subslice support in Line2D

We recently saw some breakage with our PyRAF plotting tool (which uses matplotlib as a "dumb" rendering backend) and matplotlib 0.99. It stops inside the subslice support that was added to Line2D, since subslicing requires that the Line2D object have an "axes" assigned to it. Since PyRAF doesn't use matplotlib's Axes objects, its lines don't have them.

It's a simple fix to check for the existence of an axes member and skip the subslice support if it doesn't have one. However, I wonder if it couldn't just be removed, especially since it is the only dependency on an Axes from Line2D objects. I think it may have become redundant wrt the path simplification code which now handles clipping (at the figure boundary, not the axis boundary mind you) rather reliably. The simplification code now also works in all backends, which is fairly new.

I did a little benchmarking with the attached script. It generates a 10,000 point random array and then plots a 500 point subset in the middle.

baseline: 3.94 fps (no clipping or subslicing)
subslice: 28.14 fps
clipping: 28.09 fps
clipping+subslice: 28.35 fps (i.e. current code)

The last three are close enough to be considered equal.

Of course, another benchmark may produce very different results, so I'm reluctant to just yank it. But it would be nice to remove nearly-identical optimizations.

Cheers,
Mike

clipping_benchmark.py (363 Bytes)

···

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Michael Droettboom wrote:

We recently saw some breakage with our PyRAF plotting tool (which uses matplotlib as a "dumb" rendering backend) and matplotlib 0.99. It stops inside the subslice support that was added to Line2D, since subslicing requires that the Line2D object have an "axes" assigned to it. Since PyRAF doesn't use matplotlib's Axes objects, its lines don't have them.

It's a simple fix to check for the existence of an axes member and skip the subslice support if it doesn't have one. However, I wonder if it couldn't just be removed, especially since it is the only dependency on an Axes from Line2D objects. I think it may have become redundant wrt the path simplification code which now handles clipping (at the figure boundary, not the axis boundary mind you) rather reliably. The simplification code now also works in all backends, which is fairly new.

Mike,

Was the simplification you are talking about added after I added the subslice support (by which I assume you mean the slicing in the case of monotonic x)? And is it as general? At least at the time I put in the subslicing of sorted abcissas, I am pretty sure it made a big difference when panning long timeseries--that is why I put it in. Certainly I would be happy to see it go if it is not actually doing anything useful now, but I would like to be sure. I can't look at it right now.

Eric

···

I did a little benchmarking with the attached script. It generates a 10,000 point random array and then plots a 500 point subset in the middle.

baseline: 3.94 fps (no clipping or subslicing)
subslice: 28.14 fps
clipping: 28.09 fps
clipping+subslice: 28.35 fps (i.e. current code)

The last three are close enough to be considered equal.

Of course, another benchmark may produce very different results, so I'm reluctant to just yank it. But it would be nice to remove nearly-identical optimizations.

Cheers,
Mike

------------------------------------------------------------------------

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july

------------------------------------------------------------------------

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

According to svn blame, which only gives the most recent version a line was edited, not the first time a line appeared, obviously -- subslice support was added in r7100, and clipping was fixed in r6847. So, apparently at the time subslice was added the clipping was already there. So, if you were seeing a time improvement then, your data must be doing something my benchmark here doesn't.

The clipping support is probably more general than subslicing, since it doesn't require the data to be monotonic -- it clips as the line crosses any of the boundaries of the figure. Given that, I'm surprised it competes so favorably timewise -- I suspect the important thing is to just reduce the number of points passed to the renderer -- the actually speed at which those points are located is nothing compared to stroking points.

Cheers,
Mike

Eric Firing wrote:

···

Michael Droettboom wrote:

We recently saw some breakage with our PyRAF plotting tool (which uses matplotlib as a "dumb" rendering backend) and matplotlib 0.99. It stops inside the subslice support that was added to Line2D, since subslicing requires that the Line2D object have an "axes" assigned to it. Since PyRAF doesn't use matplotlib's Axes objects, its lines don't have them.

It's a simple fix to check for the existence of an axes member and skip the subslice support if it doesn't have one. However, I wonder if it couldn't just be removed, especially since it is the only dependency on an Axes from Line2D objects. I think it may have become redundant wrt the path simplification code which now handles clipping (at the figure boundary, not the axis boundary mind you) rather reliably. The simplification code now also works in all backends, which is fairly new.

Mike,

Was the simplification you are talking about added after I added the subslice support (by which I assume you mean the slicing in the case of monotonic x)? And is it as general? At least at the time I put in the subslicing of sorted abcissas, I am pretty sure it made a big difference when panning long timeseries--that is why I put it in. Certainly I would be happy to see it go if it is not actually doing anything useful now, but I would like to be sure. I can't look at it right now.

Eric

I did a little benchmarking with the attached script. It generates a 10,000 point random array and then plots a 500 point subset in the middle.

baseline: 3.94 fps (no clipping or subslicing)
subslice: 28.14 fps
clipping: 28.09 fps
clipping+subslice: 28.35 fps (i.e. current code)

The last three are close enough to be considered equal.

Of course, another benchmark may produce very different results, so I'm reluctant to just yank it. But it would be nice to remove nearly-identical optimizations.

Cheers,
Mike

------------------------------------------------------------------------

------------------------------------------------------------------------------

Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july

------------------------------------------------------------------------

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Michael Droettboom wrote:

According to svn blame, which only gives the most recent version a line was edited, not the first time a line appeared, obviously -- subslice support was added in r7100, and clipping was fixed in r6847. So, apparently at the time subslice was added the clipping was already there. So, if you were seeing a time improvement then, your data must be doing something my benchmark here doesn't.

Mike,

Try something like this:

x = arange(1000000.0)
y = sin(x/100)
plot(x, y)
xlim(0, 1000)

Now select pan/zoom, hold down X, and pan. Notice the degree of speed and smoothness (or not). Next, disable subslicing like this:

clf()
x[1] = -0.1 # no longer monotonic
plot(x, y)
xlim(0, 1000)

And try panning again. At least on my machine, there is a big difference.

Eric

···

The clipping support is probably more general than subslicing, since it doesn't require the data to be monotonic -- it clips as the line crosses any of the boundaries of the figure. Given that, I'm surprised it competes so favorably timewise -- I suspect the important thing is to just reduce the number of points passed to the renderer -- the actually speed at which those points are located is nothing compared to stroking points.

Cheers,
Mike

Eric Firing wrote:

Michael Droettboom wrote:

We recently saw some breakage with our PyRAF plotting tool (which uses matplotlib as a "dumb" rendering backend) and matplotlib 0.99. It stops inside the subslice support that was added to Line2D, since subslicing requires that the Line2D object have an "axes" assigned to it. Since PyRAF doesn't use matplotlib's Axes objects, its lines don't have them.

It's a simple fix to check for the existence of an axes member and skip the subslice support if it doesn't have one. However, I wonder if it couldn't just be removed, especially since it is the only dependency on an Axes from Line2D objects. I think it may have become redundant wrt the path simplification code which now handles clipping (at the figure boundary, not the axis boundary mind you) rather reliably. The simplification code now also works in all backends, which is fairly new.

Mike,

Was the simplification you are talking about added after I added the subslice support (by which I assume you mean the slicing in the case of monotonic x)? And is it as general? At least at the time I put in the subslicing of sorted abcissas, I am pretty sure it made a big difference when panning long timeseries--that is why I put it in. Certainly I would be happy to see it go if it is not actually doing anything useful now, but I would like to be sure. I can't look at it right now.

Eric

I did a little benchmarking with the attached script. It generates a 10,000 point random array and then plots a 500 point subset in the middle.

baseline: 3.94 fps (no clipping or subslicing)
subslice: 28.14 fps
clipping: 28.09 fps
clipping+subslice: 28.35 fps (i.e. current code)

The last three are close enough to be considered equal.

Of course, another benchmark may produce very different results, so I'm reluctant to just yank it. But it would be nice to remove nearly-identical optimizations.

Cheers,
Mike

------------------------------------------------------------------------

------------------------------------------------------------------------------

Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july

------------------------------------------------------------------------

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Ah -- for some reason the benefit is immeasurable until about 1M points. I was using fewer points because you can't even get a baseline measurement with that many -- it exceeds the path length limit in Agg.

I see now why it's superior to the clipping I wrote -- it does a binary search to find the end points. Makes sense. We'll have to keep both approaches, since the clipping is also required to prevent rendering errors due to fixed-point number overflow in the Agg backend.

So, I'll just patch it up so subslicing is disabled when there is no associated axes (the root of my original problem) and leave it at that.

Cheers,
Mike

Eric Firing wrote:

···

Michael Droettboom wrote:

According to svn blame, which only gives the most recent version a line was edited, not the first time a line appeared, obviously -- subslice support was added in r7100, and clipping was fixed in r6847. So, apparently at the time subslice was added the clipping was already there. So, if you were seeing a time improvement then, your data must be doing something my benchmark here doesn't.

Mike,

Try something like this:

x = arange(1000000.0)
y = sin(x/100)
plot(x, y)
xlim(0, 1000)

Now select pan/zoom, hold down X, and pan. Notice the degree of speed and smoothness (or not). Next, disable subslicing like this:

clf()
x[1] = -0.1 # no longer monotonic
plot(x, y)
xlim(0, 1000)

And try panning again. At least on my machine, there is a big difference.

Eric

The clipping support is probably more general than subslicing, since it doesn't require the data to be monotonic -- it clips as the line crosses any of the boundaries of the figure. Given that, I'm surprised it competes so favorably timewise -- I suspect the important thing is to just reduce the number of points passed to the renderer -- the actually speed at which those points are located is nothing compared to stroking points.

Cheers,
Mike

Eric Firing wrote:

Michael Droettboom wrote:

We recently saw some breakage with our PyRAF plotting tool (which uses matplotlib as a "dumb" rendering backend) and matplotlib 0.99. It stops inside the subslice support that was added to Line2D, since subslicing requires that the Line2D object have an "axes" assigned to it. Since PyRAF doesn't use matplotlib's Axes objects, its lines don't have them.

It's a simple fix to check for the existence of an axes member and skip the subslice support if it doesn't have one. However, I wonder if it couldn't just be removed, especially since it is the only dependency on an Axes from Line2D objects. I think it may have become redundant wrt the path simplification code which now handles clipping (at the figure boundary, not the axis boundary mind you) rather reliably. The simplification code now also works in all backends, which is fairly new.

Mike,

Was the simplification you are talking about added after I added the subslice support (by which I assume you mean the slicing in the case of monotonic x)? And is it as general? At least at the time I put in the subslicing of sorted abcissas, I am pretty sure it made a big difference when panning long timeseries--that is why I put it in. Certainly I would be happy to see it go if it is not actually doing anything useful now, but I would like to be sure. I can't look at it right now.

Eric

I did a little benchmarking with the attached script. It generates a 10,000 point random array and then plots a 500 point subset in the middle.

baseline: 3.94 fps (no clipping or subslicing)
subslice: 28.14 fps
clipping: 28.09 fps
clipping+subslice: 28.35 fps (i.e. current code)

The last three are close enough to be considered equal.

Of course, another benchmark may produce very different results, so I'm reluctant to just yank it. But it would be nice to remove nearly-identical optimizations.

Cheers,
Mike

------------------------------------------------------------------------

------------------------------------------------------------------------------

Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july

------------------------------------------------------------------------

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA