Agg complexity exceeded

_Tornes_Ivan_E · March 12, 2010, 2:30pm

I’m working on a project that handles large data
sets. Up to this point I had not had any issues using matplotlib, but I
tried yesterday to have it plot a file that had 8 million float,float pairs in
it and dies with the following message:

pException in Tkinter
callback

Traceback (most recent call
last):

File
“/usr/lib/python2.6/lib-tk/Tkinter.py”, line 1413, in
call

return
self.func(*args)

File
“/usr/lib/pymodules/python2.6/matplotlib/backends/backend_tkagg.py”,
line 212, in resize

self.show()

File
“/usr/lib/pymodules/python2.6/matplotlib/backends/backend_tkagg.py”,
line 215, in draw

FigureCanvasAgg.draw(self)

File “/usr/lib/pymodules/python2.6/matplotlib/backends/backend_agg.py”,
line 314, in draw

self.figure.draw(self.renderer)

File
“/usr/lib/pymodules/python2.6/matplotlib/artist.py”, line 46, in
draw_wrapper

draw(artist, renderer, *kl)

File “/usr/lib/pymodules/python2.6/matplotlib/figure.py”,
line 774, in draw

for a in self.axes:
a.draw(renderer)

File
“/usr/lib/pymodules/python2.6/matplotlib/artist.py”, line 46, in
draw_wrapper

draw(artist, renderer, *kl)

File
“/usr/lib/pymodules/python2.6/matplotlib/axes.py”, line 1721, in draw

a.draw(renderer)

File
“/usr/lib/pymodules/python2.6/matplotlib/artist.py”, line 46, in
draw_wrapper

draw(artist, renderer, *kl)

File
“/usr/lib/pymodules/python2.6/matplotlib/lines.py”, line 535, in draw

drawFunc(renderer, gc, tpath,
affine.frozen())

File
“/usr/lib/pymodules/python2.6/matplotlib/lines.py”, line 874, in
_draw_lines

self._lineFunc(renderer, gc,
path, trans)

File
“/usr/lib/pymodules/python2.6/matplotlib/lines.py”, line 918, in
_draw_solid

renderer.draw_path(gc, path,
trans)

File
“/usr/lib/pymodules/python2.6/matplotlib/backends/backend_agg.py”,
line 98, in draw_path

self._renderer.draw_path(gc,
path, transform, rgbFace)

RuntimeError:
Agg rendering complexity exceeded. Consider downsampling or decimating your
data.

I
would prefer to not decimate my data if possible.

Ivan
E. Tornes

_John_Hunter1 · March 12, 2010, 4:40pm

There have been some enhancements in path simplification in svn, so
you may want to try this but no guarantees this will help

http://matplotlib.sourceforge.net/faq/installing_faq.html#install-from-svn

Alternatively, you could use something like the "clippedline" example.
The basic idea is that most screen devices have order of a couple
million pixels, so there is no way to resolve more points than that.
But you may want to be able to zoom into a certain region will full
detail, which is not possible if you decimate your data before hand.
What the clippedline demo does is just pass the points that are in the
current viewport to mpl.

http://matplotlib.sourceforge.net/examples/pylab_examples/clippedline.html

While this may not solve your case, because it looks like you may be
exceeding the rendering complexity with data in the viewport, the
design pattern may help inspire to you to write a custom class to
handle adaptively decimating your data so you can still see a sketch
of your data when panned out, but nonethless get the full detail when
zoomed in.

This is in part what what the path simplification algorithm tries to
achieve, so do take a look if things work better on svn HEAD.

Hope this helps,
JDH

···

On Fri, Mar 12, 2010 at 8:30 AM, Tornes, Ivan E <tornesi@...2996...> wrote:

I’m working on a project that handles large data sets. Up to this point I
had not had any issues using matplotlib, but I tried yesterday to have it
plot a file that had 8 million float,float pairs in it and dies with the
following message:

Eric_Firing1 · March 12, 2010, 6:15pm

John Hunter wrote:

I’m working on a project that handles large data sets. Up to this point I
had not had any issues using matplotlib, but I tried yesterday to have it
plot a file that had 8 million float,float pairs in it and dies with the
following message:

There have been some enhancements in path simplification in svn, so
you may want to try this but no guarantees this will help

http://matplotlib.sourceforge.net/faq/installing_faq.html#install-from-svn

Alternatively, you could use something like the "clippedline" example.
The basic idea is that most screen devices have order of a couple
million pixels, so there is no way to resolve more points than that.
But you may want to be able to zoom into a certain region will full
detail, which is not possible if you decimate your data before hand.
What the clippedline demo does is just pass the points that are in the
current viewport to mpl.

http://matplotlib.sourceforge.net/examples/pylab_examples/clippedline.html

John,

This example is mostly obsolete--the clipping procedure is built-in. (The value added by the example is the change in marker style.)

Path simplification does a more thorough and general job of clipping. The reason for having the pre-clipping in the special but common case of monotonic x is that it can take advantage of a binary search, which is faster than the path-simplification's linear search for large datasets.

Eric

···

On Fri, Mar 12, 2010 at 8:30 AM, Tornes, Ivan E <tornesi@...2996...> wrote:

While this may not solve your case, because it looks like you may be
exceeding the rendering complexity with data in the viewport, the
design pattern may help inspire to you to write a custom class to
handle adaptively decimating your data so you can still see a sketch
of your data when panned out, but nonethless get the full detail when
zoomed in.

This is in part what what the path simplification algorithm tries to
achieve, so do take a look if things work better on svn HEAD.

Hope this helps,
JDH

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

_John_Hunter1 · March 12, 2010, 6:37pm

Right, which is what I was trying to get at in the following:

  While this may not solve your case, because it looks like you may be
  exceeding the rendering complexity with data in the viewport, the
  design pattern may help inspire to you to write a custom class to
  handle adaptively decimating your data so you can still see a sketch
  of your data when panned out, but nonethless get the full detail when
  zoomed in.

What I was trying to point out is you can hook into the event handling
/ callback mechanism to do *custom* level of detail clipping based on
viewport or some other feature if the default simplification is not
adequate. Along these lines Ivan, you may want to take a look at the
event handling tutorial

http://matplotlib.sourceforge.net/users/event_handling.html

Perhaps we should wework the example to do something more useful that
is not already handled better by path simplification.

JDH

···

On Fri, Mar 12, 2010 at 12:15 PM, Eric Firing <efiring@...202...> wrote:

This example is mostly obsolete--the clipping procedure is built-in. (The
value added by the example is the change in marker style.)