how to make matplotlib faster?

Hi,

I am now trying to switch from ppgplot to matplotlib and I
really like the latter for the much nicer plots and functionalities
(although limitations such as the absence of contour plots is a critical one).

However, I just made a small script to plot 10 small subplots
on a single window repeatedly (going through slices of an array each time)
and it is bloody slooooooooow (a very large factor slower than anything
I can use to do the same thing with python and some graphical functions).

I personnally think this is a major limitation (with the contours) of
that piece of soft, and may discourage many (and myself).

Is there a way to improve (dramatically) this? Is there a plan there?

thanks in advance,

Eric Emsellem

P.S.: by the way I solved the cursor problem I posted (and got no answer)
by defining a new cursor class (something already hinted
by many on the web), if anyone is interested..

···

--

Observatoire de Lyon emsellem@...419...
9 av. Charles-Andre tel: +33 4 78 86 83 84
69561 Saint-Genis Laval Cedex fax: +33 4 78 86 83 86
France http://www-obs.univ-lyon1.fr/eric.emsellem

Hi Eric,

[...]

P.S.: by the way I solved the cursor problem I posted (and got no answer)
by defining a new cursor class (something already hinted
by many on the web), if anyone is interested..

I would be very interested in this - maybe you could
post it here (also for the archive ;-).
If possible it would be nice to see this integrated into matplotlib
because this sounds like a replacement of scipy.xplt/pygist's
mouse command (or ppgplot's pgband?).

Best,

Arnd

P.S.: I agree on the speed issues. Unfortunately
      most of the newer python graphics packages tend to be
      slower than older packages.
      For example scipy.xplt (aka pygist) see
      http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/python/index.html
      is reasonably fast.

      We have set up a PlottingCanvas for wxPython
      for our specific needs of fast plotting of single points
      and moving objects,
        http://www.physik.tu-dresden.de/~baecker/python/plot.html
      But this is by far not a fully featured plotting program,
      though maybe some of the ideas used there to speed
      up things could be used in matplotlib
      (note that I state this with complete ignorance
      of matplotlib's interna).

···

On Wed, 8 Dec 2004, Eric Emsellem wrote:

Hi,

I am now trying to switch from ppgplot to matplotlib and I
really like the latter for the much nicer plots and functionalities
(although limitations such as the absence of contour plots is a critical one).

There is progress being made on contour plots. We've implemented a basic
version that John Hunter is looking at now.

However, I just made a small script to plot 10 small subplots
on a single window repeatedly (going through slices of an array each time)
and it is bloody slooooooooow (a very large factor slower than anything
I can use to do the same thing with python and some graphical functions).

I personnally think this is a major limitation (with the contours) of
that piece of soft, and may discourage many (and myself).

Could you give some indication of what speed you are getting vs what you
have gotten under other plotting packages? How big is the slice (how many
points)? What kind of plot? Showing the actual script may help a lot in
understanding why it is so slow. There may be other ways that are faster,
or it will at least point to the main bottleneck that could stand improvement.
This complaint is too general to be helpful. I'm not sure it should be
matplotlib's goal to be the fastest package around, but it should be
fast enough for ordinary plotting (which means different things to
different people of course).

Is there a way to improve (dramatically) this? Is there a plan there?

thanks in advance,

Eric Emsellem

P.S.: by the way I solved the cursor problem I posted (and got no answer)
by defining a new cursor class (something already hinted
by many on the web), if anyone is interested..

I missed this post (I'm too busy at the moment to read all posts).
Yes, having this functionality is important. Some of this is possible
now but John has this at his fingertips (I can't recall the details).
If I have time I'll see if I can dig this up.

Perry

···

On Dec 8, 2004, at 3:29 AM, Eric Emsellem wrote:

Perry Greenfield wrote:

Hi,

I am now trying to switch from ppgplot to matplotlib and I
really like the latter for the much nicer plots and functionalities
(although limitations such as the absence of contour plots is a critical one).

There is progress being made on contour plots. We've implemented a basic
version that John Hunter is looking at now.

It's really nice to hear that. As Eric said, it's one thing that is sorely missed.

As it's being implemented, here is a little wish. I'd like to see the capability of contouring on an arbitrary grid. That is, matplotlab would be able to plot the contours of a function f(x_i, y_i) given on an arbitrary set of points (x_i, y_i), not necessarily set out on a regular grid.

Regards,
ST

···

On Dec 8, 2004, at 3:29 AM, Eric Emsellem wrote:

--

I have experienced some extreme inefficiency using errorbar plots for large
datasets. Obviously, the "hlines" routine is a huge bottleneck. Would it be
possible, in principle, to use an efficient collection instead?

···

Am Mittwoch, 8. Dezember 2004 16:35 schrieb Perry Greenfield:

Could you give some indication of what speed you are getting vs what you
have gotten under other plotting packages?

--
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <Norbert@...399...>

LUK ShunTim wrote:

As it's being implemented, here is a little wish. I'd like to see the capability of contouring on an arbitrary grid. That is, matplotlab would be able to plot the contours of a function f(x_i, y_i) given on an arbitrary set of points (x_i, y_i), not necessarily set out on a regular grid.

This would be nice, but it's a bit of a project. One way to do it would be to Delaunay triangulate the points, then you can compute the contours from the triangular grid. Delaunay triangulation is not trivial, and you really want to use an efficient scheme to do it. One possibility is:

http://www-2.cs.cmu.edu/~quake/triangle.html

It is very robust and fast, and can be compiled as a library. I've been planning for ages to write a Python wrapper for it, but haven't gotten to it yet.

If someone works on this, I'd like to help.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...259...

Arnd Baecker wrote:

P.S.: I agree on the speed issues. Unfortunately
      most of the newer python graphics packages tend to be
      slower than older packages.

I think this has two reasons:

1) They are written more in Python, rather than wrapping an existing library written in C or whatever.

2) They often are back-end independent. This introduces an extra layer at every drawing command, and makes it difficult to take advantage of possible optimizations available for a given back end, like Arnd has done for his stuff.

One thing that could help here is if all the drawing commands were "vectorized". This would mean that rather than asking the back-end to draw one primitive at a time, a whole set could be passed in. This would allow the back end to possibly optimize the drawing of the set. An example is wxPython's DC.DrawXXXList() methods. These methods take a python sequence of drawing primitives, and loop through that sequence in C++ code. It makes a huge difference when drawing simple objects, like points or line segments.

I haven't looked closely at the matplotlib code to see if this can be done, but it could make a difference.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...259...

Actually, I believe that the low level contour engine we are using
supports this. It takes 2-d arrays for both x and y that represent
the x and y coordinates of the array being contoured and generates
plotting points based on those x and y arrays. These arrays allow
for irregular grids. At the moment, the routine generates uniform
x and y grids as arguments to pass along, but it could be generalized
to take these as extra arguments without much trouble.

Let me know if I misunderstand what you are trying to do.

Perry

···

On Dec 9, 2004, at 2:12 PM, Chris Barker wrote:

LUK ShunTim wrote:

As it's being implemented, here is a little wish. I'd like to see the capability of contouring on an arbitrary grid. That is, matplotlab would be able to plot the contours of a function f(x_i, y_i) given on an arbitrary set of points (x_i, y_i), not necessarily set out on a regular grid.

This would be nice, but it's a bit of a project. One way to do it would be to Delaunay triangulate the points, then you can compute the contours from the triangular grid. Delaunay triangulation is not trivial, and you really want to use an efficient scheme to do it. One possibility is:

http://www-2.cs.cmu.edu/~quake/triangle.html

It is very robust and fast, and can be compiled as a library. I've been planning for ages to write a Python wrapper for it, but haven't gotten to it yet.

If someone works on this, I'd like to help.

-Chris

Correction. It already supports that feature now (but it isn't
checked in yet).

···

On Dec 9, 2004, at 2:33 PM, Perry Greenfield wrote:

Actually, I believe that the low level contour engine we are using
supports this. It takes 2-d arrays for both x and y that represent
the x and y coordinates of the array being contoured and generates
plotting points based on those x and y arrays. These arrays allow
for irregular grids. At the moment, the routine generates uniform
x and y grids as arguments to pass along, but it could be generalized
to take these as extra arguments without much trouble.

Let me know if I misunderstand what you are trying to do.

Perry

Perry Greenfield wrote:

LUK ShunTim wrote:

As it's being implemented, here is a little wish. I'd like to see the capability of contouring on an arbitrary grid. That is, matplotlab would be able to plot the contours of a function f(x_i, y_i) given on an arbitrary set of points (x_i, y_i), not necessarily set out on a regular grid.

This would be nice, but it's a bit of a project. One way to do it would be to Delaunay triangulate the points, then you can compute the contours from the triangular grid. Delaunay triangulation is not trivial, and you really want to use an efficient scheme to do it. One possibility is:

http://www-2.cs.cmu.edu/~quake/triangle.html

It is very robust and fast, and can be compiled as a library. I've been planning for ages to write a Python wrapper for it, but haven't gotten to it yet.

If someone works on this, I'd like to help.

-Chris

Actually, I believe that the low level contour engine we are using
supports this. It takes 2-d arrays for both x and y that represent
the x and y coordinates of the array being contoured and generates
plotting points based on those x and y arrays. These arrays allow
for irregular grids. At the moment, the routine generates uniform
x and y grids as arguments to pass along, but it could be generalized
to take these as extra arguments without much trouble.

I use Hardy's multiquadric interpolation to to do the math, then use imshow (or pcolor) to make a surface map. I only have data for the 120 points (where the circle are - those are actuators), and interpolate the rest.

If people are interested, I can clean up the code a little and post it.

axPnuLC2004-12-09_09-38-41.70038018.png

···

On Dec 9, 2004, at 2:12 PM, Chris Barker wrote:

Peter Groszkowski wrote:

I use Hardy's multiquadric interpolation to to do the math, then use imshow (or pcolor) to make a surface map. I only have data for the 120 points (where the circle are - those are actuators), and interpolate the rest.

If people are interested, I can clean up the code a little and post it.

Please do.

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...259...

Perry Greenfield wrote:

Actually, I believe that the low level contour engine we are using
supports this. It takes 2-d arrays for both x and y that represent
the x and y coordinates of the array being contoured and generates
plotting points based on those x and y arrays. These arrays allow
for irregular grids.

completely irregular? or only orthogonal structured grids. From your description, it sounds like the later. Could it take an unstructured set of (x,y,z) points and contour the z values?

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...259...