Hi,
I've been poking around with pcolor, trying to see what makes it tick, since its performance is biggest thing that drives me nuts about matplotlib. I do pcolor plots of weather radar data with ~100000 polygons in them. Unfortunately, these polygons are at best trapezoids, so I can't treat it as image data. With this data, pcolor takes ages to zoom and pan, even on my new workstaion. My end goal is to be able to use OpenGL to do some 2D rendering, since in my experience, this simply flies at rendering such data for interactive analysis.
I noticed that when I run the pcolor_demo.py (using current SVN trunk), 29396 separate calls are registered to RendererGDK->draw_path to simply draw the image the first time. Is there any reason why these can't be batched up and passed in one block to the renderer? This would make life easier in setting up OpenGL to render in one pass.
Thanks
Ryan
···
--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
Ryan,
The pcolor implementation is fundamentally unsuited to large arrays. Therefore I made the pcolorfast axes method, which tries to use the fastest available Agg extension code, depending on the characteristics of the spatial grid. If the grid is rectangular and regular in both directions it uses a slight modification of the image code; if it is rectangular but with irregular spacing, it uses the nonuniform image code; and if it is not rectangular it uses the quadmesh code. It sounds like what you need is the quadmesh version, which you can access either via pcolormesh or pcolorfast. Neither is exposed via the pylab or pyplot API at present; both are axes methods. The pcolorfast API also may change slightly in the future; it probably needs a little more work.
The quadmesh code has problems with masked arrays in the released version of mpl, but not in the svn version. It is *much* faster than pcolor, but may not be fast enough for your needs.
If you are looking into what sounds like an OpenGL backend, or component to a backend, then the place to start is still probably pcolormesh or pcolorfast, not pcolor.
Eric
Ryan May wrote:
···
Hi,
I've been poking around with pcolor, trying to see what makes it tick, since its performance is biggest thing that drives me nuts about matplotlib. I do pcolor plots of weather radar data with ~100000 polygons in them. Unfortunately, these polygons are at best trapezoids, so I can't treat it as image data. With this data, pcolor takes ages to zoom and pan, even on my new workstaion. My end goal is to be able to use OpenGL to do some 2D rendering, since in my experience, this simply flies at rendering such data for interactive analysis.
I noticed that when I run the pcolor_demo.py (using current SVN trunk), 29396 separate calls are registered to RendererGDK->draw_path to simply draw the image the first time. Is there any reason why these can't be batched up and passed in one block to the renderer? This would make life easier in setting up OpenGL to render in one pass.
Thanks
Ryan
Eric Firing wrote:
Ryan,
The pcolor implementation is fundamentally unsuited to large arrays. Therefore I made the pcolorfast axes method, which tries to use the fastest available Agg extension code, depending on the characteristics of the spatial grid. If the grid is rectangular and regular in both directions it uses a slight modification of the image code; if it is rectangular but with irregular spacing, it uses the nonuniform image code; and if it is not rectangular it uses the quadmesh code. It sounds like what you need is the quadmesh version, which you can access either via pcolormesh or pcolorfast. Neither is exposed via the pylab or pyplot API at present; both are axes methods. The pcolorfast API also may change slightly in the future; it probably needs a little more work.
Yeah, I think for purposes of plotting radar data, quadmesh is what i need.
The quadmesh code has problems with masked arrays in the released version of mpl, but not in the svn version. It is *much* faster than pcolor, but may not be fast enough for your needs.
My latest testing with the trunk with both pcolorfast and pcolormesh would seem to indicate that it's still a bit too slow for me.
If you are looking into what sounds like an OpenGL backend, or component to a backend, then the place to start is still probably pcolormesh or pcolorfast, not pcolor.
This is the approach I'm considering. I've been poking around, and managed to create something that rendered, but not too well. However this was basically modifying the Gtk backend, and I'm sure there are some Gtk things that could be replaced. I noticed there are some other draw_* methods in other backends that aren't in the gtk, like draw_quad_mesh and draw_rectangle. Is there a definitive list somewhere? I tried RendererBase in backend_bases, but it did not have draw_rectangle.
Ryan
···
--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
Ryan May wrote:
Eric Firing wrote:
Ryan,
The pcolor implementation is fundamentally unsuited to large arrays. Therefore I made the pcolorfast axes method, which tries to use the fastest available Agg extension code, depending on the characteristics of the spatial grid. If the grid is rectangular and regular in both directions it uses a slight modification of the image code; if it is rectangular but with irregular spacing, it uses the nonuniform image code; and if it is not rectangular it uses the quadmesh code. It sounds like what you need is the quadmesh version, which you can access either via pcolormesh or pcolorfast. Neither is exposed via the pylab or pyplot API at present; both are axes methods. The pcolorfast API also may change slightly in the future; it probably needs a little more work.
Yeah, I think for purposes of plotting radar data, quadmesh is what i need.
The quadmesh code has problems with masked arrays in the released version of mpl, but not in the svn version. It is *much* faster than pcolor, but may not be fast enough for your needs.
My latest testing with the trunk with both pcolorfast and pcolormesh would seem to indicate that it's still a bit too slow for me.
If you are looking into what sounds like an OpenGL backend, or component to a backend, then the place to start is still probably pcolormesh or pcolorfast, not pcolor.
This is the approach I'm considering. I've been poking around, and managed to create something that rendered, but not too well. However this was basically modifying the Gtk backend, and I'm sure there are some Gtk things that could be replaced. I noticed there are some other draw_* methods in other backends that aren't in the gtk, like draw_quad_mesh and draw_rectangle. Is there a definitive list somewhere? I tried RendererBase in backend_bases, but it did not have draw_rectangle.
The pure Gtk backend has no optimizations for draw_quad_mesh. You may find that the Agg (e.g. GtkAgg) backend is significantly faster with quad meshes because it uses optimized C++ code to draw the mesh. The Gtk version must make (as you discovered) a Python function call for each quadrilateral.
The definitive list of methods on rendering backends is in backend_bases.py. There are "required" methods, and methods that may be provided for optimization reasons (the latter are the ones that mention having a "fallback" implementation in RendererBase).
Hope that helps,
Mike