Hi,
I've got some performance problems with matplotlib, and would like to
ask if you know any way I can make it faster.
If there is no such way, I have to decide to (a) either enhance matplotlib
or (b) write my own plotting-library.
(I'm currently using matplotlib to plot data "live" on the screen, including
animation, scrolling, zoom+pam, custom scales (to zoom out some part of the
plot), and multiple X-/Y-Axes. I therefore already wrote some wrappers around
matplotlib to implement some of these features.)
In detail:
- I have a figure containing some plots (lines).
- About every second I "update" the plot:
- Add a few points to the lines.
(=add point to an array and call set_data(array))
- Scroll the plot, so that the latest point is on the right of the plot.
Older points disappear on the left side of the plot.
(=set_xaxis() + draw())
- I'm using GtkAgg, incl. animation, (re)storing the background, drawing
the artists and blit.
(canvas.restore_region(...), ax.draw_artist(...), canvas.blit(bbox))
This works as long as the plot only contains a few points,
although 2 figures + 5-10 lines per figure and an update every 0.5 s
already consumes about 10-20% CPU (on a 1.4 GHz Pentium).
By the way: Is this speed normal, or is matplotlib usually faster?
But as soon as the plot contains *many* points (several 10000 up to
several 100000), the plotting becomes terribly slow -- up to 30s
per update and more!
Do you know any way to make this faster?
My ideas are:
- Since I only add points to the *right* of the lines, I could reduce the
number of points, by first removing all points which are outside of
the current visible plotting-window from my array, and then calling
set_data() with the reduced array.
- This unfortunately wouldn't reduce the number of points in very dense
plots. It would be possible to (a) replace several points which all
result in the same plotted point by a single point or (b) cache the
plotted points e.g. on pixel-level. I think (b) would probably require
to write a new backend.
Any ideas?
regards,
Roland
Hi,
I've got some performance problems with matplotlib, and would like to
ask if you know any way I can make it faster.
If there is no such way, I have to decide to (a) either enhance matplotlib
or (b) write my own plotting-library.
(I'm currently using matplotlib to plot data "live" on the screen, including
animation, scrolling, zoom+pam, custom scales (to zoom out some part of the
plot), and multiple X-/Y-Axes. I therefore already wrote some wrappers around
matplotlib to implement some of these features.)
In detail:
- I have a figure containing some plots (lines).
- About every second I "update" the plot:
- Add a few points to the lines.
(=add point to an array and call set_data(array))
- Scroll the plot, so that the latest point is on the right of the plot.
Older points disappear on the left side of the plot.
(=set_xaxis() + draw())
- I'm using GtkAgg, incl. animation, (re)storing the background, drawing
the artists and blit.
(canvas.restore_region(...), ax.draw_artist(...), canvas.blit(bbox))
This works as long as the plot only contains a few points,
although 2 figures + 5-10 lines per figure and an update every 0.5 s
already consumes about 10-20% CPU (on a 1.4 GHz Pentium).
By the way: Is this speed normal, or is matplotlib usually faster?
But as soon as the plot contains *many* points (several 10000 up to
several 100000), the plotting becomes terribly slow -- up to 30s
per update and more!
Do you know any way to make this faster?
My ideas are:
- Since I only add points to the *right* of the lines, I could reduce the
number of points, by first removing all points which are outside of
the current visible plotting-window from my array, and then calling
set_data() with the reduced array.
- This unfortunately wouldn't reduce the number of points in very dense
plots. It would be possible to (a) replace several points which all
result in the same plotted point by a single point or (b) cache the
plotted points e.g. on pixel-level. I think (b) would probably require
to write a new backend.
Any ideas?
http://matplotlib.sourceforge.net/examples/animation/animation_blit_gtk2.html
The above example does something similar to (b).
It saves the previous plot (only axes area is saved) as a bitmap. In
next run, it restores the saved bitmap after shifting. And then draw
only the new data points.
The example requires the svn version of matplotlib.
Regards,
-JJ
···
On Tue, Jul 28, 2009 at 9:11 AM, Roland Koebler<r.koebler@...2079...> wrote:
regards,
Roland
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
There might be some performance improvement to be had depending on how you are adding the additional points to the array. Seeing a complete, minimal of the code showing the performance problem would be needed, since it’s the details that matter in this case.
Ryan
···
n Tue, Jul 28, 2009 at 8:11 AM, Roland Koebler <r.koebler@…2079…> wrote:
Hi,
I’ve got some performance problems with matplotlib, and would like to
ask if you know any way I can make it faster.
If there is no such way, I have to decide to (a) either enhance matplotlib
or (b) write my own plotting-library.
(I’m currently using matplotlib to plot data “live” on the screen, including
animation, scrolling, zoom+pam, custom scales (to zoom out some part of the
plot), and multiple X-/Y-Axes. I therefore already wrote some wrappers around
matplotlib to implement some of these features.)
In detail:
-
I have a figure containing some plots (lines).
-
About every second I “update” the plot:
-
Add a few points to the lines.
(=add point to an array and call set_data(array))
-
Scroll the plot, so that the latest point is on the right of the plot.
Older points disappear on the left side of the plot.
(=set_xaxis() + draw())
-
I’m using GtkAgg, incl. animation, (re)storing the background, drawing
the artists and blit.
(canvas.restore_region(…), ax.draw_artist(…), canvas.blit(bbox))
This works as long as the plot only contains a few points,
although 2 figures + 5-10 lines per figure and an update every 0.5 s
already consumes about 10-20% CPU (on a 1.4 GHz Pentium).
By the way: Is this speed normal, or is matplotlib usually faster?
But as soon as the plot contains many points (several 10000 up to
several 100000), the plotting becomes terribly slow – up to 30s
per update and more!
Do you know any way to make this faster?
My ideas are:
-
Since I only add points to the right of the lines, I could reduce the
number of points, by first removing all points which are outside of
the current visible plotting-window from my array, and then calling
set_data() with the reduced array.
-
This unfortunately wouldn’t reduce the number of points in very dense
plots. It would be possible to (a) replace several points which all
result in the same plotted point by a single point or (b) cache the
plotted points e.g. on pixel-level. I think (b) would probably require
to write a new backend.
–
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
Sent from Norman, Oklahoma, United States
Hi,
2009/7/29 Jae-Joon Lee <lee.j.joon@...287...>:
[cut]
Any ideas?
http://matplotlib.sourceforge.net/examples/animation/animation_blit_gtk2.html
The above example does something similar to (b).
It saves the previous plot (only axes area is saved) as a bitmap. In
next run, it restores the saved bitmap after shifting. And then draw
only the new data points.
The example requires the svn version of matplotlib.
I have attached another example of blit animation that does NOT
require the svn version of MPL.
Antonio
gtk_animation_refresh.py (4.66 KB)
···
On Tue, Jul 28, 2009 at 9:11 AM, Roland Koebler<r.koebler@...2079...> wrote:
How did this help with your performance problems? The animation/blit
paradigm is great for *some* use cases, eg animating a cross-hair over
a static background, but is not ideal for strip charts, where the data
marches uniformly off the left of the figure. The problem is, in the
strip chart case, most of the figure is simply shifted left, and only
the new part needs to be redrawn. This does not map naturally to the
blit paradigm as mpl expresses it, since so many artists need to be
updated, eg all the tick labels, the line, the ticks.....
I think we need some custom data structures, like numpy objects that
look like arrays but are circular buffers, so we can push stuff onto
the end, have it drop off the front, but is transparent to naive users
like mpl (or we could use the mpl units interface to support these
objects with some custom method calls to get an "unwrapped" view of
the data) and an API for shifting big chunks of the rendered axes to
the left, drawing just the right part of the updated circular buffers,
and making sure the stitching together at the seams looks good.
Roland: if you are inclined to "roll your own" plotting library as you
suggested in your OP, I encourage you to look into adding this support
to mpl or working on a toolkit, because such functionality is sorely
needed and I would encourage all developers to help out in improving
the core to support this effort.
You may also want to look at chaco, which generally far outperforms
mpl for quasi real-time interactive plotting stuff. I am not sure if
they explicitly have support for strip charting, but would not be
surprised if they did.
JDH
···
On Wed, Jul 29, 2009 at 1:25 PM, Antonino Ingargiola<tritemio@...287...> wrote:
I have attached another example of blit animation that does NOT
require the svn version of MPL.