Quite recently I started out with learning Python, IPython, SciPy and
Matplotlib, to try see if I could replace some data analysis software
that was previously written in LabVIEW. Slowly I'm getting sort-of the
hang of it, but I've run into a few problems, and I'm now looking
around for some advice...
Main issue is Matplotlib's performance. I'm trying to plot a current
trace from a physics experiment, containing about 300,000 data points.
In LabVIEW, one can easily browse through a data set like this, but I
haven't been able yet to get such a good performance with
IPython+Matplotlib. Especially scrolling/panning through the data is
sluggish. (Anyone knows how to add a scrollbar for this instead of
panning with the mouse, btw?)
I know this is an extremely vague description of the problem, but
perhaps someone can give some general advice on improving performance?
I read something about data clipping, but this doesn't work in my
version of Matplotlib --- the Line2D object doesn't seem to have a
property "data_clippied" anymore.
Also, I read that someone had written an EEG display program (both in
the user's guide and on the examples page), which certainly looked
very good. So it must be possible...
In any case, some more details about my system: Windows XP SP2, 2.8
GHz dual core processor with 1 GB RAM; running Python 2.5.1, IPython
0.8.2, Matplotlib 0.90.1, NumPy 1.0.5, SciPy 0.6.0.0006.
Very much looking forward to a small discussion on the subject...
Thanks a lot in advance!
You may find this off topic, since you seem to mean by "plot
a current trace" something different than I'm familiar with.
Suppose I have a 1280×1028 monitor and I'm willing
to have really tiny 9-pixel points to represent an
observation. Then I can display 1280×1028/9 distinct
points, about half as many as you want, and cover
the entire monitor.
So perhaps your question is really about the appropriate way
to radically down-sample your data?
Cheers,
Alan Isaac
···
On Fri, 29 Feb 2008, matplotlibuser@...1907... apparently wrote:
I'm trying to plot a current trace from a physics
experiment, containing about 300,000 data points.
You could also use the "clipped line" approach to pass in a custom
class that only plots the data in the current view limits defined by
timemin, timemax. See http://matplotlib.sf.net/examples/clippedline.py. This example
changes the marker and line style depending on how many points are in
the view port, but you could expand on this idea to do downsampling
when the number of points is too large.
BTW, I wrote the EEG viewer you referred to (it currently lives as
pbrain in the nipy project), and it was a bit pokey, but with some
judicious use of custom classes like the one in clippedline.py you can
probably get acceptable performance. In earlier versions of mpl we
tried to build some of these smarts in (eg the data_clipped) but they
were rarely used and of questionable performance benefit so we pulled
them. We may want to revisit the issue...
Hope this helps,
JDH
···
On Fri, Feb 29, 2008 at 8:27 AM, <matplotlibuser@...1907...> wrote:
Hi everyone,
Quite recently I started out with learning Python, IPython, SciPy and
Matplotlib, to try see if I could replace some data analysis software
that was previously written in LabVIEW. Slowly I'm getting sort-of the
hang of it, but I've run into a few problems, and I'm now looking
around for some advice...
Main issue is Matplotlib's performance. I'm trying to plot a current
trace from a physics experiment, containing about 300,000 data points.
In LabVIEW, one can easily browse through a data set like this, but I
haven't been able yet to get such a good performance with
IPython+Matplotlib. Especially scrolling/panning through the data is
sluggish. (Anyone knows how to add a scrollbar for this instead of
panning with the mouse, btw?)
I'm trying to plot a current trace from a physics
experiment, containing about 300,000 data points.
You may find this off topic, since you seem to mean by "plot
a current trace" something different than I'm familiar with.
[snip]
So perhaps your question is really about the appropriate way
to radically down-sample your data?
Thanks for your reply! I agree that under normal circumstances,
downsampling would be a good thing to do. However, in this case, it's
really about the tiny details in the trace, so I'd like to zoom in on
a small part of the data, and then scroll through it. This is already
possible with Matplotlib, but is very slow...
Note that with the clipped line approach I suggested, you can have the
best of both worlds. Downsampl when N>20000 or some appropriate
number, and plot the full data when you zoom.
JDH
···
On Fri, Feb 29, 2008 at 10:21 AM, Onno Broekmans <matplotlibuser@...1907...> wrote:
Thanks for your reply! I agree that under normal circumstances,
downsampling would be a good thing to do. However, in this case, it's
really about the tiny details in the trace, so I'd like to zoom in on
a small part of the data, and then scroll through it. This is already
possible with Matplotlib, but is very slow...
Note that with the clipped line approach I suggested, you can have the
best of both worlds. Downsampl when N>20000 or some appropriate
number, and plot the full data when you zoom.
Hm.. Good point. I'll try to implement that. Thanks!
