Animation for WXAgg

Arnd_Baecker1 · August 30, 2005, 2:31pm

Hi Ken,

[...]

I will resubmit the wx-anim.tar.gz patch to sourceforge with this
correction.

Many thanks for the update of the patch.
It turned out that I could not apply it to this mornings CVS checkout
because John's time machine was already active and he has put your
patch into CVS

During installation I encountered two glitches:

a) debian problem:
   Very interestingly debian stable (sarge)
   does not have wxPython.h
   As a work-around I downloaded the corresponding source from debian
    wget
http://ftp.debian.org/debian/pool/main/w/wxwindows2.4/wxwindows2.4_2.4.3.1.tar.gz
   and copied all header files wxwindows2.4-2.4.3.1/wxPython/src/*.h
   to a (newly created) /usr/include/wx/wxPython.

Then the installation went fine.

b) CVS problem:
Calling python animation_blit_wx.py from the examples directory
gives the error

  lib/python2.3/site-packages/matplotlib/__init__.py:788: UserWarning:
could not find rc file; returning defaults
    warnings.warn(message)
  Traceback (most recent call last):
    File "animation_blit_wx.py", line 10, in ?
      import pylab as p
    File "/scratch/abaecker/SOFT//lib/python2.3/site-packages/pylab.py",
line 1, in ?
      from matplotlib.pylab import *
    File
"/scratch/abaecker/SOFT//lib/python2.3/site-packages/matplotlib/pylab.py",
line 194, in ?
      import cm
    File
"/scratch/abaecker/SOFT//lib/python2.3/site-packages/matplotlib/cm.py",
line 372, in ?
      atad['Blues']=_Blues_data
  NameError: name '_Blues_data' is not defined

  Commenting out the corresponding block in matplotlib/cm.py
  everything works fine again
  (John: I think this is a leftover from rev. 1.19 for cm.py "removed
extra cmaps until license is resolved")

OK, now to the new WXAgg (and animation in general):

Running:
  python animation_blit.py
  python animation_blit_tk.py
  python animation_blit_wx.py
  python animation_blit_wx.py --no-accel

gives:

  GTKAgg FPS: 131.897223221
  TkAgg FPS: 65.5153012595
  WXAgg FPS: 100.55
     (200 frames: 1.99 seconds, 200 blits: 0.95 seconds)
     (FPS: 100.55 BPS: 211.00)
  WXAgg --no-accel FPS: 53.21
     (200 frames: 3.76 seconds, 200 blits: 2.55 seconds
     (FPS: 53.21 BPS: 78.40)

I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS
graphics card (@1600x1200).
So WXAgg is almost on par with GTGAgg - great!!

Ken, in _wxagg.cpp you mention that
you planned to write a agg_to_wx_bitmap() which
draws directly to a bitmap, but that
this might not speed up things much because
of: AGG->wx.Image->wx.Bitmap before you can blit using a MemoryDC.
Maybe you can contact Chris Barker if he has an idea on this
(or are there any solutions available in the context of wxart2d or
chaco? - quick googling did not reveal anything ...)

Anyway, the speed provided by the animate option
is a fantastic improvement - many thanks John and Ken!!

Just for comparison:

for backend in `echo "Tk TkAgg GTK GTKAgg WX WXAgg"`
do
echo $backend

python anim.py -d${backend}
done

Tk FPS: 35.5331434162
TkAgg FPS: 20.3487964649
GTK FPS: 35.528699289
GTKAgg FPS: 26.9074415174
WX FPS: 13.7550465745
WXAgg FPS: 26.0163513384

Many thanks,

Arnd

···

On Mon, 29 Aug 2005, Ken McIvor wrote:

Ken_McIvor · August 30, 2005, 3:27pm

Many thanks for the update of the patch.
It turned out that I could not apply it to this mornings CVS checkout
because John's time machine was already active and he has put your
patch into CVS

Really? Yay for me!

During installation I encountered two glitches:

a) debian problem:
Very interestingly debian stable (sarge)
does not have wxPython.h

It does have wxPython.h, it's just not in the libwxgtk2.4 package. You'll need to install the libwxgtk2.4-dev package as well to get the headers.

  GTKAgg FPS: 131.897223221
  TkAgg FPS: 65.5153012595
  WXAgg FPS: 100.55
     (200 frames: 1.99 seconds, 200 blits: 0.95 seconds)
     (FPS: 100.55 BPS: 211.00)
  WXAgg --no-accel FPS: 53.21
     (200 frames: 3.76 seconds, 200 blits: 2.55 seconds
     (FPS: 53.21 BPS: 78.40)

I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS
graphics card (@1600x1200).
So WXAgg is almost on par with GTGAgg - great!!

That's very exciting news. I hadn't had a chance to test and profile the accelerator under Debian, nor had I bothered profiling it against GtkAgg and TkAgg under OSX. I should probably get those backends up and running (darn macs) and do so.

Ken, in _wxagg.cpp you mention that you planned to write a agg_to_wx_bitmap()
which draws directly to a bitmap, but that this might not speed up things
much because of: AGG->wx.Image->wx.Bitmap before you can blit using a MemoryDC.

Yes, I am concerned that re-implementing that part of the blit into C++ won't speed things up enough. I can understand doing the whole Image/Bitmap thing for portability (especially when wxWindows got started in 1992), but it's rather obnoxious when all it does is make extra copies of RGB data. However, I think there are a couple pieces of lower-handing fruit, like not making an extra source copy when blitting (_wxagg.cpp:159-190), that will offer speedups. Unfortunately, they'd be portable across Agg backends, so GtkAgg would get faster too.

Maybe you can contact Chris Barker if he has an idea on this (or are there any
solutions available in the context of wxart2d or chaco? - quick googling did not
reveal anything ...)

I'll do so, if he doesn't see this part of the message than chime in. I have come across some old wxWindows mailing list exchanges indicating that there's no way to avoid the Image->Bitmap conversion as a requirement for drawing RGB images, so I'm not planning to loose too much sleep over it. I hadn't heard about wxart2d before, but will look into it to see how their agg canvas is implemented. I'll see if I can find the chaco repository and do the same there.

Thanks for sharing your anim.py profiling results. It's heartening to see WXAgg's full-redraw speed being limited by the rest of matplotlib, rather than the backend.

Ken

···

On Aug 30, 2005, at 9:31 AM, Arnd Baecker wrote:

Arnd_Baecker1 · August 30, 2005, 3:45pm

Hi,

[...]

> a) debian problem:
> Very interestingly debian stable (sarge)
> does not have wxPython.h

It does have wxPython.h, it's just not in the libwxgtk2.4 package.
You'll need to install the libwxgtk2.4-dev package as well to get the
headers.

I would have thought so as well. However I get:

dpkg -L libwxgtk2.4-dev
/.
/usr
/usr/bin
/usr/bin/wxgtk-2.4-config
/usr/lib
/usr/lib/wx
/usr/lib/wx/include
/usr/lib/wx/include/univ
/usr/lib/wx/include/gtk-2.4
/usr/lib/wx/include/gtk-2.4/wx
/usr/lib/wx/include/gtk-2.4/wx/setup.h
/usr/lib/libwx_gtk-2.4.a
/usr/lib/libwx_gtk_gl-2.4.a
/usr/share
/usr/share/doc
/usr/share/doc/libwxgtk2.4-dev
/usr/share/doc/libwxgtk2.4-dev/copyright
/usr/share/doc/libwxgtk2.4-dev/changelog.gz
/usr/share/man
/usr/share/man/man1
/usr/lib/libwx_gtk-2.4.so
/usr/lib/libwx_gtk_gl-2.4.so
/usr/share/man/man1/wxgtk-2.4-config.1.gz

A search for wxPython.h on
http://www.debian.org/distrib/packages#search_packages
gives no hits.

> GTKAgg FPS: 131.897223221
> TkAgg FPS: 65.5153012595
> WXAgg FPS: 100.55
> (200 frames: 1.99 seconds, 200 blits: 0.95 seconds)
> (FPS: 100.55 BPS: 211.00)
> WXAgg --no-accel FPS: 53.21
> (200 frames: 3.76 seconds, 200 blits: 2.55 seconds
> (FPS: 53.21 BPS: 78.40)
>
>
> I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS
> graphics card (@1600x1200).
> So WXAgg is almost on par with GTGAgg - great!!

That's very exciting news. I hadn't had a chance to test and profile
the accelerator under Debian, nor had I bothered profiling it against
GtkAgg and TkAgg under OSX. I should probably get those backends up
and running (darn macs) and do so.

> Ken, in _wxagg.cpp you mention that you planned to write a
> agg_to_wx_bitmap()
> which draws directly to a bitmap, but that this might not speed up
> things
> much because of: AGG->wx.Image->wx.Bitmap before you can blit using a
> MemoryDC.

Yes, I am concerned that re-implementing that part of the blit into C++
won't speed things up enough. I can understand doing the whole
Image/Bitmap thing for portability (especially when wxWindows got
started in 1992), but it's rather obnoxious when all it does is make
extra copies of RGB data.

I agree. I wonder if anything improved with wx2.6?
(for our PlottingCanvas we even dared to keep
drawing DCs around, and it works without problems...)

However, I think there are a couple pieces
of lower-handing fruit, like not making an extra source copy when
blitting (_wxagg.cpp:159-190), that will offer speedups.

That would be cool
(I have to admit, that I am not too worried when FPS>100
on my machine. However, we are currently investigating
to use matplotlib for a computational physics course
(which will be next summer) and many of the students
have *much* slower machines. So we need maximum speed
but with a minimum of coding hassle (around 30% of the students
have never programmed before ...).

Unfortunately, they'd be portable across Agg backends, so GtkAgg would
get faster too.

I would not mind too much ;-).

> Maybe you can contact Chris Barker if he has an idea on this (or are
> there any
> solutions available in the context of wxart2d or chaco? - quick
> googling did not
> reveal anything ...)

I'll do so, if he doesn't see this part of the message than chime in.
I have come across some old wxWindows mailing list exchanges indicating
that there's no way to avoid the Image->Bitmap conversion as a
requirement for drawing RGB images, so I'm not planning to loose too
much sleep over it. I hadn't heard about wxart2d before, but will look
into it to see how their agg canvas is implemented. I'll see if I can
find the chaco repository and do the same there.

Thanks for sharing your anim.py profiling results. It's heartening to
see WXAgg's full-redraw speed being limited by the rest of matplotlib,
rather than the backend.

Indeed this is good news!! Many thanks,

Arnd

···

On Tue, 30 Aug 2005, Ken McIvor wrote:

On Aug 30, 2005, at 9:31 AM, Arnd Baecker wrote:

_Matt_Newville1 · August 30, 2005, 5:15pm

Ken, all,

I'd tried this patch out on Mac OS X, with standard Mac-based python
and wxPython 2.6. I had to explicitly set the environmental variable
WX_CONFIG, but after that it built fine and runs fine.

For my scripts on a slow G4 Powerbook, I see a small improvement in
speed compared with mpl 0.81. I'll be eager to try this on Windows.

Thanks Ken,

--Matt Newville <newville at cars.uchicago.edu>

Ken_McIvor · August 30, 2005, 6:16pm

I would have thought so as well. However I get:

<snip>

A search for wxPython.h on
Debian -- Packages
gives no hits.

Sorry, due to a lack of coffee this morning I misread "wxPython.h" as "wxWidgets.h" or something daft, yielding the kneejerk "install the -dev package" response.

I wasn't aware of this situation, but it has the potential to be a big problem for me at work, where we primarily run Debian. I have emailed Ron Lee, the wxgtk2.4 package maintainer, about the situation. We'll see what he has to say on the matter.

I agree. I wonder if anything improved with wx2.6? (for our PlottingCanvas
we even dared to keep drawing DCs around, and it works without problems...)

What PlottingCanvas is this? I'd be interested in seeing what optimizations you guys performed, if the source is available.

However, I think there are a couple pieces of lower-handing fruit, like not
making an extra source copy when blitting (_wxagg.cpp:159-190), that will
offer speedups.

That would be cool (I have to admit, that I am not too worried when >100
on my machine. However, we are currently investigating to use matplotlib for
a computational physics course (which will be next summer) and many of the
students have *much* slower machines. So we need maximum speed but with a
minimum of coding hassle (around 30% of the students have never programmed
before ...).

Just leveling the playing field between WXAgg and GtkAgg is exciting for me, because that means that future efforts at general optimization will net a bigger speed improvement for WXAgg.

I'd imagine the plotting speed will be good enough for something along the lines of interactive plotting with iPython or visualizing results with pylab. Speaking as a recent survivor of a computational physics class, I expect you to see a huge benefit from using Python as the language and matplotlib as the visualization, especially if you have students who have never programmed before.

Ken

···

On Aug 30, 2005, at 10:45 AM, Arnd Baecker wrote:

Arnd_Baecker1 · August 31, 2005, 12:20pm

Hi,

> I would have thought so as well. However I get:
<snip>
> A search for wxPython.h on
> Debian -- Packages
> gives no hits.

Sorry, due to a lack of coffee this morning I misread "wxPython.h" as
"wxWidgets.h" or something daft, yielding the kneejerk "install the
-dev package" response.

I wasn't aware of this situation, but it has the potential to be a big
problem for me at work, where we primarily run Debian. I have emailed
Ron Lee, the wxgtk2.4 package maintainer, about the situation. We'll
see what he has to say on the matter.

Excellent - I was thinking about doing the same.

> I agree. I wonder if anything improved with wx2.6? (for our
> PlottingCanvas
> we even dared to keep drawing DCs around, and it works without
> problems...)

What PlottingCanvas is this? I'd be interested in seeing what
optimizations you guys performed, if the source is available.

http://www.physik.tu-dresden.de/~baecker/python/plot.html

One of our main goals was to plot many points quickly,
in such a way that one appears
after another to get a "dynamic" appearance.
See http://www.physik.tu-dresden.de/~baecker/python/StandardMap.py
as an example.
(note that it still uses the old wx style...)

[...]

> However, we are currently investigating to use
> matplotlib for
> a computational physics course (which will be next summer) and many of
> the
> students have *much* slower machines. So we need maximum speed but
> with a
> minimum of coding hassle (around 30% of the students have never
> programmed
> before ...).

Just leveling the playing field between WXAgg and GtkAgg is exciting
for me, because that means that future efforts at general optimization
will net a bigger speed improvement for WXAgg.

I'd imagine the plotting speed will be good enough for something along
the lines of interactive plotting with iPython or visualizing results
with pylab.

We have just finished the conversion of all exercices
of our course from scipy.xplt (aka pygist) to matplotlib.
Unfortunately, quite a few are prohibitively slow, even
on our fast machines. But we have some ideas
on possible improvements (both on the side of our code
and on the side of matplotlib) - this is going to be
separate thread though

Speaking as a recent survivor of a computational physics
class, I expect you to see a huge benefit from using Python as the
language and matplotlib as the visualization, especially if you have
students who have never programmed before.

I absolutely agree - we have been running our course now
for the third year, so far with scipy and scipy.xplt as plotting
programm. We had very positive feedback (of course,
those who never programmed before, had to work harder ;-).
Just in case: http://www.comp-phys.tu-dresden.de/cp2005/,
however the material (apart from the FAQ) is in German.

Best,

Arnd

···

On Tue, 30 Aug 2005, Ken McIvor wrote:

On Aug 30, 2005, at 10:45 AM, Arnd Baecker wrote: