dynamic plots

John_Hunter · December 21, 2004, 7:49pm

imaginee1> Hi, after spending a nice afternoon profiling the
    imaginee1> dynamic examples and looking a bit through the code, we
    imaginee1> can make a few comments on the performace of the wx
    imaginee1> backends. We have used kcachegrind to display the
    imaginee1> results of hotshot - all files can be found under
    imaginee1> http://www.physik.tu-dresden.de/~baecker/tmp/profiling/

Hi Arnd, thanks for your profiling information - I very much like the
hotshot graphs!

I just have two comments.

All of your suggestions are imminently reasonable. The major problem
is that the wx backend has been mostly rudderless since Jeremy, the
author, stopped maintaining it, though I've filled in when I can.
Matthew Newville has recently signed on as the new maintainer and has
CVS commit privileges, but I don't know how much time he has to
address these issues right now. I don't have any extra time to devote
to wx optimizations, currently. If you would like to do some work
here, I would be happy to add you to the developers list.

The second point is that in your previous email you appeared to
indicate that GTK wasn't a good option for you because many of your
students use win32. I use the gtk backend on win32 - you have to run
the GTK runtime installer and the pygtk installer, but it otherwise
works great, and the matplotlib gtk extension code is compiled into
the matplotlib win32 installer. There are install instructions for
win32 at http://matplotlib.sourceforge.net/backends.html#GTK .

JDH

_Matt_Newville · December 22, 2004, 4:33am

Hi Arnd, John, all,

Thanks for the profiling information on WX and WXAgg backends, and
sorry I haven't been able to participate in this conversation
more. I've been running experiments for several weeks, and am
definitely looking forward to a break! Hopefully I'll be able to
devote some time to this in January.

I'm really not a wx or matplotlib expert, buit I am definitely
interested in getting WX or WXAgg to go faster. Re-drawing line
scans at 15Hz would be plenty fast enough for my needs but I would
like that to include rescaling the axes as well as updating the
line (which is what most of the current benchmarks test).
Refreshing relatively small images at 1Hz would be OK for me - I
think that's already good enough for me. Also, I'll have to admit
my timeframe is probably going to be slower than many people on
this list! So if you or someone else wants to jump in, that would
be fine with me.

I agree that the Agg rendering itself does not seem like the
bottleneck for WXAgg. Partly because of that, I'm assuming that
the WXAgg will be good enough for my needs (as opposed to
completely rewriting backend_wx) and that getting to GTKAgg level
of performance would be the goal. I also agree that the best
solution is likely to mean converting the Agg image (pixBuffer??)
into the wx.bitmap in c++. I'm not sure I have a firm grasp on
how exactly to do that, but it's worth trying.

--Matt

Arnd_Baecker · December 22, 2004, 2:43pm

Hi John,

    imaginee1> Hi, after spending a nice afternoon profiling the
    imaginee1> dynamic examples and looking a bit through the code, we
    imaginee1> can make a few comments on the performace of the wx
    imaginee1> backends. We have used kcachegrind to display the
    imaginee1> results of hotshot - all files can be found under
    imaginee1> http://www.physik.tu-dresden.de/~baecker/tmp/profiling/

Hi Arnd, thanks for your profiling information - I very much like the
hotshot graphs!

Nikolai and I were also quite impressed, in particular
because there is another window in which one
can see the corresponding lines of code, including timings
(even down to the wxpython level!).

However, we don't understand the output completely.
One reason certainly is that we don't know the matplotlib
code well enough. Another reason is that there might
be some glitches (either kcachegrind or the
conversion script).

We just did some more profiling,
for TkAgg, GTK, GTKAgg, WX and WXAgg, see:
http://www.physik.tu-dresden.de/~baecker/tmp/profiling2/

From this we get for all **Agg backends that
  - new_gc
  - _draw_solid
  - draw_text
eat up a major part of the time.
Another important part is spread in the draw chain
(for example from 74.3% in to 47.0 %+10.3 % in GTKAgg).

Best,

Nikolai and Arnd

P.S: we just looked at backend_gtk.py.
Couldn't one safely replace

    def draw_lines(self, gc, x, y):
        x = x.astype(nx.Int16)
        y = self.height*ones(y.shape, nx.Int16) - y.astype(nx.Int16)
        self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))

by

    def draw_lines(self, gc, x, y):
        x = x.astype(nx.Int16)
        y = self.height - y.astype(nx.Int16)
        self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))

? It might give a small improvement.

P.P.S: Thanks for mentioning good experiences with GTK under
Windows - we will give it a try.

···

On Tue, 21 Dec 2004, John Hunter wrote:

Arnd_Baecker · December 22, 2004, 2:44pm

Hi Matt,

[...]

I agree that the Agg rendering itself does not seem like the
bottleneck for WXAgg. Partly because of that, I'm assuming that
the WXAgg will be good enough for my needs (as opposed to
completely rewriting backend_wx) and that getting to GTKAgg level
of performance would be the goal. I also agree that the best
solution is likely to mean converting the Agg image (pixBuffer??)
into the wx.bitmap in c++. I'm not sure I have a firm grasp on
how exactly to do that, but it's worth trying.

We did some more profiling (see the other mail) which
confirms that the calls to ConvertTotBitmap and wxEmptyImage
are the main difference to GTKAgg.
The best would be to directly blit the Agg image
onto the wxFrame (however this assumes that these are
of the same type, which we don't know.
If not, the C++ route should be the fastest.
Unfortunately we both don't speak C++ and cannot
help further here).

Best,

Nikolai and Arnd

···

On Tue, 21 Dec 2004, Matt Newville wrote:

_Chris_Barker · December 22, 2004, 4:55pm

Matt Newville wrote:

I also agree that the best
solution is likely to mean converting the Agg image (pixBuffer??)
into the wx.bitmap in c++.

This trick here is that the binary format of a wxBitmap is both platform
and instance dependent. The idea, as I understand it, is that a wxBitmap
has the same binary format as is needed for the current display, so each
platform is different, and it can also be different depending on the
depth of the display. Given all this, I doubt you're going to be able to
improve on the wx supplied methods for converting from a wxImage to a
wxBitmap. (and if you do, contribute it to wx!)

Does the Agg backend use a binary format compatible with wxImage? If
not, that means there are two conversions required, which might be the
source of the slowdown.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...259...

Steve_Chaplin · December 22, 2004, 5:31pm

From this we get for all **Agg backends that
  - new_gc
  - _draw_solid
  - draw_text
eat up a major part of the time.
Another important part is spread in the draw chain
(for example from 74.3% in to 47.0 %+10.3 % in GTKAgg).

I've noticed that that new_gc() is called many times (60 times when running
simple_plot.py) with each call creates a new GraphicsContext instance. I think
it would be more efficient to create just one GC instance and reuse it. It
would suit the way that the Cairo, PS and possibly SVG and Agg backends
do their drawing but that might mean completely changing the way the
matplotlib frontend does its drawing.

P.S: we just looked at backend_gtk.py.
Couldn't one safely replace

    def draw_lines(self, gc, x, y):
        x = x.astype(nx.Int16)
        y = self.height*ones(y.shape, nx.Int16) - y.astype(nx.Int16)
        self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))

by

    def draw_lines(self, gc, x, y):
        x = x.astype(nx.Int16)
        y = self.height - y.astype(nx.Int16)
        self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))

? It might give a small improvement.

I hope so because I made this change to the code a few weeks ago and it
is now in the file backend_gdk.py in matplotlib 0.65!
It looks like you are running an old version of matplotlib.

Steve

···

On Wed, 2004-12-22 at 15:43 +0100, Arnd Baecker wrote:

Arnd_Baecker · December 22, 2004, 5:49pm

[...]

> P.S: we just looked at backend_gtk.py.
> Couldn't one safely replace
>
> def draw_lines(self, gc, x, y):
> x = x.astype(nx.Int16)
> y = self.height*ones(y.shape, nx.Int16) - y.astype(nx.Int16)
> self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))
> by
>
> def draw_lines(self, gc, x, y):
> x = x.astype(nx.Int16)
> y = self.height - y.astype(nx.Int16)
> self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))
>
> ? It might give a small improvement.
I hope so because I made this change to the code a few weeks ago and it
is now in the file backend_gdk.py in matplotlib 0.65!
It looks like you are running an old version of matplotlib.

Well, 0.64 Last week, before we started with this, there were
no debian packages for 0.65. But now they are,
http://anakonda.altervista.org/debian/
Sorry for the duplication ...

···

On Thu, 23 Dec 2004, Steve Chaplin wrote: