Interactive backends very (suprisingly?) slow for multiple subplots

Community matplotlib-users
1

Hi, I'm trying to plot a series of ~30-50 small plots, each of which contains
3 plots of ~10-20 points (one plot is data, one plot is errorbars, one plot
is a model fit). I've tried using GtkAgg and Qt4Agg as backends, and both
are extremely slow - they take ~5-10 seconds for the first plot window and
then hang. Sometimes they'll plot subsequent windows, but sometimes they
will not.

I tried the same series of plot commands using the SVG, PS, and PDF backends
and the whole series of 50 plots takes <~1s.

So, is there any way to speed up my plotting? I've tried turning
interactive plotting off (ioff()), but that didn't help any.

Thanks,
Adam

···

--
View this message in context: http://www.nabble.com/Interactive-backends-very-(suprisingly-)-slow-for-multiple-subplots-tp23261074p23261074.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

2

That does sound exceedingly slow – it looks like you are having some problems with the GUI or environment and not just the mpl component. How are you running and profiling your script? Can you post some free-standing example code which exposes the problem? Can you provide some of the environment data detailed at http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html#reporting-problems

···

On Mon, Apr 27, 2009 at 12:24 PM, keflavich <keflavich@…287…> wrote:

Hi, I’m trying to plot a series of ~30-50 small plots, each of which contains

3 plots of ~10-20 points (one plot is data, one plot is errorbars, one plot

is a model fit). I’ve tried using GtkAgg and Qt4Agg as backends, and both

are extremely slow - they take ~5-10 seconds for the first plot window and

then hang. Sometimes they’ll plot subsequent windows, but sometimes they

will not.

3

John Hunter-4 wrote:

That does sound exceedingly slow -- it looks like you are having some
problems with the GUI or environment and not just the mpl component. How
are you running and profiling your script? Can you post some
free-standing
example code which exposes the problem? Can you provide some of the
environment data detailed at
http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html#reporting-problems

I haven't been doing any profiling; I've never been particularly comfortable
with timeit.

As for normal problem reporting details...
Darwin eta.colorado.edu 9.6.3 Darwin Kernel Version 9.6.3: Tue Jan 20
18:26:40 PST 2009; root:xnu-1228.10.33~1/RELEASE_I386 i386 i386
matplotlib version 0.98.3

My example standalone script (note that it takes the backend as a command
line argument and assumes you have not selected a backend / imported pylab):

import matplotlib
import sys
matplotlib.use(sys.argv[1])
from pylab import *

i=1; n=1;
print "Working on figure 0"
figure(0); clf()
for j in xrange(36):
    if (i % 10)==0:
        print "Working on figure %i" % n
        figure(n); clf()
        i=1
        n+=1
    subplot(3,3,i)
    plot([1,2,3],[3,2,1])
    plot([1,2,1],[1,2,3])
    plot([1,2,3],[1,2,3])
    i+=1

Results:

In [1]: %run -t code/test.py 'Qt4Agg'

backend Qt4Agg version 0.9.1

IPython CPU timings (estimated):
  User : 21.553853 s.
  System: 0.0 s.

In [1]: %run -t code/test.py 'pdf'

backend pdf version unknown

IPython CPU timings (estimated):
  User : 1.056959 s.
  System: 0.0 s.

In [1]: %run -t code/test.py 'svg'
backend svg version 0.98.5.2

IPython CPU timings (estimated):
  User : 1.056702 s.
  System: 0.0 s.

Curiously, if I add ioff() at the beginning of the script using the Qt4Agg
backend, I get the much nicer result:
  User : 1.328496 s.

and show works quite rapidly. This suggests to me that the placement of
ioff() is important, so I'm going to try playing with that a little...

Adam

···

--
View this message in context: http://www.nabble.com/Interactive-backends-very-(suprisingly-)-slow-for-multiple-subplots-tp23261074p23263295.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

4

So, as the Matplotlib help page suggests, working through a test problem
helped me narrow down my problem... but it still hasn't solved it. If I set
ioff() at the main level, rather than in a function I call, it works.
However, when I show() the plot, the code halts until I close it, which is
not helpful. I feel like this is a problem I've resolved before, but
googling hasn't helped me so far - I think the closest I have come are
"show() hangs" threads that have no replies.

I guess this changes my question to:
How can I draw all of my figures non-interactively, then show them and
return to the python command prompt?

ion()
show()

should work, I think, but does not.

Thanks,
Adam

···

--
View this message in context: http://www.nabble.com/Interactive-backends-very-(suprisingly-)-slow-for-multiple-subplots-tp23261074p23263639.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

5

Ahh, mixing matplotlib.use from an interactive ipython session – that is an important detail :slight_smile: What is your backend (import matplotlib; print matplotlib.rcParams[‘backend’]) It is quite likely that you are getting cross GUI / cross threading problems from trying to do this inside ipython, which may be using one GUI backend based on rc at pylab startup, and mpl is trying to use another one with the use directive. Basically, “use” is not supported for switching GUIs, it was designed to be the first line executed in a session but when you do this interactively in ipython pylab mode, you are asking for a world of pain.

Try running your example from a standard unix shell rather than from ipython

JDH

···

On Mon, Apr 27, 2009 at 2:28 PM, keflavich <keflavich@…120…287…> wrote:

John Hunter-4 wrote:

That does sound exceedingly slow – it looks like you are having some

problems with the GUI or environment and not just the mpl component. How

are you running and profiling your script? Can you post some

free-standing

example code which exposes the problem? Can you provide some of the

environment data detailed at

http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html#reporting-problems

I haven’t been doing any profiling; I’ve never been particularly comfortable

with timeit.

As for normal problem reporting details…

Darwin eta.colorado.edu 9.6.3 Darwin Kernel Version 9.6.3: Tue Jan 20

18:26:40 PST 2009; root:xnu-1228.10.33~1/RELEASE_I386 i386 i386

matplotlib version 0.98.3

My example standalone script (note that it takes the backend as a command

line argument and assumes you have not selected a backend / imported pylab):

import matplotlib

import sys

matplotlib.use(sys.argv[1])

from pylab import *

i=1; n=1;

print “Working on figure 0”

figure(0); clf()

for j in xrange(36):

if (i % 10)==0:

    print "Working on figure %i" % n

    figure(n); clf()

    i=1

    n+=1

subplot(3,3,i)

plot([1,2,3],[3,2,1])

plot([1,2,1],[1,2,3])

plot([1,2,3],[1,2,3])

i+=1

Results:

In [1]: %run -t code/test.py ‘Qt4Agg’

6

John Hunter-4 wrote:

Ahh, mixing matplotlib.use from an interactive ipython session -- that is
an
important detail :slight_smile: What is your backend (import matplotlib; print
matplotlib.rcParams['backend']) It is quite likely that you are getting
cross GUI / cross threading problems from trying to do this inside
ipython,
which may be using one GUI backend based on rc at pylab startup, and mpl
is
trying to use another one with the use directive. Basically, "use" is not
supported for switching GUIs, it was designed to be the first line
executed
in a session but when you do this interactively in ipython pylab mode, you
are asking for a world of pain.

Try running your example from a standard unix shell rather than from
ipython

JDH

Sorry, I should have specified: I wrote this script to use separate from my
interactive sessions as a test. I use Qt4Agg now, set in my matplotlibrc,
and I do not use matplotlib.use() at all (since it doesn't work anyway).
The faster performance I reported was achieved by setting the backend in my
matplotlibrc, not on the command line. So I don't think I'm running into
the issues you're talking about.

I don't really know how to run a script in which I expect plots to show up
from the command line. If I do something simple like 'python test.py' it
pops up windows but never shows the plots I expect to see.

Adam

···

--
View this message in context: http://www.nabble.com/Interactive-backends-very-(suprisingly-)-slow-for-multiple-subplots-tp23261074p23264479.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

7

Since there don't seem to be any forthcoming answers, I have a somewhat
different question. In the matplotlib FAQ, it states that using 'show()'
puts you in the GUI mainloop
(http://matplotlib.sourceforge.net/faq/howto_faq.html#use-show). However,
using plot commands on the ipython command line does not shut down the
command line generally. I gathered from some googling that this is because
ipython starts up the matplotlib graphics in a different 'thread', but I
don't understand how this is done and most of what I've seen says it is bad.

So, my question now: How can I exit the GUI mainloop without closing the
graphics windows?

Thanks,
Adam

···

--
View this message in context: http://www.nabble.com/Interactive-backends-very-(suprisingly-)-slow-for-multiple-subplots-tp23261074p23295883.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

8

Since there don’t seem to be any forthcoming answers, I have a somewhat

different question. In the matplotlib FAQ, it states that using ‘show()’

puts you in the GUI mainloop

(http://matplotlib.sourceforge.net/faq/howto_faq.html#use-show). However,

using plot commands on the ipython command line does not shut down the

command line generally. I gathered from some googling that this is because

ipython starts up the matplotlib graphics in a different ‘thread’, but I

don’t understand how this is done and most of what I’ve seen says it is bad.

most GUI mainloops are blocking, so after you start them you cannot issue more commands from an interactive shell. Either you need to run a GUI shell, or in the case of ipython run the GUI in a separate thread. One exception to this is tkinter (tkagg), which plays nicely with a standard python shell. I understand that recent versions of pygtk work also w/o running the mainloop in a separate thread, but I haven’t dug into the details.

So, my question now: How can I exit the GUI mainloop without closing the

graphics windows?

This question doesn’t really make sense to me. Perhaps you can clearly describe your use case (what you need to do) rather than the proposed solutions.

JDH

···

On Wed, Apr 29, 2009 at 9:07 AM, keflavich <keflavich@…120…287…> wrote:

9

Since there don't seem to be any forthcoming answers, I have a somewhat
different question. In the matplotlib FAQ, it states that using 'show()'
puts you in the GUI mainloop
(http://matplotlib.sourceforge.net/faq/howto_faq.html#use-show). However,
using plot commands on the ipython command line does not shut down the
command line generally. I gathered from some googling that this is
because
ipython starts up the matplotlib graphics in a different 'thread', but I
don't understand how this is done and most of what I've seen says it is
bad.

most GUI mainloops are blocking, so after you start them you cannot issue
more commands from an interactive shell. Either you need to run a GUI
shell, or in the case of ipython run the GUI in a separate thread. One
exception to this is tkinter (tkagg), which plays nicely with a standard
python shell. I understand that recent versions of pygtk work also w/o
running the mainloop in a separate thread, but I haven't dug into the
details.

OK, that's very helpful.

So, my question now: How can I exit the GUI mainloop without closing the
graphics windows?

This question doesn't really make sense to me. Perhaps you can clearly
describe your use case (what you need to do) rather than the proposed
solutions.

I would like to have access to the command line while simultaneously
being able to interact with and/or display plots. I think this is
what ipython does by default when you pass it a thread keyword
(-pylab, -qt4thread, etc.). I had some trouble getting ipython to
work correctly, but I think that had to do with passing the thread
keyword before/after some other keywords.

Part of my question that I hope makes sense: Is there a way to unblock
the command line without closing the plot window when using an
interactive backend?

Thanks, and sorry about the misunderstandings / lack of clarity,
Adam

···

On Wed, Apr 29, 2009 at 8:29 AM, John Hunter <jdh2358@...287...> wrote:

On Wed, Apr 29, 2009 at 9:07 AM, keflavich <keflavich@...287...> wrote: