matplotlib is slow

Community matplotlib-users
#1

Hello everyone,

I would like to draw the attention on the slow startup of matplotlib.

Indeed, running matplotlib takes a long time.

I performed the following sequence :

#!/bin/bash

for i in * ; do python2 -c "from temp import * ; plot_(\"${i}\") " ; done

with temp.py like this :

#!/usr/bin/env python2

import sys
import matplotlib.pyplot as plt
import read_data as rd
import numpy

def plot_( fname ):
    P,I = rd.read_data(fname)
    Iprime = [ l / k for k , l in zip( numpy.diff(P) , numpy.diff(I) ) ]

    fig = plt.figure()
    ax1 = fig.add_subplot(211)
    ax2 = fig.add_subplot(212)

    ax1.plot(P,I)
    ax2.plot(P[:-1],Iprime)

    fig.savefig( fname + ".pdf", format='pdf' )

And it seems the longer operation is to import matplotlib.pyplot.

Does something could be done to improve the loading time of this module ?

Thank you very much.

greatings,

David Kremer

#2

I would recommend running the import in the Python profiler to determine
where most of the time is going. When I investigated this a few years
back, it was mainly due to loading the GUI toolkits, which are
understandably quite large. You can avoid most of that by using the Agg
backend. If you're using the Agg backend and still experiencing
slowness, it may be that load-up issues have crept back into matplotlib
since then -- but we need profiling data to figure out where and how.

Mike

···

________________________________________
From: David Kremer [david.kremer.dk@...287...]
Sent: Monday, March 28, 2011 9:36 AM
To: Matplotlib Users
Subject: [Matplotlib-users] matplotlib is slow

Hello everyone,

I would like to draw the attention on the slow startup of matplotlib.

Indeed, running matplotlib takes a long time.

I performed the following sequence :

#!/bin/bash

for i in * ; do python2 -c "from temp import * ; plot_(\"${i}\") " ; done

with temp.py like this :

#!/usr/bin/env python2

import sys
import matplotlib.pyplot as plt
import read_data as rd
import numpy

def plot_( fname ):
    P,I = rd.read_data(fname)
    Iprime = [ l / k for k , l in zip( numpy.diff(P) , numpy.diff(I) ) ]

    fig = plt.figure()
    ax1 = fig.add_subplot(211)
    ax2 = fig.add_subplot(212)

    ax1.plot(P,I)
    ax2.plot(P[:-1],Iprime)

    fig.savefig( fname + ".pdf", format='pdf' )

And it seems the longer operation is to import matplotlib.pyplot.

Does something could be done to improve the loading time of this module ?

Thank you very much.

greatings,

David Kremer

------------------------------------------------------------------------------
Enable your software for Intel(R) Active Management Technology to meet the
growing manageability and security demands of your customers. Businesses
are taking advantage of Intel(R) vPro (TM) technology - will your software
be a part of the solution? Download the Intel(R) Manageability Checker
today! http://p.sf.net/sfu/intel-dev2devmar
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

#3

I would recommend running the import in the Python profiler to determine
where most of the time is going. When I investigated this a few years
back, it was mainly due to loading the GUI toolkits, which are
understandably quite large. You can avoid most of that by using the Agg
backend. If you're using the Agg backend and still experiencing
slowness, it may be that load-up issues have crept back into matplotlib
since then -- but we need profiling data to figure out where and how.

Mike

Thank you a lot for your answer.

I noticed than _matplotlib.pyplot_ is longer to be imported the first time than
if it has already been imported previously (maybe things are already loaded in
ram memory), and we don't need to fetch it from the hard drive thanks to the
kernel.

As far I see, the function calls are the same for the two logs I obtained,
except than the first took 6s instead of 1.4s.

The two logs have been obtained using :
<code>
python -m cProfile temp.py
</code>

where temp.py consist of two lines :

<code>
#!/usr/bin/env python2

import matplotlib.pyplot
</code>

diff_first_and_second_log.txt (61.3 KB)

log_matplotlib.pyplot1.txt (111 KB)

log_matplotlib.pyplot2.txt (108 KB)

#4

Resurrecting an old thread here…

I would recommend running the import in the Python profiler to determine

where most of the time is going. When I investigated this a few years

back, it was mainly due to loading the GUI toolkits, which are

understandably quite large. You can avoid most of that by using the Agg

backend. If you’re using the Agg backend and still experiencing

slowness, it may be that load-up issues have crept back into matplotlib

since then – but we need profiling data to figure out where and how.

Importing Matplotlib is very slow for me, too. For a wxPython application with embedded Matplotlib, I am getting “load” times of > 20 seconds when “cold” importing matplotlib, with this (circa mid 2004) computer setup: Windows XP, sp3, Intel Pentium, 1.70 Ghz, 1 GB RAM.

This is, by the way, an import well after Python and wxPython have already been loaded into RAM, as it happens by a user action, so none of
the time involved here is due to loading Python or wxPython (they both load more quickly–about 10 seconds to cold import them, my code, images, and some other libraries).

First of all: does that amount of time seem appropriate for that fast of a system–or is that too long? It definitely feels way too long from a user perspective (for comparison Word or PowerPoint loads on this computer in about 2.5 seconds).

Trying to improve it and following this old thread, I have switched to

matplotlib.use(‘Agg’)

instead of

matplotlib.use(‘wxAgg’)

as suggested to speed things up…but it is no faster.

I see, though, that I also have lines such as:

from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg

from matplotlib.backends.backend_wxagg import NavigationToolbar2WxAgg

Would the presence of these imports obviate the fact that I switched to using the Agg instead of the wxAgg? If so, is there any way to use something faster here (I suspect not but thought I’d ask).

Also, what else should I consider doing to reduce the import time significantly? (I have to learn how to use the profiler, so I haven’t done that yet).

Thanks,
Che

···

On Tue, Mar 29, 2011 at 3:23 PM, David Kremer <david@…3516…> wrote:

Mike

Thank you a lot for your answer.

I noticed than matplotlib.pyplot is longer to be imported the first time than

if it has already been imported previously (maybe things are already loaded in

ram memory), and we don’t need to fetch it from the hard drive thanks to the

kernel.

As far I see, the function calls are the same for the two logs I obtained,

except than the first took 6s instead of 1.4s.

The two logs have been obtained using :

python -m cProfile temp.py

where temp.py consist of two lines :

#!/usr/bin/env python2

import matplotlib.pyplot

Xperia™ PLAY

It’s a major breakthrough. An authentic gaming

smartphone on the nation’s most reliable network.

And it wants your games.

http://p.sf.net/sfu/verizon-sfdev

Matplotlib-users mailing list

Matplotlib-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-users

#5

I think using the profiler is the best bet here. We've used that in the past to track down things that take a long time to import quite successfully. I'm not seeing any slowness here, so that is likely do to an environmental difference on your machine, implying you'll really need to run the profiler yourself. I recommend runsnakerun to examine the profile output -- if you have trouble interpreting it, feel free to send me your raw profiler data to me off list.

Mike

···

On 12/31/2012 02:21 PM, C M wrote:

Resurrecting an old thread here....

On Tue, Mar 29, 2011 at 3:23 PM, David Kremer <david@…3516… > <mailto:david@…3516…>> wrote:

    > I would recommend running the import in the Python profiler to
    determine
    > where most of the time is going. When I investigated this a few
    years
    > back, it was mainly due to loading the GUI toolkits, which are
    > understandably quite large. You can avoid most of that by using
    the Agg
    > backend. If you're using the Agg backend and still experiencing
    > slowness, it may be that load-up issues have crept back into
    matplotlib
    > since then -- but we need profiling data to figure out where and
    how.

Importing Matplotlib is very slow for me, too. For a wxPython application with embedded Matplotlib, I am getting "load" times of > 20 seconds when "cold" importing matplotlib, with this (circa mid 2004) computer setup: Windows XP, sp3, Intel Pentium, 1.70 Ghz, 1 GB RAM.

This is, by the way, an import well after Python and wxPython have already been loaded into RAM, as it happens by a user action, so none of the time involved here is due to loading Python or wxPython (they both load more quickly--about 10 seconds to cold import them, my code, images, and some other libraries).

First of all: does that amount of time seem appropriate for that fast of a system--or is that too long? It definitely *feels* way too long from a user perspective (for comparison Word or PowerPoint loads on this computer in about 2.5 seconds).

Trying to improve it and following this old thread, I have switched to

matplotlib.use('Agg')

instead of

matplotlib.use('wxAgg')

as suggested to speed things up...but it is no faster.

I see, though, that I also have lines such as:

from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg
from matplotlib.backends.backend_wxagg import NavigationToolbar2WxAgg

Would the presence of these imports obviate the fact that I switched to using the Agg instead of the wxAgg? If so, is there any way to use something faster here (I suspect not but thought I'd ask).

Also, what else should I consider doing to reduce the import time significantly? (I have to learn how to use the profiler, so I haven't done that yet).

Thanks,
Che

    >
    > Mike

    Thank you a lot for your answer.

    I noticed than _matplotlib.pyplot_ is longer to be imported the
    first time than
    if it has already been imported previously (maybe things are
    already loaded in
    ram memory), and we don't need to fetch it from the hard drive
    thanks to the
    kernel.

    As far I see, the function calls are the same for the two logs I
    obtained,
    except than the first took 6s instead of 1.4s.

    The two logs have been obtained using :
    <code>
    python -m cProfile temp.py
    </code>

    where temp.py consist of two lines :

    <code>
    #!/usr/bin/env python2

    import matplotlib.pyplot
    </code>

    ------------------------------------------------------------------------------
    Xperia(TM) PLAY
    It's a major breakthrough. An authentic gaming
    smartphone on the nation's most reliable network.
    And it wants your games.
    http://p.sf.net/sfu/verizon-sfdev
    _______________________________________________
    Matplotlib-users mailing list
    Matplotlib-users@lists.sourceforge.net
    <mailto:Matplotlib-users@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/matplotlib-users

------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122412

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users