Newbie. Memory useage question

system · April 23, 2007, 10:48pm

I’m producing series of plots (spectograms)
in a program loop using imshow and saving each plot to .png. Even
though I close() each plot after each savefig(…), the memory does not
appear to be freed up, and the memory useage goes up and up as the program
runs (and stalls the computer as it thrashes the page file).

This is the essence of the code:

for i in range(…):

pylab.imshow(logPSDs[i]...)

pylab.colorbar()

pylab.savefig(plotName[i])

pylab.close()

Is there anything that I should be doing
to stop this memory “wastage”? (The plots themselves are
fantastic!)

UNITED GROUP

This email message is the property of United Group. The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, you may not disclose, copy or distribute this email, nor take or omit to take any action in reliance on it. United Group accepts no liability for any damage caused by this email or any attachments due to viruses, interference, interception, corruption or unauthorised access.

If you have received this email in error, please notify United Group immediately by email to the sender’s email address and delete this document.

Matthias_Michler · April 24, 2007, 9:01am

Hi,

a friend gave me the little example I attached.
He uses 'gca().images = ' to delete the images. I'm not sure about memory
usage of that method, but I think deleting images (using clf() or the above
way) is quite important, because otherwise one more image is drawn above the
existing ones.

best regards,
Matthias

----------------------------------------------------------------------------------------------------------

import pylab
import numpy

N = 100 # to generate a matrix
A = numpy.ones(N)[numpy.newaxis, :]*numpy.arange(N)[:, numpy.newaxis]/(N-1)

pylab.figure(0)
ax = pylab.subplot(111)
for item in pylab.cm.cmapnames[:3]: # plot some cmap examples
    print " doing cm." + item
    eval("pylab.imshow(A, cmap=pylab.cm."+item+", interpolation='nearest')")
    pylab.savefig(item+'.png')
    ax.images = # delete images

···

---------------------------------------------------------------------------------------------------------

On Tuesday 24 April 2007 00:48, brett.mcsweeney@...1537... wrote:

I'm producing series of plots (spectograms) in a program loop using imshow
and saving each plot to .png. Even though I close() each plot after each
savefig(...), the memory does not appear to be freed up, and the memory
useage goes up and up as the program runs (and stalls the computer as it
thrashes the page file).

This is the essence of the code:

for i in range(..):
    pylab.imshow(logPSDs[i]...)
    pylab.colorbar()
    pylab.savefig(plotName[i])
    pylab.close()

Is there anything that I should be doing to stop this memory "wastage"?
(The plots themselves are fantastic!)

UNITED GROUP
This email message is the property of United Group. The information in this
email is confidential and may be legally privileged. It is intended solely
for the addressee. Access to this email by anyone else is unauthorised. If
you are not the intended recipient, you may not disclose, copy or
distribute this email, nor take or omit to take any action in reliance on
it. United Group accepts no liability for any damage caused by this email
or any attachments due to viruses, interference, interception, corruption
or unauthorised access. If you have received this email in error, please
notify United Group immediately by email to the sender's email address and
delete this document.

Eric_Firing1 · April 24, 2007, 6:56pm

There seem to be big memory leak problems with all interactive backends. It looks like you don't need an interactive backend. If this is the case, then instead of starting with "import pylab", try:

import matplotlib
matplotlib.use('Agg')
import pylab

Eric

···

brett.mcsweeney@...1537... wrote:

I'm producing series of plots (spectograms) in a program loop using imshow and saving each plot to .png. Even though I close() each plot after each savefig(...), the memory does not appear to be freed up, and the memory useage goes up and up as the program runs (and stalls the computer as it thrashes the page file).

This is the essence of the code:

for i in range(..):
    pylab.imshow(logPSDs[i]...)
    pylab.colorbar()
    pylab.savefig(plotName[i])
    pylab.close()

Is there anything that I should be doing to stop this memory "wastage"? (The plots themselves are fantastic!)

_John_Hunter1 · April 24, 2007, 7:05pm

I'm producing series of plots (spectograms) in a program loop using imshow
and saving each plot to .png. Even though I close() each plot after each
savefig(...), the memory does not appear to be freed up, and the memory
useage goes up and up as the program runs (and stalls the computer as it
thrashes the page file).

This is the essence of the code:

for i in range(..):
    pylab.imshow(logPSDs[i]...)
    pylab.colorbar()
    pylab.savefig(plotName[i])
    pylab.close()

The following code does not appear to leak:

import matplotlib
matplotlib.use('Agg')
from matplotlib.cbook import report_memory
import matplotlib.numerix as nx
import pylab

for i in range(100):
    print i, report_memory(i)
    fig = pylab.figure(1)
    X = nx.mlab.rand(100,100)
    pylab.imshow(X)
    pylab.colorbar()
    pylab.savefig('_test%d'%i)
    pylab.close(1)

Are you running your program in a GUI? Eric points out there are some
leaks in the GUI canvases which we have not succeeded in tracking
down. If you only want image generation, you can use an image backend
w/o leaks. The one thing to be careful of is to make sure you are not
overplotting multiple images onto the same Axes, eg by clearing the
figure or axes if you are reusing it.

JDH

···

On 4/23/07, brett.mcsweeney@...1537... <brett.mcsweeney@...1537...> wrote:

Is there anything that I should be doing to stop this memory "wastage"?
(The plots themselves are fantastic!)
UNITED GROUP
This email message is the property of United Group. The information in this
email is confidential and may be legally privileged. It is intended solely
for the addressee. Access to this email by anyone else is unauthorised. If
you are not the intended recipient, you may not disclose, copy or distribute
this email, nor take or omit to take any action in reliance on it. United
Group accepts no liability for any damage caused by this email or any
attachments due to viruses, interference, interception, corruption or
unauthorised access.
If you have received this email in error, please notify United Group
immediately by email to the sender's email address and delete this document.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

system · April 26, 2007, 1:05am

Thanks John, that works perfectly.

Best regards,

Brett McSweeney

`On 4/23/07, brett.mcsweeney@…1537…

I’m producing series of plots (spectograms) in a program loop using
imshow

and saving each plot to .png. Even though I close() each plot
after each

savefig(…), the memory does not appear to be freed up, and the memory

useage goes up and up as the program runs (and stalls the computer
as it

thrashes the page file).

This is the essence of the code:

for i in range(…):

pylab.imshow(logPSDs[i]...)

pylab.colorbar()

pylab.savefig(plotName[i])

pylab.close()

The following code does not appear to leak:

import matplotlib

matplotlib.use(‘Agg’)

from matplotlib.cbook import report_memory

import matplotlib.numerix as nx

import pylab

for i in range(100):

print i, report_memory(i)

fig = pylab.figure(1)

X = nx.mlab.rand(100,100)

pylab.imshow(X)

pylab.colorbar()

pylab.savefig('_test%d'%i)

pylab.close(1)

Are you running your program in a GUI? Eric points out there are
some

leaks in the GUI canvases which we have not succeeded in tracking

down. If you only want image generation, you can use an image backend

w/o leaks. The one thing to be careful of is to make sure you are
not

overplotting multiple images onto the same Axes, eg by clearing the

figure or axes if you are reusing it.

JDH

Is there anything that I should be doing to stop this memory “wastage”?

(The plots themselves are fantastic!)

UNITED GROUP

This email message is the property of United Group. The information
in this

email is confidential and may be legally privileged. It is intended
solely

for the addressee. Access to this email by anyone else is unauthorised.
If

you are not the intended recipient, you may not disclose, copy or
distribute

this email, nor take or omit to take any action in reliance on it.
United

Group accepts no liability for any damage caused by this email or
any

attachments due to viruses, interference, interception, corruption
or

unauthorised access.

If you have received this email in error, please notify United
Group

immediately by email to the sender’s email address and delete this
document.

···

<brett.mcsweeney@…1537…> wrote:

This SF.net email is sponsored by DB2 Express

Download DB2 Express C - the FREE version of DB2 express and take

control of your XML. No limits. Just data. Click to get it now.

http://sourceforge.net/powerbar/db2/

Matplotlib-users mailing list

Matplotlib-users@lists.sourceforge.net

matplotlib-users List Signup and Options

This email has been scanned by the MessageLabs Email Security System.

For more information please visit Email Security

`

UNITED GROUP

This email message is the property of United Group. The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, you may not disclose, copy or distribute this email, nor take or omit to take any action in reliance on it. United Group accepts no liability for any damage caused by this email or any attachments due to viruses, interference, interception, corruption or unauthorised access.

If you have received this email in error, please notify United Group immediately by email to the sender’s email address and delete this document.