Before this, I tried what you suggested (explicitly naming the figure). The
output for 3 cycles is shown bellow in result1. It doesn't change much in the
story, the memory still increases by about 6Mb. I also show the result when I
uncomment the savefig line (result2) which also shows an increase by about 0.6
Mb.
The final code is at the bottom. I still don't uderstand why it doesn't work by
the first method, but at least it works...
···
--------------
RESULT 1:
memory used 331.55 Mb
Building individual images...
building file ...
memory used for image - step1 331.55 Mb
memory used for image - step2 331.55 Mb
memory used for image - step3 331.55 Mb
memory used for image - step4 331.55 Mb -> created fig
memory used for image - step5 331.55 Mb -> cleared fig
memory used for image - step6 329.11 Mb -> built image - imshow
memory used for image - step7 329.20 Mb -> added colorbar
memory used for image - step8 329.20 Mb -> turned axis off
memory used for image - step9 336.74 Mb -> saved fig
memory used for image - step10 336.74 Mb -> cleared cached text
memory used for image - step11 336.74 Mb -> cleared image (axes)
memory used for image - step12 336.74 Mb
memory used for image - step13 328.31 Mb -> closed figure
memory used for image - step14 328.31 Mb -> called garbage collector
total time = 1.84299993515
done
memory used 328.31 Mb
building file ...
memory used for image - step1 328.31 Mb
memory used for image - step2 328.31 Mb
memory used for image - step3 328.31 Mb
memory used for image - step4 342.26 Mb -> created fig
memory used for image - step5 342.26 Mb -> cleared fig
memory used for image - step6 339.84 Mb -> built image - imshow
memory used for image - step7 339.94 Mb -> added colorbar
memory used for image - step8 339.94 Mb -> turned axis off
memory used for image - step9 347.50 Mb -> saved fig
memory used for image - step10 347.50 Mb -> cleared cached text
memory used for image - step11 347.50 Mb -> cleared image (axes)
memory used for image - step12 347.50 Mb
memory used for image - step13 333.66 Mb -> closed figure
memory used for image - step14 333.66 Mb -> called garbage collector
total time = 1.93700003624
done
memory used 333.66 Mb
building file ...
memory used for image - step1 333.66 Mb
memory used for image - step2 333.66 Mb
memory used for image - step3 333.66 Mb
memory used for image - step4 347.81 Mb -> created fig
memory used for image - step5 347.81 Mb -> cleared fig
memory used for image - step6 345.40 Mb -> built image - imshow
memory used for image - step7 345.51 Mb -> added colorbar
memory used for image - step8 345.51 Mb -> turned axis off
memory used for image - step9 353.57 Mb -> saved fig
memory used for image - step10 353.57 Mb -> cleared cached text
memory used for image - step11 353.57 Mb -> cleared image (axes)
memory used for image - step12 353.57 Mb
memory used for image - step13 339.73 Mb -> closed figure
memory used for image - step14 339.73 Mb -> called garbage collector
total time = 1.67200016975
done
--
RESULT 2:
memory used 345.79 Mb
Building individual images...
building file
I:/EggShell/ESRF-sc1579-201104/OriginalDataPolarTransform/egg4_8_0500_polar_mask.png
...
memory used for image - step1 345.79 Mb
memory used for image - step2 345.79 Mb
memory used for image - step3 345.79 Mb
memory used for image - step4 359.92 Mb -> created fig
memory used for image - step5 359.92 Mb -> cleared fig
memory used for image - step6 357.51 Mb -> built image - imshow
memory used for image - step7 357.61 Mb -> added colorbar
memory used for image - step8 357.61 Mb -> turned axis off
memory used for image - step9 357.61 Mb -> saved fig - DISABLED
memory used for image - step10 357.61 Mb -> cleared cached text
memory used for image - step11 357.61 Mb -> cleared image (axes)
memory used for image - step12 357.61 Mb
memory used for image - step13 343.77 Mb -> closed figure
memory used for image - step14 343.77 Mb -> called garbage collector
total time = 1.25
done
memory used 343.77 Mb
building file
I:/EggShell/ESRF-sc1579-201104/OriginalDataPolarTransform/egg4_8_0499_polar_mask.png
...
memory used for image - step1 343.77 Mb
memory used for image - step2 343.77 Mb
memory used for image - step3 344.02 Mb
memory used for image - step4 358.22 Mb -> created fig
memory used for image - step5 358.22 Mb -> cleared fig
memory used for image - step6 358.28 Mb -> built image - imshow
memory used for image - step7 358.38 Mb -> added colorbar
memory used for image - step8 358.38 Mb -> turned axis off
memory used for image - step9 358.38 Mb -> saved fig - DISABLED
memory used for image - step10 358.38 Mb -> cleared cached text
memory used for image - step11 358.38 Mb -> cleared image (axes)
memory used for image - step12 358.38 Mb
memory used for image - step13 344.54 Mb -> closed figure
memory used for image - step14 344.54 Mb -> called garbage collector
total time = 0.625
done
memory used 344.54 Mb
building file
I:/EggShell/ESRF-sc1579-201104/OriginalDataPolarTransform/egg4_8_0498_polar_mask.png
...
memory used for image - step1 344.54 Mb
memory used for image - step2 344.54 Mb
memory used for image - step3 344.54 Mb
memory used for image - step4 358.74 Mb -> created fig
memory used for image - step5 358.75 Mb -> cleared fig
memory used for image - step6 358.81 Mb -> built image - imshow
memory used for image - step7 358.91 Mb -> added colorbar
memory used for image - step8 358.91 Mb -> turned axis off
memory used for image - step9 358.91 Mb -> saved fig - DISABLED
memory used for image - step10 358.91 Mb -> cleared cached text
memory used for image - step11 358.91 Mb -> cleared image (axes)
memory used for image - step12 358.91 Mb
memory used for image - step13 345.06 Mb -> closed figure
memory used for image - step14 345.06 Mb -> called garbage collector
total time = 0.640000104904
done
--
RESULT 3:
memory used 325.69 Mb
Building individual images...
building file ...
memory used for image - step1 325.69 Mb
memory used for image - step2 325.41 Mb
memory used for image - step3 325.41 Mb
memory used for image - step4 325.41 Mb -> created fig - gcf
memory used for image - step5 325.41 Mb -> cleared fig
memory used for image - step6 322.70 Mb -> built image - imshow
memory used for image - step7 322.61 Mb -> added colorbar
memory used for image - step8 322.61 Mb -> turned axis off
memory used for image - step9 330.62 Mb -> saved fig
memory used for image - step10 330.62 Mb -> cleared cached text
memory used for image - step11 330.62 Mb -> cleared image (axes)
memory used for image - step12 330.62 Mb
memory used for image - step13 330.62 Mb -> closed figure - DISABLED
memory used for image - step14 325.12 Mb -> called garbage collector
total time = 1.28099989891
done
memory used 325.12 Mb
building file ...
memory used for image - step1 325.12 Mb
memory used for image - step2 325.12 Mb
memory used for image - step3 325.12 Mb
memory used for image - step4 325.12 Mb -> created fig - gcf
memory used for image - step5 325.12 Mb -> cleared fig
memory used for image - step6 322.65 Mb -> built image - imshow
memory used for image - step7 322.65 Mb -> added colorbar
memory used for image - step8 322.65 Mb -> turned axis off
memory used for image - step9 330.63 Mb -> saved fig
memory used for image - step10 330.63 Mb -> cleared cached text
memory used for image - step11 330.63 Mb -> cleared image (axes)
memory used for image - step12 330.63 Mb
memory used for image - step13 330.63 Mb -> closed figure - DISABLED
memory used for image - step14 325.13 Mb -> called garbage collector
total time = 1.31299996376
done
memory used 325.13 Mb
building file ...
memory used for image - step1 325.13 Mb
memory used for image - step2 325.13 Mb
memory used for image - step3 325.13 Mb
memory used for image - step4 325.13 Mb -> created fig - gcf
memory used for image - step5 325.13 Mb -> cleared fig
memory used for image - step6 322.62 Mb -> built image - imshow
memory used for image - step7 322.63 Mb -> added colorbar
memory used for image - step8 322.63 Mb -> turned axis off
memory used for image - step9 330.64 Mb -> saved fig
memory used for image - step10 330.64 Mb -> cleared cached text
memory used for image - step11 330.64 Mb -> cleared image (axes)
memory used for image - step12 330.64 Mb
memory used for image - step13 330.64 Mb -> closed figure - DISABLED
memory used for image - step14 325.09 Mb -> called garbage collector
total time = 1.31299996376
done
--
def generateImshow(yarray,
xscansize,yscansize,
xscanstep,yscanstep,
outputfilename,
initimagesize = 10):
'''Use this function to generate matplotlib images without displaying
them'''
#
val = float(getMemoryUsage())/1000000
print 'memory used for image - step1 ','%.2f' % val,' Mb'
#reshape according to scan
yarray = na.reshape(yarray,(yscansize,-1))
val = float(getMemoryUsage())/1000000
print 'memory used for image - step2 ','%.2f' % val,' Mb'
#build image via matplotlib
ximagesize = xscansize*xscanstep
yimagesize = yscansize*yscanstep
xyimageratio = float(ximagesize)/yimagesize
#print xyimageratio
if xyimageratio > 1: ximagesize,yimagesize =
initimagesize,initimagesize*xyimageratio
else: ximagesize,yimagesize = initimagesize*xyimageratio,initimagesize
val = float(getMemoryUsage())/1000000
print 'memory used for image - step3 ','%.2f' % val,' Mb'
#fig1 = pylab.figure(figsize=(ximagesize,yimagesize),dpi=100)
#fig1 = pylab.figure(1,figsize=(ximagesize,yimagesize),dpi=100)
fig1 = pylab.gcf()
fig1.set_figsize_inches((ximagesize,yimagesize))
fig1.set_dpi(100)
val = float(getMemoryUsage())/1000000
print 'memory used for image - step4 ','%.2f' % val,' Mb'
fig1.clear()
val = float(getMemoryUsage())/1000000
print 'memory used for image - step5 ','%.2f' % val,' Mb'
#pylab.title('blahblah')
im1 = pylab.imshow(yarray,
origin='lower',
#interpolation='nearest', #i.e. pixel
interpolation='bicubic', #i.e. smooth
#vmin=minvalue,
#vmax=maxvalue,
cmap = pylab.cm.bone,
)
val = float(getMemoryUsage())/1000000
print 'memory used for image - step6 ','%.2f' % val,' Mb'
colbar1 = pylab.colorbar()
val = float(getMemoryUsage())/1000000
print 'memory used for image - step7 ','%.2f' % val,' Mb'
#pylab.bone()
pylab.axis('off')
val = float(getMemoryUsage())/1000000
print 'memory used for image - step8 ','%.2f' % val,' Mb'
#save figure
pylab.savefig(outputfilename)
val = float(getMemoryUsage())/1000000
print 'memory used for image - step9 ','%.2f' % val,' Mb'
mpl.text.Text.cached = {}
val = float(getMemoryUsage())/1000000
print 'memory used for image - step10 ','%.2f' % val,' Mb'
pylab.cla()
val = float(getMemoryUsage())/1000000
print 'memory used for image - step11 ','%.2f' % val,' Mb'
del im1,colbar1 #doesn't bring anything
val = float(getMemoryUsage())/1000000
print 'memory used for image - step12 ','%.2f' % val,' Mb'
#pylab.close(fig1)
#pylab.close(1)
val = float(getMemoryUsage())/1000000
print 'memory used for image - step13 ','%.2f' % val,' Mb'
#pylab.close('all')
gc.collect()
val = float(getMemoryUsage())/1000000
print 'memory used for image - step14 ','%.2f' % val,' Mb'
-------------------------
Selon John Hunter <jdhunter@...4...>:
> Dear all, I posted a message two days ago mentionning
> memory problems while saving a large number of images,
> which I guess was not so clear... I've worked a bit on
> it and I think the questions seem clearer to me now. I
> have paid particular attention to the threads in the
> user mailing list dealing with memory problems.
What happens if you explicitly name the figure (ef figure(1))
for i in range(indEnd):
figure(1)
subplot(221)
plot(ind, xx)
subplot(222)
X = rand(50,50)
imshow(X)
subplot(223)
scatter(rand(50), rand(50))
subplot(224)
pcolor(10*rand(50,50))
savefig('tmp%d' % i, dpi = 75)
close(1)
See also the FAQ http://matplotlib.sourceforge.net/faq.html#LEAKS
which shows the canonical way to make multiple plots to prevent leaks,
namely, pairing a close with each figure creation?
I'm not sure why you are seeing a problem right now, but my memory
leak test script memleak_hawaii3.py does not appear to be leaking
#!/usr/bin/env python
import os, sys, time
import matplotlib
#matplotlib.interactive(True)
#matplotlib.use('Cairo')
matplotlib.use('Agg')
from pylab import *
def report_memory(i):
pid = os.getpid()
a2 = os.popen('ps -p %d -o rss,sz' % pid).readlines()
print i, ' ', a2[1],
return int(a2[1].split()[1])
# take a memory snapshot on indStart and compare it with indEnd
indStart, indEnd = 30, 201
for i in range(indEnd):
figure(1); clf()
subplot(221)
t1 = arange(0.0, 2.0, 0.01)
y = sin(2*pi*t1)
plot(t1,y,'-')
plot(t1, rand(len(t1)), 's', hold=True)
subplot(222)
X = rand(50,50)
imshow(X)
subplot(223)
scatter(rand(50), rand(50), s=100*rand(50), c=rand(50))
subplot(224)
pcolor(10*rand(50,50))
savefig('tmp%d' % i, dpi = 75)
close(1)
val = report_memory(i)
if i==indStart: start = val # wait a few cycles for memory usage to
stabilize
end = val
print 'Average memory consumed per loop: %1.4fk bytes\n' %
((end-start)/float(indEnd-indStart))