Memory leak

All,

I followed up the 'memory leak' discussion in the sourceforge list
and I know the Matplotlib-FAQ entry about this subject. I've also
seen John Hunter's post about the need of matching figure/close pairs.
Anyway, I still feel that there are problems in this subject, which
can be exposed by the following script (for Windows, but can easily
be adapted to Unix).

As can be seen by the results (also given below), there is a steady
increase in memory usage which is not recovered!

Any clues???
Clovis

···

######################################################################
#Begin of script
######################################################################
import pylab
import os
import time
N = 10 # number of loops to execute
SAVEFIG = True # SAVEFIG execution flag
SHOWFIG = True # SHOWFIG execution flag

pylab.matplotlib.use('TkAgg')
#pylab.matplotlib.use('Agg')
#pylab.matplotlib.use('PS')

fid = file('memory_report.txt','wt')
fid.write('%s\n' % time.asctime())
fid.write('OS version = %s\n' % os.sys.version)
fid.write('Matplotlib version = %s\n' % pylab.matplotlib.__version__)
fid.write('Matplotlib revision = %s\n' % pylab.matplotlib.__revision__)
fid.write('Matplotlit backend = %s\n' % pylab.matplotlib.get_backend())
fid.write('Column #0 = figure index\n')
fid.write('Column #1 = memory usage before figure\n')
fid.write('Column #2 = memory usage after figure\n')
fid.write('Column #3 = (after-before) memory\n')

pylab.ion()
a=pylab.arange(0,10)

def report_memory():
    ### Attention: the path to the pslist utility should be adjusted
according to installation!
    ### pslist.exe is a small utility that does the same as ps in Unix!
    ### It can be found at www.sysinternals.com/Utilities/PsList.html

    if os.sys.platform == 'win32':
        ps_exe_filename = os.path.join(os.getcwd(),'pslist.exe') #Build
ps filename
        a = os.popen('%s -m python' % ps_exe_filename).readlines() #Build
and execute command
        b = a[8]
        c = b.split()
        return int(c[3])
    else:
        print 'Sorry, you have to adapt the command for your OS!'
        return 0

def figureloop(N):
    for i in range(0,N):
        memory_usage_before = report_memory()
        fid.write('Memory usage before/after figure[%2d] = %8d' % (i,
memory_usage_before))
        pylab.figure(i)
        pylab.plot(a,2*a)
        figurename = 'fig%02d.eps' % i
        if SAVEFIG:
            pylab.savefig(figurename)
        if SHOWFIG:
            pylab.show()
        pylab.close(i)
        time.sleep(1.0) # wait 1.0 second before
inspecting memory usage
        if os.path.isfile(figurename): # remove figure ...
            os.remove(figurename)
        memory_usage_after = report_memory()
        delta_memory = memory_usage_after - memory_usage_before
        fid.write(' %8d %8d\n' % (memory_usage_after, delta_memory))
        print '%2d %6d %6d %6d' % (i, memory_usage_before,
memory_usage_after, delta_memory)

print 'Column #0 = figure index'
print 'Column #1 = memory usage before figure'
print 'Column #2 = memory usage after figure'
print 'Column #3 = (after-before) memory'
print 'There is a sleep time of 1s between each figure!'
print 'Close Figure[0] to continue execution!'

SAVEFIG = True # SAVEFIG execution flag
SHOWFIG = False # SHOWFIG execution flag
print('\nConfiguration SHOWFIG=%s SAVEFIG=%s' % (SHOWFIG, SAVEFIG))
fid.write('\nConfiguration SHOWFIG=%s SAVEFIG=%s\n' % (SHOWFIG, SAVEFIG))
figureloop(N)

SAVEFIG = False # SAVEFIG execution flag
SHOWFIG = True # SHOWFIG execution flag
print('\nConfiguration SHOWFIG=%s SAVEFIG=%s' % (SHOWFIG, SAVEFIG))
fid.write('\nConfiguration SHOWFIG=%s SAVEFIG=%s\n' % (SHOWFIG, SAVEFIG))
figureloop(N)

SAVEFIG = True # SAVEFIG execution flag
SHOWFIG = True # SHOWFIG execution flag
print('\nConfiguration SHOWFIG=%s SAVEFIG=%s' % (SHOWFIG, SAVEFIG))
fid.write('\nConfiguration SHOWFIG=%s SAVEFIG=%s\n' % (SHOWFIG, SAVEFIG))
figureloop(N)
######################################################################
#End of script
######################################################################

######################################################################
#The results obtained
######################################################################

Sun May 14 08:35:09 2006
OS version = 2.4.2 (#67, Oct 30 2005, 16:11:18) [MSC v.1310 32
bit (Intel)]
Matplotlib version = 0.86.2
Matplotlib revision = $Revision: 1.104 $
Matplotlit backend = TkAgg
Column #0 = figure index
Column #1 = memory usage before figure
Column #2 = memory usage after figure
Column #3 = (after-before) memory

Configuration SHOWFIG=False SAVEFIG=True
Memory usage before/after figure[ 0] = 15632 20168 4536
Memory usage before/after figure[ 1] = 20172 22532 2360
Memory usage before/after figure[ 2] = 22532 24912 2380
Memory usage before/after figure[ 3] = 24912 27256 2344
Memory usage before/after figure[ 4] = 27256 29700 2444
Memory usage before/after figure[ 5] = 29700 31980 2280
Memory usage before/after figure[ 6] = 31980 34328 2348
Memory usage before/after figure[ 7] = 34328 36696 2368
Memory usage before/after figure[ 8] = 36696 39052 2356
Memory usage before/after figure[ 9] = 39052 43160 4108

Configuration SHOWFIG=True SAVEFIG=False
Memory usage before/after figure[ 0] = 43160 43796 636
Memory usage before/after figure[ 1] = 43796 46080 2284
Memory usage before/after figure[ 2] = 46080 48392 2312
Memory usage before/after figure[ 3] = 48392 50736 2344
Memory usage before/after figure[ 4] = 50736 53020 2284
Memory usage before/after figure[ 5] = 53020 55420 2400
Memory usage before/after figure[ 6] = 55420 57672 2252
Memory usage before/after figure[ 7] = 57672 59984 2312
Memory usage before/after figure[ 8] = 59984 62312 2328
Memory usage before/after figure[ 9] = 62312 64620 2308

Configuration SHOWFIG=True SAVEFIG=True
Memory usage before/after figure[ 0] = 64620 68460 3840
Memory usage before/after figure[ 1] = 68460 70564 2104
Memory usage before/after figure[ 2] = 70564 71992 1428
Memory usage before/after figure[ 3] = 71992 75752 3760
Memory usage before/after figure[ 4] = 75752 77612 1860
Memory usage before/after figure[ 5] = 77612 79952 2340
Memory usage before/after figure[ 6] = 79952 82824 2872
Memory usage before/after figure[ 7] = 82824 84656 1832
Memory usage before/after figure[ 8] = 84656 86784 2128
Memory usage before/after figure[ 9] = 86784 89312 2528

Hello Clovis.
(Also, hello list; I <heart> matplotlib, thank you developers!)

I followed up the 'memory leak' discussion in the sourceforge list
and I know the Matplotlib-FAQ entry about this subject. I've also
seen John Hunter's post about the need of matching figure/close pairs.
Anyway, I still feel that there are problems in this subject, which
can be exposed by the following script (for Windows, but can easily
be adapted to Unix).
As can be seen by the results (also given below), there is a steady
increase in memory usage which is not recovered!
Any clues???

Could some of your difficulties could be related to Python's ill memory
management? It keeps a death grip on certain types of allocated memory,
never freeing it to the OS. Evan Jones made a patch which
helped that problem a bit. His patch has been incorporated into
Python 2.5:

http://evanjones.ca/python-memory.html

Joseph Sheedy
Technical Specialist, 3TIER Environmental Forecast Group
josephs@...1105... | (206)325-1573 x116

···

On Sun, May 14, 2006 at 09:04:27AM -0300, clovis@...339... wrote: