Memory leak using savefig with MacOSX backend?

Damon_McDougall2 · January 19, 2009, 11:09am

I’m looping over several files (about 1000) to produce a vector field plot for each data file I have. Doing this with the MacOSX backend appears to chew memory. My guess as to the source of the problem is the ‘savefig’ function (or possibly the way the MacOSX backend handles the saving of plots).

I opened Activity Monitor to watch the usage of memory increase. Below is code that recreates the problem.

[start]

import matplotlib

matplotlib.use(‘macosx’)

matplotlib.rc(‘font’,**{‘family’:‘serif’,‘serif’:[‘Computer Modern Roman’]})

matplotlib.rc(‘text’, usetex=True)

from pylab import *

i = 0

x = []

y = []

v1 = []

v2 = []

while(True):

f = open("%dresults.dat"%i,"r")

for line in f:

	x.append(float(line.split()[0]))

	y.append(float(line.split()[1]))

	v1.append(float(line.split()[2]))

	v2.append(float(line.split()[3]))

f.close()

hold(False)

figure(1)

quiver(x, y, v1, v2, color='b', units='width', scale=1.0)

xlabel('$x$')

ylabel('$y$')

grid(True)

print i

savefig('graph-%05d.pdf'%i)

close(1)

x = []

y = []

v1 = []

v2 = []

i = i + 1

[end]

Regards,

–Damon

Michiel_de_Hoon · January 19, 2009, 2:33pm

I am also finding the continuing increase in memory usage, but this also occurs with other backends (I tried tkagg and pdf) and also without the call to savefig. One possibility is a circular reference in the quiver function that prevents data from being cleaned up.

--Michiel

···

--- On Mon, 1/19/09, Damon McDougall <damon.mcdougall@...149...> wrote:

From: Damon McDougall <damon.mcdougall@...149...>
Subject: [matplotlib-devel] Memory leak using savefig with MacOSX backend?
To: "matplotlib development list" <matplotlib-devel@lists.sourceforge.net>
Date: Monday, January 19, 2009, 6:09 AM
I'm looping over several files (about 1000) to produce a
vector field plot for each data file I have. Doing this with
the MacOSX backend appears to chew memory. My guess as to
the source of the problem is the 'savefig' function
(or possibly the way the MacOSX backend handles the saving
of plots).

I opened Activity Monitor to watch the usage of memory
increase. Below is code that recreates the problem.

[start]

import matplotlib
matplotlib.use('macosx')
matplotlib.rc('font',**{'family':'serif','serif':['Computer
Modern Roman']})
matplotlib.rc('text', usetex=True)
from pylab import *

i = 0
x =
y =
v1 =
v2 =

while(True):
  f = open("%dresults.dat"%i,"r")
  for line in f:
    x.append(float(line.split()[0]))
    y.append(float(line.split()[1]))
    v1.append(float(line.split()[2]))
    v2.append(float(line.split()[3]))
  f.close()
  hold(False)
  figure(1)
  quiver(x, y, v1, v2, color='b',
units='width', scale=1.0)
  xlabel('x')
  ylabel('y')
  grid(True)
  print i
  savefig('graph-%05d.pdf'%i)
  close(1)
  x =
  y =
  v1 =
  v2 =
  i = i + 1

[end]

Regards,
--Damon

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

_Michael_Abshoff · January 19, 2009, 2:36pm

I am also finding the continuing increase in memory usage, but this also occurs with other backends (I tried tkagg and pdf) and also without the call to savefig. One possibility is a circular reference in the quiver function that prevents data from being cleaned up.

I do not know how relevant this is to the problem at hand, but I have
observed memory leaks in the Delaunay code in matplotlib 0.98.3. I
hadn't had the chance to upgrade to 0.98.5.x yet, but judging from the
release notes the issue was not fixed. Since the leak happens from
inside Sage I need to find out what exactly causes those leaks before
poking around and fixing them.

--Michiel

Cheers,

Michael

···

On Mon, Jan 19, 2009 at 6:33 AM, Michiel de Hoon <mjldehoon@...42...> wrote:

Michael_Droettboom · January 20, 2009, 2:17pm

I'm not able to see this here (RHEL 4, matplotlib SVN trunk) with either the PDF or TkAgg backends. The attached graphs show that memory usage levels off after the first 3-4 iterations. I only did 10 iterations here to save bandwidth, but I also tested 500 and the results were the same. As I'm not on a Mac, I'm unable to test the Cocoa backend. What tool are you using to detect the memory leaks? Many tools that work at the high level (such as Windows Task Manager etc.) are full of pitfalls, so in my experience you really need to use a tool designed for the purpose of detecting memory leaks (such as Valgrind/Massif, or the Apple one whose name escapes me).

I modified Damon's script to use random data (since his data was not attached). If this script does not leak memory for either of you, then perhaps the leak is data-dependent, in which case I'll either need to obtain or better simulate that data.

Mike
Michiel de Hoon wrote:

quiver_memleak.py (523 Bytes)

massif.31653.ps (151 KB)

massif.31723.ps (126 KB)

···

I am also finding the continuing increase in memory usage, but this also occurs with other backends (I tried tkagg and pdf) and also without the call to savefig. One possibility is a circular reference in the quiver function that prevents data from being cleaned up.

--Michiel

--- On Mon, 1/19/09, Damon McDougall <damon.mcdougall@...149...> wrote:

From: Damon McDougall <damon.mcdougall@...149...>
Subject: [matplotlib-devel] Memory leak using savefig with MacOSX backend?
To: "matplotlib development list" <matplotlib-devel@lists.sourceforge.net>
Date: Monday, January 19, 2009, 6:09 AM
I'm looping over several files (about 1000) to produce a
vector field plot for each data file I have. Doing this with
the MacOSX backend appears to chew memory. My guess as to
the source of the problem is the 'savefig' function
(or possibly the way the MacOSX backend handles the saving
of plots).

I opened Activity Monitor to watch the usage of memory
increase. Below is code that recreates the problem.

[start]

import matplotlib
matplotlib.use('macosx')
matplotlib.rc('font',**{'family':'serif','serif':['Computer
Modern Roman']})
matplotlib.rc('text', usetex=True)
from pylab import *

i = 0
x =
y =
v1 =
v2 =

while(True):
  f = open("%dresults.dat"%i,"r")
  for line in f:
    x.append(float(line.split()[0]))
    y.append(float(line.split()[1]))
    v1.append(float(line.split()[2]))
    v2.append(float(line.split()[3]))
  f.close()
  hold(False)
  figure(1)
  quiver(x, y, v1, v2, color='b',
units='width', scale=1.0)
  xlabel('x')
  ylabel('y')
  grid(True)
  print i
  savefig('graph-%05d.pdf'%i)
  close(1)
  x =
  y =
  v1 =
  v2 =
  i = i + 1

[end]

Regards,
--Damon

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA