Eric,
It turns out that the key difference is in pygobject. I was using 2.13.2 (from source) here at work, and Ubuntu Feisty currently ships 2.12.3. 2.13.x is an unstable branch, of course, (gnome-related packages follow the even/odd versioning scheme), so I shouldn't have been using it to test with in the first place...
...however, there are only about 40 patches between 2.12.3 and 2.13.2. Two of them are very crucial memory leak patches:
http://svn.gnome.org/viewcvs/pygobject?limit_changes=100&view=revision&revision=653
http://svn.gnome.org/viewcvs/pygobject?limit_changes=100&view=revision&revision=642
When I patched 2.12.3 against these patches, the Python object leaks go away. There may be workarounds matplotlib can make -- I'll look into that now. Hopefully the workarounds won't be too problematic -- I would hate to complicate matplotlib code for the sake of something that should resolve itself in the next Gnome release cycle...
I updated the memleak_gui.py script to display the pygobject/glib versions as well when testing the gtk backend.
Cheers,
Mike
Michael Droettboom wrote:
···
Eric Firing wrote:
Michael Droettboom wrote:
Interesting...
When you get a chance, would you mind running the attached script? This is how I was finding object leaks before. It takes a single commandline argument that is the number of iterations. Can you send me the outputs from 1 and 2 iterations? That way we should be able to see what type of object is being leaked, which is a good first step.
And here is the result of the script modified for gtk:
efiring@...340...:~/programs/py/mpl/tests$ python memleak_gui_gtk.py 1
55352 55417
*** <type 'gtk.gdk.Colormap'>
*** <class 'matplotlib.backends.backend_gtk.FileChooserDialog'>
*** <class 'gtk._gtk.WidgetFlags'>
*** <type 'gtk.gdk.Window'>
*** <class 'matplotlib.backends.backend_gtk.FigureCanvasGTK'>
*** <class 'matplotlib.backends.backend_gtk.NavigationToolbar2GTK'>
*** <type 'gtk.gdk.Pixmap'>
*** <type 'gtk.Tooltips'>
*** <type 'gtk.Label'>
*** <type 'gtk.Window'>
*** <type 'gtk.VBox'>
uncollectable list:
efiring@...340...:~/programs/py/mpl/tests$ python memleak_gui_gtk.py 2
55352 55421
*** <type 'gtk.gdk.Colormap'>
*** <class 'matplotlib.backends.backend_gtk.FileChooserDialog'>
*** <class 'gtk._gtk.WidgetFlags'>
*** <type 'gtk.gdk.Window'>
*** <class 'matplotlib.backends.backend_gtk.FileChooserDialog'>
*** <class 'gtk._gtk.WidgetFlags'>
*** <type 'gtk.gdk.Window'>
*** <class 'matplotlib.backends.backend_gtk.FigureCanvasGTK'>
*** <class 'matplotlib.backends.backend_gtk.NavigationToolbar2GTK'>
*** <type 'gtk.gdk.Pixmap'>
*** <type 'gtk.Tooltips'>
*** <type 'gtk.Label'>
*** <type 'gtk.Window'>
*** <type 'gtk.VBox'>
uncollectable list:
Eric
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options
I wouldn't spend any time on working around a leak that is already
fixed in gobject. We just need to update the FAQ so GTK users will
know that if they have a problem with leaks in GTK*, they need to
upgrade.
JDH
···
On 7/9/07, Michael Droettboom <mdroe@...31...> wrote:
...however, there are only about 40 patches between 2.12.3 and 2.13.2.
Two of them are very crucial memory leak patches:
When I patched 2.12.3 against these patches, the Python object leaks go
away. There may be workarounds matplotlib can make -- I'll look into
that now. Hopefully the workarounds won't be too problematic -- I would
hate to complicate matplotlib code for the sake of something that should
resolve itself in the next Gnome release cycle...
I am about to update the memory leak question in the FAQ, but I thought
I’d run it by the list first. I removed language that talked about
much earlier releases of mpl, and the paragraph about leaks in older
versions of Numeric and numarray. It seems like we should recommend
numpy when the user experiences problems with Numeric or numarray, but
I wanted to confirm that before adding a paragraph to that effect.
The second question (below), can be ommitted – it really belongs in a
developer’s FAQ. Any recommendations on a better place to post that
information?
matplotlib appears to be leaking memory, what should I do?
First, determine if it is a true memory leak. Python allocates memory
in pools, rather than one object at a time, which can make memory leaks
difficult to diagnose. Memory usage may appear to increase rapidly
with each iteration, but should eventually reach a steady state if no
true memory leaks exist. (For more fine-grained memory usage
reporting, you can build a custom Python with
the --without-pymalloc flag, which disables pool allocation.)
If after sufficient iterations, you still see that memory usage is
increasing, it is a likely a bonafide memory leak that should be
reported.
memleak_gui.py
(in the unit directory of the source tree), contains an example script
useful for diagnosing and reporting memory leaks. Please use
something like it when reporting leaks so we get an idea of the
magnitude of the problem (i.e. bytes per figure). Also please provide
your platform, matplotlib version, backend and as much information
about the versions of the libraries you are using: freetype, png and
zlib. It would also help if you posted code so I could look for any
obvious coding errors vis-a-vis matplotlib.
There are some known memory leaks in matplotlib-0.90.1 when used in
conjunction with the Tk, Gtk, Wx, Qt and Qt4 GUI backends. Many of
these leaks have resolutions in the current SVN version of matplotlib.
However, the following library versions are known to have leaks that
matplotlib triggers. If you have one of these versions and are
experiencing memory leaks, you should upgrade your library. This is
not an exhaustive list.
- **Wx backend:**wxPython-2.8.2 or earlier in the 2.8 series
- **Gtk backend:**pygobject-2.12.x, pygtk-2.4.0
I’d like to help diagnose a memory leak, rather than just report
it. How do you recommend I do that?
I thought you’d never ask! Python memory leaks tend to fall into one of
the following categories:
-
Uncollectable garbage: The Python garbage collector is not
infallible. It can not collect objects that the Python that have __del__
methods or weakrefs and contain cycles. If curious, you can
read about this problem in horrific
detail. You can obtain a list of all uncollectable objects as
follows: import gc
print gc.garbage
To see what cycles these objects participate in, there is a useful
function in matplotlib.cbook: from matplotlib import cbook
cbook.print_cycles(gc.garbage)
This will print out all of the reference cycles that are preventing the
uncollectable objects from being freed. The code should then be
modified to prevent these cycles, or break the cycles during
destruction.
-
Real references: Sometimes objects are legitimately being
held onto by other Python objects. When this happens, you would see the
total number of Python objects in the interpreter increase with each
iteration of your test, even when you didn’t intend any data to “stick
around”. You can print out the total number of objects in the
interpreter: import gc
print len(gc.get_objects())
By comparing the objects before and after your test, you can determine
which of them remain between iterations: original_objects = [id(x) for x in gc.get_objects()]
# ... do something that leaks objects
new_objects = [x for x in gc.get_objects() if id(x) not in original_objects]
You can then determine what is referencing those objects and causing
them to stick around:print gc.get_referents(x)
The code should be modified to prevent these unwanted references.
-
C/C++ leaks in extension objects: These is the classic
problem of objects that are “malloc’d/new’d” and never “freed/deleted”
in C or C++ parlance. There are many memory debuggers available such as
Purify or Valgrind to help find these errors. I recommend reading the documentation
about using these tools in conjunction with Python to avoid a lot
of false positives.
vis-a-vis numpy, since we are dropping support entirely for Numeric
and numarray, you can definitely recommend numoy 
The looks very good, but I would also point them to memleak_hawaii or
something to diagnose a pure image backend. Many times it is simply
the user script that is failing (they fail to properly close a figure
they have opened) so the FAQ should at least point them to an image
generation script that is written correctly. The GUI leaks are are
current problem, but they are 2nd order. We first need to make sure
the pure MPL part is not leaking (it isn't currently, but it might one
day if we screw something up in extension code) and once we've
verified that, we can isolate the GUI leaks.
Thanks,
JDH
···
On 7/9/07, Michael Droettboom <mdroe@...31...> wrote:
I am about to update the memory leak question in the FAQ, but I thought I'd
run it by the list first. I removed language that talked about much earlier
releases of mpl, and the paragraph about leaks in older versions of Numeric
and numarray. It seems like we should recommend numpy when the user
experiences problems with Numeric or numarray, but I wanted to confirm that
before adding a paragraph to that effect.