pylab.close()

I am getting a memory leak when i am using the pylab.close() function. I am
running matplotlib-0.98.3. It happens in a very simple script such as:

#!/usr/bin/python
import time
import pylab

while True:
    time.sleep(1)
    print 'calling pylab'
    pylab.box()
    pylab.close()

Every close seems to store megabytes in physical memory. Any idea why this
happens?

Cheers.

···

--
View this message in context: http://www.nabble.com/pylab.close()-tp20486589p20486589.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Can you provide more information about the platform and backend that you are using?

D2Hitman wrote:

···

I am getting a memory leak when i am using the pylab.close() function. I am
running matplotlib-0.98.3. It happens in a very simple script such as:

#!/usr/bin/python
import time
import pylab

while True:
    time.sleep(1)
    print 'calling pylab'
    pylab.box()
    pylab.close()

Every close seems to store megabytes in physical memory. Any idea why this
happens?

Cheers.

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Michael Droettboom wrote:

Can you provide more information about the platform and backend that you are using?

Mike,

I was able to reproduce this with my ubuntu 8.10, gtkagg backend. I ran the code via cut and paste with the stock python interpreter, not ipython. I did not measure the memory use carefully, but used the system monitor to observe memory for that process climbing, maybe 500k per cycle.

Our usual memleak tester shows no problem, however.

Eric

···

D2Hitman wrote:

I am getting a memory leak when i am using the pylab.close() function. I am
running matplotlib-0.98.3. It happens in a very simple script such as:

#!/usr/bin/python
import time
import pylab

while True:
    time.sleep(1)
    print 'calling pylab'
    pylab.box()
    pylab.close()

Every close seems to store megabytes in physical memory. Any idea why this
happens?

Cheers.

Ok. Thanks, I'll look into it. Just wanted to rule out that this wasn't the known Gtk memory leak with old versions of Gtk before devoting time to it.

Cheers,
Mike

Eric Firing wrote:

···

Michael Droettboom wrote:

Can you provide more information about the platform and backend that you are using?

Mike,

I was able to reproduce this with my ubuntu 8.10, gtkagg backend. I ran the code via cut and paste with the stock python interpreter, not ipython. I did not measure the memory use carefully, but used the system monitor to observe memory for that process climbing, maybe 500k per cycle.

Our usual memleak tester shows no problem, however.

Eric

D2Hitman wrote:

I am getting a memory leak when i am using the pylab.close() function. I am
running matplotlib-0.98.3. It happens in a very simple script such as:

#!/usr/bin/python
import time
import pylab

while True:
    time.sleep(1)
    print 'calling pylab'
    pylab.box()
    pylab.close()

Every close seems to store megabytes in physical memory. Any idea why this
happens?

Cheers.

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

openSUSE 11.0 (x86_64) KDE 3.5.9 "release 49.1" with GTKAgg backend

Michael Droettboom-3 wrote:

···

Can you provide more information about the platform and backend that you
are using?

D2Hitman wrote:

I am getting a memory leak when i am using the pylab.close() function. I
am
running matplotlib-0.98.3. It happens in a very simple script such as:

#!/usr/bin/python
import time
import pylab

while True:
    time.sleep(1)
    print 'calling pylab'
    pylab.box()
    pylab.close()

Every close seems to store megabytes in physical memory. Any idea why
this
happens?

Cheers.

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the
world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
View this message in context: http://www.nabble.com/pylab.close()-tp20486589p20505291.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

I'm not at the bottom of this yet, but thought I'd share my progress so far -->

It is leaking actual Python references (meaning len(gc.get_objects()) is increasing). So it's not a malloc/free pair.

Seems to be Gtk-specific. (Both GtkAgg and Gtk). Other backends are unaffected (Qt4 has some sort of leak of a much smaller magnitude).

The proper way to destroy pygtk objects appears to be a little bit of a black art. What's there now was pretty much arrived at by experimentation and seems very brittle.

I know at one point it was working better than it is now. My next step will be to try to revert to a state where the leak wasn't so bad. Given the architecture of pygtk, I'm not sure it's possible to not leak at all when creating/destroying windows like this, but we should at least be able to reduce the leakage.

If your use case is just generating a bunch of images in batch, I'd recommend using the Agg backend, rather than Gtk. That doesn't leak for me. If you really need to be opening and closing this many windows in succession, then we still don't have a good solution.

Mike

Michael Droettboom wrote:

···

Ok. Thanks, I'll look into it. Just wanted to rule out that this wasn't the known Gtk memory leak with old versions of Gtk before devoting time to it.

Cheers,
Mike

Eric Firing wrote:
  

Michael Droettboom wrote:
    

Can you provide more information about the platform and backend that you are using?
      

Mike,

I was able to reproduce this with my ubuntu 8.10, gtkagg backend. I ran the code via cut and paste with the stock python interpreter, not ipython. I did not measure the memory use carefully, but used the system monitor to observe memory for that process climbing, maybe 500k per cycle.

Our usual memleak tester shows no problem, however.

Eric

D2Hitman wrote:
      

I am getting a memory leak when i am using the pylab.close() function. I am
running matplotlib-0.98.3. It happens in a very simple script such as:

#!/usr/bin/python
import time
import pylab

while True:
    time.sleep(1)
    print 'calling pylab'
    pylab.box()
    pylab.close()

Every close seems to store megabytes in physical memory. Any idea why this
happens?

Cheers.

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Mike,

Yes i'm using a more complicated program to batch produce images. Through
more investigation i think it's not just the close function, any repeat call
of pylab seems to have similar problem. Thanks for looking into it, i will
try the Agg backend and see if that works out.

Cheers,
Jon.

Michael Droettboom-3 wrote:

···

I'm not at the bottom of this yet, but thought I'd share my progress so
far -->

It is leaking actual Python references (meaning len(gc.get_objects()) is
increasing). So it's not a malloc/free pair.

Seems to be Gtk-specific. (Both GtkAgg and Gtk). Other backends are
unaffected (Qt4 has some sort of leak of a much smaller magnitude).

The proper way to destroy pygtk objects appears to be a little bit of a
black art. What's there now was pretty much arrived at by
experimentation and seems very brittle.

I know at one point it was working better than it is now. My next step
will be to try to revert to a state where the leak wasn't so bad. Given
the architecture of pygtk, I'm not sure it's possible to not leak at all
when creating/destroying windows like this, but we should at least be
able to reduce the leakage.

If your use case is just generating a bunch of images in batch, I'd
recommend using the Agg backend, rather than Gtk. That doesn't leak for
me. If you really need to be opening and closing this many windows in
succession, then we still don't have a good solution.

Mike

Michael Droettboom wrote:

Ok. Thanks, I'll look into it. Just wanted to rule out that this
wasn't the known Gtk memory leak with old versions of Gtk before
devoting time to it.

Cheers,
Mike

Eric Firing wrote:
  

Michael Droettboom wrote:
    

Can you provide more information about the platform and backend that
you are using?
      

Mike,

I was able to reproduce this with my ubuntu 8.10, gtkagg backend. I
ran the code via cut and paste with the stock python interpreter, not
ipython. I did not measure the memory use carefully, but used the
system monitor to observe memory for that process climbing, maybe 500k
per cycle.

Our usual memleak tester shows no problem, however.

Eric

D2Hitman wrote:
      

I am getting a memory leak when i am using the pylab.close()
function. I am
running matplotlib-0.98.3. It happens in a very simple script such as:

#!/usr/bin/python
import time
import pylab

while True:
    time.sleep(1)
    print 'calling pylab'
    pylab.box()
    pylab.close()

Every close seems to store megabytes in physical memory. Any idea
why this
happens?

Cheers.

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the
world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
View this message in context: http://www.nabble.com/pylab.close()-tp20486589p20515461.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

I found the last leak exercised by your example. The window was keeping around a reference to the file selector dialog, creating a cyclical reference. Cyclical references containing Gtk objects can not be cleaned up by the Python garbage selector. By creating the file selector on-the-fly, it no longer leaks. I've attached a pretty picture for those who like pretty pictures... :wink:

My recommendation still stands to use the Agg backend for long-running offline batch processes, just for having fewer moving parts if nothing else.

Cheers,
Mike

D2Hitman wrote:

gtk_memleak1108.png

···

Mike,

Yes i'm using a more complicated program to batch produce images. Through
more investigation i think it's not just the close function, any repeat call
of pylab seems to have similar problem. Thanks for looking into it, i will
try the Agg backend and see if that works out.

Cheers,
Jon.

Michael Droettboom-3 wrote:
  

I'm not at the bottom of this yet, but thought I'd share my progress so far -->

It is leaking actual Python references (meaning len(gc.get_objects()) is increasing). So it's not a malloc/free pair.

Seems to be Gtk-specific. (Both GtkAgg and Gtk). Other backends are unaffected (Qt4 has some sort of leak of a much smaller magnitude).

The proper way to destroy pygtk objects appears to be a little bit of a black art. What's there now was pretty much arrived at by experimentation and seems very brittle.

I know at one point it was working better than it is now. My next step will be to try to revert to a state where the leak wasn't so bad. Given the architecture of pygtk, I'm not sure it's possible to not leak at all when creating/destroying windows like this, but we should at least be able to reduce the leakage.

If your use case is just generating a bunch of images in batch, I'd recommend using the Agg backend, rather than Gtk. That doesn't leak for me. If you really need to be opening and closing this many windows in succession, then we still don't have a good solution.

Mike

Michael Droettboom wrote:
    

Ok. Thanks, I'll look into it. Just wanted to rule out that this wasn't the known Gtk memory leak with old versions of Gtk before devoting time to it.

Cheers,
Mike

Eric Firing wrote:
  

Michael Droettboom wrote:
    

Can you provide more information about the platform and backend that you are using?
      

Mike,

I was able to reproduce this with my ubuntu 8.10, gtkagg backend. I ran the code via cut and paste with the stock python interpreter, not ipython. I did not measure the memory use carefully, but used the system monitor to observe memory for that process climbing, maybe 500k per cycle.

Our usual memleak tester shows no problem, however.

Eric

D2Hitman wrote:
      

I am getting a memory leak when i am using the pylab.close() function. I am
running matplotlib-0.98.3. It happens in a very simple script such as:

#!/usr/bin/python
import time
import pylab

while True:
    time.sleep(1)
    print 'calling pylab'
    pylab.box()
    pylab.close()

Every close seems to store megabytes in physical memory. Any idea why this
happens?

Cheers.

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the
world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA