Memory leaks in a web application

Community matplotlib-users
1

Ryan May <rmay31@...287...> writes:

If you're using the full OO interface and creating a figure by making an
instance of Figure(), then you shouldn't need to close the figure by hand.
It should be deleted whenever you delete or replace your instance of
Figure. (If I understand correctly.)

Yes, in the OO interface there is no close() because matplotlib does not
retain any references to the figure object. Garbage collection will take
care of reclaiming memory once the user code has no more references to
the object either.

That said, there have been cases of memory leaks caused by circular
references among objects that have __del__ methods. I think all known
leaks have been fixed, but if I were deploying a long-lived application
that creates lots of figures, I would definitely want to watch its
memory usage in my exact use case.

···

On Mon, Feb 9, 2009 at 2:37 PM, A B <python6009@...287...> wrote:

--
Jouni K. Sepp�nen

2

instance of Figure(), then you shouldn't need to close the figure by hand.
It should be deleted whenever you delete or replace your instance of
Figure. (If I understand correctly.)

... Garbage collection will take
care of reclaiming memory once the user code has no more references to
the object either.

I guess that is how it should work in theory. However at least for the
version of mpl shipped with ubuntu 8.10 (Version: 0.98.3-4ubuntu1)
there is surely quite a serious and easily demonstrated memory leak
using the artist api. Even if you manually del the figure objects
which are created inside a loop a significant growth of memory can be
observed. In fact I think there may be two separate leaks.

In the attatched standalone code example after 1000 iterations
creating 1000 png graphs there is around 2GB of RAM consumed. If you
takeout the call to savefig around 600MB is consumed.

In my view the code in the loop should execute in constant space
without growth at all. Note that when the function returns the memory
seems to be deallocated so if you check the ram use during the sleep
you should see this.

Any ideas on this?

memleaktest.py (1.8 KB)

···

--
Gerry Steele
http://belfast.no-ip.info/

3

This is an instance of the OP's problem again. Your example is using the pyplot (i.e. Matlab-like) interface, in which case, you must explicitly close each figure when you're done with it, like follows:

   plt.close(fig)

"del fig" only deletes your local reference to the figure. There is still a reference being held by the figure manager in case you wanted to go back to an earlier figure (accessed by number) and edit it. This is essentially the price paid for Matlab-work-a-likeness.

I've tested your script on 0.98.3 (though on a RHEL4 system) and I don't see any memory leaks after adding the call to "plt.close(fig)".

If you instantiate the figure and canvas directly (as in the agg_oo.py example), you won't have this issue.

Mike

Gerry Steele wrote:

···

instance of Figure(), then you shouldn't need to close the figure by hand.
It should be deleted whenever you delete or replace your instance of
Figure. (If I understand correctly.)
      
... Garbage collection will take
care of reclaiming memory once the user code has no more references to
the object either.
    
I guess that is how it should work in theory. However at least for the
version of mpl shipped with ubuntu 8.10 (Version: 0.98.3-4ubuntu1)
there is surely quite a serious and easily demonstrated memory leak
using the artist api. Even if you manually del the figure objects
which are created inside a loop a significant growth of memory can be
observed. In fact I think there may be two separate leaks.

In the attatched standalone code example after 1000 iterations
creating 1000 png graphs there is around 2GB of RAM consumed. If you
takeout the call to savefig around 600MB is consumed.

In my view the code in the loop should execute in constant space
without growth at all. Note that when the function returns the memory
seems to be deallocated so if you check the ram use during the sleep
you should see this.

Any ideas on this?

  ------------------------------------------------------------------------

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
------------------------------------------------------------------------

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
  
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

4

Thanks Michael ,

I had somehow put myself under the impression i was using he OO
version of the api but it is much more clear now. Memory issues now
look better.

Thanks.

2009/2/10 Michael Droettboom <mdroe@...86...>:

···

This is an instance of the OP's problem again. Your example is using the
pyplot (i.e. Matlab-like) interface, in which case, you must explicitly
close each figure when you're done with it, like follows:

plt.close(fig)

"del fig" only deletes your local reference to the figure. There is still a
reference being held by the figure manager in case you wanted to go back to
an earlier figure (accessed by number) and edit it. This is essentially the
price paid for Matlab-work-a-likeness.

I've tested your script on 0.98.3 (though on a RHEL4 system) and I don't see
any memory leaks after adding the call to "plt.close(fig)".

If you instantiate the figure and canvas directly (as in the agg_oo.py
example), you won't have this issue.

Mike

Gerry Steele wrote:

instance of Figure(), then you shouldn't need to close the figure by
hand.
It should be deleted whenever you delete or replace your instance of
Figure. (If I understand correctly.)

... Garbage collection will take
care of reclaiming memory once the user code has no more references to
the object either.

I guess that is how it should work in theory. However at least for the
version of mpl shipped with ubuntu 8.10 (Version: 0.98.3-4ubuntu1)
there is surely quite a serious and easily demonstrated memory leak
using the artist api. Even if you manually del the figure objects
which are created inside a loop a significant growth of memory can be
observed. In fact I think there may be two separate leaks.

In the attatched standalone code example after 1000 iterations
creating 1000 png graphs there is around 2GB of RAM consumed. If you
takeout the call to savefig around 600MB is consumed.

In my view the code in the loop should execute in constant space
without growth at all. Note that when the function returns the memory
seems to be deallocated so if you check the ram use during the sleep
you should see this.

Any ideas on this?

------------------------------------------------------------------------

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code
to
build responsive, highly engaging applications that combine the power of
local
resources and data with the reach of the web. Download the Adobe AIR SDK
and
Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
------------------------------------------------------------------------

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

--
Gerry Steele
http://belfast.no-ip.info/

5

There is room for confusion. A common usage pattern, one I often use
myself, is to use pyplot for figure creation, closing and showing
only, and to use the OO API for everything else. To whit:

  import matplotlib.pyplot as plt
  fig = plt.figure()
  ax = fig.add_subplot(111)
  ax.plot([1,2,3])
  ax.set_xlabel('hi')
  plt.show()

this contrasts with the pure pyplot approach, which uses the pyplot
state machine for everything (figure/axes creation, current axes, etc)

  import matplotlib.pyplot as plt
  plt.plot([1,2,3])
  plt.xlabel('hi')
  plt.show()

and both are different from the pure OO approach in which you manually
create the figure canvas etc

  from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
  from matplotlib.figure import Figure

  fig = Figure()
  canvas = FigureCanvas(fig)
  ax = fig.add_subplot(111)
  ax.plot([1,2,3])
  ax.set_xlabel('hi')

Note that in the 3rd example, all the code is the same as the first
example, except the figure/canvas creation, which is why one might
call it "OO pyplot"

Eric has summarized some additional information about the different
usage modes here

  http://matplotlib.sourceforge.net/faq/usage_faq.html#matplotlib-pylab-and-pyplot-how-are-they-related

JDH

···

On Tue, Feb 10, 2009 at 7:55 AM, Gerry Steele <gerry.steele@...287...> wrote:

Thanks Michael ,

I had somehow put myself under the impression i was using he OO
version of the api but it is much more clear now. Memory issues now
look better.

6

Thank you. The reason I asked the question was that with the OO interface, I still observed growth in memory consumption (more than I saw after I added pyplot.close() calls). However, at some point (90M which I still think is a lot) the growth tapered off and eventually seemed to have stopped. Essentially, what I do is create a figure, then call plot() on it in a loop to generate multiple curves, then print_png() on the canvas object.

···

On Mon, Feb 9, 2009 at 9:59 PM, Jouni K. Seppänen <jks@…2487…> wrote:

Ryan May <rmay31@…287…> writes:

On Mon, Feb 9, 2009 at 2:37 PM, A B <python6009@…287…> wrote:

If you’re using the full OO interface and creating a figure by making an

instance of Figure(), then you shouldn’t need to close the figure by hand.

It should be deleted whenever you delete or replace your instance of

Figure. (If I understand correctly.)

Yes, in the OO interface there is no close() because matplotlib does not

retain any references to the figure object. Garbage collection will take

care of reclaiming memory once the user code has no more references to

the object either.

That said, there have been cases of memory leaks caused by circular

references among objects that have del methods. I think all known

leaks have been fixed, but if I were deploying a long-lived application

that creates lots of figures, I would definitely want to watch its

memory usage in my exact use case.

–

Jouni K. Seppänen

http://www.iki.fi/jks