Trouble with imshow

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB, but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result. Here's a transcript.

   Welcome to pylab, a matplotlib-based Python environment.
   For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]: <matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]: <matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]: <matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
   File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
     return self.func(*args)
   File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
     func(*args)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in idle_draw
     self.draw()
   File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
     FigureCanvasAgg.draw(self)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
     self.figure.draw(self.renderer)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
     draw(artist, renderer, *args, **kwargs)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
     func(*args)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
     draw(artist, renderer, *args, **kwargs)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
     a.draw(renderer)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
     draw(artist, renderer, *args, **kwargs)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
     im = self.make_image(renderer.get_image_magnification())
   File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
     transformed_viewLim)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
     x = self.to_rgba(self._A, self._alpha)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
     x = self.norm(x)
   File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
     result = (val-vmin) / (vmax-vmin)
   File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
     return divide(self, other)
   File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
     m |= filled(domain(da, db), True)
   File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
     return umath.absolute(a) * self.tolerance >= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

This isn’t new, but it isn’t well understood either:

http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html

Unfortunately, that conversation went off-list, and never resulted in a bug report being filed. Could you please file a bug report on this?

http://sourceforge.net/tracker/?group_id=80706

Thanks,
Ben Root

···

On Wed, Feb 2, 2011 at 5:33 PM, Robert Abiad <abiad@…3281…> wrote:

Hello All,

I’m very new to python, so bear with me.

I’d like to use python to do my image processing, but I’m running into behavior that doesn’t make

sense to me. I’m using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest

versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I’m loading in a fits file

that’s 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,

but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each

time. If I close the figure and replot it, imshow() crashes. I don’t know if I’m doing something

wrong, or if it’s a new or known bug. I tried the same thing on Linux and got the same result.

Here’s a transcript.

Welcome to pylab, a matplotlib-based Python environment.

For more information, type ‘help(pylab)’.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)

Out[5]: <matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin=‘lower’)

Out[7]: <matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin=‘lower’)

Out[9]: <matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback

Traceback (most recent call last):

File “C:\app\Python2.6\lib\lib-tk\Tkinter.py”, line 1410, in call

 return self.func(*args)

File “C:\app\Python2.6\lib\lib-tk\Tkinter.py”, line 495, in callit

 func(*args)

File “C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py”, line 263, in

idle_draw

 self.draw()

File “C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py”, line 248, in draw

 FigureCanvasAgg.draw(self)

File “C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py”, line 394, in draw

 self.figure.draw(self.renderer)

File “C:\app\Python2.6\lib\site-packages\matplotlib\artist.py”, line 55, in draw_wrapper

 draw(artist, renderer, *args, **kwargs)

File “C:\app\Python2.6\lib\site-packages\matplotlib\figure.py”, line 798, in draw

 func(*args)

File “C:\app\Python2.6\lib\site-packages\matplotlib\artist.py”, line 55, in draw_wrapper

 draw(artist, renderer, *args, **kwargs)

File “C:\app\Python2.6\lib\site-packages\matplotlib\axes.py”, line 1946, in draw

 a.draw(renderer)

File “C:\app\Python2.6\lib\site-packages\matplotlib\artist.py”, line 55, in draw_wrapper

 draw(artist, renderer, *args, **kwargs)

File “C:\app\Python2.6\lib\site-packages\matplotlib\image.py”, line 354, in draw

 im = self.make_image(renderer.get_image_magnification())

File “C:\app\Python2.6\lib\site-packages\matplotlib\image.py”, line 569, in make_image

 transformed_viewLim)

File “C:\app\Python2.6\lib\site-packages\matplotlib\image.py”, line 201, in _get_unsampled_image

 x = self.to_rgba(self._A, self._alpha)

File “C:\app\Python2.6\lib\site-packages\matplotlib\cm.py”, line 193, in to_rgba

 x = self.norm(x)

File “C:\app\Python2.6\lib\site-packages\matplotlib\colors.py”, line 820, in call

 result = (val-vmin) / (vmax-vmin)

File “C:\app\Python2.6\lib\site-packages\numpy\ma\core.py”, line 3673, in div

 return divide(self, other)

File “C:\app\Python2.6\lib\site-packages\numpy\ma\core.py”, line 1077, in call

 m |= filled(domain(da, db), True)

File “C:\app\Python2.6\lib\site-packages\numpy\ma\core.py”, line 772, in call

 return umath.absolute(a) * self.tolerance >= umath.absolute(b)

MemoryError

Thanks for any help,

-robert

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

    Welcome to pylab, a matplotlib-based Python environment.
    For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
    File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
      return self.func(*args)
    File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
      func(*args)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
      self.draw()
    File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
      FigureCanvasAgg.draw(self)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
      self.figure.draw(self.renderer)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
      draw(artist, renderer, *args, **kwargs)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
      func(*args)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
      draw(artist, renderer, *args, **kwargs)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
      a.draw(renderer)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
      draw(artist, renderer, *args, **kwargs)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
      im = self.make_image(renderer.get_image_magnification())
    File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
      transformed_viewLim)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
      x = self.to_rgba(self._A, self._alpha)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
      x = self.norm(x)
    File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
      result = (val-vmin) / (vmax-vmin)
    File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
      return divide(self, other)
    File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
      m |= filled(domain(da, db), True)
    File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
      return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

···

On 2/2/2011 3:33 PM, Robert Abiad wrote:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html> <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

You might want to try out the visvis module instead of matplotlib for
interactive viewing of large 2D images - my system is also Win64 with
4GB and visvis.imshow() handles a 4k*4k image. You'll probably also
want to disable ipython's object caching if you're doing a lot of this
interactive viewing of large images.

Gary R

···

On Thu, Feb 3, 2011 at 10:59 AM, Christoph Gohlke <cgohlke@...2143...> wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

Welcome to pylab, a matplotlib\-based Python environment\.
For more information, type &#39;help\(pylab\)&#39;\.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
return self.func(*args)
File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
func(*args)
File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
self.draw()
File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
FigureCanvasAgg.draw(self)
File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
self.figure.draw(self.renderer)
File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
func(*args)
File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
a.draw(renderer)
File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
im = self.make_image(renderer.get_image_magnification())
File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
transformed_viewLim)
File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
x = self.to_rgba(self._A, self._alpha)
File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
x = self.norm(x)
File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
result = (val-vmin) / (vmax-vmin)
File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
return divide(self, other)
File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
m |= filled(domain(da, db), True)
File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

The first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that doesn't seem like a great solution. I use other image processing software (ImageJ and IDL) and neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same. I also think it should free up memory when the image is closed, but maybe I'm not doing the right thing. Is there something else I should be doing to free up memory?

Things are even worse with plot.

I'll file a bug report as Benjamin suggests.

-robert

···

On 2/2/2011 3:59 PM, Christoph Gohlke wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

     Welcome to pylab, a matplotlib-based Python environment.
     For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
     File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
       return self.func(*args)
     File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
       func(*args)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
       self.draw()
     File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
       FigureCanvasAgg.draw(self)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
       self.figure.draw(self.renderer)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
       draw(artist, renderer, *args, **kwargs)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
       func(*args)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
       draw(artist, renderer, *args, **kwargs)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
       a.draw(renderer)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
       draw(artist, renderer, *args, **kwargs)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
       im = self.make_image(renderer.get_image_magnification())
     File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
       transformed_viewLim)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
       x = self.to_rgba(self._A, self._alpha)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
       x = self.norm(x)
     File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
       result = (val-vmin) / (vmax-vmin)
     File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
       return divide(self, other)
     File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
       m |= filled(domain(da, db), True)
     File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
       return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

Matplotlib converts all images to float64 RGBA before plotting. The memory usage increases 32 times when plotting 8 bit images. E.g. at least 512 MB contiguous free memory is required for plotting 16 MB uint8 data.

Christoph

···

On 2/2/2011 5:08 PM, Robert Abiad wrote:

On 2/2/2011 3:59 PM, Christoph Gohlke wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

      Welcome to pylab, a matplotlib-based Python environment.
      For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
      File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
        return self.func(*args)
      File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
        func(*args)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
        self.draw()
      File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
        FigureCanvasAgg.draw(self)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
        self.figure.draw(self.renderer)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
        draw(artist, renderer, *args, **kwargs)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
        func(*args)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
        draw(artist, renderer, *args, **kwargs)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
        a.draw(renderer)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
        draw(artist, renderer, *args, **kwargs)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
        im = self.make_image(renderer.get_image_magnification())
      File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
        transformed_viewLim)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
        x = self.to_rgba(self._A, self._alpha)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
        x = self.norm(x)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
        result = (val-vmin) / (vmax-vmin)
      File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
        return divide(self, other)
      File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
        m |= filled(domain(da, db), True)
      File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
        return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

The first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that
doesn't seem like a great solution. I use other image processing software (ImageJ and IDL) and
neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same
display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same.
   I also think it should free up memory when the image is closed, but maybe I'm not doing the right
thing. Is there something else I should be doing to free up memory?

Things are even worse with plot.

I'll file a bug report as Benjamin suggests.

-robert

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

      Welcome to pylab, a matplotlib-based Python environment.
      For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
      File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
        return self.func(*args)
      File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
        func(*args)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
        self.draw()
      File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
        FigureCanvasAgg.draw(self)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
        self.figure.draw(self.renderer)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
        draw(artist, renderer, *args, **kwargs)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
        func(*args)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
        draw(artist, renderer, *args, **kwargs)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
        a.draw(renderer)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
        draw(artist, renderer, *args, **kwargs)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
        im = self.make_image(renderer.get_image_magnification())
      File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
        transformed_viewLim)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
        x = self.to_rgba(self._A, self._alpha)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
        x = self.norm(x)
      File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
        result = (val-vmin) / (vmax-vmin)
      File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
        return divide(self, other)
      File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
        m |= filled(domain(da, db), True)
      File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
        return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

The first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that
doesn't seem like a great solution. I use other image processing software (ImageJ and IDL) and
neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same
display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same.
   I also think it should free up memory when the image is closed, but maybe I'm not doing the right
thing. Is there something else I should be doing to free up memory?

Things are even worse with plot.

I'll file a bug report as Benjamin suggests.

Please file it in the "enhancement" category, not as a "bug". The difficulty that mpl has with images is the result of a design decision long ago; as Christoph notes, mpl works primarily with float64 rgba, which is horribly memory inefficient, but is very general and works fine so long as the image is not too large. uint8 is already used in some circumstances. I don't know how hard it would be to simply switch to uint8 for images, or to make that an option, perhaps for all color specification. It may involve getting into arcana of the extension code and the agg library, which is a daunting prospect for most of us, certainly for me. I agree entirely that a major reduction in memory usage would be good; patches, or a mpl branch to achieve that, are welcome.

Eric

···

On 02/02/2011 03:08 PM, Robert Abiad wrote:

On 2/2/2011 3:59 PM, Christoph Gohlke wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

-robert

------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

How about special-casing uint8 arrays, assuming that these don't need to be normalized? This gives the user the option to bypass matplotlib's internal conversions and saves around 450 MB for plotting one 4096x4096 uint8 image. A preliminary, mostly untested, patch is attached. Similar changes could be done for other signed and unsigned integer data.

Christoph

image.diff (1.29 KB)

···

On 2/2/2011 6:06 PM, Eric Firing wrote:

On 02/02/2011 03:08 PM, Robert Abiad wrote:

On 2/2/2011 3:59 PM, Christoph Gohlke wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

       Welcome to pylab, a matplotlib-based Python environment.
       For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
       File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
         return self.func(*args)
       File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
         func(*args)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
         self.draw()
       File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
         FigureCanvasAgg.draw(self)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
         self.figure.draw(self.renderer)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
         draw(artist, renderer, *args, **kwargs)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
         func(*args)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
         draw(artist, renderer, *args, **kwargs)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
         a.draw(renderer)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
         draw(artist, renderer, *args, **kwargs)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
         im = self.make_image(renderer.get_image_magnification())
       File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
         transformed_viewLim)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
         x = self.to_rgba(self._A, self._alpha)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
         x = self.norm(x)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
         result = (val-vmin) / (vmax-vmin)
       File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
         return divide(self, other)
       File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
         m |= filled(domain(da, db), True)
       File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
         return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

The first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that
doesn't seem like a great solution. I use other image processing software (ImageJ and IDL) and
neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same
display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same.
    I also think it should free up memory when the image is closed, but maybe I'm not doing the right
thing. Is there something else I should be doing to free up memory?

Things are even worse with plot.

I'll file a bug report as Benjamin suggests.

Please file it in the "enhancement" category, not as a "bug". The
difficulty that mpl has with images is the result of a design decision
long ago; as Christoph notes, mpl works primarily with float64 rgba,
which is horribly memory inefficient, but is very general and works fine
so long as the image is not too large. uint8 is already used in some
circumstances. I don't know how hard it would be to simply switch to
uint8 for images, or to make that an option, perhaps for all color
specification. It may involve getting into arcana of the extension code
and the agg library, which is a daunting prospect for most of us,
certainly for me. I agree entirely that a major reduction in memory
usage would be good; patches, or a mpl branch to achieve that, are welcome.

Eric

As a possible stop-gap solution, maybe we could consider the compressed arrays (carry)?

I am not a fan of doing special cases, though, especially at the very base/core portions of the library.

Ben Root

···

On Wed, Feb 2, 2011 at 8:06 PM, Eric Firing <efiring@…202…> wrote:

On 02/02/2011 03:08 PM, Robert Abiad wrote:

On 2/2/2011 3:59 PM, Christoph Gohlke wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

Hello All,

I’m very new to python, so bear with me.

I’d like to use python to do my image processing, but I’m running into behavior that doesn’t make

sense to me. I’m using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest

versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I’m loading in a fits file

that’s 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,

but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each

time. If I close the figure and replot it, imshow() crashes. I don’t know if I’m doing something

wrong, or if it’s a new or known bug. I tried the same thing on Linux and got the same result.

Here’s a transcript.

  Welcome to pylab, a matplotlib-based Python environment.
  For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)

Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin=‘lower’)

Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin=‘lower’)

Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback

Traceback (most recent call last):

  File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
    return self.func(*args)
  File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
    func(*args)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in

idle_draw

    self.draw()
  File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
    FigureCanvasAgg.draw(self)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
    self.figure.draw(self.renderer)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
    func(*args)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
    a.draw(renderer)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
    im = self.make_image(renderer.get_image_magnification())
  File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
    transformed_viewLim)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
    x = self.to_rgba(self._A, self._alpha)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
    x = self.norm(x)
  File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
    result = (val-vmin) / (vmax-vmin)
  File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
    return divide(self, other)
  File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
    m |= filled(domain(da, db), True)
  File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
    return umath.absolute(a) * self.tolerance>= umath.absolute(b)

MemoryError

Thanks for any help,

-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

The first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that

doesn’t seem like a great solution. I use other image processing software (ImageJ and IDL) and

neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same

display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same.

I also think it should free up memory when the image is closed, but maybe I’m not doing the right

thing. Is there something else I should be doing to free up memory?

Things are even worse with plot.

I’ll file a bug report as Benjamin suggests.

Please file it in the “enhancement” category, not as a “bug”. The

difficulty that mpl has with images is the result of a design decision

long ago; as Christoph notes, mpl works primarily with float64 rgba,

which is horribly memory inefficient, but is very general and works fine

so long as the image is not too large. uint8 is already used in some

circumstances. I don’t know how hard it would be to simply switch to

uint8 for images, or to make that an option, perhaps for all color

specification. It may involve getting into arcana of the extension code

and the agg library, which is a daunting prospect for most of us,

certainly for me. I agree entirely that a major reduction in memory

usage would be good; patches, or a mpl branch to achieve that, are welcome.

Eric

Forget about the patch; it is too crude and breaks things. My point was that it should be possible to have huge memory savings without touching agg internals. I'll take another look.

Christoph

···

On 2/2/2011 8:22 PM, Benjamin Root wrote:

On Wed, Feb 2, 2011 at 8:06 PM, Eric Firing <efiring@…202… > <mailto:efiring@…202…>> wrote:

    On 02/02/2011 03:08 PM, Robert Abiad wrote:
    > On 2/2/2011 3:59 PM, Christoph Gohlke wrote:
    > > On 2/2/2011 3:33 PM, Robert Abiad wrote:
    > >> Hello All,
    > >>
    > >> I'm very new to python, so bear with me.
    > >>
    > >> I'd like to use python to do my image processing, but I'm
    running into behavior that doesn't make
    > >> sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of
    memory, python 2.6.6, and the newest
    > >> versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1),
    scipy. I'm loading in a fits file
    > >> that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I
    can see memory usage go up by 50MB,
    > >> but when I try displaying the image using imshow(), my memory
    usage goes up by around 500MB, each
    > >> time. If I close the figure and replot it, imshow() crashes. I
    don't know if I'm doing something
    > >> wrong, or if it's a new or known bug. I tried the same thing on
    Linux and got the same result.
    > >> Here's a transcript.
    > >>
    > >> Welcome to pylab, a matplotlib-based Python environment.
    > >> For more information, type 'help(pylab)'.
    > >>
    > >> In [1]: import pyfits
    > >>
    > >> In [2]: from Tkinter import *
    > >>
    > >> In [3]: import tkFileDialog
    > >>
    > >> In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())
    > >>
    > >> In [5]: imshow(image)
    > >> Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>
    > >>
    > >> In [6]: close()
    > >>
    > >> In [7]: imshow(image,origin='lower')
    > >> Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>
    > >>
    > >> In [8]: close()
    > >>
    > >> In [9]: imshow(image[100:3600,100:3600],origin='lower')
    > >> Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>
    > >>
    > >> In [10]: Exception in Tkinter callback
    > >> Traceback (most recent call last):
    > >> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410,
    in __call__
    > >> return self.func(*args)
    > >> File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495,
    in callit
    > >> func(*args)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py",
    line 263, in
    > >> idle_draw
    > >> self.draw()
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py",
    line 248, in draw
    > >> FigureCanvasAgg.draw(self)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py",
    line 394, in draw
    > >> self.figure.draw(self.renderer)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55,
    in draw_wrapper
    > >> draw(artist, renderer, *args, **kwargs)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798,
    in draw
    > >> func(*args)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55,
    in draw_wrapper
    > >> draw(artist, renderer, *args, **kwargs)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946,
    in draw
    > >> a.draw(renderer)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55,
    in draw_wrapper
    > >> draw(artist, renderer, *args, **kwargs)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354,
    in draw
    > >> im = self.make_image(renderer.get_image_magnification())
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569,
    in make_image
    > >> transformed_viewLim)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201,
    in _get_unsampled_image
    > >> x = self.to_rgba(self._A, self._alpha)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in
    to_rgba
    > >> x = self.norm(x)
    > >> File
    "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820,
    in __call__
    > >> result = (val-vmin) / (vmax-vmin)
    > >> File
    "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in
    __div__
    > >> return divide(self, other)
    > >> File
    "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in
    __call__
    > >> m |= filled(domain(da, db), True)
    > >> File
    "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in
    __call__
    > >> return umath.absolute(a) * self.tolerance>=
    umath.absolute(b)
    > >> MemoryError
    > >>
    > >> Thanks for any help,
    > >> -robert
    > >>
    > >
    > > These are previous discussions on the issue:
    > >
    <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
    > >
    <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
    > >
    <http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>
    > >
    > > Christoph
    > >
    >
    > The first 2 discussions lead to suggestions of more memory on a
    64-bit installation, but that
    > doesn't seem like a great solution. I use other image processing
    software (ImageJ and IDL) and
    > neither has any trouble with my images. As I mentioned ImageJ
    uses 1/10th the memory for the same
    > display, or about 1 byte of memory for 1 byte of image. I think
    matplotlib should aim for the same.
    > I also think it should free up memory when the image is closed,
    but maybe I'm not doing the right
    > thing. Is there something else I should be doing to free up memory?
    >
    > Things are even worse with plot.
    >
    > I'll file a bug report as Benjamin suggests.

    Please file it in the "enhancement" category, not as a "bug". The
    difficulty that mpl has with images is the result of a design decision
    long ago; as Christoph notes, mpl works primarily with float64 rgba,
    which is horribly memory inefficient, but is very general and works fine
    so long as the image is not too large. uint8 is already used in some
    circumstances. I don't know how hard it would be to simply switch to
    uint8 for images, or to make that an option, perhaps for all color
    specification. It may involve getting into arcana of the extension code
    and the agg library, which is a daunting prospect for most of us,
    certainly for me. I agree entirely that a major reduction in memory
    usage would be good; patches, or a mpl branch to achieve that, are
    welcome.

    Eric

As a possible stop-gap solution, maybe we could consider the compressed
arrays (carry)?

I am not a fan of doing special cases, though, especially at the very
base/core portions of the library.

Ben Root

I'll put it in as an enhancement, but I'm still unsure if there is a bug in there as well. Is there something I should be doing to clear memory after the first figure is closed other than close()? I don't understand why memory usage grows each time I replot, but I'm pretty sure it isn't desireable behavior. As I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

As for how to implement a fix for memory usage, I'll let you folks figure that out. But it seems that if I want a grayscale image, I could reduce memory usage by 4 if matplotlib could turn rgba into intensity.

-robert

···

On 2/2/2011 6:06 PM, Eric Firing wrote:

On 02/02/2011 03:08 PM, Robert Abiad wrote:

On 2/2/2011 3:59 PM, Christoph Gohlke wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

       Welcome to pylab, a matplotlib-based Python environment.
       For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
       File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
         return self.func(*args)
       File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
         func(*args)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
         self.draw()
       File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
         FigureCanvasAgg.draw(self)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
         self.figure.draw(self.renderer)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
         draw(artist, renderer, *args, **kwargs)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
         func(*args)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
         draw(artist, renderer, *args, **kwargs)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
         a.draw(renderer)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
         draw(artist, renderer, *args, **kwargs)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
         im = self.make_image(renderer.get_image_magnification())
       File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
         transformed_viewLim)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
         x = self.to_rgba(self._A, self._alpha)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
         x = self.norm(x)
       File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
         result = (val-vmin) / (vmax-vmin)
       File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
         return divide(self, other)
       File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
         m |= filled(domain(da, db), True)
       File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
         return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

The first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that
doesn't seem like a great solution. I use other image processing software (ImageJ and IDL) and
neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same
display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same.
    I also think it should free up memory when the image is closed, but maybe I'm not doing the right
thing. Is there something else I should be doing to free up memory?

Things are even worse with plot.

I'll file a bug report as Benjamin suggests.

Please file it in the "enhancement" category, not as a "bug". The
difficulty that mpl has with images is the result of a design decision
long ago; as Christoph notes, mpl works primarily with float64 rgba,
which is horribly memory inefficient, but is very general and works fine
so long as the image is not too large. uint8 is already used in some
circumstances. I don't know how hard it would be to simply switch to
uint8 for images, or to make that an option, perhaps for all color
specification. It may involve getting into arcana of the extension code
and the agg library, which is a daunting prospect for most of us,
certainly for me. I agree entirely that a major reduction in memory
usage would be good; patches, or a mpl branch to achieve that, are welcome.

Eric

Christoph, if you're looking at special casing uint8's, you might want
to keep in mind that uint16 greyscale images are also quite common as
camera outputs in experimental setups. I think that the solution to
this should ideally minimise memory usage for any greyscale image, be
it uint8, uint16, float32 or float64. i.e. avoiding conversion to RGBA
for any single-plane 2D array type would be best IMHO,

Gary R.

···

On Thu, Feb 3, 2011 at 5:38 PM, Robert Abiad <abiad@...3281...> wrote:

On 2/2/2011 6:06 PM, Eric Firing wrote:

On 02/02/2011 03:08 PM, Robert Abiad wrote:

On 2/2/2011 3:59 PM, Christoph Gohlke wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

   Welcome to pylab, a matplotlib\-based Python environment\.
   For more information, type &#39;help\(pylab\)&#39;\.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
return self.func(*args)
File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
func(*args)
File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
self.draw()
File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
FigureCanvasAgg.draw(self)
File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
self.figure.draw(self.renderer)
File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
func(*args)
File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
a.draw(renderer)
File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
im = self.make_image(renderer.get_image_magnification())
File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
transformed_viewLim)
File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
x = self.to_rgba(self._A, self._alpha)
File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
x = self.norm(x)
File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
result = (val-vmin) / (vmax-vmin)
File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
return divide(self, other)
File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
m |= filled(domain(da, db), True)
File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

The first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that
doesn't seem like a great solution. I use other image processing software (ImageJ and IDL) and
neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same
display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same.
I also think it should free up memory when the image is closed, but maybe I'm not doing the right
thing. Is there something else I should be doing to free up memory?

Things are even worse with plot.

I'll file a bug report as Benjamin suggests.

Please file it in the "enhancement" category, not as a "bug". The
difficulty that mpl has with images is the result of a design decision
long ago; as Christoph notes, mpl works primarily with float64 rgba,
which is horribly memory inefficient, but is very general and works fine
so long as the image is not too large. uint8 is already used in some
circumstances. I don't know how hard it would be to simply switch to
uint8 for images, or to make that an option, perhaps for all color
specification. It may involve getting into arcana of the extension code
and the agg library, which is a daunting prospect for most of us,
certainly for me. I agree entirely that a major reduction in memory
usage would be good; patches, or a mpl branch to achieve that, are welcome.

Eric

I'll put it in as an enhancement, but I'm still unsure if there is a bug in
there as well. Is there something I should be doing to clear memory after the
first figure is closed other than close()? I don't understand why memory usage
grows each time I replot, but I'm pretty sure it isn't desireable behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

As for how to implement a fix for memory usage, I'll let you folks figure that
out. But it seems that if I want a grayscale image, I could reduce memory usage
by 4 if matplotlib could turn rgba into intensity.

-robert

------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

[...]

I'll put it in as an enhancement, but I'm still unsure if there is a bug in
there as well. Is there something I should be doing to clear memory after the
first figure is closed other than close()? I don't understand why memory usage
grows each time I replot, but I'm pretty sure it isn't desireable behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

I'm not quite sure, but I don't think there is a specifically matplotlib memory leak bug at work here. Are you using ipython, and if so, have you turned off the caching? In its default mode, ipython keeps lots of references, thereby keeping memory in use. Also, memory management and reporting can be a bit tricky and misleading.

Nevertheless, the attached script may be illustrating the problem. Try running it from the command line as-is (maybe shorten the loop--it doesn't take 100 iterations to show the pattern) and then commenting out the line as indicated in the comment. It seems that if anything is done that adds ever so slightly to memory use while the figure is displayed, then when the figure is closed, its memory is not reused. I'm puzzled.

Eric

images.py (374 Bytes)

···

On 02/02/2011 08:38 PM, Robert Abiad wrote:

As for how to implement a fix for memory usage, I'll let you folks figure that
out. But it seems that if I want a grayscale image, I could reduce memory usage
by 4 if matplotlib could turn rgba into intensity.

-robert

As a workaround you could do your own normalization and color mapping and pass a uint8 RGB image to imshow. That avoids matplotlib's norm function. The following example saves almost 500 MB for plotting a 16 MB uint8 greyscale image, as compared to passing img directly to imshow:

import numpy
from matplotlib import pyplot

img = numpy.random.randint(0, 255, (4096, 4096)).astype('uint8')
lut = pyplot.cm.gray(numpy.arange(255), bytes=True)
rgb = lut.take(img, axis=0)
del img
pyplot.imshow(rgb)
pyplot.show()

Christoph

···

On 2/2/2011 11:00 PM, gary ruben wrote:

Christoph, if you're looking at special casing uint8's, you might want
to keep in mind that uint16 greyscale images are also quite common as
camera outputs in experimental setups. I think that the solution to
this should ideally minimise memory usage for any greyscale image, be
it uint8, uint16, float32 or float64. i.e. avoiding conversion to RGBA
for any single-plane 2D array type would be best IMHO,

Gary R.

On Thu, Feb 3, 2011 at 5:38 PM, Robert Abiad<abiad@...3281...> wrote:

On 2/2/2011 6:06 PM, Eric Firing wrote:

On 02/02/2011 03:08 PM, Robert Abiad wrote:

On 2/2/2011 3:59 PM, Christoph Gohlke wrote:

On 2/2/2011 3:33 PM, Robert Abiad wrote:

Hello All,

I'm very new to python, so bear with me.

I'd like to use python to do my image processing, but I'm running into behavior that doesn't make
sense to me. I'm using Windows 7 Pro (64-bit) with 4 gigs of memory, python 2.6.6, and the newest
versions of ipython, pyfits, matplotlib (1.0.1), numpy (1.5.1), scipy. I'm loading in a fits file
that's 26 MB (~16 Mpixels). When I load my image in ImageJ, I can see memory usage go up by 50MB,
but when I try displaying the image using imshow(), my memory usage goes up by around 500MB, each
time. If I close the figure and replot it, imshow() crashes. I don't know if I'm doing something
wrong, or if it's a new or known bug. I tried the same thing on Linux and got the same result.
Here's a transcript.

        Welcome to pylab, a matplotlib-based Python environment.
        For more information, type 'help(pylab)'.

In [1]: import pyfits

In [2]: from Tkinter import *

In [3]: import tkFileDialog

In [4]: image=pyfits.getdata(tkFileDialog.askopenfilename())

In [5]: imshow(image)
Out[5]:<matplotlib.image.AxesImage object at 0x03BCA170>

In [6]: close()

In [7]: imshow(image,origin='lower')
Out[7]:<matplotlib.image.AxesImage object at 0x0440E170>

In [8]: close()

In [9]: imshow(image[100:3600,100:3600],origin='lower')
Out[9]:<matplotlib.image.AxesImage object at 0x045D9FB0>

In [10]: Exception in Tkinter callback
Traceback (most recent call last):
        File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 1410, in __call__
          return self.func(*args)
        File "C:\app\Python2.6\lib\lib-tk\Tkinter.py", line 495, in callit
          func(*args)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 263, in
idle_draw
          self.draw()
        File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_tkagg.py", line 248, in draw
          FigureCanvasAgg.draw(self)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\backends\backend_agg.py", line 394, in draw
          self.figure.draw(self.renderer)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
          draw(artist, renderer, *args, **kwargs)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\figure.py", line 798, in draw
          func(*args)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
          draw(artist, renderer, *args, **kwargs)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\axes.py", line 1946, in draw
          a.draw(renderer)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
          draw(artist, renderer, *args, **kwargs)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 354, in draw
          im = self.make_image(renderer.get_image_magnification())
        File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 569, in make_image
          transformed_viewLim)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\image.py", line 201, in _get_unsampled_image
          x = self.to_rgba(self._A, self._alpha)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\cm.py", line 193, in to_rgba
          x = self.norm(x)
        File "C:\app\Python2.6\lib\site-packages\matplotlib\colors.py", line 820, in __call__
          result = (val-vmin) / (vmax-vmin)
        File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 3673, in __div__
          return divide(self, other)
        File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 1077, in __call__
          m |= filled(domain(da, db), True)
        File "C:\app\Python2.6\lib\site-packages\numpy\ma\core.py", line 772, in __call__
          return umath.absolute(a) * self.tolerance>= umath.absolute(b)
MemoryError

Thanks for any help,
-robert

These are previous discussions on the issue:

<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg14727.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19815.html>
<http://www.mail-archive.com/matplotlib-users@lists.sourceforge.net/msg19614.html>

Christoph

The first 2 discussions lead to suggestions of more memory on a 64-bit installation, but that
doesn't seem like a great solution. I use other image processing software (ImageJ and IDL) and
neither has any trouble with my images. As I mentioned ImageJ uses 1/10th the memory for the same
display, or about 1 byte of memory for 1 byte of image. I think matplotlib should aim for the same.
     I also think it should free up memory when the image is closed, but maybe I'm not doing the right
thing. Is there something else I should be doing to free up memory?

Things are even worse with plot.

I'll file a bug report as Benjamin suggests.

Please file it in the "enhancement" category, not as a "bug". The
difficulty that mpl has with images is the result of a design decision
long ago; as Christoph notes, mpl works primarily with float64 rgba,
which is horribly memory inefficient, but is very general and works fine
so long as the image is not too large. uint8 is already used in some
circumstances. I don't know how hard it would be to simply switch to
uint8 for images, or to make that an option, perhaps for all color
specification. It may involve getting into arcana of the extension code
and the agg library, which is a daunting prospect for most of us,
certainly for me. I agree entirely that a major reduction in memory
usage would be good; patches, or a mpl branch to achieve that, are welcome.

Eric

I'll put it in as an enhancement, but I'm still unsure if there is a bug in
there as well. Is there something I should be doing to clear memory after the
first figure is closed other than close()? I don't understand why memory usage
grows each time I replot, but I'm pretty sure it isn't desireable behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

As for how to implement a fix for memory usage, I'll let you folks figure that
out. But it seems that if I want a grayscale image, I could reduce memory usage
by 4 if matplotlib could turn rgba into intensity.

-robert

------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

[...]

I'll put it in as an enhancement, but I'm still unsure if there is a
bug in
there as well. Is there something I should be doing to clear memory
after the
first figure is closed other than close()? I don't understand why
memory usage
grows each time I replot, but I'm pretty sure it isn't desireable
behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

I'm not quite sure, but I don't think there is a specifically matplotlib
memory leak bug at work here. Are you using ipython, and if so, have you
turned off the caching? In its default mode, ipython keeps lots of
references, thereby keeping memory in use. Also, memory management and
reporting can be a bit tricky and misleading.

Nevertheless, the attached script may be illustrating the problem. Try
running it from the command line as-is (maybe shorten the loop--it
doesn't take 100 iterations to show the pattern) and then commenting out
the line as indicated in the comment. It seems that if anything is done
that adds ever so slightly to memory use while the figure is displayed,
then when the figure is closed, its memory is not reused. I'm puzzled.

I wasn't thinking straight--there is no mystery and no memory leak. Ignore my example script referred to above. It was saving rows of the z array, not single elements as I had intended, so of course memory use was growing substantially.

Eric

···

On 02/02/2011 10:17 PM, Eric Firing wrote:

On 02/02/2011 08:38 PM, Robert Abiad wrote:

Eric

As for how to implement a fix for memory usage, I'll let you folks
figure that
out. But it seems that if I want a grayscale image, I could reduce
memory usage
by 4 if matplotlib could turn rgba into intensity.

-robert

------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

You may not see a memory leak, but I still can't get my memory back without killing python. I turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from the OS that the memory does not come back after close() and that it grows after the second imshow().

Any other ideas? Looks like a bug to me otherwise.

-robert

···

On 2/3/2011 10:06 AM, Eric Firing wrote:

On 02/02/2011 10:17 PM, Eric Firing wrote:

On 02/02/2011 08:38 PM, Robert Abiad wrote:

[...]

I'll put it in as an enhancement, but I'm still unsure if there is a
bug in
there as well. Is there something I should be doing to clear memory
after the
first figure is closed other than close()? I don't understand why
memory usage
grows each time I replot, but I'm pretty sure it isn't desireable
behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

I'm not quite sure, but I don't think there is a specifically matplotlib
memory leak bug at work here. Are you using ipython, and if so, have you
turned off the caching? In its default mode, ipython keeps lots of
references, thereby keeping memory in use. Also, memory management and
reporting can be a bit tricky and misleading.

Nevertheless, the attached script may be illustrating the problem. Try
running it from the command line as-is (maybe shorten the loop--it
doesn't take 100 iterations to show the pattern) and then commenting out
the line as indicated in the comment. It seems that if anything is done
that adds ever so slightly to memory use while the figure is displayed,
then when the figure is closed, its memory is not reused. I'm puzzled.

I wasn't thinking straight--there is no mystery and no memory leak.
Ignore my example script referred to above. It was saving rows of the z
array, not single elements as I had intended, so of course memory use
was growing substantially.

Eric

Except that I tried the same things and did not get quite the same result. Let's track this down. Please try the attached script, and see if the memory usage grows substantially, or just oscillates a bit.

Eric

image_mem.py (342 Bytes)

···

On 02/03/2011 11:30 AM, Robert Abiad wrote:

On 2/3/2011 10:06 AM, Eric Firing wrote:

On 02/02/2011 10:17 PM, Eric Firing wrote:

On 02/02/2011 08:38 PM, Robert Abiad wrote:

[...]

I'll put it in as an enhancement, but I'm still unsure if there is a
bug in
there as well. Is there something I should be doing to clear memory
after the
first figure is closed other than close()? I don't understand why
memory usage
grows each time I replot, but I'm pretty sure it isn't desireable
behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

I'm not quite sure, but I don't think there is a specifically matplotlib
memory leak bug at work here. Are you using ipython, and if so, have you
turned off the caching? In its default mode, ipython keeps lots of
references, thereby keeping memory in use. Also, memory management and
reporting can be a bit tricky and misleading.

Nevertheless, the attached script may be illustrating the problem. Try
running it from the command line as-is (maybe shorten the loop--it
doesn't take 100 iterations to show the pattern) and then commenting out
the line as indicated in the comment. It seems that if anything is done
that adds ever so slightly to memory use while the figure is displayed,
then when the figure is closed, its memory is not reused. I'm puzzled.

I wasn't thinking straight--there is no mystery and no memory leak.
Ignore my example script referred to above. It was saving rows of the z
array, not single elements as I had intended, so of course memory use
was growing substantially.

Eric

You may not see a memory leak, but I still can't get my memory back without killing python. I
turned off the ipython caching and even ran without iPython on both Windows and Ubuntu, but when I
use imshow(), followed by close('all') and another imshow(), I run out of memory. I can see from
the OS that the memory does not come back after close() and that it grows after the second imshow().

Any other ideas? Looks like a bug to me otherwise.

One thing I noticed is that if I add a "def __del__(self): print 'del'" to image._AxesImageBase, it never gets called. _AxesImageBase keeps float64 and uint8 rgba images in a cache, which is never freed.

Christoph

···

On 2/3/2011 2:15 PM, Eric Firing wrote:

On 02/03/2011 11:30 AM, Robert Abiad wrote:

On 2/3/2011 10:06 AM, Eric Firing wrote:

On 02/02/2011 10:17 PM, Eric Firing wrote:

On 02/02/2011 08:38 PM, Robert Abiad wrote:

[...]

I'll put it in as an enhancement, but I'm still unsure if there is a
bug in
there as well. Is there something I should be doing to clear memory
after the
first figure is closed other than close()? I don't understand why
memory usage
grows each time I replot, but I'm pretty sure it isn't desireable
behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

I'm not quite sure, but I don't think there is a specifically
matplotlib
memory leak bug at work here. Are you using ipython, and if so, have
you
turned off the caching? In its default mode, ipython keeps lots of
references, thereby keeping memory in use. Also, memory management and
reporting can be a bit tricky and misleading.

Nevertheless, the attached script may be illustrating the problem. Try
running it from the command line as-is (maybe shorten the loop--it
doesn't take 100 iterations to show the pattern) and then commenting
out
the line as indicated in the comment. It seems that if anything is done
that adds ever so slightly to memory use while the figure is displayed,
then when the figure is closed, its memory is not reused. I'm puzzled.

I wasn't thinking straight--there is no mystery and no memory leak.
Ignore my example script referred to above. It was saving rows of the z
array, not single elements as I had intended, so of course memory use
was growing substantially.

Eric

You may not see a memory leak, but I still can't get my memory back
without killing python. I
turned off the ipython caching and even ran without iPython on both
Windows and Ubuntu, but when I
use imshow(), followed by close('all') and another imshow(), I run out
of memory. I can see from
the OS that the memory does not come back after close() and that it
grows after the second imshow().

Any other ideas? Looks like a bug to me otherwise.

Except that I tried the same things and did not get quite the same
result. Let's track this down. Please try the attached script, and see
if the memory usage grows substantially, or just oscillates a bit.

Eric

[...]

I'll put it in as an enhancement, but I'm still unsure if there is a
bug in
there as well. Is there something I should be doing to clear memory
after the
first figure is closed other than close()? I don't understand why
memory usage
grows each time I replot, but I'm pretty sure it isn't desireable
behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

I'm not quite sure, but I don't think there is a specifically
matplotlib
memory leak bug at work here. Are you using ipython, and if so, have
you
turned off the caching? In its default mode, ipython keeps lots of
references, thereby keeping memory in use. Also, memory management and
reporting can be a bit tricky and misleading.

Nevertheless, the attached script may be illustrating the problem. Try
running it from the command line as-is (maybe shorten the loop--it
doesn't take 100 iterations to show the pattern) and then commenting
out
the line as indicated in the comment. It seems that if anything is done
that adds ever so slightly to memory use while the figure is displayed,
then when the figure is closed, its memory is not reused. I'm puzzled.

I wasn't thinking straight--there is no mystery and no memory leak.
Ignore my example script referred to above. It was saving rows of the z
array, not single elements as I had intended, so of course memory use
was growing substantially.

Eric

You may not see a memory leak, but I still can't get my memory back
without killing python. I
turned off the ipython caching and even ran without iPython on both
Windows and Ubuntu, but when I
use imshow(), followed by close('all') and another imshow(), I run out
of memory. I can see from
the OS that the memory does not come back after close() and that it
grows after the second imshow().

Any other ideas? Looks like a bug to me otherwise.

Except that I tried the same things and did not get quite the same
result. Let's track this down. Please try the attached script, and see
if the memory usage grows substantially, or just oscillates a bit.

Eric

One thing I noticed is that if I add a "def __del__(self): print 'del'"
to image._AxesImageBase, it never gets called. _AxesImageBase keeps
float64 and uint8 rgba images in a cache, which is never freed.

Adding a __del__ method defeats (or blocks) the garbage collection.

Since self._imcache is an instance attribute, when the instance is no longer referenced, it should get garbage-collected, provided there is no __del__ method.

Eric

···

On 02/03/2011 12:28 PM, Christoph Gohlke wrote:

On 2/3/2011 2:15 PM, Eric Firing wrote:

On 02/03/2011 11:30 AM, Robert Abiad wrote:

On 2/3/2011 10:06 AM, Eric Firing wrote:

On 02/02/2011 10:17 PM, Eric Firing wrote:

On 02/02/2011 08:38 PM, Robert Abiad wrote:

Christoph

------------------------------------------------------------------------------
The modern datacenter depends on network connectivity to access resources
and provide services. The best practices for maximizing a physical server's
connectivity to a physical network are well understood - see how these
rules translate into the virtual world?
http://p.sf.net/sfu/oracle-sfdevnlfb
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

[...]

I'll put it in as an enhancement, but I'm still unsure if there is a
bug in
there as well. Is there something I should be doing to clear memory
after the
first figure is closed other than close()? I don't understand why
memory usage
grows each time I replot, but I'm pretty sure it isn't desireable
behavior. As
I mentioned, this effect is worse with plot.

So is this a bug or improper usage?

I'm not quite sure, but I don't think there is a specifically
matplotlib
memory leak bug at work here. Are you using ipython, and if so, have
you
turned off the caching? In its default mode, ipython keeps lots of
references, thereby keeping memory in use. Also, memory management and
reporting can be a bit tricky and misleading.

Nevertheless, the attached script may be illustrating the problem. Try
running it from the command line as-is (maybe shorten the loop--it
doesn't take 100 iterations to show the pattern) and then commenting
out
the line as indicated in the comment. It seems that if anything is done
that adds ever so slightly to memory use while the figure is displayed,
then when the figure is closed, its memory is not reused. I'm puzzled.

I wasn't thinking straight--there is no mystery and no memory leak.
Ignore my example script referred to above. It was saving rows of the z
array, not single elements as I had intended, so of course memory use
was growing substantially.

Eric

You may not see a memory leak, but I still can't get my memory back
without killing python. I
turned off the ipython caching and even ran without iPython on both
Windows and Ubuntu, but when I
use imshow(), followed by close('all') and another imshow(), I run out
of memory. I can see from
the OS that the memory does not come back after close() and that it
grows after the second imshow().

Any other ideas? Looks like a bug to me otherwise.

Except that I tried the same things and did not get quite the same
result. Let's track this down. Please try the attached script, and see
if the memory usage grows substantially, or just oscillates a bit.

Eric

One thing I noticed is that if I add a "def __del__(self): print 'del'"
to image._AxesImageBase, it never gets called. _AxesImageBase keeps
float64 and uint8 rgba images in a cache, which is never freed.

Adding a __del__ method defeats (or blocks) the garbage collection.

Sorry, never heard of that. I thought __del__() is called when the reference count reaches 0.

Christoph

···

On 2/3/2011 2:44 PM, Eric Firing wrote:

On 02/03/2011 12:28 PM, Christoph Gohlke wrote:

On 2/3/2011 2:15 PM, Eric Firing wrote:

On 02/03/2011 11:30 AM, Robert Abiad wrote:

On 2/3/2011 10:06 AM, Eric Firing wrote:

On 02/02/2011 10:17 PM, Eric Firing wrote:

On 02/02/2011 08:38 PM, Robert Abiad wrote:

Since self._imcache is an instance attribute, when the instance is no
longer referenced, it should get garbage-collected, provided there is no
__del__ method.

Eric

Christoph

------------------------------------------------------------------------------
The modern datacenter depends on network connectivity to access resources
and provide services. The best practices for maximizing a physical server's
connectivity to a physical network are well understood - see how these
rules translate into the virtual world?
http://p.sf.net/sfu/oracle-sfdevnlfb
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

------------------------------------------------------------------------------
The modern datacenter depends on network connectivity to access resources
and provide services. The best practices for maximizing a physical server's
connectivity to a physical network are well understood - see how these
rules translate into the virtual world?
http://p.sf.net/sfu/oracle-sfdevnlfb
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users