imshow memory problem

Tomas_Farago · May 18, 2010, 4:31pm

I am using "pmap -pid" on Linux and Task manager on Windows. Memory usage is comparable on both operating systems so I think the memory consumption information is accurate.

···

------------ Pôvodná správa ------------
Od: Michael Droettboom <mdroe@...86...>
Predmet: Re: [Matplotlib-users] imshow memory problem
Dátum: 18.5.2010 16:18:23
----------------------------------------
What are you using to calculate memory usage? I feel the only truly reliable tool is something instrumented like "valgrind --tool=massif".

============================================================================
BUILDING MATPLOTLIB
            matplotlib: 1.0.svn
                python: 2.5.2 (r252:60911, May 7 2008, 12:40:32) [GCC
                        3.4.6 20060404 (Red Hat 3.4.6-9)]
              platform: linux2

REQUIRED DEPENDENCIES
                 numpy: 2.0.0.dev8055
             freetype2: 9.16.3

OPTIONAL BACKEND DEPENDENCIES
                libpng: 1.2.37
               Tkinter: Tkinter: 50704, Tk: 8.4, Tcl: 8.4
              wxPython: 2.8.6.1
                        * WxAgg extension not required for wxPython >= 2.8
                  Gtk+: gtk+: 2.10.9, glib: 2.16.1, pygtk: 2.10.4,
                        pygobject: 2.13.1
       Mac OS X native: no
                    Qt: Qt: 3.3.3, PyQt: 3.17.2
                   Qt4: Qt: 4.6.2, PyQt4: 4.7.3
                 Cairo: 1.4.0

OPTIONAL DATE/TIMEZONE DEPENDENCIES
              datetime: present, version unknown
              dateutil: 1.4.1
                  pytz: 2008c

OPTIONAL USETEX DEPENDENCIES
                dvipng: 1.12
           ghostscript: 7.07
                 latex: 3.1415926
               pdftops: 3.00

verbose-helpful:

$HOME=/home/mdroe
CONFIGDIR=/home/mdroe/.matplotlib
matplotlib data path
/home/mdroe/usr/lib/python2.5/site-packages/matplotlib/mpl-data
loaded rc file /home/mdroe/.matplotlib/matplotlibrc
matplotlib version 1.0.svn
verbose.level helpful
interactive is False
units is False
platform is linux2
Using fontManager instance from /home/mdroe/.matplotlib/fontList.cache
backend GTKAgg version 2.10.4
findfont: Matching
:family=sans-serif:style=normal:variant=normal:weight=normal:stretch=normal:size=medium

to Bitstream Vera Sans
(/home/mdroe/usr/lib/python2.5/site-packages/matplotlib/mpl-data/fonts/ttf/Vera.ttf)

with score of 0.000000

Tomáš Faragó wrote:
> Thanks for replying Mike,
> I tried it on Linux as well but I ran into the same problem. Perhaps
> it has something to do with other needed libraries (GTK+, etc.). Can
> you please tell me your libraries versions? I mean GTK+, pygtk, etc.
> Also output produced by --verbose-helpful could be useful. Thank you.
> Tomas.
>
>> ------------ Pôvodná správa ------------
>> Od: Michael Droettboom <mdroe@...86...>
>> Predmet: Re: [Matplotlib-users] imshow memory problem
>> Dátum: 17.5.2010 16:48:55
>> ----------------------------------------
>> On Linux, I only see about an extra 24kb being used when the canvas
>> is added to a window vs. not adding it (i.e. commenting out the
>> window.add(canvas) line).
>>
>> In general, here's the memory usage to be expected from imshow (if
>> it's a floating-point, not-rgb(a) array as you have here):
>>
>> The original data: 4-bytes-per-pixel for float32 or 8-bytes-per-pixel
>> for float64 (in your example the array is float64).
>> Intermediate float data: *if* the original is not float64, then an
>> intermediate float64 is created (not the case here)
>> The colorized data: 4-bytes-per-pixel at original array size
>> The sized data: 4-bytes-per-pixel at the scaled figure size
>>
>> I hope I'm not forgetting anything, but the point is that to support
>> high-speed rendering of plots, the memory usage is much greater than
>> the data itself. If your data is truly large, the usual technique is
>> to decimate or downsample it before passing it to matplotlib, as
>> you're not going to see more data points than pixels on your display
>> anyway.
>>
>> Mike
>>
>> Tomáš Faragó wrote:
>> > Hello,
>> > I am writing a GUI using GTK+ library. I have a question about axes
>> class
>> imshow method memory consumtion. If I pass the imshow an array, the
>> resulting
>> memory consuption is approximatelly 46 times greater than the array
>> size. If I
>> do not add the canvas to a window (in a code below), the memory
>> consuption is
>> "only" 8 times greater. Any tips on how to reduce the memory
>> consuption would be
>> very appreciated and any explanation of how much memmory imshow
>> allocates too.
>> Configuration and script are below.
>> >
>> > os: Windowx XP
>> > matplotlib version: 0.99.1
>> > downloaded from: sourceforge.net
>> >
>> > script:
>> > from matplotlib.figure import Figure
>> > from matplotlib.backends.backend_gtkagg import FigureCanvasGTKAgg
>> > from pylab import rand
>> > import gtk
>> >
>> > window = gtk.Window()
>> > window.connect("destroy", gtk.main_quit)
>> >
>> > figure = Figure(figsize=(8,6), dpi=72)
>> > canvas = FigureCanvasGTKAgg(figure)
>> > axes = figure.add_subplot(111)
>> >
>> > window.add(canvas)
>> >
>> > axes.imshow(rand(1024,1024))
>> > canvas.draw()
>> > window.show_all()
>> >
>> > gtk.main()
>> >
>> > verbose-helpful output:
>> > $HOME=C:\Documents and Settings\Sensej
>> > CONFIGDIR=C:\Documents and Settings\Sensej\.matplotlib
>> > matplotlib data path C:\Python26\lib\site-packages\matplotlib\mpl-data
>> > loaded rc file
>> C:\Python26\lib\site-packages\matplotlib\mpl-data\matplotlibrc
>> > matplotlib version 0.99.1
>> > verbose.level helpful
>> > interactive is False
>> > units is False
>> > platform is win32
>> > Using fontManager instance from C:\Documents and
>> Settings\Sensej\.matplotlib\fontList.cache
>> > backend GTKAgg version 2.12.1
>> > findfont: Matching
>>
:family=sans-serif:style=normal:variant=normal:weight=normal:stretch=normal:size=medium

>>
>> to Bitstream Vera Sans
>> (C:\Python26\lib\site-packages\matplotlib\mpl-data\fonts\ttf\Vera.ttf)
>> with
>> score of 0.000000
>> >
>> > Thank you,
>> > Tomas.
>> >
>>
------------------------------------------------------------------------------
>>
>> >
>> > _______________________________________________
>> > Matplotlib-users mailing list
>> > Matplotlib-users@lists.sourceforge.net
>> > matplotlib-users List Signup and Options
>> >
>> --
>> Michael Droettboom
>> Science Software Branch
>> Operations and Engineering Division
>> Space Telescope Science Institute
>> Operated by AURA for NASA
>>

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA