Possible Bug

Hello devels, I chose to use matplotlib for a restaurant

    > simulation. It supplied all the features I needed and I was
    > pleasantly surprised. However the people I was doing this
    > project for mentioned that the simulation slowed down to a
    > crawl 10 or so minutes into the simulation. It sounded like
    > a memory leak to me so I investigated and found a 100 KB/sec
    > leak in Windows (via the interpreter as well as frozen with
    > py2exe) as well as in Linux. I narrowed it down to a part of
    > the code where I updated xlim and ylim via Axes::set_xlim,
    > Axes::set_ylim. Commenting that part out yielded no leak,

So you're doing animation right? Are you using images, eg imshow? I
just fixed a memory leak in image resizing that amounts to about
200K/resize -- the changes are in CVS. Of course on windows it's hard
to test because the changes are in extension code, and require
building on win32. The instructions for building on win32 are in the
file setupext.py.

If you are not using images, another place to look is in text caching.
text layout is expensive (to support arbitrary rotations, etc). So I
cache the layout information in a dictionary "cached" in the text
module. When you change the xlim/ylim, that would trigger a new
layout and an addition to the cache. We need to add some
auto-clearing mechanisms to prevent the cache from growing w/o bound
but its not done yet. I find it hard to believe, though, that this
would leak 100kb/sec. Assuming a 20Hz frame rate, the leak is about
5kb/sec, and I don't think I'm caching that much info.

You can estimate the memory effect of the cached simply by commenting
out the line

        self.cached[key] = ret

in the matplotlib.text module.

What backend are you using? Are you refreshing the same figure data,
or creating new figures? More information would certainly help.....

    > however I lost some critical functionality in my charts
    > (which behave like the Windows Task Manager resource monitor
    > graphs). So I really need this fixed. I will send shortly a
    > test case which replicates the problem.

That would be great.

    > Since the maintainer is out on vacation, I was also wondering
    > if anyone knew enough about the internals of matplotlib that
    > they could give me an idea of which modules (Python or
    > otherwise) are affected by Axes::set_xlim and Axes::set_ylim.

I'm back. I was in Rio for a couple of weeks and have suffered a 95
degree temperature drop returning to Chicago (sigh)...

JDH

Thanks John,

Sheesh, 95 degrees, I can't imagine how I'd take that. Some Christmases my
family would go visit relatives in Chicago. It seemed just a little bit more
warmer (but not that much) when we returned to Oklahoma. Never really gone
anywhere to warm for Christmas.

More below...

    > Hello devels, I chose to use matplotlib for a restaurant
    > simulation. It supplied all the features I needed and I was
    > pleasantly surprised. However the people I was doing this
    > project for mentioned that the simulation slowed down to a
    > crawl 10 or so minutes into the simulation. It sounded like
    > a memory leak to me so I investigated and found a 100 KB/sec
    > leak in Windows (via the interpreter as well as frozen with
    > py2exe) as well as in Linux. I narrowed it down to a part of
    > the code where I updated xlim and ylim via Axes::set_xlim,
    > Axes::set_ylim. Commenting that part out yielded no leak,

So you're doing animation right? Are you using images, eg imshow? I
just fixed a memory leak in image resizing that amounts to about
200K/resize -- the changes are in CVS. Of course on windows it's hard
to test because the changes are in extension code, and require
building on win32. The instructions for building on win32 are in the
file setupext.py.

I wish so, but no, I'm not using any images.

If you are not using images, another place to look is in text caching.
text layout is expensive (to support arbitrary rotations, etc). So I
cache the layout information in a dictionary "cached" in the text
module. When you change the xlim/ylim, that would trigger a new
layout and an addition to the cache. We need to add some
auto-clearing mechanisms to prevent the cache from growing w/o bound
but its not done yet. I find it hard to believe, though, that this
would leak 100kb/sec. Assuming a 20Hz frame rate, the leak is about
5kb/sec, and I don't think I'm caching that much info.

I am using 5 GTKAgg FIgure widgets with 2-3 subplots a piece (a maximum of 3
subplots get drawn at a time). The simulation runs at 1 Hz. Therefore 6.67
kB/sec at max. Which sounds about right. I am updating each Axes point set
when I do the update.

You can estimate the memory effect of the cached simply by commenting
out the line

        self.cached[key] = ret

in the matplotlib.text module.

I'll definitely check that out. With the above math that just might prove to
be the culprit.

What backend are you using? Are you refreshing the same figure data,
or creating new figures? More information would certainly help.....

    > however I lost some critical functionality in my charts
    > (which behave like the Windows Task Manager resource monitor
    > graphs). So I really need this fixed. I will send shortly a
    > test case which replicates the problem.

That would be great.

    > Since the maintainer is out on vacation, I was also wondering
    > if anyone knew enough about the internals of matplotlib that
    > they could give me an idea of which modules (Python or
    > otherwise) are affected by Axes::set_xlim and Axes::set_ylim.

I'm back. I was in Rio for a couple of weeks and have suffered a 95
degree temperature drop returning to Chicago (sigh)...

JDH

I'll get the test case up on the web ASAP, I just wanted to reply now to let
everyone know that I'm still working on the problem.

Joe

···

On Tuesday 18 January 2005 20:25, John Hunter wrote: