Can't figure out how to close figures/clear memory when looping

Community
1

Hello,

I’m trying to make bar charts of some linguistics data that’s outputted by another program. I’ve managed to get my data sorted in lists that I can then loop through with matplotlib to turn into a bar chart. However, I notice that my computer starts lagging more and more as the loop is running. Looking in task manager, the program creeps up its memory usage as the program goes on. After researching, most people can solve this my calling close() or clf(). I do stop getting the warning that 20 figures are open now, but my memory keeps creeping up!

Here is my looping code:

for x in range(0, len(visemeState)):
    plt.figure(figsize=(30, 15))
    plt.bar(visemes,visemeState[x])
    plt.title(filename)
    plt.xlabel('Visemes', fontsize = 20)
    plt.ylabel('Strength', fontsize = 20)
    plt.ylim([0,255])
    plt.savefig('images/{0}.png'.format(x), bbox_inches='tight')
    plt.close('all')
    plt.clf()

I’ve also tried using the external gc() to no avail.

The amount of images to generate varies per file inserted into my script, but it’s usually between 6,000-7,000 per file (my intention is to turn it into a 30fps video afterwards), so I’m not able to tough it out and let the memory creep up till it finishes.

Any help would be much appreciated.

2

There may or may not be a leak, but your code does not need multiple figures. The Axes limits, the text, and the number of bars do not change.

You should get the result of bar, and modify the heights at each frame, and save the same figure to a new file every time. This will not only be unlikely to run out of memory, but also a tiny bit faster since you won’t have to recreate things.

You may also wish to use FuncAnimation, which might handle most of the bookkeeping, and allow nearly realtime playback, though that’s not what you’re trying for.

3

Actually, looking at BarContainer, modifying it is a bit annoying, but the concept of not creating multiple figures is still good. Instead of modifying the result of bar, you can either remove the existing one and make a new one for each frame, or generate all of them at once and pass them to ArtistAnimation.

4

How are you making your figures, ie which backend are you using?

5

I ran the following from the command line, and no evidence of a memory leak. So I suspect the backend (or my inability to diagnose memory problems). I suspect if you are using jupyter or ipython in some way, the problem may lie there?

import matplotlib.pyplot as plt
import numpy as np

for i in range(100_000):
    fig, ax = plt.subplots()

    pc = ax.bar(np.arange(40), np.random.randn(40))
    ax.set_title(f'{i}')
    ax.set_xlabel('Visemes', fontsize = 20)
    ax.set_ylabel('Strength', fontsize = 20)
    fig.savefig(f'Boo{i}.png')
    plt.close('all')
6

I’m not exactly sure what backend I am using. I don’t define any, so it’d be the default backend used with Python 3.9 for 64-bit Windows machines.

Your code did still end up causing my memory to go up as each image was saved (though it seemed to increase slower than mine). Once I took the fig, ax statement out of the loop, it stayed stable.

I did this by taking the figure call outside of the loop and it cut down the memory usage a ton, which is awesome! Thanks so much for that.

I do now run into an issue where around the 30th image generated it starts slowing down my image generation. Around the 400th image it takes about 3 seconds to generate an image. I figure there’s still something I’m not clearing properly.

I tried keeping everything static out of the loop.

My code now looks like this:

fig, ax = plt.subplots(figsize=(30, 15))
ax.set_title(filename)
ax.set_xlabel('Visemes', fontsize = 20)
ax.set_ylabel('Strength', fontsize = 20)
ax.set_ylim((0,255))

for x,item in enumerate(visemeState):

    pc = ax.bar(visemes,item, color = ["#800000","#000099"]) #Each image seems to get a different colour if undefined, so I'm defining the same alternating colour pattern for each image.
    print("Saving image {0}".format(x))
    fig.savefig('images/{0}.png'.format(x))
    plt.close()

I think that the x labels (or are they called ticks?) being generated over and over (my visemes variable) might be causing that as they are the same in each figure, just the value associated with it changes. But I’m unsure how to fix that, as bar needs the labels and value of labels each time I modify it (from what I can see). Any suggestions to what could change here?

Thanks for both of your help so far!

7

You need to remove the bars after saving, otherwise they’ll just be there again and again for every continuing frame.

This closes the current figure without an argument, and you don’t want to close the figure in the loop.

8

I had to move all my ax calls back into the loop if I was clearing them each time, but this worked! Using ax.clear() at the very end generates about 5-6 frames/second and maintains a memory usage of about 100-120MB.
I should be able to multiprocess this to speed up generation and I’ll have a very neat way to present my data.

Thank you very much!

9

Clearing the Axes will be slow, because of regenerating the ticks and labels. Only removing pc should be quicker (minus some change if the ticks need to move a lot).

10

Ah, I understand now. The BarContainer container object (pc) has a remove() function. Calling it does indeed speed things up and I can keep the other axis labels outside my loop now.