I tried the test script on linux using matplotlib svn trunk rev8840
(which appears to include your patch) and found a leak that starts out
small but gets systematically larger. This is with Python 2.6.5 and
Tkinter built against Tcl/Tk 8.4.13.
Is anyone else seeing this?
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 36816 nan
5.0 36860 nan
10.0 36860 0.0
15.1 36860 0.0
20.1 36860 0.0
25.1 36896 1.8
...
326.5 36896 0.1
331.6 36972 0.3
...
401.9 36972 0.3
406.9 36980 0.3
411.9 37004 0.4
417.0 37016 0.4
422.0 37040 0.4
427.0 37052 0.5
432.0 37076 0.5
437.1 37088 0.5
442.1 37112 0.6
447.1 37124 0.6
452.1 37148 0.6
457.2 37160 0.7
462.2 37184 0.7
467.2 37196 0.7
472.2 37220 0.8
477.3 37232 0.8
482.3 37256 0.8
487.3 37268 0.8
492.3 37292 0.9
497.3 37304 0.9
502.3 37328 0.9
507.4 37340 1.0
512.4 37360 1.0
517.4 37376 1.0
522.4 37396 1.0
527.5 37412 1.1
532.5 37432 1.1
537.5 37448 1.1
542.5 37472 1.1
547.6 37488 1.2
552.6 37504 1.2
557.6 37516 1.2
562.6 37540 1.2
567.6 37556 1.2
572.7 37576 1.3
577.7 37592 1.3
582.7 37612 1.3
587.7 37628 1.3
592.7 37648 1.3
597.8 37664 1.4
602.8 37684 1.4
607.8 37700 1.4
This is different behavior than on Mac OS X, but still alarming.
-- Russell
In article <4D07A8A0.5000601@...31...>,
Michael Droettboom <mdroe@...31...>
wrote:
···
If you're able to try this with the 1.0.x SVN branch or the SVN trunk
(plus my text patch) let me know. I am not able to reproduce any sort
of memory leak with these newer versions, but I am able to with 1.0.0 as
released (with or without my text patch). This is with or without the x
axis limits updating you suggested. I believe there have been other
memory leak fixes since the 1.0.0 release.
Cheers,
Mike
On 12/13/2010 08:35 PM, Russell E. Owen wrote:
> I also wanted to say:
> - Thank you for the patch. I appreciate the chance to try a fix.
> - I doubt the issue is related to text because my scripts shows a memory
> leak even if the displayed text is never updated (comment out the line
> that modifies the x axis limits to see what I mean; though I confess I
> did not try that with the trunk matplotlib, only with 1.0.0).
>
> -- Russell
>
> In article<4D028BC7.1000304@...31...>,
> Michael Droettboom<mdroe@...31...>
> wrote:
>
>
>> I think I'm on to something -- it seems that text layout information has
>> a cyclical reference that prevents the Text object from being freed.
>>
>> Can you apply the attached patch and let me know if it solves your issue?
Russell,
I am curious, I recently ran into problems with mixing builds of numpy and matplotlib at various levels of revisions. I eventually had to do a complete clean rebuild. Note, what I mean by a complete clean rebuild is that I removed the numpy and matplotlib source directories and re-checkout the codes from source and then rebuild. I would be curious if the problem persists after that.
Ben Root
···
On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <rowen@…748…> wrote:
I tried the test script on linux using matplotlib svn trunk rev8840
(which appears to include your patch) and found a leak that starts out
small but gets systematically larger. This is with Python 2.6.5 and
Tkinter built against Tcl/Tk 8.4.13.
Is anyone else seeing this?
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 36816 nan
5.0 36860 nan
10.0 36860 0.0
15.1 36860 0.0
20.1 36860 0.0
25.1 36896 1.8
…
326.5 36896 0.1
331.6 36972 0.3
…
401.9 36972 0.3
406.9 36980 0.3
411.9 37004 0.4
417.0 37016 0.4
422.0 37040 0.4
427.0 37052 0.5
432.0 37076 0.5
437.1 37088 0.5
442.1 37112 0.6
447.1 37124 0.6
452.1 37148 0.6
457.2 37160 0.7
462.2 37184 0.7
467.2 37196 0.7
472.2 37220 0.8
477.3 37232 0.8
482.3 37256 0.8
487.3 37268 0.8
492.3 37292 0.9
497.3 37304 0.9
502.3 37328 0.9
507.4 37340 1.0
512.4 37360 1.0
517.4 37376 1.0
522.4 37396 1.0
527.5 37412 1.1
532.5 37432 1.1
537.5 37448 1.1
542.5 37472 1.1
547.6 37488 1.2
552.6 37504 1.2
557.6 37516 1.2
562.6 37540 1.2
567.6 37556 1.2
572.7 37576 1.3
577.7 37592 1.3
582.7 37612 1.3
587.7 37628 1.3
592.7 37648 1.3
597.8 37664 1.4
602.8 37684 1.4
607.8 37700 1.4
This is different behavior than on Mac OS X, but still alarming.
– Russell
An interesting question. I can say that this was a clean build of matplotlib since I ran it “in place” (in build/lib.linux-x86_64-2.6/) after building it rather than installing it in site-packages to avoid messing up other users (on the linux system this was a shared python). I modified the script to print matplotlib.file to make sure I was running the right version. I doubt it was a clean build of numpy. But considering no numerics are occurring in this example (it is literally just creating an Axes on a Canvas and calling canvas.draw() repeatedly) do you think numpy could possibly come into this?
Might you run the script on your system with the clean build?
– Russell
···
On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:
On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <rowen@…748…> wrote:
I tried the test script on linux using matplotlib svn trunk rev8840
(which appears to include your patch) and found a leak that starts out
small but gets systematically larger. This is with Python 2.6.5 and
Tkinter built against Tcl/Tk 8.4.13.
Is anyone else seeing this?
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 36816 nan
5.0 36860 nan
10.0 36860 0.0
15.1 36860 0.0
20.1 36860 0.0
25.1 36896 1.8
…
326.5 36896 0.1
331.6 36972 0.3
…
401.9 36972 0.3
406.9 36980 0.3
…
602.8 37684 1.4
607.8 37700 1.4
This is different behavior than on Mac OS X, but still alarming.
– Russell
Russell,
I am curious, I recently ran into problems with mixing builds of numpy and matplotlib at various levels of revisions. I eventually had to do a complete clean rebuild. Note, what I mean by a complete clean rebuild is that I removed the numpy and matplotlib source directories and re-checkout the codes from source and then rebuild. I would be curious if the problem persists after that.
I tried the test script on linux using matplotlib svn trunk rev8840
(which appears to include your patch) and found a leak that starts out
small but gets systematically larger. This is with Python 2.6.5 and
Tkinter built against Tcl/Tk 8.4.13.
Is anyone else seeing this?
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 36816 nan
5.0 36860 nan
10.0 36860 0.0
15.1 36860 0.0
20.1 36860 0.0
25.1 36896 1.8
…
326.5 36896 0.1
331.6 36972 0.3
…
401.9 36972 0.3
406.9 36980 0.3
…
602.8 37684 1.4
607.8 37700 1.4
This is different behavior than on Mac OS X, but still alarming.
– Russell
Russell,
I am curious, I recently ran into problems with mixing builds of numpy and matplotlib at various levels of revisions. I eventually had to do a complete clean rebuild. Note, what I mean by a complete clean rebuild is that I removed the numpy and matplotlib source directories and re-checkout the codes from source and then rebuild. I would be curious if the problem persists after that.
An interesting question. I can say that this was a clean build of matplotlib since I ran it “in place” (in build/lib.linux-x86_64-2.6/) after building it rather than installing it in site-packages to avoid messing up other users (on the linux system this was a shared python). I modified the script to print matplotlib.file to make sure I was running the right version. I doubt it was a clean build of numpy. But considering no numerics are occurring in this example (it is literally just creating an Axes on a Canvas and calling canvas.draw() repeatedly) do you think numpy could possibly come into this?
It is quite possible. Numpy is used extensively throughout the matplotlib system, regardless of whether you are using it or not. Also, the fact that you are on a shared system is interesting. For those situations, try doing
“python setupegg.py develop --user”
for the build command. What this does is builds everything in-place (the build directory symbolically links to those files), and then uses your ~/.local directory to install an egg.lnk file to point back to the build directory. This should have higher search precedence than the system install of matplotlib and numpy.
Note, I had mixed success with this approach on a RHEL (5?) system recently. I don’t know how recently ~/.local became widely accepted among various distros.
Might you run the script on your system with the clean build?
– Russell
I will give it a shot (once my other analysis programs are done for the day).
Ben Root
···
On Tue, Dec 14, 2010 at 4:06 PM, Russell Owen <rowen@…902…> wrote:
On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:
On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <rowen@…748…> wrote:
I ran your script from the url you posted earlier. My build does not show any leak for about a minute. I am fairly certain that what we have here is a build mixing issue or an issue with an unknown combination of libraries interacting. Could you post your build logs?
Ben Root
···
On Tue, Dec 14, 2010 at 4:22 PM, Benjamin Root <ben.root@…553…> wrote:
On Tue, Dec 14, 2010 at 4:06 PM, Russell Owen <rowen@…748…> wrote:
On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:
On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <rowen@…748…> wrote:
I tried the test script on linux using matplotlib svn trunk rev8840
(which appears to include your patch) and found a leak that starts out
small but gets systematically larger. This is with Python 2.6.5 and
Tkinter built against Tcl/Tk 8.4.13.
Is anyone else seeing this?
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 36816 nan
5.0 36860 nan
10.0 36860 0.0
15.1 36860 0.0
20.1 36860 0.0
25.1 36896 1.8
…
326.5 36896 0.1
331.6 36972 0.3
…
401.9 36972 0.3
406.9 36980 0.3
…
602.8 37684 1.4
607.8 37700 1.4
This is different behavior than on Mac OS X, but still alarming.
– Russell
Russell,
I am curious, I recently ran into problems with mixing builds of numpy and matplotlib at various levels of revisions. I eventually had to do a complete clean rebuild. Note, what I mean by a complete clean rebuild is that I removed the numpy and matplotlib source directories and re-checkout the codes from source and then rebuild. I would be curious if the problem persists after that.
An interesting question. I can say that this was a clean build of matplotlib since I ran it “in place” (in build/lib.linux-x86_64-2.6/) after building it rather than installing it in site-packages to avoid messing up other users (on the linux system this was a shared python). I modified the script to print matplotlib.file to make sure I was running the right version. I doubt it was a clean build of numpy. But considering no numerics are occurring in this example (it is literally just creating an Axes on a Canvas and calling canvas.draw() repeatedly) do you think numpy could possibly come into this?
It is quite possible. Numpy is used extensively throughout the matplotlib system, regardless of whether you are using it or not. Also, the fact that you are on a shared system is interesting. For those situations, try doing
“python setupegg.py develop --user”
for the build command. What this does is builds everything in-place (the build directory symbolically links to those files), and then uses your ~/.local directory to install an egg.lnk file to point back to the build directory. This should have higher search precedence than the system install of matplotlib and numpy.
Note, I had mixed success with this approach on a RHEL (5?) system recently. I don’t know how recently ~/.local became widely accepted among various distros.
Might you run the script on your system with the clean build?
– Russell
I will give it a shot (once my other analysis programs are done for the day).
Ben Root
I tried your suggestion – installing numpy from scratch (deleting the old numpy first) and then building matplotlib fresh. Here are my build logs:
<http://www.astro.washington.edu/users/rowen/python/numpy_build_log.txt>
<http://www.astro.washington.edu/users/rowen/python/matplotlib_build_log.txt>
Much like last time (with a fresh matplotlib) the test program shows no leak at first, then it starts growing to alarming levels (though it took longer to start leaking this time than it did last time):
matplotlib.file= /astro/users/rowen/build/matplotlib-trunk/build/lib.linux-x86_64-2.6/matplotlib/init.py
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 36860 nan
5.1 36904 nan
10.1 36904 0.0
15.1 36904 0.0
20.2 36904 0.0
25.2 36904 0.0
30.2 36904 0.0
35.2 36904 0.0
40.2 36904 0.0
45.2 36904 0.0
50.3 36904 0.0
55.3 36904 0.0
60.3 36904 0.0
65.4 36904 0.0
70.4 36904 0.0
75.4 36904 0.0
80.4 36904 0.0
85.4 36904 0.0
90.5 36904 0.0
95.5 36904 0.0
100.5 36976 0.8
105.5 36976 0.7
110.6 36976 0.7
115.6 36976 0.7
120.6 36976 0.6
125.6 36976 0.6
130.7 36976 0.6
135.7 36976 0.6
140.7 36976 0.5
145.7 36976 0.5
150.8 36976 0.5
155.8 36976 0.5
160.8 36976 0.5
165.8 36976 0.4
170.9 36976 0.4
175.9 36976 0.4
180.9 36976 0.4
185.9 36976 0.4
190.9 36976 0.4
196.0 36976 0.4
201.0 36976 0.4
206.0 36976 0.4
211.0 36976 0.3
216.1 36976 0.3
221.1 36976 0.3
226.1 36976 0.3
231.1 36976 0.3
236.1 36976 0.3
241.2 36976 0.3
246.2 36976 0.3
251.2 36976 0.3
256.3 36976 0.3
261.3 36976 0.3
266.3 36976 0.3
271.4 36976 0.3
276.4 36976 0.3
281.4 36976 0.3
286.5 36976 0.3
291.5 36976 0.3
296.6 36976 0.2
301.6 36976 0.2
306.6 36976 0.2
311.6 36976 0.2
316.6 36976 0.2
321.6 36976 0.2
326.7 36976 0.2
331.7 37048 0.4
336.7 37048 0.4
341.7 37048 0.4
346.8 37048 0.4
351.8 37048 0.4
356.8 37048 0.4
361.8 37048 0.4
366.8 37048 0.4
371.8 37048 0.4
376.9 37048 0.4
381.9 37048 0.4
386.9 37048 0.4
391.9 37048 0.4
397.0 37048 0.4
402.0 37048 0.4
407.0 37072 0.4
412.0 37084 0.4
417.1 37108 0.5
422.1 37124 0.5
427.1 37144 0.6
432.1 37156 0.6
437.2 37180 0.6
442.2 37196 0.7
447.2 37216 0.7
452.2 37228 0.7
457.2 37248 0.8
462.3 37264 0.8
467.3 37284 0.8
472.3 37300 0.8
477.3 37320 0.9
482.3 37336 0.9
487.4 37356 0.9
492.4 37372 1.0
497.4 37392 1.0
502.4 37404 1.0
507.4 37424 1.0
512.5 37444 1.1
517.5 37460 1.1
522.5 37476 1.1
527.6 37496 1.1
532.6 37512 1.2
537.6 37532 1.2
542.6 37548 1.2
547.6 37568 1.2
552.6 37584 1.2
557.7 37604 1.3
562.7 37620 1.3
567.7 37636 1.3
572.8 37652 1.3
577.8 37676 1.3
582.9 37688 1.4
587.9 37708 1.4
593.0 37724 1.4
598.0 37744 1.4
603.0 37760 1.4
608.0 37780 1.5
613.0 37796 1.5
618.1 37816 1.5
623.1 37828 1.5
628.1 37848 1.5
633.2 37864 1.5
638.2 37884 1.5
643.3 37904 1.6
648.3 37920 1.6
653.4 37936 1.6
658.4 37956 1.6
663.4 37972 1.6
668.4 37992 1.6
(control-C to stop it)
Also like last time (though I did not report this before since I had not yet discovered it) once this has occurred the honeymoon is over. Leakage is high right from the beginning. It appears that something is being cached that causes trouble, but I have no idea what. Here is a log from a run immediately after the previous run:
-bash-3.2$ python matplotlibMemoryLeak.py
matplotlib.file= /astro/users/rowen/build/matplotlib-trunk/build/lib.linux-x86_64-2.6/matplotlib/init.pyc
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 33892 nan
5.0 34168 nan
10.1 34188 4.0
15.1 34228 6.0
20.1 34248 5.3
25.1 34284 5.8
30.2 34300 5.2
35.2 34320 5.0
40.2 34400 6.6
Might you try running your test for much longer to see if the program starts leaking for you?
– Russell
···
On Dec 14, 2010, at 2:38 PM, Benjamin Root wrote:
On Tue, Dec 14, 2010 at 4:22 PM, Benjamin Root <ben.root@…553…> wrote:
On Tue, Dec 14, 2010 at 4:06 PM, Russell Owen <rowen@…748…> wrote:
On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:
On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <rowen@…748…> wrote:
I tried the test script on linux using matplotlib svn trunk rev8840
(which appears to include your patch) and found a leak that starts out
small but gets systematically larger. This is with Python 2.6.5 and
Tkinter built against Tcl/Tk 8.4.13.
Is anyone else seeing this?
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 36816 nan
5.0 36860 nan
10.0 36860 0.0
15.1 36860 0.0
20.1 36860 0.0
25.1 36896 1.8
…
326.5 36896 0.1
331.6 36972 0.3
…
401.9 36972 0.3
406.9 36980 0.3
…
602.8 37684 1.4
607.8 37700 1.4
This is different behavior than on Mac OS X, but still alarming.
– Russell
Russell,
I am curious, I recently ran into problems with mixing builds of numpy and matplotlib at various levels of revisions. I eventually had to do a complete clean rebuild. Note, what I mean by a complete clean rebuild is that I removed the numpy and matplotlib source directories and re-checkout the codes from source and then rebuild. I would be curious if the problem persists after that.
An interesting question. I can say that this was a clean build of matplotlib since I ran it “in place” (in build/lib.linux-x86_64-2.6/) after building it rather than installing it in site-packages to avoid messing up other users (on the linux system this was a shared python). I modified the script to print matplotlib.file to make sure I was running the right version. I doubt it was a clean build of numpy. But considering no numerics are occurring in this example (it is literally just creating an Axes on a Canvas and calling canvas.draw() repeatedly) do you think numpy could possibly come into this?
It is quite possible. Numpy is used extensively throughout the matplotlib system, regardless of whether you are using it or not. Also, the fact that you are on a shared system is interesting. For those situations, try doing
“python setupegg.py develop --user”
for the build command. What this does is builds everything in-place (the build directory symbolically links to those files), and then uses your ~/.local directory to install an egg.lnk file to point back to the build directory. This should have higher search precedence than the system install of matplotlib and numpy.
Note, I had mixed success with this approach on a RHEL (5?) system recently. I don’t know how recently ~/.local became widely accepted among various distros.
Might you run the script on your system with the clean build?
– Russell
I will give it a shot (once my other analysis programs are done for the day).
Ben Root
I ran your script from the url you posted earlier. My build does not show any leak for about a minute. I am fairly certain that what we have here is a build mixing issue or an issue with an unknown combination of libraries interacting. Could you post your build logs?
Ben Root
Interesting behavior to report. I did a completely clean rebuild on my other computer (Ubuntu 10.10) and the first matplotlib-related program that was used was this memory check program. It exhibited a memory leak for the very first run. However, after ending that program and starting it back up, the memory levels were lower and very steady.
I also noticed for the runs where the memory levels were steady, I could make the memory levels go slightly up if I interacted with the figure window (moving it and such).
I did do a long run (600+ seconds) and I did get a memory leak at around 600 second mark. I did not have the same problem, though, where the next run exhibits an immediate memory leak. Instead, the memory levels were consistent with previous runs.
I will take a peek at your build logs tomorrow. There is something fishy happening, but I am not convinced that it is entirely an issue with matplotlib (although that first run behavior is bad…).
Ben Root
···
On Tue, Dec 14, 2010 at 5:26 PM, Russell Owen <rowen@…748…> wrote:
On Dec 14, 2010, at 2:38 PM, Benjamin Root wrote:
On Tue, Dec 14, 2010 at 4:22 PM, Benjamin Root <ben.root@…553…> wrote:
On Tue, Dec 14, 2010 at 4:06 PM, Russell Owen <rowen@…748…> wrote:
On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:
On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <rowen@…748…> wrote:
I tried the test script on linux using matplotlib svn trunk rev8840
(which appears to include your patch) and found a leak that starts out
small but gets systematically larger. This is with Python 2.6.5 and
Tkinter built against Tcl/Tk 8.4.13.
Is anyone else seeing this?
time rss memory mem/sec
(sec) (kb) (kb/sec)
0.0 36816 nan
5.0 36860 nan
10.0 36860 0.0
15.1 36860 0.0
20.1 36860 0.0
25.1 36896 1.8
…
326.5 36896 0.1
331.6 36972 0.3
…
401.9 36972 0.3
406.9 36980 0.3
…
602.8 37684 1.4
607.8 37700 1.4
This is different behavior than on Mac OS X, but still alarming.
– Russell
Russell,
I am curious, I recently ran into problems with mixing builds of numpy and matplotlib at various levels of revisions. I eventually had to do a complete clean rebuild. Note, what I mean by a complete clean rebuild is that I removed the numpy and matplotlib source directories and re-checkout the codes from source and then rebuild. I would be curious if the problem persists after that.
An interesting question. I can say that this was a clean build of matplotlib since I ran it “in place” (in build/lib.linux-x86_64-2.6/) after building it rather than installing it in site-packages to avoid messing up other users (on the linux system this was a shared python). I modified the script to print matplotlib.file to make sure I was running the right version. I doubt it was a clean build of numpy. But considering no numerics are occurring in this example (it is literally just creating an Axes on a Canvas and calling canvas.draw() repeatedly) do you think numpy could possibly come into this?
It is quite possible. Numpy is used extensively throughout the matplotlib system, regardless of whether you are using it or not. Also, the fact that you are on a shared system is interesting. For those situations, try doing
“python setupegg.py develop --user”
for the build command. What this does is builds everything in-place (the build directory symbolically links to those files), and then uses your ~/.local directory to install an egg.lnk file to point back to the build directory. This should have higher search precedence than the system install of matplotlib and numpy.
Note, I had mixed success with this approach on a RHEL (5?) system recently. I don’t know how recently ~/.local became widely accepted among various distros.
Might you run the script on your system with the clean build?
– Russell
I will give it a shot (once my other analysis programs are done for the day).
Ben Root
I ran your script from the url you posted earlier. My build does not show any leak for about a minute. I am fairly certain that what we have here is a build mixing issue or an issue with an unknown combination of libraries interacting. Could you post your build logs?
Ben Root
I tried your suggestion – installing numpy from scratch (deleting the old numpy first) and then building matplotlib fresh. Here are my build logs:
<http://www.astro.washington.edu/users/rowen/python/numpy_build_log.txt>
<http://www.astro.washington.edu/users/rowen/python/matplotlib_build_log.txt>
Much like last time (with a fresh matplotlib) the test program shows no leak at first, then it starts growing to alarming levels (though it took longer to start leaking this time than it did last time):