Hi!
[I think the message didn’t get through the first time I sent it. Resending, and apologies if you get it twice]
I have a rather complex basemap-derived plot that I want to save as animation.
In essence, it uses the blumarble() to add a nice background, plots some
stuff on top of that, and also has an inset with a map of the world that
shows the area of the main map.
The problem is that memory usage as I iterate through the frames that will
eventually make up my animation, rises very fast. I read a post some time ago
on how to store the background, so it didn’t have to be recreated everytime
here:
<http://www.nabble.com/Save-a-plot-background-to20519596.html#a20519596>
There’s an example on how to actually do this for basemaps from J Whitaker,
but I can’t get this to work on my example. Ideally, I would like to store
the inset in a function, and my main background in another function, and
restore each axis directly. My attempt follows (which produces a blank image)
is at <http://pastebin.com/f4636fd7f>
Many thanks!
Jose
Jose Gomez-Dans wrote:
Hi!
[I think the message didn't get through the first time I sent it. Resending, and apologies if you get it twice]
I have a rather complex basemap-derived plot that I want to save as animation.
In essence, it uses the blumarble() to add a nice background, plots some
stuff on top of that, and also has an inset with a map of the world that
shows the area of the main map.
The problem is that memory usage as I iterate through the frames that will
eventually make up my animation, rises very fast. I read a post some time ago
on how to store the background, so it didn't have to be recreated everytime
here:
<http://www.nabble.com/Save-a-plot-background-to20519596.html#a20519596>
There's an example on how to actually do this for basemaps from J Whitaker,
but I can't get this to work on my example. Ideally, I would like to store
the inset in a function, and my main background in another function, and
restore each axis directly. My attempt follows (which produces a blank image)
is at <http://pastebin.com/f4636fd7f>
Many thanks!
Jose
Jose: I think the key is to only create the basemap instance only once (for the main plot and for the inset), then re-use that basemap instance each time you create an animation frame.
-Jeff
···
--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@...259...
325 Broadway Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web : Jeffrey S. Whitaker: NOAA Physical Sciences Laboratory
Jeff, thanks for your comment
2009/3/2 Jeff Whitaker <jswhit@…146…>
Jose: I think the key is to only create the basemap instance only once (for the main plot and for the inset), then re-use that basemap instance each time you create an animation frame.
That helps in memory consumption, and that’s something I looked into. However, my basemap instances are defined (in the case of the inset) as:
m2 = Basemap(projection=‘ortho’,lon_0=6,lat_0=-12,ax=ax2)
Actually, they both have references to axes. The bit I don’t really understand is how to define my basemap only once and then just “hang it” from some axes.
Additionally, I am warping a fairly large image (not shown in my example), so that it takes quite a long time to process. This is only seldom updated, so I could just save the background and overlay on it my data. As I see it, even if I recycle the basemap instance, I’d still have to go through the process of warping my image, extracting coastlines, etc. It would be faster to just save it, and recreate it whenever it is necessary. I don’t really know how to do this.
I have changed my code by shamelessly copying from your previous advice ;p (see <http://pastebin.com/f4eaedd7>). Things improve a bit, but my parllels/meridians are not aligned with the map. Clearly I’m still missing something!
Many thanks for your help,
Jose
Jose Gomez-Dans wrote:
Jeff, thanks for your comment
2009/3/2 Jeff Whitaker <jswhit@...146... <mailto:jswhit@…146…>>
Jose: I think the key is to only create the basemap instance only
once (for the main plot and for the inset), then re-use that
basemap instance each time you create an animation frame.
That helps in memory consumption, and that's something I looked into. However, my basemap instances are defined (in the case of the inset) as:
m2 = Basemap(projection='ortho',lon_0=6,lat_0=-12,ax=ax2)
Actually, they both have references to axes. The bit I don't really understand is how to define my basemap only once and then just "hang it" from some axes.
Jose: You can create the Basemap instance once, and then pass the axes with the ax keyword whenever you call a basemap method.
Additionally, I am warping a fairly large image (not shown in my example), so that it takes quite a long time to process. This is only seldom updated, so I could just save the background and overlay on it my data. As I see it, even if I recycle the basemap instance, I'd still have to go through the process of warping my image, extracting coastlines, etc. It would be faster to just save it, and recreate it whenever it is necessary. I don't really know how to do this.
The image is only warped the first time you call the bluemarble method for a given basemap instance. Subsequent calls used the cached image.
I have changed my code by shamelessly copying from your previous advice ;p (see <http://pastebin.com/f4eaedd7>\). Things improve a bit, but my parllels/meridians are not aligned with the map. Clearly I'm still missing something!
I don't have time right now to look at that code, I'll try to play with it tomorrow.
-Jeff
···
Many thanks for your help,
Jose
--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@...259...
325 Broadway Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web : Jeffrey S. Whitaker: NOAA Physical Sciences Laboratory
Jeff,
I’ve been doing some more tests today.
2009/3/3 Jeff Whitaker <jswhit@…146…>
Jose: I think the key is to only create the basemap instance only
once (for the main plot and for the inset), then re-use that
basemap instance each time you create an animation frame.
That helps in memory consumption, and that’s something I looked into. However, my basemap instances are defined (in the case of the inset) as:
m2 = Basemap(projection=‘ortho’,lon_0=6,lat_0=-12,ax=ax2)
Actually, they both have references to axes. The bit I don’t really understand is how to define my basemap only once and then just “hang it” from some axes.
Jose: You can create the Basemap instance once, and then pass the axes with the ax keyword whenever you call a basemap method.
OK, I wasn’t aware of this. However, memory consumption still flies. I am aware that it could be other bits of the program that are eating up loads of memory, but I don’t know how to test where the bottleneck is. In the end, I resorted to getting rid of basemap instances, but the problem persists. There must be something in what I’m doing that’s eating memory up, but I’m not sure how to check what it is.
I’ll try installing guppy and see whether that shines a light…
Thanks!
Jose
Jeff,
Solved, I think!
2009/3/3 Jose Gomez-Dans <jgomezdans@...287...>:
OK, I wasn't aware of this. However, memory consumption still flies. I am
aware that it could be other bits of the program that are eating up loads of
memory, but I don't know how to test where the bottleneck is. In the end, I
resorted to getting rid of basemap instances, but the problem persists.
There must be something in what I'm doing that's eating memory up, but I'm
not sure how to check what it is.
A message to the list suggested that calling pyplot.close( fig_num)
freed up the memory used, which I'm happy to report, is happening. I
still haven't managed to "cut and paste" a background into my figures,
but we'll get there... eventually!!!
Thanks!
Jose