Reusing basemap instance

Hi matplotlib basemap users,

I am doing a lot of plots of the same area but for different vertical levels,
time steps and parameters. I am therefore trying to reuse my basemap instance
(which in some cases is quite time consuming to setup). I am doing this by
making a deepcopy of a basemap instance created by this simple function
(where mapresolution is a function giving the different map resolutions for
different areas):

def getbasemap(area):
  """Returns basemap instance for a given area."""
  from matplotlib.toolkits import basemap
  mapres = mapresolution(area)
  m = basemap.Basemap(area[0], area[1], area[2], area[3], resolution=mapres)
  return m

The deepcopy operation takes almost as much time as creating a new basemap
instance. If the basemap instance was unchanged by my plotting I would of
course be able to avoid doing this and simply use a basemap instance without
copying it. Am I right in asserting that this is not the case? Any
suggestions on how to avoid deepcopying it?

Cheers,
Jesper

I forgot to mention that the reason I have to deepcopy it is that I cannot use
the basemap methods:

drawparallels()
drawmeridians()
fillcontinents()
drawcoastlines()

without an axes instance which is again tied to a figure instance. These
methods seem to modify the basemap instance (as far as I recall).

- Jesper

···

On Monday 07 May 2007 16:02, Jesper Larsen wrote:

The deepcopy operation takes almost as much time as creating a new basemap
instance. If the basemap instance was unchanged by my plotting I would of
course be able to avoid doing this and simply use a basemap instance
without copying it. Am I right in asserting that this is not the case? Any
suggestions on how to avoid deepcopying it?

Jesper Larsen wrote:

···

On Monday 07 May 2007 16:02, Jesper Larsen wrote:
  

The deepcopy operation takes almost as much time as creating a new basemap
instance. If the basemap instance was unchanged by my plotting I would of
course be able to avoid doing this and simply use a basemap instance
without copying it. Am I right in asserting that this is not the case? Any
suggestions on how to avoid deepcopying it?
    
I forgot to mention that the reason I have to deepcopy it is that I cannot use the basemap methods:

drawparallels()
drawmeridians()
fillcontinents()
drawcoastlines()

without an axes instance which is again tied to a figure instance. These methods seem to modify the basemap instance (as far as I recall).

- Jesper

Jesper: Can you be more specific about why you need a deepcopy? Those methods you mention do not modify the Basemap instance, although they do modify the axes instance they are used with. It shouldn't be a problem reusing the Basemap instance with a new axes instance (without using using deepcopy).

-Jeff

--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@...259...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

Thanks Jeff,

You are absolutely right, I have changed my code so that it does not perform a
deepcopy now. I must have done something wrong when I wrote the code some
time ago (I think, maybe I was confused by the memory issues we discussed a
while ago).

I am considering reusing an entire figure instance (which I guess I will have
to copy) as well instead of just the basemap instance but I don't know if it
is worth the effort or it is just as fast to redo the figure based on the
basemap instance. I consider doing it because the map decorations are
unchanged between the plots - a profiling of the application reveals that
they take a considerable amount of time to perform. Any recommendations on
that issue?

- Jesper

Ps. You can see a test/development version of the web application for which I
am using matplotlib/basemap in (besides for my scientific work) here (if it
is running when you read this):

http://ocean.dmi.dk/apps/forecast/

(some bugs remain)

···

On Monday 07 May 2007 16:46, Jeff Whitaker wrote:

Jesper: Can you be more specific about why you need a deepcopy? Those
methods you mention do not modify the Basemap instance, although they do
modify the axes instance they are used with. It shouldn't be a problem
reusing the Basemap instance with a new axes instance (without using
using deepcopy).

Jesper Larsen wrote:

···

On Monday 07 May 2007 16:46, Jeff Whitaker wrote:
  

Jesper: Can you be more specific about why you need a deepcopy? Those
methods you mention do not modify the Basemap instance, although they do
modify the axes instance they are used with. It shouldn't be a problem
reusing the Basemap instance with a new axes instance (without using
using deepcopy).
    
Thanks Jeff,

You are absolutely right, I have changed my code so that it does not perform a deepcopy now. I must have done something wrong when I wrote the code some time ago (I think, maybe I was confused by the memory issues we discussed a while ago).

I am considering reusing an entire figure instance (which I guess I will have to copy) as well instead of just the basemap instance but I don't know if it is worth the effort or it is just as fast to redo the figure based on the basemap instance. I consider doing it because the map decorations are unchanged between the plots - a profiling of the application reveals that they take a considerable amount of time to perform. Any recommendations on that issue?

- Jesper

Ps. You can see a test/development version of the web application for which I am using matplotlib/basemap in (besides for my scientific work) here (if it is running when you read this):

http://ocean.dmi.dk/apps/forecast/

(some bugs remain)
  

Jesper: You might consider just deleting the figure elements that you don't want to re-use (instead of figuring out everything you do want to re-use).

Very nice web site! Are the plots generated on the fly, or are they pre-generated?

-Jeff

--
Jeffrey S. Whitaker Phone : (303)497-6313
NOAA/OAR/CDC R/PSD1 FAX : (303)497-6449
325 Broadway Boulder, CO, USA 80305-3328

Jeff Whitaker wrote:

Jesper: You might consider just deleting the figure elements that you
don't want to re-use (instead of figuring out everything you do want to
re-use).

Thanks, that approach sounds more sensible than mine.

Very nice web site! Are the plots generated on the fly, or are they
pre-generated?

Thanks. The vertical profiles, time series and horizontal-vertical transect
plots are always generated on the fly. The horizontal contour maps are
generated on the fly if they do not already exist. There is a low priority
process running in the background which generates contour maps for the fields
at the sea surface (but not at depth) to reduce the time users have to wait.
This does not need to be done prior to putting the forecasts on the web but can
be done when it is already available (if someone requests a plot which has not
already been generated it will then just be generated upon the request).

Besides numpy and Matplotlib I have used the TurboGears rapid web development
megaframework (which is mainly a stack of Python technologies for web
development) and PyNIO for data access in the application.

- Jesper

I have an axes instance on which I have plotted my map decorations
(coastlines, meridians and so on). I would like to reuse that axes. On the
same axes* I would like to make filled contour plot and potentially a quiver
plot. These should be discarded after saving the plot to a file.

I know I have to discard part of the "collection" variable in the axes
instance. But it seems like other things are changed as well. Is it safe to
ignore these and simply do something like this:

nsave = len(ax.collections)

...do and save plot...

ax.collections = ax.collections[0:nsave]

or would you recommend doing something else?

Regards,
Jesper

*) I have tried to make an extra axes for the contour plot itself but it seems
like matplotlib does not create an extra axes when it is the same size as an
existing axes.

···

On Tuesday 08 May 2007 20:55, Jeff Whitaker wrote:

Jesper: You might consider just deleting the figure elements that you
don't want to re-use (instead of figuring out everything you do want to
re-use).

I reuse a Basemap instance to reduce processing time. What I do is
basically:

1. Remember all items I added to the map
2. Remove all items I don't want to keep

I use a list for every category of items (contoursets, clabels, texts,
...), as the way to remove them is slightly different.

Then I remove them from the map axes:

for contourset in contoursets_to_remove:
  for coll in contourset.collections:
    if coll in map_axes.collections:
      map_axes.collections.remove(coll)

for label in clabels_to_remove:
  if label in map_axes.artists:
   map_axes.artists.remove(label)

for txt in texts_to_remove:
  if txt in map_axes.texts:
   map_axes.texts.remove(txt)

Works well for me, but I would not be suprised if there's an more
matplotlib-like method...

While writing these lines, I start to wonder if it wouldn't be more
elegant to remember the items you want to keep (coastlines,...), then do
map_axes.cla() an then add your items to the map axes again?! For my
application, this seems to be the better way, as I'm deleting more
elements than reusing elements.

Shouldn't be too hard to rewrite my code, maybe I can tell you if it
works in a few days.

Cheers
Simon

Jesper Larsen schrieb:

···

On Tuesday 08 May 2007 20:55, Jeff Whitaker wrote:

Jesper: You might consider just deleting the figure elements that you
don't want to re-use (instead of figuring out everything you do want to
re-use).

I have an axes instance on which I have plotted my map decorations (coastlines, meridians and so on). I would like to reuse that axes. On the same axes* I would like to make filled contour plot and potentially a quiver plot. These should be discarded after saving the plot to a file.

I know I have to discard part of the "collection" variable in the axes instance. But it seems like other things are changed as well. Is it safe to ignore these and simply do something like this:

nsave = len(ax.collections)

...do and save plot...

ax.collections = ax.collections[0:nsave]

or would you recommend doing something else?

Regards,
Jesper

*) I have tried to make an extra axes for the contour plot itself but it seems like matplotlib does not create an extra axes when it is the same size as an existing axes.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Thanks, that reduced the plotting time by an additional factor two (besides
what I got from reusing the basemap instance). The remaining stuff seems hard
to do anything about:

- filling masked arrays in matplotlib ~15% of the CPU time
- writing the png file (write_png) ~15%
- drawing (non-reusable) polygon collections (draw_poly_collection) ~15%
- drawing line collections (draw_line_collection) ~7%
...

Cheers,
Jesper

···

On Thursday 10 May 2007 17:12, Simon Kammerer wrote:

I use a list for every category of items (contoursets, clabels, texts,
...), as the way to remove them is slightly different.

Then I remove them from the map axes:

for contourset in contoursets_to_remove:
  for coll in contourset.collections:
    if coll in map_axes.collections:
      map_axes.collections.remove(coll)

for label in clabels_to_remove:
  if label in map_axes.artists:
   map_axes.artists.remove(label)

for txt in texts_to_remove:
  if txt in map_axes.texts:
   map_axes.texts.remove(txt)