pickled plots?

Hi all,

A thought just occurred to me: I wonder if it would be useful to be able
to 'pickle' Matplotlib plots, using the python cPickle library. This
way, I could save my plots in a form that would allow me to load them
back later (with just the necessary source data) and fiddle with things
like titles and legends and so on. Would be useful perhaps in the
context of preparing diagrams for an article or report.

Has anyone tried this? Would it be recommended/not recommended/not even
possible?

Cheers
JP

Hi all,

A thought just occurred to me: I wonder if it would be useful to be able
to 'pickle' Matplotlib plots, using the python cPickle library. This
way, I could save my plots in a form that would allow me to load them
back later (with just the necessary source data) and fiddle with things
like titles and legends and so on. Would be useful perhaps in the
context of preparing diagrams for an article or report.

Has anyone tried this? Would it be recommended/not recommended/not even
possible?

I had a look at this a while back. It looks like the well thought out
structure of MPL should make this easy, although it would require a few
adjustments. To make a Figure object pickle-able, all the internal
objects in a Figure must also be pickle-able. Most of the innards of a
Figure are python objects which should pickle without problem. The only
parts which aren't are the "BinOps". These are custom C-coded objects
which implement 'lazy evaluation' of the transformation expressions.
They're defined in the _transforms.cxx/h files.

In theory, you can easily make these C-objects pickle-able using the
'copy_reg' module; you just register two functions, one to extracts the
object's state as a pickle-able object, the other to construct a new
instance of the object, initialised with the previously stored state.

However, I ran into a problem: there's some bug in either python or CXX
(the automatic C++ class wrapper which mpl uses for the compiled
components) which results in a segfault when I tried pickling copy_reg
enhanced BinOps. The templating techniques used by CXX are completely
beyond me so this is where things have stuck.

... but ... I just now tested this again with python-2.4.2 and
mpl-0.87.2 and it works! yeay. Thus, if every object in
matplotlib._transforms gets given a python reduction/construction
function pair and registers them with copy_reg, this *should* be enough
to make a Figure pickle-able. Unless I've missed something else...

I may try this out later this week, unless someone else tries it first.

Bryan

PS. copy_reg example follows >>>

import cPickle as pickle
import copy_reg

#let's test this on a simple 'Value' BinOp
from matplotlib._transforms import Value

def fcon(val):
    #constructor
    return Value(val)

def fred(o):
    #reduction functions
    val = o.get()
    return fcon, (val,)

#my starting object
a=Value(5)
print a, a.get()

copy_reg.pickle(type(a), fred)

data = pickle.dumps(a)

new = pickle.loads(data)
print "new", new, new.get()

···

On Sun, 2006-06-18 at 00:05 +1000, John Pye wrote:

Cheers
JP

FWIW I found that I was able to pickle C++ objects but simply adding
python methods __reduce__ and __setstate__ in my SWIG .i file -- I'm not
sure if Matplotlib uses this approach or not. I didn't need to use
copy_reg (perhaps it's preferable? I don't know)

(this is from the steam properties project that I run,
http://freesteam.sf.net/)

Cheers
JP

Bryan Cole wrote:

···

On Sun, 2006-06-18 at 00:05 +1000, John Pye wrote:

Hi all,

A thought just occurred to me: I wonder if it would be useful to be able
to 'pickle' Matplotlib plots, using the python cPickle library. This
way, I could save my plots in a form that would allow me to load them
back later (with just the necessary source data) and fiddle with things
like titles and legends and so on. Would be useful perhaps in the
context of preparing diagrams for an article or report.

Has anyone tried this? Would it be recommended/not recommended/not even
possible?
   
I had a look at this a while back. It looks like the well thought out
structure of MPL should make this easy, although it would require a few
adjustments. To make a Figure object pickle-able, all the internal
objects in a Figure must also be pickle-able. Most of the innards of a
Figure are python objects which should pickle without problem. The only
parts which aren't are the "BinOps". These are custom C-coded objects
which implement 'lazy evaluation' of the transformation expressions.
They're defined in the _transforms.cxx/h files.

In theory, you can easily make these C-objects pickle-able using the
'copy_reg' module; you just register two functions, one to extracts the
object's state as a pickle-able object, the other to construct a new
instance of the object, initialised with the previously stored state.

However, I ran into a problem: there's some bug in either python or CXX
(the automatic C++ class wrapper which mpl uses for the compiled
components) which results in a segfault when I tried pickling copy_reg
enhanced BinOps. The templating techniques used by CXX are completely
beyond me so this is where things have stuck.

... but ... I just now tested this again with python-2.4.2 and
mpl-0.87.2 and it works! yeay. Thus, if every object in
matplotlib._transforms gets given a python reduction/construction
function pair and registers them with copy_reg, this *should* be enough
to make a Figure pickle-able. Unless I've missed something else...

I may try this out later this week, unless someone else tries it first.

Bryan

PS. copy_reg example follows >>>

import cPickle as pickle
import copy_reg

#let's test this on a simple 'Value' BinOp
from matplotlib._transforms import Value

def fcon(val):
   #constructor
   return Value(val)

def fred(o):
   #reduction functions
   val = o.get()
   return fcon, (val,)

#my starting object
a=Value(5)
print a, a.get()

copy_reg.pickle(type(a), fred)

data = pickle.dumps(a)

new = pickle.loads(data)
print "new", new, new.get()

Cheers
JP
   
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

FWIW I found that I was able to pickle C++ objects but simply adding
python methods __reduce__ and __setstate__ in my SWIG .i file -- I'm not
sure if Matplotlib uses this approach or not. I didn't need to use
copy_reg (perhaps it's preferable? I don't know)

MPL uses CXX instead of SWIG; I'm no C++ export so I havn't looked at
adding __get/setstate__ functions to the objects themselves. copy_reg is
nice because you can add pickle-ability without modifying any matplotlib
code. I've hit another potential problem though: it's not possible to
access the full internal state of these objects from their python API,
meaning they can't be stored this way. It looks like I need to
understand the Figure creation process in a bit more detail.

BC

···

On Sun, 2006-06-18 at 15:38 +1000, John Pye wrote:

CVS Info for project freesteam
(this is from the steam properties project that I run,
http://freesteam.sf.net/)

Cheers
JP

Bryan Cole wrote:

>On Sun, 2006-06-18 at 00:05 +1000, John Pye wrote:
>
>
>>Hi all,
>>
>>A thought just occurred to me: I wonder if it would be useful to be able
>>to 'pickle' Matplotlib plots, using the python cPickle library. This
>>way, I could save my plots in a form that would allow me to load them
>>back later (with just the necessary source data) and fiddle with things
>>like titles and legends and so on. Would be useful perhaps in the
>>context of preparing diagrams for an article or report.
>>
>>Has anyone tried this? Would it be recommended/not recommended/not even
>>possible?
>>
>>
>
>I had a look at this a while back. It looks like the well thought out
>structure of MPL should make this easy, although it would require a few
>adjustments. To make a Figure object pickle-able, all the internal
>objects in a Figure must also be pickle-able. Most of the innards of a
>Figure are python objects which should pickle without problem. The only
>parts which aren't are the "BinOps". These are custom C-coded objects
>which implement 'lazy evaluation' of the transformation expressions.
>They're defined in the _transforms.cxx/h files.
>
>In theory, you can easily make these C-objects pickle-able using the
>'copy_reg' module; you just register two functions, one to extracts the
>object's state as a pickle-able object, the other to construct a new
>instance of the object, initialised with the previously stored state.
>
>However, I ran into a problem: there's some bug in either python or CXX
>(the automatic C++ class wrapper which mpl uses for the compiled
>components) which results in a segfault when I tried pickling copy_reg
>enhanced BinOps. The templating techniques used by CXX are completely
>beyond me so this is where things have stuck.
>
>... but ... I just now tested this again with python-2.4.2 and
>mpl-0.87.2 and it works! yeay. Thus, if every object in
>matplotlib._transforms gets given a python reduction/construction
>function pair and registers them with copy_reg, this *should* be enough
>to make a Figure pickle-able. Unless I've missed something else...
>
>I may try this out later this week, unless someone else tries it first.
>
>Bryan
>
>PS. copy_reg example follows >>>
>
>import cPickle as pickle
>import copy_reg
>
>#let's test this on a simple 'Value' BinOp
>from matplotlib._transforms import Value
>
>def fcon(val):
> #constructor
> return Value(val)
>
>def fred(o):
> #reduction functions
> val = o.get()
> return fcon, (val,)
>
>#my starting object
>a=Value(5)
>print a, a.get()
>
>copy_reg.pickle(type(a), fred)
>
>data = pickle.dumps(a)
>
>new = pickle.loads(data)
>print "new", new, new.get()
>
>
>
>>Cheers
>>JP
>>
>>
>
>
>
>
>_______________________________________________
>Matplotlib-users mailing list
>Matplotlib-users@lists.sourceforge.net
>matplotlib-users List Signup and Options
>
>
>