scipy conference

Development matplotlib-devel
1

After a two year hiatus where I inadvertently scheduled my travel
plans to overlap scipy, I will finally be able to make it to the scipy
conference this year, and plan to make up for lost time by coming
early to lead a tutorial on advanced mpl usage, stay through the
conference, and if any of you are interested, do a sprint. There are
lots of interesting things we can work on: refactoring the ticks to
work nicely with the new spines, pushing forward on the documentation,
optimizing stuff that is too slow or memory intensive, improving the
animation API and backend support, gradients, ....

Anyone interested? And if so, feel free to suggest topics or weigh in
on some I listed.

Also, if any of you will be there early for the tutorials, it would be
great to have some help from floaters, people who walk around the
room and help people who get stuck during the hands-on examples or
teachers, people who lead part of the tutorial. In particular,
Michael could do a segment on transforms and paths, JJ could do a
segment on all his fancy arrows, boxes, annotations, etc, Andrew on
his spines, Reinier on mplot3d, etc... I will probably cover all of
these even if you can't attend or don't want to teach, but it is best
ot hear from the experts. And if anyone not mentioned wants to
contribute a segment, that would be great -- just let me know what it
is. The tutorial is 2 hours and focuses on advanced mpl usage so I
want to avoid the everyday stuff and focus on transforms, paths, event
handling, animation, the newer features (spines, fancy*, mplot3d) and
everything else I am currently forgetting.

Also, we have raised a few hundred dollars in donations, so we could
either fly a worthy person out who might not otherwise be able to
attend, or pay for sprint registration for someone not getting
institutional support. Or at least provide coffee, doughnuts, pizza
and beer as fuel for participants. Fernando has also informed me
there may be some travel and conference money from other sources for
student developers so please email me us list if you are interested.

JDH

2

John Hunter wrote:

Also, we have raised a few hundred dollars in donations, so we could
either fly a worthy person out who might not otherwise be able to
attend, or pay for sprint registration for someone not getting
institutional support. Or at least provide coffee, doughnuts, pizza
and beer as fuel for participants. Fernando has also informed me
there may be some travel and conference money from other sources for
student developers so please email me us list if you are interested.
  
One small correction: sprints are free to attend. The only registration costs are for the tutorials and conference itself.

-- Dave

3

Actually, I have something I would like to discuss, but never really
could pull myself together to do it. I don't have time right now, but I
am still going to jot down the ideas.

The axes and figure paradigm inherited from matlab works well for simple
things, but when I want to more complex layouts, I actually end up
struggling with calls to axes with numerical parameters to adjust. In
addition, if I have a function that creates a plot with multiple axes,
like the figure on:
http://neuroimaging.scipy.org/site/doc/manual/html/neurospin/activation_map.html
I may want to reuse that function to create more complex figures,
stacking several of these views, with possibly other plots.

It seems to me having a level of granularity between the figure, and the
axes would help me a lot achieving these goals. I haven't had time to
hash out an API, or even solid concepts. For people who know LaTeX well,
let me draw an analogy: the figure is the page, the axis is the
character, what we are lacking in a 'minipage'. I would like a container
that can be stacked into a figure, and that can either hold axes, or
similar containers. That way, I could specify subplot, or axis relative
to this container, rather than relative to the whole figure, and it makes
it really easy for me to insert figures in larger figures.

One possible API would be 'subfigure', which would have a signature
similar to 'axes', but of course things would need to be thought a bit
more in detail: do we want clf to erase the figure and not the subfigure?
I believe so. Do we want subplot to divide the subfigure, rather than the
figure? I believe so too. How do we go back to full figure without
erasing the subfigures it contains? I think subfigure(None) might work.
How do I select a subfigure I already created? Maybe by passing it a
subfigure instance, like the way axes work.

Also, Chaco has the notion of containers, in which you can stack plots or
other containers. They have an additional feature which is that they
enable you to stack plot (these would be axes, in matplotlib terms) and
do an automatic layout of the plots. Very handy to have an extensible
canvas to display information on. Some sparse documentation:
http://code.enthought.com/projects/chaco/docs/html/api/containers.html

I have been trying to find time to think about this for more than a year,
and haven't. This is why I am sending unfinished thoughts. I do believe
more thinking has to be done, and the subfigure proposition may not hold
at all. Also, I fear I do not have time to implement this.

My 2 cents,

Gaël

···

On Wed, Jul 01, 2009 at 08:39:30AM -0500, John Hunter wrote:

Anyone interested? And if so, feel free to suggest topics or weigh in
on some I listed.

4

Gael Varoquaux wrote:

  

Anyone interested? And if so, feel free to suggest topics or weigh in
on some I listed.
    
Actually, I have something I would like to discuss, but never really
could pull myself together to do it. I don't have time right now, but I
am still going to jot down the ideas.

The axes and figure paradigm inherited from matlab works well for simple
things, but when I want to more complex layouts, I actually end up
struggling with calls to axes with numerical parameters to adjust. In
addition, if I have a function that creates a plot with multiple axes,
like the figure on:
http://neuroimaging.scipy.org/site/doc/manual/html/neurospin/activation_map.html
I may want to reuse that function to create more complex figures,
stacking several of these views, with possibly other plots.

It seems to me having a level of granularity between the figure, and the
axes would help me a lot achieving these goals. I haven't had time to
hash out an API, or even solid concepts. For people who know LaTeX well,
let me draw an analogy: the figure is the page, the axis is the
character, what we are lacking in a 'minipage'. I would like a container
that can be stacked into a figure, and that can either hold axes, or
similar containers. That way, I could specify subplot, or axis relative
to this container, rather than relative to the whole figure, and it makes
it really easy for me to insert figures in larger figures.

One possible API would be 'subfigure', which would have a signature
similar to 'axes', but of course things would need to be thought a bit
more in detail: do we want clf to erase the figure and not the subfigure?
I believe so. Do we want subplot to divide the subfigure, rather than the
figure? I believe so too. How do we go back to full figure without
erasing the subfigures it contains? I think subfigure(None) might work.
How do I select a subfigure I already created? Maybe by passing it a
subfigure instance, like the way axes work.

Also, Chaco has the notion of containers, in which you can stack plots or
other containers. They have an additional feature which is that they
enable you to stack plot (these would be axes, in matplotlib terms) and
do an automatic layout of the plots. Very handy to have an extensible
canvas to display information on. Some sparse documentation:
http://code.enthought.com/projects/chaco/docs/html/api/containers.html

I have been trying to find time to think about this for more than a year,
and haven't. This is why I am sending unfinished thoughts. I do believe
more thinking has to be done, and the subfigure proposition may not hold
at all. Also, I fear I do not have time to implement this.
  

I also have some not very fleshed out thoughts: my main feeling about
MPL is that there's just too much layout happening to keep using the
non-systematic delegation/notification "system" currently in place while
allowing the devs to maintain their sanity and day jobs. I don't mean to
disparage MPL -- it is quite a fantastic piece of code -- but there is a
lack of abstraction of layout hierarchies and layout dependencies that
makes development difficult.

Therefore, I'd suggest that before adding on too many more nice
features, we revisit the core layout and delegation system -- in the end
it will make everything much easier. So, perhaps a useful thing would be
for as many MPL devs as possible to sit together and discuss how we
could do this. My thought right now would be to investigate the use of
traits to codify the layout abstractions.

Any effort like this will also obviously benefit from having an
extensive test suite. I think all that's needed to get the tests at
http://mpl-buildbot.code.astraw.com/waterfall to pass is that someone
checks in new images made with the current MPL. I'd like to do this, but
I'm really short on time at the moment. So, please, someone -- beat me
to it -- it won't be hard!

Those are my 2 cents. Hope to see you all at SciPy 2009!

-Andrew

···

On Wed, Jul 01, 2009 at 08:39:30AM -0500, John Hunter wrote:

5

While we're dreaming big re-architecting dreams, I'll throw out an
idea related to Gael's suggestion: artist containers at the sub-axis
level. This would be a drawable / hideable container for an arbitrary
grouping of Artists that could be directly added to one (or more)
Axes. For those familiar with IDL, the IDLgrContainer in their object
graphics system is what I have in mind.

I also concur with Andrew's assessment that interactive and layout
event handling is holding back some extra fun in interactive apps. I
have mixed feelings about using Traits; in my experience with writing
(only one) app, I felt like I had to subsume everything, my data
modeling included, under the Traits paradigm, such that I was no
longer writing Python but Traits. I found it very hard to include
other data objects created by other libraries without making them
Traitified, too. This could be a knowledge gap on my part, of course.

David Beazley's course on coroutines
(:A Curious Course on Coroutines and Concurrency, see esp. Part 3) and
this talk (http://carlfk.blip.tv/file/2232349) on asynchronous vs.
threaded multitasking both have some interesting thoughts on
standard-library ways to model OS-like behaviors such as event
handling.

Thanks,
Eric

···

On Wed, Jul 1, 2009 at 9:51 PM, Andrew Straw<strawman@...36...> wrote:

Gael Varoquaux wrote:

On Wed, Jul 01, 2009 at 08:39:30AM -0500, John Hunter wrote:

Anyone interested? And if so, feel free to suggest topics or weigh in
on some I listed.

Actually, I have something I would like to discuss, but never really
could pull myself together to do it. I don't have time right now, but I
am still going to jot down the ideas.

The axes and figure paradigm inherited from matlab works well for simple
things, but when I want to more complex layouts, I actually end up
struggling with calls to axes with numerical parameters to adjust. In
addition, if I have a function that creates a plot with multiple axes,
like the figure on:
http://neuroimaging.scipy.org/site/doc/manual/html/neurospin/activation_map.html
I may want to reuse that function to create more complex figures,
stacking several of these views, with possibly other plots.

It seems to me having a level of granularity between the figure, and the
axes would help me a lot achieving these goals. I haven't had time to
hash out an API, or even solid concepts. For people who know LaTeX well,
let me draw an analogy: the figure is the page, the axis is the
character, what we are lacking in a 'minipage'. I would like a container
that can be stacked into a figure, and that can either hold axes, or
similar containers. That way, I could specify subplot, or axis relative
to this container, rather than relative to the whole figure, and it makes
it really easy for me to insert figures in larger figures.

One possible API would be 'subfigure', which would have a signature
similar to 'axes', but of course things would need to be thought a bit
more in detail: do we want clf to erase the figure and not the subfigure?
I believe so. Do we want subplot to divide the subfigure, rather than the
figure? I believe so too. How do we go back to full figure without
erasing the subfigures it contains? I think subfigure(None) might work.
How do I select a subfigure I already created? Maybe by passing it a
subfigure instance, like the way axes work.

Also, Chaco has the notion of containers, in which you can stack plots or
other containers. They have an additional feature which is that they
enable you to stack plot (these would be axes, in matplotlib terms) and
do an automatic layout of the plots. Very handy to have an extensible
canvas to display information on. Some sparse documentation:
http://code.enthought.com/projects/chaco/docs/html/api/containers.html

I have been trying to find time to think about this for more than a year,
and haven't. This is why I am sending unfinished thoughts. I do believe
more thinking has to be done, and the subfigure proposition may not hold
at all. Also, I fear I do not have time to implement this.

I also have some not very fleshed out thoughts: my main feeling about
MPL is that there's just too much layout happening to keep using the
non-systematic delegation/notification "system" currently in place while
allowing the devs to maintain their sanity and day jobs. I don't mean to
disparage MPL -- it is quite a fantastic piece of code -- but there is a
lack of abstraction of layout hierarchies and layout dependencies that
makes development difficult.

Therefore, I'd suggest that before adding on too many more nice
features, we revisit the core layout and delegation system -- in the end
it will make everything much easier. So, perhaps a useful thing would be
for as many MPL devs as possible to sit together and discuss how we
could do this. My thought right now would be to investigate the use of
traits to codify the layout abstractions.

Any effort like this will also obviously benefit from having an
extensive test suite. I think all that's needed to get the tests at
http://mpl-buildbot.code.astraw.com/waterfall to pass is that someone
checks in new images made with the current MPL. I'd like to do this, but
I'm really short on time at the moment. So, please, someone -- beat me
to it -- it won't be hard!

Those are my 2 cents. Hope to see you all at SciPy 2009!

-Andrew

6

Personally, I think traits must be kept out of MPL, for three main reasons:

1. As Eric Bruning points out, while traits is a very powerful tool,
it also closes a lot of doors by forcing everything to be written in a
trait-like fashion for it to play nice with everything else. While
this is great for many applications, I think it does not necessarily
belong everywhere, particularly given a tendancy to quickly snowball
in complexity (not to mention any new people that want to contribute
code have an extra thick manual to read).

2. Installation of traits or traitsgui should not be a necessity for
MPL... perhaps this will change in the future, but it's currently a
far bigger task to get traits and traits UI working on an arbitrary
system than it is to get MPL up and running (at least, that's been my
experience in a number of different settings).

3. Chaco - my general mindset on this is that Chaco is great for
plotting any and everything in a traits gui, but MPL is much better
for the quickcalculations and plots that I make every day as a
scientist. When a critical mass is reached it becomes better to
integrate a tool into a GUI app, MPL becomes more difficult to manage
and chaco is a better tool. But I think it makes a lot of sense to
leverage MPL most as the wonderful "quick-and-dirty" command-line
interactive plotting environment that it is rather than making it
application-centric, which is where I can envision it going as soon as
it has to run with traits.

Now a great idea would be to include optional support for things like
setting configuration files from a traits interface - I know I've seen
talk of doing this for matplotlibrc at some point, and I'd definitely
use that. Simlarly, some sort of traits-level extension to support
more interactive plots and better layout seems like it could be done
without requiring the core to use traits.

As for the question of new directions to go: One thing I would like to
see is a decent, simple 3D package in the core - nothing fancy (mayavi
is great if you want to get fancy), but just something that makes
great publication-quality figures with a quick and simple pylab-like
interface. That's the main thing I have no good response for when
people are contemplating switching to python/numpy/MPL. I also think
it might be neat to implement auto-generating function plots - that
is, plots where one of the axes is generated by a function at a
resolution that scales as you change the plot. I've seen some chaco
demos of something like this, but it's rather complicated to implement
- a function like pyplot.plotfunc(func,'--b',initialrange=(-1,1)) that
does exactly this would be a wonderful capability.

This may be more a user perspective than a "dev" perspective, but
still, there it is.

···

On Thu, Jul 2, 2009 at 10:45 AM, Eric Bruning<eric.bruning@...149...> wrote:

On Wed, Jul 1, 2009 at 9:51 PM, Andrew Straw<strawman@...36...> wrote:

Gael Varoquaux wrote:

On Wed, Jul 01, 2009 at 08:39:30AM -0500, John Hunter wrote:

Anyone interested? And if so, feel free to suggest topics or weigh in
on some I listed.

Actually, I have something I would like to discuss, but never really
could pull myself together to do it. I don't have time right now, but I
am still going to jot down the ideas.

The axes and figure paradigm inherited from matlab works well for simple
things, but when I want to more complex layouts, I actually end up
struggling with calls to axes with numerical parameters to adjust. In
addition, if I have a function that creates a plot with multiple axes,
like the figure on:
http://neuroimaging.scipy.org/site/doc/manual/html/neurospin/activation_map.html
I may want to reuse that function to create more complex figures,
stacking several of these views, with possibly other plots.

It seems to me having a level of granularity between the figure, and the
axes would help me a lot achieving these goals. I haven't had time to
hash out an API, or even solid concepts. For people who know LaTeX well,
let me draw an analogy: the figure is the page, the axis is the
character, what we are lacking in a 'minipage'. I would like a container
that can be stacked into a figure, and that can either hold axes, or
similar containers. That way, I could specify subplot, or axis relative
to this container, rather than relative to the whole figure, and it makes
it really easy for me to insert figures in larger figures.

One possible API would be 'subfigure', which would have a signature
similar to 'axes', but of course things would need to be thought a bit
more in detail: do we want clf to erase the figure and not the subfigure?
I believe so. Do we want subplot to divide the subfigure, rather than the
figure? I believe so too. How do we go back to full figure without
erasing the subfigures it contains? I think subfigure(None) might work.
How do I select a subfigure I already created? Maybe by passing it a
subfigure instance, like the way axes work.

Also, Chaco has the notion of containers, in which you can stack plots or
other containers. They have an additional feature which is that they
enable you to stack plot (these would be axes, in matplotlib terms) and
do an automatic layout of the plots. Very handy to have an extensible
canvas to display information on. Some sparse documentation:
http://code.enthought.com/projects/chaco/docs/html/api/containers.html

I have been trying to find time to think about this for more than a year,
and haven't. This is why I am sending unfinished thoughts. I do believe
more thinking has to be done, and the subfigure proposition may not hold
at all. Also, I fear I do not have time to implement this.

I also have some not very fleshed out thoughts: my main feeling about
MPL is that there's just too much layout happening to keep using the
non-systematic delegation/notification "system" currently in place while
allowing the devs to maintain their sanity and day jobs. I don't mean to
disparage MPL -- it is quite a fantastic piece of code -- but there is a
lack of abstraction of layout hierarchies and layout dependencies that
makes development difficult.

Therefore, I'd suggest that before adding on too many more nice
features, we revisit the core layout and delegation system -- in the end
it will make everything much easier. So, perhaps a useful thing would be
for as many MPL devs as possible to sit together and discuss how we
could do this. My thought right now would be to investigate the use of
traits to codify the layout abstractions.

Any effort like this will also obviously benefit from having an
extensive test suite. I think all that's needed to get the tests at
http://mpl-buildbot.code.astraw.com/waterfall to pass is that someone
checks in new images made with the current MPL. I'd like to do this, but
I'm really short on time at the moment. So, please, someone -- beat me
to it -- it won't be hard!

Those are my 2 cents. Hope to see you all at SciPy 2009!

-Andrew

While we're dreaming big re-architecting dreams, I'll throw out an
idea related to Gael's suggestion: artist containers at the sub-axis
level. This would be a drawable / hideable container for an arbitrary
grouping of Artists that could be directly added to one (or more)
Axes. For those familiar with IDL, the IDLgrContainer in their object
graphics system is what I have in mind.

I also concur with Andrew's assessment that interactive and layout
event handling is holding back some extra fun in interactive apps. I
have mixed feelings about using Traits; in my experience with writing
(only one) app, I felt like I had to subsume everything, my data
modeling included, under the Traits paradigm, such that I was no
longer writing Python but Traits. I found it very hard to include
other data objects created by other libraries without making them
Traitified, too. This could be a knowledge gap on my part, of course.

David Beazley's course on coroutines
(:A Curious Course on Coroutines and Concurrency, see esp. Part 3) and
this talk (http://carlfk.blip.tv/file/2232349) on asynchronous vs.
threaded multitasking both have some interesting thoughts on
standard-library ways to model OS-like behaviors such as event
handling.

Thanks,
Eric

------------------------------------------------------------------------------
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

7

Erik Tollerud wrote:

Personally, I think traits must be kept out of MPL, for three main reasons:
  

As these comments are in respose to my email, let me again state the main point of my email:

My main feeling about MPL is that there's just too much layout happening to keep using the non-systematic delegation/notification "system" currently in place while allowing the devs to maintain their sanity and day jobs. [...] Therefore, I'd suggest that before adding on too many more nice features, we revisit the core layout and delegation system -- in the end it will make everything much easier. So, perhaps a useful thing would be for as many MPL devs as possible to sit together and discuss how we could do this.

Thus, I was suggesting traits as a means to an end (a saner core), and not something for its own sake.

1. As Eric Bruning points out, while traits is a very powerful tool,
it also closes a lot of doors by forcing everything to be written in a
trait-like fashion for it to play nice with everything else. While
this is great for many applications, I think it does not necessarily
belong everywhere, particularly given a tendancy to quickly snowball
in complexity (not to mention any new people that want to contribute
code have an extra thick manual to read).
  

I wasn't proposing to touch the user API for MPL. I think we'd be silly to do that.

2. Installation of traits or traitsgui should not be a necessity for
MPL... perhaps this will change in the future, but it's currently a
far bigger task to get traits and traits UI working on an arbitrary
system than it is to get MPL up and running (at least, that's been my
experience in a number of different settings).
  

Well, I don't think we need traits UI, and I don't imagine traits itself is hard to build, although I could be wrong (I always use the Ubuntu/Debian packages).

3. Chaco - my general mindset on this is that Chaco is great for
plotting any and everything in a traits gui, but MPL is much better
for the quickcalculations and plots that I make every day as a
scientist. When a critical mass is reached it becomes better to
integrate a tool into a GUI app, MPL becomes more difficult to manage
and chaco is a better tool. But I think it makes a lot of sense to
leverage MPL most as the wonderful "quick-and-dirty" command-line
interactive plotting environment that it is rather than making it
application-centric, which is where I can envision it going as soon as
it has to run with traits.

Now a great idea would be to include optional support for things like
setting configuration files from a traits interface - I know I've seen
talk of doing this for matplotlibrc at some point, and I'd definitely
use that. Simlarly, some sort of traits-level extension to support
more interactive plots and better layout seems like it could be done
without requiring the core to use traits.
  

OK, but these don't address my main concern that the core layout of MPL is an unwieldy beast.

Anyhow, as I'm way too busy to think about any real coding on MPL's core right now, this is just cheap talk...

-Andrew

8

Traits itself is pretty trivial. There is one C extension that has no library dependencies like libfreetype.

···

On 2009-07-09 23:19, Andrew Straw wrote:

Erik Tollerud wrote:

2. Installation of traits or traitsgui should not be a necessity for
MPL... perhaps this will change in the future, but it's currently a
far bigger task to get traits and traits UI working on an arbitrary
system than it is to get MPL up and running (at least, that's been my
experience in a number of different settings).

Well, I don't think we need traits UI, and I don't imagine traits itself
is hard to build, although I could be wrong (I always use the
Ubuntu/Debian packages).

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

9

After a two year hiatus where I inadvertently scheduled my travel

plans to overlap scipy, I will finally be able to make it to the scipy

conference this year, and plan to make up for lost time by coming

early to lead a tutorial on advanced mpl usage, stay through the

conference, and if any of you are interested, do a sprint.

I am finally able to attend this year! I’m really looking forward to meeting everyone in person after all these years.

There are

lots of interesting things we can work on: refactoring the ticks to

work nicely with the new spines, pushing forward on the documentation,

optimizing stuff that is too slow or memory intensive, improving the

animation API and backend support, gradients, …

Anyone interested? And if so, feel free to suggest topics or weigh in

on some I listed.

There was some discussion about units a while back as well, which might be worth revisiting.

Also, if any of you will be there early for the tutorials, it would be

great to have some help from floaters, people who walk around the

room and help people who get stuck during the hands-on examples or

teachers, people who lead part of the tutorial. In particular,

Michael could do a segment on transforms and paths, JJ could do a

segment on all his fancy arrows, boxes, annotations, etc, Andrew on

his spines, Reinier on mplot3d, etc… I will probably cover all of

these even if you can’t attend or don’t want to teach, but it is best

ot hear from the experts. And if anyone not mentioned wants to

contribute a segment, that would be great – just let me know what it

is. The tutorial is 2 hours and focuses on advanced mpl usage so I

want to avoid the everyday stuff and focus on transforms, paths, event

handling, animation, the newer features (spines, fancy*, mplot3d) and

everything else I am currently forgetting.

I could possibly serve as a floater during the mpl tutorial, if you still need one.

Darren

···

On Wed, Jul 1, 2009 at 9:39 AM, John Hunter <jdh2358@…149…> wrote: