pyplot-OO convergence

One of the biggest causes of controversy in mpl, and of difficulty in teaching and learning mpl, is the divide between pyplot and the rest of the library. There are at least two aspects:

1) plt.title() versus ax.set_title(), etc; that is, all the clunky getters and setters on the OO side. JDH used to note that they were a side-effect of his C++ heritage, and that he probably wouldn't have used them if he had had more Python experience when he started mpl.

2) For interactive use, such as in the ipython console, one really wants pyplot's draw_if_interactive() functionality; one doesn't want to have to type explicit plt.draw() commands. Worse, plt.draw() operates on the current figure, which might not be the figure that one just updated with "ax2.set_title('the other plot')".

I think that both of these speed bumps can be removed fairly easily, in an entirely backwards-compatible way.

The first is just a matter of propagating some shorter-form pyplot function names back to Axes and Figure. This idea is mentioned at the end of MEP 13, as an alternative to properties.

The second requires accepting some behavior in the Axes and Figure classes that is conditional on the backend and the interactive state. I think it would look *roughly* like this:

Add a method to Figure:

def draw_if_interactive():
     if not is_interactive:
         return
     if not isinstance(self.canvas, interactive_canvases):
         return
     self.canvas.draw()

Append this method to suitable Figure methods such as suptitle().

Add a method to Axes:

def draw_if_interactive():
     self.figure.draw_if_interactive()

Append this method to suitable Axes methods--all those that execute changes, or at least those with corresponding pyplot functions.

Some additional logic (either a kwarg, or temporary manipulation of the "interactive" flag, or of an Axes instance flag) would be needed to block the drawing at intermediate stages--e.g., when boxplot is drawing all its bits and pieces.

After these changes, the pyplot functions could be simplified; they would not need their own draw_if_interactive calls.

Am I missing some major impediment? If not, I think this set of changes would make it much easier to use and teach the OO interface, with pyplot still being used where it is most helpful, such as in the subplots() call.

Eric

I am against pushing the pyplot style title/xlabel/.. function down
into the OO layer, I really do not like the different behaviour and
returns depending on the arguments. That has always struck me as a
MATLAB-ism that should be dropped, but we are stuck with to maintain
back-compatibility.

I have been thinking about going a very different route and pulling
almost all of the plotting function _off_ of the axes objects and just
having functions with signatures like

def plotter_function(ax, data1, data2, style1, style2,...)
    art = create_artists(...)
    ax.add_artists(art)
    return art_list

And then almost all of pyplot can be replaced with a wrapper function:

def wrap_for_pyplot(func):
    def inner(*args, **kwargs)
        ax = plt.gca()
        art_list = func(ax, *args, **kwargs)
        if plt.is_interactive():
             ax.figure.canvas.draw()

    inner.__name__ = func.__name__
    inner.__doc__ = strip_ax_arg(func.__doc__)
    return inner

for f in funcs_to_wrap:
    pyplot.setattr(f.__name__, wrap_for_pyplot(f))

Which pushes all of the interactive/global state related stuff up to
one place and removes the need for keywords to suppress re-drawing/the
need to manage that. This will make embedding a lot easier as well.

I have also been thinking quite a bit about the semantic
artist/manager layer of objects which I think would also go a long way
making the library easier to use, but that is a different story.

Tom

···

On Sat, Sep 27, 2014 at 7:40 PM, Eric Firing <efiring@...229...> wrote:

One of the biggest causes of controversy in mpl, and of difficulty in
teaching and learning mpl, is the divide between pyplot and the rest of
the library. There are at least two aspects:

1) plt.title() versus ax.set_title(), etc; that is, all the clunky
getters and setters on the OO side. JDH used to note that they were a
side-effect of his C++ heritage, and that he probably wouldn't have used
them if he had had more Python experience when he started mpl.

2) For interactive use, such as in the ipython console, one really wants
pyplot's draw_if_interactive() functionality; one doesn't want to have
to type explicit plt.draw() commands. Worse, plt.draw() operates on the
current figure, which might not be the figure that one just updated with
"ax2.set_title('the other plot')".

I think that both of these speed bumps can be removed fairly easily, in
an entirely backwards-compatible way.

The first is just a matter of propagating some shorter-form pyplot
function names back to Axes and Figure. This idea is mentioned at the
end of MEP 13, as an alternative to properties.

The second requires accepting some behavior in the Axes and Figure
classes that is conditional on the backend and the interactive state. I
think it would look *roughly* like this:

Add a method to Figure:

def draw_if_interactive():
     if not is_interactive:
         return
     if not isinstance(self.canvas, interactive_canvases):
         return
     self.canvas.draw()

Append this method to suitable Figure methods such as suptitle().

Add a method to Axes:

def draw_if_interactive():
     self.figure.draw_if_interactive()

Append this method to suitable Axes methods--all those that execute
changes, or at least those with corresponding pyplot functions.

Some additional logic (either a kwarg, or temporary manipulation of the
"interactive" flag, or of an Axes instance flag) would be needed to
block the drawing at intermediate stages--e.g., when boxplot is drawing
all its bits and pieces.

After these changes, the pyplot functions could be simplified; they
would not need their own draw_if_interactive calls.

Am I missing some major impediment? If not, I think this set of changes
would make it much easier to use and teach the OO interface, with pyplot
still being used where it is most helpful, such as in the subplots() call.

Eric

------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

--
Thomas Caswell
tcaswell@...149...

I am against pushing the pyplot style title/xlabel/.. function down
into the OO layer, I really do not like the different behaviour and
returns depending on the arguments. That has always struck me as a
MATLAB-ism that should be dropped, but we are stuck with to maintain
back-compatibility.

I don't understand your objection. In which cases do the returns depend on the arguments? "title" returns the title text object, regardless of whether there was an argument setting it to a new value. Same for "xlim". I haven't checked the whole list, but I expect this is the general pattern.

As for different behavior depending on arguments, what specifically do you object to, for example? I think we have all that nicely and pythonically implemented via kwargs, so you can use "xlim(xmin=3)", for example. Yes, that is different behavior than "xlim(3, 6)", but in a good way.

Maybe you are referring to the variety of signatures, such as for contour. I don't really like that either; but it is already in the OO layer, not just the pyplot layer. (The python builtin "slice()" has this characteristic, and I don't like it there, either.)

I have been thinking about going a very different route and pulling
almost all of the plotting function _off_ of the axes objects and just
having functions with signatures like

def plotter_function(ax, data1, data2, style1, style2,...)
     art = create_artists(...)
     ax.add_artists(art)
     return art_list

This has occurred to me also--I have never particularly liked having such an enormous number of Axes methods.

There is one major difference in using methods instead of plotter functions, though: it allows subclassing. Whether this is ever used in practice, I don't know.

And then almost all of pyplot can be replaced with a wrapper function:

def wrap_for_pyplot(func):
     def inner(*args, **kwargs)
         ax = plt.gca()
         art_list = func(ax, *args, **kwargs)
         if plt.is_interactive():
              ax.figure.canvas.draw()

     inner.__name__ = func.__name__
     inner.__doc__ = strip_ax_arg(func.__doc__)
     return inner

for f in funcs_to_wrap:
     pyplot.setattr(f.__name__, wrap_for_pyplot(f))

Which pushes all of the interactive/global state related stuff up to
one place and removes the need for keywords to suppress re-drawing/the
need to manage that. This will make embedding a lot easier as well.

But it does *not* take care of one of the two *big* problems I am talking about: the lack of automatic interactive plot updating when one wants to explicitly specify axes and figures in the plot call, regardless of whether this is done via plotting functions or methods.

I have also been thinking quite a bit about the semantic
artist/manager layer of objects which I think would also go a long way
making the library easier to use, but that is a different story.

I still don't really understand it, but perhaps it is orthogonal to the issues I am raising here. As far as I can see, your proposals above do not address either of the issues I raised, based on experience both in teaching matplotlib to non-programmers, and in using it day-to-day.

Regarding Matlab: it is justly popular for many reasons. It is relatively easy to learn both by design and because of its consistent high-quality documentation. Matplotlib's success has resulted in large measure from its pyplot layer, which can shield learners and users from mpl's complexity, which allows learners to build on their Matlab knowledge, and which is particularly well suited to quick interactive data exploration. The problem with the Matlab/pyplot approach is that it doesn't scale well, so we see a chorus of advice along the lines of "don't use pyplot except for subplots() and show(); use the nice, explicit OO interface for everything else". But at present, this doesn't work well, because the OO approach is not interactive enough, and using the getters and setters is clumsy when typing at the console--in demonstrating, teaching, learning, and exploring interactively, every keystroke counts!

Eric

···

On 2014/09/28, 6:20 AM, Thomas Caswell wrote:

Tom

On Sat, Sep 27, 2014 at 7:40 PM, Eric Firing <efiring@...229...> wrote:

One of the biggest causes of controversy in mpl, and of difficulty in
teaching and learning mpl, is the divide between pyplot and the rest of
the library. There are at least two aspects:

1) plt.title() versus ax.set_title(), etc; that is, all the clunky
getters and setters on the OO side. JDH used to note that they were a
side-effect of his C++ heritage, and that he probably wouldn't have used
them if he had had more Python experience when he started mpl.

2) For interactive use, such as in the ipython console, one really wants
pyplot's draw_if_interactive() functionality; one doesn't want to have
to type explicit plt.draw() commands. Worse, plt.draw() operates on the
current figure, which might not be the figure that one just updated with
"ax2.set_title('the other plot')".

I think that both of these speed bumps can be removed fairly easily, in
an entirely backwards-compatible way.

The first is just a matter of propagating some shorter-form pyplot
function names back to Axes and Figure. This idea is mentioned at the
end of MEP 13, as an alternative to properties.

The second requires accepting some behavior in the Axes and Figure
classes that is conditional on the backend and the interactive state. I
think it would look *roughly* like this:

Add a method to Figure:

def draw_if_interactive():
      if not is_interactive:
          return
      if not isinstance(self.canvas, interactive_canvases):
          return
      self.canvas.draw()

Append this method to suitable Figure methods such as suptitle().

Add a method to Axes:

def draw_if_interactive():
      self.figure.draw_if_interactive()

Append this method to suitable Axes methods--all those that execute
changes, or at least those with corresponding pyplot functions.

Some additional logic (either a kwarg, or temporary manipulation of the
"interactive" flag, or of an Axes instance flag) would be needed to
block the drawing at intermediate stages--e.g., when boxplot is drawing
all its bits and pieces.

After these changes, the pyplot functions could be simplified; they
would not need their own draw_if_interactive calls.

Am I missing some major impediment? If not, I think this set of changes
would make it much easier to use and teach the OO interface, with pyplot
still being used where it is most helpful, such as in the subplots() call.

Eric

------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

One point which is often neglected while discussing
oo- vs the pyplot-api is that today introspection tools
quite often fail to work with mpl-oo but are
perfectly fine with the pyplot module. E.g. if i am writing
some kind of helper oder plotting function taking ax, one
gets no auto-completion nor docstrings. I always have to use
a interactive console to search for them or just type in
the corresponding pyplot command for the function
signature.
Note that this is not matplotlibs fault, but
another reason while beginners may prefer the pyplot-interface:
Easier access to docstrings and available plotting functions.

This is also why i am a bit wary of using properties in matplotlib and
i don't think they are a good fit most of the time. Documenting
them is hard, discoverablity is also worse. And most setter methods in
mpl have very useful kwargs, something which is not doable with
properties.

One point which is often neglected while discussing
oo- vs the pyplot-api is that today introspection tools
quite often fail to work with mpl-oo but are
perfectly fine with the pyplot module. E.g. if i am writing
some kind of helper oder plotting function taking ax, one
gets no auto-completion nor docstrings. I always have to use
a interactive console to search for them or just type in
the corresponding pyplot command for the function
signature.

I think I know what you mean here--I do the same thing.

Note that this is not matplotlibs fault, but
another reason while beginners may prefer the pyplot-interface:
Easier access to docstrings and available plotting functions.

All of this sounds like an argument for considering a gradual move to plot functions rather than methods, as Tom is suggesting--correct? That is actually the way I implemented contour and quiver, and I have long thought that all of the moderately to very complicated plot functions, such as "plot", "hist", "boxplot", etc. should similarly be moved out of what used to be axes.py into modules containing related functionality; then it's easy to attach them as methods with a stub.

This is also why i am a bit wary of using properties in matplotlib and
i don't think they are a good fit most of the time. Documenting
them is hard, discoverablity is also worse. And most setter methods in
mpl have very useful kwargs, something which is not doable with
properties.

Good point. I think that one of the problems with the getters and setters, though, apart from clunky names, is that there is a conflict between what the name suggests--which is consistent with using them as the basis for properties--and expanding their functionality with kwargs. When functionality is expanded, a name like "set_something" probably is not the best description. That's why we have some convenience functions that can affect a variety of related parameters. Maybe we need more of those; and maybe this can be addressed using Tom's Controller idea, but I'm hazy about that.

Eric

···

On 2014/09/28, 12:42 PM, Till Stensitzki wrote:

------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

One of the biggest causes of controversy in mpl, and of difficulty in
teaching and learning mpl, is the divide between pyplot and the rest of
the library. There are at least two aspects:

1) plt.title() versus ax.set_title(), etc; that is, all the clunky
getters and setters on the OO side. JDH used to note that they were a
side-effect of his C++ heritage, and that he probably wouldn't have used
them if he had had more Python experience when he started mpl.

2) For interactive use, such as in the ipython console, one really wants
pyplot's draw_if_interactive() functionality; one doesn't want to have
to type explicit plt.draw() commands. Worse, plt.draw() operates on the
current figure, which might not be the figure that one just updated with
"ax2.set_title('the other plot')".

I'm not very familiar with matplotlib's guts, but as a user I agree
100%. I'm sure there are Reasons why 90% of pylot functions aren't
simply:

  def foo(*args, **kwargs):
      return gca().foo(*args, **kwargs)

but I have no idea what they are, so whenever I have to memorize two
different APIs for doing the same thing it ends up feeling like a
pointless waste of time.

I think that both of these speed bumps can be removed fairly easily, in
an entirely backwards-compatible way.

The first is just a matter of propagating some shorter-form pyplot
function names back to Axes and Figure. This idea is mentioned at the
end of MEP 13, as an alternative to properties.

I'd much rather write ax.xlim(...) than ax.xlim = .... The use of that
many magical properties feels unpythonic to me (too much magic!), and
pyplot.xlim() isn't going anywhere. So, it would still mean that
everyone has to learn both the pyplot and the OO APIs separately.
Learning 1 API is always going to be easier than learning two
different APIs, no matter how fancy and polished you make the second
one.

The second requires accepting some behavior in the Axes and Figure
classes that is conditional on the backend and the interactive state. I
think it would look *roughly* like this:

Add a method to Figure:

def draw_if_interactive():
     if not is_interactive:
         return
     if not isinstance(self.canvas, interactive_canvases):
         return
     self.canvas.draw()

Append this method to suitable Figure methods such as suptitle().

Add a method to Axes:

def draw_if_interactive():
     self.figure.draw_if_interactive()

Append this method to suitable Axes methods--all those that execute
changes, or at least those with corresponding pyplot functions.

In most scene-graphy frameworks, triggering redraws is simply not the
job of the mutator; it's the job of the rendering system to observe
the model for changes. (Using "observe" in the "observer pattern"
sense.) In this approach, every time an artist changes you'd call some
"note_change()" method, which would then propagate upwards. And then
some top-level piece of logic (the canvas or backend, I guess?) would
be responsible for deciding whether it wanted to respond to these
changes. So... very similar to what you're proposing, though with
different terminology :-).

Some additional logic (either a kwarg, or temporary manipulation of the
"interactive" flag, or of an Axes instance flag) would be needed to
block the drawing at intermediate stages--e.g., when boxplot is drawing
all its bits and pieces.

Presumably this would be a context manager, 'with artist.hold_changes(): ...'

···

n Sun, Sep 28, 2014 at 12:40 AM, Eric Firing <efiring@...229...> wrote:

--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

Matlab is actually slowly trying to transition to an OO-style interface of
their own. It has taken a LONG time, though.

···

On Sun, Sep 28, 2014 at 7:52 PM, Eric Firing <efiring@...229...> wrote:

Regarding Matlab: it is justly popular for many reasons. It is
relatively easy to learn both by design and because of its consistent
high-quality documentation. Matplotlib's success has resulted in large
measure from its pyplot layer, which can shield learners and users from
mpl's complexity, which allows learners to build on their Matlab
knowledge, and which is particularly well suited to quick interactive
data exploration. The problem with the Matlab/pyplot approach is that
it doesn't scale well, so we see a chorus of advice along the lines of
"don't use pyplot except for subplots() and show(); use the nice,
explicit OO interface for everything else". But at present, this
doesn't work well, because the OO approach is not interactive enough,
and using the getters and setters is clumsy when typing at the
console--in demonstrating, teaching, learning, and exploring
interactively, every keystroke counts!