Suggestion: accept callables as non-xs values arguments

Hi,

Not sure if this is the best place to post this, but I have an API
suggestion: as well as iterables, mpl plotting functions could accept
callable objects to generate y values, errors, etc..

Specifically, I'm thinking that the list of x values would always need
to be specified as an iterable like now, but that the other arguments
which currently require an iterable of the same length as the xs could
alternatively accept callables that are mapped on to the xs list. The
presence of __iter__ or __call__ attributes could be used to determine
which approach to use.

I searched a bit and couldn't find any previous mention of this idea,
and there are callable arguments already used in matplotlib. Thoughts?
I'd be happy to help with implementation if there is any interest.

Best wishes,
Andy

Andy,

This is exactly the right place to post things like this.

Fernando Perez has proposed something similar to me in person. As part of
the label data un-packing discussion there was talk of allowing users to
write expressions in (ex plot('x', 'sqrt(x)', data=df) would plot a square
root function) which is sort of related to this.

What follows is a rather rambling brain dump of me thinking about this :wink:

Do you want the functions to be evaluated at Artist creation time or Artist
draw time?

If it is at creation time, see
https://github.com/matplotlib/matplotlib/pull/4829 for how we implemented
something similar for unpacking labeled data. That might be a reasonable
template for how to do this. Would tooling to easily bind plotting
functions to the computation be good enough? For controlling any
additional args/kwargs to the mapping function we can just use `partial`

If it is at _draw_ time, I am a bit more skeptical. But if you can change
the x data, the rest of the data _should_ recompute so that would probably
be a better way to do it long term, but will require much more invasive
changes to the library.

We already have a fair number of mapping functions internally (ex, the
whole transform stack and the data -> norm -> cmap chain), I wonder if
there is a way to generalize any of that?

I am also a bit concerned that this will balloon quickly, for example, for
people plotting histograms, they may want the yerr to be a function of the
y-value (ex Poisson error), not the x-value. The complexity required to
express everything that would need to be done once we start to go down this
road is a bit daunting.

I have been thinking for a while (like a year) about a massive API
re-working (which I owe the list a long write up on!), I have heard you and
will think about how to integrate this.

One other concern is that our top-level API is _too_ magical rather than
not magical enough.

Tom

···

On Thu, Sep 17, 2015 at 4:39 PM Andy Buckley <andy.buckley at cern.ch> wrote:

Hi,

Not sure if this is the best place to post this, but I have an API
suggestion: as well as iterables, mpl plotting functions could accept
callable objects to generate y values, errors, etc..

Specifically, I'm thinking that the list of x values would always need
to be specified as an iterable like now, but that the other arguments
which currently require an iterable of the same length as the xs could
alternatively accept callables that are mapped on to the xs list. The
presence of __iter__ or __call__ attributes could be used to determine
which approach to use.

I searched a bit and couldn't find any previous mention of this idea,
and there are callable arguments already used in matplotlib. Thoughts?
I'd be happy to help with implementation if there is any interest.

Best wishes,
Andy
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
Matplotlib-devel Info Page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20150917/e7c05d14/attachment.html&gt;

Andy,

This is exactly the right place to post things like this.

Phew :slight_smile:

Fernando Perez has proposed something similar to me in person. As part
of the label data un-packing discussion there was talk of allowing users
to write expressions in (ex plot('x', 'sqrt(x)', data=df) would plot a
square root function) which is sort of related to this.

That looks like all sorts of more dangerous to me -- it would presumably
involve an exec() or equivalent, with security implications. But sure,
str funcdef args could be added on top of callables, by the users if not
mpl itself!

What follows is a rather rambling brain dump of me thinking about this :wink:

Do you want the functions to be evaluated at Artist creation time or
Artist draw time?

If it is at creation time, see
ENH: plotting methods can unpack labeled data by jankatins · Pull Request #4829 · matplotlib/matplotlib · GitHub for how we
implemented something similar for unpacking labeled data. That might be
a reasonable template for how to do this. Would tooling to easily bind
plotting functions to the computation be good enough? For controlling
any additional args/kwargs to the mapping function we can just use `partial`

If it is at _draw_ time, I am a bit more skeptical. But if you can
change the x data, the rest of the data _should_ recompute so that would
probably be a better way to do it long term, but will require much more
invasive changes to the library.

Heh, I hadn't really been thinking in that sort of detail. I don't
really use mpl interactively, and would I *expect* values to update
dynamically? Not based on the current behaviour at least, which by
construction always requires an explicit recalculation. But I'd defer to
interactive users whether (re)draw-time calculation would add enough to
be worth the extra trouble. If it's much easier, I guess creation time
is the place to start.

We already have a fair number of mapping functions internally (ex, the
whole transform stack and the data -> norm -> cmap chain), I wonder if
there is a way to generalize any of that?

Beyond my mpl expertise, I'm afraid!

I am also a bit concerned that this will balloon quickly, for example,
for people plotting histograms, they may want the yerr to be a function
of the y-value (ex Poisson error), not the x-value. The complexity
required to express everything that would need to be done once we start
to go down this road is a bit daunting.

Yes, this occurred to me, too.

Maybe it's not ambitious enough, but I would be very happy with just
parameterising everything on x.

But, thinking aloud, a more advanced possibility would be that a
ys=callable arg would operate on just x values, an xs=callable arg would
operate on just ys, and any other callables would operate on a 2-arg
pair (x,y). Obviously only one of xs and ys would be able to be callable.

I would hold off generalising beyond that -- YAGNI!

Thanks for the quick & positive reply :slight_smile:

Andy

···

On 17/09/15 22:46, Thomas Caswell wrote:

I have been thinking for a while (like a year) about a massive API
re-working (which I owe the list a long write up on!), I have heard you
and will think about how to integrate this.

One other concern is that our top-level API is _too_ magical rather than
not magical enough.

Tom

On Thu, Sep 17, 2015 at 4:39 PM Andy Buckley <andy.buckley at cern.ch > <mailto:andy.buckley at cern.ch>> wrote:

    Hi,

    Not sure if this is the best place to post this, but I have an API
    suggestion: as well as iterables, mpl plotting functions could accept
    callable objects to generate y values, errors, etc..

    Specifically, I'm thinking that the list of x values would always need
    to be specified as an iterable like now, but that the other arguments
    which currently require an iterable of the same length as the xs could
    alternatively accept callables that are mapped on to the xs list. The
    presence of __iter__ or __call__ attributes could be used to determine
    which approach to use.

    I searched a bit and couldn't find any previous mention of this idea,
    and there are callable arguments already used in matplotlib. Thoughts?
    I'd be happy to help with implementation if there is any interest.

    Best wishes,
    Andy
    _______________________________________________
    Matplotlib-devel mailing list
    Matplotlib-devel at python.org <mailto:Matplotlib-devel at python.org>
    Matplotlib-devel Info Page

--
Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Expt Group, University of Glasgow

I think sympy has plotting functions that basically work this way, right?
(Though they might only work with sympy function and variable objects, and
not generic callables and arrays). Might be worth looking into what they
have done, as they've probably wrestled with some of these API problems
along the way.

···

On Fri, Sep 18, 2015 at 9:01 AM, Andy Buckley <andy.buckley at cern.ch> wrote:

On 17/09/15 22:46, Thomas Caswell wrote:

Andy,

This is exactly the right place to post things like this.

Phew :slight_smile:

Fernando Perez has proposed something similar to me in person. As part

of the label data un-packing discussion there was talk of allowing users
to write expressions in (ex plot('x', 'sqrt(x)', data=df) would plot a
square root function) which is sort of related to this.

That looks like all sorts of more dangerous to me -- it would presumably
involve an exec() or equivalent, with security implications. But sure, str
funcdef args could be added on top of callables, by the users if not mpl
itself!

What follows is a rather rambling brain dump of me thinking about this :wink:

Do you want the functions to be evaluated at Artist creation time or
Artist draw time?

If it is at creation time, see
ENH: plotting methods can unpack labeled data by jankatins · Pull Request #4829 · matplotlib/matplotlib · GitHub for how we
implemented something similar for unpacking labeled data. That might be
a reasonable template for how to do this. Would tooling to easily bind
plotting functions to the computation be good enough? For controlling
any additional args/kwargs to the mapping function we can just use
`partial`

If it is at _draw_ time, I am a bit more skeptical. But if you can
change the x data, the rest of the data _should_ recompute so that would
probably be a better way to do it long term, but will require much more
invasive changes to the library.

Heh, I hadn't really been thinking in that sort of detail. I don't really
use mpl interactively, and would I *expect* values to update dynamically?
Not based on the current behaviour at least, which by construction always
requires an explicit recalculation. But I'd defer to interactive users
whether (re)draw-time calculation would add enough to be worth the extra
trouble. If it's much easier, I guess creation time is the place to start.

We already have a fair number of mapping functions internally (ex, the

whole transform stack and the data -> norm -> cmap chain), I wonder if
there is a way to generalize any of that?

Beyond my mpl expertise, I'm afraid!

I am also a bit concerned that this will balloon quickly, for example,

for people plotting histograms, they may want the yerr to be a function
of the y-value (ex Poisson error), not the x-value. The complexity
required to express everything that would need to be done once we start
to go down this road is a bit daunting.

Yes, this occurred to me, too.

Maybe it's not ambitious enough, but I would be very happy with just
parameterising everything on x.

But, thinking aloud, a more advanced possibility would be that a
ys=callable arg would operate on just x values, an xs=callable arg would
operate on just ys, and any other callables would operate on a 2-arg pair
(x,y). Obviously only one of xs and ys would be able to be callable.

I would hold off generalising beyond that -- YAGNI!

Thanks for the quick & positive reply :slight_smile:

Andy

I have been thinking for a while (like a year) about a massive API

re-working (which I owe the list a long write up on!), I have heard you
and will think about how to integrate this.

One other concern is that our top-level API is _too_ magical rather than
not magical enough.

Tom

On Thu, Sep 17, 2015 at 4:39 PM Andy Buckley <andy.buckley at cern.ch >> <mailto:andy.buckley at cern.ch>> wrote:

    Hi,

    Not sure if this is the best place to post this, but I have an API
    suggestion: as well as iterables, mpl plotting functions could accept
    callable objects to generate y values, errors, etc..

    Specifically, I'm thinking that the list of x values would always need
    to be specified as an iterable like now, but that the other arguments
    which currently require an iterable of the same length as the xs could
    alternatively accept callables that are mapped on to the xs list. The
    presence of __iter__ or __call__ attributes could be used to determine
    which approach to use.

    I searched a bit and couldn't find any previous mention of this idea,
    and there are callable arguments already used in matplotlib. Thoughts?
    I'd be happy to help with implementation if there is any interest.

    Best wishes,
    Andy
    _______________________________________________
    Matplotlib-devel mailing list
    Matplotlib-devel at python.org <mailto:Matplotlib-devel at python.org>
    Matplotlib-devel Info Page

--
Dr Andy Buckley, Lecturer / Royal Society University Research Fellow
Particle Physics Expt Group, University of Glasgow

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
Matplotlib-devel Info Page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20150918/1433d470/attachment-0001.html&gt;