Question on sphinxext.plot_directive...

Development matplotlib-devel
1

Hi folks,

if one were to say, think of writing something like a book (or a
paper) using sphinx and plots generated from python scripts, the plot
directive would be extremely useful. But as best as I can tell, it
generates at the end of the day 'image' directives, where as for
including figures in latex-produced PDF with captions and labels one
can later refer to, the plain sphinx 'figure' directive appears to be
more appropriate. As we can read here:

http://docutils.sourceforge.net/docs/ref/rst/directives.html#images

Their signatures are:

Image
Directive Type: "image"
Doctree Element: image
Directive Arguments: One, required (image URI).
Directive Options: Possible.
Directive Content: None.

Figure
Directive Type: "figure"
Doctree Elements: figure, image, caption, legend
Directive Arguments: One, required (image URI).
Directive Options: Possible.
Directive Content: Interpreted as the figure caption and an optional legend.

A key difference is that image takes no content, while figure accepts
content and uses it for the figure caption.

Would it be possible/sensible to switch the plot directive to be a
superset of 'figure' instead of 'image'?

Before I dive into the code too far, I figured I'd ask the experts.

Thanks!

f

2

Too late for that, common sense has never been my forte.

Here's a diff against current trunk to play with this idea.

WARNING: Please note that this is NOT meant to be applied for mpl yet!!!

I've actually modified the plot directive and renamed it to 'figplot',
so we can experiment a little to better understand things. The point
is that this now lets us write reST of this type:

.. figplot:: code/make_figure_brainx.py
   :width: 3.6 in

   **Example from fMRI data:** In the graph presented, the nodes represent the
   areas of the brain described in Figure 1. Nodes are labeled accordingly and
   etc...

The text block is then passed as a caption to latex.

I renamed it because there's a bit of a conflict with the current
'plot' directive, which allows a filename *or* a content block, but in
that case the content block is meant to be the source code, as
illustrated in sampledoc:

http://matplotlib.sourceforge.net/sampledoc/extensions.html#inserting-matplotlib-plots

Since I'm not sure if we can find a clean solution to:

- path to script: goes into arg list
- inlined (multiline) code: goes into content block
- inlined (possibly multiline) caption: goes into content block

I put this version so we can start experimenting. This does what
Ariel and I need, but I hope over time we can figure out a good
long-term solution.

Speaking of sphinx for books, as I've mentioned before to John, the
last big problem is being able to cross-reference arbitrary text
elements like you can in latex, be they chapters or sections or
whatever, and get a number or something that's meaningful in print.

I looked around, and apparently it's on the main docutils todo list:

http://docutils.sourceforge.net/docs/dev/todo.html#object-numbering-and-object-references

I hope we don't have to be the ones fixing that one...

Cheers,

f

plot_directive.diff (3.79 KB)

···

On Sat, Sep 12, 2009 at 8:29 PM, Fernando Perez <fperez.net@...149...> wrote:

Before I dive into the code too far, I figured I'd ask the experts.

3

Updated patch that handles correctly more than one option (I think the
bug was even in the original, not sure).

Cheers,

f

plot_directive.diff (4.01 KB)

···

On Sat, Sep 12, 2009 at 11:12 PM, Fernando Perez <fperez.net@...149...> wrote:

Here's a diff against current trunk to play with this idea.

4

Hi Fernando and all,

Speaking of sphinx for books, as I've mentioned before to John, the
last big problem is being able to cross-reference arbitrary text
elements like you can in latex, be they chapters or sections or
whatever, and get a number or something that's meaningful in print.

I looked around, and apparently it's on the main docutils todo list:

Docutils To Do List

I hope we don't have to be the ones fixing that one...

I should also mention bibliography - using a bibtex bibliography is
also still quite impossible at the moment, without going in and
editing the .tex and running the tex (latex=>bibtex=>latex=>latex)
commands on it yourself. Does anyone of the Sphinx mavens around here
know anything about how to get Sphinx to do that for you? I am
starting to believe that Sphinx can be made to do just about anything
we would want it to for these kinds of uses.

Cheers,

Ariel

5

Fernando Perez wrote:

I renamed it because there's a bit of a conflict with the current
'plot' directive, which allows a filename *or* a content block, but in
that case the content block is meant to be the source code, as
illustrated in sampledoc:

Sphinx extensions for embedded plots, math and more — sampledoc 1.0 documentation

Since I'm not sure if we can find a clean solution to:

- path to script: goes into arg list
- inlined (multiline) code: goes into content block
- inlined (possibly multiline) caption: goes into content block
  

I'm not sure it's that bad. It's certainly possible to do all these things with a single directive, since providing a path or providing source code is mutually exclusive. The thing one can't do is provide inline source *and* a caption. I applied your patch, renaming the directive back to "plot" and ran it over the matplotlib docs, and it doesn't seem to break anything.

Of course, we can also provide two directives, "plot" and "figplot" based on essentially the same source code. I'm kind of neutral on the matter.

Mike

···

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

6

Fernando Perez wrote:

I renamed it because there's a bit of a conflict with the current
'plot' directive, which allows a filename *or* a content block, but in
that case the content block is meant to be the source code, as
illustrated in sampledoc:

Sphinx extensions for embedded plots, math and more — sampledoc 1.0 documentation

Since I'm not sure if we can find a clean solution to:

- path to script: goes into arg list
- inlined (multiline) code: goes into content block
- inlined (possibly multiline) caption: goes into content block
  

I'm not sure it's that bad. It's certainly possible to do all these things with a single directive, since providing a path or providing source code is mutually exclusive. The thing one can't do is provide inline source *and* a caption. I applied your patch, renaming the directive back to "plot" and ran it over the matplotlib docs, and it doesn't seem to break anything.

Of course, we can also provide two directives, "plot" and "figplot" based on essentially the same source code. I'm kind of neutral on the matter.

Mike

···

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

7

Hi Michael,

I'm not sure it's that bad. It's certainly possible to do all these things
with a single directive, since providing a path or providing source code is
mutually exclusive. The thing one can't do is provide inline source *and* a
caption. I applied your patch, renaming the directive back to "plot" and
ran it over the matplotlib docs, and it doesn't seem to break anything.

Well, if you're OK with the api-by-inputs approach (name+text =
caption, text only=code), then I have no qualms using it. It felt a
bit hackish so I was perhaps overly cautious, but I'd much rather:

- have this in upstream mpl than in my own projects
- have a single directive to remember

So many thanks for integrating it!

Of course, we can also provide two directives, "plot" and "figplot" based on
essentially the same source code. I'm kind of neutral on the matter.

I'm pretty neutral too, though perhaps it might be worth thinking
about this a little more, to get a really good long-term solution.

Basically, what I'd like to have is a directive that can cover these
two scenarios:

1. Like today, plot with code:

.. plot::
  plot(x,y)

2. An extended version of today's support for files, that can handle
entry points:

.. plot:: script.py func1

  Caption for figure 1

.. plot:: script.py func2

  Caption for figure 2

This would *import* script only once, and then for each figure it
would call the given function (argumentless, to keep things simple for
now). Basically each call would be the equivalent of

import script; script.funcN() # N=1,2

This would make it easy to compute expensive data in script once, and
then render multiple plots out of it without re-executing the script
in full each time.

Or is this already possible with today's plot()?

I think this would cover all the use cases I have in mind, from years
of using a similar approach but with Makefiles.

Cheers,

f

···

On Mon, Sep 14, 2009 at 6:57 AM, Michael Droettboom <mdroe@...31...> wrote:

8

Fernando Perez wrote:

Hi Michael,

I'm not sure it's that bad. It's certainly possible to do all these things
with a single directive, since providing a path or providing source code is
mutually exclusive. The thing one can't do is provide inline source *and* a
caption. I applied your patch, renaming the directive back to "plot" and
ran it over the matplotlib docs, and it doesn't seem to break anything.
    
Well, if you're OK with the api-by-inputs approach (name+text =
caption, text only=code), then I have no qualms using it. It felt a
bit hackish so I was perhaps overly cautious, but I'd much rather:

- have this in upstream mpl than in my own projects
- have a single directive to remember

So many thanks for integrating it!
  

Yeah, I have the same "not quite right" feeling about it. What about putting the caption in an "option" such as:

  .. plot:: foo.py
     :caption: This is my caption

I don't know if the caption can have newlines in this mode, though. I will have to experiment.

Of course, we can also provide two directives, "plot" and "figplot" based on
essentially the same source code. I'm kind of neutral on the matter.
    
I'm pretty neutral too, though perhaps it might be worth thinking
about this a little more, to get a really good long-term solution.
  2. An extended version of today's support for files, that can handle
entry points:

.. plot:: script.py func1

  Caption for figure 1

.. plot:: script.py func2

  Caption for figure 2

This would *import* script only once, and then for each figure it
would call the given function (argumentless, to keep things simple for
now). Basically each call would be the equivalent of

import script; script.funcN() # N=1,2

This would make it easy to compute expensive data in script once, and
then render multiple plots out of it without re-executing the script
in full each time.

Or is this already possible with today's plot()?
  

It's not possible now. The tricky part of this is that currently we don't keep the module around after it's done plotting. This is actually very much on purpose since keeping all that data around for all of the example plots in the matplotlib documentation would quickly consume a lot of memory. Perhaps when used in this mode (with a function argument) we could. Alas, again that's api-by-inputs :slight_smile:

Mike

···

On Mon, Sep 14, 2009 at 6:57 AM, Michael Droettboom <mdroe@...31...> wrote:

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

9

Yeah, I have the same "not quite right" feeling about it. What about
putting the caption in an "option" such as:

.. plot:: foo.py
:caption: This is my caption

I don't know if the caption can have newlines in this mode, though. I will
have to experiment.

Not possible, as far as I could tell. This was my first reflex and
all of my experiments ended losing the other lines, and I couldn't
find by searching the lists whether there was any way to do it. But I
admit I didn't go as far as asking on the docutils or sphinx lists.

It's not possible now. The tricky part of this is that currently we don't
keep the module around after it's done plotting. This is actually very much
on purpose since keeping all that data around for all of the example plots
in the matplotlib documentation would quickly consume a lot of memory.
Perhaps when used in this mode (with a function argument) we could. Alas,
again that's api-by-inputs :slight_smile:

Well, even if the module isn't kept in memory, from an api perspective
I think it would still be useful to have this

.. plot:: script.py func

mode. It would make it easier to organize scripts for plot generation
in papers.

But don't worry too much about it. The real problem was having
figures with captions, and at least we have that done. Hopefully some
time in the future these extensions can be cleaned up a little bit
(the code is indeed a bit messy).

In the meantime, we have something very useful that we need *today*,
so I'm happy :slight_smile:

Cheers,

f

···

On Tue, Sep 15, 2009 at 7:57 AM, Michael Droettboom <mdroe@...31...> wrote: