What would you like to see in a book about Matplotlib?

Hello and Happy 2009!

I received the interesting proposal to author a book on Matplotlib,
the powerful 2D plotting library for Python.

While preparing the arguments list, I'd like to hear even your
opinion, because different points-of-view will lead to a better
product.

Some basic question I'd like to ask are:

- what are you using matplotlib for?
- what are the things you like the most of matplotlib, that you want
to give emphasis to? And why?
- what are the (basic) things that, when you were beginning to use
matplotlib, you wanted to see grouped up but couldn't find?
- what would you like to see in a book about matplotlib?
- what are some those advanced feature that made you yell "WOW!!" ?
- what are the things you'd like to explore of matplotlib and never
had time to do?

Your suggestions are really appreciated :slight_smile: And wish me good luck!

Cheers,

···

--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

Hi Sandro,

It's great news that a book may come out on MPL.

Speaking as an aspiring university professor in neuroscience, I would
like to see something that could be used as a resource for undergraduate
students just learning Python and MPL. Due to this perspective, I think
such a book would cover both numpy and MPL. The emphasis could clearly
by on MPL, but basic numpy idioms and concepts should also be taught. I
think an example-driven approach would be very useful -- something like
an undergraduate laboratory experiment where students measure and plot
raw values and compute histograms and statistics (e.g. mean and std).
More advanced sections might perform statistical comparisons of
different treatments (e.g. using chi squared, KS and/or T tests), do
linear least squares fitting (with np.linalg.lstsq), and possibly
non-linear curve fitting using something like scipy.optimize.fmin.

That's my self-serving $0.02, since you asked! :slight_smile:

You may also want to speak with John Hunter and Fernando Perez about the
possibility of collaborating -- they've already done some work towards a
book, too.

Andrew

Sandro Tosi wrote:

···

Hello and Happy 2009!

I received the interesting proposal to author a book on Matplotlib,
the powerful 2D plotting library for Python.

While preparing the arguments list, I'd like to hear even your
opinion, because different points-of-view will lead to a better
product.

Some basic question I'd like to ask are:

- what are you using matplotlib for?
- what are the things you like the most of matplotlib, that you want
to give emphasis to? And why?
- what are the (basic) things that, when you were beginning to use
matplotlib, you wanted to see grouped up but couldn't find?
- what would you like to see in a book about matplotlib?
- what are some those advanced feature that made you yell "WOW!!" ?
- what are the things you'd like to explore of matplotlib and never
had time to do?

Your suggestions are really appreciated :slight_smile: And wish me good luck!

Cheers,

Hello Andrew,
thanks for taking the time to reply.

First of all, let me clarify that I received a proposal (and not the
opposite) so some decision were already made about the book format.

Hi Sandro,

It's great news that a book may come out on MPL.

Speaking as an aspiring university professor in neuroscience, I would
like to see something that could be used as a resource for undergraduate
students just learning Python and MPL.

The cut of the book is for already experienced Python programmers. For
sure, in the "approaching" chapter to mpl, I won't go too much into
deep of python programming, and I suppose they'll be easy to read even
for new comers.

Due to this perspective, I think
such a book would cover both numpy and MPL. The emphasis could clearly
by on MPL, but basic numpy idioms and concepts should also be taught.

Mh, the focus the editor would like to see is about introducing mpl +
integrating into apps. The examples I have in mind for the pratical
part of apps integration already cointains some sort of "introduction"
to gather data and plot them (but I can't go to much on the scientific
side :slight_smile: ). I will propose a chapter like "Matplotlib for the science"
and here your suggestion on what you'd like to see there is welcome.

I
think an example-driven approach would be very useful -- something like
an undergraduate laboratory experiment where students measure and plot
raw values and compute histograms and statistics (e.g. mean and std).
More advanced sections might perform statistical comparisons of
different treatments (e.g. using chi squared, KS and/or T tests), do
linear least squares fitting (with np.linalg.lstsq), and possibly
non-linear curve fitting using something like scipy.optimize.fmin.

That could be a beginning of that chapter contents: more more more ideas :slight_smile:

That's my self-serving $0.02, since you asked! :slight_smile:

And they are very much welcome!!

You may also want to speak with John Hunter and Fernando Perez about the
possibility of collaborating -- they've already done some work towards a
book, too.

I'm sure be happy to hear their voice on this product, but (given the
preamble) I don't know if the editor would be fine with co-authorship;
I'll ask, just to be sure.

Cheers,

···

On Mon, Jan 5, 2009 at 20:00, Andrew Straw <strawman@...36...> wrote:
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

Note: Posted to matplotlib-devel and debian-science.

Sandro,
       Firstly, good luck with the book.

The sort of book I'd buy would explain how to use the combination of
matplotlib/ipython/scipy/numpy to analyse data.

- what are you using matplotlib for?

I want to use matplotlib/ipython/numpy/scipy for analysis of
experimental data - plotting and fitting models to it. Also perhaps
simulation of the data.

I have also wanted to use matplotlib to plot data as it was acquired -
see below.

I've not really used matplotlib in anger - but am likely to do so in
the future (and it would have been useful during my PhD had it been
around then).

- what are the things you like the most of matplotlib, that you want
to give emphasis to? And why?

Quality plots. The ability to add TeX labels.

I've been keeping an eye on matplotlib for several years - it looks
good. I really must spend some time exploring it.

- what are the (basic) things that, when you were beginning to use
matplotlib, you wanted to see grouped up but couldn't find?
- what would you like to see in a book about matplotlib?

Start off by reading data from a file, plotting it and fitting a
function to that data.

Often, several scans are in the same data file. An elegant solution to
reading data something like this example would be useful.

# Scan: 1
# Time: 18:00
# Temperature: 21
# t data
1 12
2 33
3 14
4 40
5 60

# Scan: 2
# Time: 18:02
# Temperature: 30
# t data
1 22
2 33
3 44
4 55

And so on.

Fitting a function to several data sets - with some of the parameters
fitted to both sets of data and some not would be useful.

- what are some those advanced feature that made you yell "WOW!!" ?
- what are the things you'd like to explore of matplotlib and never
had time to do?

Plotting with related scales

···

----------------------------

Sometimes it is useful to plot related scales on x1 and x2 axes. I've
come across this several times in different contexts. In its simplest
form, there is a linear relationship between the axes. In a mechanical test, you might want extension on the x1 axis and strain on the x2 axis (for example).

Sometimes there is not a linear relationship. For example you might
want to plot frequency (or photon energy) on x1 and wavelength on x2.

An even more complex example is a Hall-Petch plot:

(Yield Stress) = k/sqrt(Grain Size)

So plotting 1/Sqrt(Grain Size) on the X1 axis gives a linear
plot, but it would be useful to plot the grain size on the X2 scale.

ipython and emacs
-----------------

Suppose I want to write a script to analyse some data (perhaps I want
a record of what I've done, or perhaps I'd like to perform the same
analysis on several data sets). I'd probably do so in emacs - but it
is useful to do some experimentation in ipython - tab completion is
particularly useful. I feel there must be a good way to do my
experimentation in ipython and save the important bits in emacs - but
I've not sat down and worked out an efficient way of doing this.

Data aqcuisition and experimental control:
-----------------------------------------

Writing a simple application to acquire data - ideally from multiple
sources and plot the data as it is acquired. In my case I wanted to
combine mechanical with electrical tests. A couple of interesting
articles by G Varoquaux are listed at
http://wiki.debian.org/DebianScience/DataAcquisition

This is perhaps beyond the scope of the book, but it has come up on
the mailing lists a couple of times. The ideal application would have
a gui for simple use, but a command line (probably ipython) for more
more complex use - perhaps performing a series of tests under
different conditions.

Some discussion of plotting non gridded 2d data should also be in
there.

Your suggestions are really appreciated :slight_smile: And wish me good luck!

I don't think it is the thrust of your book, but another book I was
looking for is "A cookbook of Numerical simulations of classic
physics/engineering problems". For use by physicists/engineers who
don't want to rewrite things from scratch.

Good luck.

Chris

Hi Chris,
thanks for your reply, helpful as usual :slight_smile:

      Firstly, good luck with the book.

cheers :slight_smile:

The sort of book I'd buy would explain how to use the combination of
matplotlib/ipython/scipy/numpy to analyse data.

Sadly, that would not the book I'll write :frowning: The editor wanted to
target another audience for the book: experienced python developers,
with no knowledge of matplotlib; so an introductionary book, that will
show even how to integrate mpl on GTK/WX application and on the web.

I pushed to have something about science, and a chapter will be about
that, but I need your (all) inputs, because my science days are long
back in the past :wink:

- what are the (basic) things that, when you were beginning to use
matplotlib, you wanted to see grouped up but couldn't find?
- what would you like to see in a book about matplotlib?

Start off by reading data from a file, plotting it and fitting a
function to that data.

That sounds something that could land in the "science" chapter.

Plotting with related scales
----------------------------

Sometimes it is useful to plot related scales on x1 and x2 axes. I've
come across this several times in different contexts. In its simplest
form, there is a linear relationship between the axes. In a mechanical test, you might want extension on the x1 axis and strain on the x2 axis (for example).

Sometimes there is not a linear relationship. For example you might
want to plot frequency (or photon energy) on x1 and wavelength on x2.

An even more complex example is a Hall-Petch plot:

(Yield Stress) = k/sqrt(Grain Size)

So plotting 1/Sqrt(Grain Size) on the X1 axis gives a linear
plot, but it would be useful to plot the grain size on the X2 scale.

Err, I think I lost you :wink:

What you want is 2 plots on the same figure? so not 2 Ys for the same
X (let's say X is time, and Y1 is stock price variation, and Y2 is the
percentage change), you want X1-Y1 (let's say on the bottom-left) and
X2-Y2 (on the upper-right): did I get you?

ipython and emacs
-----------------

Suppose I want to write a script to analyse some data (perhaps I want
a record of what I've done, or perhaps I'd like to perform the same
analysis on several data sets). I'd probably do so in emacs - but it
is useful to do some experimentation in ipython - tab completion is
particularly useful. I feel there must be a good way to do my
experimentation in ipython and save the important bits in emacs - but
I've not sat down and worked out an efficient way of doing this.

I think the preferred way to do so it using ipython, and for now I
plan only to show it on the book.

Data aqcuisition and experimental control:
-----------------------------------------

Writing a simple application to acquire data - ideally from multiple
sources and plot the data as it is acquired. In my case I wanted to
combine mechanical with electrical tests. A couple of interesting
articles by G Varoquaux are listed at
http://wiki.debian.org/DebianScience/DataAcquisition

This is perhaps beyond the scope of the book, but it has come up on
the mailing lists a couple of times. The ideal application would have
a gui for simple use, but a command line (probably ipython) for more
more complex use - perhaps performing a series of tests under
different conditions.

I thought about an example for this already! :slight_smile: I thought to develop a
sample application for GTK/WX that display some system value (like cpu
usage or so, in this way everyone can run the example) plotting the
information as it comes (for 30 secs, for example).

Some discussion of plotting non gridded 2d data should also be in
there.

for example?

Your suggestions are really appreciated :slight_smile: And wish me good luck!

I don't think it is the thrust of your book, but another book I was
looking for is "A cookbook of Numerical simulations of classic
physics/engineering problems". For use by physicists/engineers who
don't want to rewrite things from scratch.

As said, even if my degree is in linear algebra, my science days are
gone, so it won't be in the book, if not for that chapter about
science and mpl.

Good luck.

Cheers,

···

On Fri, Jan 30, 2009 at 18:59, Chris Walker <chrisw@...235...> wrote:
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

Hi Chris,
thanks for your reply, helpful as usual :slight_smile:

> Firstly, good luck with the book.

cheers :slight_smile:

> The sort of book I'd buy would explain how to use the combination of
> matplotlib/ipython/scipy/numpy to analyse data.

Sadly, that would not the book I'll write :frowning: The editor wanted to
target another audience for the book: experienced python developers,
with no knowledge of matplotlib; so an introductionary book, that will
show even how to integrate mpl on GTK/WX application and on the web.

I pushed to have something about science, and a chapter will be about
that, but I need your (all) inputs, because my science days are long
back in the past :wink:

Sure - though anyone wanting to use matplotlib is likely to be
acquiring, manipulating and then plotting data.

>> - what are the (basic) things that, when you were beginning to use
>> matplotlib, you wanted to see grouped up but couldn't find?
>> - what would you like to see in a book about matplotlib?
>
> Start off by reading data from a file, plotting it and fitting a
> function to that data.

That sounds something that could land in the "science" chapter.

Indeed.

> Plotting with related scales
> ----------------------------
>
> Sometimes it is useful to plot related scales on x1 and x2 axes. I've
> come across this several times in different contexts. In its simplest
> form, there is a linear relationship between the axes. In a mechanical test, you might want extension on the x1 axis and strain on the x2 axis (for example).
>
> Sometimes there is not a linear relationship. For example you might
> want to plot frequency (or photon energy) on x1 and wavelength on x2.
>
> An even more complex example is a Hall-Petch plot:
>
> (Yield Stress) = k/sqrt(Grain Size)
>
> So plotting 1/Sqrt(Grain Size) on the X1 axis gives a linear
> plot, but it would be useful to plot the grain size on the X2 scale.

Err, I think I lost you :wink:

Figure 3b/3c at
http://dcwww.camd.dtu.dk/~schiotz/papers/risoesymp/html/node3.html
is an example - note that the y2 scale is not linear.

What you want is 2 plots on the same figure? so not 2 Ys for the same
X

2 scales on the same figure, yes.

(let's say X is time, and Y1 is stock price variation, and Y2 is the
percentage change), you want X1-Y1 (let's say on the bottom-left) and
X2-Y2 (on the upper-right): did I get you?

Exactly. http://en.wikipedia.org/wiki/File:Body_mass_index_chart.svg
is the sort of thing I had in mind.

> ipython and emacs
> -----------------
>
> Suppose I want to write a script to analyse some data (perhaps I want
> a record of what I've done, or perhaps I'd like to perform the same
> analysis on several data sets). I'd probably do so in emacs - but it
> is useful to do some experimentation in ipython - tab completion is
> particularly useful. I feel there must be a good way to do my
> experimentation in ipython and save the important bits in emacs - but
> I've not sat down and worked out an efficient way of doing this.

I think the preferred way to do so it using ipython, and for now I
plan only to show it on the book.

Whether or not this make it into the book, I'm interested in how
people do this. Surely you don't write your application using just
ipython do you?

> Data aqcuisition and experimental control:
> -----------------------------------------
>
> Writing a simple application to acquire data - ideally from multiple
> sources and plot the data as it is acquired. In my case I wanted to
> combine mechanical with electrical tests. A couple of interesting
> articles by G Varoquaux are listed at
> http://wiki.debian.org/DebianScience/DataAcquisition
>
> This is perhaps beyond the scope of the book, but it has come up on
> the mailing lists a couple of times. The ideal application would have
> a gui for simple use, but a command line (probably ipython) for more
> more complex use - perhaps performing a series of tests under
> different conditions.

I thought about an example for this already! :slight_smile:

Excellent.

I thought to develop a
sample application for GTK/WX that display some system value (like cpu
usage or so, in this way everyone can run the example) plotting the
information as it comes (for 30 secs, for example).

One of the things I liked about Gael's article was its discussion of
threading - separating the gui from the calculations from the data
acquisition.

> Some discussion of plotting non gridded 2d data should also be in
> there.

for example?

Something like:
http://matplotlib.sourceforge.net/examples/pylab_examples/griddata_demo.html

It is something I have encountered several times.

Another example I had in mind is:

http://www.sci.muni.cz/~mikulik/gallery.html#GaAlAsFishAtPM
http://www.sci.muni.cz/~mikulik/gnuplot.html

I've taken similar measurements in the past - and one often takes
measurements equally spaced in real space which results in them not
being equally spaced in reciprocal space.

>> Your suggestions are really appreciated :slight_smile: And wish me good luck!
>
> I don't think it is the thrust of your book, but another book I was
> looking for is "A cookbook of Numerical simulations of classic
> physics/engineering problems". For use by physicists/engineers who
> don't want to rewrite things from scratch.

As said, even if my degree is in linear algebra, my science days are
gone, so it won't be in the book, if not for that chapter about
science and mpl.

It is worth a book in itself.

> Good luck.

Another thing that springs to mind is to mention the
distributed/parallel abilities of the latest version of
ipython[1]. Given the thrust of your book, then perhaps it won't make
it in though.

Chris

[1] Unfortunately ipython 0.9.1 won't make Debian lenny - scheduled to
be released real soon now - but will perhaps be out there when your
book is published.

···

On Sun, Feb 01, 2009 at 11:59:06PM +0100, Sandro Tosi wrote:

On Fri, Jan 30, 2009 at 18:59, Chris Walker > <chrisw@...235...> wrote:

Thanks. Be aware that this is a rats nest, though, as threading is the
best way to reveal all the subtleties of an event loop, and the various
race conditions that you can have with it. The strong model/view
separation that is implicit in Traits allows to hide the code making the
view thread-safe in the Traits code updating implicitly the view. You can
build these constraints in your multi-threaded application (I believe I
touch a couple of words on this in my tutorial), but you have to be aware
of the problems and the good patterns to answer them.

To sum up, I am not saying this is uninteresting, on the contrary, I am
just saying that such text is hard to write (which makes a good text even
more interesting).

My 2 cents,

Gaël

···

On Mon, Feb 02, 2009 at 03:47:32PM +0000, Chris Walker wrote:

One of the things I liked about Gael's article was its discussion of
threading - separating the gui from the calculations from the data
acquisition.