[matplotlib-devel] Unifying numpy, scipy, and matplotlib docstring formats

Keir Mierle wrote:

I do not want to descend into a huge, time wasting discussion of this which is not productive.

Well, it's been said before, and this isn't really the place for it, but it's quite critical to your project. Don't let any of this get in the way of improving docs strings, however!

Note that I am not suggesting a recursive import of submodules.

What you appeared to support was that all of the matplotlib, numpy and scipy base namespaces be merged, and they should all be merged into the main namespace.

import matplotlib as plot
import numpy as N
import Scipy.whatever as whatever.

UGH. See, this is my issue. When I read someone else's code, they always chose a different convention.

Good point, that is a bit of a pain. It would certainly be a good idea to standardize at least these three, and have them be imported by default in your environment (though I'm not sure about scipy -- numpy and matplotlib would be good)

If we define the official way to use pylab as 'from pylab import *', then these problems vanish.

and others arise. One is that you then have to make sure you don't' get any name clashes you don't want, so you end up with arange, so it won't clash with range. etc.

Note that we must be *very* careful to export only exactly the names which should be exported;

Then how do you get the others? "import *" and "import pylab"?

Because it is always the case
that I use numpy and pylab together.

I use numpy every day without pylab. And I use pylab occasionally without numpy, it accepts regular old lists for quick hacking.

This is a key point -- if all anyone does is use your mega-pylab, then you may be right, but let's not cripple people. Let them start using PyLab for matlab-like quickie coding, then decide to write a real app, and be able to start using wxPython without learning a bunch of new stuff, and having namespaces clash.

Other non-core modules should be treated as usual, where it is at the author's discretion for how to import them.

Everyone's idea of non-core is different

Even if you insist on joining numpy and plotting namespaces (they are both too big at the moment, if you ask me), please tell people to import it as:

import pylab

I think a consensus is building in the python community that you should NEVER use import *!

some history:

wxPython used to be commonly used as:

from wxpython import *

And the names were all: wxSomeName.

A few years back, the names were all changed to remove the wx, and we now all do:

import wx
this = wx.SomeName(...)

Numeric was designed to be used with "import *". Now many of the old Numeric functions are available as numpy methods, and more and more people are using some variation of:

import numpy as N

and fewer are using "import *"

Does anyone else have any other examples?

One more reason: more IDEs are providing auto-completion and module browsers. Smaller, more hierarchical namespaces are much better for this. I know if I'm looking for a number crunching or a plotting function -- make it easier to find them.

I think Scott Meyer, a C++ luminary, said it best [1] when this heinous
fragment of his code was posted in comp.lang.c++.moderated

namespaces are a new add-on to C++ -- it will be a good while before they are used right there!

This is only for the core functionality. The scipy/numpy/matplotlib core API becomes similar in importance to Python's __builtins__ for the PyLab
environment.

>>> len(dir(__builtins__))
129

len(dir(pylab))

432

and __builtins__ is too big as it is. -- 31 of those are Exceptions, they should have their own namespace, if you ask me. Would it be that much harder to type

except Errors.Type:

than

except TypeError:

I argue that Python's __builtins__ should be equivalent to PyLab's from pylab
import *, and that e.g. import sys corresponds to import linalg.

But then you have __builtins__ and pylab in the same namespace!

Perhaps non-interactively; when using the system interactively the MATLAB
interface is by far the best way to go. If someone proposes an oo interface
which is as fast to type and as easy to understand as the MATLAB interface
(i.e. to demo to my friends who came over to see what I'm talking about when I
say that PyLab is great) then I'm all ears.

Look for my (and other) posts about this for more detail, but a few points:

1) Don't break long-term productivity/useability so that the quicky demos are more impressive. Python's real strength over tools like Matlab shows up when projects get bigger.

2) There is nothing about an OO interface that is inherently harder to use, or even more typing, except perhaps a few extra dots.

3) There is some work to be done to bring the matplotlib OO interface up to its potential for interactive use, particularly the docs!

Note that this discussion is early! I am waaaay not here yet; first step is to fix the docstrings.

Yes, enough said for now -- and I really appreciate your efforts to clean up the docstrings.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...259...

Hi,

I think a consensus is building in the python community that you should
NEVER use import *!

Well, I have only been coding python for a few years, but I would say,
along with writing unit tests, the great importance of not using
import * is one of the secrets that you learn slowly and painfully
with experience. Chris' point about the movement of big projects away
from that idiom is a very good one. It is convenient, but over time
you realize that the value of convenience is far outweighed by the
namespace mess and loss of clarity that results.

Best,

Matthew

Perhaps we should consider two use cases: interactive use ala Matlab
and larger code bases. In the first case, being able to import * saves
a lot of typing and the namespace polution problem isn't a big deal.
The second use, obviously, benefits from avoiding import *.

Returning to the OP's questions, why couldn't both cases be helped by
creating a "meta-package" for numpy, scipy, and matplotlib? For the
sake of argument, lets call the package "plab". Existing code could be
affected by changing the individual packages, but a package that
essentially does

from pylab import *
from numpy import *
from scipy import *

would give a standard API that future code and interactive use could
use. Code could do
import plab

plab.plot() #etc.

and interactive use could do from plab import *.

Just a thought...

Barry

···

On 2/16/07, Matthew Brett <matthew.brett@...287...> wrote:

Hi,

> I think a consensus is building in the python community that you should
> NEVER use import *!

Well, I have only been coding python for a few years, but I would say,
along with writing unit tests, the great importance of not using
import * is one of the secrets that you learn slowly and painfully
with experience. Chris' point about the movement of big projects away
from that idiom is a very good one. It is convenient, but over time
you realize that the value of convenience is far outweighed by the
namespace mess and loss of clarity that results.

Best,

Matthew

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Hi,

import plab

plab.plot() #etc.

and interactive use could do from plab import *.

Yes... It's a hard call of course. I am a long term matlab user, and
switched to python relatively recently. I do see the attraction of
persuading people that you can get something very similar to matlab
easily. The downside about making numpy / python like matlab is that
you soon realize that you really have to think about your problems
differently, and write code in a different way. I know that's
obvious, but the variables as pointers, mutable / immutable types,
zero based indexing, arrays vs matrices are all (fruitful) stumbling
blocks. Then there is the very large change of thinking in an OO way,
pulling in other large packages for doing other tasks, writing
well-structured code with tests - all the features that python gives
you for an industrial strength code base. And, the more pylab looks
like matlab, the more surprised and confused people will be when they
switch. So, I would argue that getting as close to matlab as
possible should not be the unqualified goal here - it is a real
change, with real pain, but great benefits.

Best,

Matthew

Matt,

Yes, I agree. I wasn't coming at so much from the goal of making Pylab
a Matlab clone (as you point out, that's silly, and misses much of the
advantage of Python), but rather from the goal of making interactive
use as efficient as possible. When I fire up ipython -pylab to do some
quick exploration, it's nice not to have to type N.blah or pylab.plot
etc. If I just import pylab *, however, then the commands I use may
not be what I expect from more formal coding where I use N.blah numpy,
S.foo for scipy, and pylab.bar for matplotlib. Making it easy for
users to have either namespace strategy, with consistent bindings, ala
the start of this thread is a good idea, IMO.

Well, I've said my piece. I'll get out of the way and let others have a crack...

Barry

···

On 2/18/07, Matthew Brett <matthew.brett@...287...> wrote:

Hi,

> import plab
>
> plab.plot() #etc.
>
> and interactive use could do from plab import *.

Yes... It's a hard call of course. I am a long term matlab user, and
switched to python relatively recently. I do see the attraction of
persuading people that you can get something very similar to matlab
easily. The downside about making numpy / python like matlab is that
you soon realize that you really have to think about your problems
differently, and write code in a different way. I know that's
obvious, but the variables as pointers, mutable / immutable types,
zero based indexing, arrays vs matrices are all (fruitful) stumbling
blocks. Then there is the very large change of thinking in an OO way,
pulling in other large packages for doing other tasks, writing
well-structured code with tests - all the features that python gives
you for an industrial strength code base. And, the more pylab looks
like matlab, the more surprised and confused people will be when they
switch. So, I would argue that getting as close to matlab as
possible should not be the unqualified goal here - it is a real
change, with real pain, but great benefits.

Best,

Matthew

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

There's probably a better forum for this conversation, but...

Barry Wark wrote:

Perhaps we should consider two use cases: interactive use ala Matlab
and larger code bases.

A couple key points -- yes, interactive use is different than larger code bases, but I think it's a "Bad Idea" to promite totally different coding styles for these cases for a couple reasons:

-- One usually is doing both at once. I was a long-time, every day Matlab user, and hardly did anything of consequence interactively. I learned very quickly that it made a whole lot more sense to write a five line script that I could save, edit, etc. than do stuff interactively. Once I got something working, parts of that five line script might get cut&pasted into "real" code.

I do still test one or two lines interactively, but even then, I want the style to be something I can put in my code.

2) consistency in docs and examples is important, recommending different styles for interactive and programming use is just going to confuse people more.

3) even for folks that do a lot of interactive use, they are likely to write larger scale code at some point, and then they would need to learn something new.

In the first case, being able to import * saves
a lot of typing

No, it saves a little typing, if you're using an OOO style anyway.

> and the namespace polution problem isn't a big deal.

Yes, it can be. A good interactive environment will be able to do things like method and command completion -- namespace pollution keeps that from working well.

Returning to the OP's questions, why couldn't both cases be helped by
creating a "meta-package" for numpy, scipy, and matplotlib? For the
sake of argument, lets call the package "plab". Existing code could be
affected by changing the individual packages, but a package that
essentially does

from pylab import *
from numpy import *
from scipy import *

The issue with this is that you've now hidden where things are coming from. People seeing examples using that package will have no idea where things come from.

and by the way, the current "pylab", as delivered with MPL, pretty much does this already. I think we need to move away from that, rather than putting even more into pylab.

Matthew Brett wrote:

The downside about making numpy / python like matlab is that
you soon realize that you really have to think about your problems
differently, and write code in a different way.

Good point. A part of good Pythonic code is namespaces and OOO style. New users might as well learn the whole pile at once.

That all being said, it would be nice to establish a standard convention for how to import the key packages. I use:

import numpy as N
import matplotlib as MPL

But I don't really care that much, if we can come to any kind of community consensus, I'll follow it. The goal would be for all docs, Wiki entries, examples on the mailing lists, etc. to use the same style.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...259...