mlab: please check

mlab.py in svn has now been changed quite a bit:

1) It uses straight numpy.
2) In many but not all cases, I tried to simplify algorithms by using numpy capabilities. I did some testing, but beware: there may be new bugs.
3) For functions with numpy equivalents I put in deprecation warnings, with the intention that mlab be stripped down to things that are useful and are not readily available in numpy.
4) I added some comments to docstrings.
5) I added a "demean" function that is like detrend_mean but takes an axis argument. This ability to operate on a specified axis might be added to other functions in mlab.

Please test the new version. Especially if there are functions that you contributed to mlab (Fernando, I think you provided many of them), please check to see whether I have fouled them up, or deprecated them when this should not have been done, or whether you think they should still be included at all.

This version is not intended as an endpoint but as a step in the evolution of mpl and pylab.

We should come up with a clear policy as to what belongs in mlab.py and what does not. As it settles down, the module docstring can be made more complete and useful.

Eric

Unfortunately I'm getting ready to leave for scipy'07, so I won't be
able to look at this.

But if I may make a small comment, in my view MPL should simply not
contain any numerics, period (beyond that needed for plotting
functions). I realize that it's convenient to have a layer of
numerical functionality directly in there, since you control the
library and that lets you have in there what you want, when you want
it, rather than waiting for numpy to catch up. IPython used to do the
same (IPython.numutils, which is where that stuff in mlab came from),
but I've completely stopped. IPython also had plotting
(IPython.Gnuplot*), and I also stopped that, now all IPython has is
the bare minimum needed to support MPL (-pylab), but no actual
plotting facilities of its own.

In the days of Numeric/numarray/numerix, I think mlab made a lot of
sense. But I would hope that now, any truly useful numerical
contributions should go into numpy (there's no confusion anymore), so
that we have a clear separation of responsibilities:

- numpy: base numerics
- scipy: bigger algorithms, some domain-specific functionality
- scikits: extensions to scipy, most of the domain-specific
functionality, staging area for scipy candidates.
- mpl: plotting
- ipython: interactive work (and distributed computing)

It used to be that scipy had plotting and interactive stuff, mpl had
numerics and interactive code (the little standalone GTK-threaded
interpreter), ipython had numerics and plotting, etc. We all had a
bit of each other's domain.

Moving forward, I think we should all make an effort to avoid that.
Scipy by and large has shed all of its old plotting and interactive
support, IPython carries it but I haven't touched that code in years
(and it will just stay there, it won't move to the new branches).

In that regard, I think in the long run having MPL carry a numerical
layer confuses its purpose. Unless you (a collective you, the MPL
developers) decide that MPL is really meant to be a complete numerical
system, but then you should change the description from the website,
which currently reads:

"matplotlib is a python 2D plotting library which produces publication
quality figures in a variety of hardcopy formats and interactive
environments across platforms."

I realize that this requires more work, since it means asking the
numpy team for inclusion of things you might like to have and waiting
for that to happen, rather than just doing it yourselves. That
coordination takes time, obviously, and that's a scarce resource for
all of us. But I honestly think that now that the
Numeric/Numarray/numerix mess is finally behind us, this should be an
opportunity for completing the 'separation of responsibilities'
effort. In the long run, I think it will help new users coming to
Python to better understand what is what, it will make for a
stronger/better Numpy, etc.

I know this isn't an easy problem, because you are balancing
matlab-compatibility, backwards compatibility, ease of development
workflow, etc. In the end, I trust your judgment, I just wanted to
mention this point of view in case you find it feasible.

Cheers,

f

···

On 8/12/07, Eric Firing <efiring@...229...> wrote:

mlab.py in svn has now been changed quite a bit:

1) It uses straight numpy.
2) In many but not all cases, I tried to simplify algorithms by using
numpy capabilities. I did some testing, but beware: there may be new bugs.
3) For functions with numpy equivalents I put in deprecation warnings,
with the intention that mlab be stripped down to things that are useful
and are not readily available in numpy.
4) I added some comments to docstrings.
5) I added a "demean" function that is like detrend_mean but takes an
axis argument. This ability to operate on a specified axis might be
added to other functions in mlab.

Please test the new version. Especially if there are functions that you
contributed to mlab (Fernando, I think you provided many of them),

Fernando Perez wrote:

mlab.py in svn has now been changed quite a bit:

But if I may make a small comment, in my view MPL should simply not
contain any numerics, period (beyond that needed for plotting
functions).

For the most part,m I agree with Fernando on this. However, the numpy team is making a concerted effort to have numpy be pretty straight core functionality. All the stuff in mlab will not and should not be put into numpy.

However, it's a lot of useful stuff. What to do?

I think it probably belongs in SciPy -- after all, SciPy is a lot of useful stuff (some domain-specific, some not) built on top of numpy.

However, mlab is now in MPL, and I think Eric is doing a great job of cleaning things up, so that the mlab module it self could easily be moved somewhere else in the future.

A couple points to keep in mind:

Does internal MPL code rely on mlab? if so, it needs to be part of some core lib that is stable enough to be a MPL dependency.

Despite what Fernando and I think, a lot of people DO want MPL to be an all-in-one plotting AND computing environment -- like Matlab. I think Scipy is what should be the complete environment, and I think it's getting close, but there are all still issues, so MPL does fill a need.

-Chris

···

On 8/12/07, Eric Firing <efiring@...229...> wrote:

Christopher Barker wrote:
[...]

Does internal MPL code rely on mlab? if so, it needs to be part of some core lib that is stable enough to be a MPL dependency.

Good question. With a couple of minor exceptions (which are easily handled), the only dependency is in axes methods that I don't think should be axes methods: xcorr, psd, etc. These are not the sort of core plot types that belong at the heart of mpl; they should be out on the periphery, in something like an optional toolkit. They are more analysis than plotting. I wanted to do a reorganization along these lines a long time ago, but never got around to it. The question of what should be an axes method has been raised in the context of John's mpl1 major rewrite ideas, but an incremental change might be feasible and worthwhile also. How much user code is relying on Axes.psd() etc? Probably not too much, and code modification required if these things were made functions that take an axes instance as the first argument would not be large. The methods could be moved out of Axes and converted to functions without affecting the corresponding pylab API.

Eric