[Matplotlib-users] check pylab before upcoming release

Development matplotlib-devel
#1

John,

In your list, "load" is duplicated.

It would be nice if numpy.rank could be deprecated, since it seems to be an alias for ndim (which is more descriptive and less ambiguous). We could avoid the name clash by using "matrixrank" for the mlab version.

I suggest that for every name that conflicts with a numpy name, the docstring should make this explicit.

On a more strategic note: what do you see as the future of mlab, and its place in pylab? Should mlab contain every neat function we can think of, and if so, should all of these be exposed in the pylab namespace? Do we have or need any quality-control standards for these functions? Is there a point in exposing more of mlab now than we have in the past? Probably so, but we might want to be conservative about it.

I am copying this to the devel list; it seems worth wider exposure.

Eric

John Hunter wrote:

···

On Nov 9, 2007 12:09 AM, Eric Firing <efiring@...229...> wrote:

from pyplot import *
from matplotlib.mlab import *

for mlab I would be inclined to use an explicit list for the sake of
documentation and clarity, if nothing else.

One of my colleagues using svn just got bitten because she was using
pylab load and save, which have slightly different semantics than
numpy's. In particular, we have an optional converter dictionary. It
would be nice to reconcile these (Travis has already borrowed a fair
amount from mlab which is why so many funcs are deprecated).

Here is my candidate list, which I will commit shortly but let me know
if you have a problem with any of it:

from matplotlib.mlab import load, window_hanning, window_none, conv,
detrend, demean, \
     detrend_mean, detrend_none, detrend_linear, entropy, normpdf, levypdf, \
     find, longest_contiguous_ones, longest_ones, prepca, prctile,
prctile_rank, \
     center_matrix, rk4, bivariate_normal, get_xyz_where,
get_sparse_matrix, dist, \
     dist_point_to_segment, segments_intersect, fftsurr, liaupunov, movavg, \
     save, load, slopes, stineman_interp, inside_poly, poly_below,
poly_between, exp_safe, \
     amap, rms_flat, l1norm, l2norm, norm_flat, frange,
diagonal_matrix, identity, \
     base_repr, binary_repr, log2, ispower2, fromfunction_kw, rem,
norm, orth, rank, sqrtm,\
     mfuncC, approx_real, rec_append_field, rec_drop_fields, rec_join,
csv2rec, rec2csv

#2

In your list, "load" is duplicated.

Thanks, purged.

On a more strategic note: what do you see as the future of mlab, and its
place in pylab? Should mlab contain every neat function we can think
of, and if so, should all of these be exposed in the pylab namespace?
Do we have or need any quality-control standards for these functions?
Is there a point in exposing more of mlab now than we have in the past?
  Probably so, but we might want to be conservative about it.

Generally I use cbook as a repository of general purpose helper
functions and classes. mlab started life as a repository of matlab
compatible functions to supplement those in Numeric's MLab and it has
morphed into something a bit looser than that: something like a cbook
for stuff with numerical content.

Some of the code that started life in mlab has been moved over into
numpy (polyfit, polyval, ...) and is now deprecated, some of it serves
the plotting functions (cohere, psd, hist, ...), some of it provides a
matlab-like function that clashes with a numpy function of the same
name (load), some of it is just helper code written primarily by
Fernando, me and others that didn't have a better home( eg, Fernando's
code lived in ipython for a while and then he contributed it to mlab)

One could argue that everything there belongs somewhere else: the load
and save and record array loaders and saves belong in scipy.io, the
numerical codes belong in scipy or numpy or some sandbox, some of the
stuff could just be scrapped as a relic of the past. I don't have a
bone to pick with that argument, but I do have a practical concern: I
am not a numpy/scipy committer and don't really have the time to
become one and it is easier for me to put them into mpl (where I
understand the code organization, build process, commit protocol,
tests, etc much better) than to go through a numpy or scipy patch
development and submission process.

I am more than happy for any of this code to end up there and be
pulled from mpl. In the past, I've offered this to Travis and he has
taken me up on it for some functions, and the same goes for any other
user or developer who has strong feelings on how these codes should be
organized. Having taught several courses of "scientific computing for
python" I am certainly aware of and sympathetic to the complaint that
it is difficult to know where to find stuff in the support packages
for scientific computing. I am all in favor of getting as much useful
stuff into numpy and scipy and organized and documented -- it's just
not likely to be me who is the one doing it, just from a time
standpoint. So I currently tend to use cbook and mlab as a place for
code I develop that is generally useful and is not available in numpy
or scipy.

Since we are making good progress with cleaning up the namespaces
inside mpl and in the examples and in the import layer (eg pyplot), I
am happy to *minimize* the contribution of mlab to the pylab
namespace. In my scripts I get only "figure", "close" and "show" from
pylab anyway so it won't affect me. If we want to stick to the
original view of mlab as a repository of matlab-like functions not in
scipy or numpy, we can prune the stuff it provides to pylab
significantly. I'm also happy to split stuff out of mlab into a
different module for clarity if you or others think that would be
useful ( I did that originally adding the rec* functions to
"recutils", but reverted thinking it would be preferable to add to an
existing module than create another one)

As for tests and quality control, I'm open to suggestions. For most
things, I tend to write an example and add it to the backend driver,
which is a test on "still runs" but generally not "runs correctly".
Certainly more unit tests would be welcome.

JDH

···

On Nov 12, 2007 12:15 PM, Eric Firing <efiring@...229...> wrote:

#3

Hey John,

I am willing to commit to helping migrating the relevant mlab code
over to SciPy (or NumPy). The next release of SciPy
(http://projects.scipy.org/scipy/scipy/milestone/0.7) may be pushed
back a little, but it should be out by February at the latest. One of
the things I had been planning to work on for this release was going
over the various scipy.io code anyway.

I probably won't get a chance to take a close look at things until
December, but I thought I would mention it in case it has any impact
on your own plans.

Thanks,

···

On Nov 12, 2007 1:19 PM, John Hunter <jdh2358@...149...> wrote:

On Nov 12, 2007 12:15 PM, Eric Firing <efiring@...229...> wrote:
> On a more strategic note: what do you see as the future of mlab, and its
> place in pylab? Should mlab contain every neat function we can think
> of, and if so, should all of these be exposed in the pylab namespace?
> Do we have or need any quality-control standards for these functions?
> Is there a point in exposing more of mlab now than we have in the past?
> Probably so, but we might want to be conservative about it.

One could argue that everything there belongs somewhere else: the load
and save and record array loaders and saves belong in scipy.io, the
numerical codes belong in scipy or numpy or some sandbox, some of the
stuff could just be scrapped as a relic of the past. I don't have a
bone to pick with that argument, but I do have a practical concern: I
am not a numpy/scipy committer and don't really have the time to
become one and it is easier for me to put them into mpl (where I
understand the code organization, build process, commit protocol,
tests, etc much better) than to go through a numpy or scipy patch
development and submission process.

I am more than happy for any of this code to end up there and be
pulled from mpl. In the past, I've offered this to Travis and he has
taken me up on it for some functions, and the same goes for any other
user or developer who has strong feelings on how these codes should be
organized. Having taught several courses of "scientific computing for
python" I am certainly aware of and sympathetic to the complaint that
it is difficult to know where to find stuff in the support packages
for scientific computing. I am all in favor of getting as much useful
stuff into numpy and scipy and organized and documented -- it's just
not likely to be me who is the one doing it, just from a time
standpoint. So I currently tend to use cbook and mlab as a place for
code I develop that is generally useful and is not available in numpy
or scipy.

--
Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014
http://cirl.berkeley.edu/

#4

Thanks Jarrod, for the offer. I did another read-through of the mlab
stuff and it doesn't look like an unreasonable amount of work since a
lot of it is already deprecated. Some of it is geometry, like tests
whether segments intersect and that kind of thing, and belong in mpl
but probably another module. psd and csd will be needed to support
the plotting functions, so they will have to be in numpy or maptlotlib
since we don't rely on scipy. Some of the io stuff can go into scipy
or numpy. We'll reconvene in December when you have more time...

JDH

···

On Nov 12, 2007 8:35 PM, Jarrod Millman <millman@...453...> wrote:

I am willing to commit to helping migrating the relevant mlab code
over to SciPy (or NumPy). The next release of SciPy
(http://projects.scipy.org/scipy/scipy/milestone/0.7) may be pushed
back a little, but it should be out by February at the latest. One of
the things I had been planning to work on for this release was going
over the various scipy.io code anyway.

I probably won't get a chance to take a close look at things until
December, but I thought I would mention it in case it has any impact
on your own plans.