mlab and pylab

Development matplotlib-devel
1

I have been gradually numpifying mlab.py; I hope this is not a duplication of effort with someone else. I have not finished yet (and have made no commits), and it may still take a while--there are a lot of things to change, and I am trying to do some cleaning up and rationalization, not just mechanical numpification. I will probably commit a mlab1.py version so as to facilitate some extra testing before making the switch.

I am trying to simplify and shrink the mlab namespace, with the idea that if a function exists in modern numpy it should simply be imported from there.

Similarly, after dealing with mlab, I would like to simplify pylab. Right now, we have a horrible tangle of namespaces in pylab. Cleaning this up will potentially break user code; if a numpy function formerly could be referenced with three different names and we knock that down to one, code using the other two will not work. My guess is that in practice the amount of breakage will be *very* small and easy for users to deal with.

Eric

2

Eric Firing wrote:

Similarly, after dealing with mlab, I would like to simplify pylab. Right now, we have a horrible tangle of namespaces in pylab. Cleaning this up will potentially break user code; if a numpy function formerly could be referenced with three different names and we knock that down to one, code using the other two will not work. My guess is that in practice the amount of breakage will be *very* small and easy for users to deal with.

Speaking off the top of my head, without looking at the specifics -- would it be useful to generate deprecation warnings for these functions before we ultimately remove them? The deprecated functions could just be decorated versions of the "correct" functions that raise a warning and then delegate to the correct version.

May be overkill if these functions really are in low usage, as you suspect.

Cheers,
Mike

3

Mike,

It is a good question.

Right now I am doing some of each. If a function was defined in mlab.py and it has a replacement in numpy I am issuing a deprecation warning and passing the arguments to the numpy function. If a function was not defined in mlab but was imported from numerix, then, with a couple of exceptions, I am simply not leaving it in the mlab namespace.

This is in the present (old) mlab.py:

from numerix import array, asarray, arange, divide, exp, arctan2, \
      multiply, transpose, ravel, repeat, resize, reshape, floor, ceil,\
      absolute, matrixmultiply, power, take, where, Float, Int, asum,\
      dot, convolve, pi, Complex, ones, zeros, diagonal, Matrix, nonzero, \
      log, searchsorted, concatenate, sort, ArrayType, clip, size, indices,\
      conjugate, typecode, iscontiguous

For compatibility, access to some or all of these things may need to remain in the *pylab* namespace for a while, but I don't think it should stay in the *mlab* namespace as well. I really don't want to add wrappers with warnings for all these functions.

Eric

Michael Droettboom wrote:

···

Eric Firing wrote:

Similarly, after dealing with mlab, I would like to simplify pylab. Right now, we have a horrible tangle of namespaces in pylab. Cleaning this up will potentially break user code; if a numpy function formerly could be referenced with three different names and we knock that down to one, code using the other two will not work. My guess is that in practice the amount of breakage will be *very* small and easy for users to deal with.

Speaking off the top of my head, without looking at the specifics -- would it be useful to generate deprecation warnings for these functions before we ultimately remove them? The deprecated functions could just be decorated versions of the "correct" functions that raise a warning and then delegate to the correct version.

May be overkill if these functions really are in low usage, as you suspect.

Cheers,
Mike

4

Hi Eric,

thanks for the warning. I had looked into the issue but never got around
to start working on it.

While you are at it, I would suggest that you split the graphics stuff
out of pylab into a separate
module (pyplot?) to be used for "interactive plotting".

I believe I'm not the only user who likes the stateful plotting
functionality (i.e. based on gca(), gcf() etc.) but does not want to
import all the matlab-compatibility numerics-functions along. Typically,
I prefer the nice and clean numpy stuff to do numerics.

I would suggest to create an additional module (e.g. pyplot?) that only
contains the stateful plotting routines and a module pylab that simply
imports everything from pyplot and mlab.

Greetings,
Norbert

Eric Firing wrote:

···

I have been gradually numpifying mlab.py; I hope this is not a
duplication of effort with someone else. I have not finished yet (and
have made no commits), and it may still take a while--there are a lot of
things to change, and I am trying to do some cleaning up and
rationalization, not just mechanical numpification. I will probably
commit a mlab1.py version so as to facilitate some extra testing before
making the switch.

I am trying to simplify and shrink the mlab namespace, with the idea
that if a function exists in modern numpy it should simply be imported
>from there.

Similarly, after dealing with mlab, I would like to simplify pylab.
Right now, we have a horrible tangle of namespaces in pylab. Cleaning
this up will potentially break user code; if a numpy function formerly
could be referenced with three different names and we knock that down to
one, code using the other two will not work. My guess is that in
practice the amount of breakage will be *very* small and easy for users
to deal with.

Eric

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

5

Eric Firing wrote:

Similarly, after dealing with mlab, I would like to simplify pylab. Right now, we have a horrible tangle of namespaces in pylab. Cleaning this up will potentially break user code; if a numpy function formerly could be referenced with three different names and we knock that down to one, code using the other two will not work. My guess is that in practice the amount of breakage will be *very* small and easy for users to deal with.

I for one will be happy to change my code; numerical stuff in numpy, plotting stuff in pylab (or pyplot?), though some things like linspace() may be hard to loose; that's really an mlab function and I can import mlab. It probably makes more sense.

For the C library you say "man strcpy" or "man erf" and it tells you what include file to use to get the definition (function prototype). In python you only get the help after you've already imported. But it was and still is reasonable to have an import statement like an #include to define stuff you need. If I'm looking at somebody else's code and I see "linspace" or something, it would be nice to have a "man linspace" or so, that tells me which module I need to import. Is there a way to convert doc strings to man pages?

Speaking as somebody who has bascially switched from "write it in C and use assembly for speed" to "write it in Python and use C for speed".

···

--
I say to you: good and evil which would be everlasting- it does not
exist! Of its own accord must it ever overcome itself anew.
  -- thus spoke Zarathustra

6

Tom Holroyd (NIH/NIMH) [E] wrote:

Eric Firing wrote:

Similarly, after dealing with mlab, I would like to simplify pylab.
Right now, we have a horrible tangle of namespaces in pylab. Cleaning
this up will potentially break user code; if a numpy function formerly
could be referenced with three different names and we knock that down to
one, code using the other two will not work. My guess is that in
practice the amount of breakage will be *very* small and easy for users
to deal with.

I for one will be happy to change my code; numerical stuff in numpy, plotting stuff in pylab (or pyplot?), though some things like linspace() may be hard to loose; that's really an mlab function and I can import mlab. It probably makes more sense.

I'm not sure if linspace was just a bad example, but it does exist
within numpy itself.

Ryan

···

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

7

Just a bad example. I know there are functions that were originally defined in Matlab and then got rewritten in python, but I've never been clear on which modules they really live in.

Ryan May wrote:

···

Tom Holroyd (NIH/NIMH) [E] wrote:

Eric Firing wrote:

Similarly, after dealing with mlab, I would like to simplify pylab. Right now, we have a horrible tangle of namespaces in pylab. Cleaning this up will potentially break user code; if a numpy function formerly could be referenced with three different names and we knock that down to one, code using the other two will not work. My guess is that in practice the amount of breakage will be *very* small and easy for users to deal with.

I for one will be happy to change my code; numerical stuff in numpy, plotting stuff in pylab (or pyplot?), though some things like linspace() may be hard to loose; that's really an mlab function and I can import mlab. It probably makes more sense.

I'm not sure if linspace was just a bad example, but it does exist
within numpy itself.

Ryan

--
I say to you: good and evil which would be everlasting- it does not
exist! Of its own accord must it ever overcome itself anew.
  -- thus spoke Zarathustra

8

Tom Holroyd (NIH/NIMH) [E] wrote:
[...]

For the C library you say "man strcpy" or "man erf" and it tells you what include file to use to get the definition (function prototype). In python you only get the help after you've already imported. But it was and still is reasonable to have an import statement like an #include to define stuff you need. If I'm looking at somebody else's code and I see "linspace" or something, it would be nice to have a "man linspace" or so, that tells me which module I need to import. Is there a way to convert doc strings to man pages?

I don't know of any such tool, but it could be done. I don't think it would solve the problem you refer to, though, which I understand as wanting to know where a given function actually comes from. One of the problems with creating a huge number of man pages for this is that function names may be duplicated; linspace, for example, could be coming from numpy or from mlab (until we get everything cleaned up...).

So, I think you are stuck with two approaches: actually tracing imports back through the code, or running ipython, importing a module, and then using the ? facility:

In [1]:import pylab

In [2]:pylab.linspace?
Type: function
Base Class: <type 'function'>
String Form: <function linspace at 0x8541a04>
Namespace: Interactive
File: /usr/local/lib/python2.5/site-packages/numpy/lib/function_base.py
Definition: pylab.linspace(start, stop, num=50, endpoint=True, retstep=False)
Docstring:
     Return evenly spaced numbers.

     Return num evenly spaced samples from start to stop. If
     endpoint is True, the last sample is stop. If retstep is
     True then return the step value used.

Which shows that at present, the linspace supplied by pylab is actually the numeric version, not the mlab version. This is not at all obvious from looking at pylab.py!

Eric

9

Ryan May wrote:

I for one will be happy to change my code; numerical stuff in
numpy, plotting stuff in pylab (or pyplot?), though some things
like linspace() may be hard to loose; that's really an mlab
function and I can import mlab.

I'm a big namespace fan. I'd much rather see us all do something like:

import pyplot
import numpy as N
import mlab

etc.

In this case, there should be little name overlap between these modules, and you could "import *" all of them if you really wanted.

However, they are not all maintained by the same folks, so some overlap may occur, and that's a good reason for keeping them in separate namespaces.

In any case, it needs to be decided what the point of the mlab module is. There was a discussion about this a few (quite a few!) years ago when there was just Numeric, and it came with an mlab module (at least I think it was called mlab). At that point, we decided that Matlab compatibility wasn't really the point -- the point was that there were a number of nifty utilities in Matlab that people missed, and that the stuff in mlab wasn't completely Matlab compatible anyway (numpy and Matlab are just plain different, after all). We more or less decided to what we really needed was a "Utilities" module that could hold some of those nifty useful functions (like linspace, etc), but not much was done at the time. Since then, numpy as gained quite a bit of those features anyway.

So for this mlab module -- what is the goal?

1) A Matlab-like set of tools? In that case, it way want to default to matrices, rather than arrays, etc. If that's what's wanted, I'd make it a stand-alone module, that imports what it needs from numpy, but provides a complete API.

2) A bunch of extra utilities that numpy doesn't have, but are handy. In that case, I'd have it include only the extras, and one would need to import numpy and mlab both. It could also "import *" numpy , then add the extra stuff, but I'm too much of a namespace fan to advocate for that. What happens when numpy adds a name that clashes with an mlab name, for instance?

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

10

Christopher Barker wrote:

Ryan May wrote:

I for one will be happy to change my code; numerical stuff in
numpy, plotting stuff in pylab (or pyplot?), though some things
like linspace() may be hard to loose; that's really an mlab
function and I can import mlab.

I'm a big namespace fan. I'd much rather see us all do something like:

import pyplot
import numpy as N
import mlab

etc.

In this case, there should be little name overlap between these modules, and you could "import *" all of them if you really wanted.

However, they are not all maintained by the same folks, so some overlap may occur, and that's a good reason for keeping them in separate namespaces.

In any case, it needs to be decided what the point of the mlab module is. There was a discussion about this a few (quite a few!) years ago when there was just Numeric, and it came with an mlab module (at least I think it was called mlab). At that point, we decided that Matlab compatibility wasn't really the point -- the point was that there were a number of nifty utilities in Matlab that people missed, and that the stuff in mlab wasn't completely Matlab compatible anyway (numpy and Matlab are just plain different, after all). We more or less decided to what we really needed was a "Utilities" module that could hold some of those nifty useful functions (like linspace, etc), but not much was done at the time. Since then, numpy as gained quite a bit of those features anyway.

So for this mlab module -- what is the goal?

1) A Matlab-like set of tools? In that case, it way want to default to matrices, rather than arrays, etc. If that's what's wanted, I'd make it a stand-alone module, that imports what it needs from numpy, but provides a complete API.

It looks like it started out something like this, but I don't think it should stay that way. If someone wants something like this, I think they will have to do it themselves.

2) A bunch of extra utilities that numpy doesn't have, but are handy. In that case, I'd have it include only the extras, and one would need to import numpy and mlab both. It could also "import *" numpy , then add the extra stuff, but I'm too much of a namespace fan to advocate for that. What happens when numpy adds a name that clashes with an mlab name, for instance?

I think this is the direction mlab has evolved, and my intention is to take it the rest of the way. As with pylab, some degree of matlab-similarity can be retained, but the goal is not a clone--that would be neither attainable nor desirable. If someone wants something more clone-like, they can use octave or scilab. I do not think mlab should "from numpy import *"; if that is what someone wants, they can do it explicitly themselves, very easily. Now, pylab should do something like "from mlab import *; from numpy import *; from pyplot import *", but mlab should be its own nice, clean set of functions.

The strategy here is to make a clear distinction between modules that provide unique functionality, and modules that serve as namespace aggregators.

Eric

···

-Chris

11

Eric Firing wrote:

I do not think mlab should "from numpy import *"; if that is what someone wants, they can do it explicitly themselves, very easily. Now, pylab should do something like "from mlab import *; from numpy import *; from pyplot import *", but mlab should be its own nice, clean set of functions.

+1

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...