overridding builtin variables in pylab

Here's something I find surprising (and disconcerting):

astraw@...183...:~$ python
Python 2.3.4 (#2, Sep 24 2004, 08:39:09)
[GCC 3.3.4 (Debian 1:3.3.4-12)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> min(1,2)
1
>>> from pylab import *
>>> min(1,2)
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "/usr/lib/python2.3/site-packages/numarray/linear_algebra/mlab.py", line 149, in min
     return minimum.reduce(m,axis)
ValueError: Empty array in a ufunc that has no identity value.

So, it seems that the "min" function is overridden. Digging deeper,

>>> min.__module__
'numarray.linear_algebra.mlab'

OK, I see what's going on here: in an effort to achieve maximum matlab compatibility, pylab imports everything (or at least min) from numarray.linear_algebra.mlab. The trouble with this is that a simple "from pylab import *" will break a script that uses min() on scalars. While numarray may have very good reasons for overriding builtins (although I admit that I cannot see what), I think causing normal Python scripts to break by simply importing pylab into their namespace is not the way to achieve total world domination. :slight_smile:

This may be a numarray.linear_algebra.mlab bug/issue and I should go to their mailing list. (Although here is probably as good a place to reach them :). But, even if the good numarray folks did it this way for a reason (I'd love to know, but I should hunt on their FAQ before asking), I suggest than pylab seek to not (incompatibly) override builtins.

Again, "from pylab import *" is our plan for total world domination, and I suggest we not break pure Python scripts.

The general problem does not appear serious:

>>> from pylab import *
>>>
>>> for key in globals().keys():
... if key in dir(__builtins__):
... print "'%s' was overridden."%key
...
'round' was overridden.
'sum' was overridden.
'abs' was overridden.
'max' was overridden.
'min' was overridden.
'__doc__' was overridden.
'__name__' was overridden.

Cursory checks of the above functions indicate all except min(), max(), and round() are compatible with normal (non-exception raising) Python use. Even round() is apparently deprecated by numarray to around().

So, basically, I think we're just talking about min() and max().

What do folks think?

A few weeks ago (on Dec 18), I send an email to the matplotlib-devel list indicating that "from pylab import *" overrides some builtin functions, such as min() and max(). [1]

This results from pylab enlarging its namespace greatly with statements like "from numerix import *" and "from mlab import *". In general this is a good thing, but it seems numarray.linear_algebra.mlab (amongst possibly others) overrides some builtin names. Although I don't see the benefit in this design for numarray.linear_algebra.mlab, I can live with it, since I never do "from numarray.linear_algebra.mlab import *".

However, I (and many others here, I suspect) would like to frequently use "from pylab import *", and it really pisses me off when I discover that various builtins are overridden, causing mysterious errors that may be hard to track down.

So, I offer the following patch [2] to pylab.py which fixes this. I hereby ask for your feedback indicating if you are happy with the current behavior, or if you prefer that pylab does not override builtins. (Also, if you have a better way, that would be appreciated, too.)

You may check your own systems using a little script I wrote while testing this. [3]

Cheers!
Andrew

[1] http://sourceforge.net/mailarchive/forum.php?thread_id=6190717&forum_id=36187

[2] Patch to pylab.py, to be inserted anywhere after the last "from blah import *", e.g. line 216.

# restore builtin functions which may have been overridden
min = getattr(sys.modules['__builtin__'],'min')
max = getattr(sys.modules['__builtin__'],'max')
sum = getattr(sys.modules['__builtin__'],'sum')
round = getattr(sys.modules['__builtin__'],'round')
abs = getattr(sys.modules['__builtin__'],'abs')

[3] Script to test for overriding of builtin names:

import sys

def check_globals():
    for key in globals().keys():
        if key in dir(sys.modules['__builtin__']):
            if globals()[key] != getattr(sys.modules['__builtin__'],key):
                print "'%s' was overridden in globals()."%key

print 'before pylab import'
check_globals()
print

from pylab import *

print 'after pylab import'
check_globals()
print

Andrew Straw wrote:

So, I offer the following patch [2] to pylab.py which fixes this. I hereby ask for your feedback indicating if you are happy with the current behavior, or if you prefer that pylab does not override builtins. (Also, if you have a better way, that would be appreciated, too.)

+1 on your patch.

I'd also much rather have the builtin namespace be left alone.

Cheers,

f

I fear that this "fix" will rather add to the general confusion than lighten
it. First doing "from ... import *" and then reverting it partly really seems
like a bad idea. The cleaner solution would be to fix Numeric/numarray in
such a way that the replacement min/max/round/... retain the complete
original functionality of the builtins only extending them to new types.
Independent of pylab, there obviously is a bug in the underlying libraries
which should not be obscured but solved.

I already hate the current situation as it is: Both, SciPy and matplotlib, are
based upon Numeric/numarray, but then they change and extend its behaviour in
subtle ways so that - in the end - the user is completely confused. If there
is a bug or a missing feature in the array library, it should be fixed there.
Having arrays behave similar but slightly different in SciPy, matplotlib and
numarray/Numeric should be avoided by all means.

···

--
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <Norbert@...160...>

--
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <Norbert@...160...>