Problem checking if masked arrays are finite in transforms.py

It looks like the recent modification to transforms.py to update datalims only
when valid data are available is broken for numpy-1.04. In
tranforms.Bbox.update_from_data_xy, with numpy-1.04, the following line:

xym = ma.masked_where(~npy.isfinite(xy), xy)

fails with:

  File "/usr/lib64/python2.5/site-packages/matplotlib/axes.py", line 2805, in
plot
    self.add_line(line)
  File "/usr/lib64/python2.5/site-packages/matplotlib/axes.py", line 1165, in
add_line
    self._update_line_limits(line)
  File "/usr/lib64/python2.5/site-packages/matplotlib/axes.py", line 1173, in
_update_line_limits
    self.update_datalim( xydata )
  File "/usr/lib64/python2.5/site-packages/matplotlib/axes.py", line 1221, in
update_datalim
    self.dataLim.update_from_data_xy(xys, self.ignore_existing_data_limits)
  File "/usr/lib64/python2.5/site-packages/matplotlib/transforms.py", line
699, in update_from_data_xy
    xym = ma.masked_where(~npy.isfinite(xy), xy)
  File "/usr/lib64/python2.5/site-packages/numpy/core/ma.py", line 641, in
__array_wrap__
    domain = ufunc_domain[func]
KeyError: <ufunc 'isfinite'>

It looks like isfinite doesnt like getting a masked array as input. Is xy
always a masked array? If so, maybe that line could look like:

xym = ma.masked_where(~npy.isfinite(xy.data), xy)

Darren,

In an earlier thread on matplotlib-users, when this first came up, John noted that numpy svn should be required for present mpl svn, so instead of fixing the attempted workaround for 1.04 I took it out and instead put a numpy version check in matplotlib.__init__.

Eric

Darren Dale wrote:

···

It looks like the recent modification to transforms.py to update datalims only when valid data are available is broken for numpy-1.04. In tranforms.Bbox.update_from_data_xy, with numpy-1.04, the following line:

xym = ma.masked_where(~npy.isfinite(xy), xy)

fails with:

  File "/usr/lib64/python2.5/site-packages/matplotlib/axes.py", line 2805, in plot
    self.add_line(line)
  File "/usr/lib64/python2.5/site-packages/matplotlib/axes.py", line 1165, in add_line
    self._update_line_limits(line)
  File "/usr/lib64/python2.5/site-packages/matplotlib/axes.py", line 1173, in _update_line_limits
    self.update_datalim( xydata )
  File "/usr/lib64/python2.5/site-packages/matplotlib/axes.py", line 1221, in update_datalim
    self.dataLim.update_from_data_xy(xys, self.ignore_existing_data_limits)
  File "/usr/lib64/python2.5/site-packages/matplotlib/transforms.py", line 699, in update_from_data_xy
    xym = ma.masked_where(~npy.isfinite(xy), xy)
  File "/usr/lib64/python2.5/site-packages/numpy/core/ma.py", line 641, in __array_wrap__
    domain = ufunc_domain[func]
KeyError: <ufunc 'isfinite'>

It looks like isfinite doesnt like getting a masked array as input. Is xy always a masked array? If so, maybe that line could look like:

xym = ma.masked_where(~npy.isfinite(xy.data), xy)

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

Just to make clear my thinking on this: since the svn trunk of mpl is
a major refactoring and will be a major point jump when we release it
(0.98), this is a good time to get onto numpy 1.1 (ie numpy svn) so we
can rely on all the nice features and fixes that have gone into that
release.

JDH

···

On Thu, Apr 24, 2008 at 1:32 PM, Eric Firing <efiring@...229...> wrote:

Darren,

In an earlier thread on matplotlib-users, when this first came up, John
  noted that numpy svn should be required for present mpl svn, so
instead of fixing the attempted workaround for 1.04 I took it out and
instead put a numpy version check in matplotlib.__init__.

John Hunter wrote:

···

On Thu, Apr 24, 2008 at 1:32 PM, Eric Firing <efiring@...229...> wrote:

Darren,

In an earlier thread on matplotlib-users, when this first came up, John
  noted that numpy svn should be required for present mpl svn, so
instead of fixing the attempted workaround for 1.04 I took it out and
instead put a numpy version check in matplotlib.__init__.

Just to make clear my thinking on this: since the svn trunk of mpl is
a major refactoring and will be a major point jump when we release it
(0.98), this is a good time to get onto numpy 1.1 (ie numpy svn) so we
can rely on all the nice features and fixes that have gone into that
release.

JDH

I agree completely!

On a related note, what about python >= 2.4 instead of 2.3? This is not something I have a strong opinion about, but I think it might also be a reasonable time to let 2.4 be the minimum requirement.

Eric

As a point of reference, NumPy 1.2 will require python >= 2.4 and so
will the SciPy 0.7 release.

···

On Thu, Apr 24, 2008 at 2:21 PM, Eric Firing <efiring@...229...> wrote:

On a related note, what about python >= 2.4 instead of 2.3? This is not
something I have a strong opinion about, but I think it might also be a
reasonable time to let 2.4 be the minimum requirement.

--
Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014
http://cirl.berkeley.edu/

I think python 2.4 is totally reasonable for the 0.98 release, though
I am still prone to avoiding certain magic like meta-classes where
possible.

JDH

···

On Thu, Apr 24, 2008 at 2:21 PM, Eric Firing <efiring@...229...> wrote:

On a related note, what about python >= 2.4 instead of 2.3? This is not
something I have a strong opinion about, but I think it might also be a
reasonable time to let 2.4 be the minimum requirement.

John Hunter wrote:

On a related note, what about python >= 2.4 instead of 2.3? This is not
something I have a strong opinion about, but I think it might also be a
reasonable time to let 2.4 be the minimum requirement.

I think python 2.4 is totally reasonable for the 0.98 release, though
I am still prone to avoiding certain magic like meta-classes where
possible.

JDH

Agreed. If there were a meta-class usage that was crystal-clear after a little inspection, and that actually made the code easier to understand and maintain, I would consider it.

I know that when we talked in Kona I was complaining about decorators, but they are here to stay, and as long as they are not too tricky they can be fine. The @staticmethod form is an improvement over the non-decorator equivalent, an assignment at the bottom of the method. I would be happy to see Mike's commented-out @staticmethod lines uncommented, and the corresponding assignments at the bottom of the method deleted, for example.

Requiring 2.4 would also let us remove the subprocess module--not a big deal, but an incremental simplification of the mpl package.

Eric

···

On Thu, Apr 24, 2008 at 2:21 PM, Eric Firing <efiring@...229...> wrote:

All these sound like good suggestions to me...

JDH

···

On Thu, Apr 24, 2008 at 3:32 PM, Eric Firing <efiring@...229...> wrote:

Agreed. If there were a meta-class usage that was crystal-clear after a
little inspection, and that actually made the code easier to understand and
maintain, I would consider it.

I know that when we talked in Kona I was complaining about decorators, but
they are here to stay, and as long as they are not too tricky they can be
fine. The @staticmethod form is an improvement over the non-decorator
equivalent, an assignment at the bottom of the method. I would be happy to
see Mike's commented-out @staticmethod lines uncommented, and the
corresponding assignments at the bottom of the method deleted, for example.

Requiring 2.4 would also let us remove the subprocess module--not a big
deal, but an incremental simplification of the mpl package.

I've been installing numpy through various package managers and now I'll need
to figure out how to configure the build on fedora, ubuntu and gentoo in
order to use svn matplotlib. Was it a mistake for me to develop my
application using the matplotlib trunk? If I go through the trouble of
configuring the build environments for numpy on these various OS's, am I
going to discover that the numpy trunk is not backward compatible and is
causing problems with other applications? I know my own difficulties are not
sufficient reason to alter the development path of our fine library, but I
think this might be a mistake.

···

On Thursday 24 April 2008 03:09:43 pm John Hunter wrote:

On Thu, Apr 24, 2008 at 1:32 PM, Eric Firing <efiring@...229...> wrote:
> Darren,
>
> In an earlier thread on matplotlib-users, when this first came up, John
> noted that numpy svn should be required for present mpl svn, so
> instead of fixing the attempted workaround for 1.04 I took it out and
> instead put a numpy version check in matplotlib.__init__.

Just to make clear my thinking on this: since the svn trunk of mpl is
a major refactoring and will be a major point jump when we release it
(0.98), this is a good time to get onto numpy 1.1 (ie numpy svn) so we
can rely on all the nice features and fixes that have gone into that
release.

Hum, a quite common discussion (we have had it at the nipy sprint, for
instance). My feeling is that you want to avoid depending on SVN
versions, unless there is a huge gain. The reason is that you loose
tester and potential contributors. In addition it makes it harder to get
the whole stack in a consistent shape because everything becomes a moving
target.

My 2 cents,

Ga�l

Darren Dale wrote:

Darren,

In an earlier thread on matplotlib-users, when this first came up, John
  noted that numpy svn should be required for present mpl svn, so
instead of fixing the attempted workaround for 1.04 I took it out and
instead put a numpy version check in matplotlib.__init__.

Just to make clear my thinking on this: since the svn trunk of mpl is
a major refactoring and will be a major point jump when we release it
(0.98), this is a good time to get onto numpy 1.1 (ie numpy svn) so we
can rely on all the nice features and fixes that have gone into that
release.

I've been installing numpy through various package managers and now I'll need to figure out how to configure the build on fedora, ubuntu and gentoo in order to use svn matplotlib. Was it a mistake for me to develop my application using the matplotlib trunk? If I go through the trouble of configuring the build environments for numpy on these various OS's, am I going to discover that the numpy trunk is not backward compatible and is causing problems with other applications? I know my own difficulties are not sufficient reason to alter the development path of our fine library, but I think this might be a mistake.

Darren,

It is open for discussion. Here are some factors:

1) In my experience, numpy is easy to build from source--easier than matplotlib.

2) The numpy 1.1 release is coming soon--on the order of a week. I don't know how much that will help you. Maybe not much until distro packages catch up, which can take a long time.

3) There have been a lot of bug fixes between numpy 1.04 and 1.1. The main area of *slight* incompatibility is in the masked array package. The main practical difference is that some import forms that worked with 1.04 do not work with 1.1; e.g. you can't import ma from numpy.core because that is not where it is now, and it is sub-package, not a file. The ma internals are quite different (a masked array is now a subclass of ndarray), and the overall implementation is much improved, but functions and methods are highly compatible.

Although I am sympathetic to the problems involved in making changes of this sort, I am also sympathetic to the problems of trying to keep something like mpl working with multiple versions of components like the numeric library. There were a lot of bugs and holes in the old numpy.ma. To me, it is a relief to be able to stick to the new version and forget about the limitations and quirks of the old. Ideally, it will mean that all of us can spend more time thinking about how to improve mpl and less time in duplicate testing and coming up with workarounds.

Eric

···

On Thursday 24 April 2008 03:09:43 pm John Hunter wrote:

On Thu, Apr 24, 2008 at 1:32 PM, Eric Firing <efiring@...229...> wrote:

This is my view too -- if you rely on or can build svn mpl, I seen no
reason why you can't also rely on/build svn numpy since it is a much
easier build. If mpl builds, numpy svn is pretty much guaranteed to
build on any platform.

If I am missing something Darren, please let me know, but in my
regular workflow, I pretty much assume I can bild svn ipython, numpy
and mpl, and hopefully scipy.

JDH

···

On Thu, Apr 24, 2008 at 5:03 PM, Eric Firing <efiring@...229...> wrote:

Darren,

It is open for discussion. Here are some factors:

1) In my experience, numpy is easy to build from source--easier than
matplotlib.

I have been developing a data acquisition and analysis program for my lab, and
we are actually in the process of using it it this week and next to run
experiments with our visiting scientists. I've been running matplotlib from
svn for as long as I can remember and hadn't anticipated trouble. I guess I
just wasn't reading the dev list closely enough since I rely on the trunk for
daily use.

Maybe we should consider cutting a transitional prelease at some point before
additional big changes are made, so those of us who have already transitioned
our code to the new codebase have a reference build we can install when we
run into trouble on the trunk?

···

On Thursday 24 April 2008 09:48:18 pm John Hunter wrote:

On Thu, Apr 24, 2008 at 5:03 PM, Eric Firing <efiring@...229...> wrote:
> Darren,
>
> It is open for discussion. Here are some factors:
>
> 1) In my experience, numpy is easy to build from source--easier than
> matplotlib.

This is my view too -- if you rely on or can build svn mpl, I seen no
reason why you can't also rely on/build svn numpy since it is a much
easier build. If mpl builds, numpy svn is pretty much guaranteed to
build on any platform.

If I am missing something Darren, please let me know, but in my
regular workflow, I pretty much assume I can bild svn ipython, numpy
and mpl, and hopefully scipy.

I have been developing a data acquisition and analysis program for my lab, and
we are actually in the process of using it it this week and next to run
experiments with our visiting scientists. I've been running matplotlib from
svn for as long as I can remember and hadn't anticipated trouble. I guess I
just wasn't reading the dev list closely enough since I rely on the trunk for
daily use.

I can appreciate that you are in production mode and have very little
of any time for distraction, but if you get a minute try:

  > svn co http://svn.scipy.org/svn/numpy/trunk numpy
  > cd numpy/
  > python setup.py build
  > sudo python setup.py install

Make sure you "rm -rf build" your mpl build dir and get a clean mpl
rebuild afterward. I would be very surprised if you encounter any
troubles. The numpy folks have been doing a tremendous amount of
work to get svn ready for a 1.1 release, and you owe it to yourself to
be on the latest.

JDH

···

On Thu, Apr 24, 2008 at 9:23 PM, Darren Dale <darren.dale@...143...> wrote:

I think I'm up and running. I'm not sure that the gentoo's lapack and blas
were recognized, but thats because gentoo does some non standard naming of
the f77blas. Also, there were a few changes I had to make for my project to
work with numpy-svn, related to the behavior of ndarray.flat. Hopefully none
of the other tools I depend on will have much trouble with numpy-1.1.

···

On Thursday 24 April 2008 10:45:33 pm John Hunter wrote:

On Thu, Apr 24, 2008 at 9:23 PM, Darren Dale <darren.dale@...143...> wrote:
> I have been developing a data acquisition and analysis program for my
> lab, and we are actually in the process of using it it this week and next
> to run experiments with our visiting scientists. I've been running
> matplotlib from svn for as long as I can remember and hadn't anticipated
> trouble. I guess I just wasn't reading the dev list closely enough since
> I rely on the trunk for daily use.

I can appreciate that you are in production mode and have very little

of any time for distraction, but if you get a minute try:
  > svn co http://svn.scipy.org/svn/numpy/trunk numpy
  > cd numpy/
  > python setup.py build
  > sudo python setup.py install

Make sure you "rm -rf build" your mpl build dir and get a clean mpl
rebuild afterward. I would be very surprised if you encounter any
troubles. The numpy folks have been doing a tremendous amount of
work to get svn ready for a 1.1 release, and you owe it to yourself to
be on the latest.