scipy core update

Development matplotlib-devel
1

John, It looks like you are referring to bugs in the

    > Colormap.__call__ method that were introduced when I added
    > support for masked values out-of-bounds values. I have
    > committed a change that fixes the bugs I could find there
    > (as tested with examples/image_masked.py), so that the
    > colormap call produces exactly the same output with scipy as
    > with Numeric or numarray--the same rgba values.
    > Unfortunately, although that demo now runs with all three,
    > it produces different plots. It appears that somewhere else
    > in the code--I'm pretty sure it is not within colors.py--the
    > floating point rgba values are getting rounded or, more
    > likely, truncated, when using scipy.

Hey Eric,

Thanks for taking a look at this. The only other logical place for a
problem to reside is in colors.normalize, since the pipleline is

  rgba = cmap(normalize(X))

The relevant bit of code is here -- if you or anyone else sees an
obvious candidate that might cause this truncation, let me know....

class normalize:
    def __init__(self, vmin=None, vmax=None, clip = True):
        """
        Normalize a given value to the 0-1 range

        If vmin or vmax is not given, they are taken from the input's
        minimum and maximum value respectively. If clip is True and
        the given value falls outside the range, the returned value
        will be 0 or 1, whichever is closer. Returns 0 if vmin==vmax.
        Works with scalars or arrays, including masked arrays. If clip
        is True, masked values on input will be set to 1 on output; if
        clip is False, the mask will be propagated to the output.
        """
        self.vmin = vmin
        self.vmax = vmax
        self.clip = clip

    def __call__(self, value):

        if isinstance(value, (int, float)):
            vtype = 'scalar'
            val = ma.array([value])
        else:
            vtype = 'array'
            val = ma.asarray(value)

        self.autoscale(val)
        vmin, vmax = self.vmin, self.vmax
        if vmin > vmax:
            raise ValueError("minvalue must be less than or equal to maxvalue")
        elif vmin==vmax:
            return 0.*value
        else:
            if self.clip:
                val = clip(val.filled(vmax), vmin, vmax)
            result = (1.0/(vmax-vmin))*(val-vmin)
        if vtype == 'scalar':
            result = result[0]
        return result

    def autoscale(self, A):
        if not self.scaled():
            if self.vmin is None: self.vmin = ma.minimum(A)
            if self.vmax is None: self.vmax = ma.maximum(A)

    def scaled(self):
        'return true if vmin and vmax set'
        return (self.vmin is not None and self.vmax is not None)

JDH

2

John,

The problem is that a bunch of scipy support is missing. I think I have most of it fixed; I will do a little more, commit, and send another message in a few minutes.

Eric

John Hunter wrote:

···

"Eric" == Eric Firing <efiring@...229...> writes:

    > John, It looks like you are referring to bugs in the
    > Colormap.__call__ method that were introduced when I added
    > support for masked values out-of-bounds values. I have
    > committed a change that fixes the bugs I could find there
    > (as tested with examples/image_masked.py), so that the
    > colormap call produces exactly the same output with scipy as
    > with Numeric or numarray--the same rgba values.
    > Unfortunately, although that demo now runs with all three,
    > it produces different plots. It appears that somewhere else
    > in the code--I'm pretty sure it is not within colors.py--the
    > floating point rgba values are getting rounded or, more
    > likely, truncated, when using scipy.

Hey Eric,

Thanks for taking a look at this. The only other logical place for a
problem to reside is in colors.normalize, since the pipleline is

  rgba = cmap(normalize(X))

The relevant bit of code is here -- if you or anyone else sees an
obvious candidate that might cause this truncation, let me know....

class normalize:
    def __init__(self, vmin=None, vmax=None, clip = True):
        """
        Normalize a given value to the 0-1 range

        If vmin or vmax is not given, they are taken from the input's
        minimum and maximum value respectively. If clip is True and
        the given value falls outside the range, the returned value
        will be 0 or 1, whichever is closer. Returns 0 if vmin==vmax.
        Works with scalars or arrays, including masked arrays. If clip
        is True, masked values on input will be set to 1 on output; if
        clip is False, the mask will be propagated to the output.
        """
        self.vmin = vmin
        self.vmax = vmax
        self.clip = clip

    def __call__(self, value):

        if isinstance(value, (int, float)):
            vtype = 'scalar'
            val = ma.array([value])
        else:
            vtype = 'array'
            val = ma.asarray(value)

        self.autoscale(val)
        vmin, vmax = self.vmin, self.vmax
        if vmin > vmax:
            raise ValueError("minvalue must be less than or equal to maxvalue")
        elif vmin==vmax:
            return 0.*value
        else:
            if self.clip:
                val = clip(val.filled(vmax), vmin, vmax)
            result = (1.0/(vmax-vmin))*(val-vmin)
        if vtype == 'scalar':
            result = result[0]
        return result

    def autoscale(self, A):
        if not self.scaled():
            if self.vmin is None: self.vmin = ma.minimum(A)
            if self.vmax is None: self.vmax = ma.maximum(A)

    def scaled(self):
        'return true if vmin and vmax set'
        return (self.vmin is not None and self.vmax is not None)

JDH

3

John,

I committed fixes to several files; looks OK now, at least for examples/image_masked.py.

Eric

John Hunter wrote:

···

"Eric" == Eric Firing <efiring@...229...> writes:

    > John, It looks like you are referring to bugs in the
    > Colormap.__call__ method that were introduced when I added
    > support for masked values out-of-bounds values. I have
    > committed a change that fixes the bugs I could find there
    > (as tested with examples/image_masked.py), so that the
    > colormap call produces exactly the same output with scipy as
    > with Numeric or numarray--the same rgba values.
    > Unfortunately, although that demo now runs with all three,
    > it produces different plots. It appears that somewhere else
    > in the code--I'm pretty sure it is not within colors.py--the
    > floating point rgba values are getting rounded or, more
    > likely, truncated, when using scipy.

Hey Eric,

Thanks for taking a look at this. The only other logical place for a
problem to reside is in colors.normalize, since the pipleline is

  rgba = cmap(normalize(X))

The relevant bit of code is here -- if you or anyone else sees an
obvious candidate that might cause this truncation, let me know....

class normalize:
    def __init__(self, vmin=None, vmax=None, clip = True):
        """
        Normalize a given value to the 0-1 range

        If vmin or vmax is not given, they are taken from the input's
        minimum and maximum value respectively. If clip is True and
        the given value falls outside the range, the returned value
        will be 0 or 1, whichever is closer. Returns 0 if vmin==vmax.
        Works with scalars or arrays, including masked arrays. If clip
        is True, masked values on input will be set to 1 on output; if
        clip is False, the mask will be propagated to the output.
        """
        self.vmin = vmin
        self.vmax = vmax
        self.clip = clip

    def __call__(self, value):

        if isinstance(value, (int, float)):
            vtype = 'scalar'
            val = ma.array([value])
        else:
            vtype = 'array'
            val = ma.asarray(value)

        self.autoscale(val)
        vmin, vmax = self.vmin, self.vmax
        if vmin > vmax:
            raise ValueError("minvalue must be less than or equal to maxvalue")
        elif vmin==vmax:
            return 0.*value
        else:
            if self.clip:
                val = clip(val.filled(vmax), vmin, vmax)
            result = (1.0/(vmax-vmin))*(val-vmin)
        if vtype == 'scalar':
            result = result[0]
        return result

    def autoscale(self, A):
        if not self.scaled():
            if self.vmin is None: self.vmin = ma.minimum(A)
            if self.vmax is None: self.vmax = ma.maximum(A)

    def scaled(self):
        'return true if vmin and vmax set'
        return (self.vmin is not None and self.vmax is not None)

JDH

4

John, Travis,

Now I find a genuine difference between scipy and num* that is causing a problem: for a masked array, say "A", A.mask is a property in scipy but a function in num*. I can work around it, but it seems like an anomaly in scipy. What it means is that in matplotlib we can't use A.mask or A.mask(), but must instead use the ma.getmask() function.

Here is another anomaly that I needed to work around:

>>> bb = scipy.array([1.1,2,2,3.3])
>>> 1.1 in bb
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
>>> import numarray
>>> nn = numarray.array([1.1,2.2,3.3])
>>> 1.1 in nn
True

x in A behaves completely differently if A is a scipy ndarray than if it is a numarray array. This seems to me like a bug in scipy; the num* behavior is certainly easier to use, in that it preserves the list-like behavior when used in a list-like context.

In any case, I have committed changes to make contour work; but examples/masked_demo.py still doesn't work with scipy. It generates no exception, but the plot is completely wrong. I can't spend any more time on this right now.

Eric

5

Eric Firing wrote:

I replied privately to Eric by mistake.

John, Travis,

Now I find a genuine difference between scipy and num* that is causing a problem: for a masked array, say "A", A.mask is a property in scipy but a function in num*. I can work around it, but it seems like an anomaly in scipy. What it means is that in matplotlib we can't use A.mask or A.mask(), but must instead use the ma.getmask() function.

Paul is the author of masked arrays. He implemented the new behavior. I think mask as a property/attribute is right.

Here is another anomaly that I needed to work around:

>>> bb = scipy.array([1.1,2,2,3.3])
>>> 1.1 in bb
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
>>> import numarray
>>> nn = numarray.array([1.1,2.2,3.3])
>>> 1.1 in nn
True

This is a bug in scipy core that has been fixed for several weeks. It came about as an unintentional side-effect of raising an error for truth-testing of arrays.

Notice:

>>> 1.1 in scipy.array([1.1,2.2,3.3])
True

-Travis