fill_between and masked array

Andres_Luhamaa · April 29, 2009, 11:40am

Hello,
I try to do fill_between two arrays, that have missing value (masked arrays). Following code shows behaviour that seems not correct. Images what I expect the result to be (by omitting the missing value) and what I get by plotting the whole arrays.

import pylab
import numpy as np
edatmax=np.array([10,4,6,9,np.nan,9,10])
edatmax=np.ma.masked_array(edatmax,np.isnan(edatmax))
edatmin=np.array([8,4,5,1,np.nan,8,5])
edatmin=np.ma.masked_array(edatmin,np.isnan(edatmin))
xtelg=np.arange(edatmax.size)
xtelg=np.ma.masked_array(xtelg,np.isnan(edatmin))
pylab.plot(edatmax,"gx")
pylab.plot(edatmin,"r+")
# comment out to see better
pylab.fill_between(xtelg,edatmax,edatmin,facecolor='green',alpha='0.3')
# comment in to see better
#pylab.fill_between(xtelg[:4],edatmax[:4],edatmin[:4],facecolor='green',alpha='0.3')
#pylab.fill_between(xtelg[5:],edatmax[5:],edatmin[5:],facecolor='green',alpha='0.3')
pylab.show()

Version of matplotlib is current cvs.

Best regards,
Andres

_John_Hunter1 · April 29, 2009, 3:54pm

fill_between does not currently support masked arrays, but I think we could easily extend it to support the mask using the existing support for the “where” kwarg. For now, does this behave like you expect?

valid = ~(edatmax.mask & edatmax.mask )
pylab.fill_between(xtelg,edatmax,edatmin,facecolor=‘green’,alpha=‘0.3’, where=valid)

JDH

···

On Wed, Apr 29, 2009 at 6:40 AM, Andres Luhamaa <andres.luhamaa@…1716…> wrote:

Hello,

I try to do fill_between two arrays, that have missing value (masked arrays). Following code shows behaviour that seems not correct. Images what I expect the result to be (by omitting the missing value) and what I get by plotting the whole arrays.

import pylab

import numpy as np

edatmax=np.array([10,4,6,9,np.nan,9,10])

edatmax=np.ma.masked_array(edatmax,np.isnan(edatmax))

edatmin=np.array([8,4,5,1,np.nan,8,5])

edatmin=np.ma.masked_array(edatmin,np.isnan(edatmin))

xtelg=np.arange(edatmax.size)

xtelg=np.ma.masked_array(xtelg,np.isnan(edatmin))

pylab.plot(edatmax,“gx”)

pylab.plot(edatmin,“r+”)

comment out to see better

pylab.fill_between(xtelg,edatmax,edatmin,facecolor=‘green’,alpha=‘0.3’)

comment in to see better

#pylab.fill_between(xtelg[:4],edatmax[:4],edatmin[:4],facecolor=‘green’,alpha=‘0.3’)

#pylab.fill_between(xtelg[5:],edatmax[5:],edatmin[5:],facecolor=‘green’,alpha=‘0.3’)

pylab.show()

Version of matplotlib is current cvs.

_John_Hunter1 · April 29, 2009, 3:54pm

Oops – meant

valid = ~(edatmin.mask & edatmax.mask )

JDH

···

On Wed, Apr 29, 2009 at 10:54 AM, John Hunter <jdh2358@…878…287…> wrote:

fill_between does not currently support masked arrays, but I think we could easily extend it to support the mask using the existing support for the “where” kwarg. For now, does this behave like you expect?

valid = ~(edatmax.mask & edatmax.mask )
pylab.fill_between(xtelg,edatmax,edatmin,facecolor=‘green’,alpha=‘0.3’, where=valid)

Michael_Droettboom1 · April 29, 2009, 3:56pm

I don't believe fill_between directly supports missing values -- but it does have the where parameter for this purpose.

We could also be smarter about how "where" is generated when none is provided:

        if where is None:
            where = (~np.ma.getmaskarray(x) & ~np.ma.getmaskarray(y1) &
                     ~np.ma.getmaskarray(y2))

...but I'd like more feedback from the author or users of fill_between before committing that change. (That, and Eric Firing can probably find a much more efficient way to do the masked array manipulation...

Mike

Andres Luhamaa wrote:

···

Hello,
I try to do fill_between two arrays, that have missing value (masked arrays). Following code shows behaviour that seems not correct. Images what I expect the result to be (by omitting the missing value) and what I get by plotting the whole arrays.

import pylab
import numpy as np
edatmax=np.array([10,4,6,9,np.nan,9,10])
edatmax=np.ma.masked_array(edatmax,np.isnan(edatmax))
edatmin=np.array([8,4,5,1,np.nan,8,5])
edatmin=np.ma.masked_array(edatmin,np.isnan(edatmin))
xtelg=np.arange(edatmax.size)
xtelg=np.ma.masked_array(xtelg,np.isnan(edatmin))
pylab.plot(edatmax,"gx")
pylab.plot(edatmin,"r+")
# comment out to see better
pylab.fill_between(xtelg,edatmax,edatmin,facecolor='green',alpha='0.3')
# comment in to see better
#pylab.fill_between(xtelg[:4],edatmax[:4],edatmin[:4],facecolor='green',alpha='0.3')

#pylab.fill_between(xtelg[5:],edatmax[5:],edatmin[5:],facecolor='green',alpha='0.3')

pylab.show()

Version of matplotlib is current cvs.

Best regards,
Andres

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance & Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
------------------------------------------------------------------------

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

_John_Hunter1 · April 29, 2009, 4:03pm

I’m working on a patch for this now…but I would also like Eric to take a look when I am done since I am a masked array dummy

JDH

···

On Wed, Apr 29, 2009 at 10:56 AM, Michael Droettboom <mdroe@…86…> wrote:

I don’t believe fill_between directly supports missing values – but it

does have the where parameter for this purpose.

We could also be smarter about how “where” is generated when none is

provided:
    if where is None:

        where = (~np.ma.getmaskarray(x) & ~np.ma.getmaskarray(y1) &

                 ~np.ma.getmaskarray(y2))
…but I’d like more feedback from the author or users of fill_between

before committing that change. (That, and Eric Firing can probably find

a much more efficient way to do the masked array manipulation…

_John_Hunter1 · April 29, 2009, 4:07pm

Here’s the diff I just committed::
Index: lib/matplotlib/axes.py

···

On Wed, Apr 29, 2009 at 11:03 AM, John Hunter <jdh2358@…878…287…> wrote:

…but I’d like more feedback from the author or users of fill_between

before committing that change. (That, and Eric Firing can probably find

a much more efficient way to do the masked array manipulation…

I’m working on a patch for this now…but I would also like Eric to take a look when I am done since I am a masked array dummy

===================================================================
— lib/matplotlib/axes.py (revision 7069)
+++ lib/matplotlib/axes.py (working copy)

@@ -5832,6 +5832,26 @@
self._process_unit_info(xdata=x, ydata=y1, kwargs=kwargs)
self._process_unit_info(ydata=y2)

```
   if where is None:
```

       where = np.ones(len(x), np.bool)

```
   else:
```
```
       where = np.asarray(where)
```

   maskedx = isinstance(x, np.ma.MaskedArray)

   maskedy1 = isinstance(y1, np.ma.MaskedArray)

   maskedy2 = isinstance(y2, np.ma.MaskedArray)

   if (maskedx or maskedy1 or maskedy2):

```
       if maskedx:
```
```
           where = where & (~x.mask)
```
```
       if maskedy1:
```
```
           where = where & (~y1.mask)
```
```
       if maskedy2:
```
```
           where = where & (~y2.mask)
```

   # Convert the arrays so we can work with them
   x = np.asarray(self.convert_xunits(x))
   y1 = np.asarray(self.convert_yunits(y1))

@@ -5843,10 +5863,7 @@
if not cbook.iterable(y2):
y2 = np.ones_like(x)*y2

```
   if where is None:
```

       where = np.ones(len(x), np.bool)

   where = np.asarray(where)

   assert( (len(x)==len(y1)) and (len(x)==len(y2)) and len(x)==len(where))

   polys = []

Eric_Firing1 · April 29, 2009, 7:44pm

John Hunter wrote:

        ...but I'd like more feedback from the author or users of
        fill_between
        before committing that change. (That, and Eric Firing can
        probably find
        a much more efficient way to do the masked array manipulation...

    I'm working on a patch for this now....but I would also like Eric to
    take a look when I am done since I am a masked array dummy

John,

The way you did it looked OK; but since I was looking at it, I went ahead and did some rearranging and condensation to try to make the whole thing (not just the mask handling) read better. I doubt there are any significant performance differences, but I did not test that. I also added a subplot to the fill_between.py demo to illustrate the mask handling. Then I noticed a flaw in the demo plots, and added a note about it. See revision 7071.

Eric

···

On Wed, Apr 29, 2009 at 11:03 AM, John Hunter <jdh2358@...287... > <mailto:jdh2358@…287…>> wrote:

Andres_Luhamaa · April 30, 2009, 3:19pm

John, that's exactly what I wanted, thanks.

Andres

John Hunter wrote:

···

On Wed, Apr 29, 2009 at 10:54 AM, John Hunter <jdh2358@...287... > <mailto:jdh2358@…287…>> wrote:

    fill_between does not currently support masked arrays, but I think
    we could easily extend it to support the mask using the existing
    support for the "where" kwarg. For now, does this behave like you
    expect?

      valid = ~(edatmax.mask & edatmax.mask )
         pylab.fill_between(xtelg,edatmax,edatmin,facecolor='green',alpha='0.3',
    where=valid)

Oops -- meant

valid = ~(edatmin.mask & edatmax.mask )

JDH