contour/contourf and masked arrays

The basemap toolkit relies on the 'badmask' parameter in

    > contour/contourf to mask out areas outside the desired map
    > projection region in matplotlib 0.80. It works quite
    > well. Unfortunately, the new masked array support in CVS
    > doesn't (I get all kinds of weird artifacts, especially for
    > contourf but also for contour). Can the old badmask
    > functionality be added back in for 0.81, at least
    > temporarily until the masked array support stabilizes?

I don't have anything to add to this, I'm just replying so I can CC
Eric Firing, who added the masked array support to contour, because I
don't know if he is on the devel list.

JDH

John, Jeff,

Yes, I am on the matplotlib-devel list.

Coincidentally, while walking in to work yesterday, a possible workaround for the long-standing contourf mask bug occurred to me. In the process of testing it, I found that masking was not working as expected for either contour or contourf. I had done a little testing before sending in cntr.c and related changes, but obviously I missed the problem then. The cause should be quite simple and localized. I think I can track it down and come up with a clean fix, so that at worst only the original contourf bug remains (and maybe the workaround for that will, indeed, work). Please allow me a little time to work on this.

Eric

John Hunter wrote:

···

"Jeff" == Jeff Whitaker <jswhit@...196...> writes:

    > The basemap toolkit relies on the 'badmask' parameter in
    > contour/contourf to mask out areas outside the desired map
    > projection region in matplotlib 0.80. It works quite
    > well. Unfortunately, the new masked array support in CVS
    > doesn't (I get all kinds of weird artifacts, especially for
    > contourf but also for contour). Can the old badmask
    > functionality be added back in for 0.81, at least
    > temporarily until the masked array support stabilizes?

I don't have anything to add to this, I'm just replying so I can CC
Eric Firing, who added the masked array support to contour, because I
don't know if he is on the devel list.

JDH

John,

The attached diff contains a bugfix for the masking problem that Jeff found:

diff -Naur cntr.c_as_sent cntr.c > cntr_bugfix.diff

where cntr.c_as_sent is the old version, cntr.c is the corrected version.

I was simply not transferring the information correctly from the mask to the "reg" array that cntr.c uses; I was setting a value, and then accidentally overwriting it with what should have been an initialization.

With this patch, masked arrays should be handled no better and no worse than before the change from gcntr.c to cntr.c; as far as I know, masking works perfectly for line contours, and it works correctly for filled contours under many but not all circumstances. I tried two workarounds for the latter problem:

1) in line 1359 of cntr.c, change the nchunk variable from 30 to a much smaller value.
     long nchunk = 30; /* hardwired for now */
This causes the generation of many small polygons, so it may slow things down quite a bit, and it causes the boundaries to look a little glitchy on the screen (gtkagg); but I haven't done timing tests, and I haven't tried other backends.

2) change the end of ContourSupport._contour_args(), in contour.py:

         self.ax.set_xlim((ma.minimum(x), ma.maximum(x)))
         self.ax.set_ylim((ma.minimum(y), ma.maximum(y)))
         # Workaround for cntr.c bug wrt masked interior regions:
         #if filled:
         # z = ma.masked_array(z.filled(-1e38))
         # It's not clear this is any better than the original bug.
         return (x, y, z, lev)

The workaround is to uncomment the "if filled:" block. This method also works, but can leave some artifacts around the boundary of the bad region.

I am not recommending that either of these workarounds be added to CVS, but if some individual runs into major trouble because of the bug, then either of these methods could be used as a stopgap on an individual basis.

Eric

John Hunter wrote:

cntr_bugfix.diff (657 Bytes)

···

"Jeff" == Jeff Whitaker <jswhit@...196...> writes:

    > The basemap toolkit relies on the 'badmask' parameter in
    > contour/contourf to mask out areas outside the desired map
    > projection region in matplotlib 0.80. It works quite
    > well. Unfortunately, the new masked array support in CVS
    > doesn't (I get all kinds of weird artifacts, especially for
    > contourf but also for contour). Can the old badmask
    > functionality be added back in for 0.81, at least
    > temporarily until the masked array support stabilizes?

I don't have anything to add to this, I'm just replying so I can CC
Eric Firing, who added the masked array support to contour, because I
don't know if he is on the devel list.

JDH

Eric Firing wrote:

John,

The attached diff contains a bugfix for the masking problem that Jeff found:

diff -Naur cntr.c_as_sent cntr.c > cntr_bugfix.diff

where cntr.c_as_sent is the old version, cntr.c is the corrected version.

I was simply not transferring the information correctly from the mask to the "reg" array that cntr.c uses; I was setting a value, and then accidentally overwriting it with what should have been an initialization.

With this patch, masked arrays should be handled no better and no worse than before the change from gcntr.c to cntr.c; as far as I know, masking works perfectly for line contours, and it works correctly for filled contours under many but not all circumstances. I tried two workarounds for the latter problem:

1) in line 1359 of cntr.c, change the nchunk variable from 30 to a much smaller value.
    long nchunk = 30; /* hardwired for now */
This causes the generation of many small polygons, so it may slow things down quite a bit, and it causes the boundaries to look a little glitchy on the screen (gtkagg); but I haven't done timing tests, and I haven't tried other backends.

2) change the end of ContourSupport._contour_args(), in contour.py:

        self.ax.set_xlim((ma.minimum(x), ma.maximum(x)))
        self.ax.set_ylim((ma.minimum(y), ma.maximum(y)))
        # Workaround for cntr.c bug wrt masked interior regions:
        #if filled:
        # z = ma.masked_array(z.filled(-1e38))
        # It's not clear this is any better than the original bug.
        return (x, y, z, lev)

The workaround is to uncomment the "if filled:" block. This method also works, but can leave some artifacts around the boundary of the bad region.

I am not recommending that either of these workarounds be added to CVS, but if some individual runs into major trouble because of the bug, then either of these methods could be used as a stopgap on an individual basis.

Eric

Eric: Thanks - contour indeed now seems to work perfectly with masked arrays, but I still have problems with contourf (see attached ortho_test.png). Unfortunately, neither of your suggested workarounds help, the first makes no difference and the second makes it much worse.

-Jeff

···

--
Jeffrey S. Whitaker Phone : (303)497-6313
NOAA/OAR/CDC R/CDC1 FAX : (303)497-6449
325 Broadway Web : http://www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80305-3328 Office: Skaggs Research Cntr 1D-124

Jeff,

Eric: Thanks - contour indeed now seems to work perfectly with masked arrays, but I still have problems with contourf (see attached ortho_test.png). Unfortunately, neither of your suggested workarounds help, the first makes no difference and the second makes it much worse.
-Jeff

It is not working in your basemap/examples/contour_demo.py, either, so this does not seem to be exclusively a masked array support problem. I hope I can figure it out.

Eric

John, Jeff,

Eric: Thanks - contour indeed now seems to work perfectly with masked arrays, but I still have problems with contourf (see attached ortho_test.png). Unfortunately, neither of your suggested workarounds help, the first makes no difference and the second makes it much worse.
-Jeff

The attached patch against the cntr.c in cvs does the following:

1) Fixes the bug you (Jeff) described above.

2) Adds a print function for debugging, but normally unused.

3) Fixes a minor bug in which space for an array of ints was allocated where only an array of chars was used.

The original masked array contourf bug is still there, but I suspect (and hope!) it will not present a serious problem for basemap. I still want to find and fix it, but that may take a long time.

I want to make some minor cleanups and one API change in contour.py, but I will leave that for a separate message, probably within 24 hours.

Eric

cntr_bugfix2.diff (2.23 KB)

Eric Firing wrote:

John, Jeff,

The attached patch against the cntr.c in cvs does the following:

1) Fixes the bug you (Jeff) described above.

2) Adds a print function for debugging, but normally unused.

3) Fixes a minor bug in which space for an array of ints was allocated where only an array of chars was used.

The original masked array contourf bug is still there, but I suspect (and hope!) it will not present a serious problem for basemap. I still want to find and fix it, but that may take a long time.

I want to make some minor cleanups and one API change in contour.py, but I will leave that for a separate message, probably within 24 hours.

Eric

Eric: The combination of your two patches (cntr_bugfix.diff and cntr_bugfix2.diff) does indeed fix all of the problems with the basemap examples. Many thanks!

-Jeff

···

--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/CDC R/CDC1 Email : Jeffrey.S.Whitaker@...236...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : Jeffrey S. Whitaker: NOAA Physical Sciences Laboratory