Use of contourf changed?

Hello together,

Since the 1.5 version of matplotlib together with python 3.5, I get
strange results from using the contourf function. What I'm actually
doing is plotting a lot of points in a 2D-space. The density equals the
likelihood of the data, so what I want is to plot the 1Sigma, 2Sigma and
3Sigma areas of the data.
For this purpose, I was using the following line of code:
cs = plt.contourf(hist2D, extent=extent, levels=[L1,L2,L3,Lmin],
linestyles=['-','-','-','-'], colors=['blue','blue','blue'], alpha=0.25)

The problem now is, that with matplotlib 1.4, the resulting image shows
the most dense area in white instead of darker blue, which was the
previous behavior. I did not change my code, I can get the old result
simply by moving back to 1.4.
I link to the two pictures I created, so maybe one of you can explain
me, what changed with the version so that the plot is getting messed up.
V1.4:
http://i.imgur.com/HH7jKBE.png
V1.5:
http://i.imgur.com/0LZ9Tso.png

Thank you very much,
Viktoria

Interesting, but it is hard to tell what is going on without any code or
data. In particular, matplotlib's contourf polygons has always been
"unstacked". In other words, the polygons representing a contour level does
not overlap with any other polygon of another contour level, so I am not
exactly sure how you are getting the behavior you are seeing in either
image.

Now, the default algorithm for contouring did change in v1.5. You can
access the old algorithm by passing `corner_mask='legacy'` as a keyword
argument to contourf() in v1.5. Could you try that and see if at least you
get identical results for v1.5 and v1.4? Note, that keyword argument is
not available in v1.4. This should help us narrow down the source of your
issue, but I wouldn't treat it as a solution because the old algorithm is
slated for removal at some point, and it still doesn't explain the
difference you are seeing with the new contouring algorithm.

Ben

···

On Thu, Nov 12, 2015 at 8:11 AM, Viktoria Schubert via Matplotlib-devel < matplotlib-devel at python.org> wrote:

Hello together,

Since the 1.5 version of matplotlib together with python 3.5, I get
strange results from using the contourf function. What I'm actually
doing is plotting a lot of points in a 2D-space. The density equals the
likelihood of the data, so what I want is to plot the 1Sigma, 2Sigma and
3Sigma areas of the data.
For this purpose, I was using the following line of code:
cs = plt.contourf(hist2D, extent=extent, levels=[L1,L2,L3,Lmin],
linestyles=['-','-','-','-'], colors=['blue','blue','blue'], alpha=0.25)

The problem now is, that with matplotlib 1.4, the resulting image shows
the most dense area in white instead of darker blue, which was the
previous behavior. I did not change my code, I can get the old result
simply by moving back to 1.4.
I link to the two pictures I created, so maybe one of you can explain
me, what changed with the version so that the plot is getting messed up.
V1.4:
http://i.imgur.com/HH7jKBE.png
V1.5:
http://i.imgur.com/0LZ9Tso.png

Thank you very much,
Viktoria
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
https://mail.python.org/mailman/listinfo/matplotlib-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20151112/bb54944e/attachment.html>

Hello,

Yeah, this helps indeed with "corner_mask='legacy' the result is
identical, even though I get a warning:
/usr/lib/python3.5/site-packages/matplotlib/cbook.py:136:
MatplotlibDeprecationWarning: The corner_mask='legacy' attribute was
deprecated in version 1.5. Use corner_mask=False or True instead.
  warnings.warn(message, mplDeprecation, stacklevel=1)

I tried creating a minimal example which recreates the problem with
simpler data. The result does not exactly represent 100% of the
problem, but it is screwed anyway. Again, with the option
corner_mask='legacy' option, it looks just fine.
You can find my example here:
https://paste.kde.org/poyn7upbi

Thank you very much!
Viktoria

2015-11-12 17:09 GMT+01:00 Benjamin Root <ben.v.root at gmail.com>:

···

Interesting, but it is hard to tell what is going on without any code or
data. In particular, matplotlib's contourf polygons has always been
"unstacked". In other words, the polygons representing a contour level does
not overlap with any other polygon of another contour level, so I am not
exactly sure how you are getting the behavior you are seeing in either
image.

Now, the default algorithm for contouring did change in v1.5. You can access
the old algorithm by passing `corner_mask='legacy'` as a keyword argument to
contourf() in v1.5. Could you try that and see if at least you get identical
results for v1.5 and v1.4? Note, that keyword argument is not available in
v1.4. This should help us narrow down the source of your issue, but I
wouldn't treat it as a solution because the old algorithm is slated for
removal at some point, and it still doesn't explain the difference you are
seeing with the new contouring algorithm.

Ben

On Thu, Nov 12, 2015 at 8:11 AM, Viktoria Schubert via Matplotlib-devel > <matplotlib-devel at python.org> wrote:

Hello together,

Since the 1.5 version of matplotlib together with python 3.5, I get
strange results from using the contourf function. What I'm actually
doing is plotting a lot of points in a 2D-space. The density equals the
likelihood of the data, so what I want is to plot the 1Sigma, 2Sigma and
3Sigma areas of the data.
For this purpose, I was using the following line of code:
cs = plt.contourf(hist2D, extent=extent, levels=[L1,L2,L3,Lmin],
linestyles=['-','-','-','-'], colors=['blue','blue','blue'], alpha=0.25)

The problem now is, that with matplotlib 1.4, the resulting image shows
the most dense area in white instead of darker blue, which was the
previous behavior. I did not change my code, I can get the old result
simply by moving back to 1.4.
I link to the two pictures I created, so maybe one of you can explain
me, what changed with the version so that the plot is getting messed up.
V1.4:
http://i.imgur.com/HH7jKBE.png
V1.5:
http://i.imgur.com/0LZ9Tso.png

Thank you very much,
Viktoria
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
https://mail.python.org/mailman/listinfo/matplotlib-devel

This is indeed a bug. Both the legacy code and the new code produce
incorrect output, but differently incorrect.

I have opened an issue on github for this:
https://github.com/matplotlib/matplotlib/issues/5477

Viktoria,
Thanks for reporting this. It is a bug that has been around for a long
time and needs to be dealt with. If you don't want to read through all of
the github issue, which may get a bit technical, the key thing from your
point of view is that even though the v1.4 code gave you what you wanted to
see, it wasn't actually the correct output given what your code asked for.
When we fix the underlying bug, if you keep using the same code you will
get the correct output which will be no good to you! Instead of using

    levels = [L1, L2, L3, Lmin]
    plt.contourf(hist2D, levels=levels, colors=['b', 'b', 'b'])

you should instead use something like

    levels = [L1, L2, L3, Lmin]
    for level in levels:
        plt.contourf(hist2D, levels=[level, np.inf], colors='b']

This will give you the results that you want now, as well as when we fix
the bug.

Ian

···

On 12 November 2015 at 17:34, Viktoria Schubert via Matplotlib-devel < matplotlib-devel at python.org> wrote:

Hello,

Yeah, this helps indeed with "corner_mask='legacy' the result is
identical, even though I get a warning:
/usr/lib/python3.5/site-packages/matplotlib/cbook.py:136:
MatplotlibDeprecationWarning: The corner_mask='legacy' attribute was
deprecated in version 1.5. Use corner_mask=False or True instead.
  warnings.warn(message, mplDeprecation, stacklevel=1)

I tried creating a minimal example which recreates the problem with
simpler data. The result does not exactly represent 100% of the
problem, but it is screwed anyway. Again, with the option
corner_mask='legacy' option, it looks just fine.
You can find my example here:
https://paste.kde.org/poyn7upbi

Thank you very much!
Viktoria

2015-11-12 17:09 GMT+01:00 Benjamin Root <ben.v.root at gmail.com>:
> Interesting, but it is hard to tell what is going on without any code or
> data. In particular, matplotlib's contourf polygons has always been
> "unstacked". In other words, the polygons representing a contour level
does
> not overlap with any other polygon of another contour level, so I am not
> exactly sure how you are getting the behavior you are seeing in either
> image.
>
> Now, the default algorithm for contouring did change in v1.5. You can
access
> the old algorithm by passing `corner_mask='legacy'` as a keyword
argument to
> contourf() in v1.5. Could you try that and see if at least you get
identical
> results for v1.5 and v1.4? Note, that keyword argument is not available
in
> v1.4. This should help us narrow down the source of your issue, but I
> wouldn't treat it as a solution because the old algorithm is slated for
> removal at some point, and it still doesn't explain the difference you
are
> seeing with the new contouring algorithm.
>
> Ben
>
>
> On Thu, Nov 12, 2015 at 8:11 AM, Viktoria Schubert via Matplotlib-devel > > <matplotlib-devel at python.org> wrote:
>>
>> Hello together,
>>
>> Since the 1.5 version of matplotlib together with python 3.5, I get
>> strange results from using the contourf function. What I'm actually
>> doing is plotting a lot of points in a 2D-space. The density equals the
>> likelihood of the data, so what I want is to plot the 1Sigma, 2Sigma and
>> 3Sigma areas of the data.
>> For this purpose, I was using the following line of code:
>> cs = plt.contourf(hist2D, extent=extent, levels=[L1,L2,L3,Lmin],
>> linestyles=['-','-','-','-'], colors=['blue','blue','blue'], alpha=0.25)
>>
>> The problem now is, that with matplotlib 1.4, the resulting image shows
>> the most dense area in white instead of darker blue, which was the
>> previous behavior. I did not change my code, I can get the old result
>> simply by moving back to 1.4.
>> I link to the two pictures I created, so maybe one of you can explain
>> me, what changed with the version so that the plot is getting messed up.
>> V1.4:
>> http://i.imgur.com/HH7jKBE.png
>> V1.5:
>> http://i.imgur.com/0LZ9Tso.png
>>
>> Thank you very much,
>> Viktoria
>> _______________________________________________
>> Matplotlib-devel mailing list
>> Matplotlib-devel at python.org
>> https://mail.python.org/mailman/listinfo/matplotlib-devel
>
>
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
https://mail.python.org/mailman/listinfo/matplotlib-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20151113/42dfdcba/attachment.html>

Hello,

Yeah, this helps indeed with "corner_mask='legacy' the result is
identical, even though I get a warning:
/usr/lib/python3.5/site-packages/matplotlib/cbook.py:136:
MatplotlibDeprecationWarning: The corner_mask='legacy' attribute was
deprecated in version 1.5. Use corner_mask=False or True instead.
   warnings.warn(message, mplDeprecation, stacklevel=1)

I tried creating a minimal example which recreates the problem with
simpler data. The result does not exactly represent 100% of the
problem, but it is screwed anyway. Again, with the option
corner_mask='legacy' option, it looks just fine.
You can find my example here:
https://paste.kde.org/poyn7upbi

Viktoria,

Independently of the contourf bug, I suggest you consider two other
options for 2-D histogram display: pcolormesh, and hexbin. In both
cases, you would use a BoundaryNorm with a ListedColormap. Something
like this (untested):

from matplotlib import colors
cmap = colors.ListedColormap([(1, 1, 1, 1), (0, 0, 1, 0.25), (0, 0, 1,
0.5), (0, 0, 1, 0.75)])
norm = colors.BoundaryNorm([0, Lmin, L1, L2, L3, np.inf], ncolors=cmap.N)

plt.pcolormesh(xedges, yedges, hist2D, norm=norm, cmap=cmap)

An advantage of using pcolormesh or hexbin is that the value of a cell
in a histogram represents the entire cell, not a point, so it makes
sense to color in the whole cell. I think that using contour or contourf
in cases like this gives an inaccurate visual impression unless the
histogram is reasonably smoothly varying on a fine grid.

Although in the example above I have stuck to your use of alpha to
modify the shade of blue, I don't advocate this. Instead, unless you
really need the transparency to achieve a visual effect involving other
plot elements, I suggest modifying the color directly and leaving alpha
as the default (unity). So I would use something like:

cmap = colors.ListedColormap([(1, 1, 1), (0.75, 0.75, 1), (0.5, 0.5, 1),
(0.25, 0.25, 1)])

Eric

···

On 2015/11/12 7:34 AM, Viktoria Schubert via Matplotlib-devel wrote:

Thank you very much!
Viktoria

2015-11-12 17:09 GMT+01:00 Benjamin Root <ben.v.root at gmail.com>:

Interesting, but it is hard to tell what is going on without any code or
data. In particular, matplotlib's contourf polygons has always been
"unstacked". In other words, the polygons representing a contour level does
not overlap with any other polygon of another contour level, so I am not
exactly sure how you are getting the behavior you are seeing in either
image.

Now, the default algorithm for contouring did change in v1.5. You can access
the old algorithm by passing `corner_mask='legacy'` as a keyword argument to
contourf() in v1.5. Could you try that and see if at least you get identical
results for v1.5 and v1.4? Note, that keyword argument is not available in
v1.4. This should help us narrow down the source of your issue, but I
wouldn't treat it as a solution because the old algorithm is slated for
removal at some point, and it still doesn't explain the difference you are
seeing with the new contouring algorithm.

Ben

On Thu, Nov 12, 2015 at 8:11 AM, Viktoria Schubert via Matplotlib-devel >> <matplotlib-devel at python.org> wrote:

Hello together,

Since the 1.5 version of matplotlib together with python 3.5, I get
strange results from using the contourf function. What I'm actually
doing is plotting a lot of points in a 2D-space. The density equals the
likelihood of the data, so what I want is to plot the 1Sigma, 2Sigma and
3Sigma areas of the data.
For this purpose, I was using the following line of code:
cs = plt.contourf(hist2D, extent=extent, levels=[L1,L2,L3,Lmin],
linestyles=['-','-','-','-'], colors=['blue','blue','blue'], alpha=0.25)

The problem now is, that with matplotlib 1.4, the resulting image shows
the most dense area in white instead of darker blue, which was the
previous behavior. I did not change my code, I can get the old result
simply by moving back to 1.4.
I link to the two pictures I created, so maybe one of you can explain
me, what changed with the version so that the plot is getting messed up.
V1.4:
http://i.imgur.com/HH7jKBE.png
V1.5:
http://i.imgur.com/0LZ9Tso.png

Thank you very much,
Viktoria
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
https://mail.python.org/mailman/listinfo/matplotlib-devel

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
https://mail.python.org/mailman/listinfo/matplotlib-devel