A new KNOWNFAIL, pcolormesh.png ?

_Sandro_Tosi3 · January 17, 2011, 7:28pm

Hi,
I was playing with test suite and I noticed that :

morph@...914...:~/deb/build-area/matplotlib-1.0.1$
PYTHONPATH=build/lib.linux-x86_64-2.6 python -c "import matplotlib as
m ; m.test(verbosity=1)"
/usr/lib/pymodules/python2.6/nose/plugins/manager.py:391: UserWarning:
Module matplotlib was already imported from
/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/__init__.pyc,
but /usr/lib/pymodules/python2.6 is being added to sys.path
import pkg_resources
..K........K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..K..KE.K..K..K..K..K..K/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/axes.py:2381:
UserWarning: Attempting to set identical left==right results
in singular transformations; automatically expanding.
left=730139.0, right=730139.0
+ 'left=%s, right=%s') % (left, right))
...K..KK..K..K.....K..K..K..K..K....K..K..K....K..K..K..K..K

···

======================================================================
ERROR: matplotlib.tests.test_axes.test_pcolormesh
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/testing/decorators.py",
line 32, in failer
    result = f(*args, **kwargs)
  File "/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/testing/decorators.py",
line 126, in decorated_compare_images
    '(RMS %(rms).3f)'%err)
ImageComparisonFailure: images not close:
/home/morph/deb/build-area/matplotlib-1.0.1/result_images/test_axes/pcolormesh.png
vs. /home/morph/deb/build-area/matplotlib-1.0.1/result_images/test_axes/expected-pcolormesh.png
(RMS 116.512)

----------------------------------------------------------------------
Ran 150 tests in 107.607s

FAILED (KNOWNFAIL=46, errors=1)

Except for the UserWarning, there is an error in the suite: is someone
else able to replicate it? is it another KNOWNFAIL?

Cheers,
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

Benjamin_Root1 · January 17, 2011, 7:35pm

I have seen this before, and I think it was discussed once before. Visually, there is very little difference, but supposedly there is some sort of issue with different PIL versions. What version of PIL do you have?

Ben Root

···

On Mon, Jan 17, 2011 at 1:28 PM, Sandro Tosi <morph@…12…> wrote:

Hi,

I was playing with test suite and I noticed that :

morph@…914…:~/deb/build-area/matplotlib-1.0.1$

PYTHONPATH=build/lib.linux-x86_64-2.6 python -c "import matplotlib as

m ; m.test(verbosity=1)"

/usr/lib/pymodules/python2.6/nose/plugins/manager.py:391: UserWarning:

Module matplotlib was already imported from

/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/init.pyc,

but /usr/lib/pymodules/python2.6 is being added to sys.path

import pkg_resources

…K…K…K…K…K…K…K…K…K…K…K…K…K…K…K…K…K…K…K…K…K…KE.K…K…K…K…K…K/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/axes.py:2381:

UserWarning: Attempting to set identical left==right results

in singular transformations; automatically expanding.

left=730139.0, right=730139.0

‘left=%s, right=%s’) % (left, right))

…K…KK…K…K…K…K…K…K…K…K…K…K…K…K…K…K…K

======================================================================

ERROR: matplotlib.tests.test_axes.test_pcolormesh

Traceback (most recent call last):

File “/usr/lib/pymodules/python2.6/nose/case.py”, line 183, in runTest
self.test(*self.arg)
File “/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/testing/decorators.py”,

line 32, in failer
result = f(*args, **kwargs)
File “/home/morph/deb/build-area/matplotlib-1.0.1/build/lib.linux-x86_64-2.6/matplotlib/testing/decorators.py”,

line 126, in decorated_compare_images
'(RMS %(rms).3f)'%err)
ImageComparisonFailure: images not close:

/home/morph/deb/build-area/matplotlib-1.0.1/result_images/test_axes/pcolormesh.png

vs. /home/morph/deb/build-area/matplotlib-1.0.1/result_images/test_axes/expected-pcolormesh.png

(RMS 116.512)

Ran 150 tests in 107.607s

FAILED (KNOWNFAIL=46, errors=1)

Except for the UserWarning, there is an error in the suite: is someone

else able to replicate it? is it another KNOWNFAIL?

Cheers,

–

Sandro Tosi (aka morph, morpheus, matrixhasu)

My website: http://matrixhasu.altervista.org/

Me at Debian: http://wiki.debian.org/SandroTosi

_Sandro_Tosi3 · January 17, 2011, 7:44pm

Hi Ben,
thanks for the fast reply!

I have seen this before, and I think it was discussed once before.
Visually, there is very little difference,

indeed, looking at the 2 images, I can't see any difference.

but supposedly there is some sort
of issue with different PIL versions. What version of PIL do you have?

1.1.7

Cheers,

···

On Mon, Jan 17, 2011 at 20:35, Benjamin Root <ben.root@...553...> wrote:
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

Michael_Droettboom · January 18, 2011, 2:00pm

I'm not sure PIL enters into it -- there shouldn't be any code path involving PIL in that case.

I think this a case where the image comparison tolerance needs to be increased. You would do this be passing a "tol" parameter to the image_comparison decorator on the pcolormesh test. The default is 1e-3, but it should be conservatively increased until the test passes. You can perform this experiment yourself, or attach the result image for the test to this list and I can experiment to find a correct value.

Mike

···

On 01/17/2011 02:44 PM, Sandro Tosi wrote:

Hi Ben,
thanks for the fast reply!

On Mon, Jan 17, 2011 at 20:35, Benjamin Root<ben.root@...553...> wrote:

I have seen this before, and I think it was discussed once before.
Visually, there is very little difference,

indeed, looking at the 2 images, I can't see any difference.

but supposedly there is some sort
of issue with different PIL versions. What version of PIL do you have?

1.1.7

Cheers,

_Sandro_Tosi3 · January 18, 2011, 6:29pm

Hi,

···

On Tue, Jan 18, 2011 at 15:00, Michael Droettboom <mdroe@...31...> wrote:

I'm not sure PIL enters into it -- there shouldn't be any code path
involving PIL in that case.

I think this a case where the image comparison tolerance needs to be
increased. You would do this be passing a "tol" parameter to the
image_comparison decorator on the pcolormesh test. The default is 1e-3,
but it should be conservatively increased until the test passes. You
can perform this experiment yourself, or attach the result image for the
test to this list and I can experiment to find a correct value.

I'm attaching the images, just for reference; as you can see, the
difference is very tiny and with a tolerance of 0.02 I was able to
pass the test (RMS Value: 0.0116511801977).

Cheers,
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

Michael_Droettboom · January 18, 2011, 7:00pm

Thanks. I have increased the tolerance in the SVN repository in r 8926/r8927.

Cheers,
Mike

···

On 01/18/2011 01:29 PM, Sandro Tosi wrote:

Hi,

On Tue, Jan 18, 2011 at 15:00, Michael Droettboom<mdroe@...31...> wrote:

I'm not sure PIL enters into it -- there shouldn't be any code path
involving PIL in that case.

I think this a case where the image comparison tolerance needs to be
increased. You would do this be passing a "tol" parameter to the
image_comparison decorator on the pcolormesh test. The default is 1e-3,
but it should be conservatively increased until the test passes. You
can perform this experiment yourself, or attach the result image for the
test to this list and I can experiment to find a correct value.

I'm attaching the images, just for reference; as you can see, the
difference is very tiny and with a tolerance of 0.02 I was able to
pass the test (RMS Value: 0.0116511801977).

Cheers,

Paul_Ivanov2 · January 18, 2011, 7:00pm

Sandro Tosi, on 2011-01-18 19:29, wrote:

Hi,

> I'm not sure PIL enters into it -- there shouldn't be any code path
> involving PIL in that case.
>
> I think this a case where the image comparison tolerance needs to be
> increased. You would do this be passing a "tol" parameter to the
> image_comparison decorator on the pcolormesh test. The default is 1e-3,
> but it should be conservatively increased until the test passes. You
> can perform this experiment yourself, or attach the result image for the
> test to this list and I can experiment to find a correct value.

I'm attaching the images, just for reference; as you can see, the
difference is very tiny and with a tolerance of 0.02 I was able to
pass the test (RMS Value: 0.0116511801977).

I just wanted to chime in that there *is* structure in the
differences between the images - which don't show up on screens
which don't have linearized gamma (the difference between 0 and 1
are much smaller in brightness than the difference between 127
and 128, for example).

A quick way to get around this is to just invert the colors.

in X11, I do this with 'xcalib -a -i' which toggles back to
normal if you run the command twice (Compiz also has it's own
version of this via some keyboard shortcut, IIRC, but I don't use
it). ImageMagick's convert can do this with the -negate flag. On
OS X, there's something like command-F8 to do this.

For completeness, if you're interested in doing this in
matplotlib, just subtract your (floating point 0.0-1.0) color
from (1.,1.,1.) to get the new color.

I'm attaching the difference image, with its colors inverted.

best,

···

On Tue, Jan 18, 2011 at 15:00, Michael Droettboom <mdroe@...31...> wrote:

--
Paul Ivanov
314 address only used for lists, off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7