Test failure testing binary installer - any clues?

Hi,

I'm testing the binary installer build:

https://travis-ci.org/matthew-brett/mpl-osx-binaries/builds/12703220

and I'm getting a test failure on Python 3.3 (not Python 2.7):

···

======================================================================
FAIL: matplotlib.tests.test_lines.test_invisible_Line_rendering.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/nose/case.py",
line 198, in runTest
    self.test(*self.arg)
  File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/testing/decorators.py",
line 73, in test
    self._func()
  File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/tests/test_lines.py",
line 54, in test_invisible_Line_rendering
    assert_true(slowdown_factor < slowdown_threshold)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1464 tests in 656.822s

Is this a problem? What should I do to debug further?

Cheers,

Matthew

I've never seen that failure before...

I wonder if Pierre Haessig has any thoughts, as the author of that test...

Mike

···

On 10/18/2013 02:11 AM, Matthew Brett wrote:

Hi,

I'm testing the binary installer build:

Travis CI - Test and Deploy with Confidence

and I'm getting a test failure on Python 3.3 (not Python 2.7):

======================================================================
FAIL: matplotlib.tests.test_lines.test_invisible_Line_rendering.test
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/nose/case.py",
line 198, in runTest
     self.test(*self.arg)
   File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/testing/decorators.py",
line 73, in test
     self._func()
   File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/tests/test_lines.py",
line 54, in test_invisible_Line_rendering
     assert_true(slowdown_factor < slowdown_threshold)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1464 tests in 656.822s

Is this a problem? What should I do to debug further?

--
                    _

\/|o _|_ _. _ | | \.__ __|__|_|_ _ _ ._ _
>>(_| |(_|(/_| |_/|(_)(/_|_ |_|_)(_)(_)| | |

Hi,

···

On Fri, Oct 18, 2013 at 4:47 AM, Michael Droettboom <mdroe@...31...> wrote:

On 10/18/2013 02:11 AM, Matthew Brett wrote:

Hi,

I'm testing the binary installer build:

Travis CI - Test and Deploy with Confidence

and I'm getting a test failure on Python 3.3 (not Python 2.7):

======================================================================
FAIL: matplotlib.tests.test_lines.test_invisible_Line_rendering.test
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/nose/case.py",
line 198, in runTest
     self.test(*self.arg)
   File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/testing/decorators.py",
line 73, in test
     self._func()
   File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/tests/test_lines.py",
line 54, in test_invisible_Line_rendering
     assert_true(slowdown_factor < slowdown_threshold)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1464 tests in 656.822s

Is this a problem? What should I do to debug further?

I've never seen that failure before...

I wonder if Pierre Haessig has any thoughts, as the author of that test...

Mike

Thanks. I get the same error running under Python 2.7 on a clean 10.6 machine.

Also I get:

======================================================================
FAIL: matplotlib.tests.test_contour.test_contour_manual_labels.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py",
line 197, in runTest
    self.test(*self.arg)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/testing/decorators.py",
line 40, in failer
    result = f(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/testing/decorators.py",
line 159, in do_test
    '(RMS %(rms).3f)'%err)
ImageComparisonFailure: images not close:
/Users/mb312/mpkg-test/mpl-test/result_images/test_contour/contour_manual_labels.png
vs. /Users/mb312/mpkg-test/mpl-test/result_images/test_contour/contour_manual_labels-expected.png
(RMS 15.521)

The images look identical to me...

Cheers,

Matthew

Can you send me the failed image? If we both agree they are the same, it may just need to have the RMS increased to account for font differences.

Also Cc'ing Pierre about the above issue.

Mike

···

On 10/19/2013 04:24 PM, Matthew Brett wrote:

Hi,

On Fri, Oct 18, 2013 at 4:47 AM, Michael Droettboom <mdroe@...31...> wrote:

On 10/18/2013 02:11 AM, Matthew Brett wrote:

Hi,

I'm testing the binary installer build:

Travis CI - Test and Deploy with Confidence

and I'm getting a test failure on Python 3.3 (not Python 2.7):

======================================================================
FAIL: matplotlib.tests.test_lines.test_invisible_Line_rendering.test
----------------------------------------------------------------------
Traceback (most recent call last):
    File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/nose/case.py",
line 198, in runTest
      self.test(*self.arg)
    File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/testing/decorators.py",
line 73, in test
      self._func()
    File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/matplotlib/tests/test_lines.py",
line 54, in test_invisible_Line_rendering
      assert_true(slowdown_factor < slowdown_threshold)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1464 tests in 656.822s

Is this a problem? What should I do to debug further?

I've never seen that failure before...

I wonder if Pierre Haessig has any thoughts, as the author of that test...

Mike

Thanks. I get the same error running under Python 2.7 on a clean 10.6 machine.

Also I get:

======================================================================
FAIL: matplotlib.tests.test_contour.test_contour_manual_labels.test
----------------------------------------------------------------------
Traceback (most recent call last):
   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py",
line 197, in runTest
     self.test(*self.arg)
   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/testing/decorators.py",
line 40, in failer
     result = f(*args, **kwargs)
   File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/testing/decorators.py",
line 159, in do_test
     '(RMS %(rms).3f)'%err)
ImageComparisonFailure: images not close:
/Users/mb312/mpkg-test/mpl-test/result_images/test_contour/contour_manual_labels.png
vs. /Users/mb312/mpkg-test/mpl-test/result_images/test_contour/contour_manual_labels-expected.png
(RMS 15.521)

The images look identical to me...

--
                    _

\/|o _|_ _. _ | | \.__ __|__|_|_ _ _ ._ _
>>(_| |(_|(/_| |_/|(_)(/_|_ |_|_)(_)(_)| | |