Regression tests failing?

I'm noticing that SVN r8269 is failing a great number of regression tests -- with pretty major things like the number of digits in the formatter being different. The buildbot seems to be getting the same failures I am, but I don't see any buildbot e-mails since Wednesday. Does anyone know the source of these errors? It seems to have to do with changes to the formatters.

Mike

Michael Droettboom wrote:

I'm noticing that SVN r8269 is failing a great number of regression tests -- with pretty major things like the number of digits in the formatter being different. The buildbot seems to be getting the same failures I am, but I don't see any buildbot e-mails since Wednesday. Does anyone know the source of these errors? It seems to have to do with changes to the formatters.

I suspect I'm the culprit. I will look into it.

Eric

···

Mike

------------------------------------------------------------------------------
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Michael Droettboom wrote:

I'm noticing that SVN r8269 is failing a great number of regression tests -- with pretty major things like the number of digits in the formatter being different. The buildbot seems to be getting the same failures I am, but I don't see any buildbot e-mails since Wednesday. Does anyone know the source of these errors? It seems to have to do with changes to the formatters.

Mike,

What is the easiest way to run a single test? I want to work on one failure at at time.

Eric

···

Mike

------------------------------------------------------------------------------
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Michael Droettboom wrote:

I'm noticing that SVN r8269 is failing a great number of regression tests -- with pretty major things like the number of digits in the formatter being different. The buildbot seems to be getting the same failures I am, but I don't see any buildbot e-mails since Wednesday. Does anyone know the source of these errors? It seems to have to do with changes to the formatters.

Mike,

What is the easiest way to run a single test? I want to work on one failure at at time.

You can provide a dot-separated path to the module and function, eg. (this is assuming the test is installed):

nosetests matplotlib.tests.test_simplification:test_clipping

Mike

···

On 04/26/2010 02:37 PM, Eric Firing wrote:

Eric

Mike

------------------------------------------------------------------------------

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Michael Droettboom wrote:

I'm noticing that SVN r8269 is failing a great number of regression tests -- with pretty major things like the number of digits in the formatter being different. The buildbot seems to be getting the same failures I am, but I don't see any buildbot e-mails since Wednesday. Does anyone know the source of these errors? It seems to have to do with changes to the formatters.

Mike,

I have fixed all but one test--in some cases, partly by fixing tests and/or the baseline images.

The one exception is figimage. I don't think I have changed anything that would affect that in any obvious way, but certainly I could be wrong, and I haven't looked into it. The immediate problem is that I can't see what the difference is between the baseline and what I am generating now. They look identical to my eye, on my screen. The diff plot shows up as a black square. The png files are different, differing in length by 2 bytes, so something has changed--but what? Given that I can't *see* any difference, I am tempted to just change the baseline plot so that the test will pass, but maybe that would be a mistake.

Building mpl from a version 2 months ago, I am still seeing the figimage failure.

Eric

···

Mike

------------------------------------------------------------------------------
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Thanks, Eric. I can confirm the failure on figimage, but with the same seemingly miniscule difference.

It looks like Jouni wrote that test... Jouni: could you verify that the difference isn't significant? If it's not, we can just create a new baseline image.

Mike

Eric Firing wrote:

···

Michael Droettboom wrote:

I'm noticing that SVN r8269 is failing a great number of regression tests -- with pretty major things like the number of digits in the formatter being different. The buildbot seems to be getting the same failures I am, but I don't see any buildbot e-mails since Wednesday. Does anyone know the source of these errors? It seems to have to do with changes to the formatters.

Mike,

I have fixed all but one test--in some cases, partly by fixing tests and/or the baseline images.

The one exception is figimage. I don't think I have changed anything that would affect that in any obvious way, but certainly I could be wrong, and I haven't looked into it. The immediate problem is that I can't see what the difference is between the baseline and what I am generating now. They look identical to my eye, on my screen. The diff plot shows up as a black square. The png files are different, differing in length by 2 bytes, so something has changed--but what? Given that I can't *see* any difference, I am tempted to just change the baseline plot so that the test will pass, but maybe that would be a mistake.

Building mpl from a version 2 months ago, I am still seeing the figimage failure.

Eric

Mike

------------------------------------------------------------------------------

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA