Andrew and I just spent a couple of hours on the phone trying to
reconcile our freetype rendering discrepancies, and after much tedious
poor-man's debugging, we finally figured out what was wrong. We
thought we were using the same version of freetype but were not. The
fault was mine: I configured my buildbot to recognize the right
version of freetype at buildtime but not runtime. We've fixed this,
we now both run the same version of freetype, and we are getting
identical mpl images. We are very excited by the sea of green at
http://mpl-buildbot.code.astraw.com/waterfall
Now that we have two bots passing the regression tests, we have big plans:
* enable buildbots for more platforms (solaris and win32, we already
have osx and linux)
* enable buildbots for all supported versions of python (2.4, 2.5 and 2.6)
* make the unit testing infrastructure in the "test" dir easier to
use and extend, so it becomes trivial to drop in a new test (and then
drop in a lot of tests)
* enable tarballs and platform specific binary installers from the
buildbots which upload to a nightly directory that users can grab
stuff from
* send out emails on any buildbot failures to the
matplotlib-buildbot mailing list
In preparation for the last point, I have created
https://lists.sourceforge.net/lists/listinfo/matplotlib-buildbot
so subscribe if you are interested in getting notices of failures.
Andrew is taking a break before he configures the bot to email on
failures, but he'll probably have it activated soon. We'll probably
force a failure by uploading a bad image as a baseline to debug the
email script, so there may be some false negatives coming out on the
list over the next day or two, but once we have everything working the
list should accurately reflect unit test failures.
Thanks to Andrew for pushing this through. It's a big step forward to
go from the poor-man's unit testing that we have in backend_driver to
a real framework doing image diffs across platforms triggered by svn
commits.
JDH