matplotlib build options

Development matplotlib-devel
1

I have been updating the logic in our setup.py and setupext.py files, so all
of the build options are now exposed in setup.cfg. This should make it easier
for anyone wishing to distribute matplotlib, like package managers. See
setup.cfg.template for the details.

There are a few changes. The external packages kept in lib/ now report "-mpl"
in their versions. Past version provided by matplotlib will be identified and
updated on future installs. It is also possible to disable the installation
of these external packages, details are listed in setup.cfg.template.

Finally, building of extension modules like gtkagg and gdk can be enabled,
disabled, or attempted regardless of missing dependencies, in which case the
build will fail with a meaningful error message.

Please upgrade your svn repository and report any problems, so they dont show
up after cutting the next release.

Thanks,
Darren

2

I just did a clean build and it went through -- a good start! I tried
enabling the new config with by setting NEWCONFIG = True in
__init__.py, and also enabled in setup.cfg

## Experimental config package support:
enthought.traits = True
configobj = True

I pulled my matplotlibrc file from ~/.matplotlib and dropped in a copy
of site-packages/matplotlib/mpl-data/matplotlib.conf into
~/.matplotlib, and then tried to run simple_plot.py with
--verbose-helpful. Here is the error I got:

johnh@...539...:examples> python simple_plot.py --verbose-helpful
$HOME=/home/titan/johnh
CONFIGDIR=/home/titan/johnh/.matplotlib
matplotlib data path
/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/mpl-data
loaded rc file /home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc
$HOME=/home/titan/johnh
CONFIGDIR=/home/titan/johnh/.matplotlib
matplotlib data path
/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/mpl-data
loaded rc file /home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc
Traceback (most recent call last):
  File "simple_plot.py", line 6, in ?
    from pylab import *
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/pylab.py",
line 1, in ?
    from matplotlib.pylab import *
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/__init__.py",
line 714, in ?
    from config import rcParams, rcdefaults
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/config/__init__.py",
line 10, in ? from mplconfig import rcParams, mplConfig,
save_config, rcdefaults
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/config/mplconfig.py",
line 488, in ?
    filePriority=True)
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/config/tconfig.py",
line 561, in __init__
    self.tconf = configClass(self.fconfCombined,monitor=monitor)
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/config/tconfig.py",
line 425, in __init__
    section = v(sec_config,self,monitor=monitor)
  File "/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/config/tconfig.py",
line 400, in __init__
    raise TConfigInvalidKeyError(m)
matplotlib.config.tconfig.TConfigInvalidKeyError: In config defined in
file: '/home/titan/johnh/.matplotlib/matplotlib.conf'
Error processing section: text
These subsections are invalid : ['math']
Valid subsection names : ['latex']

I am attaching my setup.cfg as well as ~/.matplotlib/matplotlib.conf.
I haven't made any changes to the [text] section that I know of....

Note there is also the somewhat confusing line in the verbose output

loaded rc file /home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/mpl-data/matplotlibrc

Thoughts?

One other minor suggestion: you probably want to include the VERBOSE
setting from setup.py in the cfg file. One of the things we want to
get to here is to allow developers to tweak their local config w/o
risking compromising the svn repository. I am frequently guilty of
this: I may tweak some variable in setup.py to get the build I need,
and then forget to unset it when I commit. If everything in the build
is configurable from a file that doesn't live in svn, we won't have
this problem. In the same vein, if you can make setup.cfg handle the
NEWCONFIG setting, this would be ideal. I realize this is tricky
because at runtime you don't know which config file to read until you
have your config, so don't worry about it if it is too tricky. One
possibility while we are in development testing phase would be assume
NEWCONFIG if ~/.matplotlib/matplotlib.conf exists.

JDH

JDH

matplotlib.conf (14.5 KB)

setup.cfg (2.75 KB)

···

On Nov 12, 2007 4:09 PM, Darren Dale <darren.dale@...143...> wrote:

I have been updating the logic in our setup.py and setupext.py files, so all
of the build options are now exposed in setup.cfg. This should make it easier
for anyone wishing to distribute matplotlib, like package managers. See
setup.cfg.template for the details.

3

> I have been updating the logic in our setup.py and setupext.py files, so
> all of the build options are now exposed in setup.cfg. This should make
> it easier for anyone wishing to distribute matplotlib, like package
> managers. See setup.cfg.template for the details.

I just did a clean build and it went through -- a good start! I tried
enabling the new config with by setting NEWCONFIG = True in
__init__.py, and also enabled in setup.cfg

I was going to mask the NEWCONFIG stuff for the upcoming release, including
providing the following packages:

## Experimental config package support:
enthought.traits = True
configobj = True

I pulled my matplotlibrc file from ~/.matplotlib and dropped in a copy
of site-packages/matplotlib/mpl-data/matplotlib.conf into
~/.matplotlib, and then tried to run simple_plot.py with
--verbose-helpful. Here is the error I got:

[...]

matplotlib.config.tconfig.TConfigInvalidKeyError: In config defined in
file: '/home/titan/johnh/.matplotlib/matplotlib.conf'
Error processing section: text
These subsections are invalid : ['math']
Valid subsection names : ['latex']

This is unrelated to the changes in the setup scripts.

It looks like config.mplconfig has been changed. The section name now is
mathtext, not text.math. It is not possible to autocreate the matplotlib.conf
files at build time, without requiring configobj and enthought.traits as
external dependencies. Instead, the developers should get in the habit of
running mplconfig.py as a script whenever we make changes to mplconfig, in
order to update the default config file (I just did so). This should also be
done immediately before any release that includes the new config package.

I am attaching my setup.cfg as well as ~/.matplotlib/matplotlib.conf.
I haven't made any changes to the [text] section that I know of....

Note there is also the somewhat confusing line in the verbose output

loaded rc file
/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/mpl-data/matpl
otlibrc

When the config package encounters a matplotlibrc file but not a
matplotlib.conf file, it reads the rc file and creates a .conf file. The
message must have been printed during that process.

One other minor suggestion: you probably want to include the VERBOSE
setting from setup.py in the cfg file. One of the things we want to
get to here is to allow developers to tweak their local config w/o
risking compromising the svn repository. I am frequently guilty of
this: I may tweak some variable in setup.py to get the build I need,
and then forget to unset it when I commit. If everything in the build
is configurable from a file that doesn't live in svn, we won't have
this problem. In the same vein, if you can make setup.cfg handle the
NEWCONFIG setting, this would be ideal. I realize this is tricky
because at runtime you don't know which config file to read until you
have your config, so don't worry about it if it is too tricky. One
possibility while we are in development testing phase would be assume
NEWCONFIG if ~/.matplotlib/matplotlib.conf exists.

Done. The last option is disabling compilation of the image extension module.
Should that still be optional?

Darren

···

On Monday 12 November 2007 05:52:55 pm John Hunter wrote:

On Nov 12, 2007 4:09 PM, Darren Dale <darren.dale@...143...> wrote:

4

I altered mplconfig to create a matplotlib.conf.template file, which gets read
and modified at build time like we do with matplotlibrc.template. It's not as
elegant as using the config package machinery to create the default config
file at build time, but I think this will serve our needs. Devs still need to
execute mplconfig.py when we make changes that would affect the default
config file.

I also updated the selection of the default backend at build time:

SVG -> Agg -> TkAgg -> WXAgg -> GTK -> GTKAagg -> selection in setup.cfg

committed in svn-4242.

Darren

···

On Monday 12 November 2007 06:39:28 pm Darren Dale wrote:

On Monday 12 November 2007 05:52:55 pm John Hunter wrote:
> On Nov 12, 2007 4:09 PM, Darren Dale <darren.dale@...143...> wrote:
> > I have been updating the logic in our setup.py and setupext.py files,
> > so all of the build options are now exposed in setup.cfg. This should
> > make it easier for anyone wishing to distribute matplotlib, like
> > package managers. See setup.cfg.template for the details.
>
> I just did a clean build and it went through -- a good start! I tried
> enabling the new config with by setting NEWCONFIG = True in
> __init__.py, and also enabled in setup.cfg

5

I'm not sure we need GTK in that pipeline, since its future status is
in question and I think we'd rather have someone on tkagg or wxagg
than plain vanilla gtk.

I think in the autogen of matplotlib.conf, there are a few things you
could do to make the file much more readable.

* for the enumeration of options, print

    # 'bilinear' or 'nearest' or 'bicubic' or 'spline16' or 'spline36' or 'hanni
    # ng' or 'hamming' or 'hermite' or 'kaiser' or 'quadric' or 'catrom' or 'gau
    # ssian' or 'bessel' or 'mitchell' or 'sinc' or 'lanczos' or 'blackman'

    like

    # bilinear | nearest | bicubic | spline16 ....

* For floating point numbers, use a str converter rather than a repr
converter, eg '%s'

    In [4]: '%s'%0.1
    Out[4]: '0.1'

    In [5]: '%r'%0.1
    Out[5]: '0.10000000000000001'

* I think your wrap algorithm is breaking in the middle of words. You
might look at cbook.wrap

    In [15]: s = 'A regular expression used to determine the amount of
space to remove. It looks for the first sequence of spaces
immediately following the first newline, or at the beginning of the
string.'

    In [16]: print cbook.wrap('', s, 76)
    A regular expression used to determine the amount of space to remove. It
    looks for the first sequence of spaces immediately following the first
    newline, or at the beginning of the string.

JDH

···

On Nov 13, 2007 8:01 AM, Darren Dale <darren.dale@...143...> wrote:

On Monday 12 November 2007 06:39:28 pm Darren Dale wrote:
> On Monday 12 November 2007 05:52:55 pm John Hunter wrote:
> > On Nov 12, 2007 4:09 PM, Darren Dale <darren.dale@...143...> wrote:
> > > I have been updating the logic in our setup.py and setupext.py files,
> > > so all of the build options are now exposed in setup.cfg. This should
> > > make it easier for anyone wishing to distribute matplotlib, like
> > > package managers. See setup.cfg.template for the details.
> >
> > I just did a clean build and it went through -- a good start! I tried
> > enabling the new config with by setting NEWCONFIG = True in
> > __init__.py, and also enabled in setup.cfg

I altered mplconfig to create a matplotlib.conf.template file, which gets read
and modified at build time like we do with matplotlibrc.template. It's not as
elegant as using the config package machinery to create the default config
file at build time, but I think this will serve our needs. Devs still need to
execute mplconfig.py when we make changes that would affect the default
config file.

I also updated the selection of the default backend at build time:

SVG -> Agg -> TkAgg -> WXAgg -> GTK -> GTKAagg -> selection in setup.cfg

6

> > > > I have been updating the logic in our setup.py and setupext.py
> > > > files, so all of the build options are now exposed in setup.cfg.
> > > > This should make it easier for anyone wishing to distribute
> > > > matplotlib, like package managers. See setup.cfg.template for the
> > > > details.
> > >
> > > I just did a clean build and it went through -- a good start! I
> > > tried enabling the new config with by setting NEWCONFIG = True in
> > > __init__.py, and also enabled in setup.cfg
>
> I altered mplconfig to create a matplotlib.conf.template file, which gets
> read and modified at build time like we do with matplotlibrc.template.
> It's not as elegant as using the config package machinery to create the
> default config file at build time, but I think this will serve our needs.
> Devs still need to execute mplconfig.py when we make changes that would
> affect the default config file.
>
> I also updated the selection of the default backend at build time:
>
> SVG -> Agg -> TkAgg -> WXAgg -> GTK -> GTKAagg -> selection in setup.cfg

I'm not sure we need GTK in that pipeline, since its future status is
in question and I think we'd rather have someone on tkagg or wxagg
than plain vanilla gtk.

Done.

I think in the autogen of matplotlib.conf, there are a few things you
could do to make the file much more readable.

* for the enumeration of options, print

    # 'bilinear' or 'nearest' or 'bicubic' or 'spline16' or 'spline36' or
'hanni # ng' or 'hamming' or 'hermite' or 'kaiser' or 'quadric' or 'catrom'
or 'gau # ssian' or 'bessel' or 'mitchell' or 'sinc' or 'lanczos' or
'blackman'

    like

    # bilinear | nearest | bicubic | spline16 ....

That is a good suggestion. However, most of the formatting is done under the
hood, by the Traits package itself. It looks like the logic is spread out
over traits code so it won't be easy to subclass and override the default
behavior.

However, when we define our own trait handlers, like we do with BackendHandler
and BoolHandler, we define an "info" method which controls the formatting. If
we end up committing to using Traits, and we started adding more complex
validation and notification, etc., I think that would be the time to add
additional TraitHandlers to improve the formatting.

* For floating point numbers, use a str converter rather than a repr
converter, eg '%s'

    In [4]: '%s'%0.1
    Out[4]: '0.1'

    In [5]: '%r'%0.1
    Out[5]: '0.10000000000000001'

This is also dictated by Traits.

I'll have to think some more about formatting. Right now, its not obvious to
me that there is a simple solution, aside from defining TraitHandlers.

* I think your wrap algorithm is breaking in the middle of words. You
might look at cbook.wrap

    In [15]: s = 'A regular expression used to determine the amount of
space to remove. It looks for the first sequence of spaces
immediately following the first newline, or at the beginning of the
string.'

    In [16]: print cbook.wrap('', s, 76)
    A regular expression used to determine the amount of space to remove.
It looks for the first sequence of spaces immediately following the first
newline, or at the beginning of the string.

Thank you for noticing this, and for pointing out the solution in cbook. I
updated tconfig in our repository and in ipython1.

Darren

···

On Tuesday 13 November 2007 09:30:52 am John Hunter wrote:

On Nov 13, 2007 8:01 AM, Darren Dale <darren.dale@...143...> wrote:
> On Monday 12 November 2007 06:39:28 pm Darren Dale wrote:
> > On Monday 12 November 2007 05:52:55 pm John Hunter wrote:
> > > On Nov 12, 2007 4:09 PM, Darren Dale <darren.dale@...143...> wrote: