Release schedule for version 1.0.1?

Development matplotlib-devel
1

In article <4CC22964.1050509@...31...>,
Michael Droettboom <mdroe@...31...>
wrote:

> I'm curious when the next release of matplotlib is due.
>
> My application is suffering badly from the issue that an incorrect font
> cache will cause matplotlib to fail (the application mysteriously exits
> partway through startup until the user deletes the font cache).
>
> That problem is allegedly fixed on the trunk and I'm trying to decide
> how best to deal with it. Depending on the timing of 1.0.1 I can decide
> whether it's worth putting in my own workaround, bundling a prerelease
> version of matplotlib or just waiting for the official release.
I'm not sure what the timeframe is on 1.0.1.

What problem with the cache are you referring to? I'm aware of a
problem where if some fonts are moved or removed after the cache is
created matplotlib will crash (and this problem is fixed in the trunk),
but is that really a problem in everyday practice? I'm just curious --
if there's another issue with the cache that I'm not aware of, I'd like
to fix it.

The known problem what I am referring to. Fortunately.

It has proven to be a very serious problem in practice. I bundle
matplotlib into a Mac application and for a significant number of my
users it crashes at startup due to problems with the matplotlib cache
files. The fix is always the same: delete the cache.

Some reasons this has happened
- The user first runs the application from the distribution dmg file
before copying to /Applications
- The user installs it and runs it, but then moves or renames it for
some reason...boom
- The user had an older version of matplotlib installed but then deleted
it for some reason.

Fortunately the fix from the trunk will do the job.

That said, it still seems risky to me that matplotlib insists on using a
shared directory for its cache and matplotlibrc file: that there's no
way to tell matplotlib to put that data somewhere else (e.g. inside the
application bundle). With bundled applications it is quite likely the
user may run multiple versions of matplotlib without even knowing it. If
any of that data is version-dependent then this is a recipe for
mysterious problems.

-- Russell

···

  On 10/22/2010 05:45 PM, Russell E. Owen wrote:

2

It doesn’t really insist on it right? There are MATPLOTLIBDIR and MPLCONFIGDIR environment variables. The former is for the location of mpl-data, and is not really documented well (that I could find, anyway, but I found it in the source code), and MPLCONFIGDIR specifies where your configuration files for matplotlib are. This is where your font cache will be, as well as your matplotlibrc.

You can set these env variables within your code, before import of matplotlib via os.environment.

···

On Mon, Oct 25, 2010 at 4:15 PM, Russell E. Owen <rowen@…552…748…> wrote:

In article <4CC22964.1050509@…31…>,

Michael Droettboom <mdroe@…31…>

wrote:

On 10/22/2010 05:45 PM, Russell E. Owen wrote:

I’m curious when the next release of matplotlib is due.

My application is suffering badly from the issue that an incorrect font

cache will cause matplotlib to fail (the application mysteriously exits

partway through startup until the user deletes the font cache).

That problem is allegedly fixed on the trunk and I’m trying to decide

how best to deal with it. Depending on the timing of 1.0.1 I can decide

whether it’s worth putting in my own workaround, bundling a prerelease

version of matplotlib or just waiting for the official release.

I’m not sure what the timeframe is on 1.0.1.

What problem with the cache are you referring to? I’m aware of a

problem where if some fonts are moved or removed after the cache is

created matplotlib will crash (and this problem is fixed in the trunk),

but is that really a problem in everyday practice? I’m just curious –

if there’s another issue with the cache that I’m not aware of, I’d like

to fix it.

The known problem what I am referring to. Fortunately.

It has proven to be a very serious problem in practice. I bundle

matplotlib into a Mac application and for a significant number of my

users it crashes at startup due to problems with the matplotlib cache

files. The fix is always the same: delete the cache.

Some reasons this has happened

  • The user first runs the application from the distribution dmg file

before copying to /Applications

  • The user installs it and runs it, but then moves or renames it for

some reason…boom

  • The user had an older version of matplotlib installed but then deleted

it for some reason.

Fortunately the fix from the trunk will do the job.

That said, it still seems risky to me that matplotlib insists on using a

shared directory for its cache and matplotlibrc file: that there’s no

way to tell matplotlib to put that data somewhere else (e.g. inside the

application bundle). With bundled applications it is quite likely the

user may run multiple versions of matplotlib without even knowing it. If

any of that data is version-dependent then this is a recipe for

mysterious problems.

–
Daniel Hyams
dhyams@…149…

3

They should be better documented, but the MPLCONFIGDIR is in the FAQ

http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html#matplotlib-directory-location

···

On Mon, Oct 25, 2010 at 3:41 PM, Daniel Hyams <dhyams@...149...> wrote:

It doesn't really insist on it right? There are MATPLOTLIBDIR and
MPLCONFIGDIR environment variables. The former is for the location of
mpl-data, and is not really documented well (that I could find, anyway, but
I found it in the source code), and MPLCONFIGDIR specifies where your
configuration files for matplotlib are. This is where your font cache will
be, as well as your matplotlibrc.

You can set these env variables within your code, before import of
matplotlib via os.environment.

4

Right, I was referring specifically to MATPLOTLIBDIR :wink:

I was just pleased as punch to find it in the source code, documented or no :slight_smile:

···

On Mon, Oct 25, 2010 at 5:06 PM, John Hunter <jdh2358@…714…> wrote:

On Mon, Oct 25, 2010 at 3:41 PM, Daniel Hyams <dhyams@…149…> wrote:

It doesn’t really insist on it right? There are MATPLOTLIBDIR and

MPLCONFIGDIR environment variables. The former is for the location of

mpl-data, and is not really documented well (that I could find, anyway, but

I found it in the source code), and MPLCONFIGDIR specifies where your

configuration files for matplotlib are. This is where your font cache will

be, as well as your matplotlibrc.

You can set these env variables within your code, before import of

matplotlib via os.environment.

They should be better documented, but the MPLCONFIGDIR is in the FAQ

http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html#matplotlib-directory-location

–
Daniel Hyams
dhyams@…149…

5

I'm guessing you mean MATPLOTLIBDATA ? And you're right, it isn't
documented (yet)...

JDH

···

On Mon, Oct 25, 2010 at 4:09 PM, Daniel Hyams <dhyams@...149...> wrote:

Right, I was referring specifically to MATPLOTLIBDIR :wink:

I was just pleased as punch to find it in the source code, documented or no
:slight_smile:

6

I'm glad I've learned about this, and will start doing it with my bundled up apps -- but that does seem pretty un-pythonic -- shouldn't it be possible to set this sort of thing without resorting to that little round trip through environment variables?

Not a big deal, but it feels kludgy.

-Chris

···

On 10/25/10 1:41 PM, Daniel Hyams wrote:

It doesn't really insist on it right? There are MATPLOTLIBDIR and
MPLCONFIGDIR environment variables.

You can set these env variables within your code, before import of
matplotlib via os.environment.

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

7

The MPLCONFIGDIR tells mpl where to find the config file. It's hard
to read the config, no matter how pythonic it is, if you don't know
where it is. Once the config file (matplotlibrc) is located you can
set the datapath with the 'datapath' parameter so you don't need
MATPLOTLIBDATA (which is only used if datapath is not set). From the
rc file:

  # Where your matplotlib data lives if you installed to a non-default
  # location. This is where the matplotlib fonts, bitmaps, etc reside
  #datapath : /home/jdhunter/mpldata

We could use command line args, but mpl is a library and shouldn't
really be using them... Arguably we are already doing too much in the
way of command line args.

JDH

···

On Thu, Oct 28, 2010 at 1:11 PM, Christopher Barker <Chris.Barker@...236...> wrote:

On 10/25/10 1:41 PM, Daniel Hyams wrote:

It doesn't really insist on it right? There are MATPLOTLIBDIR and
MPLCONFIGDIR environment variables.

You can set these env variables within your code, before import of
matplotlib via os.environment.

I'm glad I've learned about this, and will start doing it with my
bundled up apps -- but that does seem pretty un-pythonic -- shouldn't it
be possible to set this sort of thing without resorting to that little
round trip through environment variables?

Not a big deal, but it feels kludgy.

8

Isn’t this a problem with any program/library? Could we go a route where (for *nix) we check the current working directory for the file, then check the user account’s .local directory, then check the /etc directory for the any remaining settings? Maybe for Windows use (dare I say it!) the registry?

(Please excuse the registry idea if this completely flies in the face of modern Windows development practices, I haven’t programmed in Windows since my days of taking AP Computer Science ten years ago).

Ben Root

···

On Thu, Oct 28, 2010 at 1:50 PM, John Hunter <jdh2358@…149…> wrote:

On Thu, Oct 28, 2010 at 1:11 PM, Christopher Barker > > <Chris.Barker@…236…> wrote:

On 10/25/10 1:41 PM, Daniel Hyams wrote:

It doesn’t really insist on it right? There are MATPLOTLIBDIR and

MPLCONFIGDIR environment variables.

You can set these env variables within your code, before import of

matplotlib via os.environment.

I’m glad I’ve learned about this, and will start doing it with my

bundled up apps – but that does seem pretty un-pythonic – shouldn’t it

be possible to set this sort of thing without resorting to that little

round trip through environment variables?

Not a big deal, but it feels kludgy.

The MPLCONFIGDIR tells mpl where to find the config file. It’s hard

to read the config, no matter how pythonic it is, if you don’t know

where it is. Once the config file (matplotlibrc) is located you can

set the datapath with the ‘datapath’ parameter so you don’t need

MATPLOTLIBDATA (which is only used if datapath is not set). From the

rc file:

Where your matplotlib data lives if you installed to a non-default

location. This is where the matplotlib fonts, bitmaps, etc reside

#datapath : /home/jdhunter/mpldata

We could use command line args, but mpl is a library and shouldn’t

really be using them… Arguably we are already doing too much in the

way of command line args.

JDH

9

On Thu, Oct 28, 2010 at 1:11 PM, Christopher Barker

You can set these env variables within your code, before import of
matplotlib via os.environment.

The MPLCONFIGDIR tells mpl where to find the config file. It's hard
to read the config, no matter how pythonic it is, if you don't know
where it is.

OK -- so that does need to be set as the first part of the process.

Once the config file (matplotlibrc) is located you can
set the datapath with the 'datapath' parameter so you don't need
MATPLOTLIBDATA (which is only used if datapath is not set).

So the fontcache will be put/found there? That would solve part of Russell's (and my) issue.

We could use command line args, but mpl is a library and shouldn't
really be using them... Arguably we are already doing too much in the
way of command line args.

I agree -- command line args are not the way to go.

It's an interesting question. You can't call a matplotlib function to do
it because it has to happen before matplotlib is loaded. I suppose there
could be a configuration package to perform the operation.

I actually like that idea. It could even do a bit more, like have the matplotlib.use() function, and who knows how many others.

I've never liked the .matplotlibrc approach -- it makes great sense for an interactive environment, but not so much for embedding MPL in other apps, for all the reason's Russell has laid out.

If there was a mplconfig module that you could import first, and have functions in there where you could set all the defaults the way tyou like them, it would be easier to make self-contained MPL apps that didn't step on each-others toes.

* The font cache uses absolute paths.

perhaps that could be fixed -- how are parameters like that stored now? If it was an attribute of some object, it could be made a property that resolved the path is some nifty way. Or, if there were a mplconfig module, it could figure out the absolute path for the font cache at run time.

* Where to put it?

I had hoped to put it in the bundled application itself, so that it
would be thrown out when the application was thrown out.

I think the official way to do it on OS-X is to put it in ~/Library/Application Support. However, I don't now how that gets cleaned up -- if ever!

I'd like to avoid generating a new MPLCONFIGDIR directory for every
version of my application (or every version of matplotlib).

I suppose, like for any application configuration, you'd want a new version to check the config, and clean it out if it is for an old version (or, even better, update for the new version, but that's a lot of work)

An alternative is to just keep using the default MPLCONFIGDIR
(~/.matplotlib) and put up with the risk of a matplotlibrc file doing
bad things. That's my plan for now since I know few users who bother to
set exotic things in their matplotlibrc files, and my application
already explicitly set the important settings.

I've gotten away with that as well, but I still don't like it!

Is is possible for MPL to read from one place, but write to another? Doesn't that happen already for when MPL first starts up and doesn't have anything?

And it sounds like the fontcache and rc file don't NEED to be in teh same place -- they probably shouldn't be for this type of use.

-Chris

···

On 10/28/10 11:50 AM, John Hunter wrote:
On 10/28/10 12:46 PM, Russell E. Owen wrote:

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

10

In reference to the configuration package idea...

I (and the users that I support) use matplotlib both as a standalone plotter
for generating lots of plots, as an interactive plotter, and as an embedded
plot in an application environment. In all of these instances we have found
the '.matplotlibrc' mechanism to be severely lacking in control of plot
formats. Users will be generating series of plots that are going to be
embedded into documents and therefore want a uniform look and feel to those
plots. Later they want to generate some plots for use on an overhead
projector (which has a much different DPI). Another time they are
interactively plotting on their workstation and still again doing all this
in an embedded form as well.

The solution we have come up with, although a little bit of a hack, is a
configuration/formatter class, where instances are used to manage the
formatting of the plots. Each instance can represent a specific set of
configurations controlling everything from font properties to background
color.

The difference here is that having global defaults is all fine and dandy,
but sometimes just setting a global default is not enough and controlling
the setting as an encapsulated entity that can be passed around and selected
at will can be quite useful.

Just my $0.02

--James Evans

···

On 10/28/10 12:46 PM, Russell E. Owen wrote:

> It's an interesting question. You can't call a matplotlib function to
do
> it because it has to happen before matplotlib is loaded. I suppose
there
> could be a configuration package to perform the operation.

I actually like that idea. It could even do a bit more, like have the
matplotlib.use() function, and who knows how many others.

I've never liked the .matplotlibrc approach -- it makes great sense for
an interactive environment, but not so much for embedding MPL in other
apps, for all the reason's Russell has laid out.

If there was a mplconfig module that you could import first, and have
functions in there where you could set all the defaults the way tyou
like them, it would be easier to make self-contained MPL apps that
didn't step on each-others toes.