How to decouple non-GUI stuff from the GUI?

Hi. I am finding that matplotlib does not allow me to decouple certain actions from the GUI as much as I’d like.

For example, I have not been able to do any of the following tasks. Some of these tasks are certainly artificial, but they all succinctly illustrate some of the problems I’ve been running into when doing more realistic stuff.

  1. without bringing up any GUI window, generate a list of all the filetypes available to a specific matplotlib installation for saving graphics (e.g. .ps, .svg, .pdf, .png, etc.);

  2. switch back-and-forth between sending graphics objects for display to the GUI and sending them for saving to any of the available filetypes (as defined in (1));

  3. (by far the most important one) without activating any backend, create a graphics object (I explain what I mean by this below).

To clarify what I mean by (3), the best I can do is to give an example from Mathematica. In Mathematica, a Graphics object does not need to be displayed. It can be defined as a collection of Graphics primitives, without ever displaying it in the GUI. One such definition would be something like

In[1]:= g = Graphics[Line[{{1., 2.}, {2., 3.}, {3., 5.}}]];

This is what I mean in (3) by creating a graphics object without displaying it (in the GUI). From a design perspective, this makes a lot of sense, since a Graphics object, as an idea, is entirely independent of whatever mean one chooses to display it. I want my code to reflect this conceptual independence.

Regarding task (2), to make it more concrete, consider this (also artificial) exercise: write a script that iterates over all filetypes identified in (1); at each iteration, generate some random graphic (e.g. of the sort shown here http://matplotlib.sourceforge.net/examples/api/patch_collection.html), display it on the GUI, and save it as a file of the type corresponding to the iteration.

I would very much appreciate some hints/guidance on how to solve them.

As I noted, by far the most important of these tasks is (3). From looking into the matplotlib source code, it looks to me impossible to do this with standard matplotlib functions. But I am very much of a noob with mpl, so I still hope there’s a way.

In the worst case scenario (i.e. (3) can’t be done directly using standard mpl functions), then the only solution that I can think of would involve implementing a separate layer of abstract graphics objects, distinct from mpl’s. This layer would then delegate to mpl the task of displaying the graphics objects whenever the user requests it. Any suggestions on how best to do this would be much appreciated. In particular, I’d like to know what would be the right place to insert this new graphics object layer into the matplotlib objects stack. I think it would be best to make this connection at a level deeper than the axes object, but I’m not quite sure how to do this.

Thanks in advance!

kj

Hi. I am finding that matplotlib does not allow me to decouple
certain actions from the GUI as much as I’d like.

For example, I have not been able to do any of the following
tasks. Some of these tasks are certainly artificial, but they all
succinctly illustrate some of the problems I’ve been running into when
doing more realistic stuff.

  1. without bringing up any GUI window,

Set the backend to a non-GUI backend, such as agg, pdf, ps, svg or
cairo by setting the ‘backend’ rcParam in matplotlibrc, or doing:

import matplotlib

matplotlib.use(“Agg”) # Must be the first call after importing
matplotlib

generate a list of all the filetypes available to a specific
matplotlib installation for saving graphics (e.g. .ps, .svg, .pdf,
.png, etc.);

From a figure object:

figure.canvas.get_supported_filetypes()

returns a dictionary of supported file extensions, with mapping to more
detailed descriptions of each.

  1. switch back-and-forth between sending graphics objects for
    display to the GUI and sending them for saving to any of the available
    filetypes (as defined in (1));

Once set to any of the GUI backends, you can use “show()” and
“savefig()” interchangeably.

  1. (by far the most important one) without activating any
    backend
    , create a graphics object (I explain what I mean by this
    below).

To clarify what I mean by (3), the best I can do is to give an
example from Mathematica. In Mathematica, a Graphics object does not
need to be displayed. It can be defined as a collection of Graphics
primitives, without ever displaying it in the GUI. One such definition
would be something like

In[1]:= g = Graphics[Line[{{1.,
2.}, {2., 3.}, {3., 5.}}]];

This
is what I mean in (3) by creating a graphics object without displaying
it (in the GUI). From a design perspective, this makes a lot of sense,
since a Graphics object, as an idea, is entirely independent of
whatever mean one chooses to display it. I want my code to reflect
this conceptual independence.

Regarding
task (2), to make it more concrete, consider this (also artificial)
exercise: write a script that iterates over all filetypes identified in
(1); at each iteration, generate some random graphic (e.g. of the sort
shown here http://matplotlib.sourceforge.net/examples/api/patch_collection.html),
display it on the GUI, and save it as a file of the type corresponding
to the iteration.

I would
very much appreciate some hints/guidance on how to solve them.

As
I noted, by far the most important of these tasks is (3). From looking
into the matplotlib source code, it looks to me impossible to do this
with standard matplotlib functions. But I am very much of a noob with
mpl, so I still hope there’s a way.

In
the worst case scenario (i.e. (3) can’t be done directly using standard
mpl functions), then the only solution that I can think of would
involve implementing a separate layer of abstract graphics objects,
distinct from mpl’s. This layer would then delegate to mpl the task of
displaying the graphics objects whenever the user requests it. Any
suggestions on how best to do this would be much appreciated. In
particular, I’d like to know what would be the right place to insert
this new graphics object layer into the matplotlib objects stack. I
think it would be best to make this connection at a level deeper than
the axes object, but I’m not quite sure how to do this.

I think you’ll find matplotlib works exactly as you want. I think you
might find this part of the documentation useful:

Mike

···

http://matplotlib.sourceforge.net/faq/installing_faq.html#backends

Thanks
in advance!

kj



Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today


Matplotlib-devel mailing list
http://p.sf.net/sfu/msIE9-sfdev2dev


Matplotlib-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/matplotlib-devel

-- Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

Hi. I am finding that matplotlib does not allow me to decouple
certain actions from the GUI as much as I’d like.

For example, I have not been able to do any of the following
tasks. Some of these tasks are certainly artificial, but they all
succinctly illustrate some of the problems I’ve been running into when
doing more realistic stuff.

  1. without bringing up any GUI window,

Set the backend to a non-GUI backend, such as agg, pdf, ps, svg or
cairo by setting the ‘backend’ rcParam in matplotlibrc, or doing:

import matplotlib

matplotlib.use(“Agg”) # Must be the first call after importing
matplotlib

generate a list of all the filetypes available to a specific
matplotlib installation for saving graphics (e.g. .ps, .svg, .pdf,
.png, etc.);

From a figure object:

figure.canvas.get_supported_filetypes()

returns a dictionary of supported file extensions, with mapping to more
detailed descriptions of each.

Yes, but this solves the problem only for the non-GUI backends. It does not work for GUI backends, or at least for the MacOSX backend, which is the only graphics backend that I’ve managed to get to work (despite of, literally, days trying unsuccessfully to install other ones). With this GUI backend, as far as I know, generating a figure object (even an empty one), causes the window to pop up.

That’s the point of this task: to illustrate that even something like getting a list of supported files, which does not require any graphical output, is nonetheless tightly coupled with displaying the GUI. As far as I can tell, there’s simply no way to get these filetypes for GUI backends without bringing up a window. (I’m assuming here that the MacOSX backend is representative in this respect.)

By the same token, it is not possible to iterate over all the possible backends within the same script to find the filetypes they support, because the call to matplotlib.use(BACKEND) must happen before one imports matplotlib.pyplot, which is required to generate the figure.canvas to interrogate for possible filetypes. This is another illustration of what I refer to a tight coupling with the GUI.

  1. switch back-and-forth between sending graphics objects for
    display to the GUI and sending them for saving to any of the available
    filetypes (as defined in (1));

Once set to any of the GUI backends, you can use “show()” and
“savefig()” interchangeably.

Yes, but only for the filetypes associated with that backend; once the GUI backend is chosen, it blocks any other backend (even a non-GUI one) from loading. I’d argue that this is another example of tight coupling with the GUI, because there’s no reason why making a choice of GUI should restrict the filetypes that can be saved. The problems of displaying a graphic and saving a file are completely orthogonal. Picking a solution for one should have no impact at all on what one chooses for the other.

  1. (by far the most important one) without activating any
    backend
    , create a graphics object (I explain what I mean by this
    below).

To clarify what I mean by (3), the best I can do is to give an
example from Mathematica. In Mathematica, a Graphics object does not
need to be displayed. It can be defined as a collection of Graphics
primitives, without ever displaying it in the GUI. One such definition
would be something like

In[1]:= g = Graphics[Line[{{1.,
2.}, {2., 3.}, {3., 5.}}]];

This
is what I mean in (3) by creating a graphics object without displaying
it (in the GUI). From a design perspective, this makes a lot of sense,
since a Graphics object, as an idea, is entirely independent of
whatever mean one chooses to display it. I want my code to reflect
this conceptual independence.

Regarding
task (2), to make it more concrete, consider this (also artificial)
exercise: write a script that iterates over all filetypes identified in
(1); at each iteration, generate some random graphic (e.g. of the sort
shown here http://matplotlib.sourceforge.net/examples/api/patch_collection.html),
display it on the GUI, and save it as a file of the type corresponding
to the iteration.

I would
very much appreciate some hints/guidance on how to solve them.

As
I noted, by far the most important of these tasks is (3). From looking
into the matplotlib source code, it looks to me impossible to do this
with standard matplotlib functions. But I am very much of a noob with
mpl, so I still hope there’s a way.

In
the worst case scenario (i.e. (3) can’t be done directly using standard
mpl functions), then the only solution that I can think of would
involve implementing a separate layer of abstract graphics objects,
distinct from mpl’s. This layer would then delegate to mpl the task of
displaying the graphics objects whenever the user requests it. Any
suggestions on how best to do this would be much appreciated. In
particular, I’d like to know what would be the right place to insert
this new graphics object layer into the matplotlib objects stack. I
think it would be best to make this connection at a level deeper than
the axes object, but I’m not quite sure how to do this.

I think you’ll find matplotlib works exactly as you want. I think you might find this part of the documentation useful:

http://matplotlib.sourceforge.net/faq/installing_faq.html#backends

I’d read that before (that’s where I learned the little I know about backends).

Still, I don’t see how one does (3). Here’s an example problem:

  1. generate (without bringing up a GUI window!) two different graphics objects, each corresponding to a unit circle;

  2. perform a random translation/scaling transformation on each of the two circles;

  3. combine the two circles into a single graphic object (so that they have a common coordinate system);

  4. finally, display the resulting object.

As far as I can tell, steps 1-3 cannot be done before choosing (and locking in) a specific backend. If this backend is a GUI one (or at least the MacOSX one), then just carrying out (1) will cause a window to pop up.

~kj

···

On Wed, Nov 17, 2010 at 3:52 PM, Michael Droettboom <mdroe@…31…> wrote:

On 11/17/2010 03:41 PM, Kynn Jones wrote:

From a figure object:

figure.canvas.get_supported_filetypes()

returns a dictionary of supported file extensions, with mapping to more
detailed descriptions of each.

Yes,
but this solves the problem only for the non-GUI backends. It does not
work for GUI backends, or at least for the MacOSX backend, which is the
only graphics backend that I’ve managed to get to work (despite of, literally, days trying unsuccessfully to install other ones).

Works for me:

import matplotlib
matplotlib.use(“MacOSX”)
from pylab import *
f = figure()
f.canvas.get_supported_filetypes()
{‘svgz’: ‘Scalable Vector Graphics’, ‘ps’: ‘Postscript’, ‘emf’: ‘Enhanced Metafile’, ‘gif’: ‘Graphics Interchange Format’, ‘svg’: ‘Scalable Vector Graphics’, ‘eps’: ‘Encapsulated Postscript’, ‘jpeg’: ‘JPEG’, ‘raw’: ‘Raw RGBA bitmap’, ‘bmp’: ‘Windows bitmap’, ‘jpg’: ‘JPEG’, ‘rgba’: ‘Raw RGBA bitmap’, ‘tiff’: ‘Tagged Image Format File’, ‘pdf’: ‘Portable Document Format’, ‘tif’: ‘Tagged Image Format File’, ‘png’: ‘Portable Network Graphics’}

This does pop up a window though.
You should be able to use the non-interactive mode to do this without popping up a window. However, the non-interactive mode was never implemented for this backend. At least as I understood it, originally the main purpose of
the non-interactive mode was to speed up drawing. In the MacOSX backend, interactive drawing was always equally fast as non-interactive drawing (nowadays this is also true for the other backends; at least I have not seen any examples of the contrary), which is why non-interactive drawing was not implemented in the MacOSX backend.
In principle the non-interactive mode can also be implemented in this backend. If that would solve your problem, could you open a bug report for it?

–Michiel.

···

— On Wed, 11/17/10, Kynn Jones <kynnjo@…149…> wrote:

Hi. I am finding that matplotlib does not allow me to decouple
certain actions from the GUI as much as I’d like.

For example, I have not been able to do any of the following
tasks. Some of these tasks are certainly artificial, but they all
succinctly illustrate some of the problems I’ve been running into when
doing more realistic stuff.

  1. without bringing up any GUI window,

Set the backend to a non-GUI backend, such as agg, pdf, ps, svg or
cairo by setting the ‘backend’ rcParam in matplotlibrc, or doing:

import matplotlib

matplotlib.use(“Agg”) # Must be the first call after importing
matplotlib

generate a list of all the filetypes available to a specific
matplotlib installation for saving graphics (e.g. .ps, .svg, .pdf,
.png, etc.);

From a figure object:

figure.canvas.get_supported_filetypes()

returns a dictionary of supported file extensions, with mapping to more
detailed descriptions of each.

Yes, but this solves the problem only for the non-GUI backends. It does not work for GUI backends, or at least for the MacOSX backend, which is the only graphics backend that I’ve managed to get to work (despite of, literally, days trying unsuccessfully to install other ones). With this GUI backend, as far as I know, generating a figure object (even an empty one), causes the window to pop up.

Actually, the macosx backend is alone in this matter. This behavior was recently reported and is a bug. The other backends do not pop up a window unless interactive mode is turned on.

That’s the point of this task: to illustrate that even something like getting a list of supported files, which does not require any graphical output, is nonetheless tightly coupled with displaying the GUI. As far as I can tell, there’s simply no way to get these filetypes for GUI backends without bringing up a window. (I’m assuming here that the MacOSX backend is representative in this respect.)

By the same token, it is not possible to iterate over all the possible backends within the same script to find the filetypes they support, because the call to matplotlib.use(BACKEND) must happen before one imports matplotlib.pyplot, which is required to generate the figure.canvas to interrogate for possible filetypes. This is another illustration of what I refer to a tight coupling with the GUI.

There is a switch_backends function in pyplot, but it is experimental and tricky:

http://matplotlib.sourceforge.net/api/pyplot_api.html?highlight=switch_backend#matplotlib.pyplot.switch_backend

I have not checked, but maybe that get_supported_filetypes() could be made intto a static function? That could address your complaint in that
regard because an instance of the backend does not need to be made.

  1. switch back-and-forth between sending graphics objects for
    display to the GUI and sending them for saving to any of the available
    filetypes (as defined in (1));

Once set to any of the GUI backends, you can use “show()” and
“savefig()” interchangeably.

Yes, but only for the filetypes associated with that backend; once the GUI backend is chosen, it blocks any other backend (even a non-GUI one) from loading. I’d argue that this is another example of tight coupling with the GUI, because there’s no reason why making a choice of GUI should restrict the filetypes that can be saved. The problems of displaying a graphic and saving a file are completely orthogonal. Picking a solution for one should have no impact at all on what one chooses for the other.

I would argue that it is because no one has asked for this feature before. Necessity is the mother of all inventions. We have a very basic switch_backends function, but maybe you could help us make it better?

The reason for the different backends having different supported filetypes is that some of the backends are speciality backends (“pdf”, for example) and obviously only supports the pdf file type. In some other situations, the saving of the figure data is off-loaded to convenience functions in the various gui toolkits. The main reason for this is that those gui’s provide the proper clipping and other rendering tricks needed for proper production of the images. I am not as familiar with exactly what happens in the backends, but as far as I know, the available file formats are the same across all the major gui backends and Agg. It is only the specialty backends that this is not the case.

If you find a discrepency, it is probably a bug and should be reported as such.

  1. (by far the most important one) without activating any
    backend
    , create a graphics object (I explain what I mean by this
    below).

To clarify what I mean by (3), the best I can do is to give an
example from Mathematica. In Mathematica, a Graphics object does not
need to be displayed. It can be defined as a collection of Graphics
primitives, without ever displaying it in the GUI. One such definition
would be something like

In[1]:= g = Graphics[Line[{{1.,
2.}, {2., 3.}, {3., 5.}}]];

This
is what I mean in (3) by creating a graphics object without displaying
it (in the GUI). From a design perspective, this makes a lot of sense,
since a Graphics object, as an idea, is entirely independent of
whatever mean one chooses to display it. I want my code to reflect
this conceptual independence.

Regarding
task (2), to make it more concrete, consider this (also artificial)
exercise: write a script that iterates over all filetypes identified in
(1); at each iteration, generate some random graphic (e.g. of the sort
shown here http://matplotlib.sourceforge.net/examples/api/patch_collection.html),
display it on the GUI, and save it as a file of the type corresponding
to the iteration.

I would
very much appreciate some hints/guidance on how to solve them.

As
I noted, by far the most important of these tasks is (3). From looking
into the matplotlib source code, it looks to me impossible to do this
with standard matplotlib functions. But I am very much of a noob with
mpl, so I still hope there’s a way.

In
the worst case scenario (i.e. (3) can’t be done directly using standard
mpl functions), then the only solution that I can think of would
involve implementing a separate layer of abstract graphics objects,
distinct from mpl’s. This layer would then delegate to mpl the task of
displaying the graphics objects whenever the user requests it. Any
suggestions on how best to do this would be much appreciated. In
particular, I’d like to know what would be the right place to insert
this new graphics object layer into the matplotlib objects stack. I
think it would be best to make this connection at a level deeper than
the axes object, but I’m not quite sure how to do this.

I think you’ll find matplotlib works exactly as you want. I think you might find this part of the documentation useful:

http://matplotlib.sourceforge.net/faq/installing_faq.html#backends

I’d read that before (that’s where I learned the little I know about backends).

Still, I don’t see how one does (3). Here’s an example problem:

  1. generate (without bringing up a GUI window!) two different graphics objects, each corresponding to a unit circle;
  1. perform a random translation/scaling transformation on each of the two circles;
  1. combine the two circles into a single graphic object (so that they have a common coordinate system);
  1. finally, display the resulting object.

As far as I can tell, steps 1-3 cannot be done before choosing (and locking in) a specific backend. If this backend is a GUI one (or at least the MacOSX one), then just carrying out (1) will cause a window to pop up.

As already covered above, the macosx backend is the only one to do this, and it is a bug. Therefore, while backend has to be chosen at load-time, this is not an issue, as it is not used until later.

To do what you want, you might want to take a look at these:

http://matplotlib.sourceforge.net/users/path_tutorial.html
http://matplotlib.sourceforge.net/users/artists.html

And maybe some stuff in the examples library. Matplotlib makes extensive use of various artist objects, and collections of those objects to represent the components of the figure to be plotted. So, you could create Circle patch objects and others (before you make a call to figure()), and then add them to an axes when ready.

As an additional note, if you are having difficulty compiling for MacOS X, why not just ask for help with that?

Ben Root

···

On Wed, Nov 17, 2010 at 4:54 PM, Kynn Jones <kynnjo@…149…> wrote:

On Wed, Nov 17, 2010 at 3:52 PM, Michael Droettboom <mdroe@…31…> wrote:

On 11/17/2010 03:41 PM, Kynn Jones wrote:

As an additional note, if you are having difficulty compiling for MacOS X, why not just ask for help with that?

yup -- there are some issues with which Tk is used by tkInter, but wx should be easy -- how have you tried to install?

-Chris

As an additional note, if you are having difficulty compiling for MacOS X, why not just ask for help with that?

Just to keep from ranting like a lunatic, basically. The experience was traumatic enough to shake my faith in Python altogether, and made me miss good ol’ CPAN.

To summarize 2-3 day’s worth of frustration:

GTK, GTKAgg:

ImportError: Gtk* backend requires pygtk to be installed.

GTKCairo:

ImportError: No module named backend_gtkcairo

FltkAgg:

ImportError: No module named fltk

QtAgg:

ImportError: Qt backend requires pyqt to be installed.

Qt4Agg:

ImportError: Warning: formlayout requires PyQt4 >v4.3

WX, WXAgg:

ImportError: Matplotlib backend_wx and backend_wxagg require wxPython >=2.8

  1. configuration of pygtk fails:

checking for PYGOBJECT… configure: error: Package requirements (pygobject-2.0 >= 2.21.3) were not met:

No package ‘pygobject-2.0’ found

  1. configuration of pygobject fails:

checking for PYCAIRO… configure: error: Package requirements (pycairo >= 1.0.2

) were not met:

No package ‘pycairo’ found

  1. installation of pycairo fails (after > 6 hours of trying a variety of approaches); currently, importing cairo produces a fatal error:

% python

Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)

[GCC 4.2.1 (Apple Inc. build 5646)] on darwin

Type “help”, “copyright”, “credits” or “license” for more information.

import cairo

Fatal Python error: Interpreter not initialized (version mismatch?)

zsh: abort python

By this point, I just gave up on getting any more interactive backends for matplotlib. (I reiterate that the above is a barebone’s summary. There’s a lot that I’m omitting.)

As an aside, I have never in my life seen a worse installation system than pycairo’s. If you have not examined it in detail, please do so before reacting to my comments. In particular look at the waf file, and the files it invokes. Also, when you find the latter, do a global search for the number 1337 to get a whiff pycairo’s excremental aroma.

There is no point in choosing Python as one’s programming language if this means that one depends on garbage like pycairo. I think it is unfair to put unsuspecting users of matplotlib through the ordeal of dealing with pycairo’s installation. (And pygtk’s and pygobjects’s are not much better.) We can’t force people to write good Python, but at the very least we should not propagate bad code. Packages like pycairo, pygtk, and pygobjects are substandard, and they should be ostracized by the rest of the Python community until they shape up.

Even putting pycairo aside, the Python package installation system is a disaster. The problems I’ve summarized on this page are par for the course with Python packages. Just getting matplotlib to install (even without most interactive backends) was also a multi-day nightmare…

You see? I’m ranting like a lunatic. Sorry. I’ll stop.

~kj

···

On Wed, Nov 17, 2010 at 6:33 PM, Benjamin Root <ben.root@…553…> wrote:

I think we are more asking what tutorials have you read to help install matplotlib (some are better than others), as well as asking what was the source of your matplotlib installation. In addition, what version of MacOS X are you using and if you are using the built-in python interpreater or installed your own? Did you try the DMG file, or from macports, or from pypi?

There are a variety of issues depending on your Mac system that needs to be sorted out to determine the best way to go about installing everything. The particular sticking point is that Apple supplied their own interperater rather than the standard python interpreater. Unfortunately, this causes problems with numpy (and thus matplotlib). There are other pitfalls that makes things tricky.

Personally, I found using macports to be friendly (although it takes forever because it builds ATLAS…) on my wife’s Snow Leopard (32-bit) computer. Friedrich Romstedt is working on a new dmg build for numpy and matplotlib, I believe, and some of use have some school of hard-knocks experience to help with specific questions.

What we really need is improved documentation for the mac install process.

As for the python packager… tell us something we don’t know. I have heard of some people working on an improved system and is testing it out on numpy, I believe (again, my memory is hazy here). We all know the difficulties of the packaging system, that’s why we are more than willing to help you out. So, let’s start from the beginning and we can help you though. It is possible!

Ben Root

Yes, installing these GUIs on OSX is a mess, particularly GTK. I only
recommend it for the very brave and foolish. If you use the enthought
python distribution for OSX, you should get a working tkagg, qt4agg,
wxagg and macosx (I think)

http://www.enthought.com/products/epd.php

JDH

···

On Thu, Nov 18, 2010 at 4:11 PM, Kynn Jones <kynnjo@...149...> wrote:

On Wed, Nov 17, 2010 at 6:33 PM, Benjamin Root <ben.root@...553...> wrote:

As an additional note, if you are having difficulty compiling for MacOS X,
why not just ask for help with that?

Just to keep from ranting like a lunatic, basically. The experience was
traumatic enough to shake my faith in Python altogether, and made me miss
good ol' CPAN.
To summarize 2-3 day's worth of frustration:

I think we are more asking what tutorials have you read to help install matplotlib (some are better than others)

This was about 6 weeks ago, and I no longer remember the details. I do remember I read a lot of advice on installing matplotlib (and scipy, ipython, etc.), not always consistent.

In addition, what version of MacOS X are you using and if you are using the built-in python interpreater or installed your own?

I’m using 64-bit Snow Leopard. My python is /usr/bin/python, which came with the Snow Leopard installation.

Did you try the DMG file, or from macports, or from pypi?

Don’t recall what I finally ended up with, but I do know that I had to completely scrap the macports install, because my macports got totally wedged (circular dependencies, etc.). I had to uninstall all my macports, and reinstall them from scratch. On this re-install I did not install any Python ports, although python2.6 and a few other python ports were pulled in as dependencies of other ports. Still, even though I have /opt/local/bin/python2.6, I don’t use it.

What we really need is improved documentation for the mac install process.

I beg to differ: what we need is a single official download and installation procedure. Of course, this in turn requires a similarly unitary-no-two-ways-about-it download and installation procedure for everything else that matplotlib depends on, directly or indirectly. Without this, adequate documentation becomes impossible, because there’s no way that the writers of the documentation can possibly test the combinatorial explosion of possibilities.

It’s bad enough that we have to deal with multiple OSs and architectures. We don’t need to pile 31 installation flavors on top of that.

(What really boggles the mind is that of all languages out there it is Python that finds itself in this unholy mess, while wild-eyed TIMTOWTDI Perl hums along with CPAN. I can count with the fingers of one hand (and have a few left over) the number of Perl packages that I use that were not downloaded from CPAN. In contrast, tons of my Python modules come from random places: SourceForge, Google Code, github, individual authors’ websites, you name it. How is this possible? Whatever happened to Python’s “There should be one-- and preferably only one --obvious way to do it”???)

So, let’s start from the beginning and we can help you though.

Your patience puts me to shame. Thank you for the offer. I suppose that I should first clean the slate, and re-install everything? Is there clean way to do this?

~kj

···

On Thu, Nov 18, 2010 at 5:57 PM, Benjamin Root <ben.root@…553…> wrote:

Yes, installing these GUIs on OSX is a mess, particularly GTK. I only
recommend it for the very brave and foolish.

yup -- though wxPython is trivial.

Do make sure you're running the python.org python, though -- that's the one most likely to be supported by various binary packages.

If you use the enthought
python distribution for OSX, you should get a working tkagg, qt4agg,
wxagg and macosx (I think)

I don't know if enthought is worth it for that -- but it sure is if you want VTK. Mayavi, and a few other really ugly packages!

As for the wonders of CPAN:

I've never been a perl user, but the issues with packages like matplotlib on OS-X are due to compiled code that depends on libraries that aren't delivered with the OS, on an OS that can have up to four diffferent hardware architectures, and a pretty rapidly changing API (or ABI anyway). Oh, and it depends on various GUI toolkits that have those same issues.

I can't imagine how a CPAN-like system would help with that at all. Installing pure python packages isn't a problem at all.

-Chris

···

On 11/18/10 3:08 PM, John Hunter wrote:

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

If you use the MacOSX backend, you won’t need pygtk, pycairo, pyqt, pygobject, wx, tcl/tk, or anything they depend on, which was my main motivation for writing this backend. Other than the fact that the MacOSX backend currently does not support the non-interactive mode, it should work for you, doesn’t it?
–Michiel.

···

— On Thu, 11/18/10, Kynn Jones <kynnjo@…149…> wrote:

From: Kynn Jones <kynnjo@…149…>
Subject: Re: [matplotlib-devel] How to decouple non-GUI stuff from the GUI?
To: matplotlib-devel@lists.sourceforge.net
Date: Thursday, November 18, 2010, 5:11 PM

On Wed, Nov 17, 2010 at 6:33 PM, Benjamin Root <ben.root@…553…> wrote:

As an additional note, if you are having difficulty compiling for MacOS X, why not just ask for help with that?

Just to keep from ranting like a lunatic, basically. The experience was traumatic enough to shake my faith in Python altogether, and made me miss good ol’ CPAN.

To summarize 2-3 day’s worth of frustration:

GTK, GTKAgg:

ImportError: Gtk* backend requires pygtk to be installed.

GTKCairo:

ImportError: No module named backend_gtkcairo

FltkAgg:

ImportError: No module named fltk

QtAgg:

ImportError: Qt backend requires pyqt to be installed.

Qt4Agg:

ImportError: Warning: formlayout requires PyQt4 >v4.3

WX, WXAgg:

ImportError: Matplotlib backend_wx and backend_wxagg require wxPython >=2.8

  1. configuration of pygtk fails:

checking for PYGOBJECT… configure: error: Package requirements (pygobject-2.0 >= 2.21.3) were not met:

No package ‘pygobject-2.0’ found

  1. configuration of pygobject fails:

checking for PYCAIRO… configure: error: Package requirements (pycairo >= 1.0.2

) were not met:

No package ‘pycairo’ found

  1. installation of pycairo fails (after > 6 hours of trying a variety of approaches); currently, importing cairo produces a fatal error:

% python

Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)

[GCC 4.2.1 (Apple Inc. build 5646)] on darwin

Type “help”, “copyright”, “credits” or “license” for more information.

import cairo

Fatal Python error: Interpreter not initialized (version mismatch?)

zsh: abort python

By this point, I just gave up on getting any more interactive backends for matplotlib. (I reiterate that the above is a barebone’s summary. There’s a lot that I’m omitting.)

As an aside, I have never in my life seen a worse installation system than pycairo’s. If you have not examined it in detail, please do so before reacting to my comments. In particular look at the waf file, and the files it invokes. Also, when you find the latter, do a global search for the number 1337 to get a whiff pycairo’s excremental aroma.

There is no point in choosing Python as one’s programming language if this means that one depends on garbage like pycairo. I think it is unfair to put unsuspecting users of matplotlib through the ordeal of dealing with pycairo’s installation. (And pygtk’s and pygobjects’s are not much better.) We can’t force people to write good Python, but at the very least we should not propagate bad code. Packages like pycairo, pygtk, and pygobjects are substandard, and they should be ostracized by the rest of the Python community until they shape up.

Even putting pycairo aside, the Python package installation system is a disaster. The problems I’ve summarized on this page are par for the course with Python packages. Just getting matplotlib to install (even without most interactive backends) was also a multi-day nightmare…

You see? I’m ranting like a lunatic. Sorry. I’ll stop.

~kj

-----Inline Attachment Follows-----


Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today
http://p.sf.net/sfu/msIE9-sfdev2dev

-----Inline Attachment Follows-----


Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

In principle the non-interactive mode can also be implemented in this backend. If that would solve your problem, could you open a bug report for it?

How??? I feel so inept: I can’t even find matplotlib’s bug reporting/tracking page! Is there one? If so, it is well hidden. All I was able to find was

http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html?highlight=bug#reporting-problems

which basically says to send an email to the matplotlib list. But didn’t I just do this already? I’m confused.

···

On Wed, Nov 17, 2010 at 6:30 PM, Michiel de Hoon <mjldehoon@…42…> wrote:

On Thu, Nov 18, 2010 at 8:06 PM, Michiel de Hoon <mjldehoon@…744…2…> wrote:

If you use the MacOSX backend, you won’t need pygtk, pycairo, pyqt, pygobject, wx, tcl/tk, or anything they depend on, which was my main motivation for writing this backend. Other than the fact that the MacOSX backend currently does not support the non-interactive mode, it should work for you, doesn’t it?

Actually, getting the non-interactive behavior was the reason I was trying to install a backend different from the MacOSX one. As I described in an earlier post, I want to simulate a perfect separation between the creation and manipulation of graphic objects, and their output (either in a graphical display or by saving them to a file). In other words, with this simulation in place, one should be able to create graphical objects, translate them, scale them, shear them, recombine them, split them up, interrogate them, etc., and finally save these objects to files, without a window ever popping up. In fact, this code should run perfectly well on a terminal without any graphical capabilities at all.

Incidentally, one of the reasons for my difficulties with using matplotlib is 100% conceptual. I just can’t wrap my head around the idea of needing to implement a “non-interactive” mode. (Actually, I to call it “non-GUI”, since it’s perfectly possible to envision an interaction that is entirely text-based.) In my brain, such “non-GUI” mode is the default behavior, because, as I described above, the creation and manipulation of graphic objects logically precedes and is conceptually independent of their output. Therefore, the absolutely simplest system for dealing with such graphic objects would have no output at all: the user would (either through a script, or interactively, through a text-based interface) create and manipulate graphics objects that live in memory and disappear when the process/session is terminated. Of course, the usefulness of such a system would be very limited, but it would certainly be my first step. Only after that I would implement the various ways to output the graphical objects. This is the only ordering of capabilities that makes sense to me.

When I read your message about implementing the non-GUI mode, it turns this simple picture completely on its head, which tells me that matplotlib’s architecture is, at this point, beyond my comprehension. One of the reasons for looking forward to your implementation of the non-GUI mode for the MacOSX backend is that, by studying a diff between this enhanced version and the previous version I may be able to finally understand matplotlib’s basic architecture.

~kj

(The quoting got screwed in my previous post, so I'm re-posting it.
Sorry about that.)

In principle the non-interactive mode can also be implemented in this backend. If that would solve your problem, could you open
a bug report for it?

How??? I feel so inept: I can't even find matplotlib's bug
reporting/tracking page! Is there one? If so, it is well hidden.
All I was able to find was

http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html?highlight=bug#reporting-problems

which basically says to send an email to the matplotlib list. But
didn't I just do this already? I'm confused.

If you use the MacOSX backend, you won't need pygtk, pycairo, pyqt, pygobject, wx, tcl/tk, or anything they depend on, which
was my main motivation for writing this backend. Other than the fact that the MacOSX backend currently does not support the
non-interactive mode, it should work for you, doesn't it?

Actually, getting the non-interactive behavior was the reason I was
trying to install a backend different from the MacOSX one. As I
described in an earlier post, I want to simulate a perfect separation
between the creation and manipulation of graphic objects, and their
output (either in a graphical display or by saving them to a file).
In other words, with this simulation in place, one should be able to
create graphical objects, translate them, scale them, shear them,
recombine them, split them up, interrogate them, etc., and finally
save these objects to files, without a window ever popping up. In
fact, this code should run perfectly well on a terminal without any
graphical capabilities at all.

Incidentally, one of the reasons for my difficulties with using
matplotlib is 100% conceptual. I just can't wrap my head around the
idea of needing to implement a "non-interactive" mode. (Actually, I
to call it "non-GUI", since it's perfectly possible to envision an
interaction that is entirely text-based.) In my brain, such "non-GUI"
mode is the default behavior, because, as I described above, the
creation and manipulation of graphic objects logically precedes and is
conceptually independent of their output. Therefore, the absolutely
simplest system for dealing with such graphic objects would have no
output at all: the user would (either through a script, or
interactively, through a text-based interface) create and manipulate
graphics objects that live in memory and disappear when the
process/session is terminated. Of course, the usefulness of such a
system would be very limited, but it would certainly be my first step.
Only after that I would implement the various ways to output the
graphical objects. This is the only ordering of capabilities that
makes sense to me.

When I read your message about implementing the non-GUI mode, it turns
this simple picture completely on its head, which tells me that
matplotlib's architecture is, at this point, beyond my comprehension.
One of the reasons for looking forward to your implementation of the
non-GUI mode for the MacOSX backend is that, by studying a diff
between this enhanced version and the previous version I may be able
to finally understand matplotlib's basic architecture.

~kj

···

On Wed, Nov 17, 2010 at 6:30 PM, Michiel de Hoon <mjldehoon@...42...> wrote:
On Thu, Nov 18, 2010 at 8:06 PM, Michiel de Hoon <mjldehoon@...42...> wrote:

Take a look at the home page http://matplotlib.sourceforge.net on the right panel there is a box with the title “Need help?” which says :

You can file bugs, patches and feature requests on the
sourceforge
tracker,

but it is a good idea to ping us on the mailing list too.

Sorry this has been so difficult and confusing. I think you will save yourself a lot of pain by using the Enthought Python Distribution I pointed you too. There is a free academic version as well as a trial version.

JDH

···

On Fri, Nov 19, 2010 at 9:10 AM, Kynn Jones <kynnjo@…716…> wrote:

On Wed, Nov 17, 2010 at 6:30 PM, Michiel de Hoon <mjldehoon@…42…> wrote:

In principle the non-interactive mode can also be implemented in this backend. If that would solve your problem, could you open a bug report for it?

How??? I feel so inept: I can’t even find matplotlib’s bug reporting/tracking page! Is there one? If so, it is well hidden. All I was able to find was

http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html?highlight=bug#reporting-problems

The matplotlib code is conceptually divided into three parts: the
  pylab interface is the set of functions provided by matplotlib.pylab
  which allow the user to create plots with code quite similar to
  MATLAB figure generating code (Pyplot tutorial). The matplotlib
  frontend or matplotlib API is the set of classes that do the heavy
  lifting, creating and managing figures, text, lines, plots and so on
  (Artist tutorial). This is an abstract interface that knows nothing
  about output. The backends are device dependent drawing devices, aka
  renderers, that transform the frontend representation to hardcopy or
  a display device (What is a backend?). Example backends: PS creates
  PostScript? hardcopy, SVG creates Scalable Vector Graphics hardcopy,
  Agg creates PNG output using the high quality Anti-Grain Geometry
  library that ships with matplotlib, GTK embeds matplotlib in a Gtk+
  application, GTKAgg uses the Anti-Grain renderer to create a figure
  and embed it a Gtk+ application, and so on for PDF, WxWidgets,
  Tkinter etc.

  matplotlib is used by many people in many different contexts. Some
  people want to automatically generate PostScript files to send to a
  printer or publishers. Others deploy matplotlib on a web application
  server to generate PNG output for inclusion in dynamically-generated
  web pages. Some use matplotlib interactively from the Python shell
  in Tkinter on Windows?. My primary use is to embed matplotlib in a
  Gtk+ EEG application that runs on Windows, Linux and Macintosh OS X.

Elaborating a little bit: the middle part of matplotlib, the artist
hierarchy, is what I think you are looking for. That is where
abstractions like Line2D, Circle and Text live. There is no concept
of a GUI or a render at that abstraction.

http://matplotlib.sourceforge.net/users/artists.html#artist-tutorial

···

On Fri, Nov 19, 2010 at 9:10 AM, Kynn Jones <kynnjo@...149...> wrote:

When I read your message about implementing the non-GUI mode, it turns this simple picture completely on its head, which tells me that matplotlib's architecture is, at this point, beyond my comprehension. One of the reasons for looking forward to your implementation of the non-GUI mode for the MacOSX backend is that, by studying a diff between this enhanced version and the previous version I may be able to finally understand matplotlib's basic architecture.

From the introduction: http://matplotlib.sourceforge.net/users/intro.html::

Exactly, this is the part of matplotlib that Kynn should focus on – as a hierarchy of artist objects. I do think that Kynn would have a somewhat valid argument with regards to how matplotlib is presented, though. For example, the pylab interface is completely “magical” and utterly bypasses any object-oriented-ness. Then, the pyplot interface (which I use most of the time) provides a more OOP-esque interface. However, the documentation and examples for the plotting functions do not emphasize the fact that they return graphing objects, mostly because they get automatically attached to the axes object and it is unnecessary to do anything with those objects. Usually, users just let the functions return those objects without an assignment to anywhere.

While I think pyplot strikes the right balance (for me), I often wonder if the way this interface is presented causes a logical hinderance to fully understanding the final layer of the graphical artist objects and collections. Because the underlying graphical objects are typically ignored by the users and handled automatically, this layer doesn’t receive as much developer attention to make it more versatile and complete.

Just my 2 cents on this issue. Actual value of the thought varies based upon the current value of the dollar in your area…

Ben Root

···

On Fri, Nov 19, 2010 at 9:19 AM, John Hunter <jdh2358@…149…> wrote:

On Fri, Nov 19, 2010 at 9:10 AM, Kynn Jones <kynnjo@…149…> wrote:

When I read your message about implementing the non-GUI mode, it turns this simple picture completely on its head, which tells me that matplotlib’s architecture is, at this point, beyond my comprehension. One of the reasons for looking forward to your implementation of the non-GUI mode for the MacOSX backend is that, by studying a diff between this enhanced version and the previous version I may be able to finally understand matplotlib’s basic architecture.

From the introduction: http://matplotlib.sourceforge.net/users/intro.html::

The matplotlib code is conceptually divided into three parts: the

pylab interface is the set of functions provided by matplotlib.pylab

which allow the user to create plots with code quite similar to

MATLAB figure generating code (Pyplot tutorial). The matplotlib

frontend or matplotlib API is the set of classes that do the heavy

lifting, creating and managing figures, text, lines, plots and so on

(Artist tutorial). This is an abstract interface that knows nothing

about output. The backends are device dependent drawing devices, aka

renderers, that transform the frontend representation to hardcopy or

a display device (What is a backend?). Example backends: PS creates

PostScript? hardcopy, SVG creates Scalable Vector Graphics hardcopy,

Agg creates PNG output using the high quality Anti-Grain Geometry

library that ships with matplotlib, GTK embeds matplotlib in a Gtk+

application, GTKAgg uses the Anti-Grain renderer to create a figure

and embed it a Gtk+ application, and so on for PDF, WxWidgets,

Tkinter etc.

matplotlib is used by many people in many different contexts. Some

people want to automatically generate PostScript files to send to a

printer or publishers. Others deploy matplotlib on a web application

server to generate PNG output for inclusion in dynamically-generated

web pages. Some use matplotlib interactively from the Python shell

in Tkinter on Windows?. My primary use is to embed matplotlib in a

Gtk+ EEG application that runs on Windows, Linux and Macintosh OS X.

Elaborating a little bit: the middle part of matplotlib, the artist

hierarchy, is what I think you are looking for. That is where

abstractions like Line2D, Circle and Text live. There is no concept

of a GUI or a render at that abstraction.

http://matplotlib.sourceforge.net/users/artists.html#artist-tutorial

In other words, with this simulation in place, one should be able to
create graphical objects, translate them, scale them, shear them,
recombine them, split them up, interrogate them, etc., and finally
save these objects to files, without a window ever popping up. In
fact, this code should run perfectly well on a terminal without any
graphical capabilities at all.

This is exactly what the non-gui back-ends are for (agg in particular).

While it would be nice to be able to do all that with a GUI back-end, the point of the GUI is to give you a GUI -- if you don't need one, don't use one.

Incidentally, one of the reasons for my difficulties with using
matplotlib is 100% conceptual. I just can't wrap my head around the
idea of needing to implement a "non-interactive" mode. (Actually, I
to call it "non-GUI", since it's perfectly possible to envision an
interaction that is entirely text-based.)

non-GUI is what the Agg, etc back-ends are for. non-interactive is a different concept -- it's specifically a subset of GUI behavior:

"interactive" means that you are typing things interactively at the command prompt, and thus want to see changes you make to a figure after each change -- exactly the opposite of what you want.

"non-interactive" means that you want to be able to do a number of manipulations to a figure, and only show the result of those changes when you specifically ask to see them. This is kind of similar to what you want, but till really talking about behavior with a GUI back-end.

non-GUI back ends are inherently non-interactive.

I'm not totally sure about how MPL works, but it may need to query teh back-end occasinally when doing layout, etc. That may require the GUI toolkit to be initialized even before anyactual rendering is done. If that's the case, then you simpily never want to use a GUI back-end when you don't have/need a GUI (text terminal, web application, batch scripts, etc).

-Chris

Thanks! I added the bug report:

https://sourceforge.net/tracker/?func=detail&aid=3113191&group_id=80706&atid=560720

~kj

···

On Fri, Nov 19, 2010 at 10:15 AM, John Hunter <jdh2358@...149...> wrote:

On Fri, Nov 19, 2010 at 9:10 AM, Kynn Jones <kynnjo@...149...> wrote:

On Wed, Nov 17, 2010 at 6:30 PM, Michiel de Hoon <mjldehoon@...42...> wrote:

In principle the non-interactive mode can also be implemented in this
backend. If that would solve your problem, could you open a bug report for
it?

How??? I feel so inept: I can't even find matplotlib's bug
reporting/tracking page! Is there one? If so, it is well hidden. All I
was able to find was
http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html?highlight=bug#reporting-problems

Take a look at the home page http://matplotlib.sourceforge.net on the right
panel there is a box with the title "Need help?" which says :

You can file bugs, patches and feature requests on the sourceforge tracker,
but it is a good idea to ping us on the mailing list too.

Thanks, this is a very helpful clarification.

~kj

···

On Fri, Nov 19, 2010 at 5:08 PM, Christopher Barker <Chris.Barker@...236...> wrote:

"interactive" means that you are typing things interactively at the
command prompt, and thus want to see changes you make to a figure after
each change -- exactly the opposite of what you want.

"non-interactive" means that you want to be able to do a number of
manipulations to a figure, and only show the result of those changes
when you specifically ask to see them. This is kind of similar to what
you want, but till really talking about behavior with a GUI back-end.