Hi Russell (and Mike),
Is it useful in the long term to have such a packager? My impression is
that as soon as packaging is more robust we’ll switch to using pip or
easy_install.
First off, sorry for the long email - got a bit carried away 
The summary is that I propose we keep the dmg installer but maybe make it use the system Python for reasons illustrated below.
For the record, I’m using pip / easy_install to install matplotlib from source on my Mac and that has been working fine for a while now. You only need to add pkg-config (and the development tools, obviously) to a virgin Mac system and this is now really easy in the days of Homebrew.
Of course, binary packages have the extra issue of a dependency on the environment for which it is built, which complicates matters for binary eggs (thanks for the reminder of wheel, Mike - definitely something to watch). This is the main reason why I don’t use the Mac installer dmg: it’s built for python.org Python and I prefer to use system Python instead.
(On this note, it would be interesting to find out how matplotlib people get Python on their Mac these days. My gut feel tells me that Homebrew Python will be quite popular these days, followed by EPD / Anaconda and then maybe python.org Python. If you use Homebrew there is now the option of “brew install matplotlib” courtesy of Samuel John, while EPD and Anaconda ship with their own versions of matplotlib, so most of those users are taken care of.)
To answer your original question: I do see a use for a dmg installer in the long term, but one you might not have considered. I picture a Mac user who is not familiar with Python but wants to try out matplotlib (the image of Justin Long saying “Hello, I’m a Mac” somehow comes to mind :-)).
Justin has never heard of easy_install or even a compiler and might not be that comfortable with the Terminal. On the other hand, he is used to installing software by downloading and clicking on a dmg or via the App Store. This is a person who is starting out with these tools and needs as few obstacles as possible to get going. Once he is up and running and likes what he sees, he might be persuaded to install a more full-fledged Python distribution or the rest of the SciPy stack.
As an experiment I put myself in the shoes of Justin. I actually did the steps below on a spare MacBook Pro running Mac OS X 10.7.5 that was unsullied by extra Pythons and rogue matplotlibs and what not.
Someone told me about “matplotlib” (maybe after seeing a plot in a talk or a paper) which led me to matplotlib.org (first Google hit).
I see “Download” and go to the downloads page. I see a bunch of links, including these two under “Latest stable version”:
matplotlib-1.2.1-py2.7-python.org-macosx10.3.dmg
matplotlib-1.2.1-py2.7-python.org-macosx10.6.dmg
Since I am on Lion I guess I have to download the latter (although the fact that it says 10.6 and not 10.7 worries me…). I’m not sure what the rest of the filename means - what is py27-python.org? I download the dmg and open it. I am impatient like most users and click on “Continue”.
Oops, there is a problem. The third “Continue” button is grayed out with an error that says: “matplotlib 1.2.1 can’t be installed on this disk. matplotlib requires System Python 2.7 to install.” [This is ironic because, unbeknownst to Justin, he actually has System Python 2.7 installed…] Time to click on “Go Back”… Aah, Important Information (I kick myself for not reading this): “matplotlib for MacOS X 10.6 or later [cool!] and 64-bit Python 2.7 from python.org (not Apple’s built-in Python)”. So that’s probably what py2.7-python.org refers to. If Justin is patient enough he might also spot the following line: “Before running matplotlib, you must install numpy.”
[Clicking “Go Back” would have been the more useful thing to do in this case. If I had decided to return to matplotlib.org, I might have seen “Need help?” and clicked on the “faq” link and ended up at the OS-X Notes. This mentions “several alternative versions of python” such as EPD, MacPython (yikes, Leopard only!) or python.org. But the installer only works with the latter… Surprisingly enough I could not find any explicit mention in the matplotlib installation docs that you need to install NumPy first. EDIT: Oh wait, it’s well documented here but I can only reach this important page by clicking on “docs” in the toolbar below the page title and spotting the “Installing” link. Maybe the “Download” section on the front page should read “Read the installation instructions first and then visit the matplotlib downloads page.”]
Time to visit python.org. I see “Download” and then notice “Python 2.7.5 Mac OS X 64-bit/32-bit x86-64/i386 Installer (for Mac OS X 10.6 and later [2])”. I download the dmg and click on it. The installation is successful. I go back to the matplotlib dmg and retry the installation - success!
Mmm, what now?
[This is not really matplotlib’s problem, but I found surprisingly few resources that tell you how to start Python on the Mac if you know absolutely nothing about Unix and Terminals and such. If you search for “python mac” on Google you at least get some idea at the first hit. The official Mac usage page is quite technical but mentions that “your best way to get started with Python on Mac OS X is through the IDLE integrated development environment” which at least gets you to a Python prompt.]
As a typical Mac user I expect that something has appeared in /Applications. I see no matplotlib but at least there is a Python 2.7 folder. I click on “Python Launcher” which seems like the obvious place to start. Nothing happens - oh wait, something has started but it doesn’t do much. I click on IDLE and this looks more promising. At least there is a prompt that looks like the examples on the net. Alternatively I have somehow found out how to run Python in the Terminal.
Next issue… I type “import matplotlib” and up comes “ImportError: No module named numpy”. A search reveals www.numpy.org. I click on “Getting NumPy” and end up at the SciPy installation page. It mentions Mac packages but only give an example for using Macports. Otherwise it suggests installing a full Python distribution (What, start from scratch? But I’m so close!).
Argh, this is becoming a PITA. [At this stage Justin might accidentally install a compiler and enter a new world of hurt :-)] Oh well, the interwebs to the rescue. Search for “numpy mac”. The first hit is the aforementioned page. The second hit looks more promising, but contains many conflicting answers (“Lion comes with numpy installed?”). If I’m lucky I might follow the last suggestion which reveals the NumPy SourceForge page. Or I could read the Numpy User Guide (DRAFT) → Building and installing NumPy → Mac OS X which points there too.
I assume I want the latest version (1.7.1) and I stumble upon numpy-1.7.1-py2.7-python.org-macosx10.6.dmg. Hey, that has a familiar and comforting filename! It installs without a hitch and finally “import matplotlib” succeeds and I manage to make a plot from IDLE!
Now I need a drink…
This was quite an eye-opener for me (although I wish I could shut my eyes again quickly and forget this ever happened :-)). Hopefully the typical would-be matplotlib user is not as hapless as Justin, but I suspect that quite a few need guidance. I guess their best chance for happiness is to stumble upon a full-fledged Python distribution but if matplotlib is their entry point to the Python universe they might be in for an adventure ride. Although if I had read the proper matplotlib installation instructions first I might have gone straight to EPD…
I picture the following basic Mac user groups:
-
Black belt: pulls git repositories to get bleeding-edge packages, compiles from scratch, lives in the Terminal, probably uses Homebrew Python or EPD / Anaconda or some custom Python installation
-
Savvy: likes to install packages via pip / easy_install, probably has a compiler, not afraid of Terminal, probably uses Homebrew Python or EPD / Anaconda or maybe even Macports / Fink Python
-
No Fuss: likes to click on a dmg, has no compiler, rarely uses Terminal, possibly has EPD / Anaconda or just system Python
-
Justin: a hapless version of No Fuss
The problem I see with the binary dmg installer is that it is currently aimed somewhere between Savvy and No Fuss users. It won’t help the Black Belt and EPD / Anaconda users and is also not as straightforward as the No Fuss user would have hoped.
This is why I’m wondering whether it would make more sense to base the dmg installer on system Python instead. Since Lion ships with Python 2.7.1, NumPy 1.5.1, libfreetype and libpng, OS X has had the potential since 10.7 to run matplotlib out of the box with no modifications or extra dependencies (although the latest 1.3.0 might throw a spanner in the works again by shedding dateutil, pytz and friends). A No Fuss user like Justin could therefore click on the dmg as he does with all his other software installations and matplotlib will “just work” like the OS X mantra says. Having used system Python extensively for many years I can vouch that it is more than adequate for someone wanting to try out matplotlib and sure is easier to install!
Of course, maintaining the dmg installer is already a big job and your work load is therefore probably the biggest factor in these discussions
Best regards,
Ludwig