MEP11: Attempting to deal with the Python library dependency issue

Numpy is required at build time (matplotlib C++ extensions need to include Numpy's headers), but I had assumed if it was a requirement, it would get installed first and then we'd be ok to proceed. Is that not how pip works?

Mike

···

On 10/03/2012 05:26 PM, Erik Bray wrote:

On 10/03/2012 12:20 PM, Michael Droettboom wrote:

I invite comments for a new MEP about improving the situation with
respect to our bundling of third-party Python dependencies.

In particular, I'd love feedback from the various stakeholders -- those
producing binary installers and packages for the various platforms.

https://github.com/matplotlib/matplotlib/wiki/MEP11

Mike

+1 Let me know if you need/want any help with any distribute or pip intricacies.

As for Windows, I need to double check, but I believe bdist_wininst installers generated by setuptools/distribute do support installing dependencies from PyPI.

Regarding Numpy: I'm not sure, does Matplotlib require Numpy at build time? I'm guessing probably yes, but if not it can also be installed by easy_install or pip. IIRC easy_install will install one of the pre-built binary eggs. pip only installs/builds from source which can be problematic sometimes. Though I fixed Numpy a while back to be installable by pip so it *does* work so long as Numpy can be built on your system.

Yes/no. Pip does install dependencies first, but they aren't
available at run-time of the setup.py of the dependent
package--unfortunate but mostly true. I've been meaning to see if I
can fix that. And easy_install of course is flawed in that it
installs dependencies afterwards. easy_install does support
setup_requires for downloading and adding packages to the path when
running setup.py, but this does *not* work with Numpy.

So as you wrote in the MEP, Numpy will simply have to be installed
separately, I think, if the C++ modules require the Numpy headers.

Erik

···

On Wed, Oct 3, 2012 at 9:10 PM, Michael Droettboom <mdroe@...31...> wrote:

On 10/03/2012 05:26 PM, Erik Bray wrote:

On 10/03/2012 12:20 PM, Michael Droettboom wrote:

I invite comments for a new MEP about improving the situation with
respect to our bundling of third-party Python dependencies.

In particular, I'd love feedback from the various stakeholders -- those
producing binary installers and packages for the various platforms.

https://github.com/matplotlib/matplotlib/wiki/MEP11

Mike

+1 Let me know if you need/want any help with any distribute or pip
intricacies.

As for Windows, I need to double check, but I believe bdist_wininst
installers generated by setuptools/distribute do support installing
dependencies from PyPI.

Regarding Numpy: I'm not sure, does Matplotlib require Numpy at build
time? I'm guessing probably yes, but if not it can also be installed
by easy_install or pip. IIRC easy_install will install one of the
pre-built binary eggs. pip only installs/builds from source which can
be problematic sometimes. Though I fixed Numpy a while back to be
installable by pip so it *does* work so long as Numpy can be built on
your system.

Numpy is required at build time (matplotlib C++ extensions need to
include Numpy's headers), but I had assumed if it was a requirement, it
would get installed first and then we'd be ok to proceed. Is that not
how pip works?

Mike

Which is totally fine -- MPL requires a bunch of non-python
dependencies (OK, a few) anyway, so no matter how you slice it, you
need to get some stuf set up before you can build MPL.

Ideally, no on e needs to build MPL that would have trouble with this
requirement -- that's what binaries (and nifty linux .deb / .rpm ) are
for.

I personally prefer that dependencies are well documented, but not
necessarily auto-installed -- the auto stuff is just not reliable
enough.

-Chris

···

On Wed, Oct 3, 2012 at 10:36 PM, Erik Bray <erik.m.bray@...149...> wrote:

So as you wrote in the MEP, Numpy will simply have to be installed
separately, I think, if the C++ modules require the Numpy headers.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

So as you wrote in the MEP, Numpy will simply have to be installed
separately, I think, if the C++ modules require the Numpy headers.

Which is totally fine -- MPL requires a bunch of non-python
dependencies (OK, a few) anyway, so no matter how you slice it, you
need to get some stuf set up before you can build MPL.

Ideally, no on e needs to build MPL that would have trouble with this
requirement -- that's what binaries (and nifty linux .deb / .rpm ) are
for.

I personally prefer that dependencies are well documented, but not
necessarily auto-installed -- the auto stuff is just not reliable
enough.

I agree that .deb/.rpm/MacPorts solve this problem better than anything. This is mainly for folks on Windows who want a binary, those on a Mac who want a graphical installer, or those on other Unices who don't have access to the package manager or want to build from source for any myriad of reasons.

I agree that the dependencies should be well-documented and it does need improvement in certain areas.

When you say the "auto stuff" is just not reliable enough, can you be more specific? Not trying to start an argument -- but I've found it much more reliable in the distribute/pip era than in the setuptools/easy_install era, but that's only with myself as a data point. If there's still lingering issues and obstacles, it would be good to know about.

Mike

···

On 10/04/2012 11:56 AM, Chris Barker wrote:

On Wed, Oct 3, 2012 at 10:36 PM, Erik Bray <erik.m.bray@...149...> wrote: