basemap installation problem with mpl master

Eric_Firing2 · July 6, 2013, 6:53am

If I do a clean install of mpl master, and then of basemap, basemap lands in dist-packages/mpl_toolkits, as it always has. But now it is not found--I can't import it. It seems that now the *real* mpl_toolkits is cleverly hidden inside an egg directory with a monstrous name, leaving basemap orphaned in a directory with no __init__.py. As a workaround I can symlink it into the egg location. I suspect the real solution will require basemap to use setuptools, correct? I don't know how to do this, so I hope someone who does will submit a PR.

Eric

Damon_McDougall2 · July 6, 2013, 3:32pm

Actually, using the new setuptools isn't adequate, I just tried it locally
on my machine and it still doesn't install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional
dependency to matplotlib and have the user set a flag (off by default) to
pull basemap if it's desired.

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits are installed into the correct place
because they are shipped with matplotlib and installed at the same time.
We could ship basemap with matplotlib too but it's a rather large download.

Best wishes,
Damon

···

On Sat, Jul 6, 2013 at 1:53 AM, Eric Firing <efiring@...229...> wrote:

If I do a clean install of mpl master, and then of basemap, basemap
lands in dist-packages/mpl_toolkits, as it always has. But now it is
not found--I can't import it. It seems that now the *real* mpl_toolkits
is cleverly hidden inside an egg directory with a monstrous name,
leaving basemap orphaned in a directory with no __init__.py. As a
workaround I can symlink it into the egg location. I suspect the real
solution will require basemap to use setuptools, correct? I don't know
how to do this, so I hope someone who does will submit a PR.

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Jeff_Whitaker1 · July 6, 2013, 4:04pm

Damon McDougall

July 6, 2013 9:32
AM

What
if a user decides later that he/she wants to install basemap? Then they would have to reinstall matplotlib? That doesn’t sound reasonable to me.

How about having matplotlib install a symlink to the egg
location?

Why the change to using setuptools by default in the first place?

-Jeff

···

If
I do a clean install of mpl master, and then of basemap, basemap

lands in dist-packages/mpl_toolkits, as it always has. But now it is

not found–I can’t import it. It seems that now the real mpl_toolkits

is cleverly hidden inside an egg directory with a monstrous name,

leaving basemap orphaned in a directory with no init.py. As a

workaround I can symlink it into the egg location. I suspect the real

solution will require basemap to use setuptools, correct? I don’t know

how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn’t adequate, I just tried it locally on my machine and it still doesn’t install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it’s desired

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it’s a rather large download.

Best wishes,

Damon

–

Damon McDougall
http://www.damon-is-a-geek.com
Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin,
TX 78712-1229

Eric_Firing2 · July 6, 2013, 5:20pm

    If I do a clean install of mpl master, and then of basemap, basemap
    lands in dist-packages/mpl_toolkits, as it always has. But now it is
    not found--I can't import it. It seems that now the *real* mpl_toolkits
    is cleverly hidden inside an egg directory with a monstrous name,
    leaving basemap orphaned in a directory with no __init__.py. As a
    workaround I can symlink it into the egg location. I suspect the real
    solution will require basemap to use setuptools, correct? I don't know
    how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn't adequate, I just tried it
locally on my machine and it still doesn't install itself into the
matplotlib egg.

Thank you for giving it a try.

I think the proper solution here is to add basemap as an optional
dependency to matplotlib and have the user set a flag (off by default)
to pull basemap if it's desired.

Does that sound like a reasonable solution?

No, unfortunately. First, because fundamentally, matplotlib is a dependency of basemap, not the other way around. Second, because I want to be able to pull basemap from git and do the usual "setup.py build, setup.py install" independently of matplotlib.

It sounds like the only real solution will be for basemap to live as an independent package in dist-packages, and drop the mpl_toolkits umbrella entirely. I don't see that it does any good anyway. It seems setuptools has irretrievably broken the usefulness of mpl_toolkits as anything other than a place to put sub-packages that are distributed with mpl, and that live in the same git repo.

Moving basemap out of mpl_toolkits would also simplify the basemap directory structure. I don't see any downside other than the pain of the transition itself, including the problem of user code needing to have every import of basemap handle both possible locations.

The setuptools arrangement of having mpl_toolkits hidden in an egg, but still imported as "import mpl_toolkits", seems like a horrible design. I'm also uncomfortable with the new behavior in which the standard command to build and install mpl triggers an avalanche of potential package installations. Oh, well...

Eric

···

On 2013/07/06 5:32 AM, Damon McDougall wrote:

On Sat, Jul 6, 2013 at 1:53 AM, Eric Firing <efiring@...229... > <mailto:efiring@…229…>> wrote:

P.S. Note that the other mpl_toolkits are installed into the correct
place because they are shipped with matplotlib and installed at the same
time. We could ship basemap with matplotlib too but it's a rather large
download.

Best wishes,
Damon

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Damon_McDougall2 · July 6, 2013, 5:20pm

Damon McDougall <damon.mcdougall@...149...>
July 6, 2013 9:32 AM

If I do a clean install of mpl master, and then of basemap, basemap

lands in dist-packages/mpl_toolkits, as it always has. But now it is
not found--I can't import it. It seems that now the *real* mpl_toolkits
is cleverly hidden inside an egg directory with a monstrous name,
leaving basemap orphaned in a directory with no __init__.py. As a
workaround I can symlink it into the egg location. I suspect the real
solution will require basemap to use setuptools, correct? I don't know
how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn't adequate, I just tried it locally
on my machine and it still doesn't install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional
dependency to matplotlib and have the user set a flag (off by default) to
pull basemap if it's desired

Does that sound like a reasonable solution?

What if a user decides later that he/she wants to install basemap? Then
they would have to reinstall matplotlib? That doesn't sound reasonable to
me.

Actually, on reflection, I'm in agreement with you. I'm comfortable
installing from source but this poses a larger problem when users download
the basemap binary and expect it to Just Work.

How about having matplotlib install a symlink to the egg location?

If there's a way for setuptools to modify the matplotlib egg to add a
symlink, then it must be possible for setuptools to just put the files
there. I just haven't figured out how to do that.

Why the change to using setuptools by default in the first place?

Long story. The short story is that distutils was merged into setuptools.
So setuptools is now the recommended way to install python packages.

···

On Sat, Jul 6, 2013 at 11:04 AM, Jeff Whitaker <jswhit@...196...> wrote:

-Jeff

P.S. Note that the other mpl_toolkits are installed into the correct
place because they are shipped with matplotlib and installed at the same
time. We could ship basemap with matplotlib too but it's a rather large
download.

Best wishes,
Damon

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229
Eric Firing <efiring@...229...>
July 6, 2013 12:53 AM
If I do a clean install of mpl master, and then of basemap, basemap lands
in dist-packages/mpl_toolkits, as it always has. But now it is not
found--I can't import it. It seems that now the *real* mpl_toolkits is
cleverly hidden inside an egg directory with a monstrous name, leaving
basemap orphaned in a directory with no __init__.py. As a workaround I can
symlink it into the egg location. I suspect the real solution will require
basemap to use setuptools, correct? I don't know how to do this, so I hope
someone who does will submit a PR.

Eric

------------------------------

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Thomas_Kluyver · July 6, 2013, 5:26pm

*distribute*, which was a fork of setuptools, was merged into setuptools.
*distutils* is the component in the standard library, and is still there. I
still prefer distutils where possible, precisely because setuptools' eggs
are a mess.

Thomas

···

On 6 July 2013 18:20, Damon McDougall <damon.mcdougall@...149...> wrote:

Long story. The short story is that distutils was merged into setuptools.
So setuptools is now the recommended way to install python packages.

Damon_McDougall2 · July 6, 2013, 5:28pm

Long story. The short story is that distutils was merged into
setuptools. So setuptools is now the recommended way to install python
packages.

*distribute*, which was a fork of setuptools, was merged into setuptools.
*distutils* is the component in the standard library, and is still there. I
still prefer distutils where possible, precisely because setuptools' eggs
are a mess.

Sorry, yes. My mistake.

···

On Sat, Jul 6, 2013 at 12:26 PM, Thomas Kluyver <thomas@...1071...>wrote:

On 6 July 2013 18:20, Damon McDougall <damon.mcdougall@...149...> wrote:

Thomas

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Damon_McDougall2 · July 6, 2013, 5:32pm

    If I do a clean install of mpl master, and then of basemap, basemap
    lands in dist-packages/mpl_toolkits, as it always has. But now it is
    not found--I can't import it. It seems that now the *real*
mpl_toolkits
    is cleverly hidden inside an egg directory with a monstrous name,
    leaving basemap orphaned in a directory with no __init__.py. As a
    workaround I can symlink it into the egg location. I suspect the real
    solution will require basemap to use setuptools, correct? I don't
know
    how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn't adequate, I just tried it
locally on my machine and it still doesn't install itself into the
matplotlib egg.

Thank you for giving it a try.

No worries. All I did was use matplotlib's distribute_setup.py file and
add the lines

from distribute_setup import use_setuptools
use_setuptools()

to setup.py.

I'm sure there's extra setuptools foo I need to make it play nicely with
the matplotlib egg, but I haven't at all looked into it in any detail.

I think the proper solution here is to add basemap as an optional
dependency to matplotlib and have the user set a flag (off by default)
to pull basemap if it's desired.

Does that sound like a reasonable solution?

No, unfortunately. First, because fundamentally, matplotlib is a
dependency of basemap, not the other way around. Second, because I want to
be able to pull basemap from git and do the usual "setup.py build, setup.py
install" independently of matplotlib.

Ah yes, that's entirely reasonable.

It sounds like the only real solution will be for basemap to live as an
independent package in dist-packages, and drop the mpl_toolkits umbrella
entirely. I don't see that it does any good anyway. It seems setuptools
has irretrievably broken the usefulness of mpl_toolkits as anything other
than a place to put sub-packages that are distributed with mpl, and that
live in the same git repo.

That's sounds reasonable to me. But there's a part of me that can't help
thinking that what we're trying to do should be entirely possible. Perhaps
it's more hacky, though.

Moving basemap out of mpl_toolkits would also simplify the basemap
directory structure. I don't see any downside other than the pain of the
transition itself, including the problem of user code needing to have every
import of basemap handle both possible locations.

I'm not against having it as a separate package. We can deprecate the old
location and remove it in 1.5.x, say.

The setuptools arrangement of having mpl_toolkits hidden in an egg, but
still imported as "import mpl_toolkits", seems like a horrible design. I'm
also uncomfortable with the new behavior in which the standard command to
build and install mpl triggers an avalanche of potential package
installations. Oh, well...

Yes, I know. It's a mess. Also notice that it's really hard to downgrade
to maptlotlib version 1.3.x after having installed 1.4.x, because
setuptools creates an egg for each version. In principle this is nice as,
I assume, it offers the flexibility to switch between different matplotlib
versions on the fly. That said, I see no way to actually do this.

···

On Sat, Jul 6, 2013 at 12:20 PM, Eric Firing <efiring@...229...> wrote:

On 2013/07/06 5:32 AM, Damon McDougall wrote:

On Sat, Jul 6, 2013 at 1:53 AM, Eric Firing <efiring@...229... >> <mailto:efiring@…229…>> wrote:

Eric

P.S. Note that the other mpl_toolkits are installed into the correct
place because they are shipped with matplotlib and installed at the same
time. We could ship basemap with matplotlib too but it's a rather large
download.

Best wishes,
Damon

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229

Jeff_Whitaker1 · July 6, 2013, 5:59pm

Thomas Kluyver

July 6, 2013 11:26 AM

I
agree eggs are a mess. Given that it is still easy to have the old behavior, can someone explain why the change was made to have setup.py create eggs by default?

-Jeff

···

distribute, which was a fork of setuptools, was merged into setuptools. distutils is the component in the standard library, and is still there. I still prefer distutils where possible, precisely because setuptools’ eggs are a mess.

Thomas

On Sat, Jul 6, 2013
at 11:04 AM, Jeff Whitaker <jswhit@…196…>
wrote:

Damon McDougall

July 6, 2013 9:32
AM

What
if a user decides later that he/she wants to install basemap? Then they would have to reinstall matplotlib? That doesn’t sound reasonable to me.

Actually,
on reflection, I’m in agreement with you. I’m comfortable installing from source but this poses a larger problem when users download the basemap binary and expect it to Just Work.

How about having matplotlib install a symlink to the egg
location?

If there’s a way for setuptools to modify the matplotlib egg to add a symlink, then it must be possible for setuptools to just put the files there. I just haven’t figured out how to do that.

Why the change to using setuptools by default in the first place?

Long
story. The short story is that distutils was merged into setuptools. So setuptools is now the recommended way to install python packages.

-Jeff

Eric Firing

July 6, 2013 12:53 AM

If
I do a clean install of mpl master, and then of basemap, basemap lands in dist-packages/mpl_toolkits, as it always has. But now it is not found–I can’t import it. It seems that now the real mpl_toolkits
is cleverly hidden inside an egg directory with a monstrous name, leaving basemap orphaned in a directory with no init.py. As a workaround I can symlink it into the egg location. I suspect the real solution will require basemap to use setuptools, correct? I don’t know how to do this, so I hope someone who does will submit a PR.

Eric

–
Damon McDougall
http://www.damon-is-a-geek.com
Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin,
TX 78712-1229

If
I do a clean install of mpl master, and then of basemap, basemap

lands in dist-packages/mpl_toolkits, as it always has. But now it is

not found–I can’t import it. It seems that now the real mpl_toolkits

is cleverly hidden inside an egg directory with a monstrous name,

leaving basemap orphaned in a directory with no init.py. As a

workaround I can symlink it into the egg location. I suspect the real

solution will require basemap to use setuptools, correct? I don’t know

how to do this, so I hope someone who does will submit a PR.

Actually,
using the new setuptools isn’t adequate, I just tried it locally on my machine and it still doesn’t install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it’s desired

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it’s a rather large download.

Best wishes,

Damon

–

Damon McDougall
http://www.damon-is-a-geek.com

Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin

Austin,
TX 78712-1229

If
I do a clean install of mpl master, and then of basemap, basemap

lands in dist-packages/mpl_toolkits, as it always has. But now it is

not found–I can’t import it. It seems that now the real mpl_toolkits

is cleverly hidden inside an egg directory with a monstrous name,

leaving basemap orphaned in a directory with no init.py. As a

workaround I can symlink it into the egg location. I suspect the real

solution will require basemap to use setuptools, correct? I don’t know

how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn’t adequate, I just tried it locally on my machine and it still doesn’t install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it’s desired

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it’s a rather large download.

Best wishes,

Damon

–

Damon McDougall
http://www.damon-is-a-geek.com

Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin

Austin,
TX 78712-1229

If I do a clean install of mpl master, and then of basemap, basemap

lands in dist-packages/mpl_toolkits, as it always has. But now it is

not found–I can’t import it. It seems that now the real mpl_toolkits

is cleverly hidden inside an egg directory with a monstrous name,

leaving basemap orphaned in a directory with no init.py. As a

workaround I can symlink it into the egg location. I suspect the real

solution will require basemap to use setuptools, correct? I don’t know

how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn’t adequate, I just tried it locally on my machine and it still doesn’t install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it’s desired.

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits
are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it’s a rather large download.

Best wishes,

Damon

–

Damon McDougall
http://www.damon-is-a-geek.com
Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin,
TX 78712-1229

Benjamin_Root1 · July 6, 2013, 6:43pm

I have managed once before to get setup tools to install a package of mine as a directory instead of an egg, but I was never quite sure how I did that (I accidentally tripped some sort of test by doing “foo.file” somewhere in my code, I think).

In any case, isn’t this the same reason why the scikit packages are now no longer installed the way they are?

Ben Root

···

distribute, which was a fork of setuptools, was merged into setuptools. distutils is the component in the standard library, and is still there. I still prefer distutils where possible, precisely because setuptools’ eggs are a mess.

Thomas

On Sat, Jul 6, 2013
at 11:04 AM, Jeff Whitaker <jswhit@…196…>
wrote:

Damon McDougall

July 6, 2013 9:32
AM

What
if a user decides later that he/she wants to install basemap? Then they would have to reinstall matplotlib? That doesn’t sound reasonable to me.

Actually,
on reflection, I’m in agreement with you. I’m comfortable installing from source but this poses a larger problem when users download the basemap binary and expect it to Just Work.

How about having matplotlib install a symlink to the egg
location?

If there’s a way for setuptools to modify the matplotlib egg to add a symlink, then it must be possible for setuptools to just put the files there. I just haven’t figured out how to do that.

Why the change to using setuptools by default in the first place?

Long
story. The short story is that distutils was merged into setuptools. So setuptools is now the recommended way to install python packages.

-Jeff

Eric Firing

July 6, 2013 12:53 AM

If
I do a clean install of mpl master, and then of basemap, basemap lands in dist-packages/mpl_toolkits, as it always has. But now it is not found–I can’t import it. It seems that now the real mpl_toolkits
is cleverly hidden inside an egg directory with a monstrous name, leaving basemap orphaned in a directory with no init.py. As a workaround I can symlink it into the egg location. I suspect the real solution will require basemap to use setuptools, correct? I don’t know how to do this, so I hope someone who does will submit a PR.

Eric

–
Damon McDougall
http://www.damon-is-a-geek.com
Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin,
TX 78712-1229

If
I do a clean install of mpl master, and then of basemap, basemap

lands in dist-packages/mpl_toolkits, as it always has. But now it is

not found–I can’t import it. It seems that now the real mpl_toolkits

is cleverly hidden inside an egg directory with a monstrous name,

leaving basemap orphaned in a directory with no init.py. As a

workaround I can symlink it into the egg location. I suspect the real

solution will require basemap to use setuptools, correct? I don’t know

how to do this, so I hope someone who does will submit a PR.

Actually,
using the new setuptools isn’t adequate, I just tried it locally on my machine and it still doesn’t install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it’s desired

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it’s a rather large download.

Best wishes,

Damon

–

Damon McDougall
http://www.damon-is-a-geek.com

Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin

Austin,
TX 78712-1229

If
I do a clean install of mpl master, and then of basemap, basemap

lands in dist-packages/mpl_toolkits, as it always has. But now it is

not found–I can’t import it. It seems that now the real mpl_toolkits

is cleverly hidden inside an egg directory with a monstrous name,

leaving basemap orphaned in a directory with no init.py. As a

workaround I can symlink it into the egg location. I suspect the real

solution will require basemap to use setuptools, correct? I don’t know

how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn’t adequate, I just tried it locally on my machine and it still doesn’t install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it’s desired

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it’s a rather large download.

Best wishes,

Damon

–

Damon McDougall
http://www.damon-is-a-geek.com

Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin

Austin,
TX 78712-1229

If I do a clean install of mpl master, and then of basemap, basemap

lands in dist-packages/mpl_toolkits, as it always has. But now it is

not found–I can’t import it. It seems that now the real mpl_toolkits

is cleverly hidden inside an egg directory with a monstrous name,

leaving basemap orphaned in a directory with no init.py. As a

workaround I can symlink it into the egg location. I suspect the real

solution will require basemap to use setuptools, correct? I don’t know

how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn’t adequate, I just tried it locally on my machine and it still doesn’t install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it’s desired.

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits
are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it’s a rather large download.

Best wishes,

Damon

–

Damon McDougall
http://www.damon-is-a-geek.com
Institute
for Computational Engineering Sciences

201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin,
TX 78712-1229

Michael_Droettboom · July 17, 2013, 12:41pm

Sorry to just be getting to this discussion now, after a vacation.

The move to setuptools was discussed at length as part of MEP11. See here: https://github.com/matplotlib/matplotlib/wiki/MEP11 and the MEP11 mailing list thread.

It has been a long standing bug that we ship Python dependencies along with matplotlib. Users were reporting pyparsing bugs to matplotlib, assuming it was a matplotlib project, etc. That was the primary impetus for moving to setuptools: so that pip dependency resolution would work. It has not been without headaches -- the "remerge" of distribute back into setuptools in particular was handled really poorly by upstream. I do hope that now that the community has re-coalesed around the "new" setuptools and the move from eggs to wheels that things will get better in the near future. But I still think the new approach is far better than what we had.

Sorry that the namespace package declaration was missed and fell through the cracks, but I'm glad we noticed it before 1.3.0 final. Is it the case that PR #2210, or are there still issues to be resolved?

Mike

···

On 07/06/2013 01:59 PM, Jeff Whitaker wrote:

Thomas Kluyver <mailto:thomas@…1071…>
July 6, 2013 11:26 AM

*distribute*, which was a fork of setuptools, was merged into setuptools. *distutils* is the component in the standard library, and is still there. I still prefer distutils where possible, precisely because setuptools' eggs are a mess.

Thomas

I agree eggs are a mess. Given that it is still easy to have the old behavior, can someone explain why the change was made to have setup.py create eggs by default?

-Jeff

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options
Damon McDougall <mailto:damon.mcdougall@…149…>
July 6, 2013 11:20 AM

On Sat, Jul 6, 2013 at 11:04 AM, Jeff Whitaker <jswhit@...196... >> <mailto:jswhit@…196…>> wrote:

    Damon McDougall <mailto:damon.mcdougall@…149…>
    July 6, 2013 9:32 AM

        If I do a clean install of mpl master, and then of basemap,
        basemap
        lands in dist-packages/mpl_toolkits, as it always has. But
        now it is
        not found--I can't import it. It seems that now the *real*
        mpl_toolkits
        is cleverly hidden inside an egg directory with a monstrous
        name,
        leaving basemap orphaned in a directory with no __init__.py.
         As a
        workaround I can symlink it into the egg location. I
        suspect the real
        solution will require basemap to use setuptools, correct? I
        don't know
        how to do this, so I hope someone who does will submit a PR.

    Actually, using the new setuptools isn't adequate, I just tried
    it locally on my machine and it still doesn't install itself
    into the matplotlib egg.

    I think the proper solution here is to add basemap as an
    optional dependency to matplotlib and have the user set a flag
    (off by default) to pull basemap if it's desired

    Does that sound like a reasonable solution?

    What if a user decides later that he/she wants to install
    basemap? Then they would have to reinstall matplotlib? That
    doesn't sound reasonable to me.

Actually, on reflection, I'm in agreement with you. I'm comfortable installing from source but this poses a larger problem when users download the basemap binary and expect it to Just Work.

    How about having matplotlib install a symlink to the egg location?

If there's a way for setuptools to modify the matplotlib egg to add a symlink, then it must be possible for setuptools to just put the files there. I just haven't figured out how to do that.

    Why the change to using setuptools by default in the first place?

Long story. The short story is that distutils was merged into setuptools. So setuptools is now the recommended way to install python packages.

    -Jeff

    P.S. Note that the other mpl_toolkits are installed into the
    correct place because they are shipped with matplotlib and
    installed at the same time. We could ship basemap with
    matplotlib too but it's a rather large download.

    Best wishes,
    Damon

    -- Damon McDougall
    http://www.damon-is-a-geek.com
    Institute for Computational Engineering Sciences
    201 E. 24th St.
    Stop C0200
    The University of Texas at Austin
    Austin, TX 78712-1229
    Eric Firing <mailto:efiring@…229…>
    July 6, 2013 12:53 AM
    If I do a clean install of mpl master, and then of basemap,
    basemap lands in dist-packages/mpl_toolkits, as it always has. But now it is not found--I can't import it. It seems that now
    the *real* mpl_toolkits is cleverly hidden inside an egg
    directory with a monstrous name, leaving basemap orphaned in a
    directory with no __init__.py. As a workaround I can symlink it
    into the egg location. I suspect the real solution will require
    basemap to use setuptools, correct? I don't know how to do
    this, so I hope someone who does will submit a PR.

    Eric

    ------------------------------------------------------------------------

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229
Jeff Whitaker <mailto:jswhit@…196…>
July 6, 2013 10:04 AM

Damon McDougall <mailto:damon.mcdougall@…149…>
July 6, 2013 9:32 AM

    If I do a clean install of mpl master, and then of basemap, basemap
    lands in dist-packages/mpl_toolkits, as it always has. But now
    it is
    not found--I can't import it. It seems that now the *real*
    mpl_toolkits
    is cleverly hidden inside an egg directory with a monstrous name,
    leaving basemap orphaned in a directory with no __init__.py. As a
    workaround I can symlink it into the egg location. I suspect
    the real
    solution will require basemap to use setuptools, correct? I
    don't know
    how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn't adequate, I just tried it locally on my machine and it still doesn't install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it's desired

Does that sound like a reasonable solution?

What if a user decides later that he/she wants to install basemap? Then they would have to reinstall matplotlib? That doesn't sound reasonable to me.

How about having matplotlib install a symlink to the egg location?

Why the change to using setuptools by default in the first place?

-Jeff

P.S. Note that the other mpl_toolkits are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it's a rather large download.

Best wishes,
Damon

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229
Eric Firing <mailto:efiring@…229…>
July 6, 2013 12:53 AM
If I do a clean install of mpl master, and then of basemap, basemap lands in dist-packages/mpl_toolkits, as it always has. But now it is not found--I can't import it. It seems that now the *real* mpl_toolkits is cleverly hidden inside an egg directory with a monstrous name, leaving basemap orphaned in a directory with no __init__.py. As a workaround I can symlink it into the egg location. I suspect the real solution will require basemap to use setuptools, correct? I don't know how to do this, so I hope someone who does will submit a PR.

Eric

------------------------------------------------------------------------

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options
Damon McDougall <mailto:damon.mcdougall@…149…>
July 6, 2013 9:32 AM

    If I do a clean install of mpl master, and then of basemap, basemap
    lands in dist-packages/mpl_toolkits, as it always has. But now it is
    not found--I can't import it. It seems that now the *real*
    mpl_toolkits
    is cleverly hidden inside an egg directory with a monstrous name,
    leaving basemap orphaned in a directory with no __init__.py. As a
    workaround I can symlink it into the egg location. I suspect the
    real
    solution will require basemap to use setuptools, correct? I
    don't know
    how to do this, so I hope someone who does will submit a PR.

Actually, using the new setuptools isn't adequate, I just tried it locally on my machine and it still doesn't install itself into the matplotlib egg.

I think the proper solution here is to add basemap as an optional dependency to matplotlib and have the user set a flag (off by default) to pull basemap if it's desired.

Does that sound like a reasonable solution?

P.S. Note that the other mpl_toolkits are installed into the correct place because they are shipped with matplotlib and installed at the same time. We could ship basemap with matplotlib too but it's a rather large download.

Best wishes,
Damon

--
Damon McDougall
http://www.damon-is-a-geek.com
Institute for Computational Engineering Sciences
201 E. 24th St.
Stop C0200
The University of Texas at Austin
Austin, TX 78712-1229
Eric Firing <mailto:efiring@…229…>
July 6, 2013 12:53 AM
If I do a clean install of mpl master, and then of basemap, basemap lands in dist-packages/mpl_toolkits, as it always has. But now it is not found--I can't import it. It seems that now the *real* mpl_toolkits is cleverly hidden inside an egg directory with a monstrous name, leaving basemap orphaned in a directory with no __init__.py. As a workaround I can symlink it into the egg location. I suspect the real solution will require basemap to use setuptools, correct? I don't know how to do this, so I hope someone who does will submit a PR.

Eric

------------------------------------------------------------------------

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options