[sage-devel] PyCXX, Matplotlib and OS X 10.6

Development matplotlib-devel
1

If you could just try running your matplotlib build on
bsd.math.washington.edu and reporting about whether or not it works,
that would be very helpful, since it will indicate whether the problem
is somehow in Sage or in pyCXX or Matplotlib itself or something else.

I was able to build into a mostly clean environment (rebuild all deps
zlip, freetype, png) using my OSX make file in, but I had to update
the MACOSX_DEPLOYMENT_TARGET to 10.6 (I've committed this to svn
HEAD).

PREFIX=~/devtest make -f make.osx fetch deps mpl_build mpl_install

But even simple tests are failing with::

jdh2358@...776...:~> LD_LIBRARY_PATH=~/devtest/lib/
PYTHONPATH=~/devtest/lib/python2.6/site-packages/ /usr/bin/python -c
'import matplotlib; matplotlib.use("Agg"); from matplotlib.pyplot
import *; plot([1,2,3]); savefig("test")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/pyplot.py",
line 7, in <module>
from matplotlib.figure import Figure, figaspect
File "/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/figure.py",
line 16, in <module>
import artist
File "/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/artist.py",
line 6, in <module>
from transforms import Bbox, IdentityTransform, TransformedBbox,
TransformedPath
File "/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/transforms.py",
line 34, in <module>
from matplotlib._path import affine_transform
ImportError: /Users/jdh2358/devtest/lib/python2.6/site-packages/matplotlib/_path.so:
no appropriate 64-bit architecture (see "man python" for running in
32-bit mode)

I'm attaching my build output in case anyone sees anything that might
be triggering this 32bit/64bit problem (see attached for full output).
I did not rebuild numpy and this may be the problem since the failure
is in the _path module. I'll give that a try next

I think it's definitely an issue with PyCXX. I installed the latest
version of PyCXX and tried their test example program, and got the
dreaded "Abort trap":

bash-3.2$ python
Python 2.6.2 (r262:71600, Sep 27 2009, 13:41:53)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import CXX.example

CXX is installed.
The support files you need are in the PYTHON/etc/CXX directory.
The include files are in the distutils include path already.
Just refer to them as "CXX/CXX_Objects.h", etc.

range object created 0x1003140b0
range object destroyed 0x1003140b0

CXX.example.test()

Example Test starting
Trying to convert a NULL to an Py::Int
Abort trap
bash-3.2$ pwd
/Users/was/build/sage-4.1.2.alpha4/spkg/build/matplotlib/trunk/matplotlib/pycxx-6.1.1/Demo/Python2

···

On Mon, Sep 28, 2009 at 9:14 AM, John Hunter <jdh2358@...149...> wrote:

On Mon, Sep 28, 2009 at 10:48 AM, William Stein <wstein@...149...> wrote:

----

Note that installing PyCXX and their test program was not easy. I
watched their "python setup.py install" fail repeatedly, then I would
copy missing files where they should go, and iterate (about 6 times

I've cc'd the PyCXX maintainer Barry Scott. Barry, do you have access
to an OS X 10.6 install, since it seems PyCXX is broken there? If
not, I can give you an account on a machine at University of
Washington. PyCXX is the single component of Sage
(http://sagemath.org) that is holding us back from finishing the OS X
10.6 port.

-- William

2

If you could just try running your matplotlib build on
bsd.math.washington.edu and reporting about whether or not it works,
that would be very helpful, since it will indicate whether the problem
is somehow in Sage or in pyCXX or Matplotlib itself or something else.

I was able to build into a mostly clean environment (rebuild all deps
zlip, freetype, png) using my OSX make file in, but I had to update
the MACOSX_DEPLOYMENT_TARGET to 10.6 (I've committed this to svn
HEAD).

PREFIX=~/devtest make -f make.osx fetch deps mpl_build mpl_install

But even simple tests are failing with::

jdh2358@...776...:~> LD_LIBRARY_PATH=~/devtest/lib/
PYTHONPATH=~/devtest/lib/python2.6/site-packages/ /usr/bin/python -c
'import matplotlib; matplotlib.use("Agg"); from matplotlib.pyplot
import *; plot([1,2,3]); savefig("test")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/pyplot.py",
line 7, in <module>
  from matplotlib.figure import Figure, figaspect
File "/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/figure.py",
line 16, in <module>
  import artist
File "/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/artist.py",
line 6, in <module>
  from transforms import Bbox, IdentityTransform, TransformedBbox,
TransformedPath
File "/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/transforms.py",
line 34, in <module>
  from matplotlib._path import affine_transform
ImportError: /Users/jdh2358/devtest/lib/python2.6/site-packages/matplotlib/_path.so:
no appropriate 64-bit architecture (see "man python" for running in
32-bit mode)

This error has nothing to do the PyCXX isn't it a build problem not crating
a fat binary with -arch x86_64 for _path.so. Use lipo -info to see what is
inside _path.so.

I'm attaching my build output in case anyone sees anything that might
be triggering this 32bit/64bit problem (see attached for full output).
I did not rebuild numpy and this may be the problem since the failure
is in the _path module. I'll give that a try next

I think it's definitely an issue with PyCXX. I installed the latest
version of PyCXX and tried their test example program, and got the
dreaded "Abort trap":

bash-3.2$ python
Python 2.6.2 (r262:71600, Sep 27 2009, 13:41:53)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import CXX.example

CXX is installed.
The support files you need are in the PYTHON/etc/CXX directory.
The include files are in the distutils include path already.
Just refer to them as "CXX/CXX_Objects.h", etc.

range object created 0x1003140b0
range object destroyed 0x1003140b0

CXX.example.test()

Example Test starting
Trying to convert a NULL to an Py::Int
Abort trap

bash-3.2$ pwd
/Users/was/build/sage-4.1.2.alpha4/spkg/build/matplotlib/trunk/matplotlib/pycxx-6.1.1/Demo/Python2

----

Note that installing PyCXX and their test program was not easy. I
watched their "python setup.py install" fail repeatedly, then I would
copy missing files where they should go, and iterate (about 6 times

I never use the setup.py - I keep trying to delete it from the kits but have users want
it and contribute fixes.

I do not recommend installing PyCXX into a system for a couple of reasons.

On one system you may well end up using more then one version of PyCXX.
For example one older version of maintenance and a newer version for new development.

YOu may have more then one project using PyCXX that need compiler options
that conflict. If you install PyCXX into the system the object code will be wrong
so some projects.

Your experience is convincing me to delete setup.py to avoid this in future.

However the tests do all work on 10.6. You will need a new make file to use
Apple's python 2.6.1. The PyCXX 6.1.1 kit does not have one to do this.

example_mac_apple_py26.mak (412 Bytes)

py26.log (7.35 KB)

···

On 28 Sep 2009, at 17:27, William Stein wrote:

On Mon, Sep 28, 2009 at 9:14 AM, John Hunter <jdh2358@...149...> > wrote:

On Mon, Sep 28, 2009 at 10:48 AM, William Stein <wstein@...149...> >> wrote:

3

Thanks for all your ideas. Inspired by your remarks I tried again
and the first thing I attempted was to take matplotlib's build lines
for building ft2font (which uses PyCXX) and just paste them into my
console instead of letting them be built under setup.py. Amazingly
enough doing that completely fixes all the problems. That suggests
there is something amis with the environment that Python's distutils
and matplotlib create when building ft2font + pyCXX.

My solution for Sage for now will be to rerun those lines explicitly
when building Sage on OS X 10.6, and hope for John Hunter to be able
to replicate the problem and come up with something better in the next
few weeks (or I'll come back to it later).

William

···

On Tue, Sep 29, 2009 at 3:07 PM, Barry Scott <barry@...777...> wrote:

On 28 Sep 2009, at 17:27, William Stein wrote:

On Mon, Sep 28, 2009 at 9:14 AM, John Hunter <jdh2358@...149...> wrote:

On Mon, Sep 28, 2009 at 10:48 AM, William Stein <wstein@...149...> wrote:

If you could just try running your matplotlib build on
bsd.math.washington.edu and reporting about whether or not it works,
that would be very helpful, since it will indicate whether the problem
is somehow in Sage or in pyCXX or Matplotlib itself or something else.

I was able to build into a mostly clean environment (rebuild all deps
zlip, freetype, png) using my OSX make file in, but I had to update
the MACOSX_DEPLOYMENT_TARGET to 10.6 (I've committed this to svn
HEAD).

PREFIX=~/devtest make -f make.osx fetch deps mpl_build mpl_install

But even simple tests are failing with::

jdh2358@...776...:~> LD_LIBRARY_PATH=~/devtest/lib/
PYTHONPATH=~/devtest/lib/python2.6/site-packages/ /usr/bin/python -c
'import matplotlib; matplotlib.use("Agg"); from matplotlib.pyplot
import *; plot([1,2,3]); savefig("test")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File
"/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/pyplot.py",
line 7, in <module>
from matplotlib.figure import Figure, figaspect
File
"/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/figure.py",
line 16, in <module>
import artist
File
"/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/artist.py",
line 6, in <module>
from transforms import Bbox, IdentityTransform, TransformedBbox,
TransformedPath
File
"/Users/jdh2358/devtest//lib/python2.6/site-packages/matplotlib/transforms.py",
line 34, in <module>
from matplotlib._path import affine_transform
ImportError:
/Users/jdh2358/devtest/lib/python2.6/site-packages/matplotlib/_path.so:
no appropriate 64-bit architecture (see "man python" for running in
32-bit mode)

This error has nothing to do the PyCXX isn't it a build problem not crating
a fat binary with -arch x86_64 for _path.so. Use lipo -info to see what is
inside _path.so.

I'm attaching my build output in case anyone sees anything that might
be triggering this 32bit/64bit problem (see attached for full output).
I did not rebuild numpy and this may be the problem since the failure
is in the _path module. I'll give that a try next

I think it's definitely an issue with PyCXX. I installed the latest
version of PyCXX and tried their test example program, and got the
dreaded "Abort trap":

bash-3.2$ python
Python 2.6.2 (r262:71600, Sep 27 2009, 13:41:53)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import CXX.example

CXX is installed.
The support files you need are in the PYTHON/etc/CXX directory.
The include files are in the distutils include path already.
Just refer to them as "CXX/CXX_Objects.h", etc.

range object created 0x1003140b0
range object destroyed 0x1003140b0

CXX.example.test()

Example Test starting
Trying to convert a NULL to an Py::Int
Abort trap

bash-3.2$ pwd

/Users/was/build/sage-4.1.2.alpha4/spkg/build/matplotlib/trunk/matplotlib/pycxx-6.1.1/Demo/Python2

----

Note that installing PyCXX and their test program was not easy. I
watched their "python setup.py install" fail repeatedly, then I would
copy missing files where they should go, and iterate (about 6 times

I never use the setup.py - I keep trying to delete it from the kits but have
users want
it and contribute fixes.

I do not recommend installing PyCXX into a system for a couple of reasons.

On one system you may well end up using more then one version of PyCXX.
For example one older version of maintenance and a newer version for new
development.

YOu may have more then one project using PyCXX that need compiler options
that conflict. If you install PyCXX into the system the object code will be
wrong
so some projects.

Your experience is convincing me to delete setup.py to avoid this in future.

However the tests do all work on 10.6. You will need a new make file to use
Apple's python 2.6.1. The PyCXX 6.1.1 kit does not have one to do this.

Save it to the top of the PyCXX files, where all the other make files are.

Then you can build and run all tests with:

$ make -f example_mac_apple_py26.mak clean test

All the tests will pass without error.

Attached is the log of the build and test on Mac OS X 10.6.1 I ran just now
to confirm this.

I suspect that the PyCXX that you installed with setup.py is not correct.
I suggest you delete it from out of the system and point directly to the
headers and C and C++ files you need out of a exploded tarball.

This is what I do for PySVN for example.

I've cc'd the PyCXX maintainer Barry Scott. Barry, do you have access
to an OS X 10.6 install, since it seems PyCXX is broken there? If
not, I can give you an account on a machine at University of
Washington. PyCXX is the single component of Sage
(http://sagemath.org) that is holding us back from finishing the OS X
10.6 port.

-- William

Let me know how else I can help.