Doesn't find Tcl/Tk headers on Ubuntu 8.04

I'm moving this second question onto the matplotlib-devel list.

It seems that a user is unable to build the tkagg extension from source on Ubuntu 8.04. I know the Tcl/Tk header lookup mechanism is inherently complex. Could someone who knows what's going on there have a look?

Jonathan: Could you please provide the output of a compile without your modifications to make it work? We'd like this to work automatically.

Cheers,
Mike

Jonathan Wright wrote:

···

Mike,

That fixes things for me - many thanks. Unrelated, but to build from SVN I had to go diving in setupext.py to say that the tk include files are in:

/usr/include/tcl8.4

... while the tcl install home is /usr/share/tcltk. The command "locate tk.h" was particularly useful.

Many thanks again,

Jon

Michael Droettboom wrote:

I assume you're using the matplotlib 0.91.2 that's distributed with Ubuntu 8.04.

There was a recent fix for segfaulting in the exact same place (outside of any sort of freezing apparatus). Since it was related to the interpretation of a pointer, it's possible that you would see this inside of cx-freeze and not outside on the same machine, just because things get loaded into different parts of memory. I would try that fix first, and then look at problems related to freezing.

We should have a new release out shortly, but it's unclear how long that will take to trickle down into Ubuntu repositories.

You can check out the SVN maintenance branch from here (which has this bugfix):

svn co https://matplotlib.svn.sf.net/svnroot/matplotlib/branches/v0_91_maint matplotlib-0.91.x

Let us know how that works for you.

Cheers,
Mike

Jonathan Wright wrote:

Hello,

I am getting segmentation faults when I try to freeze a script which uses the TkAgg backend, on python2.5.2, gcc 4.2.3 (ubuntu 8.04, hardy heron). A trial script is:

import matplotlib
matplotlib.use("TkAgg") # unless you have it in matplotlibrc
import matplotlib.backends.backend_tkagg # explicit for freezer
from matplotlib.pylab import plot, show
plot(range(10), range(10), "+")
show()

Is anyone already familiar with the problem? Things seem to work with the GTkAgg backend, but sadly many years ago I decided to use Tk as I thought it'd be easier to distribute. In order to reproduce the problem with bbfreeze you should just need this freezing script:

from bbfreeze import Freezer
f = Freezer("dist",
           includes=("matplotlib",
                     "matplotlib.numerix.fft",
                     "matplotlib.numerix.linear_algebra",
                     "matplotlib.numerix.ma",
                     "matplotlib.numerix.mlab",
                     "matplotlib.numerix.random_array"))
f.addScript("t.py")
f()

For reproducing the problem with cx-freeze you need to (a) install it by patching the cx-freeze setup.py [so that (2, 5) -> (2, 6)] and (b) add an import for numpy.linalg.lapack_lite and edit your numpy.__init__ to remove numpy.test.

Thanks for any advice,

Jon
---

PS: gdb says
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7c806b0 (LWP 8158)]
0xb6e145a0 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/matplotlib.backends._tkagg.so

(gdb) bt
#0 0xb6e145a0 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/matplotlib.backends._tkagg.so

#1 0xb6badb6e in TclInvokeStringCommand () from /usr/lib/libtcl8.4.so.0
#2 0xb6baee56 in TclEvalObjvInternal () from /usr/lib/libtcl8.4.so.0
#3 0xb6baf0db in Tcl_EvalObjv () from /usr/lib/libtcl8.4.so.0
#4 0xb6ef96c6 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/_tkinter.so
#5 0x0827a0c8 in ?? ()
#6 0x00000005 in ?? ()
...

-------------------------------------------------------------------------

This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
  
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Hi Mike,

OK, I've just done the svn co of v0_91_maint and get the output here:

···

============================================================================
BUILDING MATPLOTLIB
             matplotlib: 0.91.2svn
                 python: 2.5.2 (r252:60911, Apr 21 2008, 11:12:42) [GCC
                         4.2.3 (Ubuntu 4.2.3-2ubuntu7)]
               platform: linux2

REQUIRED DEPENDENCIES
                  numpy: 1.0.4
              freetype2: 9.16.3

OPTIONAL BACKEND DEPENDENCIES
                 libpng: 1.2.15beta5
                Tkinter: no
                         * Tkinter present, but header files are not found.
                         * You may need to install development packages.
               wxPython: 2.8.7.1
                         * WxAgg extension not required for wxPython >= 2.8
                   Gtk+: gtk+: 2.12.9, glib: 2.16.3, pygtk: 2.12.1,
                         pygobject: 2.14.1
                         * Could not find Gtk+ headers in any of
                         * '/usr/local/include', '/usr/include', '.'
                     Qt: Qt: 3.3.8, PyQt: 3.17.4
                    Qt4: Qt: 4.3.4, PyQt4: 4.3.3
                  Cairo: 1.4.0

OPTIONAL DATE/TIMEZONE DEPENDENCIES
               datetime: present, version unknown
               dateutil: 1.3
                   pytz: 2007k

OPTIONAL USETEX DEPENDENCIES
                 dvipng: 1.9
            ghostscript: 8.61
                  latex: 3.141592
                pdftops: 3.00

EXPERIMENTAL CONFIG PACKAGE DEPENDENCIES
              configobj: 4.4.0
       enthought.traits: 2.0.1b1

=========================================================================

I then added the python-gtk2-dev package and it built the _gtkagg.so backend by default. This looks fine, but it is not making TkAgg. So I added "tkagg = True" to setup.cfg and it gets up to:

=========================================================================
building 'matplotlib.backends._tkagg' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/share/include -I/usr/share/include -I/usr/include/libpng12 -I/usr/local/include -I/usr/include -I. -Isrc -Iswig -Iagg23/include -I. -I/usr/include/freetype2 -I/usr/local/include -I/usr/include -I. -I/usr/include/python2.5 -c src/_tkagg.cpp -o build/temp.linux-i686-2.5/src/_tkagg.o
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
src/_tkagg.cpp:28:18: error: tk.h: No such file or directory
src/_tkagg.cpp:36: error: ISO C++ forbids declaration of Tcl_Interp with no type
src/_tkagg.cpp:36: error: expected ; before * token
[snip - more errors ...]

It has missed tk.h. I have the tk-dev and tcl-dev packages (and of course the "build-essential" which gives standard headers etc).

`locate tk.h` gives /usr/include/tcl8.4/tk.h

so I edited setupext.py to give:

svn diff
Index: setupext.py

--- setupext.py (revision 5276)
+++ setupext.py (working copy)
@@ -977,6 +977,12 @@
                      os.path.exists('/usr/include/tk.h')):
                      tcl_inc = '/usr/include'
                      tk_inc = '/usr/include'
+ # Ubuntu 8.04
+ if (sys.platform.startswith('linux') and
+ os.path.exists('/usr/include/tcl8.4/tcl.h') and
+ os.path.exists('/usr/include/tcl8.4/tk.h')):
+ tcl_inc = '/usr/include/tcl8.4'
+ tk_inc = '/usr/include/tcl8.4'
          else:
              message = """\
  Using default library and include directories for Tcl and Tk because a

... and now it builds and seems to work. With this patch it says:

=============================================================
...
OPTIONAL BACKEND DEPENDENCIES
...
                Tkinter: Tkinter: 50704, Tk: 8.4, Tcl: 8.4

I thought you are expected to pick up the correct include directories from the files tclConfig.sh and tkConfig.sh, which are found in the lib directory returned from your query_tcltk function? Obviously, my solution is going to give problems for people who are not using 8.4, or who have something else different etc. Parsing tclConfig.sh seems like overkill - maybe try asking the python/distutils folks?

Best

Jon

On Tue, 27 May 2008, Michael Droettboom wrote:

I'm moving this second question onto the matplotlib-devel list.

It seems that a user is unable to build the tkagg extension from source on Ubuntu 8.04. I know the Tcl/Tk header lookup mechanism is inherently complex. Could someone who knows what's going on there have a look?

Jonathan: Could you please provide the output of a compile without your modifications to make it work? We'd like this to work automatically.

Cheers,
Mike

Jonathan Wright wrote:

Mike,

That fixes things for me - many thanks. Unrelated, but to build from SVN I had to go diving in setupext.py to say that the tk include files are in:

/usr/include/tcl8.4

... while the tcl install home is /usr/share/tcltk. The command "locate tk.h" was particularly useful.

Many thanks again,

Jon

Michael Droettboom wrote:

I assume you're using the matplotlib 0.91.2 that's distributed with Ubuntu 8.04.

There was a recent fix for segfaulting in the exact same place (outside of any sort of freezing apparatus). Since it was related to the interpretation of a pointer, it's possible that you would see this inside of cx-freeze and not outside on the same machine, just because things get loaded into different parts of memory. I would try that fix first, and then look at problems related to freezing.

We should have a new release out shortly, but it's unclear how long that will take to trickle down into Ubuntu repositories.

You can check out the SVN maintenance branch from here (which has this bugfix):

svn co https://matplotlib.svn.sf.net/svnroot/matplotlib/branches/v0_91_maint matplotlib-0.91.x

Let us know how that works for you.

Cheers,
Mike

Jonathan Wright wrote:

Hello,

I am getting segmentation faults when I try to freeze a script which uses the TkAgg backend, on python2.5.2, gcc 4.2.3 (ubuntu 8.04, hardy heron). A trial script is:

import matplotlib
matplotlib.use("TkAgg") # unless you have it in matplotlibrc
import matplotlib.backends.backend_tkagg # explicit for freezer
from matplotlib.pylab import plot, show
plot(range(10), range(10), "+")
show()

Is anyone already familiar with the problem? Things seem to work with the GTkAgg backend, but sadly many years ago I decided to use Tk as I thought it'd be easier to distribute. In order to reproduce the problem with bbfreeze you should just need this freezing script:

from bbfreeze import Freezer
f = Freezer("dist",
           includes=("matplotlib",
                     "matplotlib.numerix.fft",
                     "matplotlib.numerix.linear_algebra",
                     "matplotlib.numerix.ma",
                     "matplotlib.numerix.mlab",
                     "matplotlib.numerix.random_array"))
f.addScript("t.py")
f()

For reproducing the problem with cx-freeze you need to (a) install it by patching the cx-freeze setup.py [so that (2, 5) -> (2, 6)] and (b) add an import for numpy.linalg.lapack_lite and edit your numpy.__init__ to remove numpy.test.

Thanks for any advice,

Jon
---

PS: gdb says
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7c806b0 (LWP 8158)]
0xb6e145a0 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/matplotlib.backends._tkagg.so (gdb) bt
#0 0xb6e145a0 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/matplotlib.backends._tkagg.so #1 0xb6badb6e in TclInvokeStringCommand () from /usr/lib/libtcl8.4.so.0
#2 0xb6baee56 in TclEvalObjvInternal () from /usr/lib/libtcl8.4.so.0
#3 0xb6baf0db in Tcl_EvalObjv () from /usr/lib/libtcl8.4.so.0
#4 0xb6ef96c6 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/_tkinter.so
#5 0x0827a0c8 in ?? ()
#6 0x00000005 in ?? ()
...

------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Thanks, Jonathan. The tclConfig.sh stuff was news to me (I didn't write the original Tcl/Tk header finding stuff, though).

Everyone else: I'm on this and will let you know how it goes.

Cheers,
Mike

Jonathan Wright wrote:

···

Hi Mike,

OK, I've just done the svn co of v0_91_maint and get the output here:

============================================================================

BUILDING MATPLOTLIB
            matplotlib: 0.91.2svn
                python: 2.5.2 (r252:60911, Apr 21 2008, 11:12:42) [GCC
                        4.2.3 (Ubuntu 4.2.3-2ubuntu7)]
              platform: linux2

REQUIRED DEPENDENCIES
                 numpy: 1.0.4
             freetype2: 9.16.3

OPTIONAL BACKEND DEPENDENCIES
                libpng: 1.2.15beta5
               Tkinter: no
                        * Tkinter present, but header files are not found.
                        * You may need to install development packages.
              wxPython: 2.8.7.1
                        * WxAgg extension not required for wxPython >= 2.8
                  Gtk+: gtk+: 2.12.9, glib: 2.16.3, pygtk: 2.12.1,
                        pygobject: 2.14.1
                        * Could not find Gtk+ headers in any of
                        * '/usr/local/include', '/usr/include', '.'
                    Qt: Qt: 3.3.8, PyQt: 3.17.4
                   Qt4: Qt: 4.3.4, PyQt4: 4.3.3
                 Cairo: 1.4.0

OPTIONAL DATE/TIMEZONE DEPENDENCIES
              datetime: present, version unknown
              dateutil: 1.3
                  pytz: 2007k

OPTIONAL USETEX DEPENDENCIES
                dvipng: 1.9
           ghostscript: 8.61
                 latex: 3.141592
               pdftops: 3.00

EXPERIMENTAL CONFIG PACKAGE DEPENDENCIES
             configobj: 4.4.0
      enthought.traits: 2.0.1b1

=========================================================================

I then added the python-gtk2-dev package and it built the _gtkagg.so backend by default. This looks fine, but it is not making TkAgg. So I added "tkagg = True" to setup.cfg and it gets up to:

=========================================================================
building 'matplotlib.backends._tkagg' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/share/include -I/usr/share/include -I/usr/include/libpng12 -I/usr/local/include -I/usr/include -I. -Isrc -Iswig -Iagg23/include -I. -I/usr/include/freetype2 -I/usr/local/include -I/usr/include -I. -I/usr/include/python2.5 -c src/_tkagg.cpp -o build/temp.linux-i686-2.5/src/_tkagg.o
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
src/_tkagg.cpp:28:18: error: tk.h: No such file or directory
src/_tkagg.cpp:36: error: ISO C++ forbids declaration of Tcl_Interp with no type
src/_tkagg.cpp:36: error: expected ; before * token
[snip - more errors ...]

It has missed tk.h. I have the tk-dev and tcl-dev packages (and of course the "build-essential" which gives standard headers etc).

`locate tk.h` gives /usr/include/tcl8.4/tk.h

so I edited setupext.py to give:

svn diff
Index: setupext.py

--- setupext.py (revision 5276)
+++ setupext.py (working copy)
@@ -977,6 +977,12 @@
                     os.path.exists('/usr/include/tk.h')):
                     tcl_inc = '/usr/include'
                     tk_inc = '/usr/include'
+ # Ubuntu 8.04
+ if (sys.platform.startswith('linux') and
+ os.path.exists('/usr/include/tcl8.4/tcl.h') and
+ os.path.exists('/usr/include/tcl8.4/tk.h')):
+ tcl_inc = '/usr/include/tcl8.4'
+ tk_inc = '/usr/include/tcl8.4'
         else:
             message = """\
Using default library and include directories for Tcl and Tk because a

... and now it builds and seems to work. With this patch it says:

=============================================================
...
OPTIONAL BACKEND DEPENDENCIES
...
               Tkinter: Tkinter: 50704, Tk: 8.4, Tcl: 8.4

I thought you are expected to pick up the correct include directories from the files tclConfig.sh and tkConfig.sh, which are found in the lib directory returned from your query_tcltk function? Obviously, my solution is going to give problems for people who are not using 8.4, or who have something else different etc. Parsing tclConfig.sh seems like overkill - maybe try asking the python/distutils folks?

Best

Jon

On Tue, 27 May 2008, Michael Droettboom wrote:

I'm moving this second question onto the matplotlib-devel list.

It seems that a user is unable to build the tkagg extension from source on Ubuntu 8.04. I know the Tcl/Tk header lookup mechanism is inherently complex. Could someone who knows what's going on there have a look?

Jonathan: Could you please provide the output of a compile without your modifications to make it work? We'd like this to work automatically.

Cheers,
Mike

Jonathan Wright wrote:

Mike,

That fixes things for me - many thanks. Unrelated, but to build from SVN I had to go diving in setupext.py to say that the tk include files are in:

/usr/include/tcl8.4

... while the tcl install home is /usr/share/tcltk. The command "locate tk.h" was particularly useful.

Many thanks again,

Jon

Michael Droettboom wrote:

I assume you're using the matplotlib 0.91.2 that's distributed with Ubuntu 8.04.

There was a recent fix for segfaulting in the exact same place (outside of any sort of freezing apparatus). Since it was related to the interpretation of a pointer, it's possible that you would see this inside of cx-freeze and not outside on the same machine, just because things get loaded into different parts of memory. I would try that fix first, and then look at problems related to freezing.

We should have a new release out shortly, but it's unclear how long that will take to trickle down into Ubuntu repositories.

You can check out the SVN maintenance branch from here (which has this bugfix):

svn co https://matplotlib.svn.sf.net/svnroot/matplotlib/branches/v0_91_maint matplotlib-0.91.x

Let us know how that works for you.

Cheers,
Mike

Jonathan Wright wrote:

Hello,

I am getting segmentation faults when I try to freeze a script which uses the TkAgg backend, on python2.5.2, gcc 4.2.3 (ubuntu 8.04, hardy heron). A trial script is:

import matplotlib
matplotlib.use("TkAgg") # unless you have it in matplotlibrc
import matplotlib.backends.backend_tkagg # explicit for freezer
from matplotlib.pylab import plot, show
plot(range(10), range(10), "+")
show()

Is anyone already familiar with the problem? Things seem to work with the GTkAgg backend, but sadly many years ago I decided to use Tk as I thought it'd be easier to distribute. In order to reproduce the problem with bbfreeze you should just need this freezing script:

from bbfreeze import Freezer
f = Freezer("dist",
           includes=("matplotlib",
                     "matplotlib.numerix.fft",
                     "matplotlib.numerix.linear_algebra",
                     "matplotlib.numerix.ma",
                     "matplotlib.numerix.mlab",
                     "matplotlib.numerix.random_array"))
f.addScript("t.py")
f()

For reproducing the problem with cx-freeze you need to (a) install it by patching the cx-freeze setup.py [so that (2, 5) -> (2, 6)] and (b) add an import for numpy.linalg.lapack_lite and edit your numpy.__init__ to remove numpy.test.

Thanks for any advice,

Jon
---

PS: gdb says
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7c806b0 (LWP 8158)]
0xb6e145a0 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/matplotlib.backends._tkagg.so (gdb) bt
#0 0xb6e145a0 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/matplotlib.backends._tkagg.so #1 0xb6badb6e in TclInvokeStringCommand () from /usr/lib/libtcl8.4.so.0
#2 0xb6baee56 in TclEvalObjvInternal () from /usr/lib/libtcl8.4.so.0
#3 0xb6baf0db in Tcl_EvalObjv () from /usr/lib/libtcl8.4.so.0
#4 0xb6ef96c6 in ?? () from /home/wright/testcx/build/exe.linux-i686-2.5/_tkinter.so
#5 0x0827a0c8 in ?? ()
#6 0x00000005 in ?? ()
...

------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Just a reminder that a lot of the tcl/tk finder code was lifted from
the PIL setup, so you may want to see if there have been any updates
over there and re-lift them.

JDH

···

On Tue, May 27, 2008 at 9:27 AM, Michael Droettboom <mdroe@...31...> wrote:

Thanks, Jonathan. The tclConfig.sh stuff was news to me (I didn't write
the original Tcl/Tk header finding stuff, though).

Everyone else: I'm on this and will let you know how it goes.

Alas, PIL SVN head fails to build on Ubuntu 8.04 (without Ubuntu-specific patches) as well...

Cheers,
Mike

John Hunter wrote:

···

On Tue, May 27, 2008 at 9:27 AM, Michael Droettboom <mdroe@...31...> wrote:
  

Thanks, Jonathan. The tclConfig.sh stuff was news to me (I didn't write
the original Tcl/Tk header finding stuff, though).

Everyone else: I'm on this and will let you know how it goes.
    
Just a reminder that a lot of the tcl/tk finder code was lifted from
the PIL setup, so you may want to see if there have been any updates
over there and re-lift them.

JDH
  
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

I think I have a fix in for this now.

As Jonathan Wright suggested, it parses tclConfig.sh and tkConfig.sh to find the include and lib directories. If any of that fails, it falls back to the old method (which is based on the assumption that the build and lib directories are relative to the tcl data directories). I'm afraid this makes the code more convoluted, not less, but perhaps with enough testing we can be confident that the old way can be eliminated.

Please try this on as many Linux distributions as you can. (This does not affect Windows and Mac). If in the build logs you see the message:

    Guessing the library and include directories for Tcl and Tk because the
    tclConfig.sh and tkConfig.sh could not be found and/or parsed.

that means the old method is still being used. Please let me know about that, and what distro you are running -- it would be great to merge to a single lookup method if possible.

I'm keeping this on the trunk for now, but if we're confident in the change, this is a good candidate to be merged to the maintenance branch.

Cheers,
Mike

Michael Droettboom wrote:

···

Alas, PIL SVN head fails to build on Ubuntu 8.04 (without Ubuntu-specific patches) as well...

Cheers,
Mike

John Hunter wrote:
  

On Tue, May 27, 2008 at 9:27 AM, Michael Droettboom <mdroe@...31...> wrote:
  

Thanks, Jonathan. The tclConfig.sh stuff was news to me (I didn't write
the original Tcl/Tk header finding stuff, though).

Everyone else: I'm on this and will let you know how it goes.
    

Just a reminder that a lot of the tcl/tk finder code was lifted from
the PIL setup, so you may want to see if there have been any updates
over there and re-lift them.

JDH
  
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA