binary installers for python2.6; libpng segfault, MSVCR90.DLL and mingw

Development matplotlib-devel
1

I have uploaded binary installers for python2.5 for win32 to the
website, but python2.6 is a different beast all together. I have been
trying to make Charlie's support work for mingw32 for windows in
release/win32 work, but have hit a huge wall. Apparently, mingw
doesn't play nicely with MSVCR90.DLL abd Charlie's hack to simply
remove it from the dll_libraries list doesn't work because _png.pyd
needs these functions and the mingw support is broken. The dirty
details are at

  http://bugs.python.org/issue3308

  http://www.nabble.com/localtime()-and-MSVCRT9-td18329478.html

numpy went through what looks like a painful experience with some of
these issues detailed at

  http://www.mail-archive.com/numpy-discussion@...336.../msg14552.html

  http://cournape.wordpress.com/2008/09/02/how-to-embed-a-manifest-into-a-dll-with-mingw-tools-only/

It looks like pygame is going to extreme lengths to work around these
problems, as you can see in this code

   http://github.com/ab3/pygame-mirror/blob/cf8aee7508194b0be734b080f866c5a24e242cec/msys_link_VC_2008_dlls.py

I've burnt a lot of time chasing false leads before I finally figured
out where the root of the problem is, and it appears to be in the
broken gmtime/localtime support in mingw and MSVCR90.DLL. I have
narrowed the segfault to png_write_info(png_ptr, info_ptr), and since
I believe png uses time support in the info processing, my guess is
this is where the code is failing, but my attempts to build libpng w/o
time support as a quick workaround

  CFLAGS = -Os -D_ftime=ftime64 -DPNG_NO_READ_tIME -DPNG_NO_WRITE_tIME

have not helped (this was mainly a stab in the dark from poking around
the png headers and src)

There are a variety of solutions offered in the links above ranging
from rebuilding a patched mingw from src to the approach taken by
pygame. Since the rest of mpl is working fine, I was hoping to simply
hack around the broken part of libpng itself (eg removing the time
calls) but have not succeeded yet. Unfortunately, I won't have a lot
more time to spend on this in the near future, so I was hoping that
someone with a fresh pair of eyes and a little time could find a
solution.

If you can take a look, to get started check out release/win32 from
the TRUNK and build the sdist from the branch with

  > python setup.py sdist --formats=gztar

Drop the tar.gz into release/win32 and then following the instructions
in release/win32/README.txt

Thanks,
JDH

2

I should add one more detail here. distutils for python2.6 adds a
dependency to MSVCR90.DLL, and Charlie has a distutils hack in
release/win32/data/setupwin.py to try
and banish the msvcr90.dll which is not on most systems; he sets the
distutils dll_libraries to the empty list::

    from distutils import cygwinccompiler

    try:
            # Python 2.6
            # Replace the msvcr func to return an empty list
            cygwinccompiler.get_msvcr
            cygwinccompiler.get_msvcr = lambda:

w/o this hack you will get a dll import error rather than a segfault.
You can toggle the hack on and off by changing in the Makefile::

  # on
  \{PYTHON\} setupwin\.py build\_ext \-c mingw32 \-I {PY_INCLUDE} -L
${PY_LINKER} bdist_wininst

  # off
  \{PYTHON\} setup\.py build\_ext \-c mingw32 \-I {PY_INCLUDE} -L
${PY_LINKER} bdist_wininst

The problem is that the dll cannot be found on the system and
apparently the right solution is to include it in a framework as
described by David in the link I posted

  How to embed a manifest into a dll with mingw tools only – Nothing to say

but I think this is something of a red herring because w/ Charlie's
hack _png links against MSVCRT.DLL rather than MSVCR90.DLL but still
segfaults (only on python2.6) on the png_write_info call. I also
tried forcing 'msvcr71' in the distutils hack which is what python2.5
usesbut to no avail. When I said I wasted a lot of time chasing down
false leads, most of that was spent on trying various versions of mpl,
numpy, distutils, and the distutils MSVCR hacks, but in the end none
of it mattered.

But for the record I am including some notes I took while
experimenting with different things. When I write below

  Interestingly, depwalker can find the DLL where PYTHON26.DLL depends
  on, pointing to C:\WINDOWS\WinSxS.VC90.CRT_1fgobbledygook on my box,
  but cannot find it where _PNG.PYD depends on it.

this was before I discovered David's essay on side-by-side assemblies
which explains why the linker finds python's MSVCR90.DLL but not
_pngs. What it doesn't explain is why w/ Charlie's hack, when we link
against MSVCRT.DLL or MSVCR71.DLL, png is still segfaulting, and I
think the answer there is what I pointed to in my first post, the
broken time support in mingw MSVCR90.DLL. Support for this is in my
note below::

  This time it finds it, but two of the functions are colored red
  indicating a problem: _fstati64 and gmtime.

Notes on the _png segfault and the MSVCR hacks

···

On Wed, Jun 10, 2009 at 4:41 PM, John Hunter<jdh2358@...149...> wrote:

Drop the tar.gz into release/win32 and then following the instructions
in release/win32/README.txt

======================================

I have been working on the windows installers for mpl 98.5.3. I am
able to build for python2.5 w/o problems, but when building for
python2.6, I get an DLL local error on any extension code

    cd matplotlib-0.98.5.3/build/lib.win32-2.6/matplotlib
    > /c/python26/python -c 'import _png'
    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    ImportError: DLL load failed: The specified procedure could not be found.

when I open _PNG.PYD this file in dependency walker, I see a
dependency on MSVCR90.DLL which is flagged with a question mark "Error
opening file. The system cannot find the file specified". All the
*.PYD files have this dependency, including the numpy pyd files, but
most of them have it buried under a PYTHON26.DLL dependency, and
_PNG.PYD has it as a direct dependency (top level). Interestingly,
depwalker can find the DLL where PYTHON26.DLL depends on, pointing to
C:\WINDOWS\WinSxS.VC90.CRT_1fgobbledygook on my box, but cannot find
it where _PNG.PYD depends on it.

When I look at _PNG.PYD for python2.5 (which is working fine and I've
uploaded the installers) I see a dependency for MSVCR71.DLL located in
C:/windows/system32, and the functions it provides include fclose,
fopen, putc, etc. which libpng apparently needs for file I/O.

As mentioned above, my system *does* have MSVCR90.DLL deep in
C:\WINDOWS\WinSxS.VC90.CRT_1fgobbledygook, and when I move it into
C:\windows\system32 and then reopen _png.pyd in depdendency walker, I
still get an error. This time it finds it, but two of the functions
are colored red indicating a problem: _fstati64 and gmtime.

Charlie has a distutils hack in release/win32/data/setupwin.py. Wen I
build the installer using the hacked distutils I can load the DLL file
in mpl either by importing matplotlib._png or in dependency walker.
In depwalker, the msvcrt90.DLL is replaced by MSVCRT.DLL at the top
level _PNG.PYD dependency, but the MSVCR90.DLL is still buried deep
under PYTHON26.DLL. Unfortunately, use of this module triggers a
segfault.

3

I am new to trying to build matplotlib from svn and believe I am up against
the issue described in this thread. That is, building matplotlib on Python
2.6.2/Windows, compiling/linking with MinGW seem to go fine but upon trying
to

     from pylab import *

I get

     ....
     ....(lots of traceback that culminates below)
     ....
     C:\Python262\lib\site-packages\matplotlib\image.py in <module>()
        19 # For clarity, names from _image are given explicitly in this
module:

        20 import matplotlib._image as _image
---> 21 import matplotlib._png as _png
        22
        23 # For user convenience, the names from _image are also imported
into

        ImportError: DLL load failed: The specified procedure could not be
found. )

even though matplotlib\_png.pyd exists.

I am wondering if there has been any progress on this front?

I found http://www.nabble.com/Python-2.6.2-installation--td23996471.html
this thread
which suggests using Visual Studio 2008 (VS9) instead of MinGW, but that was
for 0.98.5.3. I installed VS9 and followed the instructions in that thread
but now when I try to import pylab Visual Studio's Just-In-Time Debugger
pops up telling me I have an unhandled win32 exception in python.exe

Can anyone comment on a verified way of building the latest matplotlib on
Windows/Python 2.6.2? I keep going down dead ends and would like to figure
out if its me or if I have just been down now-known bad paths.

Thanks,

Josh

···

-----
Josh Hemann
Statistical Advisor
http://www.vni.com/ Visual Numerics
jhemann@...735... | P 720.407.4214 | F 720.407.4199

–
View this message in context: http://www.nabble.com/binary-installers-for-python2.6--libpng-segfault%2C-MSVCR90.DLL-and- mingw-tp23971661p24943445.html
Sent from the matplotlib - devel mailing list archive at Nabble.com.

4

mpl 0.99.0 installer from SF doesn't work for me (python2.6). Python
interpreter crashes without any message every time I import
matplotlib._path.pyd or _png.pyd
I have win xp sp3, mingw-tdm(4.4.0)
Previous mpl installer worked fine but mpl 0.99.0 isn't

The only way to have it was to compile mpl 0.99.0 under mingw this way:
1. make import lib from dll: original msvcr90.dll -> msvcr90.dll.a (google
this)
2. temporiarly replace mingw's libmsvcr90.a with msvcr90.dll.a
3. compile mpl as usual
4. restore libmsvcr90.a

···

–
View this message in context: http://www.nabble.com/binary-installers-for-python2.6--libpng-segfault%2C-MSVCR90.DLL-and- mingw-tp23971661p25244316.html
Sent from the matplotlib - devel mailing list archive at Nabble.com.