MinGW/Win32 Release Build Scripts

I've spent the last couple of days writing the win32 equivalent
of John's osx release scripts for MinGW. It patches distutils at
runtime to exclude the msvcr link, hence we should no longer need to
modify distutils. I have tested it on Python 2.6 and it should work
on 2.4 and 2.5 as well. It's too late for me to test those right now,
but I trust they should work. Instructions can be found in
release/win32/README.txt. It required removing a link to "gw32" in
"setupext.py" so it only works trunk sdists. All interested, please
give it a try.

- Charlie

I'm in the process of testing this -- one problem I encountered was
some trailing spaces after the version numbers and before the comments
which broke the build. nfortunately some of the compiler error
messages were fairly cryptic (c compiler cannot create executables)
which sent me down some false trails before I figured it out from the
libpng config.log. I've committed the fix (first use of tortoisesvn
-- woohoo).

My sdist command in mingw is creating zip files by default -- any way
to force it to make the tar.gz that the build script is expecting ? I
can hack around this by creating my own tarball.....

JDH

···

On Wed, May 27, 2009 at 11:08 PM, Charlie Moad <cwmoad@...149...> wrote:

    I've spent the last couple of days writing the win32 equivalent
of John's osx release scripts for MinGW. It patches distutils at
runtime to exclude the msvcr link, hence we should no longer need to
modify distutils. I have tested it on Python 2.6 and it should work
on 2.4 and 2.5 as well. It's too late for me to test those right now,
but I trust they should work. Instructions can be found in
release/win32/README.txt. It required removing a link to "gw32" in
"setupext.py" so it only works trunk sdists. All interested, please
give it a try.

It's strange that I didn't run into spacing issues.

sdist --formats=gztar

- Charlie

···

On Thu, May 28, 2009 at 9:47 AM, John Hunter <jdh2358@...149...> wrote:

On Wed, May 27, 2009 at 11:08 PM, Charlie Moad <cwmoad@...149...> wrote:

I&#39;ve spent the last couple of days writing the win32 equivalent

of John's osx release scripts for MinGW. It patches distutils at
runtime to exclude the msvcr link, hence we should no longer need to
modify distutils. I have tested it on Python 2.6 and it should work
on 2.4 and 2.5 as well. It's too late for me to test those right now,
but I trust they should work. Instructions can be found in
release/win32/README.txt. It required removing a link to "gw32" in
"setupext.py" so it only works trunk sdists. All interested, please
give it a try.

I'm in the process of testing this -- one problem I encountered was
some trailing spaces after the version numbers and before the comments
which broke the build. nfortunately some of the compiler error
messages were fairly cryptic (c compiler cannot create executables)
which sent me down some false trails before I figured it out from the
libpng config.log. I've committed the fix (first use of tortoisesvn
-- woohoo).

My sdist command in mingw is creating zip files by default -- any way
to force it to make the tar.gz that the build script is expecting ? I
can hack around this by creating my own tarball.....

JDH

Another note. I ran into problems when trying to create the build in
a directory that had spaces. (e.g. C:\Documents and Settings\....).
Try running everything in a top level dir.

- Charlie

···

On Thu, May 28, 2009 at 9:56 AM, Charlie Moad <cwmoad@...149...> wrote:

It's strange that I didn't run into spacing issues.

sdist --formats=gztar

- Charlie

On Thu, May 28, 2009 at 9:47 AM, John Hunter <jdh2358@...149...> wrote:

On Wed, May 27, 2009 at 11:08 PM, Charlie Moad <cwmoad@...149...> wrote:

I&#39;ve spent the last couple of days writing the win32 equivalent

of John's osx release scripts for MinGW. It patches distutils at
runtime to exclude the msvcr link, hence we should no longer need to
modify distutils. I have tested it on Python 2.6 and it should work
on 2.4 and 2.5 as well. It's too late for me to test those right now,
but I trust they should work. Instructions can be found in
release/win32/README.txt. It required removing a link to "gw32" in
"setupext.py" so it only works trunk sdists. All interested, please
give it a try.

I'm in the process of testing this -- one problem I encountered was
some trailing spaces after the version numbers and before the comments
which broke the build. nfortunately some of the compiler error
messages were fairly cryptic (c compiler cannot create executables)
which sent me down some false trails before I figured it out from the
libpng config.log. I've committed the fix (first use of tortoisesvn
-- woohoo).

My sdist command in mingw is creating zip files by default -- any way
to force it to make the tar.gz that the build script is expecting ? I
can hack around this by creating my own tarball.....

JDH

Did you maybe add the comments after you had built the dependencies,
so that branch wasn't executed when you compiled?

Thanks for the sdist flag -- I'll update the README.

I forgot to say -- thanks a lot for doing this. It will be a
tremendous boon for windows users (and us) and should help us plug
win32 into the buildbot down the road. BTW, have you tested these
builds with the png save error I CCd you on earlier? I'll test when I
finish the build...

JDH

···

On Thu, May 28, 2009 at 8:56 AM, Charlie Moad <cwmoad@...149...> wrote:

It's strange that I didn't run into spacing issues.

sdist --formats=gztar

Another note. I ran into problems when trying to create the build in
a directory that had spaces. (e.g. C:\Documents and Settings\....).
Try running everything in a top level dir.

That's not it -- I avoid spaces like the plague.

I am now getting a "cannot exec 'cc1plus'" error when I try and build
mpl itself (zlib, png, freetype and tcltl built fine). I can do::

  > gcc --version

but not::

  > g++ --version

Did you install something extra for g++ -- I installed mingw 5.1.4,
MSYS and wget as instructed....)

···

On Thu, May 28, 2009 at 8:57 AM, Charlie Moad <cwmoad@...149...> wrote:

Never mind -- just reran the mingw-5.1.4.exe installer and it prompted
me for additional compilers. I selected g++ and now I have it

JDH

···

On Thu, May 28, 2009 at 9:06 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 8:57 AM, Charlie Moad <cwmoad@...149...> wrote:

Another note. I ran into problems when trying to create the build in
a directory that had spaces. (e.g. C:\Documents and Settings\....).
Try running everything in a top level dir.

That's not it -- I avoid spaces like the plague.

I am now getting a "cannot exec 'cc1plus'" error when I try and build
mpl itself (zlib, png, freetype and tcltl built fine). I can do::

> gcc --version

but not::

> g++ --version

Did you install something extra for g++ -- I installed mingw 5.1.4,
MSYS and wget as instructed....)

Now I am hitting a missing setuptools import from setupwinegg.py.
Apparently python2.6 doesn't come with setuptools on windows (Is this
expected? It surprised me, I thought setuptools was supposed to be a
standard....)

Anyhow, a quick google takes me to

http://www.python-forum.org/pythonforum/viewtopic.php?f=15&t=10175

which suggests

1. Download setuptools-0.6c9.tar.gz
2. Download setuptools-0.6c9-py2.6.egg to a folder(directory) outside
your Windows Python installation folder
3. Use 7-zip to extract ez_setup.py in the same folder as
setuptools-0.6c9-py2.6.egg
4. Go to the corresponding dos prompt and run python ez_setup.py
setuptools-0.6c9-py2.6.egg from the command prompt
5. Ensure that your PATH includes the appropriate C:\Python2X\Scripts directory

Is this really the path of least resistance? Whatever happened to the
easy in ez_setup....

JDH

···

On Thu, May 28, 2009 at 9:25 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 9:06 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 8:57 AM, Charlie Moad <cwmoad@...149...> wrote:

Another note. I ran into problems when trying to create the build in
a directory that had spaces. (e.g. C:\Documents and Settings\....).
Try running everything in a top level dir.

That's not it -- I avoid spaces like the plague.

I am now getting a "cannot exec 'cc1plus'" error when I try and build
mpl itself (zlib, png, freetype and tcltl built fine). I can do::

> gcc --version

but not::

> g++ --version

Did you install something extra for g++ -- I installed mingw 5.1.4,
MSYS and wget as instructed....)

Never mind -- just reran the mingw-5.1.4.exe installer and it prompted
me for additional compilers. I selected g++ and now I have it

You found it. That's probably worth adding to the readme as well as
the "spaces in build path" issue. I think I did add some comments
after testing, btw.

···

On Thu, May 28, 2009 at 10:25 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 9:06 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 8:57 AM, Charlie Moad <cwmoad@...149...> wrote:

Another note. I ran into problems when trying to create the build in
a directory that had spaces. (e.g. C:\Documents and Settings\....).
Try running everything in a top level dir.

That's not it -- I avoid spaces like the plague.

I am now getting a "cannot exec 'cc1plus'" error when I try and build
mpl itself (zlib, png, freetype and tcltl built fine). I can do::

> gcc --version

but not::

> g++ --version

Did you install something extra for g++ -- I installed mingw 5.1.4,
MSYS and wget as instructed....)

Never mind -- just reran the mingw-5.1.4.exe installer and it prompted
me for additional compilers. I selected g++ and now I have it

I think setuptools should be a requirement in the same way numpy is.

The latest "ez_setup.py" script has 2.6 included.
http://peak.telecommunity.com/dist/ez_setup.py

You should be able to do the normal, "python ez_setup.py"

···

On Thu, May 28, 2009 at 10:34 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 9:25 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 9:06 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 8:57 AM, Charlie Moad <cwmoad@...149...> wrote:

Another note. I ran into problems when trying to create the build in
a directory that had spaces. (e.g. C:\Documents and Settings\....).
Try running everything in a top level dir.

That's not it -- I avoid spaces like the plague.

I am now getting a "cannot exec 'cc1plus'" error when I try and build
mpl itself (zlib, png, freetype and tcltl built fine). I can do::

> gcc --version

but not::

> g++ --version

Did you install something extra for g++ -- I installed mingw 5.1.4,
MSYS and wget as instructed....)

Never mind -- just reran the mingw-5.1.4.exe installer and it prompted
me for additional compilers. I selected g++ and now I have it

Now I am hitting a missing setuptools import from setupwinegg.py.
Apparently python2.6 doesn't come with setuptools on windows (Is this
expected? It surprised me, I thought setuptools was supposed to be a
standard....)

Anyhow, a quick google takes me to

http://www.python-forum.org/pythonforum/viewtopic.php?f=15&t=10175

which suggests

1. Download setuptools-0.6c9.tar.gz
2. Download setuptools-0.6c9-py2.6.egg to a folder(directory) outside
your Windows Python installation folder
3. Use 7-zip to extract ez_setup.py in the same folder as
setuptools-0.6c9-py2.6.egg
4. Go to the corresponding dos prompt and run python ez_setup.py
setuptools-0.6c9-py2.6.egg from the command prompt
5. Ensure that your PATH includes the appropriate C:\Python2X\Scripts directory

Is this really the path of least resistance? Whatever happened to the
easy in ez_setup....

OK, I got the thing to build and updated the README with instructions
for dummies like me. I tested the install and we are still getting
the libpng triggered segfault. I removed the win32 binaries from the
website yesterday since they showed the same problem. So now we have
to drill into this problem. Do you recall what version of libpng you
used for the 98.5.2 builds. I may try regressing and see if that
helps.

JDH

···

On Thu, May 28, 2009 at 9:37 AM, Charlie Moad <cwmoad@...149...> wrote:

I think setuptools should be a requirement in the same way numpy is.

The latest "ez_setup.py" script has 2.6 included.
http://peak.telecommunity.com/dist/ez_setup.py

You should be able to do the normal, "python ez_setup.py"

I'll have to check my home computer tonight. I never saw your note
about libpng either. Again, I'll poke a little more tonight once I
get home to my machine and update you.

- Charlie

···

On Thu, May 28, 2009 at 10:52 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 9:37 AM, Charlie Moad <cwmoad@...149...> wrote:

I think setuptools should be a requirement in the same way numpy is.

The latest "ez_setup.py" script has 2.6 included.
http://peak.telecommunity.com/dist/ez_setup.py

You should be able to do the normal, "python ez_setup.py"

OK, I got the thing to build and updated the README with instructions
for dummies like me. I tested the install and we are still getting
the libpng triggered segfault. I removed the win32 binaries from the
website yesterday since they showed the same problem. So now we have
to drill into this problem. Do you recall what version of libpng you
used for the 98.5.2 builds. I may try regressing and see if that
helps.

I'll have to check my home computer tonight. I never saw your note
about libpng either. Again, I'll poke a little more tonight once I
get home to my machine and update you.

The debugger indicates the dreaded mscvrt is still in the binaries,
despite your monkey patch. The only reference to that file in
distutils cygwincompiler is::

        # cygwin and mingw32 need different sets of libraries
        if self.gcc_version == "2.91.57":
            # cygwin shouldn't need msvcrt, but without the dlls will crash
            # (gcc version 2.91.57) -- perhaps something about initialization
            self.dll_libraries=["msvcrt"]
            self.warn(
                "Consider upgrading to a newer version of gcc")
        else:
            # Include the appropriate MSVC runtime library if Python was built
            # with MSVC 7.0 or later.
            self.dll_libraries = get_msvcr()

we shouldn't be matching that version string, and your monkey patch
*should* be overriding get_msvcr, but it is still getting in there.
Try in the matplotlib-0.98.6svn dir::

  > grep -ri mscvrt .

I get the following listing::

Binary file ./build/lib.win32-2.6/matplotlib/backends/_backend_agg.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/backends/_tkagg.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/ft2font.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/nxutils.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/ttconv.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_cntr.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_delaunay.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_image.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_path.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_png.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_windowing.pyd matches

JDH

···

On Thu, May 28, 2009 at 9:57 AM, Charlie Moad <cwmoad@...149...> wrote:

Hmm, not sure if this msvcrt is a red-herring or not. We may be
getting it by virtue of linking with numpy. If I run the same grep on
the numpy install from the 1.3.0 numpy binary, I get

Binary file ./core/multiarray.pyd matches
Binary file ./core/scalarmath.pyd matches
Binary file ./core/umath.pyd matches
Binary file ./core/umath_tests.pyd matches
Binary file ./core/_dotblas.pyd matches
Binary file ./core/_sort.pyd matches
./distutils/mingw32ccompiler.py: # with MSVC >= 7.0 (MinGW
standard is msvcrt)
./distutils/mingw32ccompiler.py: import msvcrt
./distutils/mingw32ccompiler.py: if hasattr(msvcrt,
"CRT_ASSEMBLY_VERSION"):
./distutils/mingw32ccompiler.py: _MSVCRVER_TO_FULLVER['90']
= msvcrt.CRT_ASSEMBLY_VERSION
./distutils/mingw32ccompiler.py: log.warn('Cannot import
msvcrt: using manifest will not be possible')
./distutils/mingw32ccompiler.py: raise ValueError("Version
%d,%d of MSVCRT not supported yet" \
Binary file ./distutils/mingw32ccompiler.pyc matches
Binary file ./distutils/mingw32ccompiler.pyo matches
Binary file ./fft/fftpack_lite.pyd matches
Binary file ./lib/_compiled_base.pyd matches
Binary file ./linalg/lapack_lite.pyd matches
Binary file ./numarray/_capi.pyd matches
./numpy.out:./distutils/mingw32ccompiler.py: # with MSVC >= 7.0
(MinGW standard is msvcrt)
./numpy.out:./distutils/mingw32ccompiler.py: import msvcrt
./numpy.out:./distutils/mingw32ccompiler.py: if hasattr(msvcrt,
"CRT_ASSEMBLY_VERSION"):
./numpy.out:./distutils/mingw32ccompiler.py:
_MSVCRVER_TO_FULLVER['90'] = msvcrt.CRT_ASSEMBLY_VERSION
./numpy.out:./distutils/mingw32ccompiler.py: log.warn('Cannot
import msvcrt: using manifest will not be possible')
./numpy.out:./distutils/mingw32ccompiler.py: raise
ValueError("Version %d,%d of MSVCRT not supported yet" \
Binary file ./random/mtrand.pyd matches

Sometimes these missing dlls are not the true cause of the segfault....

JDH

···

On Thu, May 28, 2009 at 10:01 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, May 28, 2009 at 9:57 AM, Charlie Moad <cwmoad@...149...> wrote:

I'll have to check my home computer tonight. I never saw your note
about libpng either. Again, I'll poke a little more tonight once I
get home to my machine and update you.

The debugger indicates the dreaded mscvrt is still in the binaries,
despite your monkey patch. The only reference to that file in
distutils cygwincompiler is::

   \# cygwin and mingw32 need different sets of libraries
   if self\.gcc\_version == &quot;2\.91\.57&quot;:
       \# cygwin shouldn&#39;t need msvcrt, but without the dlls will crash
       \# \(gcc version 2\.91\.57\) \-\- perhaps something about initialization
       self\.dll\_libraries=\[&quot;msvcrt&quot;\]
       self\.warn\(
           &quot;Consider upgrading to a newer version of gcc&quot;\)
   else:
       \# Include the appropriate MSVC runtime library if Python was built
       \# with MSVC 7\.0 or later\.
       self\.dll\_libraries = get\_msvcr\(\)

we shouldn't be matching that version string, and your monkey patch
*should* be overriding get_msvcr, but it is still getting in there.
Try in the matplotlib-0.98.6svn dir::

> grep -ri mscvrt .

I get the following listing::

Binary file ./build/lib.win32-2.6/matplotlib/backends/_backend_agg.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/backends/_tkagg.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/ft2font.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/nxutils.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/ttconv.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_cntr.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_delaunay.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_image.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_path.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_png.pyd matches
Binary file ./build/lib.win32-2.6/matplotlib/_windowing.pyd matches