Hi all,
With the integration of mathtex into matplotlib nearing completion (just bug fixes really) I think now is a good time to be considering the best way to include mathtex into matplotlib.
This has already been discussed on the mathtex mailing list, with Michael proposing a few ways of doing this. However, I am not an expert by any means when it comes to Python set-up/configuring. (As anyone who has looked at setup.py in mathtex will have seen.)
While including the source is not difficult (it can be done directly or using svn:external) getting it built/configured is. Lets say that mathtex was dumped into lib/, how would one go about configuring and installing it from setup.py in matplotlib.
Although I am sure that just executing a shell command would do it I am sure there must be a 'better' option for this type of 'package chaining'.
Does anyone have any suggestions or recommendations?
Regards, Freddie.
There may be a setuptools solution here, but if there is, I'm not the one to know 
matplotlib is for the most part ignorant of setuptools, and it's probably reasonable to keep it that way.
Anyway, since the mathtex setup infrastructure is based on what matplotlib was already doing, there's a common convention we can exploit. Essentially, the matplotlib setup.py builds up a list of extension modules (ext_modules) and packages and then passes those lists to distutils for building. So, in theory, all mathtex needs to do is provide a function that will add extension modules and packages to those lists (basically like all of the build_* methods in setupext.py). So basically, matplotlib's setup.py would import lib/mathtex/setupext.py (by filename) and call a method in it. Lots of details I'm missing, but that should provide a general framework.
Another issue this raises is whether to build the FT2Font and png modules twice, once as part of matplotlib, and once as part of mathtex. Once mathtex is a truly external dependency for matplotlib, I don't see a way around this, so maybe we should just pretend we're already there, despite the duplication. If we want to be clever, I could see mathtex being smart about imports: try importing its local copies of its libraries and failing that import matplotlib's. I'm not entirely sure about that idea, but I sort of feel "hacky-if-you-do, hacky-if-you-don't" here
I see the code maintenance problem of this duplication (i.e. making sure bugfixes to FT2Font make it into matplotlib and mathtex) almost as a separate issue. We know the solution to that: break out the freetype wrappers into its own project (which then both matplotlib and mathtex would rely on) -- but that's probably outside of the scope of this GSoC project.
Please try to use svn:externals if you can -- that will make pulling updates from mathtex easier. I've never used it cross-repository like this, so there may be unforeseen issues.
It also just occurred to me that we might want to take another step in preparation for mathtex as an external dependency: make it optional. That is, if importing mathtex fails, be able to render regular text, and warn if trying to render math text.
Mike
Freddie Witherden wrote:
···
Hi all,
With the integration of mathtex into matplotlib nearing completion (just bug fixes really) I think now is a good time to be considering the best way to include mathtex into matplotlib.
This has already been discussed on the mathtex mailing list, with Michael proposing a few ways of doing this. However, I am not an expert by any means when it comes to Python set-up/configuring. (As anyone who has looked at setup.py in mathtex will have seen.)
While including the source is not difficult (it can be done directly or using svn:external) getting it built/configured is. Lets say that mathtex was dumped into lib/, how would one go about configuring and installing it from setup.py in matplotlib.
Although I am sure that just executing a shell command would do it I am sure there must be a 'better' option for this type of 'package chaining'.
Does anyone have any suggestions or recommendations?
Regards, Freddie.
------------------------------------------------------------------------------
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
Could you please the advantages of using mathtext? How does it compare and contrast to the existing interface?
Any pointers to read before getting more lost?
Thanks,
···
On Thu, Jul 23, 2009 at 8:36 AM, Freddie Witherden <freddie@…718…> wrote:
Hi all,
With the integration of mathtex into matplotlib nearing completion
(just bug fixes really) I think now is a good time to be considering
the best way to include mathtex into matplotlib.
This has already been discussed on the mathtex mailing list, with
Michael proposing a few ways of doing this. However, I am not an
expert by any means when it comes to Python set-up/configuring. (As
anyone who has looked at setup.py in mathtex will have seen.)
While including the source is not difficult (it can be done directly
or using svn:external) getting it built/configured is. Lets say that
mathtex was dumped into lib/, how would one go about configuring and
installing it from setup.py in matplotlib.
Although I am sure that just executing a shell command would do it I
am sure there must be a ‘better’ option for this type of 'package
chaining’.
Does anyone have any suggestions or recommendations?
Regards, Freddie.
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
–
Gökhan
Hi,
Could you please the advantages of using mathtext? How does it compare and contrast to the existing interface?
The interface itself is mostly the same, although there are a few enhancements to make it easier to use. To main advantages of using mathtex are that, being an external project, it is likely to experience a wider audience than the current code in matplotlib.
This should result in a package that is better maintained with more features. Currently, however, it is almost exactly the same as the mathtext code in matplotlib.
Any pointers to read before getting more lost?
There is the project code site here: Google Code Archive - Long-term storage for Google Code Project Hosting. while my project blog is here: http://gsoc-mathtex.blogspot.com/
Regards, Freddie.
···
On 23 Jul 2009, at 22:46, Gökhan SEVER wrote:
Hi,
Anyway, since the mathtex setup infrastructure is based on what matplotlib was already doing, there's a common convention we can exploit. Essentially, the matplotlib setup.py builds up a list of extension modules (ext_modules) and packages and then passes those lists to distutils for building. So, in theory, all mathtex needs to do is provide a function that will add extension modules and packages to those lists (basically like all of the build_* methods in setupext.py). So basically, matplotlib's setup.py would import lib/mathtex/setupext.py (by filename) and call a method in it. Lots of details I'm missing, but that should provide a general framework.
Not sure about the extension modules + lists stuff, although setuptools is not my area of expertise. Sadly executing the lib/mathtex/setup.py file isn't really an option either, so I'll need to think on it. Might need to ask for some help though.
Another issue this raises is whether to build the FT2Font and png modules twice, once as part of matplotlib, and once as part of mathtex. Once mathtex is a truly external dependency for matplotlib, I don't see a way around this, so maybe we should just pretend we're already there, despite the duplication. If we want to be clever, I could see mathtex being smart about imports: try importing its local copies of its libraries and failing that import matplotlib's. I'm not entirely sure about that idea, but I sort of feel "hacky-if-you-do, hacky-if-you-don't" here
I don't think it hurts having two FT2Font wrappers lying around on a system.
Please try to use svn:externals if you can -- that will make pulling updates from mathtex easier. I've never used it cross-repository like this, so there may be unforeseen issues.
svn:externals seems to work quite well and is currently used in the mathtex branch to pull mathtex.
It also just occurred to me that we might want to take another step in preparation for mathtex as an external dependency: make it optional. That is, if importing mathtex fails, be able to render regular text, and warn if trying to render math text.
Shouldn't be too hard to add, I'll write in backend support today.
Regards, Freddie.
Hi,
I was thinking about the problem of including mathtex in matplotlib earlier
and came up with an alternative means of 'solving' the problem.
Instead of hacking setup.py to install mathtex on the behalf of matplotlib it
may be easier to leave it up to the user/packager to install mathtex.
While simplifying the code (not as many changed are need to the setup files) it
also eliminates the problems associated with one package installing another
package (matplotlib installing mathtex).
This could be done either through ones distribution package manger, standalone
(getting the mathtex source) or just by following the instructions in
lib/mathtex/INSTALL. (Checking out matplotlib also checks out the mathtex
source into lib/mathtex.)
However, I am interested if this solution is acceptable to the matplotlib
developers.
It may also be worth pointing out that mathtex is an optional dependency of
matplotlib and is not required for matplotlib to function.
Regards, Freddie.
I think we need (at least as a transitional stopgap) a single "python setup.py install" to install both matplotlib and mathtex. Once distributions catch up (which could take more than a year, depending on cycles), we can consider being more loosely coupled.
There are already examples of installing subpackages from matplotlib (pytz and dateutil) for example, and each of the extensions (_backend_agg etc.) have examples in setupext.py showing how to dynamically add extensions and packages to the list of things that matplotlib's setup.py will install and build. Are there fundamental differences in how that works vs. how mathtex needs to work that is a stumbling block? I must admit I haven't thought it all the way through, but I'm surprised that the existing examples there don't provide a template for how to deal with mathtex. That said, I know that distutils can be rather, um, labyrinthine.
Cheers,
Mike
Freddie Witherden wrote:
···
Hi,
I was thinking about the problem of including mathtex in matplotlib earlier and came up with an alternative means of 'solving' the problem.
Instead of hacking setup.py to install mathtex on the behalf of matplotlib it may be easier to leave it up to the user/packager to install mathtex.
While simplifying the code (not as many changed are need to the setup files) it also eliminates the problems associated with one package installing another package (matplotlib installing mathtex).
This could be done either through ones distribution package manger, standalone (getting the mathtex source) or just by following the instructions in lib/mathtex/INSTALL. (Checking out matplotlib also checks out the mathtex source into lib/mathtex.)
However, I am interested if this solution is acceptable to the matplotlib developers.
It may also be worth pointing out that mathtex is an optional dependency of matplotlib and is not required for matplotlib to function.
Regards, Freddie.
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
I second this. I would like to see the new mathtext (and png and
freetype wrappers as necessary, possibly in mathtext) live in the
matplotlib trunk under lib, the same place we put pytz and dateutil.
It can have its own setup and release cycle, but I think it would make
matters simpler to have it there. If there is compelling reason to
have it live in a separate svn repo, we can always grab it externally,
as scipy does for our sphinx extensions.
We can use the mpl configure setup.cfg to either
no-install/force-install/auto-install as we do for these other
packages. This works fine at build time, but is a little more
complicated when distributing binaries. Eg for win32 binaries, we
currently force these packages on, which is sometimes a problem. We
might be able to be a little smarter about building win32 installers,
so the user can select which subpackages to install, but I am not
holding my breath on this one.
I am unwilling to rely on any setuptools solutions at this point, as
I've always viewed them with suspicion (maybe because I rarely found
easy_install to be easy) and the setuptools community is in a bit of
disarray, with the lead developer unable to adequately maintain the
package and unwilling to let those who want to take it over do so
since he feels they are not qualified, so a fork is in progress:
http://www.mail-archive.com/distutils-sig@...147.../msg08111.html
JDH
···
On Wed, Jul 29, 2009 at 7:52 AM, Michael Droettboom<mdroe@...31...> wrote:
I think we need (at least as a transitional stopgap) a single "python
setup.py install" to install both matplotlib and mathtex. Once
distributions catch up (which could take more than a year, depending on
cycles), we can consider being more loosely coupled.
There are already examples of installing subpackages from matplotlib
(pytz and dateutil) for example, and each of the extensions
(_backend_agg etc.) have examples in setupext.py showing how to
dynamically add extensions and packages to the list of things that
matplotlib's setup.py will install and build. Are there fundamental
differences in how that works vs. how mathtex needs to work that is a
stumbling block? I must admit I haven't thought it all the way through,
but I'm surprised that the existing examples there don't provide a
template for how to deal with mathtex. That said, I know that distutils
can be rather, um, labyrinthine.
Instead of using the matplotlib package as a catch-all for these external packages, could you instead keep them all nominally separate and release a tarball that includes all of them side by side? You can even have an installation script that will run the setups for each. Even better, this installation script wouldn't use distutils, so you don't have to hack all of this "optional package" stuff into it.
···
On 2009-07-29 08:17, John Hunter wrote:
On Wed, Jul 29, 2009 at 7:52 AM, Michael Droettboom<mdroe@...31...> wrote:
I think we need (at least as a transitional stopgap) a single "python
setup.py install" to install both matplotlib and mathtex. Once
distributions catch up (which could take more than a year, depending on
cycles), we can consider being more loosely coupled.
There are already examples of installing subpackages from matplotlib
(pytz and dateutil) for example, and each of the extensions
(_backend_agg etc.) have examples in setupext.py showing how to
dynamically add extensions and packages to the list of things that
matplotlib's setup.py will install and build. Are there fundamental
differences in how that works vs. how mathtex needs to work that is a
stumbling block? I must admit I haven't thought it all the way through,
but I'm surprised that the existing examples there don't provide a
template for how to deal with mathtex. That said, I know that distutils
can be rather, um, labyrinthine.
I second this. I would like to see the new mathtext (and png and
freetype wrappers as necessary, possibly in mathtext) live in the
matplotlib trunk under lib, the same place we put pytz and dateutil.
It can have its own setup and release cycle, but I think it would make
matters simpler to have it there. If there is compelling reason to
have it live in a separate svn repo, we can always grab it externally,
as scipy does for our sphinx extensions.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
I second this. I would like to see the new mathtext (and png and
freetype wrappers as necessary, possibly in mathtext) live in the
matplotlib trunk under lib, the same place we put pytz and dateutil.
It can have its own setup and release cycle, but I think it would make
matters simpler to have it there. If there is compelling reason to
have it live in a separate svn repo, we can always grab it externally,
as scipy does for our sphinx extensions.
We can use the mpl configure setup.cfg to either
no-install/force-install/auto-install as we do for these other
packages. This works fine at build time, but is a little more
complicated when distributing binaries. Eg for win32 binaries, we
currently force these packages on, which is sometimes a problem. We
might be able to be a little smarter about building win32 installers,
so the user can select which subpackages to install, but I am not
holding my breath on this one.
After thinking about the problem a little bit I have gone for the following
solution. I added support for mathtex to use the wrappers provided by
matplotlib for FT2Font and _png. It is likely we will want to keep them in
sync anyway -- so that should not be an issue.
However, I am having trouble working around the fact that mathtex is two
levels deep (lib/mathtex/mathtex). I can either get the setup.py file to
install either just the setup* stuff from mathtex or install mathtex as
mathtex/mathtex. Not sure how to work around this.
I have committed what I have thus far to the branch.
Regards, Freddie.
Furthermore, you could put numpy in there, too. Maybe even recipes for downloading and building the C dependencies for those who need it. Basically, by freeing yourself from the shackles of distutils, you can make the "new-to-Python-just-give-me-matplotlib" experience much less painful while improving the situation for downstream packagers like Linux distributions, EPD, and python(x,y) at the same time.
···
On 2009-07-29 15:29, Robert Kern wrote:
On 2009-07-29 08:17, John Hunter wrote:
On Wed, Jul 29, 2009 at 7:52 AM, Michael Droettboom<mdroe@...31...> wrote:
I think we need (at least as a transitional stopgap) a single "python
setup.py install" to install both matplotlib and mathtex. Once
distributions catch up (which could take more than a year, depending on
cycles), we can consider being more loosely coupled.
There are already examples of installing subpackages from matplotlib
(pytz and dateutil) for example, and each of the extensions
(_backend_agg etc.) have examples in setupext.py showing how to
dynamically add extensions and packages to the list of things that
matplotlib's setup.py will install and build. Are there fundamental
differences in how that works vs. how mathtex needs to work that is a
stumbling block? I must admit I haven't thought it all the way through,
but I'm surprised that the existing examples there don't provide a
template for how to deal with mathtex. That said, I know that distutils
can be rather, um, labyrinthine.
I second this. I would like to see the new mathtext (and png and
freetype wrappers as necessary, possibly in mathtext) live in the
matplotlib trunk under lib, the same place we put pytz and dateutil.
It can have its own setup and release cycle, but I think it would make
matters simpler to have it there. If there is compelling reason to
have it live in a separate svn repo, we can always grab it externally,
as scipy does for our sphinx extensions.
Instead of using the matplotlib package as a catch-all for these external
packages, could you instead keep them all nominally separate and release a
tarball that includes all of them side by side? You can even have an
installation script that will run the setups for each. Even better, this
installation script wouldn't use distutils, so you don't have to hack all of
this "optional package" stuff into it.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
After thinking about the problem a little bit I have gone for the following
solution. I added support for mathtex to use the wrappers provided by
matplotlib for FT2Font and _png. It is likely we will want to keep them in
sync anyway -- so that should not be an issue.
Isn't this a bit backwards? The point of the GSOC project is to make
the mathtext project available as a smaller, lighter, more widely
accessible project. If it depends on mpl, what have we achieved
except complexity? Don't we want the ft2font and png to be provided
by mathtext, and let mpl depend on it to get the stuff there?
Admittedly, mathtex may not be the right name (mpltext?) but
conceptually it seems like we want the low level stuff in mathtex. Or
am I missing your point here?
However, I am having trouble working around the fact that mathtex is two
levels deep (lib/mathtex/mathtex). I can either get the setup.py file to
install either just the setup* stuff from mathtex or install mathtex as
mathtex/mathtex. Not sure how to work around this.
You can punt on the setup/dependency issue for now. In the branch,
you can assume mathtex exists and is installed. We can worry about
how to handle the setup and distribution issues once we have the
package organization rationalized. A simple set of install
instructions for developers with very explicit commands to check out
the branch, install the deps, and test would help us test as you
progress.
JDH
···
On Wed, Jul 29, 2009 at 4:32 PM, Freddie Witherden<freddie@...718...> wrote:
For linux, unix and os x src installs, this is imminently viable. We
already have a start on this in the mpl/releases dir to autofetch the
deps and build them, but this is for the lower level deps, like
freetype, png and zlib. We could easily extend the approach to pytz
and dateutil, and provide some enhancements (remove make, use python
instead, have some sort of configure file support). The bugaboo is
the win32 installer and OSX binary. If we don't build this stuff in,
suddenly the dependency list grows from "just install numpy" to
"install pytz, dateutil, numpy and mathtext" which is a very different
picture. As I alluded to in my earlier post, I assume there is some
way to build smart windows installers (eg along the lines of the ETS
install tool or native win32 solutions) but this is a question of
manpower. None of our core developers are primarily win32 users, and
we often struggle to get anyone to even test a win32 release
candidate, much less do the heavy lifting to build a smart installer.
The ETS and PythonXY solutions have gotten so good that maybe I should
just give up worrying making life easy for win32 users, since these
solutions already satisfy the needs of someone who wants it to work
with one or two clicks. I already aggressively point them to these
solutions on our installing page. But windows users still make up
about half of our known installations, so I don't want to be cavalier
in dismissing the importance of making it easy for them.
···
On Wed, Jul 29, 2009 at 5:05 PM, Robert Kern<robert.kern@...149...> wrote:
Furthermore, you could put numpy in there, too. Maybe even recipes for
downloading and building the C dependencies for those who need it. Basically, by
freeing yourself from the shackles of distutils, you can make the
"new-to-Python-just-give-me-matplotlib" experience much less painful while
improving the situation for downstream packagers like Linux distributions, EPD,
and python(x,y) at the same time.
Him
Isn't this a bit backwards? The point of the GSOC project is to make
the mathtext project available as a smaller, lighter, more widely
accessible project. If it depends on mpl, what have we achieved
except complexity? Don't we want the ft2font and png to be provided
by mathtext, and let mpl depend on it to get the stuff there?
Admittedly, mathtex may not be the right name (mpltext?) but
conceptually it seems like we want the low level stuff in mathtex. Or
am I missing your point here?
I should have made myself clearer. Mathtex will attempt to firstly import its own versions of FT2Font and friends. Should this fail it will then attempt to import those in matplotlib. So yes, mathtex still has all of the required dependencies, nothing has changed there.
This means that if you have matplotlib installed you can use/install mathtex without compiling a single C-code extension. It also means that when matplotlib wishes to install mathtex it only need bother with copying over .py files -- the same way it does for pytz.
You can punt on the setup/dependency issue for now. In the branch,
you can assume mathtex exists and is installed. We can worry about
how to handle the setup and distribution issues once we have the
package organization rationalized. A simple set of install
instructions for developers with very explicit commands to check out
the branch, install the deps, and test would help us test as you
progress.
Okay, I'll revert to the previous way of doing so (cd'ing to the directory where mathtex is checked out and running setup.py there).
Regards, Freddie.
···
On 30 Jul 2009, at 02:27, John Hunter wrote: