# zorder seems to cause problems when embed python in latex files

LaTeX can accept embedded Python code with a python.sty file.

This is handy to dynamically generate plots with Matplotlib for a LaTeX slide
presentation.

I successfully embedded lots of matplotlib plot code into my slides
and then had problems with zorder.

For some reason zorder seems to mess up the footer of my Beamer/LaTeX slides.
(For some reason zorder setting make the footer shrink in size.)

Is there any weirdness or side effects about zorder I should be aware of that
would explain this?

Chris

chris@...1388... wrote:

LaTeX can accept embedded Python code with a python.sty file.

This is handy to dynamically generate plots with Matplotlib for a LaTeX slide
presentation.

I successfully embedded lots of matplotlib plot code into my slides
and then had problems with zorder.

For some reason zorder seems to mess up the footer of my Beamer/LaTeX slides.
(For some reason zorder setting make the footer shrink in size.)

Is there any weirdness or side effects about zorder I should be aware of that
would explain this?

My best guess is that when the elements of the plot are in a particular order, the "last drawn" element has some setting that is not getting reverted back when going back to the LaTeX part of the slide. In general, matplotlib doesn't explicitly try to be careful about state in its output since it is really the embedding applications job (in this case Beamer/LaTeX) that is supposed to ensure that anything it embeds does not have external side effects.

But I've never tried this combination of python.sty/Beamer/LaTeX personally. Can you send us the output of your plot on its own (ps or pdf...)?

Mike

···

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

But I've never tried this combination of python.sty/Beamer/LaTeX
personally. Can you send us the output of your plot on its own (ps or
pdf...)?

Mike

Wow thanks. I was afraid no one would respond. I suppose if Matplotlib is so
mature that we are left worrying about bugs when embedding it in Latex then it
has come a long way.

I created a tiny test.tex that shows the problem. Just type

pdflatex -shell-escape test.tex ; xpdf test.pdf

in the same directory with the 2 files test.tex and python.sty below and
attached. I also attached the final PDF since you asked for it.

\documentclass{beamer}
\usepackage{beamerthemesplit}
\usepackage{graphicx}
\usepackage{python}
\begin{document}
\title[Example of zorder trouble]{Example of zorder trouble}
\author[Matplotlib fan]{Matplotlib fan}
\date{}
\frame{\titlepage}

\begin{frame}[fragile]\frametitle{No zorder - notice footer ok}
\begin{python}
import pylab
pylab.plot(range(10), range(10))
pylab.scatter([5], [5])
pylab.savefig('plot1.pdf')
print r'\includegraphics[width=200pt]{plot1.pdf}'
\end{python}
\end{frame}

\begin{frame}[fragile]\frametitle{zorder - notice footer messed up}
\begin{python}
import pylab
pylab.plot(range(10), range(10), zorder = 1)
pylab.scatter([5], [5], zorder = 9)
pylab.savefig('plot2.pdf')
print r'\includegraphics[width=200pt]{plot2.pdf}'
\end{python}
\end{frame}

\end{document}

%% This program is free software; you can redistribute it and/or
%% modify it under the terms of the GNU General Public License
%%
%% This program is distributed in the hope that it will be useful,
%% but WITHOUT ANY WARRANTY; without even the implied warranty of
%% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
%% GNU General Public License for more details.
%%
%% You should have received a copy of the GNU General Public License
%% along with this program; if not, write to the Free Software
%% Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
%%
%% Author: Martin R. Ehmsen, ehmsen@...2204...
%% Department of Mathematics and Computer Science,
%% University of Southern Denmark, DK
%%
%% You can find an online copy of the GPL at
%% http://www.gnu.org/copyleft/gpl.html .
%%
%% Note: shell-escape needs to be activated for this to work.
%% This can either be done by passing -shell-escape as an option to
%% latex or by adding/changing "shell_escape = t" in your texmf.cnf .

% 0.2 -> 0.21: Moved \newwrite\@module from \@writemodule and out, since
% no more than 15 \newwrites are allowed (and the previous version created a new
% every time \@writemodule was called.

\NeedsTeXFormat{LaTeX2e}[1994/12/01]
\ProvidesPackage{python}[2007/06/07 v0.21 Python in LaTeX]

\newwrite\@out
\newwrite\@module

\begingroup \catcode |=0 \catcode [=1
\catcode]=2 \catcode \{=12 \catcode \}=12
\catcode\\=12 |gdef|@xpython#1\end{python}[|immediate|write|@out[#1]|end[python]]

endgroup

\def\python{\kernel@...2205... [{\@python}{\@python[]}}

\def\@python[#1]{%
\gdef\@pythoninclude{#1}
\immediate\openout\@out=\jobname.py
\newlinechar='15
\begingroup \catcode\^^M=12 %
\let\do\@makeother\dospecials\obeyspaces%
\@xpython}

\def\endpython{%
\endgroup
\immediate\closeout\@out
\@writemodule
\immediate\write18{cat \@pythoninclude\space\jobname.py | python > \jobname.py.out 2> \jobname.py.err}
\immediate\input\jobname.py.out}
%\immediate\write{\begin{verbatim}}
%\immediate\input\jobname.py.err
%\immediate\write{\end{verbatim}}}

\def\@writemodule{%
\immediate\openout\@module=latex.py
\immediate\write\@module{jobname="\jobname"}
\immediate\closeout\@module}

% BUGS:
% 1. If anything gets send to stderr then it should be included
% in \begin{verbatim}...\end{verbatim} to be properly displayed
%
% \immediate\write18{cat \@pythoninclude\space\jobname.py | python > \jobname.py.out 2>\jobname.py.err}
%
% 2. Watch out for indentation done by aucTeX in Emacs
%
% 3. Let the package accept a "final version" option, such
% that the output of each python run is saved such that it can be
% inserted into the document by hand
% (conference, journals are not likely to compile with
% shell_escape or have python).
%
% \gdef\@prepython{}
% \def\prepython#1{%
% \gdef\@prepython{#1}
% }
% sed -e 's/^ //g' cluster.py
% \immediate\write18{\@prepython\space\jobname.py > \

test.tex (829 Bytes)

python.sty (2.97 KB)

test.pdf (41.5 KB)

···

On Tue, Oct 14, 2008 at 08:21:08AM -0400, Michael Droettboom wrote:

With the file you sent, I can see the messed up footer in xpdf, but not in acroread. There are a number of times that I have seen xpdf not completely support the PDF spec, and this may be one of them.

Creating my own files, however, I'm not able to reproduce this here.

When I compile your test.tex, I get an error, even though it seems to have loaded the python.sty file. Not sure why (see attached log).

When I generate the plots "offline", and then hack test.tex to simply include the files, everything works fine, and I don't see a problem with the footers with either xpdf or acroread.

There was a recent bug discovered in matplotlib where PDF files weren't always getting flushed completely. I don't *think* that's the cause of this, but if you could reproduce what I did (generate the plots independently of TeX and then load them), and that works for you, that might point to something like that.

There was also a bug a few months back where xpdf didn't like the way matplotlib handled reusing the same graphic multiple times (which is used for markers). That may be why you're seeing the footer bug and I'm not -- you didn't mention which version of matplotlib you're running, so it's hard to say.

Here's my versions of all the various moving pieces:

pdfTeX (Web2C 7.4.5) 3.14159-1.10b
kpathsea version 3.4.5
Copyright (C) 1997-2003 Han The Thanh.

Beamer 3.00
python 2.5.2
matplotlib SVN (today)
xpdf 3.00

That python.sty stuff looks really cool, by the way. I haven't come across it before.

Mike

chris@...1388... wrote:

test.broken.log (25.7 KB)

test.tex (537 Bytes)

···

On Tue, Oct 14, 2008 at 08:21:08AM -0400, Michael Droettboom wrote:

But I've never tried this combination of python.sty/Beamer/LaTeX
personally. Can you send us the output of your plot on its own (ps or
pdf...)?

Mike

Wow thanks. I was afraid no one would respond. I suppose if Matplotlib is so
mature that we are left worrying about bugs when embedding it in Latex then it
has come a long way.

I created a tiny test.tex that shows the problem. Just type

pdflatex -shell-escape test.tex ; xpdf test.pdf

in the same directory with the 2 files test.tex and python.sty below and
attached. I also attached the final PDF since you asked for it.

\documentclass{beamer}
\usepackage{beamerthemesplit}
\usepackage{graphicx}
\usepackage{python}
\begin{document}
\title[Example of zorder trouble]{Example of zorder trouble}
\author[Matplotlib fan]{Matplotlib fan}
\date{}
\frame{\titlepage}

\begin{frame}[fragile]\frametitle{No zorder - notice footer ok}
\begin{python}
import pylab
pylab.plot(range(10), range(10))
pylab.scatter([5], [5])
pylab.savefig('plot1.pdf')
print r'\includegraphics[width=200pt]{plot1.pdf}'
\end{python}
\end{frame}

\begin{frame}[fragile]\frametitle{zorder - notice footer messed up}
\begin{python}
import pylab
pylab.plot(range(10), range(10), zorder = 1)
pylab.scatter([5], [5], zorder = 9)
pylab.savefig('plot2.pdf')
print r'\includegraphics[width=200pt]{plot2.pdf}'
\end{python}
\end{frame}

\end{document}

%% This program is free software; you can redistribute it and/or
%% modify it under the terms of the GNU General Public License
%%
%% This program is distributed in the hope that it will be useful,
%% but WITHOUT ANY WARRANTY; without even the implied warranty of
%% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
%% GNU General Public License for more details.
%%
%% You should have received a copy of the GNU General Public License
%% along with this program; if not, write to the Free Software
%% Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
%%
%% Author: Martin R. Ehmsen, ehmsen@...2204...
%% Department of Mathematics and Computer Science,
%% University of Southern Denmark, DK
%%
%% You can find an online copy of the GPL at
%% http://www.gnu.org/copyleft/gpl.html .
%%
%% Note: shell-escape needs to be activated for this to work.
%% This can either be done by passing -shell-escape as an option to
%% latex or by adding/changing "shell_escape = t" in your texmf.cnf .

% 0.2 -> 0.21: Moved \newwrite\@module from \@writemodule and out, since
% no more than 15 \newwrites are allowed (and the previous version created a new
% every time \@writemodule was called.

\NeedsTeXFormat{LaTeX2e}[1994/12/01]
\ProvidesPackage{python}[2007/06/07 v0.21 Python in LaTeX]

\newwrite\@out
\newwrite\@module

\begingroup \catcode |=0 \catcode [=1
\catcode]=2 \catcode \{=12 \catcode \}=12
\catcode\\=12 |gdef|@xpython#1\end{python}[|immediate|write|@out[#1]|end[python]]
>endgroup

\def\python{\kernel@...2205... [{\@python}{\@python[]}}

\def\@python[#1]{%
\gdef\@pythoninclude{#1}
\immediate\openout\@out=\jobname.py
\newlinechar='15
\begingroup \catcode\^^M=12 %
\let\do\@makeother\dospecials\obeyspaces%
\@xpython}

\def\endpython{%
\endgroup
\immediate\closeout\@out
\@writemodule
\immediate\write18{cat \@pythoninclude\space\jobname.py | python > \jobname.py.out 2> \jobname.py.err}
\immediate\input\jobname.py.out}
%\immediate\write{\begin{verbatim}}
%\immediate\input\jobname.py.err
%\immediate\write{\end{verbatim}}}

\def\@writemodule{%
\immediate\openout\@module=latex.py
\immediate\write\@module{jobname="\jobname"}
\immediate\closeout\@module}

% BUGS:
% 1. If anything gets send to stderr then it should be included
% in \begin{verbatim}...\end{verbatim} to be properly displayed
%
% \immediate\write18{cat \@pythoninclude\space\jobname.py | python > \jobname.py.out 2>\jobname.py.err}
%
% 2. Watch out for indentation done by aucTeX in Emacs
%
% 3. Let the package accept a "final version" option, such
% that the output of each python run is saved such that it can be
% inserted into the document by hand
% (conference, journals are not likely to compile with
% shell_escape or have python).
%
% \gdef\@prepython{}
% \def\prepython#1{%
% \gdef\@prepython{#1}
% }
% sed -e 's/^ //g' cluster.py
% \immediate\write18{\@prepython\space\jobname.py > \

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

With the file you sent, I can see the messed up footer in xpdf, but not
in acroread. There are a number of times that I have seen xpdf not
completely support the PDF spec, and this may be one of them.

I installed acroread and I was also able to view everything fine with it
whereas xpdf and evince showed the bug.

Creating my own files, however, I'm not able to reproduce this here.

Hmmm. This would imply that your versions somehow do something that makes xpdf
happy.

When I compile your test.tex, I get an error, even though it seems to
have loaded the python.sty file. Not sure why (see attached log).

Looks like it doesn't know what \begin{python} means. Did you put python.sty
in the same directory as text.tex?

When I generate the plots "offline", and then hack test.tex to simply
include the files, everything works fine, and I don't see a problem with
the footers with either xpdf or acroread.

I see the problem with xpdf when I simply include the plots which means that
python.sty was just a red herring. python.sty isn't the problem.

There was a recent bug discovered in matplotlib where PDF files weren't
always getting flushed completely. I don't *think* that's the cause of
this, but if you could reproduce what I did (generate the plots
independently of TeX and then load them), and that works for you, that
might point to something like that.

xpdf still croaks for me. Perhaps using the bleeding edge of matplotlib would
fix? I'm using Ubuntu 8.04 which has python-matplotlib version
0.91.2-0ubuntu1.

There was also a bug a few months back where xpdf didn't like the way
matplotlib handled reusing the same graphic multiple times (which is
used for markers). That may be why you're seeing the footer bug and I'm
not -- you didn't mention which version of matplotlib you're running, so
it's hard to say.

Here's my versions of all the various moving pieces:

pdfTeX (Web2C 7.4.5) 3.14159-1.10b
kpathsea version 3.4.5
Copyright (C) 1997-2003 Han The Thanh.

Beamer 3.00
python 2.5.2
matplotlib SVN (today)
xpdf 3.00

Here is what I got:

Beamer 3.07-1
python 2.5.2
matplotlib 0.91.2-0ubuntu1
xpdf 3.02-1.3ubuntu1

Ironically, all my stuff is newer than yours except for matplotlib. Sounds
like bleeding edge of matplotlib must be tried next.

That python.sty stuff looks really cool, by the way. I haven't come
across it before.