[Matplotlib-users] encoding of files included in sphinx

Development matplotlib-devel
1

Hi,

just wanted to raise this problem on the devel list, where it probably belongs. Also, if nobody has time to look at it now and you prefer me to file a bug, please don’t hesitate to tell it.

the original post is there: http://thread.gmane.org/gmane.comp.python.matplotlib.general/20411

Cheers

···

Le 21 novembre 2009 17:50, Sébastien Barthélemy <barthelemy@…804…> a écrit :

Le 18 novembre 2009 17:24, Michael Droettboom <mdroe@…31…> a écrit :

This is a bug – but it has a fairly straightforward fix: to use Sphinx’s “include” directive rather than roll our own as we currently do. This has been fixed in SVN r7972. plot-directive now takes an “encoding” option, exactly like the Sphinx include directive. It does not do automatic encoding detection (meaning it ignores the “# coding: latin1” comments), just as the Sphinx include directive does.

Hello Michael,

thank you for your fast reply and action. I just tried with the version from trunk (r7978) and I still have an encoding problem on the same test case. It seems to happen when the file is ran (to produce the figure) rather than when it is included. I had a look at the code, but cannot understand what is happenning, I would have expected imp to proprely guess the encoding.

Could you tell me if you have the same problem ? Do you have any idea of what is going on ?

Thanks !

$ git clone git://github.com/sbarthelemy/SphinxEncoding.git

cd SphinxEncoding/ make html
sphinx-build -b html -d _build/doctrees . _build/html
Making output directory…
Running Sphinx v0.6.2
loading pickled environment… not found
building [html]: targets for 1 source files that are out of date

updating environment: 1 added, 0 changed, 0 removed
/home/barthelemy/.local/lib/python2.6/site-packages/matplotlib/sphinxext/plot_directive.py:273: UserWarning: Exception running plot ./fileutf8.py

Traceback (most recent call last):
File “/home/barthelemy/.local/lib/python2.6/site-packages/matplotlib/sphinxext/plot_directive.py”, line 270, in render_figures
run_code(plot_path, function_name, plot_code)

File “/home/barthelemy/.local/lib/python2.6/site-packages/matplotlib/sphinxext/plot_directive.py”, line 182, in run_code
“plot”, fd, fname, (‘py’, ‘r’, imp.PY_SOURCE))

File “fileutf8.py”, line 2, in 

print(u"accent aigus é")

UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xe9’ in position 13: ordinal not in range(128)

2

Sorry this thread fell through the cracks. Thanks for the reminder.

The error is not actually on importing and parsing the .py file (it
seems to do that just fine). The error is on printing to the console,
at which point it tries to convert the Unicode string to ascii (which
fails because it has character points > 127). One way around this
is to encode Unicode as UTF-8 (which seems to be the default for most
modern Linux X terminals etc.), eg.:

print(u"accent aigus é".encode("utf8"))

Mike

···

On 11/25/2009 02:11 PM, Sébastien Barthélemy wrote:

Hi,

just wanted to raise this problem on the devel list, where it probably
belongs. Also, if nobody has time to look at it now and you prefer me
to file a bug, please don’t hesitate to tell it.

the original post is there: http://thread.gmane.org/gmane.comp.python.matplotlib.general/20411

Cheers

Le 21 novembre 2009 17:50, Sébastien > Barthélemy <barthelemy@…804…> > a écrit :

Le
18 novembre 2009 17:24, Michael Droettboom <mdroe@…31…>
a écrit :

This
is a bug – but it has a fairly straightforward fix: to use Sphinx’s
“include” directive rather than roll our own as we currently do. This
has been fixed in SVN r7972. plot-directive now takes an “encoding”
option, exactly like the Sphinx include directive. It does not do
automatic encoding detection (meaning it ignores the “# coding: latin1”
comments), just as the Sphinx include directive does.

Hello Michael,

thank you for your fast reply and action. I just tried with the version
from trunk (r7978) and I still have an encoding problem on the same
test case. It seems to happen when the file is ran (to produce the
figure) rather than when it is included. I had a look at the code, but
cannot understand what is happenning, I would have expected imp to
proprely guess the encoding.

Could you tell me if you have the same problem ? Do you have any idea
of what is going on ?

Thanks !

$ git clone git://github.com/sbarthelemy/SphinxEncoding.git

$ cd SphinxEncoding/

$ make html

sphinx-build -b html -d _build/doctrees . _build/html

Making output directory…

Running Sphinx v0.6.2

loading pickled environment… not found

building [html]: targets for 1 source files that are out of date

updating environment: 1 added, 0 changed, 0 removed

/home/barthelemy/.local/lib/python2.6/site-packages/matplotlib/sphinxext/plot_directive.py:273:
UserWarning: Exception running plot ./fileutf8.py

Traceback (most recent call last):

File
“/home/barthelemy/.local/lib/python2.6/site-packages/matplotlib/sphinxext/plot_directive.py”,
line 270, in render_figures

run_code(plot_path, function_name, plot_code)

File
“/home/barthelemy/.local/lib/python2.6/site-packages/matplotlib/sphinxext/plot_directive.py”,
line 182, in run_code

"__plot__", fd, fname, ('py', 'r', imp.PY_SOURCE))

File “fileutf8.py”, line 2, in 

print(u"accent aigus é")

UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xe9’ in
position 13: ordinal not in range(128)