Arabic Character Support

Hi all,

I have been trying to follow your tips regarding matplotlib and arabic support.

Indeed matplotlib with Latex and unicode work great together :

#-*- coding: utf-8 -*-
from pylab import *
from matplotlib import rcParams
rcParams['text.usetex']=True
rcParams['text.latex.unicode']=True
figure(1)
plot([1,2,3])
title(unicode(r'\textit{éèê}','utf8'))
show()

But then I came to the conclusion that Latex did not support Arabic.
This few lines generate a "missing def" error.

\documentclass{article}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}
\title{éèê هه}
\begin{document}
\maketitle
\end{document}

Is it related to my installation ? (Ubuntu with package
texlive-latex-recommended)
I asked here : Latex Unicode : missing .def for arabic ?

Somebody pointed me out XeTeX, which has wider support for Unicode.
www.tug.org/TUGboat/Articles/tb27-2/tb87kew.pdf

I was wondering if there could be a way to get matplotlib work with XeTeX.

What do you think ? Is it the right track ? Should I switch to
grandpa's gnuplot :slight_smile: ?

Thank you all,

Mathieu.

Michael Droettboom wrote:

···

Unfortunately, I believe this is a fundamental incompatibility within
matplotlib. matplotlib uses a very simple algorithm for layout out a
line of characters which assumes ltr order and all kinds of other things.

That said, there is something you could try. Matplotlib has a "usetex"
mode which will do all text rendering using LaTeX. Assuming you can get
LaTeX to handle Arabic correctly, you can set the following in your
.matplotlibrc file:

       text.usetex : True
       text.latex.unicode : True

You'll need to install the LaTeX Unicode extension "ucs" (Ubuntu
probably has packages for that.) If you need to load any LaTeX packages
to get arabic support, you can tell matplotlib about them in your
matplotlibrc like so:

       text.latex.preamble: \usepackage{foo}

If Arabic-encoded-as-Unicode doesn't work, you could try setting
"text.latex.unicode" to False, and then typing your Arabic strings using
one of the other Arabic encodings that LaTeX understands.

Please let us know if you have success or hit another roadblock.

Cheers,
Mike

Burhan Khalid wrote:

Hello All:

  Having an issue with Arabic font support using matplotlib. When using
the correct font, Arabic characters are displayed, but are not joined
together
properly; and they are also not in the correct order (the font rendered is
ltr, but Arabic is a rtl language). Is this an issue with the render I
am using, or some incompatibility within matplotlib? Sample source code
used (please note, your email client should support utf8 to display the
code correctly).

#-*- coding: utf-8 -*-
from pylab import *
figure(1) # the first figure
plot([1,2,3])
figure(1) # figure 1 current
title(u'برهان',name='Times New Roman') # figure 1 title
savefig('test.png')
savefig('test.eps')
show()

Output when run from the command line:

matplotlib data path /usr/share/matplotlib/mpl-data
$HOME=/home/burhan
CONFIGDIR=/home/burhan/.matplotlib
loaded rc file /etc/matplotlibrc
matplotlib version 0.90.1
verbose.level helpful
interactive is False
units is True
platform is linux2
numerix numpy 1.0.3
font search path ['/usr/share/matplotlib/mpl-data/fonts/ttf',
'/usr/share/matplotlib/mpl-data/fonts/afm']
loaded ttfcache file /home/burhan/.matplotlib/ttffont.cache
backend TkAgg version 8.4
Could not match Bitstream Vera Serif, New Century Schoolbook, Century
Schoolbook L, Utopia, ITC Bookman, Bookman, Nimbus Roman No9 L, Times
New Roman, Times, Palatino, Charter, serif, normal, normal. Returning
/usr/share/fonts/truetype/ttf-bitstream-vera/Vera.ttf

This is on a clean Ubuntu Gutsy install, using python 2.5.1.

Thanks,
Burhan

------------------------------------------------------------------------

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

------------------------------------------------------------------------

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

Mathieu Leplatre wrote:

Hi all,

I have been trying to follow your tips regarding matplotlib and arabic support.
  Is it related to my installation ? (Ubuntu with package
texlive-latex-recommended)
I asked here : Latex Unicode : missing .def for arabic ?
  

I have no idea. Try asking on a texlive forum, perhaps.

Do you have a real need for using "usetex", or are you just doing that given my suggestion from last October?

Here's the thing to keep in mind: matplotlib's built-in text layout algorithm is completely English-centric. It assumes all text is left to right and doesn't deal with any of Unicode's complications like combining characters etc. We really should be using a proper text layout engine like Pango or ICU TextLayout that has all these things worked out. matplotlib in general avoids (L)GPL'd libraries for core functionality so Pango is probably out. And ICU TextLayout doesn't appear to have Python bindings, adding another speed bump to its use. I think it needs a highly-motivated developer to take it on. Unfortunately, I think many of us are in the head-in-the-sand about internationalization United States... :wink:

So the trick here for matplotlib (ignoring the LaTeX/XeTeX issues for now), is to avoid its built-in text layout. The Agg, Pdf and Ps backends are basically forced to use it by the nature of the format and implementations. The Cairo backend could do it if we used pangocairo, but at the moment we use Cairo's "toy" text API which doesn't have all the Pango goodness. It turns out that (basically by accident) the SVG backend *does* work, if you set "svg.embed_char_paths" to False. In that mode, it outputs "real" SVG text as strings that appear (to my non-Arabic-reading eyes) to be interpreted correctly r-to-l in Inkscape. Note, however, that matplotlib will get the length of the text wrong, so the text is slightly off-center. But it's a start.

Now, on to the LaTeX-based solution. I'm a bit out of my element here and only suggesting things that I might try next. No guarantees of working at all. I haven't spent much time with matplotlib's usetex code.

Somebody pointed me out XeTeX, which has wider support for Unicode.
www.tug.org/TUGboat/Articles/tb27-2/tb87kew.pdf

I was wondering if there could be a way to get matplotlib work with XeTeX.
  

I don't have XeTeX installed here (and don't have root access to my machine at work), so I can't easily try it. You could experiment, however, by replacing

            command = self._get_shell_cmd('cd "%s"'% self.texcache,
                            'latex -interaction=nonstopmode %s > "%s"'\
                            %(os.path.split(texfile)[-1], outfile))

with

            command = self._get_shell_cmd('cd "%s"'% self.texcache,
                            'xelatex --no-pdf -interaction=nonstopmode %s > "%s"'\
                            %(os.path.split(texfile)[-1], outfile))

Make sure text.latex.unicode is set to False, so that the ucs packages, which are incompatible with XeTeX, are not loaded.

A couple of things may stand in the way -- xetex, with the --no-pdf option, outputs "xdv" (extended dvi), rather than "dvi". matplotlib inserts LaTeX text by converting dvi to png or ps (with dvipng or dvips respectively). I don't know how backward-compatible xdv is as a format and whether that's a show stopper.

If you get somewhere and get stuck again, please post your progress and all console output etc. here and I can try to get you unstuck again.

What do you think ? Is it the right track ? Should I switch to
grandpa's gnuplot :slight_smile: ?
  

Well, gnuplot does use pango, apparently, so it has that advantage, if nothing else... :wink:

Cheers,
Mike

···

Thank you all,

Mathieu.

Michael Droettboom wrote:
  

Unfortunately, I believe this is a fundamental incompatibility within
matplotlib. matplotlib uses a very simple algorithm for layout out a
line of characters which assumes ltr order and all kinds of other things.

That said, there is something you could try. Matplotlib has a "usetex"
mode which will do all text rendering using LaTeX. Assuming you can get
LaTeX to handle Arabic correctly, you can set the following in your
.matplotlibrc file:

       text.usetex : True
       text.latex.unicode : True

You'll need to install the LaTeX Unicode extension "ucs" (Ubuntu
probably has packages for that.) If you need to load any LaTeX packages
to get arabic support, you can tell matplotlib about them in your
matplotlibrc like so:

       text.latex.preamble: \usepackage{foo}

If Arabic-encoded-as-Unicode doesn't work, you could try setting
"text.latex.unicode" to False, and then typing your Arabic strings using
one of the other Arabic encodings that LaTeX understands.

Please let us know if you have success or hit another roadblock.

Cheers,
Mike

Burhan Khalid wrote:
    

Hello All:

  Having an issue with Arabic font support using matplotlib. When using
the correct font, Arabic characters are displayed, but are not joined
together
properly; and they are also not in the correct order (the font rendered is
ltr, but Arabic is a rtl language). Is this an issue with the render I
am using, or some incompatibility within matplotlib? Sample source code
used (please note, your email client should support utf8 to display the
code correctly).

#-*- coding: utf-8 -*-
from pylab import *
figure(1) # the first figure
plot([1,2,3])
figure(1) # figure 1 current
title(u'�����',name='Times New Roman') # figure 1 title
savefig('test.png')
savefig('test.eps')
show()

Output when run from the command line:

matplotlib data path /usr/share/matplotlib/mpl-data
$HOME=/home/burhan
CONFIGDIR=/home/burhan/.matplotlib
loaded rc file /etc/matplotlibrc
matplotlib version 0.90.1
verbose.level helpful
interactive is False
units is True
platform is linux2
numerix numpy 1.0.3
font search path ['/usr/share/matplotlib/mpl-data/fonts/ttf',
'/usr/share/matplotlib/mpl-data/fonts/afm']
loaded ttfcache file /home/burhan/.matplotlib/ttffont.cache
backend TkAgg version 8.4
Could not match Bitstream Vera Serif, New Century Schoolbook, Century
Schoolbook L, Utopia, ITC Bookman, Bookman, Nimbus Roman No9 L, Times
New Roman, Times, Palatino, Charter, serif, normal, normal. Returning
/usr/share/fonts/truetype/ttf-bitstream-vera/Vera.ttf

This is on a clean Ubuntu Gutsy install, using python 2.5.1.

Thanks,
Burhan

------------------------------------------------------------------------

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

------------------------------------------------------------------------

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
      

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
  
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Hi Mathieu,

Are there perhaps some latex packages you could load using the
text.latex.preamble rc setting? This setting is officially unsupported, but
it would be much easier than adding an option to use the xetex executable
(which would probably not be officially supported by mpl).

Darren

···

On Wednesday 25 June 2008 09:02:41 am Mathieu Leplatre wrote:

Hi all,

I have been trying to follow your tips regarding matplotlib and arabic
support.

Indeed matplotlib with Latex and unicode work great together :

#-*- coding: utf-8 -*-
from pylab import *
from matplotlib import rcParams
rcParams['text.usetex']=True
rcParams['text.latex.unicode']=True
figure(1)
plot([1,2,3])
title(unicode(r'\textit{éèê}','utf8'))
show()

But then I came to the conclusion that Latex did not support Arabic.
This few lines generate a "missing def" error.

\documentclass{article}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}
\title{éèê هه}
\begin{document}
\maketitle
\end{document}

Is it related to my installation ? (Ubuntu with package
texlive-latex-recommended)
I asked here : Latex Unicode : missing .def for arabic ?

Somebody pointed me out XeTeX, which has wider support for Unicode.
www.tug.org/TUGboat/Articles/tb27-2/tb87kew.pdf

I was wondering if there could be a way to get matplotlib work with XeTeX.

What do you think ? Is it the right track ? Should I switch to
grandpa's gnuplot :slight_smile: ?

Thank you all,

Mathieu.

Michael Droettboom wrote:
> Unfortunately, I believe this is a fundamental incompatibility within
> matplotlib. matplotlib uses a very simple algorithm for layout out a
> line of characters which assumes ltr order and all kinds of other things.
>
> That said, there is something you could try. Matplotlib has a "usetex"
> mode which will do all text rendering using LaTeX. Assuming you can get
> LaTeX to handle Arabic correctly, you can set the following in your
> .matplotlibrc file:
>
> text.usetex : True
> text.latex.unicode : True
>
> You'll need to install the LaTeX Unicode extension "ucs" (Ubuntu
> probably has packages for that.) If you need to load any LaTeX packages
> to get arabic support, you can tell matplotlib about them in your
> matplotlibrc like so:
>
> text.latex.preamble: \usepackage{foo}
>
> If Arabic-encoded-as-Unicode doesn't work, you could try setting
> "text.latex.unicode" to False, and then typing your Arabic strings using
> one of the other Arabic encodings that LaTeX understands.
>
> Please let us know if you have success or hit another roadblock.
>
> Cheers,
> Mike
>
> Burhan Khalid wrote:
>> Hello All:
>>
>> Having an issue with Arabic font support using matplotlib. When using
>> the correct font, Arabic characters are displayed, but are not joined
>> together
>> properly; and they are also not in the correct order (the font rendered
>> is ltr, but Arabic is a rtl language). Is this an issue with the render
>> I am using, or some incompatibility within matplotlib? Sample source
>> code used (please note, your email client should support utf8 to display
>> the code correctly).
>>
>> #-*- coding: utf-8 -*-
>> from pylab import *
>> figure(1) # the first figure
>> plot([1,2,3])
>> figure(1) # figure 1 current
>> title(u'برهان',name='Times New Roman') # figure 1 title
>> savefig('test.png')
>> savefig('test.eps')
>> show()
>>
>> Output when run from the command line:
>>
>> matplotlib data path /usr/share/matplotlib/mpl-data
>> $HOME=/home/burhan
>> CONFIGDIR=/home/burhan/.matplotlib
>> loaded rc file /etc/matplotlibrc
>> matplotlib version 0.90.1
>> verbose.level helpful
>> interactive is False
>> units is True
>> platform is linux2
>> numerix numpy 1.0.3
>> font search path ['/usr/share/matplotlib/mpl-data/fonts/ttf',
>> '/usr/share/matplotlib/mpl-data/fonts/afm']
>> loaded ttfcache file /home/burhan/.matplotlib/ttffont.cache
>> backend TkAgg version 8.4
>> Could not match Bitstream Vera Serif, New Century Schoolbook, Century
>> Schoolbook L, Utopia, ITC Bookman, Bookman, Nimbus Roman No9 L, Times
>> New Roman, Times, Palatino, Charter, serif, normal, normal. Returning
>> /usr/share/fonts/truetype/ttf-bitstream-vera/Vera.ttf
>>
>> This is on a clean Ubuntu Gutsy install, using python 2.5.1.
>>
>> Thanks,
>> Burhan
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> ------------------------------------------------------------------------
>>- This SF.net email is sponsored by: Splunk Inc.
>> Still grepping through log files to find problems? Stop.
>> Now Search log events and configuration files using AJAX and a browser.
>> Download your FREE copy of Splunk now >> http://get.splunk.com/
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Matplotlib-users mailing list
>> Matplotlib-users@lists.sourceforge.net
>> matplotlib-users List Signup and Options
>
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems? Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Matplotlib-users mailing list
> Matplotlib-users@lists.sourceforge.net
> matplotlib-users List Signup and Options

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

Well, I'll give a try towards another possibility :
1 - use SVG output with rc setting : svg.embed_char_paths" to False
2 - use cairo to export to PNG/JPG etc..

Step 1 works great natively with Arabic (at least viewing it with Inkscape).
Let's see if I have problems with step 2.

2008/6/25 Darren Dale <dsdale24@...287...>:

···

Hi Mathieu,

Are there perhaps some latex packages you could load using the
text.latex.preamble rc setting? This setting is officially unsupported, but
it would be much easier than adding an option to use the xetex executable
(which would probably not be officially supported by mpl).

Darren

On Wednesday 25 June 2008 09:02:41 am Mathieu Leplatre wrote:

Hi all,

I have been trying to follow your tips regarding matplotlib and arabic
support.

Indeed matplotlib with Latex and unicode work great together :

#-*- coding: utf-8 -*-
from pylab import *
from matplotlib import rcParams
rcParams['text.usetex']=True
rcParams['text.latex.unicode']=True
figure(1)
plot([1,2,3])
title(unicode(r'\textit{éèê}','utf8'))
show()

But then I came to the conclusion that Latex did not support Arabic.
This few lines generate a "missing def" error.

\documentclass{article}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}
\title{éèê هه}
\begin{document}
\maketitle
\end{document}

Is it related to my installation ? (Ubuntu with package
texlive-latex-recommended)
I asked here : Latex Unicode : missing .def for arabic ?

Somebody pointed me out XeTeX, which has wider support for Unicode.
www.tug.org/TUGboat/Articles/tb27-2/tb87kew.pdf

I was wondering if there could be a way to get matplotlib work with XeTeX.

What do you think ? Is it the right track ? Should I switch to
grandpa's gnuplot :slight_smile: ?

Thank you all,

Mathieu.

Michael Droettboom wrote:
> Unfortunately, I believe this is a fundamental incompatibility within
> matplotlib. matplotlib uses a very simple algorithm for layout out a
> line of characters which assumes ltr order and all kinds of other things.
>
> That said, there is something you could try. Matplotlib has a "usetex"
> mode which will do all text rendering using LaTeX. Assuming you can get
> LaTeX to handle Arabic correctly, you can set the following in your
> .matplotlibrc file:
>
> text.usetex : True
> text.latex.unicode : True
>
> You'll need to install the LaTeX Unicode extension "ucs" (Ubuntu
> probably has packages for that.) If you need to load any LaTeX packages
> to get arabic support, you can tell matplotlib about them in your
> matplotlibrc like so:
>
> text.latex.preamble: \usepackage{foo}
>
> If Arabic-encoded-as-Unicode doesn't work, you could try setting
> "text.latex.unicode" to False, and then typing your Arabic strings using
> one of the other Arabic encodings that LaTeX understands.
>
> Please let us know if you have success or hit another roadblock.
>
> Cheers,
> Mike
>
> Burhan Khalid wrote:
>> Hello All:
>>
>> Having an issue with Arabic font support using matplotlib. When using
>> the correct font, Arabic characters are displayed, but are not joined
>> together
>> properly; and they are also not in the correct order (the font rendered
>> is ltr, but Arabic is a rtl language). Is this an issue with the render
>> I am using, or some incompatibility within matplotlib? Sample source
>> code used (please note, your email client should support utf8 to display
>> the code correctly).
>>
>> #-*- coding: utf-8 -*-
>> from pylab import *
>> figure(1) # the first figure
>> plot([1,2,3])
>> figure(1) # figure 1 current
>> title(u'برهان',name='Times New Roman') # figure 1 title
>> savefig('test.png')
>> savefig('test.eps')
>> show()
>>
>> Output when run from the command line:
>>
>> matplotlib data path /usr/share/matplotlib/mpl-data
>> $HOME=/home/burhan
>> CONFIGDIR=/home/burhan/.matplotlib
>> loaded rc file /etc/matplotlibrc
>> matplotlib version 0.90.1
>> verbose.level helpful
>> interactive is False
>> units is True
>> platform is linux2
>> numerix numpy 1.0.3
>> font search path ['/usr/share/matplotlib/mpl-data/fonts/ttf',
>> '/usr/share/matplotlib/mpl-data/fonts/afm']
>> loaded ttfcache file /home/burhan/.matplotlib/ttffont.cache
>> backend TkAgg version 8.4
>> Could not match Bitstream Vera Serif, New Century Schoolbook, Century
>> Schoolbook L, Utopia, ITC Bookman, Bookman, Nimbus Roman No9 L, Times
>> New Roman, Times, Palatino, Charter, serif, normal, normal. Returning
>> /usr/share/fonts/truetype/ttf-bitstream-vera/Vera.ttf
>>
>> This is on a clean Ubuntu Gutsy install, using python 2.5.1.
>>
>> Thanks,
>> Burhan
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> ------------------------------------------------------------------------
>>- This SF.net email is sponsored by: Splunk Inc.
>> Still grepping through log files to find problems? Stop.
>> Now Search log events and configuration files using AJAX and a browser.
>> Download your FREE copy of Splunk now >> http://get.splunk.com/
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Matplotlib-users mailing list
>> Matplotlib-users@lists.sourceforge.net
>> matplotlib-users List Signup and Options
>
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems? Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Matplotlib-users mailing list
> Matplotlib-users@lists.sourceforge.net
> matplotlib-users List Signup and Options

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

Thank you Michael and Darren.

I'll keep your information for further polishing :

So far, I am not having problems with my two-steps solution :

1) SVG backend
2) rsvg + Cairo, code snippet :

svg = rsvg.Handle( filesvg )
width = svg.props.width
height = svg.props.height
surface = cairo.ImageSurface (cairo.FORMAT_ARGB32, width, height)
cr = cairo.Context (surface)
svg.render_cairo (cr)
surface.write_to_png (filepng)

Thanks again for your assistance ! Long live sub-cultures and their
exotic writings !

2008/6/25 Mathieu Leplatre <leplatre@...287...>:

···

Well, I'll give a try towards another possibility :
1 - use SVG output with rc setting : svg.embed_char_paths" to False
2 - use cairo to export to PNG/JPG etc..

Step 1 works great natively with Arabic (at least viewing it with Inkscape).
Let's see if I have problems with step 2.

2008/6/25 Darren Dale <dsdale24@...287...>:

Hi Mathieu,

Are there perhaps some latex packages you could load using the
text.latex.preamble rc setting? This setting is officially unsupported, but
it would be much easier than adding an option to use the xetex executable
(which would probably not be officially supported by mpl).

Darren

On Wednesday 25 June 2008 09:02:41 am Mathieu Leplatre wrote:

Hi all,

I have been trying to follow your tips regarding matplotlib and arabic
support.

Indeed matplotlib with Latex and unicode work great together :

#-*- coding: utf-8 -*-
from pylab import *
from matplotlib import rcParams
rcParams['text.usetex']=True
rcParams['text.latex.unicode']=True
figure(1)
plot([1,2,3])
title(unicode(r'\textit{éèê}','utf8'))
show()

But then I came to the conclusion that Latex did not support Arabic.
This few lines generate a "missing def" error.

\documentclass{article}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}
\title{éèê هه}
\begin{document}
\maketitle
\end{document}

Is it related to my installation ? (Ubuntu with package
texlive-latex-recommended)
I asked here : Latex Unicode : missing .def for arabic ?

Somebody pointed me out XeTeX, which has wider support for Unicode.
www.tug.org/TUGboat/Articles/tb27-2/tb87kew.pdf

I was wondering if there could be a way to get matplotlib work with XeTeX.

What do you think ? Is it the right track ? Should I switch to
grandpa's gnuplot :slight_smile: ?

Thank you all,

Mathieu.

Michael Droettboom wrote:
> Unfortunately, I believe this is a fundamental incompatibility within
> matplotlib. matplotlib uses a very simple algorithm for layout out a
> line of characters which assumes ltr order and all kinds of other things.
>
> That said, there is something you could try. Matplotlib has a "usetex"
> mode which will do all text rendering using LaTeX. Assuming you can get
> LaTeX to handle Arabic correctly, you can set the following in your
> .matplotlibrc file:
>
> text.usetex : True
> text.latex.unicode : True
>
> You'll need to install the LaTeX Unicode extension "ucs" (Ubuntu
> probably has packages for that.) If you need to load any LaTeX packages
> to get arabic support, you can tell matplotlib about them in your
> matplotlibrc like so:
>
> text.latex.preamble: \usepackage{foo}
>
> If Arabic-encoded-as-Unicode doesn't work, you could try setting
> "text.latex.unicode" to False, and then typing your Arabic strings using
> one of the other Arabic encodings that LaTeX understands.
>
> Please let us know if you have success or hit another roadblock.
>
> Cheers,
> Mike
>
> Burhan Khalid wrote:
>> Hello All:
>>
>> Having an issue with Arabic font support using matplotlib. When using
>> the correct font, Arabic characters are displayed, but are not joined
>> together
>> properly; and they are also not in the correct order (the font rendered
>> is ltr, but Arabic is a rtl language). Is this an issue with the render
>> I am using, or some incompatibility within matplotlib? Sample source
>> code used (please note, your email client should support utf8 to display
>> the code correctly).
>>
>> #-*- coding: utf-8 -*-
>> from pylab import *
>> figure(1) # the first figure
>> plot([1,2,3])
>> figure(1) # figure 1 current
>> title(u'برهان',name='Times New Roman') # figure 1 title
>> savefig('test.png')
>> savefig('test.eps')
>> show()
>>
>> Output when run from the command line:
>>
>> matplotlib data path /usr/share/matplotlib/mpl-data
>> $HOME=/home/burhan
>> CONFIGDIR=/home/burhan/.matplotlib
>> loaded rc file /etc/matplotlibrc
>> matplotlib version 0.90.1
>> verbose.level helpful
>> interactive is False
>> units is True
>> platform is linux2
>> numerix numpy 1.0.3
>> font search path ['/usr/share/matplotlib/mpl-data/fonts/ttf',
>> '/usr/share/matplotlib/mpl-data/fonts/afm']
>> loaded ttfcache file /home/burhan/.matplotlib/ttffont.cache
>> backend TkAgg version 8.4
>> Could not match Bitstream Vera Serif, New Century Schoolbook, Century
>> Schoolbook L, Utopia, ITC Bookman, Bookman, Nimbus Roman No9 L, Times
>> New Roman, Times, Palatino, Charter, serif, normal, normal. Returning
>> /usr/share/fonts/truetype/ttf-bitstream-vera/Vera.ttf
>>
>> This is on a clean Ubuntu Gutsy install, using python 2.5.1.
>>
>> Thanks,
>> Burhan
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> ------------------------------------------------------------------------
>>- This SF.net email is sponsored by: Splunk Inc.
>> Still grepping through log files to find problems? Stop.
>> Now Search log events and configuration files using AJAX and a browser.
>> Download your FREE copy of Splunk now >> http://get.splunk.com/
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Matplotlib-users mailing list
>> Matplotlib-users@lists.sourceforge.net
>> matplotlib-users List Signup and Options
>
> --
> Michael Droettboom
> Science Software Branch
> Operations and Engineering Division
> Space Telescope Science Institute
> Operated by AURA for NASA
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems? Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Matplotlib-users mailing list
> Matplotlib-users@lists.sourceforge.net
> matplotlib-users List Signup and Options

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

Thanks for the report. It's a bit convoluted, but it is automatable, at least, and I'm glad to see it works for you.

I'd like to see a Cairo expert take on the task of upgrading our Cairo backend to use Pango. That would make this a little cleaner and theoretically fix the text alignment problem with your method (i.e. the titles are not exactly centered since its width is based on a naive rendering of the text). I have no concept of how much work that would be. Though it's also possible that that would just run up against other assumptions that matplotlib makes about text.

Cheers,
Mike

Mathieu Leplatre wrote:

···

Thank you Michael and Darren.

I'll keep your information for further polishing :

So far, I am not having problems with my two-steps solution :

1) SVG backend
2) rsvg + Cairo, code snippet :

svg = rsvg.Handle( filesvg )
width = svg.props.width
height = svg.props.height
surface = cairo.ImageSurface (cairo.FORMAT_ARGB32, width, height)
cr = cairo.Context (surface)
svg.render_cairo (cr)
surface.write_to_png (filepng)

Thanks again for your assistance ! Long live sub-cultures and their
exotic writings !

2008/6/25 Mathieu Leplatre <leplatre@...287...>:
  

Well, I'll give a try towards another possibility :
1 - use SVG output with rc setting : svg.embed_char_paths" to False
2 - use cairo to export to PNG/JPG etc..

Step 1 works great natively with Arabic (at least viewing it with Inkscape).
Let's see if I have problems with step 2.

2008/6/25 Darren Dale <dsdale24@...287...>:
    

Hi Mathieu,

Are there perhaps some latex packages you could load using the
text.latex.preamble rc setting? This setting is officially unsupported, but
it would be much easier than adding an option to use the xetex executable
(which would probably not be officially supported by mpl).

Darren

On Wednesday 25 June 2008 09:02:41 am Mathieu Leplatre wrote:
      

Hi all,

I have been trying to follow your tips regarding matplotlib and arabic
support.

Indeed matplotlib with Latex and unicode work great together :

#-*- coding: utf-8 -*-
from pylab import *
from matplotlib import rcParams
rcParams['text.usetex']=True
rcParams['text.latex.unicode']=True
figure(1)
plot([1,2,3])
title(unicode(r'\textit{���}','utf8'))
show()

But then I came to the conclusion that Latex did not support Arabic.
This few lines generate a "missing def" error.

\documentclass{article}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}
\title{��� ��}
\begin{document}
\maketitle
\end{document}

Is it related to my installation ? (Ubuntu with package
texlive-latex-recommended)
I asked here : Latex Unicode : missing .def for arabic ?

Somebody pointed me out XeTeX, which has wider support for Unicode.
www.tug.org/TUGboat/Articles/tb27-2/tb87kew.pdf

I was wondering if there could be a way to get matplotlib work with XeTeX.

What do you think ? Is it the right track ? Should I switch to
grandpa's gnuplot :slight_smile: ?

Thank you all,

Mathieu.

Michael Droettboom wrote:
        

Unfortunately, I believe this is a fundamental incompatibility within
matplotlib. matplotlib uses a very simple algorithm for layout out a
line of characters which assumes ltr order and all kinds of other things.

That said, there is something you could try. Matplotlib has a "usetex"
mode which will do all text rendering using LaTeX. Assuming you can get
LaTeX to handle Arabic correctly, you can set the following in your
.matplotlibrc file:

       text.usetex : True
       text.latex.unicode : True

You'll need to install the LaTeX Unicode extension "ucs" (Ubuntu
probably has packages for that.) If you need to load any LaTeX packages
to get arabic support, you can tell matplotlib about them in your
matplotlibrc like so:

       text.latex.preamble: \usepackage{foo}

If Arabic-encoded-as-Unicode doesn't work, you could try setting
"text.latex.unicode" to False, and then typing your Arabic strings using
one of the other Arabic encodings that LaTeX understands.

Please let us know if you have success or hit another roadblock.

Cheers,
Mike

Burhan Khalid wrote:
          

Hello All:

  Having an issue with Arabic font support using matplotlib. When using
the correct font, Arabic characters are displayed, but are not joined
together
properly; and they are also not in the correct order (the font rendered
is ltr, but Arabic is a rtl language). Is this an issue with the render
I am using, or some incompatibility within matplotlib? Sample source
code used (please note, your email client should support utf8 to display
the code correctly).

#-*- coding: utf-8 -*-
from pylab import *
figure(1) # the first figure
plot([1,2,3])
figure(1) # figure 1 current
title(u'�����',name='Times New Roman') # figure 1 title
savefig('test.png')
savefig('test.eps')
show()

Output when run from the command line:

matplotlib data path /usr/share/matplotlib/mpl-data
$HOME=/home/burhan
CONFIGDIR=/home/burhan/.matplotlib
loaded rc file /etc/matplotlibrc
matplotlib version 0.90.1
verbose.level helpful
interactive is False
units is True
platform is linux2
numerix numpy 1.0.3
font search path ['/usr/share/matplotlib/mpl-data/fonts/ttf',
'/usr/share/matplotlib/mpl-data/fonts/afm']
loaded ttfcache file /home/burhan/.matplotlib/ttffont.cache
backend TkAgg version 8.4
Could not match Bitstream Vera Serif, New Century Schoolbook, Century
Schoolbook L, Utopia, ITC Bookman, Bookman, Nimbus Roman No9 L, Times
New Roman, Times, Palatino, Charter, serif, normal, normal. Returning
/usr/share/fonts/truetype/ttf-bitstream-vera/Vera.ttf

This is on a clean Ubuntu Gutsy install, using python 2.5.1.

Thanks,
Burhan

------------------------------------------------------------------------

------------------------------------------------------------------------
- This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

------------------------------------------------------------------------

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
            

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
          

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
        

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options
  
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA