how to avoid import backend in the batch job?

Chao_YUE1 · September 3, 2012, 12:53pm

Dear all,

I have a module written by myself which use python-netcdf/matplotlib/numpy/basemap for treating and mapping of netcdf files.
so in the module I have a line:

import matplotlib.pyplot as plt

But now I am using some function of this module which do not involve the visualisation. And I want to use it in a batch mode submitted by qsub command.
And then I have traceback like following:

File “/home/orchidee01/ychao/SCRIPT/combine.py”, line 1, in
import gnc
File “/home/users/ychao/python/python_lib/gnc.py”, line 4, in
import matplotlib.pyplot as plt
File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/pyplot.py”, line 95, in
new_figure_manager, draw_if_interactive, _show = pylab_setup()
File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/init.py”, line 25, in pylab_setup
globals(),locals(),[backend_name])
File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/backend_gtkagg.py”, line 10, in
from matplotlib.backends.backend_gtk import gtk, FigureManagerGTK, FigureCanvasGTK,
File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/backend_gtk.py”, line 8, in
import gtk; gdk = gtk.gdk
File “/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/init.py”, line 64, in
_init()
File “/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/init.py”, line 52, in _init
_gtk.init_check()
RuntimeError: could not open display

From the last line, I think this is because I cannot use this module in a non-interactive mode. So is there a way I can bypass this issue and still
use the module in a non-interactive mode? Thanks a lot for any information.

Chao

···

–

Chao YUE
Laboratoire des Sciences du Climat et de l’Environnement (LSCE-IPSL)
UMR 1572 CEA-CNRS-UVSQ
Batiment 712 - Pe 119
91191 GIF Sur YVETTE Cedex

Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16

Benjamin_Root · September 3, 2012, 1:30pm

import matplotlib

matplotlib.use(“AGG”)

Or set that in the matplotlibrc file.

Cheers,
Ben Root

···

On Monday, September 3, 2012, Chao YUE wrote:

Dear all,

I have a module written by myself which use python-netcdf/matplotlib/numpy/basemap for treating and mapping of netcdf files.

so in the module I have a line:

import matplotlib.pyplot as plt

But now I am using some function of this module which do not involve the visualisation. And I want to use it in a batch mode submitted by qsub command.
And then I have traceback like following:

File “/home/orchidee01/ychao/SCRIPT/combine.py”, line 1, in
import gnc
File “/home/users/ychao/python/python_lib/gnc.py”, line 4, in
import matplotlib.pyplot as plt
File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/pyplot.py”, line 95, in
new_figure_manager, draw_if_interactive, _show = pylab_setup()
File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/init.py”, line 25, in pylab_setup
globals(),locals(),[backend_name])

File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/backend_gtkagg.py”, line 10, in
from matplotlib.backends.backend_gtk import gtk, FigureManagerGTK, FigureCanvasGTK,\

File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/backend_gtk.py”, line 8, in
import gtk; gdk = gtk.gdk
File “/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/init.py”, line 64, in
_init()
File “/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/init.py”, line 52, in _init
_gtk.init_check()
RuntimeError: could not open display

From the last line, I think this is because I cannot use this module in a non-interactive mode. So is there a way I can bypass this issue and still

use the module in a non-interactive mode? Thanks a lot for any information.

Chao

Chao_YUE1 · September 3, 2012, 5:08pm

Hi Ben,

I changed the backend in matplotlibrc file. It works fine. Yet if I don’t want to change the rc file, I guess I must change in the modules where matplotlib has been imported for the first time?

cheers,

Chao

···

On Mon, Sep 3, 2012 at 3:30 PM, Benjamin Root <ben.root@…1304…> wrote:

On Monday, September 3, 2012, Chao YUE wrote:
Dear all,

I have a module written by myself which use python-netcdf/matplotlib/numpy/basemap for treating and mapping of netcdf files.

so in the module I have a line:

import matplotlib.pyplot as plt

But now I am using some function of this module which do not involve the visualisation. And I want to use it in a batch mode submitted by qsub command.
And then I have traceback like following:

File “/home/orchidee01/ychao/SCRIPT/combine.py”, line 1, in
import gnc
File “/home/users/ychao/python/python_lib/gnc.py”, line 4, in
import matplotlib.pyplot as plt
File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/pyplot.py”, line 95, in
new_figure_manager, draw_if_interactive, _show = pylab_setup()
File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/init.py”, line 25, in pylab_setup
globals(),locals(),[backend_name])

File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/backend_gtkagg.py”, line 10, in
from matplotlib.backends.backend_gtk import gtk, FigureManagerGTK, FigureCanvasGTK,\

File “/usr/local/install/matplotlib-1.1.0/lib64/python2.6/site-packages/matplotlib/backends/backend_gtk.py”, line 8, in
import gtk; gdk = gtk.gdk
File “/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/init.py”, line 64, in
_init()
File “/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/init.py”, line 52, in _init
_gtk.init_check()
RuntimeError: could not open display

From the last line, I think this is because I cannot use this module in a non-interactive mode. So is there a way I can bypass this issue and still

use the module in a non-interactive mode? Thanks a lot for any information.

Chao

import matplotlib

matplotlib.use(“AGG”)

Or set that in the matplotlibrc file.

Cheers,
Ben Root

–

Chao YUE
Laboratoire des Sciences du Climat et de l’Environnement (LSCE-IPSL)
UMR 1572 CEA-CNRS-UVSQ
Batiment 712 - Pe 119
91191 GIF Sur YVETTE Cedex

Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16

Damon_McDougall3 · September 3, 2012, 6:13pm

If you're changing the rc file in your home directory:

~/.matplotlib/matplotlibrc

then just make a copy before you change it:

cd ~/.matplotlib
cp matplotlibrc matplotlibrc_orig

then edit matplotlibrc. Just rename matplotlibrc_orig back to
matplotlibrc when you're done.

···

On Mon, Sep 3, 2012 at 6:08 PM, Chao YUE <chaoyuejoy@...287...> wrote:

Yet if I don't want to change the rc file, I guess I must change in the
modules where matplotlib has been imported for the first time?

--
Damon McDougall
http://www.damon.is-a-geek.com
B2.39
Mathematics Institute
University of Warwick
Coventry
West Midlands
CV4 7AL
United Kingdom

Chao_YUE1 · September 3, 2012, 9:20pm

Thanks Damon, it’s very clear now.
I explicit set the backend as “GTKAgg” in my ipython user configuration file. and always set the backend to ‘Agg’ in my rc file. So that when I use ipython, I can see the window interactively, but when I use script, it also works

Chao

···

On Mon, Sep 3, 2012 at 8:13 PM, Damon McDougall <damon.mcdougall@…287…> wrote:

On Mon, Sep 3, 2012 at 6:08 PM, Chao YUE <chaoyuejoy@…287…> wrote:

Yet if I don’t want to change the rc file, I guess I must change in the

modules where matplotlib has been imported for the first time?

If you’re changing the rc file in your home directory:

~/.matplotlib/matplotlibrc

then just make a copy before you change it:

cd ~/.matplotlib

cp matplotlibrc matplotlibrc_orig

then edit matplotlibrc. Just rename matplotlibrc_orig back to

matplotlibrc when you’re done.

–

Damon McDougall

http://www.damon.is-a-geek.com

B2.39

Mathematics Institute

University of Warwick

Coventry

West Midlands

CV4 7AL

United Kingdom

–

Chao YUE
Laboratoire des Sciences du Climat et de l’Environnement (LSCE-IPSL)
UMR 1572 CEA-CNRS-UVSQ
Batiment 712 - Pe 119
91191 GIF Sur YVETTE Cedex

Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16

Damon_McDougall3 · September 3, 2012, 10:15pm

Thanks Damon, it's very clear now.
I explicit set the backend as "GTKAgg" in my ipython user configuration
file. and always set the backend to 'Agg' in my rc file. So that when I use
ipython, I can see the window interactively, but when I use script, it also
works

That's an awesome idea.

Glad it's working now.

···

On Mon, Sep 3, 2012 at 10:20 PM, Chao YUE <chaoyuejoy@...287...> wrote:

Chao

On Mon, Sep 3, 2012 at 8:13 PM, Damon McDougall <damon.mcdougall@...287...> > wrote:

On Mon, Sep 3, 2012 at 6:08 PM, Chao YUE <chaoyuejoy@...287...> wrote:
> Yet if I don't want to change the rc file, I guess I must change in the
> modules where matplotlib has been imported for the first time?

If you're changing the rc file in your home directory:

~/.matplotlib/matplotlibrc

then just make a copy before you change it:

cd ~/.matplotlib
cp matplotlibrc matplotlibrc_orig

then edit matplotlibrc. Just rename matplotlibrc_orig back to
matplotlibrc when you're done.

--
Damon McDougall
http://www.damon.is-a-geek.com
B2.39
Mathematics Institute
University of Warwick
Coventry
West Midlands
CV4 7AL
United Kingdom

--
***********************************************************************************
Chao YUE
Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL)
UMR 1572 CEA-CNRS-UVSQ
Batiment 712 - Pe 119
91191 GIF Sur YVETTE Cedex
Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16
************************************************************************************

--
Damon McDougall
http://www.damon.is-a-geek.com
B2.39
Mathematics Institute
University of Warwick
Coventry
West Midlands
CV4 7AL
United Kingdom