puzzle: interactive backend case with ipython

While investigating a bug pointed out by Andrew (and one that I had introduced some time ago) I found to my horror that I had also caused another bug: interactive plotting was not working with ipython -pylab. I backed out part of r5420, and that fixed the problem. This temporary fix is to restore the mixed case backend names in the interactive_bk list that is now set in rcsetup.py and imported by backends/__init__.py.

I am baffled as to why this is needed at present; I have not been able to figure out why ipython threading seems to get confused when that list has lower case names for gtkagg, qtagg, and wxagg, but not for tkagg or qt4agg. I don't see anywhere in either the ipython or the mpl code that should care about the case of the names in that list, now that I have ensured that comparisons against backend names in mpl (everywhere I could find them--maybe I missed on) are case-insensitive, and get_backend() returns only lower case. rcParams['backend'] can still have mixed case, however.

Any suggestions as to what I am missing or where to look?

Related question: I just noticed that there is also a config/rcsetup.py. Is there any reason why there has to be so much duplication (and now gradual divergence) between this and the regular rcsetup.py? Is there any reason the traits config system can't use the regular version, maybe with some changes merged in?

Eric

We should probably discuss the future of TConfig in mpl at some point. Its been
a while since I looked at it, but basically I was trying to isolate that code
so it didnt creep into mpl until we were ready. There is lots of duplication
in config, if you want to change it to import from rcsetup, its alright with
me.

···

On Monday 09 June 2008 03:55:47 Eric Firing wrote:

While investigating a bug pointed out by Andrew (and one that I had
introduced some time ago) I found to my horror that I had also caused
another bug: interactive plotting was not working with ipython -pylab.
I backed out part of r5420, and that fixed the problem. This temporary
fix is to restore the mixed case backend names in the interactive_bk
list that is now set in rcsetup.py and imported by backends/__init__.py.

I am baffled as to why this is needed at present; I have not been able
to figure out why ipython threading seems to get confused when that list
has lower case names for gtkagg, qtagg, and wxagg, but not for tkagg or
qt4agg. I don't see anywhere in either the ipython or the mpl code that
should care about the case of the names in that list, now that I have
ensured that comparisons against backend names in mpl (everywhere I
could find them--maybe I missed on) are case-insensitive, and
get_backend() returns only lower case. rcParams['backend'] can still
have mixed case, however.

Any suggestions as to what I am missing or where to look?

Related question: I just noticed that there is also a config/rcsetup.py.
  Is there any reason why there has to be so much duplication (and now
gradual divergence) between this and the regular rcsetup.py? Is there
any reason the traits config system can't use the regular version, maybe
with some changes merged in?

Hi Eric,

show() isn't bringing up my plots with r5430 in normal scripts (i.e. not
using IPython).

-Andrew

Eric Firing wrote:

···

While investigating a bug pointed out by Andrew (and one that I had
introduced some time ago) I found to my horror that I had also caused
another bug: interactive plotting was not working with ipython -pylab.
I backed out part of r5420, and that fixed the problem. This temporary
fix is to restore the mixed case backend names in the interactive_bk
list that is now set in rcsetup.py and imported by backends/__init__.py.

I am baffled as to why this is needed at present; I have not been able
to figure out why ipython threading seems to get confused when that list
has lower case names for gtkagg, qtagg, and wxagg, but not for tkagg or
qt4agg. I don't see anywhere in either the ipython or the mpl code that
should care about the case of the names in that list, now that I have
ensured that comparisons against backend names in mpl (everywhere I
could find them--maybe I missed on) are case-insensitive, and
get_backend() returns only lower case. rcParams['backend'] can still
have mixed case, however.

Any suggestions as to what I am missing or where to look?

Related question: I just noticed that there is also a config/rcsetup.py.
  Is there any reason why there has to be so much duplication (and now
gradual divergence) between this and the regular rcsetup.py? Is there
any reason the traits config system can't use the regular version, maybe
with some changes merged in?

Eric

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Andrew Straw wrote:

Hi Eric,

show() isn't bringing up my plots with r5430 in normal scripts (i.e. not
using IPython).

Sure enough. It seems that in the process of bisecting the svn versions to track down this problem and the original one you reported, things got mixed up in my directory, so that what worked on what I thought was the svn head was actually working on some mongrel. After a fresh checkout with clean build and install, I find that neither plain show nor ipython -pylab is working.

Back to the drawing board. I really would like to get everything working with complete case-insensitivity, and lower-case defaults, for the backend names. There is no reason one should have to remember that it is TkAgg, not TKAgg, but GTKAgg, not GtkAgg.

Eric

···

-Andrew

Eric Firing wrote:

While investigating a bug pointed out by Andrew (and one that I had introduced some time ago) I found to my horror that I had also caused another bug: interactive plotting was not working with ipython -pylab. I backed out part of r5420, and that fixed the problem. This temporary fix is to restore the mixed case backend names in the interactive_bk list that is now set in rcsetup.py and imported by backends/__init__.py.

I am baffled as to why this is needed at present; I have not been able to figure out why ipython threading seems to get confused when that list has lower case names for gtkagg, qtagg, and wxagg, but not for tkagg or qt4agg. I don't see anywhere in either the ipython or the mpl code that should care about the case of the names in that list, now that I have ensured that comparisons against backend names in mpl (everywhere I could find them--maybe I missed on) are case-insensitive, and get_backend() returns only lower case. rcParams['backend'] can still have mixed case, however.

Any suggestions as to what I am missing or where to look?

Related question: I just noticed that there is also a config/rcsetup.py. Is there any reason why there has to be so much duplication (and now gradual divergence) between this and the regular rcsetup.py? Is there any reason the traits config system can't use the regular version, maybe with some changes merged in?

Eric

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
  

Andrew Straw wrote:

Hi Eric,

show() isn't bringing up my plots with r5430 in normal scripts (i.e. not
using IPython).

Andrew,

Please try now, with r5435 or later. There is still the underlying puzzle to be solved before all this can be cleaned up, but I think the basic functionality is back.

Eric

···

-Andrew

Eric Firing wrote:

While investigating a bug pointed out by Andrew (and one that I had introduced some time ago) I found to my horror that I had also caused another bug: interactive plotting was not working with ipython -pylab. I backed out part of r5420, and that fixed the problem. This temporary fix is to restore the mixed case backend names in the interactive_bk list that is now set in rcsetup.py and imported by backends/__init__.py.

I am baffled as to why this is needed at present; I have not been able to figure out why ipython threading seems to get confused when that list has lower case names for gtkagg, qtagg, and wxagg, but not for tkagg or qt4agg. I don't see anywhere in either the ipython or the mpl code that should care about the case of the names in that list, now that I have ensured that comparisons against backend names in mpl (everywhere I could find them--maybe I missed on) are case-insensitive, and get_backend() returns only lower case. rcParams['backend'] can still have mixed case, however.

Any suggestions as to what I am missing or where to look?

Related question: I just noticed that there is also a config/rcsetup.py. Is there any reason why there has to be so much duplication (and now gradual divergence) between this and the regular rcsetup.py? Is there any reason the traits config system can't use the regular version, maybe with some changes merged in?

Eric

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
  

Eric: in Shell,py, around line 1167, you'll find this code:

    backends = {'gthread' : 'GTKAgg',
                'wthread' : 'WXAgg',
                'qthread' : 'QtAgg',
                'q4thread' :'Qt4Agg',
                'tkthread' :'TkAgg',
                }

That is used to SET the rcParams['backend'] value. I don't know if
that can cause issues.

But more importantly, I think the problem you see comes from this code
(just a bit later in the same file):

        else:
            backend = matplotlib.rcParams['backend']
            if backend.startswith('GTK'):
                th_mode = 'gthread'
            elif backend.startswith('WX'):
                th_mode = 'wthread'
            elif backend.startswith('Qt4'):
                th_mode = 'q4thread'
            elif backend.startswith('Qt'):
                th_mode = 'qthread'
            else:
                # Any other backend, use plain Tk
                th_mode = 'tkthread'

        return mpl_shell[th_mode]

IPython does an explicit check on the names of the backends with that
exact capitalization. I could add a .lower() call to the line that
starts:

            backend = matplotlib.rcParams['backend'].lower()

and change the .startswith() calls to do all lowercase checks. Could
you see if that helps? That would make the fix backwards compatible.

Cheers,

f

···

On Mon, Jun 9, 2008 at 10:16 AM, Eric Firing <efiring@...229...> wrote:

Please try now, with r5435 or later. There is still the underlying
puzzle to be solved before all this can be cleaned up, but I think the
basic functionality is back.

Fernando Perez wrote:

Please try now, with r5435 or later. There is still the underlying
puzzle to be solved before all this can be cleaned up, but I think the
basic functionality is back.

Eric: in Shell,py, around line 1167, you'll find this code:

    backends = {'gthread' : 'GTKAgg',
                'wthread' : 'WXAgg',
                'qthread' : 'QtAgg',
                'q4thread' :'Qt4Agg',
                'tkthread' :'TkAgg',
                }

That is used to SET the rcParams['backend'] value. I don't know if
that can cause issues.

No, that is OK with old or svn versions of mpl.

But more importantly, I think the problem you see comes from this code
(just a bit later in the same file):

        else:
            backend = matplotlib.rcParams['backend']
            if backend.startswith('GTK'):
                th_mode = 'gthread'
            elif backend.startswith('WX'):
                th_mode = 'wthread'
            elif backend.startswith('Qt4'):
                th_mode = 'q4thread'
            elif backend.startswith('Qt'):
                th_mode = 'qthread'
            else:
                # Any other backend, use plain Tk
                th_mode = 'tkthread'

        return mpl_shell[th_mode]

Yes, I was looking for this sort of thing but managed to miss it yesterday. This is where the problem is. The workaround that I have in mpl svn now is to simply retain the mixed cases (although the input and checking are completely case-insensitive). My suggested change for ipython, so that some day mpl would not need to retain the mixed cases, is in the attached diff against ipython svn. Fairly quick checks against mpl svn, mpl svn with all lower case, and mpl maintenance branch indicated that this change would be backwards and forwards compatible, so long as mpl retains rcsetup.validate_backend().

It also makes it so that you could go to lower case in:
     backends = {'gthread' : 'GTKAgg',
                 'wthread' : 'WXAgg',
                 'qthread' : 'QtAgg',
                 'q4thread' :'Qt4Agg',
                 'tkthread' :'TkAgg',
                 }
and this should also be backwards and forwards compatible; by using validate_backend when setting the backend, one ensures that rcParams gets a form that the particular version of mpl is happy with.

You might be able to use matplotlib.use(backends[th_mode]) to replace
the following in my suggested diff, provided you have not already performed some magic that would foil the sys.modules checking done by matplotlib.use():

             backend = mrcsetup.validate_backend(backends[th_mode])
             matplotlib.rcParams['backend'] = backend

The rationale for using "matplotlib.use" and "matplotlib.get_backend" is that they can take advantage of validation that is not available when you directly get and set the rcParams value. On the other hand there is some appeal in the directness of going straight to/from rcParams.

Anyway, regardless of whether you decide to make any changes in ipython, thanks for solving the puzzle.

Eric

ipython_diff1 (1.43 KB)

···

On Mon, Jun 9, 2008 at 10:16 AM, Eric Firing <efiring@...229...> wrote:

IPython does an explicit check on the names of the backends with that
exact capitalization. I could add a .lower() call to the line that
starts:

            backend = matplotlib.rcParams['backend'].lower()

and change the .startswith() calls to do all lowercase checks. Could
you see if that helps? That would make the fix backwards compatible.

Cheers,

f

Thanks for providing a patch. I'll be mostly offline for a few days,
but I'll add the fix in some form (the beauty of bzr...). It will
show up in the ipython trunk probably by weekend.

Cheers,

f

···

On Mon, Jun 9, 2008 at 8:00 PM, Eric Firing <efiring@...229...> wrote:

Anyway, regardless of whether you decide to make any changes in ipython,
thanks for solving the puzzle.