Hello all,
[sent to mpl-dev, enthought-dev and ipython-dev]
This summer, we are doing some major refactoring of IPython’s core. One of the things I am working on is changing how IPython’s works with GUI toolkits. These changes will have a significant impact (hopefully for the better) on your project, so I wanted to open a discussion about this issue.
Here is the current situation: currently, IPython uses threads to allow GUI event loops. This code lives in IPython.Shell and is extremely subtle, hard to maintain and fragile. Fernando and John Hunter have done a fantastic job in developing this code, but in the long run we need a more robust approach.
Here is the proposal: Python has an obscure hook called PyOS_InputHook. By using this hook, GUI toolkits can interleave their event loop with a command line program without threads. Even though PyOS_InputHook is not well known, this is how Python’s built-in integration with Tk works. The good news is that other GUI toolkits are starting to support PyOS_InputHook:
- PyGTK 2.15.1 has this.
- The mpl MacOSX backend works this way
- Recent versions of PyQT 4 have this.
- I am working with Robin Dunn to implement this in wxPython 2.8 and 2.9
Bottom line: once people are using these recent/upcoming versions of the GUI toolkits, IPython will no longer need to maintain the code in Shell.py and IPython won’t need to have -pylab/-wthread/etc options.
So, how does affect your project?
- People will be able to use your project interactive from the regular python prompt.
- You will need to make small changes to your GUI toolkits initialization code.
- All of us will need to coordinate version transitions to make sure that there is a clean transition to this new approach.
- I need help testing the new approach (especially with wxPython) to make sure that your project actually works with the new approach.
What needs to be done at this point?
- I would like to discuss how the transition should be made in terms of versions.
- I need help testing this new approach in the various toolkits - especially with wx.
- I want to see if there are other issues related to this that I am missing.
Cheers,
Brian
Would it be possible for IPython to expose both solutions for a while
(say 6 month to a year), so that we all have time to adapt?
Ga�l
···
On Thu, Jul 16, 2009 at 10:04:33AM -0700, Brian Granger wrote:
So, how does affect your project?
* People will be able to use your project interactive from the regular
python prompt.
* You will need to make small changes to your GUI toolkits initialization
code.
* All of us will need to coordinate version transitions to make sure that
there is a clean transition to this new approach.
* I need help testing the new approach (especially with wxPython) to make
sure that your project actually works with the new approach.
This is a great project and I'm sure we're all looking forward to having "just one way to do it".
However, do not overestimate how up-to-date these packages will be, particularly in managed environments. For instance the RHEL4 boxes we run at my employer still have pygtk 2.4. (Yes, that is 4 years old!) It would be unfortunate if our users couldn't update matplotlib without the pain of recompiling a large part of the gtk stack underneath.
So just a plea to keep the old code paths working -- perhaps surrounded in big flashing "REMOVE ME LATER" comments. I understand that maintaining code that fewer and fewer users will be running is like a time bomb. Maybe we could raise a deprecation warning if a user has an old version of a toolkit, so at least when the bomb finally goes off the user has a first guess as to why. But I think dropping all support for these older versions in one step would be a mistake.
Cheers,
Mike
Brian Granger wrote:
···
Hello all,
[sent to mpl-dev, enthought-dev and ipython-dev]
This summer, we are doing some major refactoring of IPython's core. One of the things I am working on is changing how IPython's works with GUI toolkits. These changes will have a significant impact (hopefully for the better) on your project, so I wanted to open a discussion about this issue.
Here is the current situation: currently, IPython uses threads to allow GUI event loops. This code lives in IPython.Shell and is extremely subtle, hard to maintain and fragile. Fernando and John Hunter have done a fantastic job in developing this code, but in the long run we need a more robust approach.
Here is the proposal: Python has an obscure hook called PyOS_InputHook. By using this hook, GUI toolkits can interleave their event loop with a command line program *without threads*. Even though PyOS_InputHook is not well known, this is how Python's built-in integration with Tk works. The good news is that other GUI toolkits are starting to support PyOS_InputHook:
* PyGTK 2.15.1 has this.
* The mpl MacOSX backend works this way
* Recent versions of PyQT 4 have this.
* I am working with Robin Dunn to implement this in wxPython 2.8 and 2.9
Bottom line: once people are using these recent/upcoming versions of the GUI toolkits, IPython will no longer need to maintain the code in Shell.py and IPython won't need to have -pylab/-wthread/etc options.
So, how does affect your project?
* People will be able to use your project interactive from the regular python prompt.
* You will need to make small changes to your GUI toolkits initialization code.
* All of us will need to coordinate version transitions to make sure that there is a clean transition to this new approach.
* I need help testing the new approach (especially with wxPython) to make sure that your project actually works with the new approach.
What needs to be done at this point?
* I would like to discuss how the transition should be made in terms of versions.
* I need help testing this new approach in the various toolkits - especially with wx.
* I want to see if there are other issues related to this that I am missing.
Cheers,
Brian
------------------------------------------------------------------------
------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge
------------------------------------------------------------------------
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options
--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
So, how does affect your project?
- People will be able to use your project interactive from the regular
python prompt.
- You will need to make small changes to your GUI toolkits initialization
code.
- All of us will need to coordinate version transitions to make sure that
there is a clean transition to this new approach.
- I need help testing the new approach (especially with wxPython) to make
sure that your project actually works with the new approach.
Would it be possible for IPython to expose both solutions for a while
(say 6 month to a year), so that we all have time to adapt?
Yes and no. In the short term, this is definitely possible. Also, as Ville mentions, the new approach will also work with any older version of IPython.
However, one of the main things that needs to be refactored in IPython is the threading model. This is needed for a variety of reasons, but the bottom line is this - a number of projects are using IPython as if it were thread safe (including IPython itself) - but it is not. When I fix these things (which will be soon), I am not sure (I am doubtful) that the threaded Shell.py code will continue to work. But, this is something that I can look into.
But, at some point, we will have to say “if you want the old threaded shells, you will have to use an older version of IPython”. We just have to figure out what the transition looks like.
Cheers,
Brian
···
On Thu, Jul 16, 2009 at 10:20 AM, Gael Varoquaux <gael.varoquaux@…427…> wrote:
On Thu, Jul 16, 2009 at 10:04:33AM -0700, Brian Granger wrote:
Gaël
As long as the packages sitting outside of IPython have a way of dealing
with both reasonnably easily that fine. What we don't want to happen is
having confusing errors or instructions depending on the version of
IPython people use.
As Michael points out, there will be people using the old model for a
long time.
Ga�l
···
On Thu, Jul 16, 2009 at 10:38:56AM -0700, Brian Granger wrote:
But, at some point, we will have to say "if you want the old threaded
shells, you will have to use an older version of IPython".� We just have
to figure out what the transition looks like.
This is a great project and I’m sure we’re all looking forward to having “just one way to do it”.
However, do not overestimate how up-to-date these packages will be, particularly in managed environments. For instance the RHEL4 boxes we run at my employer still have pygtk 2.4. (Yes, that is 4 years old!) It would be unfortunate if our users couldn’t update matplotlib without the pain of recompiling a large part of the gtk stack underneath.
I am fully aware that some environments upgrade things like GUI toolkits over very long time scales. More important - there isn’t a released version of wx that has these capabilities in it. It will be a long time before everyone is running the needed versions of wx/gtk/qt. In the meantime, we will do everything we can to make sure that people running older versions of GUI toolkits can continue to use IPython (and mpl).
On the mpl side of things, there will probably have to be some changes to the GUI toolkit initialization code that detects which way things are being run (old/new) and then does the right thing. If we coordinate this well, that should allow newer versions of mpl to work with both new and old versions of IPython and the GUI toolkits. However at some point, we will have to make a clean break.
So just a plea to keep the old code paths working – perhaps surrounded in big flashing “REMOVE ME LATER” comments. I understand that maintaining code that fewer and fewer users will be running is like a time bomb.
This is especially true of the threaded Shells in IPython. Honestly, even with thousands of people using it (as is currently the case) it is a time bomb.
Maybe we could raise a deprecation warning if a user has an old version of a toolkit, so at least when the bomb finally goes off the user has a first guess as to why. But I think dropping all support for these older versions in one step would be a mistake.
Yes, there will definitely have to be some sort of transition. One possibility is that during the transition, the old threaded Shells would slowly loose features that become impossible to maintain (such as interrupt handling in the threaded shells).
With our very limited IPython manpower, we have a very fine line to walk between moving forward and maintaining backwards compatibility.
Cheers,
Brian
···
Cheers,
Mike
Brian Granger wrote:
Hello all,
[sent to mpl-dev, enthought-dev and ipython-dev]
This summer, we are doing some major refactoring of IPython’s core. One of the things I am working on is changing how IPython’s works with GUI toolkits. These changes will have a significant impact (hopefully for the better) on your project, so I wanted to open a discussion about this issue.
Here is the current situation: currently, IPython uses threads to allow GUI event loops. This code lives in IPython.Shell and is extremely subtle, hard to maintain and fragile. Fernando and John Hunter have done a fantastic job in developing this code, but in the long run we need a more robust approach.
Here is the proposal: Python has an obscure hook called PyOS_InputHook. By using this hook, GUI toolkits can interleave their event loop with a command line program without threads. Even though PyOS_InputHook is not well known, this is how Python’s built-in integration with Tk works. The good news is that other GUI toolkits are starting to support PyOS_InputHook:
-
PyGTK 2.15.1 has this.
-
The mpl MacOSX backend works this way
-
Recent versions of PyQT 4 have this.
-
I am working with Robin Dunn to implement this in wxPython 2.8 and 2.9
Bottom line: once people are using these recent/upcoming versions of the GUI toolkits, IPython will no longer need to maintain the code in Shell.py and IPython won’t need to have -pylab/-wthread/etc options.
So, how does affect your project?
-
People will be able to use your project interactive from the regular python prompt.
-
You will need to make small changes to your GUI toolkits initialization code.
-
All of us will need to coordinate version transitions to make sure that there is a clean transition to this new approach.
-
I need help testing the new approach (especially with wxPython) to make sure that your project actually works with the new approach.
What needs to be done at this point?
-
I would like to discuss how the transition should be made in terms of versions.
-
I need help testing this new approach in the various toolkits - especially with wx.
-
I want to see if there are other issues related to this that I am missing.
Cheers,
Brian
Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World™ will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
–
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA
After playing a bit more, I have a ctypes based prototype that can set PyOS_InputHook in an appropriate way for wx 2.9 (trunk). I think it is possible to have a ctypes based version that would work for all the major GUI toolkits that we could ship with IPython until the transition is over (i.e., everyone is using a GUI toolkits recent enough).
BUT. I need to begin to narrow the GUI toolkits and versions that we will support.
The total number of possibilities (for testing, etc.) is approximately:
(number of OSs~3) x (number of GUI toolkits~qt4/wx/gtk) x (number of supported versions of each GUI)
Because the first 2 terms are pretty fixed, I want to bring sanity to the picture by keeping the 3rd term as small as possible. So, here is a question.
What versions of what GUI toolkits do we realistically need to support moving forward (today and beyond)? By this, I mean that:
“unsupported” = older versions of GUIs toolkits that will have to use older versions of IPython.
“supported” = versions of GUIS that will be able to use the PyOS_InputHook approach in newer releases of IPython.
Cheers,
Brian
For Wx, we would obviously need to support 2.8. We can toy with the idea
of not supporting 2.6, I believe. A lot of people are starting to feel
that 2.6 is depreciated.
Ga�l
···
On Thu, Jul 16, 2009 at 01:36:21PM -0700, Brian Granger wrote:
What versions of what GUI toolkits do we realistically need to support
moving forward (today and beyond)?� By this, I mean that:
"unsupported" = older versions of GUIs toolkits that will have to use
older versions of IPython.
�"supported" = versions of GUIS that will be able to use the
PyOS_InputHook approach in newer releases of IPython.
For Wx, we would obviously need to support 2.8. We can toy with the idea
of not supporting 2.6, I believe. A lot of people are starting to feel
that 2.6 is depreciated.
Yes, 2.8 is latest stable, so we should support it. How many people would complain if 2.6 were not supported? I take it that all of ETS is 2.8 based?
Cheers,
Brian
···
Gaël
ETS works better on wx 2.8, but it also works on 2.6, with small bugs.
Ga�l
···
On Thu, Jul 16, 2009 at 02:34:01PM -0700, Brian Granger wrote:
Yes, 2.8 is latest stable, so we should support it.� How many people would
complain if 2.6 were not supported?� I take it that all of ETS is 2.8
based?