For review and merging: new GUi support for Qt4 and Wx

[_Brian_Granger]

Hello all,

I would like to submit the following branch on github for review and
merging into matplotlib trunk:

http://github.com/ellisonbg/matplotlib/commits/guisupport

This branch implements the logic needed for the qt4 and wx backends to
fully work with the upcoming IPython 0.11 release. In our testing we
have run many of the mpl examples (including the new animation
examples) in both qt4/wx in both the terminal based IPython and the
new IPython Qt GUI. For background on these changes please see this
thread:

http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel

It is important to note that we have not updated the other matplotlib
backends (gtk, tk, etc.) to have this logic. This is mainly because
we know almost nothing about these toolkits and could really use some
help from folks who are experts at the respective toolkits. We have
done some minimal testing and these other backends do work for simple
examples in the terminal IPython, but they won't work in all cases and
will definitely not work in the new IPython Qt based GUI.

We would love feedback and help testing as these changes are
significant (even though only a few lines of code). To test this
stuff you will need to grab the following IPython development branch:

http://github.com/ipython/ipython/tree/newkernel

You should be able to run the examples in regular ipython:

ipython --pylab qt4|wx

Or the new GUI

ipythonqt --pylab qt4|wx

Cheers,

Brian

[Eric_Firing2]

Brian and others,

It's not quite that simple. After some initial thrashing around, I installed zmq from source, and then pyzmq--but I can't import zmq:

In [1]: import zmq

···

On 09/03/2010 12:37 PM, Brian Granger wrote:

Hello all,

I would like to submit the following branch on github for review and
merging into matplotlib trunk:

http://github.com/ellisonbg/matplotlib/commits/guisupport

This branch implements the logic needed for the qt4 and wx backends to
fully work with the upcoming IPython 0.11 release. In our testing we
have run many of the mpl examples (including the new animation
examples) in both qt4/wx in both the terminal based IPython and the
new IPython Qt GUI. For background on these changes please see this
thread:

Thread: [matplotlib-devel] Uniform GUI support across matplotlib, ets and ipython | matplotlib

It is important to note that we have not updated the other matplotlib
backends (gtk, tk, etc.) to have this logic. This is mainly because
we know almost nothing about these toolkits and could really use some
help from folks who are experts at the respective toolkits. We have
done some minimal testing and these other backends do work for simple
examples in the terminal IPython, but they won't work in all cases and
will definitely not work in the new IPython Qt based GUI.

We would love feedback and help testing as these changes are
significant (even though only a few lines of code). To test this
stuff you will need to grab the following IPython development branch:

http://github.com/ipython/ipython/tree/newkernel

You should be able to run the examples in regular ipython:

ipython --pylab qt4|wx

Or the new GUI

ipythonqt --pylab qt4|wx

---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
/home/efiring/<ipython console> in <module>()

/usr/local/lib/python2.6/dist-packages/zmq/__init__.py in <module>()
      24 #-----------------------------------------------------------------------------

      25
---> 26 from zmq import _zmq
      27 from zmq._zmq import *
      28

ImportError: libzmq.so.0: cannot open shared object file: No such file or directory

But it is right where it should be:

efiring@...340...:~/programs/py/ipython/pyzmq_git$ ll /usr/local/lib/libzmq*
-rw-r--r-- 1 root root 4736930 2010-09-03 14:35 /usr/local/lib/libzmq.a
-rwxr-xr-x 1 root root 825 2010-09-03 14:35 /usr/local/lib/libzmq.la
lrwxrwxrwx 1 root root 15 2010-09-03 14:35 /usr/local/lib/libzmq.so -> libzmq.so.0.0.0
lrwxrwxrwx 1 root root 15 2010-09-03 14:35 /usr/local/lib/libzmq.so.0 -> libzmq.so.0.0.0
-rwxr-xr-x 1 root root 1993116 2010-09-03 14:35 /usr/local/lib/libzmq.so.0.0.0

And it looks like the extension was built OK, complete with linking:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include -I/usr/include/python2.6 -c zmq/_zmq.c -o build/temp.linux-x86_64-2.6/zmq/_zmq.o
zmq/_zmq.c:1995: warning: ‘__pyx_doc_3zmq_4_zmq_7Message___len__’ defined but not used
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/zmq/_zmq.o -L/usr/local/lib -lzmq -o build/lib.linux-x86_64-2.6/zmq/_zmq.so

I'm stumped.

Eric

Cheers,

Brian

[_Brian_Granger]

Eric,

Looks like you are on Linux. If so, you will have to make sure you do
two things before doing python setup.py install on pyzmq:

1. Edit the setup.cfg (copy the template) to point to your zeromq install.
2. Use the --rpath option:

python setup.py build_ext --rpath=/opt/zeromq-dev/lib --inplace

Alternatively, you can set the LD_LIBRARY_PATH to point to your zeromq lib path.

Also, for now, we suggest using the stable 2.0.7 releases of both
pyzmq and zeromq.

Sorry about the hitches, parts of this are definitely still rough and
we hope to iron out these things. I should mention that zeromq/pyzmq
are only required for the new Qt gui, the old terminal ipython doesn't
require them.
Let us know if you run into any other issues.

Cheers,

Brian

···

On Fri, Sep 3, 2010 at 6:57 PM, Eric Firing <efiring@...229...> wrote:

On 09/03/2010 12:37 PM, Brian Granger wrote:

Hello all,

I would like to submit the following branch on github for review and
merging into matplotlib trunk:

http://github.com/ellisonbg/matplotlib/commits/guisupport

This branch implements the logic needed for the qt4 and wx backends to
fully work with the upcoming IPython 0.11 release. In our testing we
have run many of the mpl examples (including the new animation
examples) in both qt4/wx in both the terminal based IPython and the
new IPython Qt GUI. For background on these changes please see this
thread:

Thread: [matplotlib-devel] Uniform GUI support across matplotlib, ets and ipython | matplotlib

It is important to note that we have not updated the other matplotlib
backends (gtk, tk, etc.) to have this logic. This is mainly because
we know almost nothing about these toolkits and could really use some
help from folks who are experts at the respective toolkits. We have
done some minimal testing and these other backends do work for simple
examples in the terminal IPython, but they won't work in all cases and
will definitely not work in the new IPython Qt based GUI.

We would love feedback and help testing as these changes are
significant (even though only a few lines of code). To test this
stuff you will need to grab the following IPython development branch:

http://github.com/ipython/ipython/tree/newkernel

You should be able to run the examples in regular ipython:

ipython --pylab qt4|wx

Or the new GUI

ipythonqt --pylab qt4|wx

Brian and others,

It's not quite that simple. After some initial thrashing around, I
installed zmq from source, and then pyzmq--but I can't import zmq:

In [1]: import zmq
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
/home/efiring/<ipython console> in <module>()

/usr/local/lib/python2.6/dist-packages/zmq/__init__.py in <module>()
24
#-----------------------------------------------------------------------------

 25

---> 26 from zmq import _zmq
27 from zmq._zmq import *
28

ImportError: libzmq.so.0: cannot open shared object file: No such file
or directory

But it is right where it should be:

efiring@...340...:~/programs/py/ipython/pyzmq_git$ ll /usr/local/lib/libzmq*
-rw-r--r-- 1 root root 4736930 2010-09-03 14:35 /usr/local/lib/libzmq.a
-rwxr-xr-x 1 root root 825 2010-09-03 14:35 /usr/local/lib/libzmq.la
lrwxrwxrwx 1 root root 15 2010-09-03 14:35 /usr/local/lib/libzmq.so
-> libzmq.so.0.0.0
lrwxrwxrwx 1 root root 15 2010-09-03 14:35
/usr/local/lib/libzmq.so.0 -> libzmq.so.0.0.0
-rwxr-xr-x 1 root root 1993116 2010-09-03 14:35
/usr/local/lib/libzmq.so.0.0.0

And it looks like the extension was built OK, complete with linking:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/local/include -I/usr/include/python2.6
-c zmq/_zmq.c -o build/temp.linux-x86_64-2.6/zmq/_zmq.o
zmq/_zmq.c:1995: warning: ‘__pyx_doc_3zmq_4_zmq_7Message___len__’
defined but not used
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
build/temp.linux-x86_64-2.6/zmq/_zmq.o -L/usr/local/lib -lzmq -o
build/lib.linux-x86_64-2.6/zmq/_zmq.so

I'm stumped.

Eric

Cheers,

Brian

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger@...882...
ellisonbg@...149...

[_Fernando_Perez1]

Mhh, sorry to see you burn up on this, Eric. Brian is the zmq expert,
not me, but it *may* be a version compatibility issue, perhaps? I am
using these versions of zmq/pyzmq:

- zmq 2.0.8: http://www.zeromq.org/local--files/area:download/zeromq-2.0.8.tar.gz
- pyzmq 2.0.7: http://github.com/downloads/zeromq/pyzmq/pyzmq-2.0.7.tar.gz

These two worked fine for me, with:

- for zmq:

./configure --prefix=$HOME/usr/opt
make && make install

note that $HOME/usr/opt/lib is in my LD_LIBRARY_PATH, and the include/
dir in my include path, etc.

- for pyzmq:

python setup.py install --prefix=$HOME/usr/opt

And at that point it just worked. A few questions:

- is it possible that you did a build from the head of either
zmq/pyzmq git tree? that might cause an incompatibility.
- could it be that /usr/local/lib isn in your LD_LIBRARY_PATH?

I'll be available over the weekend and will be more than happy to try
and help sort this out.

Regards, and sorry for the hassle

f

···

On Fri, Sep 3, 2010 at 6:57 PM, Eric Firing <efiring@...229...> wrote:

It's not quite that simple. After some initial thrashing around, I
installed zmq from source, and then pyzmq--but I can't import zmq:

[Eric_Firing2]

Hello all,

I would like to submit the following branch on github for review and
merging into matplotlib trunk:

http://github.com/ellisonbg/matplotlib/commits/guisupport

This branch implements the logic needed for the qt4 and wx backends to
fully work with the upcoming IPython 0.11 release. In our testing we
have run many of the mpl examples (including the new animation
examples) in both qt4/wx in both the terminal based IPython and the
new IPython Qt GUI. For background on these changes please see this
thread:

Thread: [matplotlib-devel] Uniform GUI support across matplotlib, ets and ipython | matplotlib

It is important to note that we have not updated the other matplotlib
backends (gtk, tk, etc.) to have this logic. This is mainly because
we know almost nothing about these toolkits and could really use some
help from folks who are experts at the respective toolkits. We have
done some minimal testing and these other backends do work for simple
examples in the terminal IPython, but they won't work in all cases and
will definitely not work in the new IPython Qt based GUI.

We would love feedback and help testing as these changes are
significant (even though only a few lines of code). To test this
stuff you will need to grab the following IPython development branch:

http://github.com/ipython/ipython/tree/newkernel

You should be able to run the examples in regular ipython:

ipython --pylab qt4|wx

Or the new GUI

ipythonqt --pylab qt4|wx

Brian, Fernando,

I have been doing a little testing with ipython 0.10 versus ipython-newkernel, both modes, and with mpl svn versus your guisupport. There are so many possible modes of operation and combinations of versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when you use mpl *svn* with ipython-newkernel, in either or both of the console or gui modes, but *does* work with your guisupport version?

Thanks.

Eric

···

On 09/03/2010 12:37 PM, Brian Granger wrote:

Cheers,

Brian

[_Fernando_Perez1]

Hi Eric,

I have been doing a little testing with ipython 0.10 versus
ipython-newkernel, both modes, and with mpl svn versus your guisupport.
There are so many possible modes of operation and combinations of
versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when
you use mpl *svn* with ipython-newkernel, in either or both of the
console or gui modes, but *does* work with your guisupport version?

Thanks for your testing, Eric.

With matplotlib *alone*, I can't find a way to crash/lock/whatever the
combo of matplotlib(svn)+ipython-newkernel.

The reason, i believe, is that guisupport.py is available in ipython
itself, and it goes out of its way to avoid creating a second main qt
app, letting matplotlib create it. Since that main app is alive all
the time, there's only one app and one event loop and life is good.
But if I were to open another library that uses Qt and makes a new
main qApp unconditionally, we'd have problems.

Brian and Evan have recently just added the guisupport.py patch to
Enthought's ETS, so that now it probably will be pretty hard to
actually see the problem: if both ipython and ets go out of their way
to avoid the nested main app issue, mpl can get away with making one
unconditionally and things will probably work OK.

But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
collaborative protocol with a simple api: check for one special
'_in_event_loop' flag in the main toolkit before making one. That
will make it easier to have interactive code that uses Wx or Qt from
more than one library coexisting in the same process.

I'll let Brian fill in with more details when he has some
availability, but I think that's the gist of it.

Regards,

f

···

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing <efiring@...229...> wrote:

[Eric_Firing2]

Hi Eric,

I have been doing a little testing with ipython 0.10 versus
ipython-newkernel, both modes, and with mpl svn versus your guisupport.
There are so many possible modes of operation and combinations of
versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when
you use mpl *svn* with ipython-newkernel, in either or both of the
console or gui modes, but *does* work with your guisupport version?

Thanks for your testing, Eric.

With matplotlib *alone*, I can't find a way to crash/lock/whatever the
combo of matplotlib(svn)+ipython-newkernel.

The reason, i believe, is that guisupport.py is available in ipython
itself, and it goes out of its way to avoid creating a second main qt
app, letting matplotlib create it. Since that main app is alive all
the time, there's only one app and one event loop and life is good.
But if I were to open another library that uses Qt and makes a new
main qApp unconditionally, we'd have problems.

Brian and Evan have recently just added the guisupport.py patch to
Enthought's ETS, so that now it probably will be pretty hard to
actually see the problem: if both ipython and ets go out of their way
to avoid the nested main app issue, mpl can get away with making one
unconditionally and things will probably work OK.

But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
collaborative protocol with a simple api: check for one special
'_in_event_loop' flag in the main toolkit before making one. That
will make it easier to have interactive code that uses Wx or Qt from
more than one library coexisting in the same process.

Fernando,

There are two parts to guisupport: ensure a single main app, and ensure no more than one call to the mainloop. The first makes perfect sense, and cannot cause any problems that I can see. The second one is the one that I think may be both unnecessary and undesirable. The reason is that the gui toolkit mainloop functions or methods are designed for nested calls. This permits blocking within a running mainloop, and allows show() to block when pyplot is not in interactive mode. This is what is lost with the guisupport mods. Some changes to mainloop logic may well be needed, but I don't think that prohibiting nested calls to the underlying toolkit mainloop function is necessary or desirable.

Eric

···

On 09/07/2010 11:07 AM, Fernando Perez wrote:

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<efiring@...229...> wrote:

I'll let Brian fill in with more details when he has some
availability, but I think that's the gist of it.

Regards,

f

[_Brian_Granger]

Eric,

Sorry about the delay, I was on vacation last week...comments inline below...

Hi Eric,

I have been doing a little testing with ipython 0.10 versus
ipython-newkernel, both modes, and with mpl svn versus your guisupport.
There are so many possible modes of operation and combinations of
versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when
you use mpl *svn* with ipython-newkernel, in either or both of the
console or gui modes, but *does* work with your guisupport version?

Thanks for your testing, Eric.

With matplotlib *alone*, I can't find a way to crash/lock/whatever the
combo of matplotlib(svn)+ipython-newkernel.

The reason, i believe, is that guisupport.py is available in ipython
itself, and it goes out of its way to avoid creating a second main qt
app, letting matplotlib create it. Since that main app is alive all
the time, there's only one app and one event loop and life is good.
But if I were to open another library that uses Qt and makes a new
main qApp unconditionally, we'd have problems.

Brian and Evan have recently just added the guisupport.py patch to
Enthought's ETS, so that now it probably will be pretty hard to
actually see the problem: if both ipython and ets go out of their way
to avoid the nested main app issue, mpl can get away with making one
unconditionally and things will probably work OK.

But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
collaborative protocol with a simple api: check for one special
'_in_event_loop' flag in the main toolkit before making one. That
will make it easier to have interactive code that uses Wx or Qt from
more than one library coexisting in the same process.

Fernando,

There are two parts to guisupport: ensure a single main app, and ensure
no more than one call to the mainloop.

Yes, that is a good summary.

The first makes perfect sense,
and cannot cause any problems that I can see. The second one is the one
that I think may be both unnecessary and undesirable. The reason is
that the gui toolkit mainloop functions or methods are designed for
nested calls. This permits blocking within a running mainloop, and
allows show() to block when pyplot is not in interactive mode. This is
what is lost with the guisupport mods. Some changes to mainloop logic
may well be needed, but I don't think that prohibiting nested calls to
the underlying toolkit mainloop function is necessary or desirable.

This is a very good point and is something that we have thought
carefully about. You are very correct, that the functions in
guisupport cannot be used to do a nested mainloop. Nested calls to
the mainloop should be done in the usual manner by simply calling the
appropriate gui toolkit method for doing so. We probably need to
clarify this point, but the focus of the functions in guisupport are
*only* the first and main invocation of the event loop. Basically, we
want to ensure that:

* Projects don't accidentally do nested mainloops because there were
not aware that the outermost eventloop was already running.
* Projects start the outermost eventloop in a manner that other
projects will be able to reliably detect.

I should mention the other approach that we have tried, but that failed:

* Have IPython startup, create an app and start the main loop.
* Then monkeypatch the gui toolkit so that the mainloop calls are no-ops.
* Further monkeypatch the gui toolkit so that it appears that the
mainloop is running (even when it may not be because of PyOS_InputHook
magic).

This allowed us to do everything, BUT obviously, nested mainloops
failed. Thus, making sure that nested mainloops still work was the
main reason we have created guisupport. We should better document
these details though.

Cheers,

Brian

···

On Tue, Sep 7, 2010 at 2:26 PM, Eric Firing <efiring@...229...> wrote:

On 09/07/2010 11:07 AM, Fernando Perez wrote:

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<efiring@...229...> wrote:

Eric

I'll let Brian fill in with more details when he has some
availability, but I think that's the gist of it.

Regards,

f

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger@...882...
ellisonbg@...149...

[_Brian_Granger]

Hello all,

I would like to submit the following branch on github for review and
merging into matplotlib trunk:

http://github.com/ellisonbg/matplotlib/commits/guisupport

This branch implements the logic needed for the qt4 and wx backends to
fully work with the upcoming IPython 0.11 release. In our testing we
have run many of the mpl examples (including the new animation
examples) in both qt4/wx in both the terminal based IPython and the
new IPython Qt GUI. For background on these changes please see this
thread:

Thread: [matplotlib-devel] Uniform GUI support across matplotlib, ets and ipython | matplotlib

It is important to note that we have not updated the other matplotlib
backends (gtk, tk, etc.) to have this logic. This is mainly because
we know almost nothing about these toolkits and could really use some
help from folks who are experts at the respective toolkits. We have
done some minimal testing and these other backends do work for simple
examples in the terminal IPython, but they won't work in all cases and
will definitely not work in the new IPython Qt based GUI.

We would love feedback and help testing as these changes are
significant (even though only a few lines of code). To test this
stuff you will need to grab the following IPython development branch:

http://github.com/ipython/ipython/tree/newkernel

You should be able to run the examples in regular ipython:

ipython --pylab qt4|wx

Or the new GUI

ipythonqt --pylab qt4|wx

Brian, Fernando,

I have been doing a little testing with ipython 0.10 versus
ipython-newkernel, both modes, and with mpl svn versus your guisupport.
There are so many possible modes of operation and combinations of
versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when
you use mpl *svn* with ipython-newkernel, in either or both of the
console or gui modes, but *does* work with your guisupport version?

The problems are when matplotlib and enthought.traits/pyface/mayavi
stuff are used together. There are two types of problems:

* Multiple apps are created. The enthought stuff used to not check
for existing apps, but that has been fixed.
* Event loop is started multiple times. This one is more subtle and
on some toolkits the error is not fatal. This problem shows up when
IPython is run in terminal mode and event loop integration is handled
through PyOS_InputHook. In these situations, if matplotlib or ets
start the event loop themselves, IPython will hang.

Unfortunately, I am having trouble getting an install of both
matplotlib svn and ets svn on the same machine, so I can't reproduce
any of the failures ATM. I am trying to get things installed so I can
reproduce the problems.

Cheers,

Brian

···

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing <efiring@...229...> wrote:

On 09/03/2010 12:37 PM, Brian Granger wrote:

Thanks.

Eric

Cheers,

Brian

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger@...882...
ellisonbg@...149...

[Eric_Firing2]

Eric,

Sorry about the delay, I was on vacation last week...comments inline below...

Hi Eric,

I have been doing a little testing with ipython 0.10 versus
ipython-newkernel, both modes, and with mpl svn versus your guisupport.
There are so many possible modes of operation and combinations of
versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when
you use mpl *svn* with ipython-newkernel, in either or both of the
console or gui modes, but *does* work with your guisupport version?

Thanks for your testing, Eric.

With matplotlib *alone*, I can't find a way to crash/lock/whatever the
combo of matplotlib(svn)+ipython-newkernel.

The reason, i believe, is that guisupport.py is available in ipython
itself, and it goes out of its way to avoid creating a second main qt
app, letting matplotlib create it. Since that main app is alive all
the time, there's only one app and one event loop and life is good.
But if I were to open another library that uses Qt and makes a new
main qApp unconditionally, we'd have problems.

Brian and Evan have recently just added the guisupport.py patch to
Enthought's ETS, so that now it probably will be pretty hard to
actually see the problem: if both ipython and ets go out of their way
to avoid the nested main app issue, mpl can get away with making one
unconditionally and things will probably work OK.

But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
collaborative protocol with a simple api: check for one special
'_in_event_loop' flag in the main toolkit before making one. That
will make it easier to have interactive code that uses Wx or Qt from
more than one library coexisting in the same process.

Fernando,

There are two parts to guisupport: ensure a single main app, and ensure
no more than one call to the mainloop.

Yes, that is a good summary.

  The first makes perfect sense,
and cannot cause any problems that I can see. The second one is the one
that I think may be both unnecessary and undesirable. The reason is
that the gui toolkit mainloop functions or methods are designed for
nested calls. This permits blocking within a running mainloop, and
allows show() to block when pyplot is not in interactive mode. This is
what is lost with the guisupport mods. Some changes to mainloop logic
may well be needed, but I don't think that prohibiting nested calls to
the underlying toolkit mainloop function is necessary or desirable.

This is a very good point and is something that we have thought
carefully about. You are very correct, that the functions in
guisupport cannot be used to do a nested mainloop. Nested calls to
the mainloop should be done in the usual manner by simply calling the
appropriate gui toolkit method for doing so. We probably need to
clarify this point, but the focus of the functions in guisupport are
*only* the first and main invocation of the event loop. Basically, we
want to ensure that:

* Projects don't accidentally do nested mainloops because there were
not aware that the outermost eventloop was already running.
* Projects start the outermost eventloop in a manner that other
projects will be able to reliably detect.

I should mention the other approach that we have tried, but that failed:

* Have IPython startup, create an app and start the main loop.
* Then monkeypatch the gui toolkit so that the mainloop calls are no-ops.
* Further monkeypatch the gui toolkit so that it appears that the
mainloop is running (even when it may not be because of PyOS_InputHook
magic).

This allowed us to do everything, BUT obviously, nested mainloops
failed. Thus, making sure that nested mainloops still work was the
main reason we have created guisupport. We should better document
these details though.

Brian,

Thanks for the clarification. Your basic approach sounds fine. I need to think a little more about how to handle this in mpl. The proposed modification is not quite right yet. I probably can't work on it until the weekend.

Eric

···

On 09/13/2010 08:54 AM, Brian Granger wrote:

On Tue, Sep 7, 2010 at 2:26 PM, Eric Firing<efiring@...229...> wrote:

On 09/07/2010 11:07 AM, Fernando Perez wrote:

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<efiring@...229...> wrote:

Cheers,

Brian

Eric

I'll let Brian fill in with more details when he has some
availability, but I think that's the gist of it.

Regards,

f

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

[Eric_Firing2]

Hello all,

I would like to submit the following branch on github for review and
merging into matplotlib trunk:

http://github.com/ellisonbg/matplotlib/commits/guisupport

This branch implements the logic needed for the qt4 and wx backends to
fully work with the upcoming IPython 0.11 release. In our testing we
have run many of the mpl examples (including the new animation
examples) in both qt4/wx in both the terminal based IPython and the
new IPython Qt GUI. For background on these changes please see this
thread:

Thread: [matplotlib-devel] Uniform GUI support across matplotlib, ets and ipython | matplotlib

It is important to note that we have not updated the other matplotlib
backends (gtk, tk, etc.) to have this logic. This is mainly because
we know almost nothing about these toolkits and could really use some
help from folks who are experts at the respective toolkits. We have
done some minimal testing and these other backends do work for simple
examples in the terminal IPython, but they won't work in all cases and
will definitely not work in the new IPython Qt based GUI.

We would love feedback and help testing as these changes are
significant (even though only a few lines of code). To test this
stuff you will need to grab the following IPython development branch:

http://github.com/ipython/ipython/tree/newkernel

You should be able to run the examples in regular ipython:

ipython --pylab qt4|wx

Or the new GUI

ipythonqt --pylab qt4|wx

Brian, Fernando,

I have been doing a little testing with ipython 0.10 versus
ipython-newkernel, both modes, and with mpl svn versus your guisupport.
There are so many possible modes of operation and combinations of
versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when
you use mpl *svn* with ipython-newkernel, in either or both of the
console or gui modes, but *does* work with your guisupport version?

The problems are when matplotlib and enthought.traits/pyface/mayavi
stuff are used together. There are two types of problems:

* Multiple apps are created. The enthought stuff used to not check
for existing apps, but that has been fixed.

That one is easy.

* Event loop is started multiple times. This one is more subtle and
on some toolkits the error is not fatal. This problem shows up when
IPython is run in terminal mode and event loop integration is handled
through PyOS_InputHook. In these situations, if matplotlib or ets
start the event loop themselves, IPython will hang.

So, this can be checked with nothing but IPython and mpl. I think I may have seen this with some combination of configurations, though not with what I typically use.

Unfortunately, I am having trouble getting an install of both
matplotlib svn and ets svn on the same machine, so I can't reproduce
any of the failures ATM. I am trying to get things installed so I can
reproduce the problems.

At least twice in the last couple of years I have tried to get mayavi compiled and running, without fouling up my development versions of numpy and mpl. I never succeeded. Granted, I didn't allocate much time and mental energy to it.

In any case, with the help of your recent explanations, I expect we can make mpl play by your suggested rules without sacrificing anything. Part of the change for mpl 1.0 was to factor all of the show logic out into backend_bases.py, leaving only the core mainloop call in the specific backends. I hope we can keep that aspect, retaining the ability to work with earlier ipython (0.10) and the ability for show to block or not depending on the interactivity state.

Eric

···

On 09/13/2010 05:46 PM, Brian Granger wrote:

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<efiring@...229...> wrote:

On 09/03/2010 12:37 PM, Brian Granger wrote:

[_Brian_Granger]

Hello all,

I would like to submit the following branch on github for review and
merging into matplotlib trunk:

http://github.com/ellisonbg/matplotlib/commits/guisupport

This branch implements the logic needed for the qt4 and wx backends to
fully work with the upcoming IPython 0.11 release. In our testing we
have run many of the mpl examples (including the new animation
examples) in both qt4/wx in both the terminal based IPython and the
new IPython Qt GUI. For background on these changes please see this
thread:

Thread: [matplotlib-devel] Uniform GUI support across matplotlib, ets and ipython | matplotlib

It is important to note that we have not updated the other matplotlib
backends (gtk, tk, etc.) to have this logic. This is mainly because
we know almost nothing about these toolkits and could really use some
help from folks who are experts at the respective toolkits. We have
done some minimal testing and these other backends do work for simple
examples in the terminal IPython, but they won't work in all cases and
will definitely not work in the new IPython Qt based GUI.

We would love feedback and help testing as these changes are
significant (even though only a few lines of code). To test this
stuff you will need to grab the following IPython development branch:

http://github.com/ipython/ipython/tree/newkernel

You should be able to run the examples in regular ipython:

ipython --pylab qt4|wx

Or the new GUI

ipythonqt --pylab qt4|wx

Brian, Fernando,

I have been doing a little testing with ipython 0.10 versus
ipython-newkernel, both modes, and with mpl svn versus your guisupport.
There are so many possible modes of operation and combinations of
versions and backends that all this will take some time to sort out.

Can you give me simple examples of what does *not* work correctly when
you use mpl *svn* with ipython-newkernel, in either or both of the
console or gui modes, but *does* work with your guisupport version?

The problems are when matplotlib and enthought.traits/pyface/mayavi
stuff are used together. There are two types of problems:

* Multiple apps are created. The enthought stuff used to not check
for existing apps, but that has been fixed.

That one is easy.

* Event loop is started multiple times. This one is more subtle and
on some toolkits the error is not fatal. This problem shows up when
IPython is run in terminal mode and event loop integration is handled
through PyOS_InputHook. In these situations, if matplotlib or ets
start the event loop themselves, IPython will hang.

So, this can be checked with nothing but IPython and mpl. I think I may have
seen this with some combination of configurations, though not with what I
typically use.

Unfortunately, I am having trouble getting an install of both
matplotlib svn and ets svn on the same machine, so I can't reproduce
any of the failures ATM. I am trying to get things installed so I can
reproduce the problems.

At least twice in the last couple of years I have tried to get mayavi
compiled and running, without fouling up my development versions of numpy
and mpl. I never succeeded. Granted, I didn't allocate much time and
mental energy to it.

Yes, it can be a challenge, we will see how far I get...

In any case, with the help of your recent explanations, I expect we can make
mpl play by your suggested rules without sacrificing anything. Part of the
change for mpl 1.0 was to factor all of the show logic out into
backend_bases.py, leaving only the core mainloop call in the specific
backends. I hope we can keep that aspect, retaining the ability to work
with earlier ipython (0.10) and the ability for show to block or not
depending on the interactivity state.

The refactored show logic does help a lot. I am pretty sure that my
branch already does all the things you want though. If there are
things that it is missing we can definitely add them. But when you
get a chance to look at this, let us know and we can continue the
discussion.

Cheers,

Brian

···

On Tue, Sep 14, 2010 at 12:59 AM, Eric Firing <efiring@...229...> wrote:

On 09/13/2010 05:46 PM, Brian Granger wrote:

On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing<efiring@...229...> wrote:

On 09/03/2010 12:37 PM, Brian Granger wrote:

Eric

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger@...882...
ellisonbg@...149...