Updating MPlot3D to a more recent matplotlib.

Hi,

I saw that 3D plotting was dropped from matplotlib since the last time
I used it. Unfortunately, it is pretty necessary for some of the work
I am doing. Thus, I have started the process of refactoring the code
to work with recent versions of matplotlib.

Right now, it is still in very early stages and is quite flaky but I
do have some functionality. In particular, I am able to do a regular
3d plot, a wireframe plot and a scatter plot. If this interests
anyone I am making the code available via git. Instructions are
available on my website at:

http://jonathantaylor.ca/projects.shtml

Feel free to send any patches my way.

Best,
Jon.

That’s great – a number of people were very disappointed to see the functionality removed, even though it was primitive compared to a good 3D toolkit. The problem was, we could never find a core developer who was interested in taking it under his wing. Once you get this to a satisfactory point, I suggest you develop it as an mpl toolkit. That way, it will get installed with every mpl distro (the plain vanilla toolkits we ship, the complex ones like basemap are distributed separately) but without the implicit promise of full support until someone is willing to step up and offer to fully support it.

JDH

···

On Mon, Mar 2, 2009 at 10:13 PM, Jonathan Taylor <jonathan.taylor@…247…> wrote:

Hi,

I saw that 3D plotting was dropped from matplotlib since the last time

I used it. Unfortunately, it is pretty necessary for some of the work

I am doing. Thus, I have started the process of refactoring the code

to work with recent versions of matplotlib.

Right now, it is still in very early stages and is quite flaky but I

do have some functionality. In particular, I am able to do a regular

3d plot, a wireframe plot and a scatter plot. If this interests

anyone I am making the code available via git. Instructions are

available on my website at:

That sounds reasonable. Can I ask what it is that was primitive?
Having looked through the code I see that a few shortcuts were made to
minimize the amount of code written that makes it especially
susceptible to changes in the 2D code. That said, it seems like it
was comparable functionally to matlab's 3d plots, which is my goal for
it.

Best,
Jon.

P.S. I saw your talk at NIPS 2008 this year. I have used mpl for a
while now but that demo where you url.opened() yahoo finance and
plotted it with those nice dates in 2/3 lines was very nice. :wink:

···

On Tue, Mar 3, 2009 at 9:14 AM, John Hunter <jdh2358@...149...> wrote:

On Mon, Mar 2, 2009 at 10:13 PM, Jonathan Taylor > <jonathan.taylor@...247...> wrote:

Hi,

I saw that 3D plotting was dropped from matplotlib since the last time
I used it. Unfortunately, it is pretty necessary for some of the work
I am doing. Thus, I have started the process of refactoring the code
to work with recent versions of matplotlib.

Right now, it is still in very early stages and is quite flaky but I
do have some functionality. In particular, I am able to do a regular
3d plot, a wireframe plot and a scatter plot. If this interests
anyone I am making the code available via git. Instructions are
available on my website at:

That's great -- a number of people were very disappointed to see the
functionality removed, even though it was primitive compared to a good 3D
toolkit. The problem was, we could never find a core developer who was
interested in taking it under his wing. Once you get this to a satisfactory
point, I suggest you develop it as an mpl toolkit. That way, it will get
installed with every mpl distro (the plain vanilla toolkits we ship, the
complex ones like basemap are distributed separately) but without the
implicit promise of full support until someone is willing to step up and
offer to fully support it.

JDH

That sounds reasonable. Can I ask what it is that was primitive?

Having looked through the code I see that a few shortcuts were made to

minimize the amount of code written that makes it especially

susceptible to changes in the 2D code. That said, it seems like it

was comparable functionally to matlab’s 3d plots, which is my goal for

it.

Well, it is painfully slow, since it does everything in software, and there are some corner cases where the zorder clipping is broken in the presence of alpha transparency, and it doesn’t do lighting, shadows, etc… But it does do enough for basic stuff, so we would be happy if you could resurrect it cleanly enough for a toolkit.

P.S. I saw your talk at NIPS 2008 this year. I have used mpl for a

while now but that demo where you url.opened() yahoo finance and

plotted it with those nice dates in 2/3 lines was very nice. :wink:

Yep, that is a favorite example of mine :slight_smile: I’m giving a talk at SIAM on Thursday, and I think I’ll do this one again, time permitting.

JDH

···

On Tue, Mar 3, 2009 at 9:56 AM, Jonathan Taylor <jonathan.taylor@…247…> wrote:

I'd just like to add that having a *bare minimum* 3D capability in mpl
is extremely useful to me -- i.e. being to visualize 3D data as a
point cloud or a wireframe mesh. While teaching with python and doing
numerical experiments in my research it's invaluable to be able to
throw together a wholly non-publication quality 3D plot to get a quick
idea of what's going on. I would imagine that others who use mpl
professionally (and not necessarily only for public consumption) would
agree on the value of maintaining this bare minimum even if there is
no short- or medium-term expectation that it will develop into
anything more sophisticated.

Cheers,
Rob

···

On Tue, Mar 3, 2009 at 11:39 AM, John Hunter <jdh2358@...149...> wrote:

Well, it is painfully slow, since it does everything in software, and there
are some corner cases where the zorder clipping is broken in the presence of
alpha transparency, and it doesn't do lighting, shadows, etc.... But it
does do enough for basic stuff, so we would be happy if you could resurrect
it cleanly enough for a toolkit.

Hi all,

I was also a bit disappointed about the fact that 3d plotting support
was dropped. I'm happy to help out to get it going again, so here's a
patch to get surface plotting working; I'm busy with the contour plots
as well.
(We might want to do some code refactoring as well when it's functional).

Regards,
Reinier

0001-Fix-surface-plot.patch (3.98 KB)

···

On Wed, Mar 4, 2009 at 5:01 AM, Rob Clewley <rob.clewley@...149...> wrote:

On Tue, Mar 3, 2009 at 11:39 AM, John Hunter <jdh2358@...149...> wrote:

Well, it is painfully slow, since it does everything in software, and there
are some corner cases where the zorder clipping is broken in the presence of
alpha transparency, and it doesn't do lighting, shadows, etc.... But it
does do enough for basic stuff, so we would be happy if you could resurrect
it cleanly enough for a toolkit.

I'd just like to add that having a *bare minimum* 3D capability in mpl
is extremely useful to me -- i.e. being to visualize 3D data as a
point cloud or a wireframe mesh. While teaching with python and doing
numerical experiments in my research it's invaluable to be able to
throw together a wholly non-publication quality 3D plot to get a quick
idea of what's going on. I would imagine that others who use mpl
professionally (and not necessarily only for public consumption) would
agree on the value of maintaining this bare minimum even if there is
no short- or medium-term expectation that it will develop into
anything more sophisticated.

Cheers,
Rob

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

--
Reinier Heeres
Waalstraat 17
2515 XK Den Haag
The Netherlands

Tel: +31 6 10852639

Great. I applied your patch and pushed it to the web repository.

I agree, that some more serious refactoring might be good. I have
been leaving comments throughout the code with my thoughts on this.

Cheers,
Jon.

···

On Wed, Mar 4, 2009 at 4:53 AM, Reinier Heeres <reinier@...707...> wrote:

Hi all,

I was also a bit disappointed about the fact that 3d plotting support
was dropped. I'm happy to help out to get it going again, so here's a
patch to get surface plotting working; I'm busy with the contour plots
as well.
(We might want to do some code refactoring as well when it's functional).

Regards,
Reinier

On Wed, Mar 4, 2009 at 5:01 AM, Rob Clewley <rob.clewley@...149...> wrote:

On Tue, Mar 3, 2009 at 11:39 AM, John Hunter <jdh2358@...149...> wrote:

Well, it is painfully slow, since it does everything in software, and there
are some corner cases where the zorder clipping is broken in the presence of
alpha transparency, and it doesn't do lighting, shadows, etc.... But it
does do enough for basic stuff, so we would be happy if you could resurrect
it cleanly enough for a toolkit.

I'd just like to add that having a *bare minimum* 3D capability in mpl
is extremely useful to me -- i.e. being to visualize 3D data as a
point cloud or a wireframe mesh. While teaching with python and doing
numerical experiments in my research it's invaluable to be able to
throw together a wholly non-publication quality 3D plot to get a quick
idea of what's going on. I would imagine that others who use mpl
professionally (and not necessarily only for public consumption) would
agree on the value of maintaining this bare minimum even if there is
no short- or medium-term expectation that it will develop into
anything more sophisticated.

Cheers,
Rob

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

--
Reinier Heeres
Waalstraat 17
2515 XK Den Haag
The Netherlands

Tel: +31 6 10852639

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Hi,

Great. I applied your patch and pushed it to the web repository.

I agree, that some more serious refactoring might be good. I have
been leaving comments throughout the code with my thoughts on this.

John just pointed me to this thread, so I just wanted to mention that
we have 3d plots in sympy, that are pure python and use pyglet, here
are some examples and docs:

http://wiki.sympy.org/wiki/Plotting_Module

http://docs.sympy.org/modules/plotting.html

and if anyone would be interested in taking the code and use it for
some of the 3D stuff that you want to do, I would fully support it.
Ideally, if any of you would take it and maintain it, so that we don't
have to, it'd be really awesome. We would like to just concentrate on
symbolic manipulation with sympy and leave all the plotting to
matplotlib, or other packages, if needed.

Let me know if you are interested, we can help with integrating it.

Ondrej

···

On Wed, Mar 4, 2009 at 11:38 AM, Jonathan Taylor <jonathan.taylor@...247...> wrote:

wouaouh..... if I had known that sumpy had this functionality, I would have downloaded it ages ago. This is a good example of justified 'taylorisation', IMHO.
Big +1 on seing this moved from sympy to matplotlib. I am not expert at coding guis et al, but if you need reviewers/testers or doc writers, I will be happy to give a hand (even two).
best,
Johann

Ondrej Certik wrote:

···

Hi,

On Wed, Mar 4, 2009 at 11:38 AM, Jonathan Taylor > <jonathan.taylor@...247...> wrote:
  

Great. I applied your patch and pushed it to the web repository.

I agree, that some more serious refactoring might be good. I have
been leaving comments throughout the code with my thoughts on this.
    
John just pointed me to this thread, so I just wanted to mention that
we have 3d plots in sympy, that are pure python and use pyglet, here
are some examples and docs:

http://wiki.sympy.org/wiki/Plotting_Module

http://docs.sympy.org/modules/plotting.html

and if anyone would be interested in taking the code and use it for
some of the 3D stuff that you want to do, I would fully support it.
Ideally, if any of you would take it and maintain it, so that we don't
have to, it'd be really awesome. We would like to just concentrate on
symbolic manipulation with sympy and leave all the plotting to
matplotlib, or other packages, if needed.

Let me know if you are interested, we can help with integrating it.

Ondrej

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Yes, I didn't know that either. But it's not clear if I can plot
discrete data using this interface - at least the examples on the wiki
make it look that way. I'm also +1 on seeing it moved into mpl, but I
don't know if the APIs and dependencies are too dissimilar to make it
work.

···

On Thu, Mar 5, 2009 at 5:14 AM, Johann Cohen-Tanugi <cohen@...585...> wrote:

wouaouh..... if I had known that sumpy had this functionality, I would
have downloaded it ages ago. This is a good example of justified
'taylorisation', IMHO.
Big +1 on seing this moved from sympy to matplotlib. I am not expert at
coding guis et al, but if you need reviewers/testers or doc writers, I
will be happy to give a hand (even two).
best,
Johann

wouaouh..... if I had known that sumpy had this functionality, I would
have downloaded it ages ago. This is a good example of justified
'taylorisation', IMHO.
Big +1 on seing this moved from sympy to matplotlib. I am not expert at
coding guis et al, but if you need reviewers/testers or doc writers, I
will be happy to give a hand (even two).
best,
Johann

Yes, I didn't know that either. But it's not clear if I can plot
discrete data using this interface - at least the examples on the wiki

I am not sure if I understand your question, but It only plots
discrete data --- it takes some sympy expression, evaluates it on a
discrete grid and plots it. So you would just take the plotting stuff.

make it look that way. I'm also +1 on seeing it moved into mpl, but I
don't know if the APIs and dependencies are too dissimilar to make it
work.

There are no dependencies besides pyglet (e.g. it does not depend on
sympy, or if it does, it can be trivially fixed).

As to the API, just look into sympy/plotting. And play with the 3D
plots in sympy to see how it looks like and how fast/slow it is (I
think it's pretty fast). Then you will have to decide, if it's easier
for you to implement something from scratch, or take our stuff.

As I said, I would love if mpl has similar capability, so that we can
get rid of this from sympy. If you decide to go this way, I (and other
sympy developers) will be at hand to help you integrate it.

Ondrej

···

On Thu, Mar 5, 2009 at 10:02 AM, Rob Clewley <rob.clewley@...149...> wrote:

On Thu, Mar 5, 2009 at 5:14 AM, Johann Cohen-Tanugi > <cohen@...585...> wrote:

Yes, I didn't know that either. But it's not clear if I can plot
discrete data using this interface - at least the examples on the wiki

I am not sure if I understand your question, but It only plots
discrete data --- it takes some sympy expression, evaluates it on a
discrete grid and plots it. So you would just take the plotting stuff.

OK, but it wasn't clear from the example that I could plot a 3D array
of arbitrary data points. The way that you put together the demo plots
involved a symbolic function that would be called to generate the
points. Maybe you could add an example that plots some arbitrary data
that has been imported from a text file?

make it look that way. I'm also +1 on seeing it moved into mpl, but I
don't know if the APIs and dependencies are too dissimilar to make it
work.

There are no dependencies besides pyglet (e.g. it does not depend on
sympy, or if it does, it can be trivially fixed).

Well, I meant more like is there a design dependency that is
incompatible with mpl. I'll shut up now because I know zilch about
mpl's internals!

Yes, I didn't know that either. But it's not clear if I can plot
discrete data using this interface - at least the examples on the wiki

I am not sure if I understand your question, but It only plots
discrete data --- it takes some sympy expression, evaluates it on a
discrete grid and plots it. So you would just take the plotting stuff.

OK, but it wasn't clear from the example that I could plot a 3D array
of arbitrary data points. The way that you put together the demo plots

As I understand it, it plots triangles and/or wireframe in the end.
Currently I think our plotting mainly works with surfaces. How can you
plot a 3D array of arbitrary data points? You need to convert it to
some triangles first, e.g. do you want to plot contours (isosurfaces)?
Or do you want to cut a plane in your 3D data points and plot that
plane?

involved a symbolic function that would be called to generate the
points. Maybe you could add an example that plots some arbitrary data
that has been imported from a text file?

make it look that way. I'm also +1 on seeing it moved into mpl, but I
don't know if the APIs and dependencies are too dissimilar to make it
work.

There are no dependencies besides pyglet (e.g. it does not depend on
sympy, or if it does, it can be trivially fixed).

Well, I meant more like is there a design dependency that is
incompatible with mpl. I'll shut up now because I know zilch about
mpl's internals!

The best thing is to just look into our sources, it's pretty well documented.

Ondrej

···

On Thu, Mar 5, 2009 at 10:15 AM, Rob Clewley <rob.clewley@...149...> wrote:

OK, but it wasn't clear from the example that I could plot a 3D array
of arbitrary data points. The way that you put together the demo plots

As I understand it, it plots triangles and/or wireframe in the end.
Currently I think our plotting mainly works with surfaces. How can you
plot a 3D array of arbitrary data points?

E.g., by plotting a dot at the coordinate (x,y,z).

You need to convert it to
some triangles first, e.g. do you want to plot contours (isosurfaces)?
Or do you want to cut a plane in your 3D data points and plot that
plane?

If I have a set of scalar sample data on a rectangular 2D mesh that I
want to plot in the 3D I'd want a simple wireframe rectangular surface
plot. Can it do that?

-Rob

···

On Thu, Mar 5, 2009 at 10:39 AM, Ondrej Certik <ondrej@...583...> wrote:

OK, but it wasn't clear from the example that I could plot a 3D array
of arbitrary data points. The way that you put together the demo plots

As I understand it, it plots triangles and/or wireframe in the end.
Currently I think our plotting mainly works with surfaces. How can you
plot a 3D array of arbitrary data points?

E.g., by plotting a dot at the coordinate (x,y,z).

I guess more like a small ball in opengl. I think this can be done
directly in pyglet.

You need to convert it to
some triangles first, e.g. do you want to plot contours (isosurfaces)?
Or do you want to cut a plane in your 3D data points and plot that
plane?

If I have a set of scalar sample data on a rectangular 2D mesh that I
want to plot in the 3D I'd want a simple wireframe rectangular surface
plot. Can it do that?

Yes, that can be done. I can post a simple example, but right now I am
urgently working on getting mayavi2 working in the offline mode in the
Sage notebook for my presentation on Friday, so I'll get back to this
later.

Ondrej

···

On Thu, Mar 5, 2009 at 11:44 AM, Rob Clewley <rob.clewley@...149...> wrote:

On Thu, Mar 5, 2009 at 10:39 AM, Ondrej Certik <ondrej@...583...> wrote:

My experience from trying to design a simple API to do simple 3D plotting
of data stored as numpy array is that everybody has his own usecase (that
he often believes is universal and obvious), and that designing an API to
make a particular usecase simple is easy, but keeping reasonnably
versatile and powerful API while making it easy to use for all the
'simple' usecases is terribly hard.

That said, Mayavi2 is incredibly powerful and with time (thanks to
feedback from users) we are getting better at having a simple and
powerful API, and a documentation that tries to 'solve your problem'.

I know Mayavi2 has its problems (mainly difficult to instal because of
historical reasons, and big dependencies), but for non trivial 3D
plotting problems, especially with large datasets, keep it in mind. (It
also works well for trivial problems, it is just that if it is not
packaged, I understand that the hassle of installing it may be higher
than the gain of using it).

By the way, I am happy to take feedback on any problems with Mayavi2 (I
am not pretending I will solve them :>). I believe that we have made huge
progress in usability recently, and that it is no longer a monster hard
to master and useless for simple problems, but I may be mistaken.

Cheers,

Gaël

···

On Thu, Mar 05, 2009 at 11:44:16AM -0500, Rob Clewley wrote:

If I have a set of scalar sample data on a rectangular 2D mesh that I
want to plot in the 3D I'd want a simple wireframe rectangular surface
plot. Can it do that?

Hi,

I think that except for contourf3D everything is functioning fine
again, so I'm thinking about packaging/distributing it. (I will try to
fix contourf3D as well when I can find some time)

2009/3/3 John Hunter <jdh2358@...149...>:

Hi,

I saw that 3D plotting was dropped from matplotlib since the last time
I used it. Unfortunately, it is pretty necessary for some of the work
I am doing. Thus, I have started the process of refactoring the code
to work with recent versions of matplotlib.

Right now, it is still in very early stages and is quite flaky but I
do have some functionality. In particular, I am able to do a regular
3d plot, a wireframe plot and a scatter plot. If this interests
anyone I am making the code available via git. Instructions are
available on my website at:

That's great -- a number of people were very disappointed to see the
functionality removed, even though it was primitive compared to a good 3D
toolkit. The problem was, we could never find a core developer who was
interested in taking it under his wing. Once you get this to a satisfactory
point, I suggest you develop it as an mpl toolkit. That way, it will get
installed with every mpl distro (the plain vanilla toolkits we ship, the
complex ones like basemap are distributed separately) but without the
implicit promise of full support until someone is willing to step up and
offer to fully support it.

JDH

That sounds good. I assume you mean adding it to lib/mpl_toolkits in
the matplotlib source? It would be great to do this soon.

I also tried to create a separate mpl_toolkit installer with
setuptools, but can't get that to work properly:

In [2]: import mpl_toolkits.mplot3d
/usr/lib/python2.5/site-packages/mpl_toolkits/__init__.py:2:
UserWarning: Module mpl_toolkits was already imported from
/usr/lib/python2.5/site-packages/mpl_toolkits/__init__.pyc, but
/usr/lib/python2.5/site-packages/mplot3d-0.1-py2.5.egg is being added
to sys.path
  __import__('pkg_resources').declare_namespace(__name__)

This can probably be fixed, but I would prefer things to live in the
matplotlib tree, and I'm willing to volunteer to support it.

Cheers,

···

On Mon, Mar 2, 2009 at 10:13 PM, Jonathan Taylor > <jonathan.taylor@...247...> wrote:

--
Reinier Heeres
Waalstraat 17
2515 XK Den Haag
The Netherlands

Tel: +31 6 10852639

Since this is pure python, my plan was to put it in the mpl source tree in mpl_toolkits so that it gets included in all mpl installs, rather than distribute it separately. This might put some additional burden on you since we still use svn, but we have some notes on developing mpl via git with a git/svn gateway

http://matplotlib.sourceforge.net/devel/coding_guide.html#id3

Have you done any work on bringing it up to compatibility with svn HEAD? I’d like at least the basic demos to work with svn HEAD before I do the import. I think the major missing piece there is the missing TextWithDash.

Thanks,
JDH

···

On Mon, Mar 30, 2009 at 2:13 AM, Reinier Heeres <reinier@…707…> wrote:

That sounds good. I assume you mean adding it to lib/mpl_toolkits in

the matplotlib source? It would be great to do this soon.

I also tried to create a separate mpl_toolkit installer with

setuptools, but can’t get that to work properly:

In [2]: import mpl_toolkits.mplot3d

/usr/lib/python2.5/site-packages/mpl_toolkits/init.py:2:

UserWarning: Module mpl_toolkits was already imported from

/usr/lib/python2.5/site-packages/mpl_toolkits/init.pyc, but

/usr/lib/python2.5/site-packages/mplot3d-0.1-py2.5.egg is being added

to sys.path

import(‘pkg_resources’).declare_namespace(name)

This can probably be fixed, but I would prefer things to live in the

matplotlib tree, and I’m willing to volunteer to support it.