Source of inaccuracies in the matplotlib library

Konrad_Bartkowski · April 11, 2011, 9:16am

Ok, forwarding it to the matplotlib-devel list.

Best wishes,

Konrad (on behalf of our workgroup)

···

Subject:
Source of inaccuracies in the matplotlib library
Date:
Fri, 8 Apr 2011 18:12:47 +0200
From:
Bartkowski, Konrad
To:

,
,
,
,
CC:
Bartkowski, Konrad , , Matthias Bolten , Grotendorst, Johannes , Steffen, Bernhard
<k.bartkowski@…954…>dd55@…143…<dd55@…143…>mdroe@…31…<mdroe@…31…>efiring@…229…<efiring@…229…>jdhunter@…5…<jdhunter@…5…>jdh2358@…149…<jdh2358@…149…>
<k.bartkowski@…954…>
elmod@…955…
<elmod@…955…>
<bolten@…956…>
<j.grotendorst@…954…>
<b.steffen@…954…>


Dear Matplotlib developers,
I am writing about the matplotlib library with the mpl_toolkits. First of all let me emphasize how great software it is. Recently, in one of our projects we were rendering big surfaces and encountered the following problem:
It's not a bug (which all in all is a natural and unavoidable ingredient of the software, and especially in such a big and complex system like matplotlib would be fully natural), since the software does exactly the projection mathematics that it is expected to do, but a source of the inaccuracies, which is especially visible in the critical examples. For the profit of the Python community we are sending You a proposition of a modification of the surface plotting rendering system, in case You find it interesting enough to include in the consecutive version of the library. In the source code from the attachment we redesigned a little bit the computation process – since the computations are especially sensible to numerical errors, that are for example amplified while norming or processing the quaterions in the various stages (for example division over coordinate in the perspective projection). Therefore the computational focus can be shifted from the Polygon collection to the polygons itself. In the example from the above forum or the slightly modified one, one can observe a big difference in the numerical precision while the speed of the computations does not decrease (at least visibly). While instead of the surfaces from the forum, the following surfaces are rendered:
u = np.linspace(0, 2 * np.pi, 100)
v = np.linspace(0, np.pi, 100)
x = 10 * np.outer(np.cos(u), np.sin(v))
y = 10 * np.outer(np.sin(u), np.sin(v))
z = 10 * np.outer(np.ones(np.size(u)), np.cos(v))
ax.plot_surface(x, y, z, rstride=8, cstride=8, color='y', alpha=0.5)
shiftX=28
shiftY=28
X,Y=np.meshgrid(range(-20+shiftX,20+shiftX),range(-20+shiftY,20+shiftY))
Z=np.ones((X.shape[0], Y.shape[1]))
ax.plot_surface(X, Y, Z, color='r', rstride=10, cstride=10, alpha=1.0)
the issue is visible for example at the azimuth=40 , elevation=70 – with those parameters the mentioned case is visible on the red surface, while with elevation=68 not. Moreover, now also the stride is big (in the new approach the influence of increasing stride on the numerical precision grows).
So again let me use this opportunity to thank You for empowering the Python community worldwide in a great, powerful scientific visualization tool.
Best wishes,
Konrad Bartkowski

http://www.mail-archive.com/matplotlib-devel@lists.sourceforge.net/msg06869.html

Forschungszentrum Juelich GmbH

52425 Juelich

Sitz der Gesellschaft: Juelich

Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498

Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher

Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),

Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,

Prof. Dr. Sebastian M. Schmidt

Besuchen Sie uns auf unserem neuen Webauftritt unter www.fz-juelich.de

Benjamin_Root1 · April 14, 2011, 11:13pm

Konrad,

Thank you for this contribution. It seems that I am currently the de facto maintainer of the mplot3d code, but I have been focused on my PhD for the past month. I will take a look at the code in more detail over the weekend.

Thanks!
Ben Root

···

On Mon, Apr 11, 2011 at 4:16 AM, Konrad Bartkowski <k.bartkowski@…954…> wrote:

Ok, forwarding it to the matplotlib-devel list.

Best wishes,

Konrad (on behalf of our workgroup)

-------- Original Message --------
Subject:
Source of inaccuracies in the matplotlib library
Date:
Fri, 8 Apr 2011 18:12:47 +0200
From:
Bartkowski, Konrad
<k.bartkowski@…954…>
To:
dd55@…143…
<dd55@…143…>,
mdroe@…31…
<mdroe@…31…>,
efiring@…229…
<efiring@…930…>,
jdhunter@…5…
<jdhunter@…5…>,
jdh2358@…149…
<jdh2358@…714…>
CC:
Bartkowski, Konrad
<k.bartkowski@…954…>,
elmod@…955…
<elmod@…955…>, Matthias Bolten
<bolten@…956…>, Grotendorst, Johannes
<j.grotendorst@…954…>, Steffen, Bernhard
<b.steffen@…954…>


Dear Matplotlib developers,
I am writing about the matplotlib library with the mpl_toolkits. First of all let me emphasize how great software it is. Recently, in one of our projects we were rendering big surfaces and encountered the following problem:
[http://www.mail-archive.com/matplotlib-devel@lists.sourceforge.net/msg06869.html](http://www.mail-archive.com/matplotlib-devel@...712...et/msg06869.html)

It's not a bug (which all in all is a natural and unavoidable ingredient of the software, and especially in such a big and complex system like matplotlib would be fully natural), since the software does exactly the projection mathematics that it is expected to do, but a source of the inaccuracies, which is especially visible in the critical examples. For the profit of the Python community we are sending You a proposition of a modification of the surface plotting rendering system, in case You find it interesting enough to include in the consecutive version of the library. In the source code from the attachment we redesigned a little bit the computation process – since the computations are especially sensible to numerical errors, that are for example amplified while norming or processing the quaterions in the various stages (for example division over coordinate in the perspective projection). Therefore the computational focus can be shifted from the Polygon collection to the polygons itself. In the example from the above forum or the slightly modified one, one can observe a big difference in the numerical precision while the speed of the computations does not decrease (at least visibly). While instead of the surfaces from the forum, the following surfaces are rendered:
u = np.linspace(0, 2 * np.pi, 100)
v = np.linspace(0, np.pi, 100)
x = 10 * np.outer(np.cos(u), np.sin(v))
y = 10 * np.outer(np.sin(u), np.sin(v))
z = 10 * np.outer(np.ones(np.size(u)), np.cos(v))
ax.plot_surface(x, y, z, rstride=8, cstride=8, color='y', alpha=0.5)
shiftX=28
shiftY=28
X,Y=np.meshgrid(range(-20+shiftX,20+shiftX),range(-20+shiftY,20+shiftY))
Z=np.ones((X.shape[0], Y.shape[1]))
ax.plot_surface(X, Y, Z, color='r', rstride=10, cstride=10, alpha=1.0)
the issue is visible for example at the azimuth=40 , elevation=70 – with those parameters the mentioned case is visible on the red surface, while with elevation=68 not. Moreover, now also the stride is big (in the new approach the influence of increasing stride on the numerical precision grows).
So again let me use this opportunity to thank You for empowering the Python community worldwide in a great, powerful scientific visualization tool.
Best wishes,
Konrad Bartkowski

Benjamin_Root1 · April 15, 2011, 8:42pm

Konrad,

Ok, I have examined the attached file, and I see what you have done. First, the shading issue has long since been resolved and is in the development branch (but probably won’t be backported to v1.0.x because the changes were extensive).

Second, the zorder sorting issue is a PITA to say the least. Your approach, however, only pushes the problem down to smaller parts, and still doesn’t address the same problems with PatchCollection objects. That being said, I would still be inclined to have the problem solved for surfaces and leave patches to be buggy, except that this approach completely breaks the API. plot_surface returns a Poly3DCollection object, not a list of Poly3DCollection objects.

The Collections object is a double-edge sword. It allows for easy manipulation of many artist objects, but ultimately the Collection object must report a single z-order value to represent the z-order for plotting all of the artist elements.

There might be a possible work-around, though. Maybe (and I am just speculating here) if we can get the core part of matplotlib to specially treat 3d collection objects in such a way that allows the collection to return provide elements and z-order pairs. It is either that, or we finally try to get OpenGL working again in matplotlib and allow ourselves to specify coordinates in 3-D.

Ben Root

···

On Mon, Apr 11, 2011 at 4:16 AM, Konrad Bartkowski <k.bartkowski@…954…> wrote:

Ok, forwarding it to the matplotlib-devel list.

Best wishes,

Konrad (on behalf of our workgroup)

-------- Original Message --------
Subject:
Source of inaccuracies in the matplotlib library
Date:
Fri, 8 Apr 2011 18:12:47 +0200
From:
Bartkowski, Konrad
<k.bartkowski@…954…>
To:
dd55@…143…
<dd55@…143…>,
mdroe@…31…
<mdroe@…31…>,
efiring@…229…
<efiring@…930…>,
jdhunter@…5…
<jdhunter@…5…>,
jdh2358@…149…
<jdh2358@…714…>
CC:
Bartkowski, Konrad
<k.bartkowski@…954…>,
elmod@…955…
<elmod@…955…>, Matthias Bolten
<bolten@…956…>, Grotendorst, Johannes
<j.grotendorst@…954…>, Steffen, Bernhard
<b.steffen@…954…>


Dear Matplotlib developers,
I am writing about the matplotlib library with the mpl_toolkits. First of all let me emphasize how great software it is. Recently, in one of our projects we were rendering big surfaces and encountered the following problem:
[http://www.mail-archive.com/matplotlib-devel@lists.sourceforge.net/msg06869.html](http://www.mail-archive.com/matplotlib-devel@...712...et/msg06869.html)

It's not a bug (which all in all is a natural and unavoidable ingredient of the software, and especially in such a big and complex system like matplotlib would be fully natural), since the software does exactly the projection mathematics that it is expected to do, but a source of the inaccuracies, which is especially visible in the critical examples. For the profit of the Python community we are sending You a proposition of a modification of the surface plotting rendering system, in case You find it interesting enough to include in the consecutive version of the library. In the source code from the attachment we redesigned a little bit the computation process – since the computations are especially sensible to numerical errors, that are for example amplified while norming or processing the quaterions in the various stages (for example division over coordinate in the perspective projection). Therefore the computational focus can be shifted from the Polygon collection to the polygons itself. In the example from the above forum or the slightly modified one, one can observe a big difference in the numerical precision while the speed of the computations does not decrease (at least visibly). While instead of the surfaces from the forum, the following surfaces are rendered:
u = np.linspace(0, 2 * np.pi, 100)
v = np.linspace(0, np.pi, 100)
x = 10 * np.outer(np.cos(u), np.sin(v))
y = 10 * np.outer(np.sin(u), np.sin(v))
z = 10 * np.outer(np.ones(np.size(u)), np.cos(v))
ax.plot_surface(x, y, z, rstride=8, cstride=8, color='y', alpha=0.5)
shiftX=28
shiftY=28
X,Y=np.meshgrid(range(-20+shiftX,20+shiftX),range(-20+shiftY,20+shiftY))
Z=np.ones((X.shape[0], Y.shape[1]))
ax.plot_surface(X, Y, Z, color='r', rstride=10, cstride=10, alpha=1.0)
the issue is visible for example at the azimuth=40 , elevation=70 – with those parameters the mentioned case is visible on the red surface, while with elevation=68 not. Moreover, now also the stride is big (in the new approach the influence of increasing stride on the numerical precision grows).
So again let me use this opportunity to thank You for empowering the Python community worldwide in a great, powerful scientific visualization tool.
Best wishes,
Konrad Bartkowski

_John_Hunter · April 15, 2011, 8:54pm

It should be fairly easy to get a collections object to support multiple z-orders within the collection. Across artists, damn near impossible. I don’t think you need to provide elements and z-order pairs per-se. The typical way a property is specified for a collection if you want it to vary over the elements of the collection is that the property is a sequence, and the property is accessed as prop[i%N] where i is the element number and N is the length of the property vector. So if we make zorder a len(elements) sequence of z-orders, we can order the collection by the zorder at draw time. Presumably external code would modify the zorder of the collection before each draw depending on the view.

···

On Fri, Apr 15, 2011 at 3:42 PM, Benjamin Root <ben.root@…553…> wrote:

There might be a possible work-around, though. Maybe (and I am just speculating here) if we can get the core part of matplotlib to specially treat 3d collection objects in such a way that allows the collection to return provide elements and z-order pairs. It is either that, or we finally try to get OpenGL working again in matplotlib and allow ourselves to specify coordinates in 3-D.

Benjamin_Root1 · April 15, 2011, 9:02pm

Within a collection, this is already done correctly (or at least, as well as one can except with a 2D rendering engine). It is when you have multiple artists that exists over parts of the z-order “axis” that there are problems. Essentially, if the 3-D bounding boxes overlap, then trouble ensues.

Also, that speculation I had wouldn’t work either, as the problem still would exist for intersecting patches. No, what we really need is a 3D rendering engine, and a logical separation between z-order for sorting/layering and z-coordinates.

Ben Root

···

On Fri, Apr 15, 2011 at 3:54 PM, John Hunter <jdh2358@…552…149…> wrote:

On Fri, Apr 15, 2011 at 3:42 PM, Benjamin Root <ben.root@…553…> wrote:

There might be a possible work-around, though. Maybe (and I am just speculating here) if we can get the core part of matplotlib to specially treat 3d collection objects in such a way that allows the collection to return provide elements and z-order pairs. It is either that, or we finally try to get OpenGL working again in matplotlib and allow ourselves to specify coordinates in 3-D.

It should be fairly easy to get a collections object to support multiple z-orders within the collection. Across artists, damn near impossible. I don’t think you need to provide elements and z-order pairs per-se. The typical way a property is specified for a collection if you want it to vary over the elements of the collection is that the property is a sequence, and the property is accessed as prop[i%N] where i is the element number and N is the length of the property vector. So if we make zorder a len(elements) sequence of z-orders, we can order the collection by the zorder at draw time. Presumably external code would modify the zorder of the collection before each draw depending on the view.

Konrad_Bartkowski · April 21, 2011, 9:52am

Dear Ben, dear John,

        Thank You for the emails. Well, we have modified only the part
influencing the modules, that we need or will need in our projects. I
admit that apart from the deceptions connected with human perception
(for example
CiteSeerX ) still
the inaccuracy remains. Nevertheless all the surfaces we have rendered
were unproblematic. Also the examples from the forum rendered correctly
even for big stride. If it is unsatisfactory one can control the
precision with the stride size. The modification was written under the
assumption not to modify too much in the existing system. While writing
the code we were naturally keeping in mind the ratio of accuracy to the
computational effort.
        Since in our projects we didn't need the return value from the
plot_surface we return the obtained list for the efficiency reasons. But
if the API should remain unchanged, then one can generate also the
previous collection and return it instead of the list, while the list is
added.
        I admit, that I didn't consider the changes You mention in the
consecutive e-mails, since it would mean redesigning Your code. From the
same reason I also cannot comment them – I simply don't know the
internal structure of Your code sufficienty. Change on the level of
Artists would probably imply a lot of other modifications in many, many
modules.

···

------
Mit freundlichen Grüssen / With friendly greetings,

Konrad Bartkowski

On 04/15/11 23:02, Benjamin Root wrote:

On Fri, Apr 15, 2011 at 3:54 PM, John Hunter <jdh2358@...149... > <mailto:jdh2358@…149…>> wrote:

    On Fri, Apr 15, 2011 at 3:42 PM, Benjamin Root <ben.root@...553... > <mailto:ben.root@…553…>> wrote:

        There might be a possible work-around, though. Maybe (and I am
        just speculating here) if we can get the core part of matplotlib
        to specially treat 3d collection objects in such a way that
        allows the collection to return provide elements and z-order
        pairs. It is either that, or we finally try to get OpenGL
        working again in matplotlib and allow ourselves to specify
        coordinates in 3-D.

    It should be fairly easy to get a collections object to support
    multiple z-orders *within* the collection. Across artists, damn
    near impossible. I don't think you need to provide elements and
    z-order pairs per-se. The typical way a property is specified for a
    collection if you want it to vary over the elements of the
    collection is that the property is a sequence, and the property is
    accessed as prop[i%N] where i is the element number and N is the
    length of the property vector. So if we make zorder a len(elements)
    sequence of z-orders, we can order the collection by the zorder at
    draw time. Presumably external code would modify the zorder of the
    collection before each draw depending on the view.

Within a collection, this is already done correctly (or at least, as
well as one can except with a 2D rendering engine). It is when you have
multiple artists that exists over parts of the z-order "axis" that there
are problems. Essentially, if the 3-D bounding boxes overlap, then
trouble ensues.

Also, that speculation I had wouldn't work either, as the problem still
would exist for intersecting patches. No, what we really need is a 3D
rendering engine, and a logical separation between z-order for
sorting/layering and z-coordinates.

Ben Root

------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------

Besuchen Sie uns auf unserem neuen Webauftritt unter www.fz-juelich.de

Benjamin_Root1 · April 21, 2011, 2:52pm

Dear Ben, dear John,
   Thank You for the emails. Well, we have modified only the part
influencing the modules, that we need or will need in our projects. I
admit that apart from the deceptions connected with human perception
(for example
http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.151.2303 ) still
the inaccuracy remains. Nevertheless all the surfaces we have rendered
were unproblematic. Also the examples from the forum rendered correctly
even for big stride. If it is unsatisfactory one can control the
precision with the stride size.
Just to make sure, you never really described exactly what “numerical inaccuracies” you were talking about. From the example you posted, I am assuming that it is the overlapping/z-sorting issue. If it is something else, then I am going to need to see some “before / after” images showing the difference you are talking about. If it is the overlapping issue, then the strides are only incidental to the problem. The problem exists for any 3d object, with or without strides.
The problem is one of dimensionality. While mplot3d does correctly calculate the 3d coordinates of each vertex in the perspective coordinate system, only one value for the depth axis can be reported for each artist. If the artist has many vertexes with different depth values, then a single value will never be sufficient. It is this reduction of the depth coordinates from many to one that is the source of the problem.
What your solution does is to not reduce the depth coordinates as much. But it still does a reduction, therefore not solving the problem. The only way to properly solve this is with a 3D rendering engine (like OpenGL).

The modification was written under the
assumption not to modify too much in the existing system. While writing
the code we were naturally keeping in mind the ratio of accuracy to the
computational effort.

   Since in our projects we didn't need the return value from the
plot_surface we return the obtained list for the efficiency reasons. But
if the API should remain unchanged, then one can generate also the

previous collection and return it instead of the list, while the list is
added.

This would cause many inconsistencies within the code. For example, if a user wants to remove the collection from the axes, it won’t work because the collection doesn’t exist in the axes object. Also, having a collection object unconnected to any axis might cause unexpected behavior or could even throw exceptions.

   I admit, that I didn't consider the changes You mention in the
consecutive e-mails, since it would mean redesigning Your code. From the
same reason I also cannot comment them – I simply don’t know the

internal structure of Your code sufficienty. Change on the level of
Artists would probably imply a lot of other modifications in many, many
modules.

That’s ok. It was actually this problem that pushed me to learn matplotlib to the point that I could become a developer. However, it took me another 6 months before I could understand the problem well enough to realize the source of the issue and what was needed to solve it properly. If this patch works well for your purposes, that’s great, and I hope it continues to serve you well. However, for a broader audience, it introduces too many potential problems for us.

I hope you continue to use matplotlib and find it useful. And feel free to continue commenting on the mplot3d module and how it can be improved!

Cheers,
Ben Root

···

On Thursday, April 21, 2011, Konrad Bartkowski <k.bartkowski@…954…> wrote: