[Matplotlib-users] Problems upgrading to mpl 0.87.4

_Chris.Barker · August 15, 2006, 6:05pm

I moved this to the devel list -- that seemed more appropriate.

Stefan van der Walt wrote:

Another (or additional) option is for both MPL and wx to support
the new array interface protocol in numpy.

Travis posted a patch to PIL for support a while back, I don't know
if it's going to get applied or not, but it's worth looking at.

Looks like it has been added already.

Very cool. Now we just need MPL and wx to support it...

In the meantime, this note from Robin Dunn on the wxpython-users list:

wx.Image.SetData makes a copy and gives ownership of the copy to the
image. This allows it to work even when the source buffer doesn't
live as long as the image does.

This is probably how the original wx back-end worked.

There is also the
wx.Image.SetDataBuffer method, which will have the wxImage use the
buffer passed in without making a copy, but if the buffer doesn't
live as long as the image does it will likely cause a crash.

This would probably be the easiest way to get top performance at the moment. If we can make the Agg data a buffer, and somehow do the reference counting right so it doesn't get deleted while the wxImage is still around (that may not be hard -- the wxImage has to be converted to a wxBitmap to be drawn anyway)

Another issue is the data layout of the Agg buffer: is it RGB or RGBA (or something else?) wxImage requires 24bit RGB. It can support an alpha channel, but as far as I can tell, the alpha data is stored in a separate array, rather than as an RGBA array.

I'm poking into the code a bit now, and see:

def _py_convert_agg_to_wx_image(agg, bbox):

image = wx.EmptyImage(int(agg.width), int(agg.height))
image.SetData(agg.tostring_rgb())

We might be able to just change that to:
image.SetDataBuffer(agg.tostring_rgb())

Does the agg.tostring_rgb() call make a copy?

If so, then maybe a agg.Getbuffer() method is in order.

Also, if it does, then that string will get garbage collected right away, and delete the memory buffer. We'd need to keep that around somehow. maybe a subclass of wx.Image that holds a reference to the string. Then they'll both get deleted when the wx.Image is deleted..

I also took a look at:

def _clipped_image_as_bitmap(

This is way too ugly!

1) maybe it would be a good idea to have a:

agg.subImageToString_rgb(x,y,w,h)

This must be used in all the back-ends

2) If you do need to do the sub-sampling with wx code, I think you could do:

def _clipped_image_as_bitmap(image, bbox):
     """
     Convert the region of a wx.Image described by bbox to a wx.Bitmap.
     """
     l, b, width, height = bbox.get_bounds()
     r = l + width
     t = b + height

subImage = image.GetSubImage( wx.Rect( (l,b),(width,height) ) )
destBmp = wx.BitmapFromImage(subImage)

return destBmp

Much simpler (and probably faster)

3) Another option would be to use numpy arrays to pass this kind of data around. We could then use numpy to sub-sample the image. MPL depends on numpy anyway, so it's not an added dependency. That may have to wait until the 'grand unification" is complete, and we can drop support for Numeric and numarray.

Sorry I don't have time to build and test this code right now.

-Chris

···

On Mon, Aug 14, 2006 at 11:12:29AM -0700, Christopher Barker wrote:

--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

_Chris.Barker · August 17, 2006, 11:19pm

Hi all,

I seem to be talking to myself here, but if someone (including myself) wants to pick this up in the future, it'll be good to have it in the archives.

Christopher Barker wrote:

There is also the
wx.Image.SetDataBuffer method, which will have the wxImage use the
buffer passed in without making a copy, but if the buffer doesn't
live as long as the image does it will likely cause a crash.

This would probably be the easiest way to get top performance at the moment. If we can make the Agg data a buffer, and somehow do the reference counting right so it doesn't get deleted while the wxImage is still around (that may not be hard -- the wxImage has to be converted to a wxBitmap to be drawn anyway)

I've been suing this for another use, and Robin just added an improved version of new wx.Image factory function I wrote for just this purpose:

def ImageFromBuffer(width, height, dataBuffer, alphaBuffer=None):
     """
     Creates a `wx.Image` from the data in dataBuffer. The dataBuffer
     parameter must be a Python object that implements the buffer
     interface, such as a string, array, etc. The dataBuffer object is
     expected to contain a series of RGB bytes and be width*height*3
     bytes long. A buffer object can optionally be supplied for the
     image's alpha channel data, and it is expected to be width*height
     bytes long.

     A reference to the data and alpha buffer objects are kept with the
     wx.Image, so that they won't get deleted until after the wx.Image
     is deleted. However please be aware that it is not guaranteed that
     an object won't move its memory buffer to a new location when it
     needs to resize its contents. If that happens then the wx.Image
     will end up referring to an invalid memory location and could cause
     the application to crash. Therefore care should be taken to not
     manipulate the objects used for the data and alpha buffers in a
     way that would cause them to change size.
     """
     image = wx.EmptyImage(width, height)
     image.SetDataBuffer(dataBuffer)
     if alphaBuffer is not None:
         image.SetAlphaBuffer(alphaBuffer)
     image._buffer = dataBuffer
     image._alpha = alphaBuffer
     return image

Does the aggDrawer already implement a Python buffer object? If so, using this would be easy. If not, then it probably should, unless we go straight to the numpy array protocol.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

_Darren_Dale · August 18, 2006, 12:15am

I think all the high rollers are at SciPy 2006.

···

On Thursday 17 August 2006 7:19 pm, Christopher Barker wrote:

I seem to be talking to myself here

_Charlie_Moad · August 18, 2006, 12:46am

I haven't had time to thoroughly absorb all your post. From skimming
it looks like we could do a similar approach as the qtagg blitting.
If you look at the else clause in the FigureCanvasQTAgg.paintEvent
method, you'll see we added a to_string method to the BufferRegion
object. Using this string buffer we create a QImage and blit it. It
includes the alpha channel. I'll paste the block of interest to save
you a little digging.

            bbox = self.replot
            w, h = int(bbox.width()), int(bbox.height())
            l, t = bbox.ll().x().get(), bbox.ur().y().get()
            reg = self.copy_from_bbox(bbox)
            stringBuffer = reg.to_string()
            qImage = qt.QImage(stringBuffer, w, h, 32, None, 0,
qt.QImage.IgnoreEndian)
            self.pixmap.convertFromImage(qImage, qt.QPixmap.Color)
            p.drawPixmap(qt.QPoint(l, self.renderer.height-t), self.pixmap)

- Charlie

···

On 8/17/06, Christopher Barker <Chris.Barker@...236...> wrote:

Hi all,

I seem to be talking to myself here, but if someone (including myself)
wants to pick this up in the future, it'll be good to have it in the
archives.

Christopher Barker wrote:
>> There is also the
>> wx.Image.SetDataBuffer method, which will have the wxImage use the
>> buffer passed in without making a copy, but if the buffer doesn't
>> live as long as the image does it will likely cause a crash.
>
> This would probably be the easiest way to get top performance at the
> moment. If we can make the Agg data a buffer, and somehow do the
> reference counting right so it doesn't get deleted while the wxImage is
> still around (that may not be hard -- the wxImage has to be converted to
> a wxBitmap to be drawn anyway)

I've been suing this for another use, and Robin just added an improved
version of new wx.Image factory function I wrote for just this purpose:

def ImageFromBuffer(width, height, dataBuffer, alphaBuffer=None):
     """
     Creates a `wx.Image` from the data in dataBuffer. The dataBuffer
     parameter must be a Python object that implements the buffer
     interface, such as a string, array, etc. The dataBuffer object is
     expected to contain a series of RGB bytes and be width*height*3
     bytes long. A buffer object can optionally be supplied for the
     image's alpha channel data, and it is expected to be width*height
     bytes long.

     A reference to the data and alpha buffer objects are kept with the
     wx.Image, so that they won't get deleted until after the wx.Image
     is deleted. However please be aware that it is not guaranteed that
     an object won't move its memory buffer to a new location when it
     needs to resize its contents. If that happens then the wx.Image
     will end up referring to an invalid memory location and could cause
     the application to crash. Therefore care should be taken to not
     manipulate the objects used for the data and alpha buffers in a
     way that would cause them to change size.
     """
     image = wx.EmptyImage(width, height)
     image.SetDataBuffer(dataBuffer)
     if alphaBuffer is not None:
         image.SetAlphaBuffer(alphaBuffer)
     image._buffer = dataBuffer
     image._alpha = alphaBuffer
     return image

Does the aggDrawer already implement a Python buffer object? If so,
using this would be easy. If not, then it probably should, unless we go
straight to the numpy array protocol.

_Chris.Barker · August 18, 2006, 5:16am

Darren Dale wrote:

I think all the high rollers are at SciPy 2006.

Oh right. This is the third year in a row that I have ALMOST gone myself. Maybe next year.

But what's the deal? Aren't they all hooked up to wifi and checking email constantly?

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

_Chris.Barker · August 18, 2006, 5:20am

Charlie Moad wrote:

I'll paste the block of interest to save
you a little digging.

           bbox = self.replot
           w, h = int(bbox.width()), int(bbox.height())
           l, t = bbox.ll().x().get(), bbox.ur().y().get()
           reg = self.copy_from_bbox(bbox)

So this is one copy.

stringBuffer = reg.to_string()

Is this another? or does to_string not copy?

qImage = qt.QImage(stringBuffer, w, h, 32, None, 0,
qt.QImage.IgnoreEndian)

Here we have difference from wx -- AFAICT, a wx.Image doesn't do rgba32, it does RGB24, with a separate 8bit alpha channel. if to_string copies anyway, then that wouldn't make much difference.

But do we really need alpha here anyway?

self.pixmap.convertFromImage(qImage, qt.QPixmap.Color)
p.drawPixmap(qt.QPoint(l, self.renderer.height-t), self.pixmap)

And this looks like wx.

I wish I had more time to work on this, but I'm in a crunch that I'll be in for while....

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...