[Fwd: [wxPython-users] Re: using wxImage in C++ python extension]

Development matplotlib-devel
1

As a probably final installment in the thread about optimizing the wx back-end, here is a post from the wxPython list, in which someone posted SWIG code for making a PyBuffer from his data set, then using it to create a wx.Image without copying. A similar approach could be used for the wxAgg back-end.

-Chris

···

-------- Original Message --------
Subject: [wxPython-users] Re: using wxImage in C++ python extension
Date: Fri, 18 Aug 2006 16:48:08 +0000 (UTC)
From: Andrew Murray <andrew.murray@...447...>
Reply-To: wxPython-users@...73...
To: wxpython-users@...73...
References: <20060815150714.19CE.JCARLSON@...244...> <44E2BEDC.9070005@...328...> <20060816101951.19E0.JCARLSON@...244...> <44E39F6B.90501@...236...> <44E4BD0F.8080308@...328...> <44E4F700.1050001@...236...>

Hi all :slight_smile:

I've followed the combined advice of Robin and Christopher in using a python
buffer object to instance a wx.Image in the python wrapper layer. It all seems
to be working - and I don't think there are any 'data copies' going on :wink:

To keep my underlying C++ python-free, I introduced the buffer (and creation of
the wx.Image) via my .i Swig file. In case anyone who is reading is keen trying
to solve a similar problem, I've included the code that I added to my .i file in
order to implement the change below. (I'd appreciate any comments regarding
errors or shortcomings that anyone spots in my implementation.)

Sorry I didn't reply to the thread earlier. A combination of being tied up with
more boring work, moving house and background research into using python buffers
with Swig means that I've only just got this far.

One remaining question I have is: Does the call to wx.EmptyImage that I make
cause any memory to be allocated? I see from Robin's recent post that in my
case this call is now redundant (as I will soon be using ImageFromBuffer) - but
I'm curious to know the answer anyway :wink:

Thanks for all the help. The more I use wxPython the more I like it...

Andrew :wink:

/* the RawImage C++ class that we are wrapping offers a method that
    returns a pointer to an internal 'unsigned char' buffer of display
    data. To make the python class generated by SWIG a bit more
    friendly to the rest of our wxPython code we instruct SWIG to make
    two modifications during the wrapping process. First we add a
    method to the C++ class that will return a version of the internal
    display buffer wrapped up as a 'python buffer object' (performing
    this in the C++ wrapper class keeps the wrapped C++ python-free)...
  */
%extend RawImage {
  PyObject* RawImage::getDisplayBuffer(void) {
    // return a new 'python buffer object' that intialised using
    // the memory address and length of our display buffer...
    void* pvBufferData = (void*)self->pcGetDisplayData();
    int iBufferSize = self->getDisplayBufferSize();
    return PyBuffer_FromMemory(pvBufferData, iBufferSize);
  }
}
/* ... we also add a method to the python wrapper class that will use
    the buffer offered by our new C++ method to create a wx.Image
    (neither of these methods actually copies the image data)...
  */
%extend RawImage {
%pythoncode %{
def getImage(self):
     # create an unintialised wx.Image of the correct size...
     wximage = wx.EmptyImage(self.getWidth(), self.getHeight(), clear=False)
     # then define the data content of the image...
     wximage.SetDataBuffer(self.getDisplayBuffer())
     return wximage
%}
}

---------------------------------------------------------------------
To unsubscribe, e-mail: wxPython-users-unsubscribe@...73...
For additional commands, e-mail: wxPython-users-help@...73...

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

2

Chris,

Thanks for the cross-post. I'm not sure this approach will help speed up the wxAgg accelerator, but I'll put it on the list of things to look into. The problem I foresee is that the Agg renderer's RGBA data has to be converted to RGB before a wxImage can be created by convert_agg2image().

Ken

···

On Aug 18, 2006, at 6:14 PM, Christopher Barker wrote:

As a probably final installment in the thread about optimizing the wx
back-end, here is a post from the wxPython list, in which someone posted
SWIG code for making a PyBuffer from his data set, then using it to
create a wx.Image without copying. A similar approach could be used for
the wxAgg back-end.

-Chris

-------- Original Message --------
Subject: [wxPython-users] Re: using wxImage in C++ python extension
Date: Fri, 18 Aug 2006 16:48:08 +0000 (UTC)
From: Andrew Murray <andrew.murray@...447...>
Reply-To: wxPython-users@...73...
To: wxpython-users@...73...
References: <20060815150714.19CE.JCARLSON@...244...>
<44E2BEDC.9070005@...328...> <20060816101951.19E0.JCARLSON@...244...>
<44E39F6B.90501@...236...> <44E4BD0F.8080308@...328...>
<44E4F700.1050001@...236...>

Hi all :slight_smile:

I've followed the combined advice of Robin and Christopher in using a python
buffer object to instance a wx.Image in the python wrapper layer. It
all seems
to be working - and I don't think there are any 'data copies' going on :wink:

To keep my underlying C++ python-free, I introduced the buffer (and
creation of
the wx.Image) via my .i Swig file. In case anyone who is reading is
keen trying
to solve a similar problem, I've included the code that I added to my .i
file in
order to implement the change below. (I'd appreciate any comments regarding
errors or shortcomings that anyone spots in my implementation.)

Sorry I didn't reply to the thread earlier. A combination of being tied
up with
more boring work, moving house and background research into using python
buffers
with Swig means that I've only just got this far.

One remaining question I have is: Does the call to wx.EmptyImage that I make
cause any memory to be allocated? I see from Robin's recent post that in my
case this call is now redundant (as I will soon be using
ImageFromBuffer) - but
I'm curious to know the answer anyway :wink:

Thanks for all the help. The more I use wxPython the more I like it...

Andrew :wink:

/* the RawImage C++ class that we are wrapping offers a method that
    returns a pointer to an internal 'unsigned char' buffer of display
    data. To make the python class generated by SWIG a bit more
    friendly to the rest of our wxPython code we instruct SWIG to make
    two modifications during the wrapping process. First we add a
    method to the C++ class that will return a version of the internal
    display buffer wrapped up as a 'python buffer object' (performing
    this in the C++ wrapper class keeps the wrapped C++ python-free)...
  */
%extend RawImage {
  PyObject* RawImage::getDisplayBuffer(void) {
    // return a new 'python buffer object' that intialised using
    // the memory address and length of our display buffer...
    void* pvBufferData = (void*)self->pcGetDisplayData();
    int iBufferSize = self->getDisplayBufferSize();
    return PyBuffer_FromMemory(pvBufferData, iBufferSize);
  }
}
/* ... we also add a method to the python wrapper class that will use
    the buffer offered by our new C++ method to create a wx.Image
    (neither of these methods actually copies the image data)...
  */
%extend RawImage {
%pythoncode %{
def getImage(self):
     # create an unintialised wx.Image of the correct size...
     wximage = wx.EmptyImage(self.getWidth(), self.getHeight(), clear=False)
     # then define the data content of the image...
     wximage.SetDataBuffer(self.getDisplayBuffer())
     return wximage
%}
}

---------------------------------------------------------------------
To unsubscribe, e-mail: wxPython-users-unsubscribe@...73...
For additional commands, e-mail: wxPython-users-help@...73...

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

3

Ken McIvor wrote:

I'll put it on the list of things to look into.

Great. I'm glad someone is working on this.

The problem I foresee is that the Agg renderer's RGBA data has to be converted to RGB before a wxImage can be created by convert_agg2image().

darn. I figured as much. wx is really due for an update to support alpha properly. But I guess you'll always have problems with different data formats.

> I'm not sure this approach will help speed
> up the wxAgg accelerator, but

Anyway, this thread started because people were having binary compatibility issues. Even if this doesn't speed up the accelerator, it may be possible to get the same performance without using wx-version-specific compiled code -- i.e. pure python.

I do have one question -- does the agg back-end really need to use an alpha channel for it's buffer? Isn't it the whole image anyway? What is is it going to get blended with?

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

4

Ken McIvor wrote:

The problem I foresee is that the Agg renderer's RGBA data has to be converted to RGB before a wxImage can be created by convert_agg2image().

As if by magic, this from Robin Dunn today:

You may want to take a look at my CVS commits for the last couple weeks. I've now got some raw bitmap access code in place. Both 2.6 and 2.7 will have wx.BitmapFromBuffer and wx.BitmapFromBufferRGBA factory functions which can copy from a buffer object directly into the bitmap's pixel buffer, and 2.7 will also have wx.NativePixelData and wx.AlphaPixelData which allow direct access to the pixel buffer from Python. (The latter needed a bug fix that I'm not sure (yet) can be backported to 2.6...) For example, I can now do this (in a PyShell):

>>> import wx
>>> import numarray
>>> f = wx.Frame(None)
>>> p = wx.Panel(f)
>>> dc = wx.ClientDC(p)
>>> f.Show()
>>> dim=100
>>> R=0; G=1; B=2; A=3
>>> arr = numarray.array(shape=(dim, dim, 4), typecode='u1')
>>> for row in xrange(dim):
... for col in xrange(dim):
... arr[row,col,R] = 0
... arr[row,col,G] = 0
... arr[row,col,B] = 255
... arr[row,col,A] = int(col * 255.0 / dim)
...
>>> bmp = wx.BitmapFromBufferRGBA(dim, dim, arr)
>>> dc.DrawBitmap(bmp, 20, 20, True)

This is looking pretty promising.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

5

wx is really due for an update to support alpha properly. But I guess
you'll always have problems with different data formats.

I think they added preliminary support for alpha channels in 2.5, in the form of wx.Image.HasAlpha(). My beef is that you have to convert everything to a wx.Bitmap before you can do anything useful with it.

Anyway, this thread started because people were having binary compatibility issues. Even if this doesn't speed up the accelerator, it may be possible to get the same performance without using wx-version-specific compiled code -- i.e. pure python.

As near as I can tell, the primary slowdown at this point is the way wxWidgets distinguishes from RGB image data (wx.Image) as opposed to displayed image data (wx.Bitmap). Right now you cannot draw a wx.Image without first converting it into a wx.Bitmap, nor can you use a MemoryDC to blit or otherwise munge a wx.Image directly. My impression is that this made sense when wxWindows was getting started (Win16 and Motif), but is more of an artificial distinction at this point.

I don't think we're going to be able to get performance similar to that of the accelerator using straight Python code unless something changes in the wxWidgets' Image/Bitmap/MemoryDC department. That being said, I'd love to be proven wrong! If you're interested in the gory details, you should check out the pure-Python implementation of the image conversion functions, at the end of `backend_wxagg.py'.

I do have one question -- does the agg back-end really need to use an alpha channel for it's buffer? Isn't it the whole image anyway? What is is it going to get blended with?

I don't know enough about Agg to venture an educated guess. My un-educated guess is that there's an RGBA buffer to support alpha in the drawing operations... how can Agg alpha-composite new pixels into the buffer when you draw something, unless you know the alpha values of the existing pixels?

Ken

···

On 08/28/06 12:19, Christopher Barker wrote:

6

Ken McIvor wrote:

I think they added preliminary support for alpha channels in 2.5, in the form of wx.Image.HasAlpha().

right, but it uses the old 24 bit RGB buffer, and separate Alpha buffer, it's kind of tacked on, rather than native.

> My beef is that you have to convert

everything to a wx.Bitmap before you can do anything useful with it.

And that DCs don't support alpha, even though the underlying device often does. This has been discuses a lot, but no one has done much about it yet -- wxTNG will have something better, but who knows how far out that is?

As near as I can tell, the primary slowdown at this point is the way wxWidgets distinguishes from RGB image data (wx.Image) as opposed to displayed image data (wx.Bitmap). Right now you cannot draw a wx.Image without first converting it into a wx.Bitmap, nor can you use a MemoryDC to blit or otherwise munge a wx.Image directly. My impression is that this made sense when wxWindows was getting started (Win16 and Motif), but is more of an artificial distinction at this point.

a wxBitmap is the same format as the native rendering system. While most systems use 24b RGB (or 32b RGBA), people can still run displays at 16bpp or whatever, so it's still needed. Also, I wouldn't be surprised if some less common systems use ARGB or something else weird.

I don't think we're going to be able to get performance similar to that of the accelerator using straight Python code

But whether it's Python or C++, you still need to do the Image->Bitmap conversion -- so if we can get rid of the data copying from Agg buffer to wxImage in Python, we don't need C++.

unless something changes in the wxWidgets' Image/Bitmap/MemoryDC department.

And it has. For wxPython 2.7 (and now in CVS) there are methods for dumping 32 bit RGBA data directly into a wxBitmap with no copying, if the data source is a Python Buffer object. I think I posted a note about this here yesterday.

I'd love to be proven wrong! If you're interested in the gory details, you should check out the pure-Python implementation of the image conversion functions, at the end of `backend_wxagg.py'.

I did, and I suggested some improvements a couple messages back. To really get it to work, the 24bit RGB Agg buffer needs to be a Python Buffer object -- is it now? I'm sorry I don't have the time to mess with this now -- maybe some day.

I do have one question -- does the agg back-end really need to use an alpha channel for it's buffer? Isn't it the whole image anyway? What is is it going to get blended with?

I don't know enough about Agg to venture an educated guess. My un-educated guess is that there's an RGBA buffer to support alpha in the drawing operations... how can Agg alpha-composite new pixels into the buffer when you draw something, unless you know the alpha values of the existing pixels?

You can alpha composite into a non-alpha background. You just lose the alpha there, so that the background couldn't be alpha-composited onto anything else -- but does it ever need to be?

However, there is something to be said for just using alpha everywhere, and as we'll soon be able to dump RGBA data straight into a wx.Bitmap, this should work great.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                         
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@...236...

7

Ken McIvor wrote:

a wxBitmap is the same format as the native rendering system. While most systems use 24b RGB (or 32b RGBA), people can still run displays at 16bpp or whatever, so it's still needed.

I can understand why it's still necessary, although it's nice to sometimes pretend that everyone's running 24-bit color displays. I hope I didn't sound too judgemental!

I don't think we're going to be able to get performance similar to that of the accelerator using straight Python code

But whether it's Python or C++, you still need to do the Image->Bitmap conversion -- so if we can get rid of the data copying from Agg buffer to wxImage in Python, we don't need C++.

I think we got some wires crossed at some point in the conversation, although it could be that I'm wearing the Stupid Hat today. I was talking about the image-from-a-buffer business not helping us with WX 2.4/2.6 due to the RGBA to RGB conversion.

And it has. For wxPython 2.7 (and now in CVS) there are methods for dumping 32 bit RGBA data directly into a wxBitmap with no copying, if the data source is a Python Buffer object. I think I posted a note about this here yesterday.

Yes, you did mention it. I agree completely with this analysis of the situation. When I replied I wasn't thinking in terms of wxPython 2.7.

To really get it to work, the 24bit RGB Agg buffer needs to be a Python Buffer object -- is it now? I'm sorry I don't have the time to mess with this now -- maybe some day.

I guess Guido lets John borrow his time machine, because RendererAgg appears to already have a buffer_rgba() method.

You can alpha composite into a non-alpha background. You just lose the alpha there, so that the background couldn't be alpha-composited onto anything else -- but does it ever need to be?

I thought that the buffer's accumulated alpha played a role in compositing new pixels onto it, but I apparently misunderstood. It must be time to read "Compositing Digital Images. Anyway, if the buffer's alpha channel isn't used, then the whole situation does seem a bit odd. Could the information be retained for PNGs or something?

However, there is something to be said for just using alpha everywhere, and as we'll soon be able to dump RGBA data straight into a wx.Bitmap, this should work great.

Yes, it will be a great improvement over the current situation.

Ken

···

On 08/31/06 13:43, Christopher Barker wrote: