On Sat, Dec 1, 2012 at 6:44 AM, Michiel de Hoon
Since the Python/C glue code is modified only very rarely, there may not be a need for regenerating the Python/C glue code by developers or users from a Cython source code.
True.
In addition, it is much easier to maintain the Python/C glue code than to write it from scratch. Once you have the Python/C glue code, it's relatively straightforward to modify it by looking at the existing Python/C glue code.
not so true -- getting reference counting right, etc is difficult -- I
suppose once the glue code is robust, and all you are changing is a
bit of API to the C, maybe....
This argues against making the Cython source code a part of the matplotlib codebase.
huh? are you suggesting that we use Cython to generate the glue, then
hand-maintain that glue? I think that is a really, rally bad idea --
generated code is ugly and hard to maintain, it is not designed to be
human-readable, and we wouldn't get the advantages of bug-fixes
further development in Cython.
So -- if you use Cython, you want to keep using, and theat means the
Cython source IS the source. I agree that it's a good idea to ship the
generated code as well, so that no one that is not touching the Cython
has to generate. Other than the slight mess from generated files
showing up in diffs, etc, this really works just fine.
Any reason MPL couldn't continue with EXACTLY the same approach now
used with C_XX -- it generates code as well, yes?
Michael Droettboom wrote:
For the PNG extension specifically, it was creating callbacks that can
be called from C and the setjmp magic that libpng requires. I think
it's possible to do it, but I was surprised at how non-obvious those
pieces of Cython were. I was really hoping by creating this experiment
that a Cython expert would step up and show the way 
Did you not get the support you expected from the cython list? Anyway,
there's no reason you can't keep stuff in C that's easier in C (or did
C_XX make this easy?). I think making basic callbacks is actually
pretty straightforward, but In don't know about the setjmp magic (I
have no idea hat that means!).
The Agg backend has more C++-specific challenges, particularly
instantiating very complex template expressions --
I'm guessing you'd do the complex template stuff in C++ -- and let
Cython see a more traditional static API.
but some of that complexity could be reduced by using Numpy arrays in place of the
image buffer types that each of them contain
OR Cython arrays and/or memoryviews -- this is indeed a real strength of Cython.
The Cython version isn't that much shorter than the C++ version.
I think some things make sense to keep in C++, though I do see a fair
bit of calls (in the C++) to the python API -- I'm surprised there
isn't much code advantage, but anyway, the goal is more robust/easier
to maintain, which may correlate with code-size, but not completely.
These declarations aren't exact matches to what one would find in the header file(s) >because Cython doesn't support exact-width data types etc.
It does support the C99 fixed-width integer types:
from libc.stdint cimport int16_t, int32_t,
Or are you talking about something else?
I'm not sure why some of the Python/C API calls I needed were not defined in Cython's include wrappers.
I suspect that's an oversight -- for the most part, stuff has been
added as it's needed.
One other note -- from a quick glance at your Cython code, it looks
like you did almost everything is Cython-that-will-compile-to-pure-C
-- i.e. a lot of calls to the CPython API. But the whole point of
Cython is that it makes those calls for you. So you can do type
checking, and switching on types, and calling np.asarray(), etc, etc,
etc, in Python, without calling the CPython api yourself. I know
nothing of the PNG API, and am pretty week on the CPython API (and C
for that matter), but I it's likely that the Cython code you've
written could be much simplified.
Once things compiled, due to my own mistake, calling the function segfaulted. Debugging
that segfault in gdb required, again, wading through the generated code. Using gdb on
hand-written code is *much* nicer.
for sure -- there is a plug-in/add-on/something for using gdb on
Cython code -- I haven't used it but I imagine it would help.
Ian Thomas wrote:
I have never used Cython, but to me the code looks like an inelegant combination of
Python,C/C++ and some Cython-specific stuff.
well, yes, it is that!
I can see the advantage of this approach for small sections of code, but I have strong > reservations about using it for complicated modules that have extensive use of
templated code and/or Standard Template Library collections (mpl has examples of
both of these).
So far, I've found that Cython is good for:
- The simple stuff -- basic loops through numpy arrays, etc.
- wrapping/calling more complex C or C++
-- essentially handling the reference counting and python type
packing/unpacking of python types.
So we find we do write some shim code in C++ to make the access to the
core libraries Cython-friendly. We haven't dealt with complex
templating, etc, but I'd guess if we did I'd keep that in C++. And
since the resulting actual glue code is pretty simple, it makes the
debugging easier.
Maybe rather than asking "if we switched to using Cython, would more participate", I
should be asking "among those that can participate in removing the PyCXX
dependency, what is the preferred approach?"
I don't know that we need a one-sieze fits all approach -- perhaps
some bits make the most sense to move to plain old C/C++, and some to
Cython, either because of the nature of the code itself, or because of
the experience/preference of the person that takes ownership of a
particular problem.
-Chris
···
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@...236...