The Matplotlib frontend is no longer graphics library neutral

Also, I see the frontend file has: import

    > matplotlib.agg

matplotlib does not require the agg backend, but it does require a
minimal bit of agg functionality. Specifically, agg::path_storage us
the cross-backend path storage class, which you use in backend_cairo
in the draw_markers function. As you probably remember, I started out
with a simple list of tuples to represent a path, but eventually
decided we would be better off with a full blown, bug vetted
implementation rather than rolling our own. agg's seemed a natural
choice because the code is already in the mpl code base, and it has
the additional benefit that backend_agg can use it with no overhead.

So backend matplotlib.backends._backend_agg and matplotlib.agg are
different beasts. The first is pycxx hand written code to implement
the Renderer and GC classes; the latter is a SWIG wrapper of agg. I
would like to do away with the former and replace all of it's
functionality with the latter (I started experimenting with this in
backend_agg2). The advantage of this is that agg users could also
draw directly onto the canvas using agg primitives and would not be
limited by the mpl drawing model.

This is a long winded way of saying that I have polluted the agg
module's minimal set of cross-backend agg functionality with some
graphics capability. I think the solution is to factor matplotlib.agg
this into two modules: one lightweight which has the rendering
independent computational geometry stuff which can be used across
backends, and one which has the rendering stuff . mpl has reinvented
(for the 40th time) many of the primitives that every graphics library
needs (color storage, path storage, affine transformations, etc...).
In the grand refactoring that I've wanted to do for many moons but
haven't found the time, I've envisioned replacing a lot of this custom
stuff with agg's. The code in matplotlib.agg layed the groundwork for
this, but as I say I got a little carried away and included other
antigrain stuff too.

How about two modules, the first required ("basics") that has
rectangles, path_storage, affine transformations, etc, and another
optional ("agg" or "antigrain") which has the rendering buffers,
renderers, strokes and so on, thus keeping the cross-backend required
component as light as possible. The only objection I can think of is
that is makes C++ a core requirement for mpl, but it already is for
the _transforms and ft2font modules.

Does this sound reasonable to you? Do you have any objection to
reusing a minimal set of agg across backends?