A few days ago I committed a patch with which I tried to improve the
coordinate handling of annotation.
And I updated the annotation guide which describes the change in some detail.
And here is an example
http://matplotlib.sourceforge.net/trunk-docs/examples/pylab_examples/annotation_demo3.html
I hope this is useful for others as this has been one of my wishlist.
Regards,
-JJ