matplotlib-0.98.5.3.win32-py2.6.exe was compiled without support for GTK.
If you don't mind trying, I have a build of the matplotlib trunk available on my homepage that has GTK support enabled:
It should work with the PyGTK 2.12 Windows binaries from http://www.pygtk.org/downloads.html.
SVG support seems broken: the window.set_icon_from_file() function in backend_gtk.py will raise an exception, not recognizing SVG files. The PNG icon works.