Mike,
I apologize for not reading through your script completely to test, but does this re-write the __init__.py files so that they don't declare namespace packages using pkg_resources?
If you aren't doing this, then you still won't get to the time savings Fernando and I did because a significant part of the overhead was in setuptools/pkg_resources declaring namespace packages and importing from them. In fact, in Fernando's small test script using Traits, there were over 5,000 calls(!!!) to pkg_resources even when we'd de-eggified, but not de-package-namespace-ified.
-- Dave
Michael McLay wrote:
ยทยทยท
The attached script creates an enthought packages out of enthought
eggs. It uses symbolic links so it won't work on Windows and the eggs
need to be kept on the filesystem. I'll rework it to copy the trees
instead of just setting up symbolic links.On 8/18/07, Fernando Perez <fperez.net@...149...> wrote:
On 8/16/07, Eric Firing <efiring@...229...> wrote:
So, 10% slower with backend_agg, 18% slower with backend_svg. It's not
devastating, but it's not so great, either.Those results look fine to me. I dont know what has changed since we last
discussed this, but when Eric brought up the speed issue I remember the
traited config was significantly slower at that time. Maybe this is very good
news!
Maybe there is some sensitivity to machine architecture; my tests were
on a Lenovo T60 laptop, Core2 at 2 GHz.For Fernando's simple_plot, using /usr/bin/time, I get:
0.53user 0.11system 0:00.66elapsed without traits
0.66user 0.21system 0:00.89elapsed with traits(The figures are quite repeatable; numbers above are representative. CPU
is 98% in both cases.)So the total time for this very simple plot (which makes it something of
a worst case) is more than 30% longer with traits than without, implying
that the startup time increase is even larger as a percentage. One
might argue that the difference of 0.23 seconds doesn't matter, but I
think it is still worth considering. Maybe it can be beaten down to a
small fraction of that.
Here's some interesting info. I'm sitting here with Dave Peterson,
from Enthought, and we've done a bunch of profiling that pointed to
setuptools, not Traits, being the culprit for the time increase.
We've now just done an install of Traits *without* any setuptools
(right now that requires manual surgery, but later it can be done
automatically if needed). Here are the resulting times:# Using traits
maqroll[mpl-traits-debug]> time ./simple_plot.py
*** Using Traits!!!
1.844u 0.212s 0:02.13 96.2% 0+0k 0+0io 0pf+0w
maqroll[mpl-traits-debug]> time ./simple_plot.py
*** Using Traits!!!
1.840u 0.216s 0:02.58 79.4% 0+0k 0+0io 0pf+0w
maqroll[mpl-traits-debug]> time ./simple_plot.py
*** Using Traits!!!
1.836u 0.196s 0:02.12 95.2% 0+0k 0+0io 0pf+0w# NOT Using traits
maqroll[mpl-traits-debug]> time ./simple_plot.py
2.200u 0.280s 0:02.67 92.8% 0+0k 0+0io 0pf+0w
maqroll[mpl-traits-debug]> time ./simple_plot.py
2.248u 0.220s 0:02.74 89.7% 0+0k 0+0io 0pf+0w
maqroll[mpl-traits-debug]> time ./simple_plot.py
2.216u 0.244s 0:02.72 90.0% 0+0k 0+0io 0pf+0wAs you'll notice, the traits times are *lower*. This is what my gut
told me, since I know how tight the traits code is. The point is that
traits can actually give you a performance *benefit*, not a cost. The
problem is with setuptools, which goes ballistic on the filesystem at
init time. The profiles I sent earlier have already all the
information on that, if you use kcachegrind to see it (that's how Dave
and I pinned it down).This hopefully settles the argument on the performance side. We'll
leave the final decision up to you guys, obviously. For IPython, this
settles the matter and we're going with traits, with setuptools banned
til further notice from ipython.Cheers,
f