[IPython-user] profiling advice

[Fernando Perez]

Hi all,

On 8/7/07, skip@pobox.com <skip@pobox.com> wrote:
>
>     Stefan> On his webpage, Fernando shows how to combine hotshot and
>     Stefan> kcachegrind to visualize profiling results:
>
>     Stefan> http://amath.colorado.edu/faculty/fperez/python/profiling/
>
>     Stefan> I find his recipe very useful.
>
> Since hotshot is kind (pretty much?) deprecated in favor of the cProfile
> module, any idea how much work would be involved in replacing hotshot with
> cProfile in this recipe?

Yup, that page above should now be deprecated, since someone wrote a
little tool to do the same thing with cProfile.  Just a couple of days
ago on the MPL list, Eric Fring sent this very useful nugget:

######################
Fernando,

Here is some more information about profiling that others may find useful:

1) The profile module reports times that are much longer than normal; I
don't know to what extent the relative times are still correct; my
impression is that they do still give a decent idea of where the
bottlenecks and resource hogs are.

2) The combination of hotshot and hotshot2calltree (or similar) gets
badly confused by duplicate function names such as __init__, so the
kcachegrind view can be quite misleading.

3) A solution is to use the cProfile module in Python 2.5 with an
appropriate reformatter.  The following script runs a python script
through cProfile and generates a *.log file that kcachegrind reads:

http://www.gnome.org/~johan/lsprofcalltree.py

So the sequence is simply

python lsprofcalltree.py testscript.py
kcachegrind testscript.py.log &

Here are a couple of relevant Google results:

https://lists.dulug.duke.edu/pipermail/yum-devel/2007-January/003045.html
http://ddaa.net/blog/python/lsprof-calltree

Eric
################################

So use that lsprofcalltree instead, and you'll get the nice
KCacheGrind tool but with more accurate results.

Cheers,

f