Store cProfile output in pandas dataframe
There are several posts discussing python profiling using cProfile, as well as parsing the output due to the output file retranslating from the example code below - it is not a regular text file.
(the displayed code is just a sample from https://docs.python.org/2/library/profile.html , not directly reproducible).
import cProfile
import re
cProfile.run('re.compile("foo|bar")', 'restats')
There is one discussion here: Profile python script using cProfile to external file
And docs.python.org has more details on how to parse the output using pstats.Stats
(still only sample, not reproducible):
import pstats
p = pstats.Stats('restats')
p.strip_dirs().sort_stats(-1).print_stats()
I may be missing some very important details, but I would really like to store the output in a pandas frame and do further analysis from there.
I thought it would be pretty simple, since the output in iPython from running cProfile.run () looks pretty neat:
In[]:
cProfile.run('re.compile("foo|bar")'
Out[]:
Any suggestions on how to get this in a pandas dataframe in the same format?
source to share
It looks like https://github.com/ssanderson/pstats-view can do what you want (albeit with unnecessary dependencies related to data visualization and how it is interactive):
>>> from pstatsviewer import StatsViewer
>>> sv = StatsViewer("/path/to/profile.stats")
>>> sv.timings.columns
Index(['lineno', 'ccalls', 'ncalls', 'tottime', 'cumtime'], dtype='object')
source to share