Even though Python allows many ways to easily debug and profile your code, it is not uncommon to see people overusing simple print statements for this. The presentation will provide an overview of the most common basic debugging techniques that every Python programmer should know. Additionally, for debugging speed or memory problems, couple profilers are presented. Outline:
Basic techniques (print statements, logging)
Debuggers (pdb, winpdb/rpdb2)
Profiling (cProfile, guppy, ...)
2. Who am I
Tuomas Suutari
Software Developer at Anders Inno
8 years Python experience
Anders Inno
Software Company from Turku
specialized in e-commerce
www.andersinno.fi
2013-10-21
5. Print statements
●
●
●
The simplest way to see
what's happening
Sometimes also the fastest
and easiest, so don't
overthink it
Could cause problems
when temporary print
statements slip into
production
2013-10-21
def do_stuff():
something = do_something()
print 'Something:', something
do_more(something)
print 'Calling do_last'
do_last()
6. Logging
●
●
●
●
Python has powerful and
flexible logging module
Like print statements with
more context information
Used through a logger
object with name
Log messages have levels
2013-10-21
import logging
LOG = logging.getLogger(__name__)
def do_stuff():
something = do_something()
LOG.debug(
'Something: %s',
something)
do_more(something)
LOG.info('Calling do_last')
do_last()
7. Logging
●
Log output can be easily
controlled
•
•
●
Outputting to stdout, file,
network socket, ...
Filtering based on log levels
or logger names
Logging statements are
OK in production
2013-10-21
import logging
msg_format = (
'%(asctime)s %(name)s'
' %(levelname)s %(funcName)s'
' %(message)s')
logging.basicConfig(
format=msg_format,
level=logging.DEBUG)
9. PDB
●
The Python Debugger
●
In standard library
●
Console interface
●
Launch script with PDB or
jump into debugger with
set_trace()
python -m pdb some_script.py
def do_stuff():
import pdb; pdb.set_trace()
something = do_something()
# ...
2013-10-21
10. PDB
●
Basic commands
•
c, cont: continue running
•
s, step: step into function
•
p, print: print value of variable/expression
•
b, break: set a breakpoint
•
u, up: move the current frame one level up
•
d, down: move the current frame one level down
•
l, list: list source code for the current file
2013-10-21
11. Winpdb
●
●
●
A Platform Independent
Python Debugger
Run Python script from the
GUI or embed to script
Attaching to embedded
debugger is possible from
a remote host too
2013-10-21
def do_stuff():
import rpdb2
rpdb2.start_embedded_debugger(
'passwd')
something = do_something()
# ...
13. cProfile
●
●
For profiling time
consumption of parts of the
program
python -m cProfile -s cumulative
Visualizers for the data
could be useful
python -m cProfile -o out.pyprof
•
Run Snake Run
•
KCacheGrind (with
pyprof2calltree)
2013-10-21
some_script.py
some_script.py
runsnake out.pyprof
14. Guppy
●
●
For profiling memory
consumption of parts of the
program
# Injecting Guppy memory dumping
Data structures in the
program that are growing
can be detected by
dumping memory profile
periodically with Guppy
and watching the process
with profile browser
hp.setref()
2013-10-21
# to the program:
from guppy import hpy
hp = hpy()
for x in a_loop_in_the_program:
hp.heap().dump('memprof.hpy')
# To start Profile Browser:
python -c "
from guppy import hpy;
hpy().pb()"
15. Guppy
●
Objects can be tracked
down by connecting a
monitor to running program
# Allow monitor to connect to the
# program
import guppy
from guppy.heapy import Remote
Remote.on()
# To connect with the monitor:
python -c "from guppy import hpy;
hpy().monitor()"
<Monitor> sc 1
<Annex> int
>>> hp.heap()
2013-10-21