Linux User Space Debugging & Profiling

© 2010-15 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Debugging & Profiling

2© 2010-15 SysPlay Workshops <workshop@sysplay.in>
What to Expect?
Debugging
Various Techniques
Using Debugger: gdb
Profiling using
Program Checkers
Code Coverage Tools
Performance Analyzers

Debugging

Ways of Debugging
Basic: Printing
Querying
Kernel Windows: /proc, /sys
Using Commands like ipcs
Tracing: strace
Debugger: gdb

Debugger: gdb
Text mode debugger
Repetition of previous command
Very powerful
All kind of options: Breakpoint, Watch, ...
Extensive Help
GUI interfaces: ddd, ...

gdb: Debugging Strategies
Oh no!!! Another tool to learn
Worth learning a new tool
Segmentation faults & Core dumps
Execution intercepted by gdb
Allows examining the state, backtrace, etc
Debugging is Narrowing Down
Reaching close & then stepping

'gdb' Usage
Compile with -g option
gcc -g file.c -o file
Run gdb (with desired options)
gdb [options] ./file [core file]
Pass command line arguments by setting args
set args <cmd_line_args>
Run the program by
Typing c, or
run <cmd_line_args>
For debugging: break, watch, backtrace, …
For help: help <command>

gdb: Examining Source Files
list <line>
Print 10 lines around <line>
list [<filename>:]<function>
Print 10 lines around beginning of <function>
list <start>,<end>
Print lines from <start> to <end>
list
Print 10 more lines

gdb: Other ways of Running
step [ count ] – step into sub-routines
next – run over sub-routines in a go
finish – run till current function returns
return – make selected stack frame return to its
caller
jump <address> – continue program at specified
line or address

gdb: Breakpoints
info break – Show list & status of breakpoints
Setting breakpoints (returns its number)
[t]break <function> [ if <expr> ]
[t]break [<file>:]<line>
Breakpoint Operations
disable <break_no>
enable <break_no>
delete <break_no>
commands <break_no>
Execute gdb commands on reaching <break_no>

gdb: Examining the Stack
Stack grows down
Each call state is referred as a frame
backtrace – Show stack frames
frame <frame_no> – Select <frame_no>
down – Select callee's frame
up – Select caller's frame
info args – Show args of current frame
info locals – Show locals of current frame

gdb: Examining Data
print[/xduotcf] <expr>
Evaluate the valid C <expr> within current frame
set variable <var> = <expr>
Assign <expr> to <var> in the current scope
Variables starting with $ are local to gdb
display <expr>
Print <expr> whenever program stops
undisplay
Cancels all previous display requests

gdb: Other Examinations
Watchpoints / Catchpoints
info watchpoints (Same as info break)
watch <expr> - Break on Write watchpoint
rwatch <expr> - Break on Read watchpoint
awatch <expr> - Break on Both watchpoint
Default is hardware watchpoint
Registers: info registers
Set-able options: info set

gdb: Shortcuts
Just enough of command
To make it unique
Added with tab completion
First letter only for
break, delete, run, continue, step, next, print
Repeat last by <Enter>
Executing the same set of initial commands
Put them in .gdbinit
-x <gdb command file>

gdb: Miscellaneous
Set editmode for gdb command line
editmode [ emacs | vi | dumb ]
Execute a shell command
shell <cmd>
Print command history
history
Set logging
set logging <on | off>
set logging file <log_file> (Default: gdb.txt)

gdb: Thread Debugging
Automatic notification of new threads
Switching among threads
thread <thread_no>
Auto switching on stop by breakpoint or signal
Inquiry on existing threads
info threads
Applying command to a list of threads
thread apply <thread_no> | all <cmd>
Thread specific breakpoints
Example: break <line> thread <thread_no>

gdb: Multi-Process Debugging
gdb version >= 7.1
Multiple processes from same session
Before that
Run different programs with different gdb
If parent-child program
Run the parent using gdb
Attach another gdb with running child pid

Let's Debug
Debug
debugging_cprog.c
find_the_bug.c
Thread Debugging
multi_threads.c

User Process Memory Debugging
ElectricFence → DUMA
MEMWATCH
All these tools replaces the usual memory
allocation functions like malloc, free, … by the ones
to keep track of the same

Profiling

W's of Profiling
Analyzing & Understanding Programs
Many varieties
Coding Style Checker
Logic Analysis
Memory Analysis
Execution Time Profiling
Run Time Error Capture
Execution Flow Analysis

Profiling Categories
Program Check
Static (splint)
Dynamic (memwatch, ElectricFence → DUMA)
Code Coverage (Dynamic)
Execution Flow using test vectors (gcov)
Performance Analysis (Dynamic)
Execution Time using test vectors (gprof)
All in One: valgrind

Static Program Checkers
Compiler itself
Really fast
But too naive
Other Extreme: Formal Verification Tools
Extensive
But needs huge resources
Preference: In between solutions
Example: splint

'splint' detects
Memory related Problems
Dereferencing a possibly null pointer, Using possibly undefined storage
Returning storage that is not properly defined
Dangling references, Memory leaks, Buffer overflow, ...
Dangerous Coding
Likely infinite loops, fall-through cases or incomplete switches
Suspicious statements, Dangerous aliasing
Dangerous macro implementations or invocations
Type mismatches with greater precision and flexibility, ...
Specific Violations of
Customized naming conventions
Information hiding
Modifications & Usage of global variable as per specified interfaces
...

'splint' Usage
Run splint (with desired options)
splint [options] file[.c]
Prints out all the messages about the file
On standard output
Errors on standard error (using -usestderr)
Some interesting options
-weak for lesser messages
-I <incl_dirs>
Try 'splint splint_cprog'

Dynamic Program Checkers
Memory Debugging
mtrace (export MALLOC_TRACE=outfile.txt)
memwatch
ElectricFence → DUMA
...
Replaces the memory allocation functions
Like malloc, free, ...
By the ones to keep track of the same

Code Coverage
Dynamic Execution Flow Analysis
Done by actually running the Program
Usually done with sets of test vectors
Providing the Test Code Coverage
Coverage of: Line of Code, Branches, Blocks, ...
Typically provides
Entity executed or not executed
% or times of execution of various entities
Tool: gcov
Could provide performance info when used with gprof

'gcov' Usage
Compile with -fprofile-arcs -ftest-coverage options
gcc -fprofile-arcs -ftest-coverage file.c -o file
Execute the program with the test vectors
./file [ <cmd_line_args> ]
Generates .gcda (Data) & .gcno (Graph) files
Run gcov (with desired options)
gcov [options] file.c
Interesting options: --all-blocks --branch-probabilities
Generates coverage output in .gcov file
Try with gcov_cprog.c

Performance Analysis
Dynamic Execution Time Analysis
Done by actually running the Program
Usually done with sets of test vectors
Providing the Test Performance Analysis
Typically provides info on Functions
Time Spent, Times Called
Call Graph Profile Data
Performance Bottlenecks
Tool: gprof

'gprof' Usage
Compile with -pg option
gcc -pg file.c -o file
Run the program
./file [ <cmd_line_args> ]
Generates gmon.out file
Run gprof (with desired options)
gprof [options] ./file [gmon.out]
Dumps profiling info on stdout
Try with gprof_cprog.c

Interesting 'gprof' Options
-b # Brief output
-p # Flat Profile
-q # Call Graph Analysis
-A # Print with Annotated source code
-x # Annotate all lines
-l # Line-by-line profiling
-s # Sum
-z # Display unused functions
-r, -R <mapfile> # Reordering suggestions

Tool Suite: valgrind
Provides a number of debugging & profiling tools
Most popular being Memcheck
Best behaviour without optimization
Other possibly useful tools
Cachegrind: Profiler for cache (miss) and branch (misprediction) events
Callgrind: Profiler to show cost relationships across function calls,
optionally with cache simulation
Massif: Space profiler to explore in detail which parts of your program
allocate memory
Helgrind: Debugging tool for threaded programs
Looks for various kinds of synchronisation errors in code
That uses the POSIX pthreads API
GUI frontend: valkyrie

valgrind: Memcheck detects
Touching “shouldn’t” memories
Overrunning heap block boundaries
Reading/Writing freed/unallocated memory
Memory leaks
Not freeing heap blocks
Incorrect freeing of memory
Double-freeing heap blocks
Using values before initialization

'valgrind' Usage
Compile with -g option
gcc -g file.c -o file
Run using valgrind & desired options
valgrind [options] ./file [ <cmd_line_args> ]
By default, runs Memcheck
Profiles out the memory usage of the program
For using other tools of valgrind, add
--tool=<toolname>
Try with valgrind_cprog.c

What all have we learnt?
Debugging
Various Techniques
Using Debugger: gdb
Profiling using
Program Checkers
splint, mtrace, memwatch, ...
Code Coverage Tools
gcov
Performance Analyzers
gprof, valgrind, ...

Any Queries?

Linux User Space Debugging & Profiling

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Linux User Space Debugging & Profiling

Similar to Linux User Space Debugging & Profiling (20)

More from Anil Kumar Pugalia

More from Anil Kumar Pugalia (14)

Recently uploaded

Recently uploaded (20)

Linux User Space Debugging & Profiling