A simple talk on the art of debugging presented in Tainan, 2011

1. SOFTWARE ENGINEERING:
various concepts around
- implementation
- testing
- debugging
- development rules
O. Teytaud

2. I will here give a list of rules that
one can meet as a developer.
I don't claim that all these rules are
Good. In fact, I keep only some of
them for myself.
It also depends on the context in
which you develop.
Make your own choice :-)

3. Various sayings around software development (1)
Debugging and maintaining takes
much more time than implementing.
==> good development saves up time.
Typesetting is a very small part of
implementation.
Data structures are more important
than codes/algorithms ==> discuss data
structures more than codes

4. Various sayings around software development (2)
Premature optimization is
the source of all evil.
Slow bug-free code is often ok.
Only a few parts of the program are worth
being optimized.
People who never code don't know.
Trust only people who code. If the
boss does not code, don't trust the boss.

5. ASSERT
We already discussed this,
do you remember ?
Function tralalaItou(.....)
{
…
x=a/b;
…
y=log(z);
...
}

6. ASSERT
We already discussed this,
do you remember ?
Function tralalaItou(.....)
{
…
assert(b!=0); x=a/b;
…
assert(z>0); y=log(z);
...
}

7. UNIT TESTS
Anyone who knows
what is a unit test ?
Example:
I implement a function “half(x) {return x/2}”
Its unit test is
===================
y=drand48()
assert(half(y)==y/2);
===================
Principle of unit tests:
unit tests for all functions.

8. NO OBESE CODE (1)
Implement only the methods
which are really necessary.
Many programs die (no users,
no maintenance, …) because
they are too big and unreadable.
Keep it simple.
Four small classes/files
are better than one huge class/file.

9. NO OBESE CODE (2)
Do not include a code which is not
extremely useful.
Because:
- license issues
- maintenance issues
- compatibility/portability issues

10. MAINTENANCE RULES
If you find a bug,
- first, correct the bug :-)
- put several asserts which prevent the
bug from coming back
- if unit testing, build the unit test
corresponding to the bug

11. PREMATURE OPTIMIZATION
is the source of all evil.
Do you understand what I mean
by “optimization” ?
1) Implement with slow functions
2) Find where you should optimize
(how ?)
3) Optimize this place only; possibly, then,
go back to (2)

12. Find where your program is
slow
Profiling:
Finding where and why your
program is slow.
Possible tool: instrumentation
t1=clock();
…some code...
t2=clock();
timeSpent=t2-t1;

13. Find where your program is
slow
Profiling:
Finding where and why your
program is slow.
Possible tool: instrumentation
static timeSpent=0;<== do you understand this ?
t1=clock();
…some code...
t2=clock();
timeSpent+=t2-t1; // cumulated time matters!

14. Or profiling with instrumentation
Valgrind is just great!
1. valgrind –tool=callgrind ./myprogram
2. kcachegrind
First line: runs your program (slower than
without valgrind)
Second line: displays a graphical interface,
showing where the run has spent time.

15. Pair coding
Typesetting is a small part of the job.
So a second person,
in front of the keyboard, is not useless.
Two pairs of eyes are better than one.
Also very good for transferring
knowledge and competencies.

16. Release often
Do not keep a new version private during
all months of development.
Release intermediate versions.
Because:
- earlier feedback on usefulness
- earlier feedback on bugs
- happy users

17. Intellectual property
Think about open source; keep only
a proprietary interface to
specialized applications.
Because
- No security cost (this is a huge benefit).
- Free feedback / help.
- Motivation by wide diffusion

18. Management (very personal
opinion) (1)
Big developments are a team-work.
People afraid of being fired produce
unreadable code so that they become
necessary.
People afraid of problems will just decide
not to code anything which is important.
Don't kill motivation.

19. Management (very personal
opinion) (2)
Big developments are a team-work.
Hierarchy among developers is bad;
when people are civilized,
discussion/democracy
is better than hierarchy.

20. Software development methodologies
What is a SDM ?
A list of stages, for going from an
idea of software to the software itself.
Is there only one SDM ?
No:
- Structured Programming
- Object Oriented Programming (focus on data)
- Scrum
- Extreme Programming
- ...

21. Software development methodologies
Are all these SDM contradictory ?
No.
E.g. you can do “extreme programming” (XP)
for object-oriented programmming and with
structure programming (no “goto”).

22. Example 1: Waterfall development (Royce 1970)
Dividing the project into phases
First phases = specification, detailed
specification
Specify dates for phases
Extensive documentation
Strong checking at the end of each phase.
= a “linear” methodology

23. Example 2: Prototyping
Quickly doing a first version:
- possibly incomplete;
- possibly with some features missing.
Ideas:
- see problems early (thanks to prototype)
- release early so that people can test.
- prototype can be discarded or evolved to final model.
= an “iterative” process (variants: “agile”, “extreme”...)

24. Example 2: Prototyping
Remark on prototyping:
- specification does not matter “so much”;
- software matters much more;
- I like that, because it gets rid of
false contributions like philosophical
remarks on high-level specifications;
- stupid people/lazy people can discuss
on high-level specifications.
(please do not understand this as
“dirty programming” without thinking...)

25. Example 3: “incremental development”
Idea = combining both.
Becomes a bit unclear.
E.g.:
- Do detailed specifications as in “waterfall”
- Then implement each block in a prototyping manner
- Release quicky as in the prototyping approach

26. Other examples
You can find various methodologies on internet:
- spiral development
- many variants of prototyping,
with plenty of philosophical
pictures which don't always
look serious
==> suggests that
prototyping is a good idea
==> not sure we need a lot of
theory on this idea

27. Other examples
So many examples now...
See e.g. Unified Process
(based on Unified Modeling Language).
With plenty of non-convincing graphs with
boxes, circles, arrows, philosophical statements.

28. In my opinion:
- know SVN (and/or other such tools)
- read and understand object-oriented programming
(like it or not, but you will see it
in your professional life...)
- maybe give a try to functional programming
- and do prototyping-approaches (whenever you don't call
it prototyping: fast release, no premature optimization,
prefer the easy way first)

29. What is software testing ?
- unit tests (each functions does
what it is intended to do)
- global test (the program does
what it is intended to do)
- performance test (is the program
fast enough / does it save up
resources)

30. Should we have testings ?
Yes, very often.
Well, some people say that
you should not, and that you
should prove correctness
(good luck).

31. Should we have testings ?
But not necessarily yourself:
- have users (release often) with
automatic upgrades
- use open source
- include automatic bug report
in case of “assert”
(e.g. “core” files;
do you remember ?)

32. Terminology of testing
Unit testing: remember ?
Real-time testing: assert!

33. Terminology of testing
Functional testing
choose a functionality
in the documentation and test it.
= testing high-level feature, and
only features included in doc.

34. Terminology of testing
Non-functional testing:
check something which is not
in the doc, but which might be
Important: scalability (w.r.t
number of cores, or memory, or
size of problem, or compatibility...)

35. Static testing
Re-read the code.
Re-read the specifications.
Two readers: very efficient.

36. Dynamic testing
Debugger.
Profiler.

37. Black box, white box
Black-box:
testing without knowing
any thing about how it works.
White-box: testing with the API
or using internal pieces of codes for
driving tests.

38. Regression testing
Checking that the program
performs at least as well
as the previous version.

39. Greek letters
Alpha-testers: testers from
your company.
Beta-testers: testers who accept
a possibly highly unstable code.

40. Usability testing: is it easy to use ?
Security testing: can we cheat for getting
confidential data ?

41. How to have plenty of free testers ?
Cloud development:
if the code is mainly on your side
(on your cluster), then you can easily
switch some users to a new version.
E.g.: gmail

42. How to have plenty of free testers ?
Automatic updates:
Linux, Windows, many codes.
Incidentally: bugfixes are immediately
spread everywhere.

43. How to have plenty of free testers ?
Open source:
when using is free, you get many users.
When they can see the source, they
might even find the bug.

44. Related stuff
Software verification:
includes testing; answers
the question “does it work” ?
Also includes “program proving”.
Software validation:
Is it really what we need ?
Integration by the customer ok ?

45. Related stuff
Warranty: should you give some warranty
to your user ?
==> quite dangerous.
==> if your user looses 15 billions
US dollars and can claim that it is
because your software
had a bug, you have a trouble
==> limited warranty at most.

46. Want to test your programming skills ?
Enter a programming contest.
Often possible just by internet.
E.g.: 24 hours, 3 persons
for developing a program
which performs a given task.
You get celebrity if you win,
you get experience if you loose.
I'll organize one soon, join :-)

47. DISCLAIMER (1)
These slides might be biased by my personal experience.
Many programs in real life are incredibly dirty and buggy.
A brief sample of my experience:
- nobody wanted to be responsible of anything (afraid
of troubles; so people prefer
coding minor unimportant easy stuff);
- someone was angry, and decided to make
it impossible to get rid of bugs;
- the crucial person who could make it work
has gone away (happens often);
- no time for testing, because everybody had to
justify that he is very very fast;

48. DISCLAIMER (2)
- bugs were considered as features, because
nobody really wanted to maintain a dirty obese code;
- delays were unrealistic, because the guy who
decides has no idea of what it is about (the
boss is often not a developper);
- everything was blocked by intellectual property people
(when there's no use protecting a code, they protect it
anyway just because they must justify that they work).

49. PERSONAL CONCLUSION (1)
I think there's much more respect than 10 years
ago for developpers in software companies (e.g. Google)
Switching from “waterfall” to “prototyping” illustrates this.
My take-home message:
I think that “assert” is great, as well as
test suites, profiling, debuggers,
automatic upgrades, open source, pair coding, cool
atmosphere.
Many sophisticated philosophical discussions
on software development are (for me...)
a little bit useless and just a posteriori rationalization
of what people actually decided to do.

50. PERSONAL CONCLUSION (2)
Think of specialized languages
- Matlab / Octave: fast
prototyping / modelization with maths
- R for statistics / machine learning
- Cobol for business applications
- Bash/Perl for fast prototyping of file manipulation

51. Do you know the COBOL language ?
A main victim of the year 2000 problem
(because it is dedicated to business applications).
COBOL is also a self-modifying language:
“Alter X to proceed to Y”
means: “goto X” becomes “goto Y”.
Original COBOL had no local variable;
no while; no dynamic allocation.
A nice challenge for software testing:
who had guessed the year 2000 problem
10 years in advance ?

52. “Testing can only prove the presence of
defects, never their absence.”
E. Djikstra
(author of« A Case against the GOTO
Statement »)
(militant against software testing)

53. “Object-oriented programming is an
exceptionally bad idea which could only
have originated in California.”
-- Edsger Dijkstra
I do not consider that object oriented
programming is always a good idea.
Don't say this in a job interview; OOP is
now politically correct.

54. “It is practically impossible to teach good
programming to students that have had a
prior exposure to BASIC: as potential
programmers they are mentally mutilated
beyond hope of regeneration.”
-- Edsger W. Dijkstra, SIGPLAN Notices,
Volume 17, Number 5
(I started programming with Basic...)

55. “The use of COBOL cripples the mind;
its teaching should, therefore, be
regarded as a criminal offense.”
Edsger W. Dijkstra

56. Don't consider that programming
languages will never change.
In old programming languages:
- lines had a number;
impossible to concatenate
two functions.
- variables should be written
with two letters at most.
- there were “goto” everywhere.
And some people, at that time, claimed that
extending variable names to 3 or more
characters was a stupid idea.