1. Formal Verification

2. Formal Verification…
“…is the act of proving or disproving the correctness of
intended algorithms underlying a system in respect to a
certain formal specification or property, using formal
methods of mathematics” – Wikipedia

3. ☺
System S → Model M, where M ∋ [finite state
machine, labeled transition system, timed automata,
Petri net,…)
!
(M ∈ ⊨) ∧ ((S ⩰ M) ∈ ⊨) ⇒ S ∈ ⊨

4. Why
• find more bugs in design
and implementation
• prove correctness
• cut down testing and
maintenance cost
• Common Criteria (ISO
15408), FIPS-140-2
Level 4,…
• build high-assurance
systems

5. Formal Verification of Design
• Make a formal design by using a specification
language etc.
• Define and formalize desired properties
• Verify that the chosen formal property holds for
the formal design
!
describe → design → formalise → verify

6. Formal Verification of Design
• Making formal design is (mostly) not fun
• May not be enough details for interesting
properties
• Far away from actual implementation

7. Formal Verification of Program
• Use code as formal design document
• Easier to combine with debugging and testing
• Closer to actual implementation
• Can benefit from additional methods like static
program analysis etc.

8. Code to Model Example
#include <stdio.h>
#include <stdlib.h>
!
int main(int argc, const char* argv[])
{
FILE* f;
int c, n = 0, w = 0;
!
if (argc < 2)
{
printf("Filename?n");
return 1;
}
!
f = fopen(argv[1], "r");
if (f == NULL)
{
printf("open error:%sn", argv[1]);
return 1;
}
!
!
!
!
!
while ((c = fgetc(f)) != -1)
{
if (!isspace(c))
{
if (!w)
++n, w = 1;
else
w = 0;
}
}
printf("Word count: %dn", n);
!
fclose(f);
return 0;
}

9. Code to Model Example
#include <stdio.h>
#include <stdlib.h>
!
int main(int argc, const char* argv[])
{
FILE* f;
int c, n = 0, w = 0;
!
if (argc < 2)
{
printf("Filename?n");
return 1;
}
!
f = fopen(argv[1], "r");
if (f == NULL)
{
printf("open error:%sn", argv[1]);
return 1;
}
!
!
!
!
!
while ((c = fgetc(f)) != -1)
{
if (!isspace(c))
{
if (!w)
++n, w = 1;
else
w = 0;
}
}
printf("Word count: %dn", n);
!
fclose(f);
return 0;
}

10. Code to Model Example
#include <stdio.h>
#include <stdlib.h>
!
int main(int argc, const char* argv[])
{
FILE* f;
int c, n = 0, w = 0;
!
if (argc < 2)
{
printf("Filename?n");
return 1;
}
!
f = fopen(argv[1], "r");
if (f == NULL)
{
printf("open error:%sn", argv[1]);
return 1;
}
!
!
!
!
!
while ((c = fgetc(f)) != -1)
{
if (!isspace(c))
{
if (!w)
++n, w = 1;
else
w = 0;
}
}
printf("Word count: %dn", n);
!
fclose(f);
return 0;
}

11. Code to Model Example
#include <stdio.h>
#include <stdlib.h>
!
int main()
{
FILE* f;
int c, n = 0, w = 0;
enum {FALSE,TRUE} B1;
!
if (B1 = any())
{
return 1;
}
!
f = fopen(“sample”, "r");
if (f == NULL)
{
return 1;
}
!
!
!
!
!
while ((c = fgetc(f)) != -1)
{
if (!isspace(c))
{
if (!w)
++n, w = 1;
else
w = 0;
}
}
fclose(f);
return 0;
}

12. Code to Model Example
#include <stdio.h>
#include <stdlib.h>
!
int main()
{
FILE* f;
int w = 0;
enum {FALSE,TRUE} B1, B2, B3;
!
if (B1 = any())
{
return 1;
}
!
f = fopen(“sample”, "r");
if (f == NULL)
{
return 1;
}
!
!
!
!
!
while (B2 = any())
{
if (B3 = any())
{
if (!w)
w = 1;
else
w = 0;
}
}
fclose(f);
return 0;
}

13. Code to Model Example
int main()
{
enum {FALSE,TRUE} B1, B2, B3;
enum {FCLOSED, FOPEN, FERROR} fileState;
enum {V0, V1} W = V0;
fileState = FCLOSED;
!
if (B1 = any())
{
return 1;
}
if (any())
{
fileState = FOPEN;
}
else
{
fileState = FERROR;
}
if (fileState == FERROR)
{
return 1;
}
!
!
!
!
!
while (B2 = any())
{
if (B3 = any())
{
if (!w)
W = V1;
else
W = V0;
}
}
fileState = FCLOSED;
return 0;
}

14. Code to Model Example
int main()
{
enum {FCLOSED, FOPEN, FERROR} fileState;
!
fileState = FCLOSED;
!
if (any())
{
fileState = FERROR;
}
else if (any())
{
fileState = FOPEN;
while (any());
fileState = FCLOSED;
}
return 0;
}
!
entry
FCLOSED1
FERROR FOPEN
FCLOSED2
exit

15. Formal Verification Techniques
• Theorem Proving
• Model Checking
• Ad hoc voodoo magic
!
!
!
!
!
Although these appear to be contradictory approaches, there has been considerable
effort to incorporate theorem proving and model checking

16. Theorem Proving
• Logical deduction at the intersection of
mathematics and logics
• Prove that implementation satisfies specification
by mathematical reasoning
• implementation and specification as formulae in a some
formal logic
• required relationship (equivalence/implication) as a
theorem to prove by using a set of axioms and inference
rules (simplification, induction, etc.)

17. Theorem Proving
by hand – classic approach
• human errors (similar to programming)
• unsuitable for verifying large software and
hardware systems

18. Theorem Proving
with some machine support – machine
checks the syntax, fixes layout, translates into
different (meta)languages
• less errors, readable consistent statements
• still unsuitable for verifying large software and
hardware systems

19. Theorem Proving
interactive – proofs are fully checked by machine
based on proof steps provided by user
• guaranteed correctness (if the prover is correct)
• can prove everything that is provable by hand

20. Theorem Proving
automated – theorem is shown by machine; user
must state it in a proper form so that the prover can
solve the problem within reasonable amount of
time
• may be used by non-specialists
• may be faster than interactive proving
• many problems cannot be proved automatically
• can often deal only with finite problems

21. Limitations of Theorem Proving
• Large manpower investment to prove small
theorems, difficult to prove large or hard
theorems
• Usable only by experts, requires deep
understanding of both the system design and
proving methodology
• Automated for narrow class of system designs

22. Model Checking
Automated technique that, given a finite-state
model of a system and a formal property,
systematically checks whether this property holds
for (a given state in) that model.
!
Modelling → Running → Analysing

23. Modelling
• Model the system under consideration using the
model description language of the model
checker at hand
• As a first sanity check and quick assessment of
the model perform some simulations
• Formalise the property to be checked using the
property specification language.

24. Running
• Run the model checker to check the validity of
the property in the system model.

25. Analysis
• Property satisfied?
• Check next property (if any)
• Property violated?
• Analyse generated counterexample by simulation
• Refine the model, design, or property
• Repeat the entire procedure.
• Out of memory?
• Try to reduce the model and try again.

26. Temporal Logic Model Checking
• Specifications are written in propositional
temporal logic to specify time related system
behaviour properties either in absolute time or
in terms of sequences of state transitions
• Verification is an exhaustive search of the state
space of the design

27. Basic Temporal Operators
Let p be an atomic proposition, e.g. device_Enabled
• Xp – p holds next time
• Fp – p holds sometime in a future
• Gp – p holds globally in a future (always)
• pUq – p holds until another atomic proposition
q holds
!
Use with ¬∧∨ etc. to form temporal logic formulae

28. Temporal Logic Formula Example
The microwave oven does not heat up until
the door is closed
!
Not heat_up holds until door_closed
!
(¬heat_up) U door_closed

29. Temporal Logic Formula Example
The microwave oven does not heat up until
the door is closed
!
Not heat_up holds until door_closed
!
(¬heat_up) U (door_closed ∧ powered_on)

30. Temporal Logic Model Checking Goals
Safety – something bad will never happen.
To satisfy, the system must not demonstrate
forbidden behaviour
!
Liveness – something good will happen in future.
To satisfy, the system must demonstrate desired
behaviour

31. Model Checking: Older Approaches
• Construct a model of a system in terms of a
finite state machine
• Choose a property, express it in some sort of
temporal logic and use a tool to explore the
state space and to check if the property holds
• Explicit-state checker enumerates all possible states
[SPIN]
• Symbolic checker uses binary decision diagram (BDD) to
encode the state transition function [NuSMV etc.]

32. Model Checking: Older Approaches
• Good for liveness and safety properties
• “is a state with property p reachable?”
• “if I see x, will I eventually see y?”
• “is property p true for all states?”
• Ill-suitable for functional properties
• “does this function compute AES-256?”

33. Model Checking: Newer Approaches
SAT-based model checkers
• Bounded model-checking (BMC) – an otherwise
large or even infinite-state FSM can be analysed
by checking a finite number of state transitions
• Property directed reachability (PDR) – combination
of induction, over-approximation and SAT
solving

34. Model Checking: Newer Approaches
Going into details of SAT and SMT solvers is
probably too much to handle for this session.
!
X(explain_sat ∧ explain_smt)

35. Model Checking Strengths
• General verification approach that is applicable
to a wide range of systems
• Supports partial verification (properties can be
checked individually, essential ones first)
• Provides diagnostic information in case a
property is invalidated (useful for debugging)

36. Model Checking Strengths (cont.)
• Requires neither a high degree of user
interaction nor a high degree of expertise
• Can be easily integrated in existing
development cycles
• Sound mathematical underpinning, based on
theory of graph algorithms, data structures,
logic etc.

37. Model Checking Weaknesses
• Mainly for control-intensive systems, less
suitable for data-intensive ones
• Verifies a model and not the actual system itself;
complimentary techniques are needed (testing
etc.)
• Check only stated requirements, completeness is
not guaranteed

38. Model Checking Weaknesses (cont.)
• Suffers from the state-space explosion problem
• Requires some expertise in finding appropriate
abstractions to obtain smaller system models
and to state properties in the logical formalism
used
• Does not allow checking generalisations
• A model checker may contain software defects

39. Model Checking
Provided various approaches, it is an effective
technique to expose design errors

40. Formally Verified System
Design
Model
Implementation
theorem proving
model checking
logic/syntax checkers
testing

41. Thank you
This presentation is based on materials provided courtesy of and various publicly available Internet sources.
All grammar and syntax errors are, let’s say, for watermarking purposes
2013