7. 100% code coverage does not mean:
the compiler translated your code correctly
const volatile int x;
volatile int y;
void foo(void){
for(y=0; y>10; y++)
{
int z = x;
}
}
foo:
mov1 $0, y
mov1 x, %eax
jmp .L2
.L3:
mov1 y, %eax
incl %eax
mov1 %eax, y
.L2:
mov1 y, %eax
cmp1 $10, %eax
jg .L3
ret
At the -0s optimization level, GCC 4.3.0 for IA32
compiles the C code on the left to the assembly on
the right.
The compiler output is wrong: the access to volatile-
qualified variable x should not be hoisted out of the
loop.
8. 100% code coverage does not mean:
you have covered 100% of your object code
Source code Object codeCompiled to
9. Does it matter which code coverage criteria I'm
using?
Coverage type DO-178B/C
ISO 26262
(Software architecture)
ISO 26262
(unit test)
Function Used with MC/DC
ASIL C, D Highly
Recommended
ASIL A, B Recommended
-
Call Coverage Not required
ASIL C, D Highly
Recommended
ASIL A, B Recommended
-
Statement
Level A, B, C
Required
-
ASIL A, B Highly
Recommended
ASIL C, D Recommended
Decision
Level A, B
Required
- -
Branch Not required -
ASIL B, C, D Highly
Recommended
ASIL A Recommended
MC/DC Level A Required -
ASIL D Highly Recommended
ASIL A, B, C Recommended
10. What are the barriers to 100% code coverage?
Missing requirements.
Missing or incorrect tests.
Dead code.
Deactivated code.
Defensive programming.
Impossible combinations of events.
Compiler-introduced errors.
11. Barrier #1: Missing Requirements
Add new requirements
Develop tests
Run the tests
What to do about it
12. Barrier #2: Missing/incorrect tests
What to do about it
Implement additional tests
Ensure that they trace to
the correct requirement(s)
Run the new tests
Implement additional tests
Ensure that they trace to
the correct requirement(s)
Run the new tests
What to do about it
13. Barrier #3: Dead code
foo(a,c){
int b;
mode = MOVING;
if(mode == STATIONARY){
b = 1 + a;
}
c = a;
return c;
}
Remove the dead code if
possible or
Justify why the code cannot
be run
What to do about it
DO-178C recommends "The [dead] code should be removed and
an analysis performed to assess the effect and the need for
reverification." [6.4.4.3c]
For ISO 26262, the general guidance for "insufficient" coverage
applies, namely to provide a rationale for why the dead code is
acceptable [6:9.4.5].
What to do about it…
14. Barrier #4: Deactivated code
settings = read_input(CONFIG_PINS);
if ((settings & CONFIG_MASK_A) == CONFIG_A) {
/* code to execute in configuration A */
...
} else if ((settings & CONFIG_MASK_B) == CONFIG_B) {
/* code to execute in configuration B */
...
}
Justify why code won’t be
executed
Never executed
during normal
operation
Will not affect the
execution of the
system
What to do about it
15. Barrier #5: Defensive Programming
switch(a % 4){
case 0:
..........
break;
case 1:
..........
break;
case 2:
..........
break;
case 3:
..........
break;
default:
..........
break;
}
Justify why defensive
programming cannot be
executed
Provide evidence that if
triggered, the defensive
programming works
correctly
What to do about it
16. Barrier #6: Impossible Combinations of Events
-- “enabled” might already be set at this point
if speed < 1700 then
enabled := true;
end if;
if speed < 1500 and enabled then
...
end if;
Simplify the condition?
or
Justify why 100% could not
be achieved
What to do about it
17. Barrier #7: Compiler Introduced Errors
int a,x;
if a < 10{ /*Compiler generates “a <=10”*/
result = x / 5;
} else {
result = x / 10;
}
a x
Test case 1 3 4
Test case 2 10 4
Review existing code base
Add to coding guidelines /
code review guidelines
What to do about it
18. Justifying code: in-line justifications
void fatal_error (void)
{
#pragma RVS justification( "COV_FUNCTIONS", "The fatal_error
function cannot be executed during normal execution")
/* some code */
}
...
void check_status (int status)
{
if ( (check_enabled) && (FATAL_ERROR == status) ) {
#pragma RVS justification( "COV_DECISIONS", "Variable 'status'
cannot take value FATAL_ERROR during normal execution")
fatal_error();
}
}
...
19. Justifying code: justifications in a separate file
#pragma RVS [selftest.c:34] justification("COV_MCDC", "2: (RAM_ERROR ==
status)", "RAM_ERROR cannot be triggered during normal test");
#pragma RVS [selftest.c:439] justification("COV_FUNCTIONS", "Cannot trigger
emergency_shutdown() during normal test");
22. Review – establish
missing coverage
Identify cause Identify action to take
Dealing with less than 100% code coverage
23. Review – establish
missing coverage
Identify cause Identify action to take
Dealing with less than 100% code coverage
Missing requirements.
Missing or incorrect tests.
Dead code.
Deactivated code.
Defensive programming.
Impossible combinations of events.
Compiler-introduced errors.
24. Review – establish
missing coverage
Identify cause Identify action to take
Dealing with less than 100% code coverage
Remove code
Add (or fix)
requirements
Justify why code
won't be executed
Justify why it isn't
possible to test
25. reduce testing timescales by running fewer on-target tests
reduce risk through greater tool flexibility
reduced effort for certification activities
Structural coverage analysis to DO-178B/C Level A and ISO 26262
26. Logos and images can be positioned within this area.
Change layout to blank (without position box) before use