Tdd ec-agile2012

Topics
TDD for Embedded C
Talk to us on Twitter
http://twitter.com/jwgrenning
• The why and what of TDD
http://twitter.com/basvodde • Embedded adaptation
Find us on linkedin.com • Abstracting HW and OS
http://www.linkedin.com/in/jwgrenning
http://www.linkedin.com/in/basvodde • Mocking the silicon
Remind us how we met.
• Test-double substitution options
http://www.renaissancesoftware.net
http:// www.jamesgrenning.com Scaling Lean & Agile
Practices for
Scaling Lean & Agile
• Function pointer substitution
Development Development
http://www.odd-e.com Thinking and Organizational Tools
for Large-Scale Scrum

Craig Larman
Bas Vodde
Large, Multisite, and Offshore Products
with Large-Scale Scrum

Craig Larman
Bas Vodde
• Preprocessor substitution
• TDD next to an RTOS
Come to a full version of James’ TDD
for Embedded C
October 2012. Phoenix, AZ

www.renaissancesoftware.net www.renaissancesoftware.net
Copyright © 2008-2012 James W. Grenning james@renaissancesoftware.net 1 Copyright © 2008-2012 James W. Grenning james@renaissancesoftware.net 2
All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX basv@odd-e.com All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX basv@odd-e.com

This Work Flow is Designed to Allow
Defects

Why TDD?
What is TDD? Development

Test

Defects

The Physics of Debug Later
Your program will have Programming (DLP)
bugs. And they will
surprise you when you Td Tfind T fix

find them.
Time

Mistake made Bug discovery
(bug injection)
Bug found Bug fixed

• As Td increases, Tfind increases dramatically
• Tfix is usually short, but can increase with Td


A Bug’s Life Edsger Dijkstra
Those who want really reliable
software will discover that they
must find means of avoiding the
majority of bugs to start with, and
as a result, the programming
process will become cheaper. If you
want more effective programmers,
you will discover that they should
not waste their time debugging,
they should not introduce the bugs
to start with.
From http://www.softwaretestinghelp.com/bug-life-cycle/
All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX basv@odd-e.com All Rights Reserved. For use by training attendees. basv@odd-e.com

Can we Realize Being g
Dijkstra’s Dream and ood at c
is not T hasing
Prevent Defects with echnica bugs
l Excelle
Test Driven Development? nce

All Rights Reserved. For use by training attendees. basv@odd-e.com All Rights Reserved. For use by training attendees. basv@odd-e.com

Development and Test Work Together
Preventing Defects A Complex system that
works is invariably
found to have evolved
from a simple system
Development that worked.

Test


The Physics of Test Driven
Why is it difficult to sustain?
Development
T d Tfind T fix

Time

Mistake
Mistake fixed
made
Mistake Root cause
discovery found

• When Td approaches zero, Tfind approaches zero TDD: Traditional:
• In many cases, bugs are not around long enough to be considered • Steady pace • Spurts
bugs. • Speed limits • Fast when no problems!
• See: http://www.renaissancesoftware.net/blog/archives/16 • No traffic lights (bugs) • Debugging
• Might feel slow!! • Feels fast! But often slow

Sustainability Time developers do
not notice nor plan for

Traditional Speculate Code Test Debug
development
Demo and Exercise
Time vs
Feels
slower

Test-driven
Spec
development ulate
Test and Code Debug

www.renaissancesoftware.net
Copyright © 2008-2012 James W. Grenning james@renaissancesoftware.net 15 Copyright © 2008-2012 James W. Grenning www.renaissancesoftware.net 16
All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX basv@odd-e.com All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX james@renaissancesoftware.net

Cheat sheet
/* in CheatSheetTest.cpp */
#include "CppUTest/TestHarness.h"

/* Declare TestGroup with name CheatSheet */
TEST_GROUP(CheatSheet)
{
/* declare a setup method for the test group. Optional. */
void setup ()

CppUTest Quick Intro
{
/* Set method real_one to stub. Automatically restore in teardown */
UT_PTR_SET(real_one, stub);
}

/* Declare a teardown method for the test group. Optional */
void teardown()
{
}
}; /* Do not forget semicolumn */ /* In allTest.cpp */
IMPORT_TEST_GROUP(CheatSheet);
/* Declare one test within the test group */
TEST(CheatSheet, TestName) /* In main.cpp */
{
/* Check two longs are equal */ #include "CppUTest/CommandLineTestRunner.h"
LONGS_EQUAL(1, 1); #include "AllTests.h"

/* Check a condition */ int main(int ac, char** av)
CHECK(true); {
return CommandLineTestRunner::RunAllTests(ac, av);
/* Check a string */ }
STRCMP_EQUAL("HelloWorld", "HelloWorld");
}

Test Fixture Output Format
Data needed
in each test. TEST_GROUP(TemplateEngineTest)
Run before
{ each test No news is good news
Template aTemplate;
TemplatePlaceholderValues replacementValues;
./TestFirst
EmailFormatter *emailFormat;
.
void setup() {
OK (1 tests, 1 ran, 1 checks, 0 ignored, 0 filtered out, 0 ms)
emailFormatter = createMockFormatter();
aTemplate.setEmailFormat(emailFormat);

The data and
}
void teardown(){
Give precise information when fails
destroyFormatter(emailFormatter); ./TestFirst
setup / teardown }
is also }; Run after TestFirst.cpp:10: error: Failure in TEST(FirstTest, First)
make: *** [all] Error 1
sometimes TEST(TemplateEngineTest,
templatesWithoutPlaceHoldersDoNotChange)
each test expected <1 0x1>

called: {
but was <0 0x0>

test ﬁxture aTemplate.set("Nothing here");
STRCMP_EQUAL("Nothing here",
.
Errors (1 failures, 1 tests, 1 ran, 1 checks, 0 ignored, 0 filtered out, 0 ms)
aTemplate.replaceValues(replacementValues).c_str());
}
49

More info

• CppUTest Github Project page:
– http://cpputest.github.com/cpputest/ Adaptation for Embedded
• The CppUTest Github repository: Software
– https://github.com/cpputest/cpputest

• The CppUTest Man page:
– http://www.cpputest.org/


What are the special What is special about embedded
challenges you have over a
• Concurrent HW development
non-embedded developer? • HW bottleneck
• Cross compilation
• Limited memory and IO
• Target debug
• Architecture

Copyright © 2008-2012 James W. Grenning www.renaissancesoftware.net 23 Copyright © 2008-2012 James W. Grenning james@renaissancesoftware.net 24
All Rights Reserved. For use by training attendees. james@renaissancesoftware.net All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX basv@odd-e.com

Hardware is
Scarce!
• It does not exist.
• It is being used by
someone else.
• It has bugs of its
own.


All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX basv@odd-e.com All Rights Reserved. For use by training attendees. basv@odd-e.com

Minimize DoH!

Debug
on
Copyright Fox Broadcasting. Used under fair use.
Hardware
All Rights Reserved. For use by training attendees. basv@odd-e.com All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX basv@odd-e.com

Why Use Your Development System How to Use Your Development
for a Test Bed? System for a Test Bed?
• Waiting for hardware. • Multi-targeted code.
• Waiting for restart. • Must make code portable
• Waiting for downloads. • Must beware of hardware and OS dependencies.
• Waiting for long compiles. • Object Oriented approach to Dependency
• Debugging on the target. Management.
• Target has bugs.
• (Re)Configuring the lab

Avoid wasteful practices

But There are Risks with TDD Adaptation for Embedded
Development System Tests Development
• Architecture differences
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
– Word size
– Big-endian Little-endian
– Alignment Write a Test Compile Run Tests Run Tests Acceptance
Make it Pass for Target in the Eval in Target Tests
• Compiler differences Refactor Processor Hardware Hardware
• Library differences
• Execution differences
More Frequent Less Frequent


TDD Adaptation for Embedded Continuous Integration - Embedded
Development 3. CIS notices changes, refreshes
its local copy, builds and runs host

2. Developer
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 checks in work
Source Code Continuous Integration Test Management
regularly.
Repository Server System
4. After
successful host
build/test, CIS
Write a Test Compile Run Tests Run Tests Acceptance builds and hands
Make it Pass for Target in the Eval in Target Tests images to TMS to
deploy to
Refactor Processor Hardware Hardware simulator, eval/
reference boards
1. Developer has or target system.
all unit tests

Eval/Reference
More Frequent Less Frequent System
Developer Simulator
Target System
Workstation
See : http://renaissancesoftware.net/files/articles/ProgressBeforeHardware.pdf
The SCR, CIS and TMS are not necessarily
separate hardware systems.
Copyright © 2008-2012 James W. Grenning james@renaissancesoftware.net 35 Copyright © 2008-2012 James W. Grenning
35 james@renaissancesoftware.net

Problems Found at Appropriate Stage
Stage Problems Likely to Find in Stage

1 Logic, design, modularity, interface, boundary conditions

2 Compiler compatibility (language features)
Library compatibility (header files, declarations)
Hardware and OS Abstraction
3 Processor executions problems (compiler and library bugs)
Portability problems (word size, alignment, endian)

4 Ditto stage 3
Hardware integration problems
Misunderstood hardware specifications

5 Ditto stage 4
Misunderstood feature specification


Separation of Responsibilities Light Scheduler Design
• Every minute, the
RTOS wakes up
Admin
Console
Light
Scheduler
• Program to Admin
Console
Light
Scheduler

the Light +ScheduleTurnOn() Interfaces + ScheduleTurnOn()
+RemoveSchedule() + RemoveSchedule()
Scheduler. +WakeUp() <<anonymous callback>> • Separate interface +WakeUp() <<anonymous callback>>

• If it is time for and implementation
as separate entities.
<<interface>> <<interface>>
Light Controller Time Service
one of the lights to + On(id) + GetTime()
Light Controller
+ On(id)
Time Service
+ GetTime()
be controlled, the + Off(id) + SetPeriodicAlarm() • This design has + Off(id) + SetPeriodicAlarm()

LightController is good separation of <<implements>> <<implements>>

told to turn on/off
Hardware RTOS
responsibilities Model 42 RTOS
Light Controller Time Service
the light.
Model 42 Hardware RTOS


LightScheduler Test Fixture Design Testing the Scheduler
Light TEST(LightScheduler, light_not_changed_at_the_wrong_time)
• Use the real Scheduler
Light
Scheduler {
Test
collaborators if + ScheduleTurnOn()
LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1200);
FakeTimeService_SetMinute(1199);
+ RemoveSchedule()
you can. +wakeUp() LightScheduler_Wakeup();
LONGS_EQUAL(NO_LIGHT_ID, LightControllerSpy_GetLastId());
• Use fakes when LONGS_EQUAL(LIGHT_STATE_UNKNOWN, LightControllerSpy_GetLastState());
}
you must. <<interface>>
Light Controller
<<interface>>
Time Service
+ On(id) + GetTime() TEST(LightScheduler, light_changed_at_the_right_time)
+ Off(id) + SetPeriodicAlarm() {
FakeTimeService_SetMinute(1200);
<<implements>> <<implements>> LightScheduler_Wakeup();
Light Controller Fake LONGS_EQUAL(3, LightControllerSpy_GetLastId());
Spy Time Service LONGS_EQUAL(LIGHT_ON, LightControllerSpy_GetLastState());
}


Light Scheduler Test List
A Complex system that
Light Scheduler Tests works is invariably
Lights are not changed at initialization
found to have evolved
Time is wrong, day is wrong, no lights are changed
Day is right, time is wrong, no lights are changed from a simple system
Day is wrong, time is right, no lights are changed
Day is right, time is right, the right light is turned on
Day is right, time is right, the right light is turned off
that worked.
Schedule every day
Schedule a specific day
Schedule all weekdays
Schedule weekend days
Remove scheduled event
Remove non-existent event
Multiple scheduled events at the same time
Multiple scheduled events for the same light
Remove non scheduled light schedule
Schedule the maximum supported number of events (128)
Schedule too many events


Partial Skeleton Let’s Your Try the Copy/Paste THINK
Design and Test Ideas Early
TEST(LightScheduler, light_not_changed_at_the_wrong_time)
{
TransitionClockTo(SUNDAY, 1199);
ExpectLightsUnchanged();
}

TEST(LightScheduler, light_changed_at_the_right_time)
{
TransitionClockTo(SUNDAY, 1200);
ExpectLightOn(3);
}


Replacing collaborators
• Linker-stubbing
Test-Double Substitution • Function pointer
• Pre-processor
Options


Linker Stubbing
#include "sqlite3.h"

int sqlite3_open(const char *filename, sqlite3 **ppDb)
{
FAIL("sqlite3_open");
return 0;
}

Linker Substitution int sqlite3_step(sqlite3_stmt*)
{

FAIL("sqlite3_step");
return 0;
}
Don’t link the real library.
int sqlite3_reset(sqlite3_stmt *pStmt)
{ Instead provide a fake

FAIL("sqlite3_reset");
return 0; that does something
}

sqlite3_int64 sqlite3_last_insert_rowid(sqlite3*)
useful for the test
{
FAIL("sqlite3_last_insert_rowid");
return 0;
}

int sqlite3_bind_int64(sqlite3_stmt*, int, sqlite3_int64)
{
FAIL("sqlite3_bind_int64");
return 0;
}


Use linker stubs across subsystems

Interface TinyXML
Function Pointer Substitution
• Use linker stubs on other
Sqlite
subsystems
• Linker stubs requires no
change in code!
Engine dbus
✔ fzshelltext

putty

Reasons to Use Function Pointers for
Example Usage - Printed Output
Test-Doubles
• When you already use function pointers for other • Logs and printed output are helpful for checking
purposes (you have a built-in test hook!) program correctness and debugging
– Swappable device drivers • But...
– Swappable implementations – They require that you look at the output.
• You need the production implementation in the test – They doom you to a lifetime of manually verified tests
build, and you must substitute a test double in the • So...
same test build – Design your code so that printed output can be captured
– printf and verified in your unit tests
• You don’t want to have too many test builds due to
using link-time substitution


Manual/Tedious/Error Prone Function Pointer - Runtime
Output Inspection Substitution
• Sometimes we want to use the real function.
• Sometimes we want to use a test version.
• Define a function pointer that has the same
declaration as the function to override, in this case
printf()
#ifndef D_FormatOutput_H
#deﬁne D_FormatOutput_H

extern int (*FormatOutput)(const char *, ...);

#endif // D_FormatOutput_H


By default, FormatOutput points to the Production Code
printf Before:
void someProductionCode()
{
printf(“hello %sn”, “world”);
//FormatOutput.c }
#include "FormatOutput.h"
#include <stdio.h>
After:
int (*FormatOutput)(const char* format, ...) = printf; #include "FormatOutput.h"

. . . . .

void someProductionCode()
{
FormatOutput(“hello %sn”, “world”);
}


Runtime Substitution
UT_PTR_SET
Test setup and teardown
• We can override the function by assigning the test • CppUTest has a mechanism for overriding and
version of the function during setup restoring pointers that must be restored after each
• Don’t forget to restore the original test.
• How should I do this more safely? • UT_PTR_SET() assigns the pointer and restores the
original function pointer after teardown()
void setup()
{
void setup()
FormatOutput = FormatOutputSpy;
{
}
UT_PTR_SET(FormatOutput, FormatOutputSpy);
}
void teardown()
{
void teardown()
FormatOutput = printf;
{
}
}

Sometime we Write Tests for Test Code Spy Overflow
We must be able to trust the spy.
Tests are documentation!

TEST(FormatOutput, ATestThatPrintsThings) TEST(FormatOutput, LimitTheOutputBufferSize)
{ {
FormatOutputSpy_Create(20); FormatOutputSpy_Create(4);
FormatOutput("Hello, Worldn"); FormatOutput("Hello, Worldn");
STRCMP_EQUAL("Hello, Worldn", STRCMP_EQUAL("Hell", FormatOutputSpy_GetOutput());
FormatOutputSpy_GetOutput()); }
}


More Checking on our Spy Spying on the Output

TEST(CircularBufferPrint, PrintNotYetWrappedOrFull)
{
CircularBuffer_Put(buffer, 10);
TEST(FormatOutput, PrintMultipleTimes) CircularBuffer_Put(buffer, 20);
{ CircularBuffer_Put(buffer, 30);
FormatOutputSpy_Create(25); CircularBuffer_Print(buffer);
FormatOutput("Hello");
FormatOutput(", Worldn"); expectedOutput = "Circular buffer content:n<10, 20, 30>n";
STRCMP_EQUAL("Hello, Worldn", STRCMP_EQUAL(expectedOutput, FormatOutputSpy_GetOutput());
FormatOutputSpy_GetOutput()); }
}


Test Fixture
TEST_GROUP(CircularBufferPrint)
{
CircularBuffer buffer;
const char * expectedOutput;
const char * actualOutput;
Preprocessor Substitution
void setup()
{
UT_PTR_SET(FormatOutput, FormatOutputSpy);
FormatOutputSpy_Create(100);
buffer = CircularBuffer_Create(10);
}

void teardown()
{
CircularBuffer_Destroy(buffer);
FormatOutputSpy_Destroy();
}
};

Copyright © 2008-2012 James W. Grenning james@renaissancesoftware.net 65 Copyright © 2008-2012 James W. Grenning www.renaissancesoftware.net 66
All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX basv@odd-e.com All Rights Reserved. For use by training attendees. Agile 2012, Grapevine, TX james@renaissancesoftware.net

Preprocessor Substitution Preprocessor Substitution Variants

• Is the least desirable form of substitution • Header file test double
• Though necessary when – Change the include path during test builds
– Header files won’t compile off-target • Force includes
– Header files have are the start to a massive dependency – Force in a header file that substitutes problem
chain dependencies
– A direct function call API cannot be changed and you • Command line definition
need to override the direct call and use the direct call in – Override a symbol one at a time
the implementation of the fake
e.g. malloc(), free()


Tdd ec-agile2012

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Tdd ec-agile2012

Ähnlich wie Tdd ec-agile2012 (20)

Mehr von drewz lin

Mehr von drewz lin (20)

Tdd ec-agile2012