The document discusses agile engineering practices and concepts. It begins by outlining objectives like understanding practices teams can use to handle complexity and the benefits they provide. The agenda then covers topics like what is meant by agile engineering, how practices relate to agile principles, iterative processes, user stories, and specific practices like source code control, pair programming, continuous integration, and testing. It emphasizes that these practices help produce quality code, allow for reliable and frequent deployments, and protect teams from issues.
2. Agile Engineering Objectives
We want to answer the following Qs:
• What practices can Agile teams use (to handle
increasing complexity)?
• Conceptually, how do they work?
• What benefits can a team derive from them?
• What are some common challenges in
implementing them?
• What tools can be used to support them?
3. Agile Engineering Agenda
• What do we mean by Agile Engineering?
• What does this mean in terms of change?
• What are the relationships of engineering practices to..?
– The Manifesto
– Iterative and Flow-based Processes
– User Stories
• Agile Engineering Practices
– Source Code Control
– Pair Programming
– Continuous Integration
– Testing; Types of Testing, TDD/BDD, etc.
– Refactoring
– Continuous Delivery
• Debrief
– Reminder on Change
5. Meet Mr. Spock
Mr. Spock develops software.
He wants to develop software his users
will love.
It’s creative problem solving. (How logical.)
He wants to deploy it often so his users
can love him more.
He wants tools and techniques to help
him accomplish these things.
6. Meet Mr. Scott
Mr. Scott runs systems.
He wants the systems to run reliably.
Problem-solving almost always means
bad news for him.
Users usually do not love it when he
has to do problem-solving.
Fast deployments hopefully don’t
mean more work and outages.
7. When things don’t go well…
CAPT Kirk, the CIO, shows more
emotion than Mr. Spock would like.
You’d thing the Klingons had taken his
son or something…
8. Agile engineering practices help us…
Protect us from ourselves
Produce quality code
Deploy often and reliably
10. An Abbreviated History
Lesson
1970:
2006: Dan North
XP Invented
Most engineering
practices come
from XP
1968: Edsger Dijkstra
Forerunner of XP/TDD
is Constructive
Approach to Dev
1996: Kent Beck
Project Mercury
Origins of
Test-First
William Royce
Waterfall
BDD Invented
Automated
Acceptance Tests
2010:
Manifesto
Ola Ellnestam
Daniel Brolund
Mikado Method
Safely Refactor
Legacy Code
2001
12. Adopting
Agile Engineering Practices
Introduces change for most teams
Understanding how change affects teams is important
13. The Satir Change Curve
Status Quo
New Status Quo
disruption
amount
disruption
time
A detailed depiction of the Satir Change model -
http://stevenmsmith.com/ar-satir-change-model/
14. The Satir Change Curve
Status Quo
New Status Quo
disruption
amount
disruption
time
Allow teams time to learn
A detailed depiction of the Satir Change model -
http://stevenmsmith.com/ar-satir-change-model/
Give teams
encouragement
Expect a
degradation
15. How does
Agile
Engineering
relate
to
the
Agile Manifesto?
16. Manifesto for Agile Software
Development
We are uncovering better ways of developing software
by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes
and tools Working software over comprehensive
documentation Customer collaboration over
contract negotiation Responding to change over
following a plan
That is, while there is value in the items on the right, we
17. We follow these principles:
Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.
Welcome changing requirements, even late in
development. Agile processes harness change for
the customer's competitive advantage.
Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.
Business people and developers must work
together daily throughout the project.
Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.
18. The most efficient and effective method of
conveying information to and within a development
team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
Continuous attention to technical excellence
and good design enhances agility.
Simplicity--the art of maximizing the amount
of work not done--is essential.
The best architectures, requirements, and designs
emerge from self-organizing teams.
At regular intervals, the team reflects on how
to become more effective, then tunes and adjusts
its behavior accordingly.
19. How does
Agile
Engineering
relate
to
the agile processes?
20. Iterative Process
Acceptance
Iteration Iteration
Deploy Deploy
& Unit
Testing
Continuous
Integration
Continuous
Integration
Acceptance
& Unit
Testing
21. Flow-Based (Kanban)
Backlog Accept Tests Dev Deploy Done
Deploy
Acceptance
Tests
Written
Unit
Testing
Acceptance
Testing
Continuous
Integration
23. User Stories
Planning
[WBS]
through prioritized ordering of
stories into backlog
Story as
backbone
of Agile
Release
through acceptance
criteria of story
through combination of
story and Definition of Done
Requirements Testing
24. User Stories
Planning
[WBS]
through prioritized ordering of
stories into backlog
Story as
backbone
of Agile
Release
through acceptance
criteria of story
through combination of
story and Definition of Done
TDD
Requirements Testing
CODE
28. Source Code Control
• Fundamental to almost every other technical
practice
• Does Ø good w/out frequent code check-ins
• Provides the safety net for
making changes
29. What is wrong with using a source
code repository like this?
Shared File Space
Release
Candidate
Declare
Release
Release
Who is this?
30. What considerations do you need to
think about if you do it this way?
Release Who is this?
Candidate
Declare
Release
Release
31. What considerations should we
consider if developers horde code and
don’t commit it to the repository?
Declare
Release
Release
Release
Candidate
X
34. What is Pair Programming?
In a classic
it’s
XP sense,
2 Developers, 1 Keyboard
35. Pair Programming Upside
• Better Code
– Peer review baked in
– Different ways to approach need
• Faster Output
– Design decisions/implications thought through
– Once people feel comfortable, throughput tends to go
up
• Shared Understanding of Work
– More people know code/decisions
– Higher Bus Lottery Factor
36. Pair Programming Flipside
• Personality Fits Are Important
– Not everyone feels comfortable exposing their work
– Need to avoid dominant controller
– Good listening skills
• Breather Time
– Need space/time to selves, particularly for introverts
– Consider allowing OK to check out
– Space for people to go to…
• Expect Some ‘Adjustment’
– Human dynamics of learning to share space/thoughts
38. Pair Poetry
We’re going to simulate two
people working on the same
code in the form of poetry.
There once was a man from Nantucket….
39. Pair Poetry
Round 1:
• Pair Up; you’ll each need a pen and a piece of paper, and two story dice.
• Each of you is going to write a Limerick together;
– Limericks follow an end of a sentence rhyming pattern of A-A-B-B-A
– 1st, 2nd, and 5th lines are 8-9 syllables
– 3rd and 4th lines are 5 syllables
– They are almost always humorous, if not a little bawdy
• We’ll first write a Limerick that isn’t using pairing
– Lines 1 and 3 and the first 3 words on the last line are written by the first person
– Lines 2 and 4 and the last set of words on the last line by the second
• Here’s how to play:
– Roll the story dice, decide who will take the one picture and the other the 2nd
picture. Decide who goes first….
– Put your name on your sheet, whether you are 1st or 2nd and write your lines…
– No discussion
• Once you complete your part, put your hands up; I’ll collect your sheets
once I see both hands.
40. An Example Limerick…
There once was a young lady named Bright
Whose speed was much faster than light
She set out one day
In a relative way
And returned on the previous night.
41. Round 1 Debrief
• How well did the Limericks flow?
– Any roughness?
– Subject get off the rails?
– How about that last sentence?
– Did anyone mess up the rhyming?
• What did the dice represent? (Think from a coding
standpoint.)
• How about each line of the poem?
• What shared mental model did you have?
42. Pair Poetry
Round 2:
• Same pairs; you’ll each need a pen and will share a piece of paper, and two
story dice.
• Again we’ll write a Limerick together;
– Limericks follow an end of a sentence rhyming pattern of A-A-B-B-A
– 1st, 2nd, and 5th lines are 8-9 syllables
– 3rd and 4th lines are 5 syllables
– They are almost always humorous, if not a little bawdy
• We’ll now write a Limerick that ‘pairs’ sequentially
– Lines 1 and 3 and the first 3 words on the last line are written by the first person
– Lines 2 and 4 and the last set of words by the second
• Here’s how to play:
– Roll the story dice, SWAP who will take the one picture and the other the 2nd
picture. Decide who goes first….
– Put your name on your sheet, whether you are 1st or 2nd and write your lines…
– Take turns writing your lines passing the paper between you.
• Once both people have completed their parts, put your hands up; I’ll collect
your sheets.
43. Round 2 Debrief
• Any improvement in your Limerick’s flow?
– Any roughness?
– Subject get off the rails?
– How about that last sentence?
– Did anyone mess up the rhyming?
• How did it feel not being able to change a prior line?
• What shared mental model did you have?
• Is working sequentially on code like this a realistic
approach?
44. Pair Poetry
Round 3:
• Same pairs; you’ll each need only ONE pen and ONE piece of paper, and two
story dice.
• Again we’ll write a Limerick together;
– Limericks follow an end of a sentence rhyming pattern of A-A-B-B-A
– 1st, 2nd, and 5th lines are 8-9 syllables
– 3rd and 4th lines are 5 syllables
– They are almost always humorous, if not a little bawdy
• We’ll now write a Limerick that pairs fully
– Either of you can write, the other can comment, critique, provide input, etc.
– You can also REQUEST THE PEN!
– Change any line/word you want as long as both people agree
• Here’s how to play:
– Roll the story dice, use these two pictures when writing your Limerick
– Put both your names on your sheet and write your lines together…
• Once it is complete, put your hands up; I’ll collect your sheets.
45. Round 3 Debrief
• Any improvement in your Limerick’s flow?
– Any roughness?
– Subject get off the rails?
• What shared mental model did you have?
• Did you finish more quickly?
• How did you feel about the quality of your
limerick?
46. Beyond Pairs… Try Triads!
Dev Tester Dev
Dev BA Dev
Dev PO/SME Tester
Dev Tester BA
47. Some Tools for Remote Pairing
• RealVNC
• TeamViewer
• Windows Remote Desktop
• AeroAdmin
• Chrome Remote Desktop
• Remote Utilities
• join.me
• BeamYourScreen
53. CI w/Lego, Part 1
• Split into two pairs; each pair gets a kit of
Legos (either the left side or the right side), a
set of build specifications for their side, and an
8x8 Lego plate
• Read the instructions and then we’ll start all at
the same time
• When both the left and right side have
completed their structures, we are going to
integrate them
54. CI w/Lego, Part 2
• Stay in your two pairs; each gets a new kit of
Legos (either the left side or the right side), a
set of build specifications for their side, and
keeps their set of plates
• Read the instructions and then we’ll start all at
once; you will be working in 3 iterations
• At the end of each iteration, we are going to
integrate the structures
58. Agile Jenga :: Let’s Get Started
• Separate into pairs
• Each pair takes a pair of dice (one red, one
white), and a set of Jenga blocks; these are
numbered. (Some pair may get the more
challenging Jenga Extreme blocks…)
• Decide who will be the developer and who will be
the tester
• You will need a small sheet
of paper and a pen
59. Agile Jenga :: Building Requirements
The Developer will build a structure; it must
meet the following requirements:
• Use ALL Blocks
• It must be at least 3 “stories” tall (a story
consists of at least one horizontal piece and
one vertical piece)
We will build this structure
in 3 rounds, with each
person being the tester at
least once
60. Agile Jenga :: Round 1
Developer
• Build the structure
Tester
• At the End, Run Your Tests: Roll the Dice 4 times
(match the red and black numbers rolled to the
blocks)
• Blocks that match the rolls
must be removed by the
Developer and the tower
must be rebuilt if necessary
61. Agile Jenga :: Round 2
Developer
• Build 9 blocks of the structure
Tester
• At the end of each 9 Block iteration, Run Your Tests: Roll the
Dice 4 times minus any previously found bugs times, but
always at least once (match the red and black numbers, reroll
any already pulled)
• Blocks that match the rolls must
be removed by the developer and
the tower must be rebuilt if
necessary
Move onto the next iteration of 9
blocks to be built (4 iterations)
62. Agile Jenga :: Round 3
Tester
• At the Start, Run Your Tests: Roll the Dice 4 times and record the
numbers (reroll if any have been previously removed); keep these
“private”
Developer
• Select and build 9 blocks of the structure
Tester
• As the structure is built, remove
any matching numbered blocks as
you see them
Move onto the next iteration of 9
blocks to be built (4 iterations)
64. Benefits to Bringing Testing Forward
Release
Iteration
Day
• Finds defects quicker
(like before you are
ready to deploy a
release!)
• Defects are a form of
feedback! (too late &
the feedback is painful)
Deploy
65. Test to
Specification
Test to
Failure
Product
Design
Technical
Design
Interaction
Design
Acceptance
(Functional)
Tests
xUnit
Tests
UI
Tests
Exploratory
Testing
Stress
Testing
Security
Testing
Usability
Testing
(A/B
Testing)
Finite, Repeatable,
Automated
Environmental,
Less Automated
Diagram concept from p.76, Leading Lean
Development, Mary & Tom Poppendieck, 2010
A Full Complement of Tests
68. Traditional Testing
Approach
Requirements
&
Design Specs
Develop
Test Scripts
Error Found!
Execute
Test Scripts
Test Expectation
≠ Executable Implementation
Develop Code,
Compile Executable
69. WTF We’re supposed to be releasing
next week and you tell me this
isn’t working? Get that bug fixed!
70. The objective is
to kill bugs quickly,
but to
AVOID
them altogether!
71. What if…
Requirements Specs = Test Specs = Acceptance Criteria
We built the right thing.
Design Specs = Test Specs = Quality Criteria
We built the thing right.
Greater Detail
72. 1968
How do we know our
programs are correct?
1) Construct a proof showing what the
program should do
2) Develop code that
meets the proof
REPEAT until done
“A Constructive Approach
To the Problem of Program Correctness”, 1968, EdsgerWybe Dijkstra
73. In 1968,
in fact not until the late 90s,
the tools
didn’t exist
to implement Dr. Dijkstra’s vision.
Today,
the Constructive Approach = Test Driven Development (TDD)
(Specs = Tests = Proofs)
75. Greater Detail
Acceptance Tests Stories/Functions
Mark as ready for test when completed; based on Product Owner priority
xUnit Tests Specific Detailed Methods/Functions
Test completed code
76. Write
Test(s)
Execute Test
to
Show Failure
Create Mock Data
To
Show Passing Test(s)
Write Code
to
Pass Tests
Why?
Know test will show
fail
Why?
Tests = Specs
X Unit Tests
Why?
Know test will show
pass (expected
results)
Recurse over this pattern!
77. What does an xUnit
Test look like?
Say you've got a method,
getName(String path)
that parses a file name from a file path.
You can write a JUnit test class to feed it a path
and ensure that the results are what you
expected.
Simple example from jGuru.com
78. import java.io.File;
import junit.framework.*;
import junit.extensions.*;
public class ThingTester extends TestCase
{
public ThingTester (String name)
{
super (name);
}
public static void main(String[] args)
{
junit.textui.TestRunner.run(ThingTester.class);
}
public void testGetName() throws Exception
{
File myFile = new File("c:xxxyyyzzz.txt");
assertEquals("zzz.txt", myFile.getName());
}
}
79. To exercise the test:
Compile & run ThingTester.
• Little Dot = Passing Test;
• Assertion Failure & Stack Trace = Failure & location
• Write test code BEFORE program code used to
make it pass.
No problems? Add more test cases.
Examples: long paths, files with spaces in the name,
etc.
Stop when you feel you have enough edge cases
covered.
80. Understand Your Test Coverage
Tests should cover all critical methods
Tests should cover all critical edge cases
Tests should cover all critical error detection
Goal ≠ 100% coverage, but
Goal = 100% passing for what is covered
82. Write Test
Execute Test
to
Show Failure
Create Feature
To
Pass Test
Why?
Know test will show
fail
Why?
Tests = Specs
Acceptance Tests
Describe
Desired
Behavior
Describe
Feature
Test Steps
Recurse over this pattern!
Underneath…
…are Unit Tests
83. What does an Acceptance Test1
look like?
Say you've got a feature you want to implement
to compute a factorial…
You can write a feature that is the proof of this
function or story (including edge cases)
You also implement “steps” that poke at the
implementation. As the implementation changes
the steps may change, but it is less common for
the feature to change.
1aka Executable Specification
Example from tutorial at lettuce.it
84. computefactorial.feature
Feature: Compute factorial
In order to play with Lettuce
As beginners
We'll implement factorial
Scenario: Factorial of 0
Given I have the number 0
When I compute its factorial
Then I see the number 1
Business Users can get this!
85. computefactorialsteps.py
from lettuce import *
@step('I have the number (d+)')
def have_the_number(step, number):
world.number = int(number)
@step('I compute its factorial')
def compute_its_factorial(step):
world.number = factorial(world.number)
@step('I see the number (d+)')
def check_number(step, expected):
expected = int(expected)
assert world.number == expected,
"Got %d" % world.number
86. To exercise the test:
Run Feature (duh!)
Add more code to fix problems and re-run….
88. To exercise the test:
Re-run Feature (duh!)
Need more than one case of course, so let’s add more
test cases; i.e. feature scenarios
89. Feature: Compute factorial
In order to play with Lettuce
As beginners
We'll implement factorial
Scenario: Factorial of 0
Given I have the number 0
When I compute its factorial
Then I see the number 1
Scenario: Factorial of 1
Given I have the number 1
When I compute its factorial
In this case my steps
Then I see the number 1
}Given I have the number 2
When I compute its factorial
Then I see the number 2 Scenario: Factorial of 2
& asserts were OK,
sometimes you have
to create more
90. To exercise the test: Rerun Feature (duh!)
Change code to fix problems and re-run….
92. To exercise the test:
Rerun Feature (duh!)
Add a few more cases so we get realistic…
93. Feature: Compute factorial
In order to play with Lettuce
As beginners
We'll implement factorial
Scenario: Factorial of 0
…
Scenario: Factorial of 1
…
Scenario: Factorial of 2
…
Scenario: Factorial of 3
Given I have the number 3
Again my steps &
When I compute its factorial
Then I see the number 6
} Given I have the number 4
When I compute its factorial
Then I see the number 24 Scenario: Factorial of 4
asserts were Ok this
time,
sometimes you have
to create more
97. Acceptance Tests work best when:
• Driven off of methods (functions) or several functions
• Perform below the UI level
If needed, Acceptance Tests can:
• Be driven off of the UI (usually performs a little slower; must
be done this way for something like PowerBuilder or other 4GL
type of client/server environment)
• Can be done manually
The specification is independent of the flow or implementation.
The steps that poke at the underlying code may need modification
over time.
99. How often?
For unit tests…
Every time code (which includes a test) is changed… This is probably
multiple times a day and should be at a minimum daily.
For acceptance tests…
Whenever a test is completed and then the code is completed. The
test may be completed on Tuesday, then code that pokes at the test
will be completed as the skeletal object is completed say around
Wednesday, and then perhaps Friday is when all the code for that
poke is completed.
100. Some Testing Frameworks
Unit Testing
• pyUnit
• jUnit
• nUnit
• plUnit
• cfUnit
• FlexUnit
• Jasmine (JavaScript)
• Ruby has built in framework
environment
ATDD/BDD
• Cucumber
• Lettuce
• Fitnesse
• Jbehave
• rbSpec
UI Testing
• Selenium
102. Test-Driven Tinkering
As an avid beach reader, I want to elevate my book a bit
to protect it from the incoming wave edges so that my
book remains dry when I set it down to sip on my
umbrella drink.
• Acceptance criteria are shown by the dimensions
indicated on the sheets of paper being passed out.
– Given a book of X and Y,
When I place the book on it,
Then it does not fall off.
– Given the depth of the water will be about one inch where
I sit,
When I place the book on it and incoming water splashes
up based on the lowest dimension shown,
Then my book remains dry.
103. Test-Driven Tinkering, Part 1
• Get a set of Tinker Toys
• Objective is to build a platform can hold a book
the dimensions on the sheet of paper a height
between the dimensions off the ground. (There is
some tolerance as to not meeting these exactly
as Tinker Toys are not the most exacting…)
• Can you build the tests first using Tinker Toy
pieces and then use it to guide your design?
• Are these the simplest test you can build?
• Now use these tests to build your platform and
place the book on it (performance test).
104. Test-Driven Tinkering, Part 2
Given the waves come in closer
When I need to move back towards the dunes
Then I need a handle when I stand up to move the
book elevator back with me
• Now I want the structure’s height to be the height
based on my new test (the handle I can use when
I stand up).
• Can you build a structure to pass this test while
staying true to the other tests?
106. Test-Driven Tinkering Debrief
• What do you think the Tinker Toys represent?
• What tests were you able to construct before
beginning your construction? How did they guide
your design?
• When you added onto your structure, did you break
any of your prior tests?
Congrats!
AND…This
brings us
to …
107.
108. Refactoring is…
• Modifying code to handle additional scenarios
• Adding features (or in the smaller case, methods)
• Changing code so that it handles additional
performance characteristics
- AND -
• No prior “functions” are invalidated; all prior tests
SHOULD pass
This test fail, make test pass, improve design is known as
Red-Green-Blue (Refactor) and is what allows emergent
design as opposed to over-designing upfront.
109. Refactoring also is…
• Modifying code to remove escaped defects
- WHILE –
• Ensuring no prior “functions” are invalidated;
all prior tests SHOULD pass
110. Think of TDD & Refactoring as
Transforming Your Code into
Something More Useful
113. Mikado Method Mikado Game
Making a change to your legacy code is not
unlike picking out the Mikado from the other
sticks in the pile.
Removing one impacts the others.
And not in a good way…
What is the biggest difference?
115. The Guts of the Mikado Method
• Make the change you need/want to make
• See what broke – these are pre-requisites (the
next set of changes to make before doing this
one)
• Visualize (add to graph)
• REVERT !!
Repeat until no breakages occur.
116. Enable Database
Concurrency
Configure
High
Availability
Reimplement
DB Interface
Change
Connection
Pooling
Remove
Redundant
Code
Change
Properties
File
Isolate JDBC
Connection
Create
RESTful
Interface
D i s c o v e r y
R e f a c t o r
✓ ✓
✓
✓
✓
✓ ✓
✓
117. Adding Tests to Legacy Code
• Add unit (perhaps acceptance) tests as defects
are removed or changes are made; basically
anytime you make changes to the code.
• Tests should capture what results should be
displayed (Think assertions! -- Just like before)
• I riddle you this: Why should we not go and
simply add tests to everything?
120. Building to Every Environment
Execute
Build
Scripts
Execute
Test
Scripts
121. Build to Binaries; then Promote
Execute
Build
Scripts
Execute
Test
Scripts
Execute
Deployment
Scripts
Promote
Execute
Test
Scripts
122. Deploy to all Environments
Execute
Build
Scripts
Build to Binaries;
Execute
Test
Scripts
Execute
Deployment
Scripts
Execute
Test
Scripts
123. Some Popular CI-centric Tools
Build Scripts
• Ant
• Maven
• Gradle
• nAnt
• make
Deployment/Config Tools
• Chef
• Puppet
• Buildbot
Scripting for Promotion
• bash
• c-’Shell’
• TCL
• perl
Execute via the CI Server!
125. Types of (Post-Deployment) Feedback
User Experience
• Response times
• Dwell times
• Heatmap of “clicks”
• Flow through the site
Security
• Attempted penetrations
• Attempted DoS attacks
Performance
• Response times
• Memory usage/Page faults
• Storage growth
Business Metrics
• Improved cycle-time/
throughput on
business process
• Data quality (errors)
• Additional fees/revenue
• Reduced processing costs
127. Following Good Engineering Practices…
Makes for a more productive team
– Faster feedback
Faster feedback =
Technical excellence
– Catch defects before
release
Ø Defects = Happy CIO
128. So Support the Change!
Status Quo
New Status Quo
disruption
amount
disruption
time
Allow teams time to learn
A detailed depiction of the Satir Change model -
http://stevenmsmith.com/ar-satir-change-model/
Give teams
encouragement
Expect a
degradation
There’s a lot of irony in this slide; Waterfall is really more like a canal with it’s locks and gates; here though is an innovative lock and gate system from Falkirk Scotland. The incoming “bridge” is a canal and a wheel system takes the last segment down to to the water level, so as it iterates it moves the boats up or down.