2. Agenda
• What are technical stories
– User stories vs Technical stories
– Technical Story examples
– Why are we not creating Technical Stories
– What makes a good User/Technical Story
– Acceptance and Demo
– Where to technical stories come from
– When NOT to write technical stories
• Tech Debt/System Hygiene
– Lehmans’s Law
– System Hygiene
– Common causes of Technical Debt
– Why worry about Technical Debt
– Technical Debt can be intentional
• Architecture
– What is Architecture
– Systems are growing rapidly
– Why do we need Architecture
– Don’t over engineer
– Can’t we just refactor?
– Use Informed Anticipation
– Architecture Runway
– Technical Stories on Story Map
– YAGNI vs BUFD
– Architecture work as systems grow
– Feedback loops
– Architecture Opportunities
– The architecture is in the code
– User vs Technical Stories
– Architecture Workshops – What?
– Architecture Workshops – When and Who?
• Green Field
– SAFe
– LSS
• Processing Technical Stories
– How to address Technical Stories
– Track Technical Backlog with Kanban
– Communicating Technical Stories
– Team Allocation
• Learning
– Tech Community of Practice
– Learning
– Good Architect/Tech Lead
– The wrong approach
• Summary
• Backup
3. Technical Stories
• How we can
– Manage Technical Debt
– Manage Architecture
– Educate and mentor our technical team
5. User Stories vs Technical Stories
User stories…..
…. highlight the role of the user
…. the goal they are trying to
achieve
…. the value of that goal
As a <user type>
I want to <achieve a goal>
So that I can <get some value>
Technical stories…..
no user
…. the goal you are trying to achieve
…. the value of that goal
I want to <achieve a goal>
So that <some value>
6. Technical Story Examples
I want to refactor the xyz subsystem using abc
pattern to improve robustness.
I want to create an enterprise service bus
for communication between components
to reduce coupling.
7. Why are we not addressing Technical Stories?
Not clear who is
responsible?
Not part of the
standard workflow?
Product Owner
doesn’t see the
need?
Don’t know how to
architect/refactor/fix
8. What makes a good User (and Technical) Story
INVEST
• Independent – No dependencies
• Negotiable – Does the team have other ideas to achieve the
goal?
• Valuable – Provides something to the customer
• Estimable – Team can understand and approximate the effort
• Small – designed, coded, and tested in a sprint
• Testable – Good acceptance criteria leads to good test cases
9. Technical Story - Acceptance and Demo
• Acceptance criteria is still needed
– Define success
– Define how you will determine the story is complete
• You should still Demo the results of the technical story. Examples
– Show before and after of refactoring
– Review performance measurements spreadsheet (and discuss
decisions on next steps)
– Review key aspects (data that supports making a decision) for tools
researched
– Demonstrate how new scripts improve the development process
10. Where do technical stories come from
• A metaphor referring to the eventual consequences of poor system
design, software architecture or software development within a codebase
Technical Debt
• Areas of code that are difficult to change after implemented
• Code that does NOT itself provide user function
• Only includes elements understood by all developers
Architecture
• Research into “how”
Spikes
• Experiments with “how”
Technical Prototypes
11. When NOT to write technical stories
• For User Story work
– Create tasks or acceptance criteria
– Non functional requirements may be acceptance criteria on User Stories or
may be cross cutting technical story
13. Lehman’s Laws of Software Evolution (1980)
• Architecture/code degenerates over time
• “As a system evolves, its
complexity increases
unless work is done to
maintain or reduce it”
• We only understand how the code
should be written after it’s done
(the first time)
Geek And Poke – Oliver Widder
14. System Hygiene
Technical debt - coined by
Ward Cunningham
The obligation that a software
organization incurs when it
chooses a design or
construction approach that’s
expedient in the short term but
that increases complexity and is
most costly in the long term
What happens if you don’t pay your credit
card debt?
Touch your ear mini exercise
15. But wait
• “we don’t have time… the PO bla bla bla,
our release date bla bla”
• Spare me …
• Just Stop It.
16. Common causes of technical debt
•leads to lack of knowledge
Lack of collaboration
•work in same branches that will need
to be merged
Parallel development
•putting more code into areas that need
refactoring piles on more debt
Delayed Refactoring
•developer didn’t know better way to
write code
Lack of knowledge
Lack of test suite
• pushing to release sooner, rather than
completing all necessary changes/tests … quick
and messy is ok
Business pressures
• business decisions are made that ignore the
consequences of building technical debt
Lack of process or understanding
• partially implemented architecture changes,
partial adoption of new
featurestechniques/patterns
Drift
• inflexible software that’s difficult to adapt to
changing needs
Tightly coupled components
17. Smells that more refactoring is needed
1. Team is taking longer to deliver features
2. Quality is on a downward trend
3. Team is increasingly frustrated
4. Team won’t touch certain areas of code
5. Team struggles more with estimates
6. Team wants to start fresh, rather than work with current code
7. Team wants more refactoring time
8. Debugging takes longer
9. Fixes or features are implemented through the same change in multiple
places
18. Why worry about Technical Debt
• Future development costs increase as technical debt increases – so less
new function can be produced as technical debt rises.
• Ignored Technical Debt continues to grow – more and more code is
interwoven in smelly areas
Productivity
Time
Clean up old crap
19. Why worry about Technical Debt
• Debt limits how fully you can develop the Product
Function
Time
IntroductoryStage
GrowthStage
MaturityStage
DeclineStage
High Technical Debt
Product never
reaches full
potential
Low Technical
Debt Product
20. Cost of Change - Jim Highsmith
• Increased Technical Debt
can greatly increase the cost
of change
• To the point where the
system is umanageable
“Managing Technical Debt helps ensure reliable delivery today and ready
adaptation to tomorrow’s customer needs.” – Jim Highsmith
21. Technical Debt can be intentional
• Technical debt may result from prudent decisions – e.g. ignoring some
finer design points so as to meet an important deadline
• No Design may be appropriate to get to market fast, even though more
debt is incurred
CumulativeFunctionality
Time
Good Design
No Design
Down here it may be worth
trading off design quality
for time to market ...
… but up here, there’s no
useful trade-off
22. Good Technical Debt – Henrik Kniberg
New feature
• We don’t know best approach
• Try several ways – see how they work – comment out as you go
• To be good debt
• Must Clean Up
• Add Tests
Time
TotalTechDebt
23. When incur debt
• Don’t lose track of the debt
• Add stories to the backlog for that debt, as you go
24. Exercise
• Discuss with your team, then draw a curve of your component’s
accumulation of technical debt over time
• Next draw a curve of your ideal technical debt over time
• How might you control your technical debt?
25. Managing Tech Debt – Henrik Kniberg
Measure Debt Ceiling/Baseline?
• KISS
• Subjective measure – 5 = Great, 1 = Crap
Time
TotalTechDebt
Debt Baseline
Debt Ceiling
Time
TotalTechDebt
Debt Baseline
Debt Ceiling
Quality < 3
Quality = 4
Quality = 5
Time
TotalTechDebt
Debt Baseline
Debt Ceiling
Accumulating Debt
Cleanup
Time
TotalTechDebt
Debt Baseline
27. What is Architecture
RUP - “the highest level concept of a system in its
environment. The architecture of a software system (at a
given point in time) is its organization or structure of
significant components interacting through interfaces,
those components being composed of successively
smaller components and interfaces.”
Or – “Architecture is the set of design
decisions that must be made early in a
project.”
Or – “Architecture is about the important stuff.”
Or– Customers have a different view
than developers, so perhaps
Architecture is the highest level concept
of expert developers.
IEEE – “Software architecture is the fundamental
organization of a system, embodied in its components,
their relationships to each other and the environment,
and the principles governing its design and evolution”
Geek And Poke –
Oliver Widder
28. What is Architecture
Francis Ford Coppola – “The
key to making a great movie
is getting everyone to make
the same movie”
Ralph Johnson - 'In most successful software projects, the
expert developers working on that project have a shared
understanding of the system design. This shared
understanding is called "architecture." This understanding
includes how the system is divided into components and
how the components interact through interfaces.'
Simple definition
Architecture:
• is about areas of code that are difficult to change after implemented
• does NOT itself provide user function
• only includes elements understood by all developers
Ralph Johnson –
“Architecture is shared
understanding”
29. Systems are growing rapidly
Growth of Total
lines of code for
~5K Open Source
projects (Jun 1994
to Feb 2008)
Total software is now estimated to double every two years
30. Why do we need Architecture
To avoid large
scale/unnecessary
rework
To ensure overall
efficiency of
Development
To minimize impact
on deployed
systems/users
So the code is
easier to maintain,
easier to extend, is
more stable, more
comprehensible …
Geek And Poke –
Oliver Widder
31. Where’s the architect
• The code is the architecture, and so coders are the architects ….. And the
architect codes
• Conformance to an architecture plan is
determined at the code review and via demos
• Nonetheless - Architects whose responsibility
spans teams/components can often best
address cross component architecture (and
process) issues, new tools and new technology
32. Don’t over engineer
• Challenge - ensure that desirable emergent behaviors result instead of the normal chaos of
large organizations
• Trying to control this apparently chaotic environment with traditional architectural artifacts,
processes, and procedures will only slow the organization down, thus hampering innovation.
Andrew Binstock – “Agile does not account sufficiently for architecture and only minimally
understands design — two practices crucial to enterprises. Because of these unaddressed needs,
enterprises are better off weaving Agile practices with other familiar methodologies and finding a
blend that works for them rather than transforming their processes to adopt an approach that,
while commendably good at many tasks, remains distressingly incomplete
Adrian Cockroft – Thought Leader at Netflix - “Our architecture was changing faster than you
can draw it” ,,, ‘Netflix never considered their architecture to be a prerequisite to building
software’
33. Can’t we just refactor?
• Nope
• TDD has been claimed to be a design technique …
leading to simpler designs … and Refactoring a key part
of TDD
• IEE 2008 study:
– TDD and refactoring had local benefit to code
– But … coupling and cohesion were not helped
– In fact, there was evidence that TDD hurt Architecture
• Can’t create Architecture at the last responsible
moment:
– Leads to undisciplined form
– Leaves a trail of poorly informed code as we go.
A good book is not a collection of well formed
sentences. There are paragraphs, chapters….
Geek And Poke – Oliver Widder
34. YAGNI vs BUFD
• Balance
– Unearth and act on fundamental knowledge early – consciously try to solve
architecture early – take a reasonable guess
– But allow knowledge to emerge from local, shortsighted actions
– Collaborate on YAGNI decisions
YAGNI (your aren’t
going to need) – don’t
add architectural
aspects until you’ve
proven you need them
BUFD (Big up front
design) – try to think of
everything you’ll need
up front, and lock in the
design
Which do you think is correct?
36. Use Informed anticipation
1. Technical Stories should be written and
implemented that anticipate the
architecture needed by upcoming User
Stories
1. Informed –User Stories must exist that
show architectural need
2. Anticipation – Must have a plan for when
User Stories will be implemented, so we
can properly prepare the runway
User Story
Technical Story
37. Architecture Runway
2. Architecture Runway (a SAFe term) – the technical infrastructure (code)
should exist so that high priority features can be implemented without
excessive, delay inducing, redesign
– User Stories should always be built on an appropriate architecture
User Story
Technical Story
Time
Architectural Runway
• A user story implies architecture changes
• The Architect creates a technical story
• The Technical story is implemented in anticipation of the user story
(Architectural runway).
38. Good runways reduce risk
• Consistently completing 100% of your work in an iteration
is a bad smell – indicating the team is working
conservatively, building in contingency
• When good teams badly miss completion targets, often
Architecture impacts were misunderstood or missed
• Having a good runway – completing technical stories prior
to working on the user stories – reduces risk
39. How long should the runway be?
• Small systems need less runway
– If initially wrong, the team can refactor.
– Likely is cheaper than trying to anticipate all needs
• Larger systems need more
– Larger requirements may affect multiple components requiring more
widespread architectural changes
– Investing in more foresight … and more planning …. can reduce risk
41. Architecture Roadmap Opportunities
• Mutable
• Visible to stakeholders
• Integrated into development process
• Executed incrementally
• Leads to epics/features/stories
What:
• Big issues/impediments
• New tools and technology
Can address:
42. Big Picture Backlog Item
Technical Debt
Story
Architecture
Story
User Story
StoryFeatureEpic
Business Epic
Architecture
Epic
Business
Feature
Architecture
Feature
System Test
Acceptance
Test
Unit Test
System
Qualities Test
Nonfunctional
Requirement
Is One Of
Is One Of
Is One Of
Done When Passes
Constrained by
Compliant when passes
Realized by Realized by
43. Theme Theme
Feature Feature
Architecture
Sprint 13
Sprint 14
Sprint 15
Backlog
Epic Epic Epic Epic
Feature Feature Feature Feature Feature Feature
Story
StoryTech Debt
Story Story
Story
StoryStory Architecture
Story
Story Story
Story
Story Story
Story
Story
Story
Story
Story
Story Story
Story
Story
Story
Story
Story
Story Story
Story
Story
Story
Rqmt
• Story Map Technical and User Stories
– Plan Technical Stories vs User Stories to ensure Architectural Runway exists for
User Stories
Architecture
Architecture
Tech Debt
Tech Debt
Architecture Epic
44. Implementing Architectural Epics
• Pre-Agile: Create a new branch, code it all up, merge it back in (good luck
with that)
• Agile – Three choices
1. Big but incremental – best choice – the system always runs.
The architectural epic is split into releasable stories.
2. Big but not entirely incremental –
1. Stop and rethink. Isn’t there some way to use the first option?
2. Accomplish across sprints, then ship
3. Increases risk, but may be more efficient
3. Really big and NOT incremental – but you must ship anyway
1. Write the code incrementally as above
2. Initially, isolate the code so that it does no harm
3. Evolve the code to implement the architecture
45. Architectural Work as Systems Grow
Set up foundation
Major Features
Rewrite
ArchitectureWork
Time
46. User Stories
Are not meant to completely
document a requirement –
difficult to achieve and time
consuming to produce
Are meant to be evocative –
remind us of the full story
Backlog Grooming leads to a
shared understanding by the team
and the PO
The User Story records enough to
remind us of the shared
understanding
Technical Stories
and Models
Models can help lead us to a shared
understanding
Technical Stories need only remind us
of that shared understanding
Architecture workshops create/refine
technical stories
Models and stories do NOT dictate
what must be done, but should be an
initial direction that can be modified
as we learn more
Agile – Make it visible
47. Agile - Feedback loops
• Command and control stifles
creativity, is more brittle
– Turns off brain power we’ve already
paid for
• Instead - Use communication and
feedback loops to prevent chaos
Architecture
Opportunities
CoP – for
discussion /
technical stories
Team pairing,
mentoring,
learning, reviews
PDM – User
Stories/Tech
Stories
48. The
architecture
is in the code
– Frederick Brooks in “Silver Bullet” 1987- The building metaphor has outlived its
usefulness… If, as I believe, the conceptual structures we construct today are
too complicated to be accurately specified in advance, and too complex to be
built faultlessly, then we must take a radically different approach.. The secret is
that it is grown, not built . Harlan Mills proposed that any software system
should be grown by incremental development.. Nothing in the past decade has
so radically change my own practice, or its effectiveeness …
•Large orgs tend to have architecture groups that lose touch with
the code/real architecture (powerpoint architects):
•Hand-off waste
•Disrespect from coders due to disconnect with code.
•Leaders out of touch
Powerpoint Architects
•Not in documents maintained by an architecture team
•The architecture improves and degrades day by day with every
line of code added.
•The real architecture is a mix of good/bad,
intentional/unintentional
•Like it or not, at some level, all coders are architects
The real architecture is in the code
50. Architecture Workshops – What?
Get to a shared understanding
Use face to face
communication with visual
models to guide conversation
Two primary objectives
• Education/coaching/mentoring by
more senior/knowledgeable engineers
• Shared understanding – documented
sufficiently only to remind attendees
of the shared understanding
Commonly held within team,
but may be cross team as
needed
51. Architecture Workshops –
When? and Who?
• Right before implementing a requirement or
• When design decision needed mid development
• May include Tech Lead, Developer(s) and Test
Just in time
• With groomed requirements, clarify design impacts, and approach to be used. May involve whole
team, or subset.
• For Dependencies – a cross team meeting is likely appropriate – both teams may draw/discuss
their impacts
or After Backlog Grooming
• May spend a few days or a couple of weeks
• Followed by more workshops after backlog grooming
• Start with “Walking Skeleton”
or Before first iteration of larger project
• Review and update opportunities with stakeholders – e.g. via town hall
Periodically/at Architecture Opportunity Reviews/Workshops/Milestones
52. Communicating stories
Technical stories and models
are NOT standalone
documentation
They are intentionally
lightweight, meant to evoke
memories of a shared
understanding
When communicating to others
(e.g. a rep at a joint workshop
explaining outcomes to team
mates), the shared
understanding needs to be
explained, with details filled in.
53. Green Field fast start - SAFe
• Start with one-several teams to implement
architecture underpinnings
• Teams are like any other Agile team
• Credit for working code, not models or designs
• It should take no more than a couple of iterations
to create/prove architectural elements
Start with Architecture
• Add Feature teams to initial set of teams
Then move to features
54. Green Field fast start - LSS
Tiger Team
• Form a smaller team with the right expertise
• Get the project off to a running start
Walking
Skeleton
• Tiger team works closely together to quickly create a Walking Skeleton
• Walking Skeleton has main architecture elements in place, but there is
no meat on the bones … most user stories are not yet implemented.
Expand
• After Walking Skeleton is in place (in short time frame) – seed new
teams with tiger team members to expand the ability to satisfy user
stories
56. How to address Technical Stories
Backlog
• Collect technical stories in the
same backlog as the user stories
Prioritize
• Regularly prioritize technical
and user stories
Effort
• Estimate effort for both
technical and user stories
57. How to address Technical Stories
Define
• At or before Release/MVP Planning
Refine
• In Backlog Grooming and Architecture workshops
• For High Priority technical and user stories
• Acceptance Criteria
• Refine Story Points
Product
Owner
•Involve the PO
•Developing a strategy for implementing Technical as well as User Stories
Integrate
• Continuously integrate technical stories into development process
58. Track Technical Backlog via Kanban
58
Product Analysis Ready for Grooming Ready for Dev
• Use same Kanban states to manage Technical stories and User Stories.
PBI PBI TBI
PBI TBI PBI
PBI PBI PBI
TBI
PBI TBI PBI
PBI PBI TBI
TBI
PBI PBI
TBI PBI
PBI TBI
Proposed
TBI PBI
TBI
Solution Analysis
PBI
PBI
PBI
59. Team Allocation
• There’s an ongoing cost for system hygiene
• Might appropriate capacity to technical stories (for System Hygiene)
Team Capacity 100
pts
Product Capacity 80
pts
System Hygiene
20 pts
61. Learning
• Lack of skills, and lack of knowledge will always
be an issue
• Education/coaching/mentoring is the remedy
In workshops, Technical
Leaders, et al need to help
others understand the
problems, solutions and
techniques
On the team, technical
leaders should look for
opportunities to Pair
Program, teach
Everybody learns - Helping
others is key for the
organization, and educating
will always be a valuable skill
in a lead – Folks that don’t
understand teach you
Code Reviews are a great
opportunity to teach
Technical Community of
Practice
62. Tech Community of Practice (CoP)
Tech CoP
• Share problems and
successes
• Discuss approaches,
modeling techniques
• Review Architecture
Opportunities
64. The wrong approach
Team members aren’t
sufficiently skilled to make
important decisions
Important decisions must be made early
So the architect makes all the important
decisions
65. Good Architect/Tech Lead
Good Architects elevate
the team competence so
that they can take on
more complex issues
An Architect’s value is
inversely proportional to
the number of decisions
she makes
67. Architecture Documentation Workshop
• One or two days
• Team splits into small groups to work in parallel at
whiteboards:
– UML-ish designs
Domain Model
Logical model
Component model
Physical model
Deployment model
Glossary
Notes
See appendix of this presentation
• Show and tell walls – learning and fixing as we go
• Preserve on a wiki – photos of white boards
• Outcome
– Architecture/Design documentation
– Much learning
Supplies
• Lots of whiteboards
• Lots of fresh markers
• Erasers
• Lots of sticky flip charts
• Roll of painters tape
• 1-2 projectors
• Wiki
• Video cameras – can
record “show and tell”
70. Summary
• We need Technical Stories
– Tech Debt –
Balance investment in User Stories vs System Hygiene
– Architecture –
Just enough architecture
Balance YAGNI vs BUFD
71. Summary – Architecture Workshops
• Architectural Workshops can be done
– JIT
– Periodically after Backlog Grooming
– Major workshops for MVP/Release planning
– Periodically to review Architectural Opportunities
72. Summary - Learning
• Learning is an important aspect of Architecture
– Coaching, mentoring, teaching is an important aspect of
architecture
75. Dell - Internal Use - Confidential
Modeling advice
• Model what’s helpful
• Don’t try to cover all aspects of your code
• Don’t worry about including all aspects of the model
76. Dell - Internal Use - Confidential
Domain model
The domain is the
actual world from which
the problem comes.
78. Dell - Internal Use - Confidential
Deployment model
A component is a code module. Component diagrams similar to class
diagrams
Deployment diagrams show the physical configurations of software and
hardware.
This deployment
diagram shows the
relationships among
software and
hardware
components involved
in real estate
transactions
79. Dell - Internal Use - Confidential
Use Case
• Use case diagrams describe what a system does from the standpoint of an external observer.
The emphasis is on what a system does not how.
• A scenario is an example of what happens when someone interacts with the system. Here is a
scenario for a medical clinic.
– "A patient calls the clinic to make an appointment for a yearly checkup. The receptionist finds the nearest
empty time slot in the appointment book and schedules the appointment for that time slot. "
• A use case is a summary of scenarios for a single task or goal
80. Dell - Internal Use - Confidential
Activity Diagram (Business Process Model)
• Essentially a flowchart
overlaid on
roles/components
81. Dell - Internal Use - Confidential
Sequence Diagrams
• Interaction diagrams that
show how objects
collaborate
82. Dell - Internal Use - Confidential
Collaboration Diagrams
• Another form of interaction
diagram
83. Dell - Internal Use - Confidential
Package, Class and Object Diagrams
• Should be able to
generate package
and class
diagrams
Package Diagram
Class Diagram
Object Diagram
84. Removing Architecture?
• Ralph Johnson –
– Difference between building architecture and software architecture => It is hard to go back and change
your basement, though it is possible.
– Irreversibility is one of the prime drivers of complexity.
– There’s no theoretical reason that anything is hard to change about software.
– But we don’t know how to make everything easy to change..
• Martin Fowler
I think that one of an architect’s most
important tasks is to remove architecture by
finding ways to eliminate irreversibility in
software designs.
86. 1. Partition by subsystem/product/component
• Split epics/features so as to work on one component at a time
87. 2. System qualities incrementally
• Technical stories often address performance, reliability, security,
robustness. Often the improvements can be performed
incrementally
– Improve performance by 10%, by 20%, by 30% …
– Support 5,000 users per server, then 7,000 …
88. 3. Incremental Functionality
• Stage features of the solution over time
• Provide wireless software update for thermostats
– Enable update of the device boot loader
– Enable update for the radio
– Enable update of settings
89. 4. Build scaffolding first
• Sometimes we need scaffolding to get to the next step. It might be
throwaway code, but it facilitates progress
• Build an API to support a third-party interface.
– Mock the data initially
– Enable customers/partners to test integrating with the service
90. 5. Major/Minor Effort
• Productize APIs
– First (major) – separate identified APIs from core service, and use them
internally
– Then (minor) – Refactor APIs to support standards and third part use
91. 6. Complex/Simple
• Where the architecture is getting increasing complex, imagine the
simplest core piece first, adding variations and complexities later
• Complex – Productize the APIs
• Simple – Build an adapter to expose a single API
92. 7. Variations in data
• Implement consumer home energy management devices through
the new backhaul grid infrastructure
• Implement on meter device using backhaul A protocol
• Implement on meter device using backhaul B protocol
93. 8. Break out a Spike
• A spike or a series of spikes may be needed to establish feasibility,
effort or perhaps even to decide how to split the stories
• Implement a new messaging queue protocol
1. Create a technical prototype (throwaway) using the new
protocol
2. Create test loads with performance tests
3. Run performance tests to determine value