Kanban Risk Management Scaling Up with Probabilistic Forecasting

No Crystal Ball Gazing
Risk Management &
Delivery with Kanban

Using qualitative
risk assessment &
probabilistic
forecasting for
better business
results
Devlin
Linkoping, March 2013

dja@djaa.com, @djaa_dja

Understanding Kanban Systems


Kanban are virtual!
Backlog

Engineering
Ready

5

Development

Test
Ready

Testing

UAT

3

5

3

∞

Ongoing

Done

Change
Requests

Deployment
Ready
∞

B
Pull
These are the virtual kanban
F J
G
C
F
Pull
Boards are not required to do
F F
Kanban!
F
I
F
The board D a visualization of the
is
F
*
The first system used database
workflow process, the work-intriggers to signal pull. There was no
progress and the kanban
PTCs
board!

Pull

I


Commitment is deferred
Backlog
Pool
of
Ideas

Engineering
Ready

5

Development

Test
Ready

Testing

UAT

3

5

3

∞

Ongoing

Done

Items in the backlog remain
optional………………………..
optional and unprioritized

Pull

F F
F
F F
F
F

D
G

E

Wish to avoid discard after commitment

I

Commitment point


We are committing to getting
started. We are certain we want
to take delivery.

Deployment
Ready
∞

Specific delivery commitment may be
deferred even later
DeployEnginPool
of
Ideas

eering
Ready

5

Development

Test
Ready

Testing

UAT

3

5

3

∞

Ongoing

Done

ment
Ready
∞

Change
Requests

Pull

F F
F
F F
F
F

D
G

E

PTCs

We are now committing to a
specific deployment and
delivery date

Discarded

*This may happen earlier if
I
circumstances demand it

I

2nd
Commitment
point*

Replenishment Cadence
Pool
of
Ideas

Engineering
Ready

5

Replenishment
Change
Requests

Pull

F F
F
F F
F
F

Development

Test
Ready

Testing

UAT

3

5

3

∞

Ongoing

Done

Frequent replenishment is
more agile.

On-demand replenishment is
D
most agile!
G

PTCs

Discarded

I

E
The frequency of system
replenishment should reflect
arrival rate of new
information and the
transaction & coordination
I
costs of holding a meeting

Deployment
Ready
∞

Delivery Cadence
Pool
of
Ideas
Change
Requests

Pull

F F
F
F F
F
F

Engineering
Ready

Development

Test
Ready

Testing

UAT

3

5

3

∞

Ongoing

Done

Frequent deployment is
more agile.

5

Deployment buffer size can
On-demand deployment
reduce as frequency of
D deliveryagile!
most increases

G

PTCs

Discarded

I

is

E
The frequency of delivery
should reflect the transaction
& coordination costs of
deployment plus costs &
toleranceI of customer to take
delivery

Deployment
Ready
∞

Delivery

Defining Kanban System Lead Time
Pool
of
Ideas

Engineering
Ready

The clock starts ticking when
Test
UAT
we Ready
customers
Development accept theTesting
5
∞
3 order, not when it3is placed!

Ongoing

5

Change
Requests

Pull

F F
F
F F
F
F

D
G

Done

Until then customer orders are
merely available options
Lead time ends
when the item
reaches the first

∞ queue.
E

System Lead Time

PTCs

I

Discarded

I

Deployment
Ready
∞

This provides the
correct result for
Little’s Law and
visualization on a
Cumulative Flow
Diagram

Little’s Law & Cumulative Flow
Delivery Rate

Pool
of
Ideas

=

Lead Time

Avg. Lead Time
WIP


WIP

Ready
To
Deploy

Avg. Delivery Rate

Defining Customer Lead Time
Pool
of
Ideas

Engineering
Ready

Development

Test
Ready

Testing

UAT

3

5

3

∞

Ongoing

5

Change
Requests

Done

The clock still starts ticking
when we accept the customers
order, not when it is placed!

Deployment
Ready
∞

Pull

F F
F
F F
F
F

D
G

E

Customer Lead Time

PTCs

Discarded

I

The frequency of delivery
cadence will affect customer
I
lead time in addition to system
capability

Done
∞

Flow the
Efficiency
Flow efficiency measures

Pool
Enginpercentage of total lead time
of spent actually adding value
eering
is
Development
Ideas
Ready

(or knowledge) versus waiting
3
Ongoing

2

Done

Testing

3

Verification Acceptance

Deployment
Ready
∞

Until then customer orders are
merely available options
Flow efficiency = Work Time

Multitasking means time spent
E in working columns is often
waiting time

PB
GY

DE

Waiting Working

MN
AB

Waiting

Working

Waiting

Lead Time
* Zsolt Fabok, Lean Agile Scotland, Sep 2012, Lean Kanban France, Oct 2012


x 100%

Lead Time

Flow efficiencies of 2% have been
F
reported*. 5% -> 15% D normal, P1
is
>
40% is good!
G

I

Done

Observe Lead Time Distribution as an enabler
of a Probabilistic Approach to Management
Lead Time Distribution
3.5
3

CRs & Bugs

2.5
2
1.5
1
0.5

1

4

7

0

3

6

8
14

14

13

12

12

11

10

99

92

85

78

71

64

57

50

43

36

29

22

8

15

1

0

Days

This is multi-modal data!

Mean of 31
days

The workexpectation of
SLA is of two types:
Change Requests (new
105 and Production
features);days with 98 %
Defects

SLA expectation of
44 days with 85% on-time

on-time

Mean
5 days

Change Requests

Production Defects

Filter Lead Time data by Type of Work (and
Class of Service) to get Single Mode
Distributions

98% at
25 days
85% at
10 days


98% at
150 days

Mean
50 days

85% at
60 days

Allocate Capacity to Types of Work
Pool
of
Ideas

Engineering
Ready

Ongoing

2
Change
Requests

Development

4

3

Done

Testing

3


Consistent capacity allocation
E
some consistency to
should bring more consistency to
MN
delivery rate of work of each
D
AB
type

F

Lead Time

PB
DE
Productio
n
Defects

I

Deployment
Ready

3

G

P1

GY


Separate understanding of
Separate understanding of Lead
Lead Time for each type of
Time for each type of work
work
Lead Time

∞

Done

impact

The Optimal Time to Start
If we start too early, we forgo
the option and opportunity to do
something else that may provide
value.
If we start too late we risk
Ideal Start
incurring the cost of delay

When we
need it

Here

With a 6 in 7 chance of on-time
delivery, we can always expedite
to insure on-time delivery
85th
percentile
Commitment point


Metrics for Kanban Systems
Cumulative flow integrates demand,
WIP, approx. avg. lead time and delivery
rate capabilities
Lead time histograms show us actual
lead time capability

Flow efficiency, value versus failure
demand (rework), initial quality, and
impact of blocking issues are also
useful

Scaling Up
(Probabilistic Forecasting)


Scaling Up - Planning a big project
Device Management Ike II Cumulative Flow

2008

30
-M
ar

23
-M
ar

5x

9M
ar

2M
ar

eb
24
-F

eb

2006

17
-F

10
-F

Slope in middle
3.5x - 5x slope
at ends

16
-M
ar

240
220
200
180
160
140
120
100
80
60
40
20
0

eb

Features

Required (local average) delivery rate

During the middle 60% of the project schedule we
Time
need Throughput (velocity) to average 220 features
Inventory Started Designed Coded Complete
per month


Little’s Law

Determines
staffing level

Calculated based on
known lead time
capability & required
PlandeliveryChanging the WIP limit without
based onrate
currently observed
maintaining the staffing level
capability and current working
ratio assume process
practices. Do not represents a change to the
way of working. It is a change to
improvements.
the process and will produce a
Delivery Rateto reduce undesirable‘common
change in the observed
If changing WIP
cause’ capability new
effects (e.g. multitasking), get of the system
Lead Time
sample data (perform a spike) to
observe the new capability
From observed
capability
Target
Treat as a fixed
to
variable
achieve plan

=


WIP

Using Little’s Law

Determines
staffing level

Calculated based on
known lead time
capability & required
At this point perhaps just a little
delivery rate
black magic and experience may
be required.
If our current working

WIP = 22

practices/process exhibited an
Rounding 22 up to 25 would
average WIP of 1 item per person then
55/week 25 people organized infor 5 teams
we require conveniently provide 5
with to complete 5 items each
teams of 5 peoplea WIP limit of the
0.4 weeks
project on-time

=

From observed
capability
Target
to
achieve plan


Treat as a fixed
variable

2-tiered board
Kanban System within a Kanban System

1 lane per team


WIP in this area
should be 25
items*
*photo taken
early in the
project before it
was fully
staffed/loaded
Lead time

Median lead time
target is 2 days
Alert managers if
beyond 5 days


Risks & Qualitative Assessment


A Lean approach to alignment with business
risks uses Qualitative Assessment
But how do we determine the risks in
We need a work item that we must manage?
a fast, cheap, accurate, consensus

forming approach to risk assessment. We need
Lean Risk Assessment!

The answer is to use a set of qualitative
methods to assess different dimensions of risk
such as urgency


Sketch market payoff function
Room nights
sold per day

Cost of delay for an online Easter holiday marketing
promotion is difference in integral between the two curves

Estimated additional
rooms sold
Cost of delay
Actual rooms sold

time
When we need it


When it arrived

impact

Cost of Delay based on Market Payoff
Sketches
Treat as a Standard Class item
Total cost
of delay

time
Cost of delay function for an online Easter holiday
marketing campaign delayed by 1 month from mid-January
(based on diff of 2 integrals on previous slide)

impact

impact

Establish urgency by qualitative
matching of cost of delay sketches

time

impact

impact

time

time

time

Intangible – cost of delay may be
significant but is not incurred until much
later; important but not urgent

impact

time

Fixed date – cost of delay goes up
significantly after deadline; Start early
enough & dynamically prioritize to insure
on-time delivery
Standard - cost of delay is shallow but
accelerates before leveling out; provide a
reasonable lead-time expectation

impact

impact

time

Expedite – critical and immediate cost of
delay; can exceed kanban limits (bumps
other work)

time

impact

Cost of Delay has a 2nd Dimension
Working capital

impact

time
Working capital

impact

time

Extinction Level Event – a short delay will
completely deplete the working capital of
the business

Major Capital – the cost of delay is such
that a major initiative or project will be
lost from next year’s portfolio or
additional capital will need to be raised
to fund it
Discretionary Spending – departmental
budgets may be cut as a result or our
business misses its profit forecasts

impact

time

Intangible – delay causes embarrassment,
loss of political capital, affects brand
equity, mindshare, customer confidence, etc

?
time


Risk is a multi-dimensional problem
So understanding cost of delay
Yes, however, it isn’t always relevant! Cost of
enables us to know what to pull
delay attaches to a deliverable item. What if
next?
that item is large? Whole projects, minimum
marketable features (MMFs) or minimum viable
products (MVPs) consist of many smaller items.
We need to understand the risks in those
smaller items too, if we are to know how to
schedule work, replenish our system and make
pull decisions wisely


Market Risk of Change

Profits
Market Share
etc

Start
Late

Differentiators
Spoilers
Regulatory
Changes

Cost Reducers

Scheduling

Potentia
l Value

Highly
likely
to
change

Market Risk

Build
(as rapidly as
possible)

Table Stakes
Buy (COTS)
Rent (SaaS)

Highly
unlikely
to change

Start
Early

Product Lifecycle Risk
High

Not well understood
High demand for innovation &
experimentation

Low

Major
Growth
Market

Investment

Growth
Potentia
l

Product Risk

Innovative/New

Cash Cow
Low


Well understood
Low demand for
innovation

Low

High

Shelf-Life Risk
Short
(days, weeks,
months)

Known
Expiry
Date,
Seasonal
(fixed window
of
opportunity)

Medium
(months, quarters,
1-2 years)

Long
(years, decades)


Fashion
Craze
Fad

Shelf-Life Risk
Many

High

High

Schedule Risk

(days, weeks,
months)

Innovation

Number of Options

Short

Known
Expiry
Date,
Seasonal
(fixed window
of
opportunity)

Medium
(months, quarters,
1-2 years)

Long
(years, decades)

Few

Low

Low


Fashion
Craze
Fad

Shelf-Life Risk
Schedule Risk
Low

High

High

Many

(months,
quarters,
1-2 years)

Number of Options

(window of
opportunity)

Medium

Fashion
Craze
Fad

Innovation

Known
Expiry
Date,
Seasonal

Differentiator

(days, weeks,
months)

Spoiler/Follower

Short

Low

Low

Few

Long
(years,
decades)


Low
If we are market leading our
innovations are less time
critical

Risk is a multi-dimensional contextual
problem
These are just useful examples!
We must develop a set of
We can easily envisage other risk dimensions risk
such as technical risk, vendor dependencycontext for
taxonomies that work in risk,
organizational maturity risk and so forth.
a specific business.
It may be necessary to run a workshop with
stakeholders to explore and expose the real
business risks requiring management


(Just a taste of)

Risk Management with Kanban


How much risk do you want to take?
Given our current capabilities, our
desired strategic position and goWe only have capacity to do so much work. How
we allocate that capacity across different risk
to-market strategies, how much risk
dimensions will determine how aggressive we
do you want to take?
are being from a risk management perspective.

The more aggressive we are in allocating
capacity to riskier work items the less likely it
is that the outcome will match our expectations


Hedge Delivery Risk by allocating capacity
in the kanban system DeployEngineering
Ready

2
Expedite

Development
Ongoing

3

Testing

3

Done


1
P1

AB
Fixed
Date

2

E

D

MN

PB
Standard

3

F

G

GY
Intangible

3

I

DE

ment
Ready
∞

Done

Aligning with Strategic Position
or Go-to-Market Strategy
DeployEngineering
Ready

Cost
Reducer
s

3

2

3

Testing

3

Ongoing

Done


niche! Enabling early delivery for narrower
P1
markets but potentially including value
AB
DA
generating differentiating features

E

D

MN

PB
Spoilers

1

F
GY

Differenti
ators

Done

The concept of a minimum viable
∞
product (MVP) will contain the table
Market segmentation can be used to narrow the
stakes for at least given market
necessary table stakes for any 1 market niche
G
2

Table
Stakes

Development

ment
Ready

1


I

DE

Trade off growing market reach against
growing share & profit within Deploya niche
EnginCapacity allocated to Table Stakes will

ment
eering
determine how fast new niches can be Ready
Development
Testing
Ready

Cost
Reducer
s

3

developed.

3

It is important to define a MVP in
∞
Allocate more to Table Stakes to speed market
terms of table stakes and
reach/breadth.
differentiators required to enter a
G
specific market segment
Allocate more to differentiators to grow
P1
2

Table
Stakes

3

Ongoing

Done


market share or AB
profit margins
DA

2

E
Allocate D
more to spoilers to defend market
share MN
PB

Spoilers

1

F
GY

Differenti
ators

Done

1


I

DE

An underlying philosophy of
pragmatism


Some simple rules to improve delivery
forecasting
1. Limit WIP
2. Observe what really happens in your
Assume that the past is
own environment a strong predictor of
the future
3. Measure current system capability as
In low flow distribution, environmental
lead timeefficiency systems,throughput
conditions (system
rate and flow factors) outweigh technical
efficiency
performance factors by up to 20 times in
4. Forecast Probabilistically
determining the outcome. If the environment
isn’t changing neither should results.


Prediction based on qualitative risk
assessment
Stop Crystal Ball Gazing!
Do not speculate!
Do not ‚estimate‛ the
size, weight, complexity
of an item. Instead
qualitatively assess the
risks inherent in a work
item


Some simple rules to improve risk
management
1. Establish a list of risks that are
applicable in your business domain
2. Create a qualitative taxonomy of 2 to
6 categories for each dimension of
Cost of delay, shelf-life, product adoption
risk lifecycle, market risk of change
3. Work with things that can be
All can be established as (soft*) facts. Risks
established by consensus as (soft)
associated with different classifications within
facts.risk dimensions are understood and the
these Do not speculate about the
dynamics
future! of how they might affect an outcome
are business
4. Use meaningful predictablelanguage

* Where hard facts cannot be established by measurement or
market research, a strong consensus opinion is achieved


Prediction based on qualitative risk
assessment
For example, if we load our
entire capacity with fixed
delivery date demand then it is
highly likely that some items
will be delivered late and we
will incur a (significant) cost of
delay


Allocate capacity to hedge risks
5. Our key strategy to manage risk
is to allocate capacity in
accordance with our capability,
risk tolerance and business risks
under management
6. Set kanban limits across risk
categories
7. Allow the kanban to signal what
type of risk item to pull next


Defer Commitment. Banish Backlogs
8. Defer Commitment to manage
uncertainty
When developing options upstream of the
commitment point, classify the item for each
9. The Lean concept of ‚Last
dimension of risk under management. at the
Responsible Moment‛ is
A good mixpoint of commitment –withinpoint
of options, providing choices the
each risk category is required. The more risks
of replenishment
under10. ‚Backlog‛ more options will be
management the implies committed.
required. The greater the min-max upstream
Uncommitted items are options.
kanban limits will need to be
Develop a ‚pool of ideas‛


Abandon Prioritization. Banish Priority
11. Prioritization is waste!
Prioritization is an exercise to
schedule variable for real business
Priority is a proxya sequence of items at a
risk information.
specific point in time. Only at the
Do not point ofbehind a proxy. Enable better
mask risk commitment can a proper
assessment be made making by
governance and better decision of what to
exposing the business risks under management on
pull next. Filter options based
throughout the workflow
kanban signals. Select from
filtered subset


Abandon Formulas & Calculations
12. Do not try to give relative
weight to risk categories or
calculate a risk number using a
Without a formula calculating a priority should
be impossible!
formula
Embrace the idea that formulas and proxy
Weightings and formulas mask
variables such as ‚priority‛ have no place in real
sound risk management decisionmay lead us
risk information and making

to unbalanced, misallocated

Transparently expose business risks throughout
capacity the system risk management
and poor

decisions


Visualize Risks on the Ticket
Decorators

Title

Typically used
to indicated
technical or
skillset risks

H

Checkboxes…
risk 1
risk 2
risk 3
risk 4

SLA or
Target Date

Letter

req
complete

Color of the
ticket

Business
risk
visualizatio
n
highlighted
in green

Visualize Risks on the Board
Engineering
Ready

2
Expedite

Development
Ongoing

3

Testing

3

Done


1
P1

AB
Fixed
Date

2

E

D

MN

PB
Standard

3

F

G

GY
Intangible

3

I

DE

Deployment
Ready
∞

Done

Conclusions


Focus on Sources of Delay
In low flow efficiency systems,
focusing on sources of delay – queues,
blocking issues, rework, provides high
leverage improvement in observed
Lead time performance
capability

is

strongly biased to
environmental factors, not
technical capabilities


Forecast Probabilistically
Use observed capability data!

Abandon Cartesian

Accept that the future is likely to
decomposition and speculative
strongly reflect past performance.
Use historical data todeterministically
attempts to forecast
probabilistically

estimate size, complexity or
level of effort


Qualitative Approaches are Lean
Qualitative approaches to risk
assessment are fast, cheap and drive
Stop speculating about
consensus

business value and ROI.
There is no crystal ball gazing! Riskrisks
Instead assess real
analysis is not speculative!
and design kanban systems
to manage them!


Kanban enables more predictable
delivery and better risk management
Kanban systems address variability in,
and focus attention on improving,
Exploit predictability in
flow!
delivery with qualitative

risk management.
Kanban enables predictable delivery

Stop Crystal Ball Gazing!


Thank you!

About

David Anderson is a thought
leader in managing effective
software teams. He leads a
consulting, training and
publishing and event planning
business dedicated to
developing, promoting and
implementing sustainable
evolutionary approaches for
management of knowledge
workers.
He has 30 years experience in the high technology industry
starting with computer games in the early 1980’s. He has led
software teams delivering superior productivity and
quality using innovative agile methods at large companies
such as Sprint and Motorola.
David is the pioneer of the Kanban Method an agile and
evolutionary approach to change. His latest book is
published in June 2012, Lessons in Agile Management – On the

Road to Kanban.

David is a founder of the Lean Kanban University, a business
dedicated to assuring quality of training in Lean and Kanban
for knowledge workers throughout the world.


Acknowledgements

Donald Reinertsen directly influenced the adoption of virtual
kanban systems and the assessment of cost of delay & shelf-life as
criteria for scheduling work into a kanban system.
Daniel Vacanti helped with a deeper understanding of Little’s Law
and the long term planning approach. Troy Magennis has been
inspiring with his work on probabilistic planning, risk management
and Monte Carlo simulation.
I borrowed the term ‚Stop Crystal Ball Gazing‛ from Chris Matts.

David J Anderson
& Associates, Inc.


Appendix



Fixed Date

Intangible

Standard

Expedite

Example Distributions

Kanban Risk Management Scaling Up with Probabilistic Forecasting

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Kanban Risk Management Scaling Up with Probabilistic Forecasting

Ähnlich wie Kanban Risk Management Scaling Up with Probabilistic Forecasting (20)

Mehr von David Anderson

Mehr von David Anderson (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Kanban Risk Management Scaling Up with Probabilistic Forecasting