Forecasting and portfolio management workshop I delivered at Agile 2017 in Orlando FL

1. Data-driven Portfolio Management
Concrete tools for managing products in an uncertain environment

3. Me
Not Allowed to Learn Math

4. 1. How do you decide what product to build?
2. How do you choose which to do first?
@AdamYuret

5. Troy Magennis
FocusedObjective.com
Troy Magennis (@t_magennis)
FocusedObjective.com
Every spreadsheet and
exercise worksheet is here:
Bit.ly/SimResources (gitHub)
#LASCOT16@AdamYuret

7. If you can this, you can Monte Carlo.

9. Planning Poker

10. 0 1
0.5
0.5
You are here You could be here
Why Forecasting Using Monte Carlo?
0.85
#LASCOT16@AdamYuret

11. Cut to the chase, How do I predict delivery?

13. Statistics!

14. Probability
Refresher
Sampling
Getting
Data
Forecasting
with Data
Undo all of the statistics you’ve learned in school
Learn how much data we need to forecast
Learn how to get historical data and estimates
Practice using historical data to forecast

16. Finish the rest of the probability questions
3 minutes

17. Probability
Refresher
Sampling
Undo all of the statistics you learnt in school
Learn how much data we need to forecast
Defining
Value
Learn how to define value of product options
Capacity
Planning
Learn how forecast encourages prioritization

18. Sampling
A way to use real data we actually have
to make predictions & forecasts
It helps discover the range of possible
values quickly and reliably

19. Q. How quickly do we discover a
range of values by sampling?
Why? Because as we get data (e.g. story
count, story size, velocity, throughput,
cycle-time) how confident should we be of
having found the full range values.

20. Lowest sample
so far
Actual
Maximum
Actual
Minimum
Highest
sample so far
Q. What is the chance of the 4th
sample being between the range seen
after 3 random samples? (hint: The
numbers don't matter, only the gaps)
(no duplicates, uniform distribution)
A. ?1st
2nd
3rd
4th

21. Lowest sample
so far
Actual
Maximum
Actual
Minimum
25% chance higher than
previous highest seen
25% chance lower than
previous lowest seen
Highest
sample so far
Q. What is the chance of the 4th
sample being between the range seen
after 3 random samples?
(no duplicates, uniform distribution)
A. ?1st
2nd
3rd
4th
25%
25%

22. Lowest sample
so far
Actual
Maximum
Actual
Minimum
25% chance higher than
previous highest seen
25% chance lower than
previous lowest seen
Highest
sample so far
Q. What is the chance of the 4th
sample being between the range seen
after 3 random samples?
(no duplicates, uniform distribution)
A. 50%
% = (n – 1)/(n+1)
% = (3-1)/(3+1)
% = 2/4 = 1/2
% = 0.5
1st
2nd
3rd
4th
25%
25%

23. Actual
Maximum
Actual
Minimum
8.5% chance higher than
previous highest seen
8.5% chance lower
than previous lowest
seen
Highest
sample so far
Lowest sample
so far
Q. What is the chance of the 12th
sample being between the range seen
after 11 random samples?
(no duplicates, uniform distribution)
A. 83%
% = (n-1)/(n+1)
% = (11-1)/(11+1)
% = 0.833
1st
2
3
4
5
6
7
8
9
10
11th
12

24. Prediction Intervals
• “n” = number of prior samples
• % chance next sample in previous
range for prior sample count
n (n-1)/(n+1) n (n-1)/(n+1)
2 33% 16 88%
3 50% 17 89%
4 60% 18 89%
5 67% 19 90%
6 71% 20 90%
7 75% 21 91%
8 78% 22 91%
9 80% 23 92%
10 82% 24 92%
11 83% 25 92%
12 85% 26 93%
13 86% 27 93%
14 87% 28 93%
15 88% 29 93%
30 94%

25. Experiment
From a *known* range of values, take
samples at random and see how fast we can
determine what the full range *might* be.
Compare two ways –
1. From the (n-1)/(n+1) formula
2. By doubling the average (double what you are told)

27. 42
7
99
Sum all rolls
up to each
row and
divide by n
00 & 0 = 100 10 minutes

28. Come to the front when completed. Compare with expected.
How close to 9 samples is range of 80 found? (80% range, 10% above?)
Group # samples >
range > 80
# samples until
2 x avg > 80
1
2
3
4
5
6
7
Group # samples >
range > 80
# samples until
2 x avg > 80
8
9
10
11
12
13
14

29. Cut to the chase, How do I predict delivery?

30. http://bit.ly/ThroughputForecast

31. The German Tank Problem

32. The German Tank Problem96 Bogie Wheels!

33. The German Tank Problem96 Bogie Wheels!
08/1942
Intelligence Estimate
1550
Statistical Estimate
327

34. The German Tank Problem96 Bogie Wheels!
08/1942
Intelligence Estimate
1550
Statistical Estimate
327
ACTUAL POST-WAR NUMBER
342! Statistical Estimate = 96%

37. Probability
Refresher
Sampling
Undo all of the statistics you learnt in school
Learn how much data we need to forecast
Defining
Value
Learn how to define value of product options
Capacity
Planning
Learn how forecast encourages prioritization

38. 38

39. 39
September?!

40. 40

41. 41

42. 42
How do I make Veruca
Salt Happy?

44. 44
How can I move the work to make the stated goal possible?
• Move some of the work to different people with available capacity
• Work faster/harder?

45. “Manage the work, not
the worker” –W Edwards
Deming Deming never said this… but he would’ve
#fakeDemingQuote

46. 46
NAILED IT!

47. 47
Multi-feature Forecaster

48. 48
Crunch Mode!

49. 49
http://bit.ly/HicssTroy

50. 50

51. 51

52. 52
How many stories?
What Order?
How Likely?

53. 53

54. 54
How Short?

55. 55
Weight

56. 56
Weights?
Balanced Portfolio of Investments
Create
Revenue
Protect
Revenue
Reduce
Costs
Eliminate
Costs
Exploratory
Innovation
Exploitative
Innovation

57. 57
First!

58. Study This Guy’s Work
@AdamYuret #Agile2017

59. Principles of Product Development
Flow
@AdamYuret Adam.Yuret@gmail.com

60. 60
http://productinstitute.com/
Melissa Perri
@lissijean

61. http://bit.ly/ThroughputForecast

62. Summary:
•Forecast Often
•Use Multiple Measures to Check Your
Work
•Associate Revenue Outcomes with
Decisions

63. Every spreadsheet and
exercise worksheet is here:
Bit.ly/SimResources (gitHub)
or FocusedObjective.com (free stuff)
http://bit.ly/HicssTroy
or @AdamYuret
Consulting, training and coaching:
Adam.Yuret@contextdrivenagility.com

64. @AdamYuret Sayat.me/ayuret@t_magennisBit.ly/agilesim