Estimating is hard to get right;
Why is estimating hard to get right?;
Why do we need to estimate;
Agile estimating and planning;
Determine the teams velocity;
Identify features and stories;
Define stories or features;
Planning Poker;
Agile Release Plan;
What if you don’t know the teams velocity?;
Estimating from ideal team structure;
The effect of rework;
Proposals and SOW’s;
2. Estimating is hard to get right
Suez Canal 3X more than estimated
Sydney Opera House 15X more than
estimated
“On average Victorian Government ICT
projects will have more than doubled in
cost by the time they are finished” Victorian
Ombudsman Report into ICT Projects
Myki $1.5B vs $1B estimated
3. Why is estimating hard to get right?
Overly optimistic predictions of scope and budget
to get a project approved and funded
Faulty forecasting techniques
Inadequate information
Scope creep
Everything looks easy at a high level
6. Why do we need to estimate
Stakeholders need estimates of how long things will
take and cost:
To decide if they are worth doing,
To compare alternative investments and
solutions;
To allocate resources and
To plan product launches.
7. Agile estimating and planning
Review Progress &
Identify Velocity
Develop an
iteration and
release plan
Estimate each
feature or story vs
known stories
Analyze stakeholder
requirements
Identify and rank
features or stories
8. Determine the teams velocity
Team Velocity
120
100
80
60
40
20
0
1
2
3
4
5
6
Estimated velocity
7
8
9
10
11
Actual velocity
12
13
14
15
16
17
Power (Actual velocity)
18
19
9. Identify features and stories
Feature
High level
story
Low level
story
Create
and send
basic email
Search by
keyword
Open &
read basic
email
Create sub
folders
Open RTF
email
Move
emails
Send RTF
email
Send
HTML
email
Delete
email
Create
HTML
email
Import &
process
contacts
10. Define stories or features
Acceptance Criteria
Given that I am {this actor}
And {the situation is X}
When I {do this step}
Then {Y happens}
Business Rule:
When A and B then C
Interface Design:
UI Wireframe for X
14. What if you don’t know the teams velocity?
Traditional
estimation
Start
Project Timeline
When velocity is unknown use a combination of traditional and agile estimating
approaches
Determine features and estimate stories in points as before
Team provides an optimistic and pessimistic estimate of the features and stories they can
commit to in an iteration. Use the pessimistic estimate as the velocity
As a check do a bottom up estimate of days effort taking into account that developer
effort is only half the team effort and rework and defect fixing is often as much as the
original effort again
15. Estimating from ideal team structure
Online
Application
Back end
Application
BA
12%
BA
14%
PM
10%
DEV
50%
UX
11%
QA
17%
PM
11%
0%
QA
19%
DEV
56%
16. The effect of rework
Outstanding team
90% test case pass rate
Average team
70% test case pass rate
Rework
10%
Initial
work
90%
Poor quality team
40% test case pass rate
Rework
25%
Initial
work
75%
Rework = Defect fixes = Failed tests
Initial
work
50%
Rework
50%
17. Proposals and SOW’s
Software development projects are always new
We uncover the real requirements and solutions by doing
Big up front designs lead to over specification of the wrong things
Fixed price & scope waterfall projects become variable price, scope
and time on the first change request
Agile can guarantee delivery on time and on budget of the features
that are most important to the customer
Move to Agile fixed price, fixed team projects with a target variable
scope
19. Future topics
Initiating an Agile project
Agile Kanban vs Agile Scrum
Scaling Agile for Large Teams
Reduce wasted time and effort in software
development
Project retrospectives
The business case for Agile
KiandraKiandra is a high quality software development and services company.Our Mission: To build great software. To deliver great IT. To provide an exceptional experience. To do this we aim to recruit and retain the best and most knowledgeable IT professionals the industry has to offer.Questions for the audienceHave you estimated a software project?Are your up front estimates usually accurate?Are you measuring your teams velocity?Do you use planning poker?
The Suez Canal cost 20 times more than the earliest estimates; and three times more than the estimate produced the year before constructionThe Sydney Opera House cost 15 times more than was originally projectedThe Myki ticketing system cost 1.5 times more than costedOn average Victorian Government ICT projects will have more than doubled in cost by the time they are finished.Research shows that one in 6 ICT projects have cost overruns of 200% and schedule overruns of 70%http://www.ombudsman.vic.gov.au/resources/documents/Investigation_into_ICT_enabled_projects_Nov_2011.pdf http://hbr.org/2011/09/why-your-it-project-may-be-riskier-than-you-think/Dr R. Young, Case Studies- How Boards and Senior Management Have Governed ICT Projects to Succeed (or Fail), Standards Australia, Sydney, 2006.
Lets assume you enjoy cycling and often ride your bikes for 1 to hours around Melbourne for an hour or two on the weekend covering 20 to 40 km a time. You have 5 days off over Easter so you decide to cycle to Sydney. It’s 800km away but if you cycle 20 km an hour for ten hours a day it will take 4 days.At the high level everything looks easy
The cycling analogyAt the low level there a lot of hills, twists and turns, unexpected problems and delaysEnd up averaging 100 km a day instead of 200 km a day so it takes 8 days instead of 5Why software development is like the cycling analogySoftware development is about automating manual processesWhen we know something really well we automate it by developing products, frameworks, tools and putting it in productionWhich means that every software development project is new – either we are using new tools or building something new with themWhich means that there are always a lot of unknown requirements and solutions that we cannot know until we have done the projectAttempting to remove all the unknowns through big up front analysis and design is impossible and results in over specifying and over building solutions
If estimating is hard to get right why do we need to estimate at all?
Review Progress & Identify VelocityEvery iteration and release reviewVelocity = number of story or feature points the team delivered last iteration or releaseCost per point = total cost of the team per iteration or release / total team costRemaining stories in the product backlogAnalyze stakeholder requirements at the high levelWork one iteration ahead of the teamIdentify and prioritize high level featuresFor the highest priority features develop Persona’s, process maps, information architecture,wireframes & acceptance testsIdentify and rank features or storiesDefine the stories with the product owners Identify the new stories or features the product owners want to doTeam estimates each feature or story vs known storiesForm a baseline by identifying completed small, medium and large stories or featuresUse planning poker to estimate new stories to completed storiesDevelop an iteration and release planEstimate your velocity for the next iteration and releaseChoose the highest priority stories that fit within this velocity window
Determine the teams actual velocityTeam velocity = number of story points that were “done” in an iteration.A story is an end to end feature of the system that provides value to a user. Stories are agile requirements. The team breaks features and large stories down into stories that the team can complete within one iteration or less than 2 weeks. Done = the product owner accepts that the story meets the agreed acceptance criteria and is ready to deploy to production.If a story has defects or requires rework it is not done and it stays in the backlog for the next iterationIncreases and changes in scope generate new storiesThe team measuresit’s actual velocity and uses it to predict it’s future velocity.Most teams over estimate their velocity at the start of the project but over time the teams actual and planned velocity converges.Velocity improves over time but gradually flattens off.The concept of velocity is similar to the concept of earned value in project management.This graph shows planned vs actual velocity from a real software development team
Identify features and storiesIdentify the work that you need to estimateWork at different levels of abstraction – features, high level stories and detailed storiesMap out the users business processes to identify the features and stories required and ensure that they support the users needs
Epics (features) and Stories (agile requirements) are defined incrementally and independently (where possible) using given/when/then acceptance criteria, business rules and interface designsA feature or epic normally contains many stories
The team or a representative of each function in the team meets with the product owners to estimate the storiesThe team picks some completed small, medium and large stories to use as a baseline.For each story the team asks questions, clarify assumptions, vote, explain differences, ask more questions then revote until everyone agrees.Only people actually doing the work get to vote.The team can estimate at different levels of abstraction – features or storiesAll estimates should be about overall team effort not just individual role effort
The estimating meeting proceeds as follows:A Moderator, who will not play, chairs the meeting.The Product Manager provides a short overview. The team is given an opportunity to ask questions and discuss to clarify assumptions and risks. A summary of the discussion is recorded by the Business Analyst.Each individual lays a card face down representing their estimate. Units used vary - they can be days duration, ideal days or story points. During discussion, numbers must not be mentioned at all in relation to feature size to avoid anchoring.Everyone calls their cards simultaneously by turning them over.People with high estimates and low estimates are given a soap box to offer their justification for their estimate and then discussion continues.Repeat the estimation process until a consensus is reached. The developer who was likely to own the deliverable has a large portion of the "consensus vote", although the Moderator can negotiate the consensus.The cards are numbered as they are to account for the fact that the longer an estimate is, the more uncertainty it contains. Thus, if a developer wants to play a 6 he is forced to reconsider and either work through that some of the perceived uncertainty does not exist and play a 5, or accept a conservative estimate accounting for the uncertainty and play an 8.A study by K. Molokken-Ostvold and N.C. Haugen found that estimates obtained through the Planning poker process were less optimistic and more accurate than estimates obtained through mechanical combination of individual estimates for the same tasks.
After the estimates have been done the team works with the product owners to organize the work into releases according to team velocity, business priority and dependencies. The product owner can move stories from one release or iteration to another as long as they have been started yet.You can have stories for infrastructure elements.Functional and performance testing is done progressively by the team and stakeholders throughout the release so there is no need for a test fix phase at the end of the release.The product owner can move stories the customer wants an estimate for 6 months out then estimate on featuresDifferent people and teams work at different rates so if the team structure changes the estimate is no longer validFor costing purposes you can divide the team cost and effort for one iteration by the velocity to get the cost and effort per story point
When velocity is unknown the team tends to produce overly optimistic projections. Then the inevitable complications and delays of set up tasks mean less is delivered than projected.The best approach is to use a combination of more traditional estimation measures to begin with and incorporate velocity feedback progressively as it becomes available.
If you don’t know the teams velocity then you may need to estimate the work from average days of effort.Break the work down as before into stories and ask the team to estimate the days of work they have to do.Keep in mind that development effort is half the teams work as shown in the pie chart.TypicallyOne QA can support 3 Dev’s, One UX can support4-5 Dev’s, One BA can support 4 Dev’s and 1 UX, One PM or team leader can support a team of 10. For example: if the developers say that a story will take them 4 average days then it will take the BA 1 day, UX 1 day, QA 1.5 days and the PM/TL 0.5 days. So it will take the whole team 8 days work .Why do we need a PM, BA, UX and QA? It’s because every story has to have requirements defined, UI designed, tests defined and executed and bugs identified plus the team needs a lead to organise things for them with the client and other areas and remove blockers.
If you’re estimating hours because you don’t know the teams velocity then after you’ve done the estimate for up front work you need to add an estimate for rework.Changes that are required after the requirements have been agreed is rework. Bugs are a consequence of the nature of human factors in the programming task. They arise from oversights or mutual misunderstandings made by a software team during specification, design, coding, data entry and documentation. More complex bugs can arise from unintended interactions between different parts of a computer program.Most defects and changes are found during testing and raised as bugs. Significant changes in requirements are new stories or change requests.Rework happens when we learn what the real requirements, design, code and tests should be. For example a tester or user finds that a part of a story that a developer has completed does not meet the requirements as defined in the story and test cases or it does meet them at the high level but it causes unwanted and unexpected beahviour.Defect resolution requires work by the whole team the BA, tester, UI designer and developer.Developers rarely estimate for rework when estimating a story.Rework increases when: communication between the stakeholders and the team is difficult. I.e. When the team is in another office, organization, country or time zone and has a different language, culture and KPI’s.the team is inexperienced in software development i.e. if most of the team has <= 3 years experience in developmentthe person who fixes the defect is different to the person who wrote the codeThe team has processes that slow down learning and communication i.e. waterfallThe team does not write automated acceptance tests before developing The client is learning what they need as they goExample: If you have an average team that says a story will take 8 days effort (4 days dev, 1 BA, 1 UZ, 1.5 QA, 0.5 PM) then you need to add another 3 days effort for rework by the team. It’s worthwhile to invest in reducing rework.
Software development is about automating manual processesWhen we know something really well we automate it by developing products, frameworks, tools and putting it in productionWhich means that every software development project is new – either we are using new tools or building something new with themWhich means that there are always a lot of unknown requirements and solutions that we cannot know until we have done the projectAttempting to remove all the unknowns through big up front analysis and design is impossible and results in over specifying and over building solutionsThe customer and the team work together iteratively to identify and develop the most important features to the business
Waterfall projects are often big, bloated, slow and never endingAgile projects are lean, fast and adaptable
Agile Tools and Graphs – 60%Project retrospectives – 60%The business case for Agile – 60%Initiating an Agile project – 50%Agile Deployment and Dev Op’s – 50%Scaling Agile for Large Teams – 50%Reduce wasted time and effort in software development – 40%Agile Kanban vs Agile Scrum – 10%Agile Embedded Systems – 5%
Should discuss that planning and estimation is iterativeTalk about using burn down charts for planning once in progressCan we get a copy of the slidesGood but targeted at beginnersYour estimates of rework in different teams was good.Really enjoyed it. Will go back to the office and relook at estimates.The estimation cycling analogy was good.Good organization. Slides will be good.Its good that you know both agile and other PM approaches well and can compare them.Great talk. Excellent perspective with strong experience.Inclusion of metrics from own experience was excellent.Was great when discussion started.Good that you mentioned that Agile does not equal Scrum. Might be good to do that earlier.Thanks. It’s a great session. Keep it up.Good. Commercial perspective,. Agile reality.There was a lot to discuss. May need more time.The effect of rework was good.Well presented and questions answered well.Good sessions. Thanks a lot.Informative and valuable thanks.Great to hear other people have the same problems as we do.Good to get people in the room talking through real life examples and questions.Should estimate activities related to deployment as well as development.The team pie charts seemed a little prescriptive. Reality is very context dependent. E.g. multi skilled teams.Very well explained and realistic.