1. SoberIT
Software Business and Engineering Institute
Leftovers from Tuesday
… and excellent bridge to today’s topic
HELSINKI UNIVERSITY OF TECHNOLOGY
2. SoberIT
Software Business and Engineering Institute
Software project success rates 2000
(2003)
23%
(15%)
Successful: on time, on
budget, all features
49% Challenged: Completed and
(51%) operational, but over-budget,
over time, fewer features
Failed: Cancelled
28%
(34%)
Challenged
Succeeded
Failed
HELSINKI UNIVERSITY OF TECHNOLOGY
(Standish Group, based on US data)
3. SoberIT
Software Business and Engineering Institute
Failure statistics – Improvements?
Average cost overrun 189 % (1994)->45 % (2000)-> 43% (2003)
Average time overrun 222% (1994)->63 % (2000)-> 82% (2003)
On average 61 % (1994) of required features were delivered -> 67
% (2000) -> 52% (2003)
HELSINKI UNIVERSITY OF TECHNOLOGY
(Standish Group, based on US data)
4. SoberIT
Software Business and Engineering Institute
Reasons for success and failure
Reasons for failure: Recipe for success:
“Most projects failed for lack Smaller project size and
of skilled project management shorter duration
and executive support”
“Underestimating project
More manageable
complexity and ignoring “Growing”, instead of
changing requirements are “developing”, software
basic reasons why projects engages the users earlier and
fail” confers ownership.
“The problem – and the -> Iterative and
solution – lay in people and interactive process
processes”
HELSINKI UNIVERSITY OF TECHNOLOGY
(Standish Group, based on US data)
5. SoberIT
Software Business and Engineering Institute
T-76.5612 Software Project Management
Spring 2010
2: Selecting a Process Model
Tuomas Niinimäki
Department of Computer Science and Engineering
Helsinki University of Technology
HELSINKI UNIVERSITY OF TECHNOLOGY
6. SoberIT
Software Business and Engineering Institute
Process
A sequence of steps performed for a given
purpose
[IEEE Std 610.12-1990]
HELSINKI UNIVERSITY OF TECHNOLOGY
7. SoberIT
Software Business and Engineering Institute
“A set of activities, methods, practices, and
transformations that people use to develop and
maintain software and the associated products”
Reasons for failure:
“Most projects failed for lack of skilled project management and
executive support”
“Underestimating project complexity and ignoring changing
requirements are basic reasons why projects fail”
“The problem – and the solution – lay in people and processes”
HELSINKI UNIVERSITY OF TECHNOLOGY
8. SoberIT
Software Business and Engineering Institute
Why do we need processes in
software projects?
HELSINKI UNIVERSITY OF TECHNOLOGY
9. SoberIT
Software Business and Engineering Institute
Process modeling objectives
Harmonize and standardize work at the project level
Support project planning and management
Enable understanding and communication about the process
Gain better overall control of the projects in the process
Facilitate process reuse
HELSINKI UNIVERSITY OF TECHNOLOGY
10. SoberIT
Software Business and Engineering Institute
Software Development Process Models
Waterfall model
Iterative and incremental models
Synchronize and Stabilize
Unified Process
Agile models
XP (eXtreme Programming)
Scrum
HELSINKI UNIVERSITY OF TECHNOLOGY
11. SoberIT
Software Business and Engineering Institute
Project constraints
Resources
Time Scope
HELSINKI UNIVERSITY OF TECHNOLOGY
12. SoberIT
Software Business and Engineering Institute
Different viewpoints on a project
£
Cost / Benefit
Temporary organization
Project X
€$
Sequence of work to be done
Collection of products
HELSINKI UNIVERSITY OF TECHNOLOGY
14. SoberIT
Software Business and Engineering Institute
The Waterfall Model
The “classical” model for
software development
Royce 1970
Document driven
Each phase produces
documents
Freeze
Change management process
used after each phase
“The whole application is
developed in one go”
“Limited scope for iteration”
HELSINKI UNIVERSITY OF TECHNOLOGY
15. SoberIT
Software Business and Engineering Institute
The Waterfall Model
What Royce actually said:
1. Complete program design
before analysis and coding
begins
2. Documentation must be
current and complete
3. Do the job twice if possible
4. Testing must be planned,
controlled and monitored
5. Involve the customer
Fig 3. Hopefully, the iterative interaction between
the various phases is confined to successive steps
HELSINKI UNIVERSITY OF TECHNOLOGY
17. SoberIT
Software Business and Engineering Institute
“[Figure above] summarizes the five steps that I feel necessary to
transform a risky development process into one that will provide the
desired product. I would emphasize that each item costs some additional
sum of money.
If the relatively simpler process without the five complexities described
here would work successfully, then of course the additional money is not
well spent.
In my experience, however, the simpler method has never worked on
large software development efforts and the costs to recover far exceeded
those required to finance the five-step process listed.”
HELSINKI UNIVERSITY OF TECHNOLOGY
Royce 1970
18. SoberIT
Software Business and Engineering Institute
Under what conditions
waterfall model could be
applicable?
HELSINKI UNIVERSITY OF TECHNOLOGY
19. SoberIT
Software Business and Engineering Institute
The Waterfall Model
Strengths
Easily manageable process (manager’s favorite?)
Probably the most effective model, if you know the requirements
Extensive documentation
Weaknesses
Inflexible partitioning of the project into distinct stages
Difficult to respond to changing customer requirements
Feedback on system performance available very late and changes
can be very expensive
Applicability
Appropriate when the requirements are well-understood
Short, clearly definable projects
Very large, complex system development, that requires extensive
documentation, safety critical systems
HELSINKI UNIVERSITY OF TECHNOLOGY
20. SoberIT
Software Business and Engineering Institute
Iterative and incremental
development
HELSINKI UNIVERSITY OF TECHNOLOGY
21. SoberIT
Software Business and Engineering Institute
Iterative and incremental development
(IID)
Divide the project into iterations
Each iteration builds on previous iteration = adds new
functionality = increment
Freeze requirements during each iteration
Each iteration is a self-contained mini-project composed of
nearly all software development activities (e.g. requirements
analysis, design, implementation, testing)
At the end of each iteration: an iteration release = a stable,
integrated and tested partially complete system
Most intermediate releases are internal
Release from the final iteration is the complete product
HELSINKI UNIVERSITY OF TECHNOLOGY
22. SoberIT
Software Business and Engineering Institute
Timeboxing
Fixing the iteration end date and not allowing it to change
1-6 weeks is normal length for a timeboxed iteration
Over 2 months is usually too long and misses the point and value
Shorter steps have:
Lower complexity and risk, better feedback, and higher
productivity and success rates
Higher overhead due to administrative tasks, planning, releasing
etc.
HELSINKI UNIVERSITY OF TECHNOLOGY
23. SoberIT
Software Business and Engineering Institute
Timeboxing and requirements
Prioritize user requirements
MOSCOW priorities: must, should, could, want
High-priority requirements into early iteration
Involve developers and the customer in planning
Freeze requirements during each iteration
HELSINKI UNIVERSITY OF TECHNOLOGY
24. SoberIT
Software Business and Engineering Institute
Timeboxing and scoping
If iteration goals are not going to be met:
The iteration scope is to be reduced, rather than moving the
iteration end date further
=> Lower priority requests back on the wish list
Timeboxing should not be used to pressure developers to work
long hours to meet the deadline
If normal pace of work is not enough, do less!
HELSINKI UNIVERSITY OF TECHNOLOGY
25. SoberIT
Software Business and Engineering Institute
Concurrent iterations
At least in a larger software project/program, iterations / tasks
can be concurrent:
RE 1 RE 2 RE 3 RE 4 RE 5
Impl 1 Impl 2 Impl 3 Impl 4
Testing 1 Testing 2 Testing 3 Testing 4
HELSINKI UNIVERSITY OF TECHNOLOGY
26. SoberIT
Software Business and Engineering Institute
Concurrent iterations
At least in a larger software project/program, iterations / tasks
can be concurrent:
RE 1 RE 2 RE 3 RE 4 RE 5
Feedback!
Impl 1 Impl 2 Impl 3 Impl 4
Testing 1 Testing 2 Testing 3 Testing 4
HELSINKI UNIVERSITY OF TECHNOLOGY
27. SoberIT
Software Business and Engineering Institute
Iterative and incremental development
In the beginning:
Prioritize features
Make a release plan
Plan the scope for
the first iteration Release 5
Release 4
Release 3
Release 2
Release 1
Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5
Starting
date Predefined
Delivery date
HELSINKI UNIVERSITY OF TECHNOLOGY
28. SoberIT
Software Business and Engineering Institute
Iterative and incremental development
Plan A:
Synchronize & Stabilize! project’s outcome if
Plan B: everything goes fine
project’s outcome if
something goes wrong
Release 5
Release 4
Release 3
Release 2
Release 1
Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5
Starting
date Predefined
Delivery date
HELSINKI UNIVERSITY OF TECHNOLOGY
29. SoberIT
Software Business and Engineering Institute
Under what conditions
iterative and incremental
development (IID) model
could be applicable?
Release 5
Release 4
Release 3
Release 2
Release 1
Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5
Starting
date Predefined
Delivery date
HELSINKI UNIVERSITY OF TECHNOLOGY
30. SoberIT
Software Business and Engineering Institute
IID advantages
Added value for the customer value is delivered at the end of each
increment
System functionality is available earlier
Early increments act as a prototype to help
elicit requirements for later increments
get feedback on system performance
increase the job satisfaction and motivation of the development
team, as results of their work is seen earlier
Smaller sub-projects are easier to control and manage
A meaningful progress indicator: tested software
Final product better matches the true customer needs
Lower risk of overall project failure
The highest priority system services tend to receive the most testing
HELSINKI UNIVERSITY OF TECHNOLOGY
31. SoberIT
Software Business and Engineering Institute
IID disadvantages
Can be more difficult to plan and control than waterfall development
Can end up being more expensive than waterfall development
Later increments may require modifications to earlier increments
System architecture must be adaptive to change
May require more experienced staff
Software project contracts are still mostly drawn according to the
waterfall model and all changes cause renegotiations
HELSINKI UNIVERSITY OF TECHNOLOGY
33. SoberIT
Software Business and Engineering Institute
Manifesto for Agile Software Development
We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
http://agilemanifesto.org/ 2001
HELSINKI UNIVERSITY OF TECHNOLOGY
34. SoberIT
Software Business and Engineering Institute
Agile software development
Agile software development methods are a subset of IID methods
Scrum, eXtreme Programming (XP), ...
Timeboxed, iterative and evolutionary development
Adaptive planning
Evolutionary delivery
Rapid and flexible response to change
Self-organized, self-directed and cross-functional teams
Programming over documenting
HELSINKI UNIVERSITY OF TECHNOLOGY
35. SoberIT
Software Business and Engineering Institute
Agile software development methods
Deliver something useful to the client
Cultivate commited stakeholders
Employ a leadership-collaboration style
Build competent, collaborative teams
Enable team decision making
Use short timeboxed iterations to deliver quickly features
Encourage adaptability
Focus on delivery, not process-compliace activities
(Jim Highsmith)
HELSINKI UNIVERSITY OF TECHNOLOGY
37. SoberIT
Software Business and Engineering Institute
Scrum - terms and definitions
Scrum meeting:
Stand-up meeting for 15 minutes (usually daily) with 3
questions
Heartbeat:
Time between two scrum meetings (usually 24h)
Sprint:
= iteration (usually 2-4 weeks)
Release:
Product delivered to a customer
Backlog:
Prioritized list of backlog items (“tasks”)
HELSINKI UNIVERSITY OF TECHNOLOGY
38. SoberIT
Software Business and Engineering Institute
Scrum - roles
ScrumMaster:
Primary job is to remove impediments (obstacles) in order to
make the team able to fulfill the sprint goals
Ensures that Scrum process is used as intended
Product owner:
Represents the customer
Ensures that the Scrum team works efficiently in business
point of view
Team:
Responsible for delivering the product
HELSINKI UNIVERSITY OF TECHNOLOGY
39. Example: Using Scrum based Framework
Product backlog
Strategic Release Management
Allocated into roadmap
as
Release backlogs (R&D Process)
Parts allocated into Release Project Cycle 3 months
Sprint backlog
Sprint 1 month
Development
Business
Release process with go/kill gates
41. SoberIT
Software Business and Engineering Institute
Process Choice
There are several factors Risks ~
that affect which type of - novelty of
process you should use: methods,
tools,
Product type application
Business type Iterative, prototyping
domain
Project size - fuzziness of
requirements
Project initial Incremental
uncertainty
Environmental
uncertainty
Organizational culture Plan-driven
…
Product
size/complexity
HELSINKI UNIVERSITY OF TECHNOLOGY
42. SoberIT
Software Business and Engineering Institute
Process Choice
There is no uniformly applicable process which
should be standardized within an organization if there are, e.g., different
types of products and/or markets!
High costs may occur if you force an inappropriate process on a
development team
However, one organization cannot have a large number of different
processes in use
Sometimes you don’t have a possibility to choose
A chosen process model can be tailored to suite the organization/
project
Taking a new process into use is always a large task – that should be
planned from the point of view of the whole organization
HELSINKI UNIVERSITY OF TECHNOLOGY
43. SoberIT
Software Business and Engineering Institute
Process Choice
For large systems, management is usually the principal problem
so you need a strictly managed process
For smaller systems more informality is possible
Hybrid models:
Large systems are usually made up of several sub-systems
The same process model need not be used for all
subsystems, e.g.
Prototyping for high-risk specification
Waterfall model for well-understood developments
HELSINKI UNIVERSITY OF TECHNOLOGY
44. SoberIT
Software Business and Engineering Institute
(Boehm & Turner 2003)
Plan-driven vs. agile
Plan-driven (e.g. Waterfall) Agile
Application: Application:
• Primary goals: Predictability, stability, • Primary goals: Rapid value, responding
high assurance to change
• Size: Larger teams and projects • Size: Smaller teams and projects
• Environment: Stable, low change, project • Environment: Turbulent, high-change,
and organization focused project focused
HELSINKI UNIVERSITY OF TECHNOLOGY
(Boehm & Turner 2003)
45. SoberIT
Software Business and Engineering Institute
(Boehm & Turner 2003)
Plan-driven vs. agile
Plan-driven (e.g. Waterfall) Agile
Management: Management:
• Customer relations: As-needed • Customer relations: Dedicated onsite
customer interactions, customers,
focused on contract provisions focused on prioritised increments
• Planning and control: Documented • Planning and control: Internalised plans,
plans, quantitative control qualitative control
• Communications: Explicit documented • Communications: Tacit interpersonal
knowledge knowledge
HELSINKI UNIVERSITY OF TECHNOLOGY
(Boehm & Turner 2003)
46. SoberIT
Software Business and Engineering Institute
(Boehm & Turner 2003)
Plan-driven vs. agile
Plan-driven (e.g. Waterfall) Agile
Technical: Technical:
• Requirements: Formalized project, • Requirements: Prioritised informal stories
capability, interface, quality, foreseeable and test cases, undergoing unforeseeable
evolution requirements change
• Development: Extensive design, longer • Development: Simple refactoring, short
increments, refactoring assumed expensive increments, refactoring assumed inexpensive
• Test: Documented test plans and • Test: Executable test cases define
procedures requirements, testing
HELSINKI UNIVERSITY OF TECHNOLOGY
(Boehm & Turner 2003)
47. SoberIT
Software Business and Engineering Institute
(Boehm & Turner 2003)
Plan-driven vs. agile
Plan-driven (e.g. Waterfall) Agile
Personnel: Personnel:
• Customers: Knowledgeable, not always • Customers: Knowledgeable, dedicated and
collocated collocated (customer proxy)
• Developers: 50 % Level 3s early; 10 % • Developers: At least 30 % full-time level 2
throughout; 30 % 1B’s workable; and 3 experts;
No level -1s No level 1B or Level -1 personnel
• Culture: Comfort and empowerment via • Culture: Comfort and empowerment via
framework of policies and procedures many degrees of freedom (thriving on chaos)
(thriving on order)
HELSINKI UNIVERSITY OF TECHNOLOGY
(Boehm & Turner 2003)
48. SoberIT
Software Business and Engineering Institute
People-factor: (Boehm & Turner 2003,
modified from Cockburn)
A Developer Classification
Level Characteristics
3 Able to revise a method (break its rules) to fit an
unprecedented new situation
2 Able to tailor a method to fit a precedented new situation
1A With training able to perform discretionary method steps (e.g.,
sizing stories to fit increments). With experience, can become
Level 2.
1B With training, able to perform procedural method steps (e.g.,
simple refactoring). With experience, can master some Level
1A skills.
-1 May have technical skills, but unable or unwilling to collaborate
or follow shared methods
HELSINKI UNIVERSITY OF TECHNOLOGY
49. Dimensions Affecting Process Model
Selection Personnel (Boehm & Turner 2003)
(Percent level 1B) (Percent level 2 and 3)
40 15
30 20
20 25
Criticality Dynamism
(Loss due to impact (Percent requirements
on defects) Single 10 30 change/month)
life
Discretionary
Many 1
funds 0 35 5
lives
10
Essential
funds 30
Comfort 50
3
90 Ag
10 ile
70
Pla
30 n-d
r ive
50 n
100
30
Size 300 Culture
(# of personnel) (% of thriving on chaos vs. order)
10
Size Culture
(Number of personnel) (Percent thriving on chaos versus order)
50. SoberIT
Software Business and Engineering Institute
(Boehm & Turner 2003)
Plan-driven vs. Agile: 5 Critical Factors
Plan-driven discriminators Agility discriminators
Size: Size:
• Methods evolved to handle large • Well matched to small products and
products and teams; hard to tailor teams; reliance on tacit knowledge
down to small projects limits scalability
Criticality: Criticality:
• Methods evolved to handle highly • Untested on safety-critical products;
critical products; hard to tailor down potential difficulties with simple design
efficiently to low-criticality products and lack of documentation
Dynamism: Dynamism:
• Detailed plans and "big design up • Simple design and continuous
front" excellent for highly stable refactoring are excellent for highly
environment, but a source of dynamic environments, but present
expensive rework for highly a source of potentially expensive
dynamic environments rework for highly stable environments
HELSINKI UNIVERSITY OF TECHNOLOGY
51. SoberIT
Software Business and Engineering Institute
(Boehm & Turner 2003)
Plan-driven vs. Agile: 5 Critical Factors
Plan-driven discriminators Agility discriminators
Personnel:
• Need a critical mass of scarce level 2
Personnel:
and 3 experts during project definition,
• Require continuous presence of
but can work with fewer later in the
critical mass of scarce level 2
project - unless the environment is
and 3 experts
highly dynamic
• Risky to use non-agile level 1B people
• Can usually accommodate some level 1B
people.
Culture:
Culture:
• Thrive in a culture where people feel
• Thrive in a culture where people feel
comfortable and empowered by having
comfortable and empowered by having
their roles defined by clear policies and
many degrees of freedom
procedures
• Thrive on chaos
• Thrive on order.
HELSINKI UNIVERSITY OF TECHNOLOGY
52. SoberIT
Software Business and Engineering Institute
Plan-driven vs. Agile: Conclusions
Neither agile nor plan-driven methods provide a silver bullet
Both have areas where one may suit better than the other
Both agility and discipline are critical to future software success
Increasingly rapid pace of change
E.g. larger projects using agile methods need plan-driven
process aspects to be integrated and to be successful
Build your method up – don’t tailor it down
HELSINKI UNIVERSITY OF TECHNOLOGY
53. SoberIT
Software Business and Engineering Institute
“[Figure above] summarizes the five steps that I feel “We are uncovering better ways of developing software
necessary to transform a risky development process by doing it and helping others do it.
into one that will provide the desired product. I
would emphasize that each item costs some Through this work we have come to value:
additional sum of money.
Individuals and interactions over processes and tools
If the relatively simpler process without the five
Working software over comprehensive documentation
complexities described here would work vs.
successfully, then of course the additional money is
not well spent. Customer collaboration over contract negotiation
Responding to change over following a plan
In my experience, however, the simpler method has
never worked on large software development efforts
and the costs to recover far exceeded those
required to finance the five-step process listed.”
That is, while there is value in the items on the right,
we value the items on the left more.”
HELSINKI UNIVERSITY OF TECHNOLOGY
54. SoberIT
Software Business and Engineering Institute
Recipe for success
Smaller project size and shorter duration
More manageable
“Growing”, instead of “developing”, software engages the users
earlier and confers ownership.
-> Iterative and interactive process
HELSINKI UNIVERSITY OF TECHNOLOGY
(Standish Group, based on US data)
55. SoberIT
Software Business and Engineering Institute
Thank you.
Questions?
Tuomas Niinimäki
tuomas.niinimaki@soberit.hut.fi
HELSINKI UNIVERSITY OF TECHNOLOGY