2. Agenda
• Phase 1 – Introduction and Why Agile
– Identify the need for change
– Apprise Executive Management on Agile Organizations
• Phase 2 – Familiarize Current Picture
– Gauge current state of affairs
– Propose the change and ways to measure impact
• Phase 3 - In-Detail on Principles and Practices
– Apprise Team
– Identify and Impart Trainings
• Phase 4 – Applying Principles and Practices
– Select Pilot Project
– Showcase agile principles and methods in practice
– Challenges
– Next Steps
• Phase 5 – Inspect and Adapt
5. How Did We Get Here?
Incremental and iterative development has been around for a
long time.
1960s 1970s 1980s 1986 1990s 1991 2000s 2001 2003
EVO Spiral
Model
Scrum
First Coined
XP
DSDM
RUP
Scrum
Formalized
AUP Agile LeanWaterfall
Defined
Time Line
6. February 2001 Snowbird, Utah
• 17 proponents of iterative, incremental and lightweight
methods
• They agreed on values based on trust and respect
• The Agile Alliance formed
• The Agile Manifesto written
8. “Agile” is …
• a way of developing software that is consistent
with the principles of the Agile Manifesto
• a framework within which complex products in
complex environments are developed
• the ability to adapt to change and evolve
throughout the lifecycle of the project
9. Agile Practices
Scrum
Automated Testing Open Workspace
Test Driven Development
Executable Requirements
XP
Small Frequent Releases Coding Standards
OOAD and Patterns
Customer Team Member
Acceptance Testing Sustainable Pace
11. Usage of Features
and Functions
Always
7%
Often
13%
Sometimes
16%
Rarely
19%
Never
45%
Source: Standish Group Study of 2000 projects at 1000
companies
Usage of Features and Functions In Typical System
12. More Bad News…
• 66%1 ofprojectssignificantlyoverruntheirbudget.
• 70%2 ofdefectsinproductionareintroducedintherequirementsstage
• 70%3 ofthetotalcostofownershipforsoftwareismaintenance.
14. Effectiveness of standard “Waterfall” methods
1. Source: Standish Group, 2005 Chaos Report
2. Source: J. Johnson, Keynote speech, XP 2002 Italy
• Theaverageprojectexceedsitsscheduleby84%1
• 64%2offeaturesdeliveredareneverused.
15. What Businesses are interested in:
• Delivering fewer defects
• Delivering what the customer wants
• Open communication with the customer
• Being trusted by your customer
• Showing working software frequently
• Having control over the way you work
• Understanding who is supposed to do what
• Working at a sustainable pace
• Finding more satisfaction in your work
16. Agile is a Response to
• Too many artifacts
• Too much documentation
• Inflexible plans
• Late, over budget and buggy software
• Not delivering what the customer wants
17. Business Case for Agile
• Quicker to Market
• Better Understanding of Customer Needs
• Higher Customer Satisfaction
• Lower Risk Through Early Discoveries
• Lower Risk by Delivering Incrementally
• Higher Return On Investment
• Accelerates ROI
• Progress is Measured by Actual Working Software
Thereisan85%
overheadreduction
goingtoanAgile
model
18. Paradigm Shift
(Changing a “mass manufacturing, assembly line” mindset to an “innovative, new-product” mindset)
19. Acceptance Test
Driven Development
(Executable
Requirements)
Automated
Integration Testing
(Interface Driven
High Level Design)
Unit Test
Driven Design
(Low Level Design)
Unit Test
Driven Development
(Implementation)
V-Model to I-Model
(Changing a “waterfall” verification and validation testing mindset to test-driven defect prevention mindset)
26. Agile Organizations
• Cross-functional teams aligned directly to
solve business problems
• Products
• Features
• Programs
• Components
• Services
• Business Capabilities
27. Agile Organizations
• Clear voice of the business and a willingness
to make tradeoffs to meet time and cost
constraints
• Highly engaged product ownership
• Willingness to deal with reality
• Focus on maximizing value and reducing risk
28. Agile Organizations
• Individual empowerment and shared
accountability for outcomes
• Establish boundaries and ownership but empower
within those boundaries
• Teams own outcomes not activities
29. Agile Organizations
• Disciplined attention to technical excellence
and product quality
• Technical excellence stabilizes the requirements
delivery function
30. Agile Organizations
• Predictable, accountable, able to consistently
make and meet commitments
• Teams have the ability to consistently do what they say
they are going to do
• Predictable agile teams are the foundational element
of a predictable agile enterprise
32. Adoption vs. Transformation
• First... Let’s understand the two terms that often
used interchangeably
– Agile Adoption is about what we do...
practices, tools, techniques, ceremonies, and habits
– Agile Transformation is about who we are...
reflected in both the structure of the organization
and who we are as people
• For long term success organizations need both adoption and
transformation
33. Adoption vs. Transformation
• Second... we want clearly explain the three major
focus areas that must be addressed
• Organizational Structure is about how you create teams
and how you organize them
• Agile Practice is about the methods and tools you choose
to introduce
• People and Culture is about changing hearts and minds of
the individuals in the organization
– All three aspects are essential to sustain agility of any
organization
34. Incremental vs. Iterative
• Third... we want introduce the notion that
introducing Agile is an iterative and incremental
process for you organization
• Iterative is when parts of the system are developed at
different times and integrated as they are completed
• Incremental is when you go back over parts of the system
making improvements
– The strategy is to increment the organization by
building teams and iterate the teams over time
35. Adoption/Transformation Cycle
• Incrementing and Iterating the Agile
Enterprise
– Change physical structures and introduce teams
– Teach people new practices and ways of working
– Help people internalize the value system
36. Adoption/Transformation Cycle
• Organizational Transformation
– Establish top to bottom structure and roadmap
– Incrementally make changes and establish teams
– Define policies and working agreements between
teams
37. Adoption/Transformation Cycle
• Adopting Practices
– Sprint planning, daily stand-ups, product
reviews, and retrospectives
– Identify and train a Product Owner and Scrum
Master
– Teach TDD, CI, Story Maps, and MMF
38. Adoption/Transformation Cycle
• Personal Transformation
– Develop an ability to deal with uncertainty and
adaptation
– Help people work toward common organizational
goals
– Help foster empathy, trust, and teamwork
39. Common Anti-Patterns
• Establishing teams without breaking down
the strict functional silos and rigid role
definitions
• Running daily standup meetings that
devolve into status updates for the project
manager
• Coming back from Agile training only to find
that there is no way to form agile teams and
no interest in agile
42. Understanding Current State of Affairs
• Management Engagement
• Team Engagement
• Measure Current Process Maturity
• SWOT Analysis
43. Management/Leadership Engagement
• Engage with mid-level to upper level managers
• Talk to them to understand on how they perceive
general work trends and issues/concerns and
current state of affairs
• Create openings for collaboration
• Attend as many currently running meetings as
possible
• Just Listen – Don’t offer any suggestions or
solutions
44. Team Engagement
• Engage with teams on the shop floor
• Talk to as much individual team members on
general work trends and issues/concerns and
solutions, if any
• Create rapport with teams
• Attend as many currently running meetings as
possible
• Just Listen – Don’t offer any suggestions or
solutions
45. Measure Current Process Maturity
• Engineering Processes (Quality Management
System)
• Engineering Practices
• Compliance to Mandatory Regulations
(FDA, SFDA, etc.)
46. SWOT Matrix
• Prepare SWOT Matrix to find out
– Areas of strength
– Areas for improvement
– New opportunities/practices to adopt
– Threats if not adapted to changing environment
w.r.t business
48. Current State of Affairs
• Inorganically grown organization
• Waterfall + iterative model for development
• Specialised teams for each function
(Development, Testing, Product
Management, etc.)
• Teams are always busy with new programs or
maintaining existing ones
49. Other Challenges
• Persistent customer complaints w.r.t quality and
performance of the products
• Significant cost and schedule overrun
• Silos
• No clear understanding of progress made
• No common processes followed
• Frequently changing requirements
• Most time spent on fire fighting
• High attrition rates
52. Proposed Solution
• Adopt Agile principles and practices to have short-
cycled releases to facilitate instant/early feedback from
Customers
• Invite selected customers for demos and pilot
programs to build trust and transparent
communication
• Encourage and incentivise cross-team communication
and collaboration
• Provide and environment conducive of collaborative
development (as proposed by Agile principles)
• Create Product Owners with responsibility for the ROI
of the product lines
53. Which Agile Practices to Adopt
• Scrum
– Lightweight Framework for Program/Product
Management
– Very minimal prescriptions
– Importance to Customer value generation all the time
• XP (eXtreme Programming)
– Solid engineering practices and principles
• Portions of UP (Unified Process)
– Enhances product vision and goals for better
execution of business strategy and goals (long and
short term)
54. Why Scrum
• Widely adopted agile practice
• Promotes Short-cycle releases instead of big-bang releases
– Thereby increases opportunity for early customer feedback and
improvement opportunities
• Bring in the concept of Product Owner (PO)
– Responsible for product life cycle and ROI
• Promotes the idea of Potentially Shippable Product Increments
(PSPI)
• Promotes Iterative development (called Sprints)
– Short, time-boxed delivery of mutually agreed items between team
and PO.
• Requires cross-functional teams (Dev+Test+Product Management)
– Breaks the silos and creates a multi-talented teams capable of tackling
business priorities
55. Why Scrum
• Moots the concept of self-managed teams
– Places the job of estimating tasks and feature into the people who
actually develops them
– Improved estimates
• Team only picks items/features that are clear and
unambiguous
• Requires Prod. Mgr ( or Product Owner) to constantly
reprioritise requirements (product backlog) according to
changing business needs
• Promotes minimal upfront design so as to improve scalability
and minimised impact of requirement changes down the line
• Promotes review and retrospection of the iterations
completed to improve practices and process
56. Why XP
• Promotes a set of Engineering Practices that will
enhance productivity and quality of the products
developed
– TDD (Test Driven Development)
– Pair Programming (detects and help to remove code
defects very early)
– CI (Continuous Integration) to make sure code quality
is maintained
– Refactoring to remove any technical debts
• Inline with agile principles and complements
Scrum framework
57. Why UP (Unified Process)
• Establish product and release vision
– Helpful in prioritising requirement for a release
– Helps to identify the targeted customer base w.r.t a
release
• Agile Modelling
– Helps to have an overall architectural design in mind
– Brings in architectural clarity w.r.t items chosen for a
given iteration
– Helps in measuring architectural impact of changed or
new requirements brought into the release
60. Proposed steps for Rollout – Principles and Practices
• Ensure Executive buy-in
• Implement Scrum as program/product
management framework
• Introduce XP Engineering practices
• Small, incremental rollout is proposed
• Identify pilot product to implement the agile
principles and practices
– Create cross-functional, co-located teams
– Create open workspace for better collaboration
62. Pilot Rollout – Trainings
• Agile and Scrum trainings for the entire team
– Developers
– Testers
– Product Owner/Product Manager
– Identify Scrum Master(s)
– Requirements writing (Stories, Use Cases)
– Estimation Techniques
• Relative Estimation Technique
• Story point Estimates
• Planning Poker
63. Pilot Rollout – Trainings
• Engineering Practices Trainings
– Test Driven Development
– Writing Unit Test Cases
– CI tools (Hudson, CruiseControl, etc.)
– Configuration Management Tools (ClearCase, SVN,
etc.)
– Architectural best practices
– Automated Testing (functional, performance, and
some GUI)
– Automated Build Management including automated
testing (unit and functional)
64. Pilot Rollout – Product Development
• Coach PO and teams on how to plan releases
• Includes
– Vision Workshop
– Identify the release timeframe and requirements that
need to be implemented
– Prioritize the requirements (user stories)
– Estimate requirements
– Participated by Team, PO, Scrum Master, stakeholders
(if any)
– Product Backlog Creation
65. Pilot Rollout – Product Development
• Coaching teams on Sprint Planning
• Includes
– Tactical development plan
– Team commitment
– User Stories are broken down into tasks
– “Definition of Done”
66. Pilot Rollout – Product Development
• Coaching teams on Daily Planning
– Shared accountability
– Daily adjustments
– Generates burndown metrics
70. Pilot Rollout – Tracking Progress
• Collect Metrics
– Velocity (number of story points delivered per
iteration)
– Sprint Burndown (tasks completion rate in an
iteration)
– Release Burndown (stories/features completion
rate)
71. Pilot Rollout – Tracking Progress
• Identify improvement opportunities by
discussing the metrics with teams
– Improving estimation techniques
– Improving requirements/user stories
– Improving technical skills
– Improving physical assets of the team, etc.
• Elicit suggestions from team for improvement
• Assist teams in implementing the suggestions
by talking to appropriate stakeholders
73. Pilot Rollout – Challenges
• High cost of training and related changes in
the organisational structure
• Fear of Loss of productivity during the pilot
phase
• Executive Leadership push for all-out rollout of
agile practices
• Training vs. implementation of Agile and
Scrum principles and Practices
• New methods for estimation (Story points)
74. Pilot Rollout – Challenges
• Management expectation of high/improved
productivity from first iteration itself
• Low productivity during initial iterations
(sprints) induces low morale and confidence
among teams and management
• Push back/inertia from existing team
members against Scrum and Agile adoption
75. Pilot Rollout – Resolving Challenges
• Reiterating the value-based delivery model of
agile
• Clear upfront communication about “reduced”
productivity during initial stages of adoption
• Hand-holding team whenever and wherever
necessary to allay any doubts they might have
w.r.t agile and scrum practices
• Explain the importance of organisational change
and the associated cost
• On-the-shop floor coaching and mentoring
76. Pilot Rollout – Resolving Challenges
• Regular executive updates on adoption
progress, challenges, and resolution to sustain
the confidence level
• Progress towards less-intrusive coaching and
mentoring by creating scrum masters and
technical experts to carry forward the
momentum
• Targeted sessions on different aspects of project
and product management w.r.t agile and scrum
77. Pilot Rollout – Resolving Challenges
• Technical Colloquium to find and moot
reusability across product lines
– Facilitated by Chief/Enterprise Architect
– Can result in creating a platform approached
owned and developed by different teams as
required
– Explore newer architectural best practices that
could be adopted to reduce overall development
cost
78. Pilot Rollout – Resolving Challenges
• Impart trainings and touch-points to improve
– Technical, functional, and domain expertise within
teams
– Cross-pollination via short sessions of different
product lines and their applicability in the market
• Open Workspace
– Improved collaboration
– Reduce email clutter
79. Pilot Rollout – Resolving Challenges
• Weekly Architects’ Round-up
– Identify common components/code
– Continual design (means ahead of next sprint)
• Daily Scrum-of-Scrum
– SMs of all teams to share update with each other
– Identify common problems
– Fix or escalate
80. Pilot Rollout – Resolving Challenges
• Monthly Product Management Round-up
– To recalibrate the product backlogs according to changing
business priorities and release content according to team
velocity
• Monthly deep dive
– To be done with Business Leadership/stakeholders
– To identify and escalate any issues that need leadership
intervention or help
– To be done by scrum masters of each team
82. Next Steps
• Introspect on the outcome of the pilot rollout
with executive management and senior
leadership
• Explain the metrics, areas of improvement,
and future plan
• Secure buy-in from the business stakeholders
for continued rollout
83. Next Steps
• With the help of executive management, select teams
that need to transformed next
– Big bang is still not preferred as we have 40+ teams across
multiple sites
– Make sure to include a multi-site team for transformation
• Introduce Large scale Agile transformation using Scrum
for multi-site teams
• Repeat Phase 2 (if necessary, else start from Phase 3)
through phase 5
• Inspect and adapt practices as and when
transformation progresses
85. Rollout – Sustaining the Change
• Regular executive updates on adoption
progress, challenges, and resolution of issues to
sustain the confidence level
• Progress towards less-intrusive coaching and
mentoring by creating internal coaches, scrum
masters and technical experts to carry forward
the transformation
• Targeted sessions on different aspects of project
and product management w.r.t agile and scrum
87. Appendix
• The contents of this material was influenced
by below mentioned people
– Craig Larman – his work, teachings, and books
have inspired the section on multi-site scaling of
Scrum
– Mike Cottmeyer – his ideas were adopted in the
agile organization and transformation journey
sections
Editor's Notes
This presentation tries to impart strategies to adopt and transform large businesses and organizations into Agile Principles and Practices.This presentation may contain materials that are procured from external sources.Please reach out to me in case you find any material that is been used infringes upon someone’s copyrights.
Agile not born of the dot-com eraIncremental and iterative software development has been around for a long timeMassive growth and development during the last decade.1950’s – Deming Quality CycleEVO – Evolutionary Delivery: still in use today in Europe. Short cycles of mini waterfalls.Waterfall, Royce, 19701970’s – IBM “Integration Engineering Model” was a well known iterative and incremental development approach.Scrum 1986: Source: “The New Product Development Game” by Takeuchi and Nonaka. Harvard Business Review, January 1986.XP – extreme programming.DSDM – Dynamic Systems Development Method (Mostly used in UK)AUP – Agile Unified Process (Scott Ambler)Lean – popularized in 2003 from the Toyota Production WayOthers:KaizenRUP – rational unified processUPInteraction DesignOpenUP/BasicSpiral – spiral model by Boehm (influenced XP)Methodology and mindset go hand-in-hand
http://www.flickr.com/photos/vxla/4266105037/sizes/z/in/photostream/Jim Highsmith’s comments about what happened on that date in that place:On February 11-13, 2001, at The Lodge at Snowbird ski resort in the Wasatch mountains of Utah, seventeen people met to talk, ski, relax, and try to find common ground. And of course, to eat. What emerged was the Agile Software Development Manifesto.Representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming, and others sympathetic to the need for an alternative to documentation driven, heavyweight software development processes convened. Now, a bigger gathering of organizational anarchists would be hard to find, so what emerged from this meeting was symbolic: a Manifesto for Agile Software Development, signed by all participants. The only concern with the term agile came from Martin Fowler (a Brit for those that don’t know) who said that most Americans didn’t know how to pronounce the word “agile”. Alistair Cockburn’s initial concerns reflected the early thoughts of many participants. "I personally didn't expect that this particular group of agilites to ever agree on anything substantive." But his post-meeting feelings were also shared, "Speaking for myself, I am delighted by the final phrasing [of the Manifesto. I was surprised that the others appeared equally delighted by the final phrasing. So we did agree on something substantive."Naming ourselves "The Agile Alliance," this group of independent thinkers about software development, and sometimes competitors to each other, agreed on the Manifesto for Agile Software Development displayed on the title page http”//agilemanifesto.org.But while the Manifesto provides some specific ideas, there is a deeper theme that drives many, but not all, to be sure, members of the alliance. At the close of the two-day meeting, Bob Martin joked that he was about to make a "mushy" statement. But while tinged with humor, few disagreed with Bob’s sentiments that we all felt privileged to work with a group of people who held a set of compatible values, a set of values based on trust and respect for each other and promoting organizational models based on people, collaboration, and building the types of organizational communities in which we would want to work. At the core, Agile Methodologists are really about "mushy" stuff, about delivering good products to customers by operating in an environment that does more than talk about "people as our most important asset" but actually "acts" as if people were the most important, and lose the word "asset". So in the final analysis, the meteoric rise of interest in and sometimes tremendous criticism of Agile Methodologies is about the mushy stuff of values and culture.
The “Manifesto for Agile Software Development” consists of these four core values and 12 principles.
“Agile is an iterative and incremental (evolutionary) approach to software development which is performed in a highly collaborative manner with ‘just enough’ ceremony that produces high quality software in a cost effective and timely manner which meets the changing needs of its stake holders.”Scott AmblerAgile is:a way of developing software that is consistent with the principles of the Agile Manifestoa framework within which complex products in complex environments are developeda collective term for the principles, manifestos, tools, and best practices used to develop software that has, at its fundamental core, the ability to adapt to change and evolve throughout the lifecycle of the project
What does it take to become agile?This is a set of best practices for software development Scrum – follow the scrum a framework Open Workspace - To facilitate communication Executable Requirements – directly testable user stories OOAD and Patterns –Simple Design is hard! Sustainable Pace - The team needs to stay fresh to effectively produce software. Customer Team Member (otherwise known as the product owner) Small Frequent Releases Test Driven Development - Programmers write software in very small verifiable steps. Continuous Integration - Programmers integrate and test the software many times a day. Acceptance Testing - The tests demonstrate that the story is complete. Automated testing – tests of all types automated, unit, acceptance, UI, etc. Run continuously. Coding Standards - The code needs to have a common style to facilitate communicationXP – The Extreme Programming practice set is used is a pragmatic manner.Metaphor or Vision - The system metaphor provides an idea or a model for the system and a domain language. Planning Game - The customer and the development team determine the scope of the next releaseCollective Ownership - Knowledge is distributed across the team to avoid single points of failure.Pair Programming – Collaboration and code reviewUser Story - A User Story represents a feature of the system. Refactoring - Refactoring is the process of keeping the design clean incrementally.Simple Design - The design is kept as simple as possible. Only implement code for things that are needed.
See: “Empirical Software Engineering” by K Molokken and M Jorgensen. 2003 International Symposium on Environmental Standards for Electronics. Digital Object Identifier: 10.1109/ISESE.2003.1237981Most projects encounter effort and/or schedule overruns. Why? Formal estimation models: no evidence that these produce more accurate estimates (I.e. Cocomo: Basic COCOMO is a static, single-valued model that computes software development effort (and cost) as a function of program size expressed in estimated lines of code). Do customers buy lines of code? Add in complexity that isn’t required. Teams lack a solid and agreed upon definition of what it means to be done with a requirement.Agile is an empirical process model which recognizes that software by its nature cannot be completely known in advance. Why do requirements change? Because they can – software is tractable (changeable) Because markets change Software is often has a high degree of novelty – “never been done before” type stuff Because new knowledge is gained during the project Because customers don’t know what they want – even though they think they do Because customers aren’t good at describing what they want – but they know it when they see it Because needs changeThe numbers run between 60% to 80%, depending on the source (see http://www.stsc.hill.af.mil/crosstalk/1997/07/maintenance.asp for George Stark, of the MITRE Corporation). Writing the wrong software results in bloated code and feature bases, dramatically increasing TCO Even if you did a wonderful job delivering high quality software, there is going to be on-going “maintenance” to address changing and new needs. Software spends most of its life in maintenance therefore it needs to be initially developed with “changeable” as one of its primary characteristics. A piece of software that is difficult to change will not be used for very long – generally.Source: “Empirical Software Engineering” by K. Molokken and M. Jorgensen, 2003Source: Gartner, 2006Source: George Stark, MITRE Corporation,www.stsc.hill.af.mil/crosstalk/1997/07/maintenance.asp
The longer it takes to find a defect, the more it costs to fix it.Scott Ambler, at http://www.ambysoft.com/essays/whyAgileWorksFeedback.html then maps some popular development methodologies on this curve to show the effectiveness of many popular techniques.Agile techniques are on the good, left-hand side; the more traditional waterfall techniques occupy the bad, right-hand side.
Standish Group, 1994 Chaos Report: 222% schedule overrun in 1994 Standish Group, 2000 Chaos Report: 63% schedule overrun in 2000 Standish Group, 2005 Chaos Report: 84% schedule overrun in 2004 Another widely accepted number is 100%.See Capers Jones “Software Estimating Rules of Thumb” and “Applied Software Measurements.”Why? We can’t define all of the requirements up front We have no way to manage emerging requirements We have silos in our organizations (no cross functional teams with daily interaction and participation from the business) The waterfall fantasy: analyze a problem, design a solution, develop that solution, then test to validate. Do we find bugs in the solution? New or misunderstood requirements? Is it possible to get every detail right in one pass?Why do we spend effort and money on the wrong features? Subconsciously, we add add needless complexity. Unclear requirements; have to make assumptions about the requirements actually mean. Unclear definition of done.
See David Anderson, experience report at Agile Conference 2005, given in Denver, CO. (see http://www.agilemanagement.net/Articles/Papers/Agile_2005_Paper_DJA_v1_6.pdf) Work only on features customers want/need; eliminates 2/3 of work currently being done Add in 85% reduction in overhead Agile offers compared with heavyweight waterfall/CMMI process Deliver projects on time, on budget, and deliver higher quality code