Keynote at the SPA Software in Practice, London, 26 June 2019.  Agile methods were aimed at small, co-located teams developing non-critical software products. The success of these methods for small teams have led to use in projects with tens of teams and hundreds of developers. Are agile methods suited in this new context? What fundamental assumptions in agile methods become challenging with scale? What can we learn from prior studies on key areas such as managing uncertainty, coordination, sharing knowledge, self management and tailoring of development method?

1. Agile Development in Large-Scale:
Challenges and Insight from Research
Keynote
SPA Software in Practice, London, 26 June 2019
Torgeir Dingsøyr
Chief Scientist, SINTEF Digital
Adjunct Professor, Department of Computer Science
Norwegian University of Science and Technology

2. Agenda
■ Why large-scale?
■ How do we do research large-scale agile development?
■ Insight on main challenges with scale:
■ Autonomy
■ Knowledge sharing
■ Coordination
■ Summary of insight

3. IKT
Home Ground of Agile Methods
«agile value set and practices best suit colocated
teams of about 50 people or fewer who have
easy access to user and business experts and
are developing projects that are not life-critical»
Williams and Cockburn, 2003
Williams, L. and Cockburn, A., "Agile Software Development: It’s about Feedback and Change," IEEE Computer, vol. 36, pp. 39-43, 2003.

4. Why Large-Scale Agile
Development?

5. IKTFlyvbjerg, B. and Budzier, A., "Why Your IT Project May Be Riskier Than You Think," Harvard Business Review, vol. 89, pp. 23-25, Sep 2011.

6. Arguments against Scaling Agile
The Folly of Scaling Agile
19 Jun 2014
I’m jotting down a few notes on Scaling Agile software development as Bucharest
Agile group invited me to talk about doing this. I have already warned them that I am
very skeptical about attempts to apply agile practices on large endeavours. While
preparing for our conversation, I thought it might be helpful for me to blog about the
reasons why I’m not a fan of Scaling Agile as this may make our conversation easier
to follow and help the group to come up with some questions.
When we apply Agile principles, we strip away process so that software developers
can work more collaboratively with business people to identify what is the most
valuable thing for them to deliver next. We focus on building working software and
releasing as early as we can to help us figure out what to build based on feedback
from users. Working this way is much harder when a lot of people are involved!
A bunch of things break down as you scale up. The biggest one is not being able to
maintain interpersonal relationships through which rich information flows, these are
replaced with weaker lossy forms of communication and misunderstandings about
what is the right thing to build next follow.
Typical things that become difficult at scale are access to business people and
infrastructure controlled by others outside immediate team. Meetings get long and
tedious, we start sending a representative from each team, which introduces more
secondhand information, emails and documentation.
When a project is big and is being changed by many hands it becomes much
harder to understand the whole, we start to introduce hierarchy with a select few
looking at the bigger picture and paying attention to separating concerns to allow
different teams to work in parallel. As a result, choice is removed from the team and
it can feel in teams that edicts come down from on high through a series of chutes
and screens that mask the reasoning behind them.
Often the initial attraction of Agile approaches to a business is to reduce delivery
timescales and enable developers to work faster with a lightweight approach.
Working in small teams allows individuals to feel more engaged because they have
Thoughts on Agile Coaching About Blog Events Talks Feed
The Folly of Scaling Agile http://rachelcdavies.github.io/2014/06/19/the-folly-of-scaling-agi...
1 av 3 20/02/2017, 13:44
«…not being able to maintain interpersonal
relationships through which rich information flows …»
«…meetings gets long and tedious, we start sending a
representative from each team, which introduces
more secondhand information, emails and
documentation.»
https://agilecoach.typepad.com/agile-coaching/2014/06/the-folly-of-scaling-agile.html

7. IKT

8. Programme Organisation
Dingsøyr, Torgeir, Moe, Nils Brede, Fægri, Tor Erlend, and Seim, Eva Amdahl, "Exploring Software Development at the Very Large-Scale: A Revelatory Case Study and Research Agenda for
Agile Method Adaptation," Empirical Software Engineering, 2018. http://rdcu.be/tcT3

9. Open Work Area
Dingsøyr, Torgeir, Moe, Nils Brede, Fægri, Tor Erlend, and Seim, Eva Amdahl, "Exploring Software Development at the Very Large-Scale: A Revelatory Case Study and Research Agenda for
Agile Method Adaptation," Empirical Software Engineering, 2018. http://rdcu.be/tcT3

10. Delivery Model
Dingsøyr, T., Dybå, T., Gjertsen, M., Jacobsen, A. O., Mathisen, T.-E., Nordfjord, J. O., Røe, K., and Strand, K., "Key Lessons from Tailoring Agile Methods for Large-Scale Software Development," IEEE IT
Professional, vol. 21, pp. 34-41, 2019.

11. Scaling Frameworks
0% 5% 10% 15% 20% 25% 30% 35%
Scaled agile framework (SAFe)
Scrum of Scrums
Internal method
Disciplined agile delivery (DaD)
Spotify model
Enterprise Scrum
Lean management
Agile portfolio management
Large-scale Scrum (LeSS)
Nexus
Recipe for agile governance ... (RAGE)
Kanban
Scrum at scale
Popularity of scaling frameworks from 13th State of Agile Survey 2019, VersionOne. https://www.stateofagile.com/
Conboy, K. and Carroll, N., "Implementing Large-Scale Agile Frameworks: Challenges and Recommendations," IEEE Software, vol. 36, pp. 44-50, 2019.

12. Project Success and Size
«Large-scale software development succeeds more often when using agile methods»
(Jørgensen 2019)
M A R CH /A PR I L 2019 | IE E E S O F T WA R E 41
the software proj-
ized as successful,
e, and 7% as failed.
medium-sized proj-
performances with
gorized as success-
as acceptable, and
d, respectively. The
ad 5% categorized
1% as acceptable,
d. The decrease in
nce with increased
sponds to findings
percent of the proj-
zed as agile. These
n in Table 2, had a
ccess rate than the
or all three size cat-
llustrates the effect
n for projects with
rmance. An analy-
l linear model with
elopment method
nagile) nested into
get size (i.e., small, FIGURE 1. The interaction plot of projects with acceptable performance.
0.7
0.6
0.5
0.4
0.3
0.2
ProportionAcceptable
Small Medium Large
Budget Category
Agile Nonagile
Development Method
Acceptable (n = 102) Agile 65% 58% 50%
Nonagile 19% 31% 25%
Failed (n = 13) Agile 2% 3% 14%
Nonagile 23% 6% 13%
*The percentages are the proportion of successful, acceptable, and failed projects for projects of the same budget size, category, and
development method.
Jørgensen, M., "Relationships Between Project Size, Agile Practices, and Successful Software Development: Results and Analysis," IEEE Software, vol. 36, pp. 39-43, 2019.

13. Main Challenges with Scale

14. Ineffective coordination due to «misaligned planning activities of specification, prioritization,
estimation and allocation between agile team and traditional inter-team levels»
(Bick et al. 2017)
«inter-group coordination becomes a major challenge when groups enjoy high levels of
autonomy»
(Ingvaldsen and Rolfsen 2012)
Challenges in Large-Scale
«Scrum-of-Scrum meetings involving representatives from all teams were severely
challenged: the audience was too wide to keep everybody interested ... often ending up not
reporting anything»
(Paasivaara et al. 2012)
Ingvaldsen, J. A. and Rolfsen, M., "Autonomous work groups and the challenge of inter-group coordination," Human Relations, vol. 65, pp. 861-881, Jul 2012.
Paasivaara, M., Lassenius, C., Heikkil, V. T., "Inter-team coordination in large-scale globally distributed scrum: do scrum-of-scrums really work?," Proceedings of the ACM-IEEE international symposium on
Empirical software engineering and measurement, Lund, Sweden, 2012.
Bick, S., Spohrer, K., Hoda, R., Scheerer, A., and Heinzl, A., "Coordination Challenges in Large-Scale Software Development: A Case Study of Planning Misalignment in Hybrid Settings," IEEE Transactions on
Software Engineering, 2017.

15. Challenges with Scale
■ Autonomy
■ Sharing knowledge in a large project / programme
■ Coordinating many development teams

16. How do we do Research on
Large-Scale Agile
Devlopement?

17. From Bent Hamer, the movie “Kitchen stories”, 2003.

18. Autonomy

19. Degrees of Autonomy
Slide design: Darja Smite. Model from Hackman, J. R. (1986). The psychology of self-management in organizations. Psychology and work: Productivity, change, and employment.
Autoritetsmatrise
Setting overall direction
Designing the team and its
organizational context
Monitoring and managing
work processes
Executing the task
Manager-led
teams
Self-managing
teams
Self-designing
teams
Self-governing
teams
Management responsibility
Team’s own responsibility

20. Barriers to Autonomy
Team-Level Barriers
if team-memb
the iteration p
to the iteration
much as possi
Another co
commitment w
tic plans. In e
tasks than the
became unrea
their plans to
one would fin
around the t
master (Comp
commented o
Shared resources
Organizational control
Specialist culture
Organizational-level
barriers
Individual commitment
Individual leadership
Failure to learn
Team-level
barriers
am- and
nal-level
self-
oftware
actual
e of a
ing team
t only on
ence of the
in managing
ng its
lso on
ational
vided by
Moe, N. B., Dingsøyr, T., and Dybå, T., "Overcoming Barriers to Self-Management in Software Teams," IEEE Software, vol. 26, pp. 20-26, 2009.

21. Insight from Studies
n “Organic” addition of roles (Tessem and Maurer, 2007)
n Technical architect
n Business responsible
n Test responsible
n Remove interdependencies across teams (Crowston et al. 2016)
n Ensure communication amongst teams to avoid silos (Elshamy, 2007)
Tessem, B. and Maurer, F., "Job satisfaction and motivation in a large agile team," in International Conference on Extreme Programming and Agile Processes in Software Engineering, 2007, pp. 54-61
Crowston, K., Chudoba, K., Watson-Manheim, M. B., and Rahmati, P., "Inter-team coordination in large-scale agile development: A test of organizational discontinuity theory," Proceedings of the Scientific Workshop
Proceedings of XP2016, Edinburgh, Scotland, UK, 2016.
Elshamy, A. and Elssamadisy, A., "Applying Agile to Large Projects: New Agile Software Development Practices for Large Projects," Berlin, Heidelberg, 2007, pp. 46-53.

22. Knowledge Sharing

23. Knowledge Integration and Development
Amrit Tiwana: An Empirical Study of the Effect of Knowledge Integration on Software Development Performance, Information and Software Technology, 2004.
“Higher levels of knowledge
integration are needed as the
complexity of a software project
(measured in person hours of
development effort) grows.”
(Tiwana 2005, page 905)

24. IKT
Retrospectives
«the single most important practice in agile development»
Agile practice guide
«number-one-most-important thing in Scrum»
Henrik Kniberg
Project Management Institute and Agile Alliance”, Agile Practice Guide: Project Management Institute, 2017.
Kniberg, H., Scrum and XP from the Trenches, 2nd edition ed.: InfoQ, 2015.

25. IKT
Retrospective Action Items after Topic
n 6 / 36 action items on project level
Action items after topic group
Process (13)
Other topics (7)
Tools (5)
People & relationships (5)
Project (4)
Other teams (2)
Product (0)
Action items after topic group
Process (13)
Other topics (7)
Tools (5)
People & relationships (5)
Project (4)
Other teams (2)
Product (0)
Dingsøyr, T., Mikalsen, M., Solem, A., and Vestues, K., "Learning in the Large - An Exploratory Study of Retrospectives in Large-Scale Agile Development," in XP2018, Porto, 2018, pp. 191-198.

26. IKT
Communities of Practice at Spotify
Basic Organizational Structures: Squads Chapters Tribes Guilds
U.S.
Offices
The U.K.
Office
Swedish
Offices
Teams at Spotify are
alled squads,
nd should feel
ke mini-start-ups,
A guild is a
group of people
with similar skills
and interests that
All squads are organized into
tribes containing 30–200
people each. Tribes have a
clear mission, a set of principles,
A chapter is a group of engineers
who has the same manager
(chapter lead) and is focused
on personal growth and skills
Smite, D., Moe, N. B., Levinta, G., and Floryan, M., "Spotify Guilds: How to Succeed With Knowledge Sharing in Large-Scale Agile Organizations," IEEE Software, vol. 36, pp. 51-57, 2019.

27. IKT
Knowledge Sharing in Large-Scale
n Establish retrospectives on inter-team level / raise
awareness of inter-team issues on team level retrospectives
n Establish Communities of Practice across teams
n Studies provide guidelines from companies such as
n Spotify (Smite et al. 2019)
n Ericsson (Paasivaara et al. 2014)
Smite, D., Moe, N. B., Levinta, G., and Floryan, M., "Spotify Guilds: How to Succeed With Knowledge Sharing in Large-Scale Agile Organizations," IEEE Software, vol. 36, pp. 51-57, 2019.
Paasivaara, M. and Lassenius, C., "Communities of practice in a large distributed agile software development organization - Case Ericsson," Information and Software Technology, vol. 56, pp. 1556-1577, Dec 2014.

28. Coordination

29. Importance of Coordination
«While there is no single cause of the software crisis, a major
contribution is the problem of coordinating activities while
developing large software systems. We argue that coordination
becomes much more difficult as project size and complexity
increases»
Kraut and Streeter, Communications of the ACM, 1995
Kraut, R. E. and Streeter, L. A., "Coordination in software development," Communications of the ACM, vol. 38, pp. 69-81, 1995.

30. The Scrum of Scrums
Kniberg, H., Scrum and XP from the Trenches: InfoQ, 2007, 2nd edition 2015.
Scrum of scrums

31. Coordination Modes
Plans
Routines
Personal Impersonal
Horisontal
Vertical
Individual
Scheduled meetings
Unscheduled meetings
Group
Van de Ven, A. H., Delbecq, A. L., & Koenig Jr, R. (1976). Determinants of coordination modes within organizations. American sociological review, 322-338.

32. Many Practices of Coordination
■ All three modes used
■ 19 coordination practices in total
«I think the combination of scheduled and unscheduled
coordination that just appeared was very important»
(scrum master and developer)
Dingsøyr, T., Moe, N. B., and Seim, E. A., "Coordinating Knowledge Work in Multi-Team Programs: Findings from a Large-Scale Agile Development Program," Project Management Journal, vol. 49, pp. 64-77, 2018.
Dingsøyr, T., Moe, N. B., Fægri, T. E., and Seim, E. A., "Exploring Software Development at the Very Large-Scale: A Revelatory Case Study and Research Agenda for Agile Method Adaptation," Empirical Software
Engineering, 2018.

33. Coordination Modes
Plans
Routines
Personal Impersonal
Horisontal
Vertical
Individual
Scheduled meetings
Unscheduled meetings
Group
Van de Ven, A. H., Delbecq, A. L., & Koenig Jr, R. (1976). Determinants of coordination modes within organizations. American sociological review, 322-338.

34. Summary of insight

35. Insight from Research
n Large-scale projects fail at higher rates than smaller projects
n Agile development seems to be best choice in large-scale
n Frameworks can give valuable insight, but also require much effort
n Autonomy must be balanced to enable efficient decision-making
n Make sure learning practices are also in place at inter-team level
n There need to be sufficient practices in place for coordination

36. Advice
n Avoid large-scale projects if you can
n Make efforts to mitigate increased risks
n Learn from rich descriptions of existing cases
n Beware of over-representation of success stories
n Remember context, tailor method to own needs
n Other factors also influence outcome

37. How can Practitioners Help Establish Knowledge?
n Share own lessons in experience reports
n Engage with academic community;
n Project work
n Master thesis
n Research project (be a “case” or engage in “action research”)
n Ask critical questions: Academics and to consultants

38. IKT
Follow the “Agile 2.0” Project
https://www.researchgate.net/project/Agile-20

39. IKT
https://agileresearchnetwork.org/
Leonor Barroca, Open University
Peggy Gregory, University of Central Lancashire
Helen Sharp, Open University
Katie Taylor, University of Central Lancashire

40. IKT
Special Issue in IEEE Software
Guest Editors’ Introduction: Agile Development at Scale: The Next Frontier
Torgeir Dingsøyr, Davide Falessi, and Ken Power (https://arxiv.org/abs/1901.00324)
Relationships Between Project Size, Agile Practices, and Successful Software Development: Results and Analysis
Magne Jørgensen
Implementing Large-Scale Agile Frameworks: Challenges and Recommendations
Kieran Conboy and Noel Carroll (https://arxiv.org/abs/1901.08130)
Spotify Guilds: How to Succeed With Knowledge Sharing in Large-Scale Agile Organizations
Darja Šmite, Nils Brede Moe, Georgiana Levinta and Marcin Floryan
Tailoring Product Ownership in Large-Scale Agile Projects: Managing Scale, Distance, and Governance
Julian M. Bass and Andy Haxby (https://arxiv.org/abs/1812.06524)
Empower Your Agile Organization: Community-Based Decision Making in Large-Scale Agile Development at Ericsson
Maria Paasivaara and Casper Lassenius
https://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=8648250&punumber=52