Um Estudo Experimental sobre a Relação entre o Uso de Práticas Ágeis e o Sucesso de Projetos de Software que utilizam Scrum Leila M. R. de Souza Mariz, A. César C. França, Fabio Q. B. da Silva Abstract-This work has the goal of investigating the relationship between the use of agile practices and the success of software projects that use Scrum. The method used in the investigation was a cross-sectional survey that collected data from 62 software engineers associated to 11 projects in 9 different software firms, representing 75% of the population in the studied projects. The results show that only 8 of the 25 attributes associated with agile practices have significant correlation with project success. These results suggest that it is important to consider carefully which agile practices must be prioritized in order to increase the effectiveness of the deployment of agile methodologies in the software industry. Paper presented at the XXIV Simpósio Brasileiro de Engenharia de Software, Salvador, 2010. http://www.haseresearch.com

1. S
O
P
E
I
Influences
Is influenced
An Empirical Study on the Relationship between the Use of
Agile Practices and the Success of Software Projects that
Use Scrum
Leila Mariz; A. César C. França; Fabio Q. B. da Silva
< leilamariz@yahoo.com.br, cesarfranca@gmail.com,
fabio@cin.ufpe.br>

2. S
O
P
E
I
Influences
Is influenced
Research Project
2
(empirical) research project on the
influence of human factors on software
engineering
Long Term Objective: to understand
the influence of personality, behavior,
and cognition on individuals, teams,
projects, processes, and organizations
in software engineering.

3. S
O
P
E
I
Influences
Is influenced
Research Project
3
(empirical) research project on the
influence of human factors on software
engineering
Centro de Informática – UFPE
Since 2003 ...
1 professor/researcher
4 PhD students
18 master students

4. S
O
P
E
I
Influences
Is influenced
Research Project
4
(Book) Um Estudo sobre Motivação de
Integrantes de Equipes de Engenharia
de Software

5. S
O
P
E
I
Influences
Is influenced
Context
5
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority
T. Chow, e D. Cao, "A Survey
Study of Critical Success Factors
in Agile Software Projects.", The
Journal of Systems and Software,
n. 81, 2007, pp. 961–971

6. S
O
P
E
I
Influences
Is influenced
Context and Goals
6
Critical Success
Factors
Agile Attibutes
Delivery strategy A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and
expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management
style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement
management process
A14 Following agile-oriented project management
process
A15 Following agile-oriented configuration
management process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-
face meetings
A18 Honoring regular working Schedule
Team environment A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority
Is the adoption of the
25 agile attributes
related to the
success of software
development projects
managed using
Scrum?

7. S
O
P
E
I
Influences
Is influenced
Study Design
7
Method
Cross sectional survey

8. S
O
P
E
I
Influences
Is influenced
Study Design
8
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Method
Cross sectional survey

9. S
O
P
E
I
Influences
Is influenced
Study Design
9
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner

10. S
O
P
E
I
Influences
Is influenced10
Study Design
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner

11. S
O
P
E
I
Influences
Is influenced
Study Design
11
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception of use of agile attribute
Perception of project success
Hackman (1990) and Hallows (1998)

12. S
O
P
E
I
Influences
Is influenced
Study Design
12
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25: level of use of Agile Attribute
S: project success
Scale
Five point Likert type

13. S
O
P
E
I
Influences
Is influenced
Study Design
13
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design

14. S
O
P
E
I
Influences
Is influenced
Development
14
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009

15. S
O
P
E
I
Influences
Is influenced
Development
15
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009Context
9 firms
11 projects
Recife Scrum Group

16. S
O
P
E
I
Influences
Is influenced
Development
16
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009
Context
9 firms
11 projects
Recife Scrum Group
Participants (62)
48 Scrum Team
10 Scrum Master
4 Product Owner

17. S
O
P
E
I
Influences
Is influenced
Development
17
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009
Context
9 firms
11 projects
Recife Scrum Group
Participants (62)
48 Scrum Team
10 Scrum Master
4 Product Owner
Questionnaire
65 questionnaire sent
62 answered
95% response rate

18. S
O
P
E
I
Influences
Is influenced
Statistical Analysis
18
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009
Context
9 firms
11 projects
Recife Scrum Group
Participants (62)
48 Scrum Team
10 Scrum Master
4 Product Owner
Questionnaire
65 questionnaire sent
62 answered
95% response rate
Correlation
Spearman's rank
correlation coefficient
Development

19. S
O
P
E
I
Influences
Is influenced
Results: Correlation
19
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority

20. S
O
P
E
I
Influences
Is influenced
Results: PCA
20
C1
Management style
A17 Strong communication focus with daily face-to-face meetings
A10 Managers knowledgeable in agile
A18 Honoring regular working Schedule
A11 Managers who have adaptative management style
A20 Coherent, self-organizing team-work
C2
Software process
A23Good customer relationship
A13Following agile-oriented requirement management process
A15 Following agile-oriented configuration management process
A14Following agile-oriented project management process
A02Delivering most important features first
C3
Team Structure
A22Projects with no multiple independent teams
A21Projects with small team
C4
Technical Procedures
A03 Well-defined coding Standards up front
A07 Correct integration testing
A12 Appropriate technical training to team
A04 Pursuing simple design
C5
Team capability
A09Team members with great motivation
A08Team members with high competence and expertise
C6
Customer commitment
A24Strong customer commitment and presence
A06Right amount of documentation
C7
Delivery strategy
A01Regular delivery of software
C8
Team Location
A19Collocation of the whole team
A05 Rigorous refactoring activities
C9
Customer awareness
A25Customer having full authority
A16Good progress tracking mechanism

21. S
O
P
E
I
Influences
Is influenced
Results:Discussion
21
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority
Helps to predict risks
Contributes to product quality and
customer satisfaction

22. S
O
P
E
I
Influences
Is influenced
Results:Discussion
22
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority
Key activity

23. S
O
P
E
I
Influences
Is influenced
Results:Discussion
23
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority
Not a surprise. Happens with non Agile
methodologies.

24. S
O
P
E
I
Influences
Is influenced
Results:Discussion
24
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority
Supported by SCRUM

25. S
O
P
E
I
Influences
Is influenced
Results:Discussion
25
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority
Agrees with research on team work!
Not ease to achieve in SE!

26. S
O
P
E
I
Influences
Is influenced
Results:Discussion
26
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority
Apparently a hygienic factor

27. S
O
P
E
I
Influences
Is influenced
“Surprises”
27
Critical Success
Factors
Agile Attibutes
Delivery
strategy
A01 Regular delivery of software
A02 Delivering most important features first
Agile software
engineering
techniques
A03 Well-defined coding Standards up front
A04 Pursuing simple design
A05 Rigorous refactoring activities
A06 Right amount of documentation
A07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivation
A10 Managers knowledgeable in agile
A11 Managers who have adaptative management style
A12 Appropriate technical training to team
Project
management
process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management
process
A16 Good progress tracking mechanism
A17 Strong communication focus with daily face-to-face
meetings
A18 Honoring regular working Schedule
Team
environment
A19 Collocation of the whole team
A20 Coherent, self-organizing team-work
A21 Projects with small team
A22 Projects with no multiple independent teams
Customer
involvement
A23 Good customer relationship
A24 Strong customer commitment and presence
A25 Customer having full authority

28. S
O
P
E
I
Influences
Is influenced
“Surprises” (PCA)
28
C1
Management style
A17 Strong communication focus with daily face-to-face meetings
A10 Managers knowledgeable in agile
A18 Honoring regular working Schedule
A11 Managers who have adaptative management style
A20 Coherent, self-organizing team-work
C2
Software process
A23Good customer relationship
A13Following agile-oriented requirement management process
A15 Following agile-oriented configuration management process
A14Following agile-oriented project management process
A02Delivering most important features first
C3
Team Structure
A22Projects with no multiple independent teams
A21Projects with small team
C4
Technical Procedures
A03 Well-defined coding Standards up front
A07 Correct integration testing
A12 Appropriate technical training to team
A04 Pursuing simple design
C5
Team capability
A09Team members with great motivation
A08Team members with high competence and expertise
C6
Customer commitment
A24Strong customer commitment and presence
A06Right amount of documentation
C7
Delivery strategy
A01Regular delivery of software
C8
Team Location
A19Collocation of the whole team
A05 Rigorous refactoring activities
C9
Customer awareness
A25Customer having full authority
A16Good progress tracking mechanism

29. S
O
P
E
I
Influences
Is influenced
Main Contributions:
Academia
29
We need further research!
Available study package for
replicating the experience

30. S
O
P
E
I
Influences
Is influenced
Main Contributions:
Academia
30
Agile practices
Some of those practices
may have no influence on
project success
Although we need a in-depth
understanding on those influences
We need further
research!
Replicating the
expercience

31. S
O
P
E
I
Influences
Is influenced
Main Contributions:
Practitioners
31
Agile practices
Some of those
practices may have
no influence on
success
Decision-making
Organizations may
choose carefully what
agile practices to
prioritize
We need further
research!
Replicating the
expercience
Academia

32. S
O
P
E
I
Influences
Is influenced
Four Areas of Further Research
32
Self-organizing Teams
What are they in SE?
How to build one?
How to keep them cohesive?

33. S
O
P
E
I
Influences
Is influenced
Four Areas of Further Research
33
Motivation
How is motivation related to
performance?
How is motivation related to
success?
Self-organizing
Teams
What are they in SE?
How to build one?
How to keep them
cohesive?

34. S
O
P
E
I
Influences
Is influenced
Four Areas of Further Research
34
Motivation
How is motivation
related to
performance?
How is motivation
related to success?
Leadership Style
How the style of
leadership and
management affects
success?
Self-organizing
Teams
What are they in SE?
How to build one?
How to keep them
cohesive?

35. S
O
P
E
I
Influences
Is influenced
Four Areas of Further Research
35
Leadership Style
How the style of
leadership and
management affects
success?
Personality and
Behavior
Does personality matter?
How different team work
behavior affects team
work effectiveness?
Motivation
How is motivation
related to
performance?
How is motivation
related to success?
Self-organizing
Teams
What are they in SE?
How to build one?
How to keep them
cohesive?

36. S
O
P
E
I
Influences
Is influenced
Conclusions
• Limitations and Threats to Validity:
– Restricted context
– Small number of participants, projects, and firms
• Contributions:
– New research questions
– More detailed factor structure
• Future work
– To quantify the observed effects
– Describe the conditions where the effects occur
36

37. S
O
P
E
I
Influences
Is influenced
An Empirical Study on the Relationship between the Use of
Agile Practices and the Success of Software Projects that
Use Scrum
Leila Mariz; A. César C. França; Fabio Q. B. da Silva
< leilamariz@yahoo.com.br, cesarfranca@gmail.com,
fabio@cin.ufpe.br>

38. S
O
P
E
I
Influences
Is influenced
Um Estudo Experimental sobre a Relação entre o Uso de Práticas Ágeis e o Sucesso de
Projetos de Software que utilizam Scrum by Leila M. R. de Souza Mariz, A. César C. França,
Fabio Q. B. da Silva is licensed under a Creative Commons Attribution-NonCommercial-
NoDerivs 3.0 Unported License.
Based on a work at the XXIV Simpósio Brasileiro de Engenharia de Software, Salvador, 2010.
Permissions beyond the scope of this license may be available
at http://www.haseresearch.com