Presentation slides for the "Necessary and Neglected? An Empirical Study of Internal Documentation in Agile Software Development Teams" paper presented at SIGDOC 2011

1. Necessary
and
Neglected?
Empirical
Study
of
Internal
Documenta?on
in
Agile
SoAware
Development
Teams
SIGDOC
2011,
Pisa,
Italy
Christoph J. Stettina (stettina@liacs.nl)
Werner Heijstek (heijstek@liacs.nl)
This research has been kindly supported by the Leiden University Fund
Leiden
University.
The
university
to
discover.

2. Introduc?on
Agile
Knowledge
Transfer
• Adap&ve
rather
then
predic&ve:
No
heavy
documenta&on
created
up-­‐front
• Direct
communica&on
rather
than
documenta&on
• Lean:
Documenta&on
→
“No
value
for
the
end
user”
?
All
fine,
but:
l Project
handover
and
maintenance
l Loss
of
undocumented
knowledge
(Abrahamsson et al, 2003)
l LiDle
empirical
data
(Dybå and Dingsøyr, 2008)
Leiden
University.
The
university
to
discover.

3. Objec?ves
Percep?ons
from
within
teams:
How
do
prac&&oners
feel
about
documenta&on?
Research
Ques?ons
1. How
do
team
members
in
agile
soIware
development
projects
document
their
work?
2. How
do
they
perceive
the
amount
and
importance
of
their
internal
documenta&on?
Leiden
University.
The
university
to
discover.

4. Methodology
Qualita?ve
ScrumMaster
interviews:
Project
and
team
environment
Quan?ta?ve
ques?onnaire:
Comparable
Likert
scale
data
on:
A) Documenta&on:
Perceived
amount,
effort,
importance
B) SoIware
Tools:
Usage,
usability
and
importance
(issue
tracking,
revision
control,
electronic
discussion,
Scrum
support,
document
management
and
calendar
&
scheduling)
Leiden
University.
The
university
to
discover.

5. Methodology:
Ques?onnaire
Design
Percep?ons
Regarding
Documenta?on
-­‐
Perceived
amount,
effort
and
importance
1. How
much
+me
do
you
spend
on
wri+ng
documenta+on
daily?
2. How
do
you
feel
about
documenta+on
at
work?
3. How
effec+ve
do
you
consider
finding
internal
documenta+on?
4. How
important
do
you
consider
documenta+on
for
your
project?
5. How
important
do
you
consider
physical
ar+facts
like
story
cards
or
“the
wall”
for
your
project?
Leiden
University.
The
university
to
discover.

6. Methodology:
Data
Collec?on
Par?cipant
and
team
iden?fica?on:
l SNS,
Google
Groups,
SlideShare,
Flickr,
etc.
l Ac+vely
involved
in
Scrum
at
collec+on
+me
Leiden
University.
The
university
to
discover.

7. Data:
79
individuals,
13
countries,
8
teams
Experience
Country
(in years)
Roles
Leiden
University.
The
university
to
discover.

8. Data:
Team
Sample
T1
(UK)
MMO
Game
back-­‐end
T2
(US)
Collabora&ve
SW
for
construc&on
T3
(UK)
Digital
media
agency
T4
(NO)
Smart
Card
key
solu&ons
T5
(NL)
Corporate
sites
and
web
shops
T6
(SE)
News
guide,
community
website
T7
(IN)
E-­‐commerce
T8
(NZ)
State
insurance
company
→
Broad
mul&na&onal
sample
Leiden
University.
The
university
to
discover.

9. Results:
Too
li^le
documenta?on?
"Code
comments
are
used
in
an
effec+ve
manner.
During
project
development
any
needed
documenta+on
is
generally
available.
However,
finding
documenta+on
for
older
projects
is
not
always
easy,
and
some+mes
this
documenta+on
is
missing.”
-­‐-­‐
Team
member
T6
Leiden
University.
The
university
to
discover.

10. Results:
Too
li^le
documenta?on?
5 35
5 35
Effort:
Majority
15
mins
or
less
daily
Effec?veness:
Normal
distribu?on
How important do you consider documentation for your project?
Amount:
Distribu?on
towards
too
li^le
Importance:
Important
or
very
important
Leiden
University.
The
university
to
discover.

11. Results:
Tools
as
a
“Backchannel”?
Adapta+on
of
support
tools
”[W]e
have
a
wiki
that
we
are
supposed
to
use”
-­‐-­‐
ScrumMaster
T6
Virtual
Teams
&
GSD
"We
have
good
experience
using
physical
ar+facts
for
local
projects,
but
most
of
our
projects
are
mul7
loca7on
and
require
an
electronic
solu7on.”
-­‐-­‐
ScrumMaster
T4
Leiden
University.
The
university
to
discover.

12. 13-)9 $ *3&,' $ &;;*-3< $ 4-'6-) $ ,0:'4&73 $ 3)9-)337-)9 $ '3&C," $ %0=>C3)' $ M&)&93C3)' $ ,=073, $ '63 $ *&,'$
;0,-'-0)$136-)<$N=7>C$N>;;07'$&)<$,0*>'-0),$:07$O*3='70)-=$%-,=>,,-0)"$563$=&'3907+$3)=*0,3,$'63$
Results:
Tools
as
a
“Backchannel”?
,'07&93 $ 0: $ 73L>-73C3)',H $ <3,-9) $ ;&;37,H $ >,3< $ -)'37:&=3,H $ =73&'3< $ :>)='-0), $ &)< $ ,>1.:>)='-0), $ -)$
739>*&7$,0:'4&73$;70P3=',"
60H2I('$%J"(-$ 60H2I('$%J"(L+3+2M
!""#$%&'()*+,- BC?
83% !""#$%&'()*+,- D>B @>EE
.$/+"+0,%10,2'03 BD? .$/+"+0,%10,2'03 @>D F>CB
43$)2'0,+)%5+")#""+0, DE? 43$)2'0,+)%5+")#""+0, @>CG @>BG !780'2(,)$
N(8
J"(L+3+2M
6)'#7%6#880'2 DB? 6)'#7%6#880'2 D>KE @>BA
50)#7$,2%9(,(-$7$,2 CE? 50)#7$,2%9(,(-$7$,2 D>@D @>AB
37%
1(3$,:('%;%6)<$:#3+,- D>KD @>DC
1(3$,:('%;%6)<$:#3+,- FG?
= G D F B A= AG AD
=>==? @=>==? A==>==?
Document
Management:
$ $
Least
usage
and
believed
usability
!""#$%&'%()*+,-.+/)01'&($)*+)2+#$34+$)2%5'&3+5(%6+(%$+13&73(834+#$'9("(%:+'*4+93"(3834+(01)&%'*73
Scrum
Support:
Wide
applica?on
of
soAware
tools
to
support
the
process
Q)3$3D;*&)&'-0)$'0$'63$*04$;37=3-@3<$-C;07'&)=3$0:$<0=>C3)'$C&)&93C3)'$=0>*<$13$'63$3C;6&,-,$
0:$&9-*3$C3'60<0*09-3,$EN=64&137H$#RRST$N=64&137H$UJJSG$0)$<-73='$=0CC>)-=&'-0)$,3''-)9$47-''3)$
<0=>C3)'&'-0) $ -)'0 $ '63 $ 1&=8970>)< $ &)< $ C&8-)9
$
-'
$
73<>)<&)'H $ &) $ -<3& $ ,>;;07'3< $to
d$ '63 $(9-*3$
Leiden
University.
The
university
1+ iscover.

13. Discussion
Documenta?on
l Confirm
lack
of
undocumented
knowledge
l Majority
spends
<
15
mins
on
wri&ng
documenta&on
daily
l Believe
documenta&on
is
important
and
too
liDle
SoAware
l Pure
availability
of
support
tools
not
enough
l Global
SoIware
Development,
Virtual
Teams
l Traceability
of
decisions
valued
Leiden
University.
The
university
to
discover.

14. Conclusion
?
What
is
an
appropriate
balance
of
explicit
and
tacit
knowledge
in
agile
soXware
development
projects?
Leiden
University.
The
university
to
discover.

15. Conclusion
→
Future
Work
Expecta+ons
&
Sa+sfac+on
in
agile
seZngs
→
Who
needs
what
documenta&on?
Process
Alignment
and
Cost/Quality
balance
→
Effects
of
documen&ng
itera&vely
SoXware
support
and
codifica+on
of
informa+on
→
How
to
code
informa&on
in
wikis
and
issue
trackers?
Visual
methods
→
Collabora&ve,
Agile
Modeling,
ICONIX,
ADSD
Leiden
University.
The
university
to
discover.

16. Ques?ons?
Thank
you
for
your
aDen&on!
stegna@liacs.nl
Leiden
University.
The
university
to
discover.

17. Validity
Validity
Considera?ons
l Consistency
of
data
→
Likert
scales
l Low
amount
of
data
→
Team
agreement
l Socially
Desirable
Responding
→
Anonymity
Leiden
University.
The
university
to
discover.

18. Results:
Team
Sample
Table 2: Descriptive variables, team results (x) and agreement (σ 2 )
T1 T2 T3 T4 T5 T6 T7 T8 avg.
agr.
country UK US UK NO NL SE IN NZ
team size ( pers.) 4 9 5 12 6 4 8 6
collected answers 4 6 5 6 5 3 8 4
avg. exp. ( yrs.) 7.75 13.7 6.6 12.7 2.6 10 7 3.5
spacial distribution co-loc. co-loc. co-loc. distrib. co-loc. co-loc. distrib. co-loc.
documentation tool Wiki Con- Google - - Wiki Con- -
fluence Docs fluence
Wiki Wiki
perceived doc. x -0.25 -0.50 -0.40 -1.30 -1.00 -0.75 -0.13 0
amount σ2 (0.19) (0.25) (1.44) (0.89) (0.40) (0.67) (0.61) (0) (.56)
perceived eff.. x 0.65 0.76 0.44 0.60 0.52 0.50 0.75 0.45
finding doc σ2 (0.69) (0.47) (0.16) (1.33) (1.44) (0.89) (0.69) (0.69) (.80)
perceived x 1.00 0.70 0.76 0.57 0.72 0.75 0.7 0.85
importance artif. σ2 (0) (2.25) (0.16) (0.47) (1.04) (0.67) (0.50) (0.69) (.72)
average
agreement σ2 (.29) (.99) (.59) (.90) (.96) (.74) (.60) (.46) (.69)
6.2 Software, More Than a Backchannel documentation, suggesting that documentation should only
The teams in our study predominantly adopt collabora- include information that is used, we found that agile soft-
tion tools to document and share agile artifacts such as user ware development practitioners perceive their internal doc-
stories or sprint backlogs. An interesting finding is the per- umentation as important but that they feel that too little
ceived importance and application of software that directly
documentation is available.TAnalogously to to
dobservations
Leiden
University.
he
university
the iscover.

19. References
Abrahamsson,
P.
Warsta,
J.,
Siponen,
M.
T.
and
Ronkainen,
J.
(2003)
New
direc&ons
on
agile
methods:
a
compara&ve
analysis.
In
Proceedings
of
the
25th
Interna&onal
Conference
on
SoIware
Engineering,
ICSE
2003,
pages
244–254,
Washington,
DC,
USA,
2003.
IEEE
Computer
Society.
Clear,
T.
(2003)
Documenta&on
and
agile
methods:
striking
a
balance.
SIGCSE
Bulle&n,
35(2):12–13
Dyba,
T.,
Moe,
N.
B.
(1999)
Rethinking
the
concept
of
soIware
process
assessment.
In
Proceedings
of
European
SoIware
Process
Improvement
Conference
(EuroSPI
1999),
Pori,
Finland
Dyba,
T.
and
Dingsøyr,
T.
(2008)
Empirical
studies
of
agile
soIware
development:
A
systema&c
review.
Informa&on
SoIware
Technology,
50(9-­‐10):833–859,
2008.
Fægri,
T.E.,
Dyb˚a,
T.,
Dingsøyr,
T.:
Introducing
knowledge
redundancy
prac&ce
in
soIware
development:
Experiences
with
job
rota&on
in
support
work.
Inf.
SoIw.
Technol.
52,
1118–1132
(2010)
Rubin,
E.
and
Rubin,
H.
(2011)
Suppor&ng
agile
soIware
development
through
ac&ve
documenta&on.
Requirements
Engineering,
16:117–132
Leiden
University.
The
university
to
discover.