Presented at RSSE'14 at ICSE'14 in Hyderabad, India. Authors: Ahmed Shah Mashiyat, Michalis Famelis, Rick Salay, Marsha Chechik Absract: Software development is a social process: tasks such as implementing a requirement or fixing a bug typically spark conversations between the stakeholders of a software project, where they identify points of uncertainty in the solution space and explore proposals to resolve them. Due to the fluid nature of these interactions, it is hard for project managers to maintain an overall understanding of the state of the discussion and to know when and how to intervene. We propose an approach for extracting the uncertainty information from developer conversations in order to provide managers with analytics. Using these allows us to recommend specific actions that managers can take to better facilitate the resolution of uncertainty.

1. Ahmed Shah Mashiyat, Michalis Famelis,
Rick Salay, Marsha Chechik
University of Toronto
RSSE@ICSE 2014
Using Developer Conversations
to Resolve Uncertainty in
Software Development:
A Position Paper

2. Software Development is Social
2
Conversation between developers on
jazz.net about an enhancement ticket

3. Software Development is Social
3
Conversation between developers on
jazz.net about an enhancement ticket
Ideal conversation: Question, Debate, Solution

4. The Manager’s Perspective
Goals:
• Product quality
• Speedy delivery
So, conversations should:
• cover all issues
• include all stakeholders
• not take longer than
necessary
• But that is not always the
case…
4
Manager

5. Software Development is Social
5
(The conversation has been abstracted and
simplified for presentation purposes. The
listing to the left is not intended to be legible.

6. Software Development is Social
6
“Ali” is leading the
development of a
component.
(The conversation has been abstracted and
simplified for presentation purposes. The
listing to the left is not intended to be legible.

7. Software Development is Social
7
He seeks input from
stakeholders.
` Color?
Orientation?

8. Software Development is Social
8
Stakeholders start
discussing.
?
?
“Christine”

9. Software Development is Social
9
Stakeholders start
discussing.
?
?
“Bao”

10. Software Development is Social
10
Stakeholders start
discussing.
?
?

11. Software Development is Social
11
Ali thinks he has found
the answers he needs
=
=
Bao agrees

12. Software Development is Social
12
Ali marks the issue as
resolved
=
=

13. Software Development is Social
13
=
=
But “Dimitra” thinks that
the issue is not settled
Size??

14. Software Development is Social
14
=
=
But nobody takes her
objection into account
Size??
Guys?

15. Conversations are Messy
• The decision about
orientation was not debated
• The question about size
was left unanswered
• Stakeholder Dimitra was not
consulted until late
• Changing the status to
“resolved” was premature
15
Manager

16. Proposed Solution
16
Mine conversations
Gather Analytics
Generate Action
Recommendations

17. Proposed Solution
17
Mine conversations
Gather Analytics
Generate Action
Recommendations

18. Generating Recommendations
Use metrics to support managers in facilitating conversations:
18
Intervene when questions are left
unanswered for a “long” period of time
Alert when questions are prematurely
marked as resolved
Report when a question has been
discussed enough to be considered mature
If a question is re-opened, create a context
reminder of previously examined alternatives

19. Proposed Solution
19
Mine conversations
Gather Analytics
Generate Action
Recommendations

20. • Online vs Offline (post-hoc)
• Main challenge: determining resolution of a point of
uncertainty:
• Trusting status changes not necessarily a good idea
• Who should be part of discussion?
• People’s stance may not be obvious
• E.g. does elaboration imply endorsement?
• What does absence of comment (a person’s “default”) mean?
• For now:
Compute metrics from the structure of the conversation
20
Gathering Analytics about Uncertainty

21. Dashboard Mock-ups
21

22. Proposed Solution
22
Mine conversations
Gather Analytics
Generate Action
Recommendations

23. Mining the Conversations
23
`
• Identify unique Points of Uncertainty
from question utterances

24. Mining the Conversations
24
• Identify unique Points of Uncertainty
from question utterances
• Associate them with Proposed
Alternatives

25. Mining the Conversations
25
• Identify unique Points of Uncertainty
from question utterances
• Associate them with Proposed
Alternatives
• Track the stakeholders’ arguments

26. Mining the Conversations
• Identify unique Points of Uncertainty
from question utterances
• Associate them with Proposed
Alternatives
• Track the stakeholders’ arguments
• Recognize the Resolution of
Uncertainty
26

27. Construct a Model of the Conversation
27
• Twente Argumentation Schema (TAS)
• Time: left-to-right, top-to-bottom

28. Constructing the TAS Model
28
Automation Challenges
• Identifying unique points of
uncertainty among many
utterances of questions
• Identifying proposals when they
are not stated explicitly
• e.g. cultural differences in
assertiveness
• Mapping utterances of proposals
to points of uncertainty
Benefits
• Associates points of
uncertainty with proposed
answers
• Easy to compute structural
metrics
• Can be used to trace
uncertainty to software
artifacts

29. Summary
29
quality
speed

30. Future Work
• How to best automate the creation of the TAS model?
• How can we safely determine that a point of uncertainty
has been resolved?
• What are the most useful metrics, analytics and action
recommendation for managers?
• How to connect uncertainty in conversations with relevant
organizational, social and software models?
30

31. Questions?
icons by:
31