Communication between software developers: challenges of sentiment analysis

Communication
between
software
developers
Unknown (17 century). Conversation of three men. Church of Our Lady, Poperinge.

Unknown (19 century). Marionette “van Duivel Blekkas”.Ethnographic museum, Antwerp .
Why should
we care?

More
“central”
developers
commit
more!
Christian Bird, Alex Gourley, Prem Devanbu, Michael
Gertz, Anand Swaminathan, Mining Email Social
Networks, MSR 2016

Fabio Palomba, Damian Andrew Tamburri, Alexander Serebrenik, Andy Zaidman, Francesca Arcelli Fontana, Rocco Oliveto,
How Do Community Smells Inﬂuence Code Smells? ICSE (Companion Volume) 2018: 240-241
“Community smells”, problems
in team communication
“Code smells”, problems in
source code

Jacques Hersleven. Postier télégraphiste de l'Hôtel des postes et
télégraphes de Bruxelles, ca. 1930, Reportage. Bruxelles.
Sulpice Guillaume Gavarni (1804 - 1866). Man
reading a letter in front of his bed. Museum
Mayer van den Bergh. Antwerpen.

Bogdan Vasilescu, Alexander Serebrenik, Premkumar T. Devanbu, Vladimir Filkov: How social Q&A sites are changing
knowledge sharing in open source software communities. CSCW 2014: 342-354

reputation ratings are a nice little initiative
r-help used to be very helpful. But as the
number of posts has gone up, I find that
reading it is not as useful as it had been
peer recognition/gamification
within StackOverflow
Google is getting better at finding answers
related to R so I use it more. I rely less on
going directly to mailing lists now
Bogdan Vasilescu, Alexander Serebrenik, Premkumar T. Devanbu, Vladimir Filkov: How social Q&A sites are changing
knowledge sharing in open source software communities. CSCW 2014: 342-354

What is a team?
Survey Archive analysis
Archive analysis Survey
Sequential exploratory strategy
Sequential explanatory strategy

Bin Lin, Alexey Zagalsky, Margaret-Anne D. Storey, Alexander Serebrenik: Why Developers Are Slacking Off: Understanding
How Software Teams Use Slack. CSCW Companion 2016: 333-336
Community-wide
13%
Personal
14%
Team-wide
73%
Discovery and
news/information
aggregation
Networking and
social activities
Fun
Communication
Collaboration
Customer support
DevOps
Communities of practice

Kids like
new
toys?
Actualit. Boudewijn & Josephine-Charlotte with a toy train. 1938. Koninklijke schenking. Koningin Elisabeth, Brussels.

• Download 1999-February.txt from the folder
• This is the mail archive of GNOME developer
mailing list
• Using any technique you like answer the following
questions
• How many people are involved in the
conversation?
• How many European countries are represented?

Frans Floris de Vriendt (1519-1570). Three heads. The Royal Museum of Fine Arts, Brussels, Belgium.

What do developers try
to achieve when they ask
a question?

Albert Daenens (1925) L'homme machine. De automaat-mensch. Der Maschinenmensch. The machine man. Collection Buelinckx
[© KIK-IRPA, Brussels (Belgium), cliché X038603]

Exploratory
case study
https://www.arcgames.com/en/games/star-trek-online/news/detail/9582273-star-trek-online%3A-galaxy-dreadnought-remodel
Early
work!

Maybe introduce an
additional line between
‘abc’ and ‘def’?
Soliciting an action
Can you make these
diﬀerent?
Suggestion
Request
for action

Information seeking
Information When can this be null?
Shouldn’t this just be
a failure?
Conﬁrmation
Opinion Which name do you
suggest?
Clariﬁcation What’s happening
here?
Why is this included?Rationale

Do you really want to
return the address of a
local variable here?
Attitudes and emotions
Criticism
Anger
Surprise
wtf? you really want
reﬂection here
Is this true? that
seems mildly
surprising

Hypothetical scenarios
Rhetorical questions
What about if an
already Jack server is
running?
Isn’t the case that you
illustrated (0.9ms
being decremented as
0) applicable in both
solutions? Yes

soliciting an
action
suggestion, request
for action 40%
information
seeking
clariﬁcation,
information,
conﬁrmation,
rationale, opinion
49%
attitudes and
emotions
criticism, anger,
surprise 8%
hypothetical
scenario’s 3%
rhetorical
questions 1%

Hypotheses:
Questions are frequently used to trigger
an action of the interlocutor
While the largest group of questions
aim at satisfying information needs, this
group constitutes < 50% questions
Questions express
cognitive/emotional state to induce
critical reﬂection in the interlocutor

James A. Russell. A circumplex model of affect. Journal of personality and social psychology, 39(6):1161–1178, 1980.
Paul E. Ekman, & R.J. Davidson. The nature of emotion: Fundamental questions. 1994 Oxford University Press.
valence/arousal discrete

Information Retrieval Human-Computer Interaction
polarity lexicons

• E. Guzman, B. Bruegge, Towards emotional awareness in
software development teams, FSE 2013
• D. Garcia, M. S. Zanetti, F. Schweitzer, The role of emotions in
contributors activity: A case study on the Gentoo community, in
ICCGC, 2013
• E. Guzman, D. Azócar,, Y. Li, Sentiment analysis of commit
comments in GitHub: An empirical study, MSR 2014
• A.-I. Rousinopoulos, G. Robles, and J. M. González-Barahona,
Sentiment analysis of Free/Open Source developers:
preliminary ﬁndings from a case study, Revista Eletrônica de
Sistemas de Informação, 2014
• D. Pletea, B. Vasilescu, A. Serebrenik, Security and emotion:
Sentiment analysis of security discussions on GitHub, MSR
2014
• M. M. Rahman, C.K. Roy, I. Keivanloo: Recommending
insightful comments for source code using crowdsourced
knowledge. SCAM 2015: 81-90
• M. Ortu, B. Adams, G. Destefanis, P. Tourani, M. Marchesi, R.
Tonelli: Are Bullies More Productive? Empirical Study of
Affectiveness vs. Issue Fixing Time. MSR 2015: 303-313
• F. Calefato, F. Lanubile, M. C. Marasciulo, N. Novielli: Mining
Successful Answers in Stack Overﬂow. MSR 2015: 430-433
• …

What is the best
way to kill a critical
process?
I’m missing a
parenthesis. But
where?
Nicole Novielli, Fabio Calefato, Filippo Lanubile: The challenges of sentiment detection in the social programmer ecosystem.
SSE@SIGSOFT FSE 2015: 33-40
Robbert Jongeling, Proshanta Sarkar, Subhajit Datta, Alexander Serebrenik. On Negative Results when using Sentiment
Analysis Tools for Software Engineering Research. Accepted subject to minor revision at Empirical Software Engineering
What is going on?

What is the best way to kill a critical process?
pos. neg. neg.

What is the best
process?
I’m missing a
parenthesis. But
where?
Tools tend to disagree (κ ≤ 0.25)

What is the best
process?
I’m missing a
parenthesis. But
where?
Tools disagree and this affects
previously published results

Tools are trained on movie/product reviews
and
misidentify (or fail to identify) a sentiment in a
software engineering text

Robbert Jongeling,
Proshanta Sarkar, Subhajit
Datta, Alexander
Serebrenik. On Negative
Results when using
Sentiment Analysis Tools
for Software Engineering
Research. Accepted
subject to minor revision
at Empirical Software
Engineering

Robbert Jongeling,
Proshanta Sarkar, Subhajit
Datta, Alexander
Serebrenik. On Negative
Results when using
Sentiment Analysis Tools
for Software Engineering
Research. Empirical
Software Engineering

Any help is hugely
appreciated!
Wow, great!
Thanks.
Is this really positive?

Any help is hugely
appreciated!
Wow, great!
Thanks.
Is this really positive?
Including politeness makes
things worse

David Teniers II (1601-1700), The monkey-painter. Coppée collection. Brussels

D. Pletea, B. Vasilescu, A.
Serebrenik, Security and
emotion: Sentiment analysis of
security discussions on
GitHub, MSR 2014
E. Guzman, D. Azócar,, Y. Li,
Sentiment analysis of commit
comments in GitHub: An
empirical study, MSR 2014

D. Pletea, B. Vasilescu, A.
Serebrenik, Security and
emotion: Sentiment analysis of
security discussions on
GitHub, MSR 2014
E. Guzman, D. Azócar,, Y. Li,
Sentiment analysis of commit
comments in GitHub: An
empirical study, MSR 2014
Analysis Tools for Software Engineering Research. Empirical Software Engineering
Guzman et al. Java projects are more negative than projects in other languages ✖
Guzman et al. There is no difference between other programming languages ✖
Guzman et al. Comments on Mondays are more negative than on other days ✖
Guzman et al. Afternoon comments are more positive than the evening ones ✖
Pletea et al.
Security comments/discussions are more often negative than
non-security comments/discussions
✔
Pletea et al. Security discussions are more emotional than non-security ones ✖

ASE 2017
ESEM 2018
JSS 2018
+ shared datasets, benchmarking studies, replications…

DEVA EmoTxt
Md. Rakibul Islam, Minhaz F. Zibran. DEVA: sensing emotions in the valence arousal space in software engineering text.
SAC '18 Proceedings of the 33rd Annual ACM Symposium on Applied Computing, 1536-1543
Fabio Calefato, Filippo Lanubile, Nicole Novielli. EmoTxt: A Toolkit for Emotion Recognition from Text. International
Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACII Workshops 2017, San Antonio,
USA, Oct. 23-26, 2017, pp. 79-80,

Frequent Important
Daviti Gachechiladze, Filippo Lanubile, Nicole Novielli,
Alexander Serebrenik Anger and Its Direction in
Collaborative Software Development ICSE NIER 2017
Etching Bernard Picart after
Charles Le Brun (1713)

I don’t have to ensure that
the classloader knows groovy
classes, *you* must do that.
I am an idiot - this was a
dupe of GUVNOR-84
Damn maven!

dupe of GUVNOR-84
Damn maven!
other
object
self
Andrew Ortony, Gerald L. Clore, and Allan Collins. The cognitive structure of emotions. Cambridge University Press, 1990.

dupe of GUVNOR-84
Damn maven!
other
object
self

Acta. Postier - postmannen in het sorteerlokaal ca. 1930

Daviti Gachechiladze, Filippo Lanubile, Nicole Novielli, Alexander Serebrenik. Anger and Its Direction in Collaborative
Software Development 39th International Conference on Software Engineering, New Ideas and Emerging Results 2017

Dutch Transorma machine for post sorting. Photograph unknown.

Classiﬁed as >>> Other Object Self
Other 12 53 2
Object 3 515 8
Self 0 52 78

(1) theory
(2) manual
creation of the
golden set
(3) automation:
train a classiﬁer

(1) theory
(2) manual
creation of the
golden set
(3) automation:
train a classiﬁer
Exercise: Label
the data!

Primary Secondary Tertiary
emotions emotions emotions
love
A↵ection Compassion, Sentimentality, Liking, Caring, . . .
Lust/Sexual desire Desire, Passion, Infatuation
Longing
Joy
Cheerfulness Amusement, Enjoyment, Happiness, Satisfaction, . . .
Zest Enthusiasm, Zeal, Excitement, Thrill,Exhilaration
Contentment Pleasure
Optimism Eagerness, Hope
Pride Triumph
Enthrallment Enthrallment, Rapture
Surprise Surprise Amazement, Astonishment
Anger
Irritability Aggravation, Agitation, Annoyance, Grumpy, . . .
Exasperation Frustration
Rage Outrage, Fury, Hostility, Bitter, Hatred, Dislike, . . .
Disgust Revulsion, Contempt, Loathing
Envy Jealousy
Torment Torment
Sadness
Su↵ering Agony, Anguish, Hurt
Sadness Depression, Despair, Unhappy, Grief, Melancholy, . . .
Disappointment Dismay, Displeasure
Shame Guilt, Regret, Remorse
Neglect Embarrassment, Humiliation, Insecurity, Insult, . . .
Sympathy Pity, Sympathy
Fear
Horror Alarm, Shock, Fright, Horror, Panic, Hysteria, . . .
Nervousness Suspense, Uneasiness, Worry, Distress, Dread, . . .

love
Longing
Joy
Pride Triumph
Anger
Envy Jealousy
Torment Torment
Sadness
Fear
through computer-mediated communication systems [18, 21],
development artifacts like mailing lists or the discussion board
of an issue tracking system could be a promising source for
mining developer emotions during software evolution.

Love Joy Surprise Anger Sadness Fear Polarity
x Negative
Negative emotion and negative
polarity
x Positive
Positive emotion and positive
polarity
x Negative
Surprise is intrinsically
ambiguous, all polarity values are
allowed
x Positive
x Neutral
x x Positive Multiple emotion label
x x Mixed
Opposite polarity emotion, mixed
polarity
Neutral Absence of emotion
Love Joy Surprise Anger Sadness Fear Polarity
Negative No emotion and negative spolarity
Positive No emotion and positive polarity
Mixed No emotion and mixed polarity
x Neutral Emotion label different from
surprise and neutral polarityx Neutral
GOOD
BAD

• Please download annotation_examples from the
Google folder
• We will annotate the statements individually and
then discuss the results

love
Longing
Joy
Pride Triumph
Anger
Envy Jealousy
Torment Torment
Sadness
Fear
through computer-mediated communication systems [18, 21],Try to avoid mixed emotions as much as possible

Text Love Joy Surprise Anger Sadness Fear Polarity
Thank you very much! I
appreciate your efforts.
Oops. It needs to be added to
Makefile
WTF, a package refactoring and
class renaming in a patch?
Wish I had pay more attention in
my english class… now its pay
back time…
I'm most concerned with some of
the timeouts
This bug is 3 years old. Anybody
looking to fix it?
Andreas; I can see the benefit;
but I'm not entirely sure how to
do it. Perhaps you could provide
a patch? David
This is now fixed!
Oppss... sorry late reply.... Thx
for handling this issue Claus. I
prefered it to be the way it is now
Thanks Tom. Sorry for the mess.

M. E. Jordan, D. L. Schallert, Y. Park, S. Lee, Y. hui Vanessa Chiang, A.-C. J. Cheng, K. Song, H.-N. R. Chu, T. Kim, and H.
Lee, "Expressing uncertainty in computer-mediated discourse: Language as a marker of intellectual work," Discourse
Processes, vol. 49, no. 8, pp. 660–692, 2012.

Felipe Ebert, Fernando Castor, Nicole Novielli, Alexander Serebrenik: Confusion Detection in Code Reviews. ICSME 2017: 549-553

Communication
between software
developers
• Who is talking to
whom?
• What do they want to
achieve?
• What sentiment/
emotion is reﬂected in
their communication?
Unknown (17 century). Conversation of three men. Church of Our Lady, Poperinge.

Jan van Scorel (?). Portret van een twaalfjarige scholier. Museum Boĳmans Van Beuningen. Rotterdam, Nederland. © KIK-IRPA, Brussels (Belgium), cliché B145759
Homework

• Please download commit_comments from the
Google folder
• Please annotate 300 statements individually
• I will send you individual mails with the IDs of
the statements you need to annotate
• Please do not discuss your annotations with each
other
• Please mail your annotated ﬁles to
a.serebrenik@tue.nl
1) Annotation
Deadline: Oct. 30

• Identify a paper published in 2015 or later in an important software
engineering (conferences: ICSE, FSE, ASE, ICSME, MSR; journals: TOSEM, TSE, EMSE) or
collaborative development venue (CSCW, CHI) about one of the topics
discussed in the class. Please do not summarise my papers.
• Summarise the paper: what problem do the authors want study, why is it
important, what research method(s) did they use, what results did they
obtain and how are these results interpreted.
• Review the paper: are the results technically sound? are they novel, i.e.,
how did the authors position their work with respect to earlier results? can
you reproduce the analysis? is the paper well-written?
• Give a personal opinion: why did/didn’t you like this paper? How would it
contribute to your PhD research?
• IEEE Conference Proceedings Formatting (two columns) https://
www.ieee.org/conferences/publishing/templates.html Max: 4 pages +
bibliography.
2) Essay
Deadline: Oct. 30

Communication between software developers: challenges of sentiment analysis

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Communication between software developers: challenges of sentiment analysis

Ähnlich wie Communication between software developers: challenges of sentiment analysis (20)

Mehr von Alexander Serebrenik

Mehr von Alexander Serebrenik (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Communication between software developers: challenges of sentiment analysis