This document discusses industry-academia communication in empirical software engineering. It provides context on a conference in 1968 that aimed to improve communication between industry and academia. It notes key differences in time horizons and languages between the two. Industry focuses on short-term market changes and profits, while academia focuses on long-term learning and publications. The document advocates for both sides to learn each other's languages and cultures to improve collaboration and help tear down walls between the two. It provides examples of successful collaboration projects over time that have helped improve practice.

1. Industry-Academia Communication
in Empirical Software Engineering
PROFESSOR PER RUNESON

2. Once upon a time…
It was suggested that about 50 experts from all areas
concerned with software problems — computer
manufacturers, universities, software houses, computer
users, etc. — be invited to attend the conference.
In late 1967 the Study Group recommended the holding
of a working conference on Software Engineering.
It was hoped that the Conference would be able to identify
present necessities, shortcomings and trends and that the
findings could serve as a signpost to manufacturers of
computers as well as their users.
The phrase ‘software engineering’ was deliberately chosen
as being provocative, in implying the need for software
manufacture to be based on …theoretical foundations and
practical disciplines…

3. Participants 1968
• Industry – 27
• Academia – 21
• Government – 7
Examples:
• A/S Regnecentralen, Denmark (Naur)
• IBM Corporation, USA (Randell)
• TUM, TUE, Stanford, DTU…
• Atomic Weapons Research
Establishment, England

4. What happened then?
East is East and West is West
and never the twain shall meet
Rudyard Kipling, 1892.
Barrack-Room Ballads
Academia…
...developed the
”theoretical foundations”
Industry…
…developed the
”practical disciplines”
ENIAC programmers

5. A Tale of Two Countries
Industria Academia
Praguge, 1968
Lund, 1968
Berlin, 1968

6. The wall between industry and academia

7. Is SE unique?
The experienced marketing
practitioners interviewed knew
very little about the current state
of academic research in
marketing, and considered that
academic researchers did not
understand the realities of
business life and could not
communicate effectively with
managers.
Marketing practitioners prefer
to work with consultants, whom
they consider understand
business realities better and are
more effective communicators.

8. Is SE unique?
From Human-Computer
Interaction:
Researchers, who would
like their ideas to impact
practice, complain that
… practitioners do so
incorrectly…
Practitioners, in turn, complain
that research results, even if
relevant, never exist in any form
that can readily be translated
into practice.

9. Mr. Gorbachev, tear
down this wall
Software Engineering
tear down this wall

11. Bi-lingual

12. Erik Axel Karlfält
(1864-1931)
He is talking to farmers in farmers’ ways
but with learned men in Latin
Han talar med bönder på böndernas sätt
men med lärde män på latin
Sång efter skördeanden (Fridolins visor)

13. Language
Industry-speak
”We do agile”
“We do deep learning”
Good-enough for the
purpose
Academia-speak
”The company applies some
agile practices, mostly
influenced by Scrum”
“The company applies
stacked generalization
combining several classifiers”
Rigorous language improves
validity

14. The academic languague
• 22 pages
• 28 references
• IMRAD structure

15. The practitioner language…
… as defined by the academics
• 7 pages
• 4 references
• Colors and graphs
• “Catch the reader” structure

16. Multiple languagues 2005:
”When we used our traditional
development methodology, you
could squeeze in all the new features
we needed. I feel as though I’m no
longer in control here.”
2017:
”They lash out against the loss of
control. The more independent
teams, the less control you have [as
a manager], says Ola Berg.”
…are still not sufficient

18. Stand up!
• Talk to you neighbor for 2 minutes
• about one example of
• language confusion
• between industry and academia

19. Who should learn the other language(s)?
Development aid as an example – energy efficiency
Eastern Usambara, Tanga, Tanzania
© Isaac Malugu/ WWF TanzaniaCC BY 2.0 Tony Alter @Flickr

20. Who should learn the other language(s)?
Industry-academia communication

21. Supporting mechanisms – Visual Abstracts
• ESEM 2017 short paper

22. Approach to
understand
problem
To achieve an effect in a situation apply this intervention
Problem
instance(s)
Addressed problem instance(s)
Solution(s)
Proposed solution(s)
Evaluation approach Approach to
design
solution
Problem relevance
Scientific rigor
Novel contributions

23. Related work
quantifies the
scale of the
problem in real
projects
To achieve more effective assignment of bugs to teams in large scale industrial
contexts use ensemble-based machine learning to automate bug assignment
Problem
Labour-intensive & error-
prone bug assignment in
two companies from
telecom and automation
domains
Solution
Stacked
generalization (SG),
combining several
classifiers, automates
bug assignment
Application of
solution to bug data.
Applied
state-of-the
art ensemble
learner
Problem observed in real projects: Eclipse Platform (Anvik & Murphy 2011), Mozilla foundation (Bhattacharya
et al. 2012), and at Ericsson (Jonsson et al. 2012). Evaluated on data from Telecom and Automation domains.
Evaluated in 5 real projects across 2 companies/domains, on 50 k bug reports, using K-fold cross-
validation and sliding window validation.
Precision in automated bug assignment on par with manual (50-89%), which makes it useful in practice, saving
cost and time. SG outperforms individual classifiers. When training SG, aim for at least 2,000 bug reports in the
training set. Relying only on K-fold cross-validation is not enough to evaluate automated bug assignment.

24. Supporting mechanisms – Evidence Briefings
• ESEM 2016 paper

25. Evidence Briefings
1. The title of the briefing.
2. A short paragraph to present the goal of the
briefing.
3. The main section that present the findings
extracted from the original systematic review.
4. Informative box that outlines the intended
audience and explains the nature of the
briefings’ content.
5. The reference to the original systematic review.
6. The logos of our research group and university.

26. Voice of evidence

27. Audiences of a (case study) report
• The sponsor(s)
• Stakeholders and participants
• Practitioner communities
• Government and policy making bodies
• Research communities
• Educational communities
[Runeson et al, 2012]

29. Bi-cultural

30. https://xkcd.com/664/

31. Understand business logic
Industry
• Market shares
• Profit
• Funding from customers
• NDA
Academia
• Publications and citations
• Break-even
• Funding from agencies
• Secrecy act

32. What matters?
• Credibility of knowledge from
industry practitioner's perspective
• Academia: Localize and
translate
• Industry: Learn to distinguish
opinions from knowledge
Source of
knowledge
Type of knowledge
Opinion Empirical
Local 1 (most) 2
Remote 3 4 (least)
Persuading Developers to ‘Buy into’ Software Process Improvement: Local
Opinion and Empirical Evidence
Austen Rainer, Tracy Hall, Nathan Baddoo
Centre for Empirical Software Process Research (CESPR)
University of Hertfordshire
Department of Computer Science
Hatfield Campus, College Lane
Hertfordshire, AL10 9AB
England
a.w.rainer@herts.ac.uk
In order to investigate practitioners’ opinions of software
process and software process improvement, we have
collected a large volume of qualitative evidence from 13
companies. At the same time, other researchers have
reported investigations of practitioners, and we are
interested in how their reports may relate to our evidence.
Thus, other research publications can also be treated as a
form of qualitative data. In this paper, we review advice
on a method, content analysis, that is used to analyse
qualitative data. We use content analysis to describe and
analyse discussions on software process and software
process improvement. We report preliminary findings
from an analysis of both the focus group evidence and
four publications.
Our main finding is that there is an apparent
contradiction between developers saying that they want
evidence for software process improvement, and what
developers will accept as evidence. This presents a serious
Some other research, however, suggests possible negative
effects of SPI. For example, Kuilboer and Ashrafi’s [5]
survey of developers suggests that companies conducting
SPI for a longer period of time showed an overall increase
in development cost and project duration. Gray and Smith
[6] criticise process assessment and improvement on
theoretical grounds. Their most fundamental criticism is
that the software research community still only has a poor
understanding of the software process. This criticism is
similar to previous observations made by Abdel-Hamid
and Madnick [7] and Remenyi and Williams [8]. For
example, Remenyi and Williams [8] observed that we lack
an established theory of software development, and
proceeded to argue for a grounded-theory approach (e.g.
[9] [10]) to investigating the software process.
One important aspect of process engineering is
implementing a new, or modified, process. While the
research community and industry needs to better
understand process, so the research community and

33. Time horizons
”Industry is concerned with market
changes and product plans, which
have months rather than years as their
time horizon.
Academia is based on learning cycles
of generations, and years are the time
horizon for education programs.”
Get the Cogs in Synch – Time Horizon Aspects of
Industry–Academia Collaboration
Per Runeson
Dept. of Computer Science
Lund University
Sweden
per.runeson@cs.lth.se
Sten Minör
MAPCI – Mobile and
Pervasive Computing Institute
at Lund University
Sweden
sten.minor@cs.lth.se
Johan Svenér
Sony Mobile Communications
Lund
Sweden
johan.svener@sonymobile.com
ABSTRACT
In industry–academia collaboration projects, therearemany
issues related to different time horizons in industry and aca-
demia. If not adressed upfront, they may hinder collabora-
tion in such projects. We analyze our experiences from a 10
year industry–academia collaboration program, the EASE
Industrial Excellence Center in Sweden, and identify issues
and feasible practices to overcome the hurdles of different
time horizons. Specifically, we identify issues related to con-
tracts, goals, results, organization (in)stability, and work
practices. We identify several areas where the time horizon
is different, and concludethat mutual awareness of these dif-
ferences and management commitment to the collaboration
are the key means to overcome the differences. The launch
of a mediating institute may also be part of the solution.
1. INTRODUCTION
Long-term industry–academia collaboration involvesmany
challenges. One key challenge is the difference in time hori-
zon between industry and academia. Industry is concerned
with market changes and product plans, which have months
rather than years as their time horizon. Academia is based
on learning cycles of generations, and years are the time
horizon for education programs. These differences may cre-
ate friction and frustration in industry–academia challenges,
like cogs rotating at different speed. However, since the dif-
ferences are mostly by design, we should rather learn to live
with them, or changethe design, i.e. to get thecogs in synch.
In thispaper, weanalyzethetimehorizon aspectsof a long
term industry–academia project, based on our experiences
from shaping and running the EASE Industrial Excellence
Center1
, and the Sigrun Software Innovation and Engineer-
ing Institute2
. The authors represent three stakeholders in
1
http:/ / ease.cs.lth.se
2
http:/ / www.sigrun.se. The organizational host for Sigrun
is currently under investigation, but for simplicity, we refer
to it as a separate unit here as it has been up till now.
this collaboration project, the largest industry and academia
partners, and an institute organisation, aimed at mediating,
among others, the time horizon aspects.
Published models for industry–academia collaboration fo-
cus on activities, as proposed by Gorschek et al. [3], or re-
lations, as proposed by Sandberg et al. [12]. Other work
includes surveys of success factors [14], experience reports
on specific projects [8] and summaries of challenges for the
industry–academia collaboration [13].
Sandberg et al. stress the time horizon aspect, stating:
Although it’s relatively easy to agree on challenges and goals,
view-points differ regarding variables such as relevance, rigor,
time horizons, planning practices, and predictability[12]. Woh-
lin’s identified challenge to Integrate into daily work [13]
is highly related to time horizons. Runeson notices that
[a]ppreciation in industry ... comes with fulfillment of short
to medium term project goals, while the incentives in aca-
demia is related to publications and citations – with year-
long feed-back cycles[8]. Further, he observes these differ-
ences scaled down to the daily planning: Researchers make
commitments far ahead of time for e.g. conference organi-
zation and teaching, while industry staff re-plan their com-
mitments on daily, or even hourly basis, for higher manage-
ment [8].
The paper is structured as follows. In Section 2, the in-
dustry and academia contexts are described. In Section 3
we discuss the different time horizons in industry and aca-
demia, and consequences for the collaboration. We conclude
the paper in Section 4.
2. CONTEXT
The context in which these observations are made is an
industry–academia ecosystem on softwareengineering in south-
ern Sweden, which consists of three universities, a dozen of
larger industry players and many small software companies.
Within thisecosystem, we specifically focus on the EASE In-
dustrial Excellence Center, the Sigrun Software Innovation
and Engineering Institute, and their industrial partners.
The EASE Industrial Excellence Center is a ten year en-
Table 1: T ypical t im e hor izons in indust r y–academ ia
collaborat ion (years)
Area Industry Academia
Contracts 1 – 3 3 – 5
Goals 1/ 4 – 3 3 – 5
Results 0 – 3 3 – 10
Organization 1 – 3 5 – 10
Work practice 0 – 1/ 2 0 – 3

34. Knowledge cycles
Knowledge
Career
40 years
20 years
10 years
1 years
5 years
Paradigm shift
Research program
Company operations
Education program

36. Carl Linneus (1707–1778)
botanist, physician, and zoologist
• Student in Lund 1727–
1728
• Observed nature and
farming practices
• Developed theoretical
foundations and
practical disciplines

37. SERP- Software Engineering Research and Practice
http://serpconnect.cs.lth.se

39. Communication

40. Science communication … is not about moving
information from point A to point B.
Megan K Halpern
https://meganhalpern.com/2017/11/06/science-communication-as-experience-a-bit-about-
the-book-in-progress/

42. Collaboration tears down the wall
Academia Industry
Industrial
Research
Research
Institute
Applied
Research
Industry
Evidence-
based
practice

43. Education tears down the wall
Academia Industry
Applied
Courses
Master
Theses Employ
Students

44. Example
2005 – Master thesis project
2007 – ICSE paper
2010 – PhD thesis project
2015 – EMSE paper
2017 – Industry practice

45. Collaboration – or Business?
• Universities – a warehouse
full of products and advice?
– Fix a problem
– Seeks some tips
– Hire some students
– …

46. Collaboration – not Business!
• Industry-academia
collaboration is not
transactional, it is relational
– Academia has sometimes
disruptive answers/questions
– To get ready-made answers, use
consultants, if that’s what you
are looking for 
A university can supply
multiple solutions or
approach problems in a
way that makes it clear the
corporation wasn’t looking
for the right solution to
begin with.

47. Relational model for collaboration
Need
orientation Research result
Industry goal
alignment
Research activity
Deployment
impact
Industry
benefit
Innova-
tiveness
Management
engagement
Network
access
Collaborator
match
Communi-
cation ability
Continuity
Sandberg et al 2011

49. 1989

50. THE END
• Learn each others languages and cultures
• Embrace the differences
• Communicate!

51. Bibliography
• P. Runeson, M. Alexandersson, and O. Nyholm. Detection of duplicate defect reports using natural language processing.
ICSE 2007.
• P. Runeson. It takes two to tango – an experience report on industry–academia collaboration. In Testing: Academic and
Industrial Conference - Practice and Research Techniques (TAIC-PART), pages 872–877, 2012.
• P. Runeson and S. Minör. The 4+1 view model of industry–academia collaboration. In International Workshop on Long-term
Industrial Collaboration on Software Engineering (WISE). ACM, 2014.
• P. Runeson, S. Minör, and J. Svenér. Get the cogs in synch – time horizon aspects of industry–academia collaboration. In
International Workshop on Long-term Industrial Collaboration on Software Engineering (WISE). ACM, 2014.
• L. Jonsson, M. Borg, D. Broman, K. Sandahl, S. Eldh, and P. Runeson. Automated bug assignment: Ensemble-based
machine learning in large scale industrial contexts. Empirical Software Engineering, 21(4):1579–1585, 2016.
• B. Cartaxo, G. Pinto, E. Vieira, and S. Soares, Evidence Briefings: Towards a Medium to Transfer Knowledge from
Systematic Reviews to Practitioners, ESEM 2016, pp. 57:1–57:10.
• M.-A. Storey, E. Engstrom, M. Host, P. Runeson and E. Bjarnason
Using a Visual Abstract as a Lens for Communicating and Promoting Design Science Research in Software Engineering,
ESEM 2017

52. Image attributions
• By US Army - Modified version of
http://www.globalsecurity.org/military/library/report/other/us-
army_germany_1944-46_map3.htm, Public Domain,
https://commons.wikimedia.org/w/index.php?curid=7849320
• By The Central Intelligence Agency - 10 Soviet Invasion of Czechoslovakia,
Public Domain, https://commons.wikimedia.org/w/index.php?curid=29195095
• CC BY-NC-ND 2.0 Eddie @ Flickr
• CC BY-NC-ND 2.0 jcsuperstar60 @ Flickr
• XKCD