Calls for both practical and scholarly activities to be grounded more in actual evidence have become louder, especially in the last decade. Four domains in particular have embraced evidence-based thinking, resulting in the respective developments of evidence-based medicine, evidence-based management, evidence-based education and evidence-based policy. Despite the presumed benefits of drawing on different sources of evidence for decision-making in practice, whether in medicine, management, education or policy, this does not seem to prevail. Whilst one likely reason for this slow uptake could simply be down to practitioners not always having much time to consult the evidence-base in their day-to-day work, another reason might be that they are not aware of specific insights applicable to their domain of work or to practice in general. This is where the workshop contributes: Representatives from the four key domains engaged with evidence-based practice will share with the audience their latest insights and the consequences thereof for practice. Further, all speakers will discuss questions such as: What do we have in common? How can we learn from one another? How can we combine insights from the four domains? These will be discussed as part of a concluding panel. Workshop organiser: Dr Celine Rojon, University of Edinburgh, celine.rojon@ed.ac.uk

1. Evidence-Based Management
Evidence-Based Practice
Insights from Key Domains
University of Edinburgh, May 2nd 2014
Eric Barends Rob Briner

2.  A brief history
 Current developments
 Future developments
 Challenges
 Implications for practice & research
2
Evidence-Based Management

3. 3
Evidence based management:
A brief history

4. 4
Manager > ‘Manage’
introduced mid 16th century: from Italian
maneggiare, “to handle”, based on Latin
manus, “hand”.
Management?
(we’re all managers)

5. History of management/business schools
 Late 1800s - started as trade schools
 1950s - Ford and Carnegie reports criticised business
schools for not being sufficiently academic or scientific
 1980s onwards – criticised for producing research
irrelevant to practice and students without relevant skills
or knowledge
5

6.  On the one hand they fear . . . the scorn of other, more
traditional academic subjects. On the other hand, they
often stand accused of being less than relevant to
business. (Grey, 2001)
 2000s onwards – criticized for producing ideas and
students that lead to financial crisis and poor ethical
behaviour (e.g., Enron, only one US president has MBA,
MBA students cheat more than other graduate students)
6
History of management/business schools

7. Management schools are big business
 Nat. Center for Education Statistics (USA, 2011-’12)
 20% of all Batchelor’s Degrees
 25% of all Master’s Degrees
 Higher Education Statistics Agency (UK, 2012-’13)
 27% of all Higher Degrees awarded
 18% of all Higher Education qualifications obtained
 UK Council for International Student Affairs (2011-’12)
 36% of all business school students are international
 30% of all international students studying business
7

8. EBP in Management
 Many similar ideas in past (e.g., Mode 1-Mode 2,
collaborative research, rigour and relevance)
 Rousseau (2005) Presidential address
 Peffer and Sutton (2006) Hard Facts, Dangerous Half-
Truths, and Total Nonsense: Profiting from Evidence-
based Management
8

9. EBP in Management
 EBMgt Collaborative (2007-’08) Rousseau/Carnegie Mellon
 Wiley Evidence-Based Management Insights (2007-’08)
(attempt to create Cochrane database for management)
Briner and Denyer
 Presentation to AoM Board of Governors (2008) (attempt to
persuade to develop systematic reviews in management)
 Center for Evidence-Based Management – more later
9

10. Evidence based management:
What is it?
10

11. 11

12. Evidence-based practice
Central Premise:
Decisions should be based on a
combination of critical thinking
and the ‘best available evidence‘.
12

13. Evidence?
findings from scientific research,
organizational facts & figures,
benchmarking, best practices,
professional experience
13

14. All managers base their
decisions on ‘evidence’
14

15. But…many managers pay
little or no attention to the
quality of the evidence they
base their decisions on
15

16. Trust me, 20 years of
management experience
16

17. SO ...
17

18. Teach managers how to
critically evaluate the validity,
and generalizability of the
evidence and help them find
‘the best available’ evidence
18

19. Evidence-based practice is about making decisions through
the conscientious, explicit and judicious use of
the best available evidence from multiple sources by
1. Asking: translating a practical issue or problem into an answerable question
2. Acquiring: systematically searching for and retrieving the evidence
3. Appraising: critically judging the trustworthiness and relevance of the evidence
4. Aggregating: weighing and pulling together the evidence
5. Applying: incorporating the evidence in the decision-making process
6. Assessing: evaluating the outcome of the decision taken
to increase the likelihood of a favorable outcome.
Definition
19

20. Evidence based decision
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ values
and concerns
Scientific
research
findings
Ask
Acquire
Appraise
Aggregate
Apply
Assess
20

21. Evidence-based practice:
Focus on the decision making process
Think in terms of probability
21

22. Differences?
 Research: low internal validity

23. 0
50
100
150
200
80-84 85-89 90-94 95-99 00-04 05-09
uncontrolled controlled
Controlled vs uncontrolled studies

24. Differences?
 Research: low internal validity
 Lack of evidence summaries
 Focus on multiple sources
 Organizational (BIG) data
 Stakeholders’ concerns

25. How evidence-based are we (managers)?
“I’ve never thought I need more
evidence before making a decision;
I know what needs to be done, we
get on with it and we get results.”
25

26. How evidence-based are we?
 959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body of evidence
 true / false / uncertain
HR Professionals' beliefs about effective human resource practices: correspondence
between research and practice, (Rynes et al, 2002, Sanders et al 2008)
26

27. 1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group performance while
relational conflict harms it.
3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.
True (likely) or false (not likely)?
27

28. Outcome: not better than random chance
28

29. Evidence based management:
Current developments
29

31. Postgraduate Course
31

32. Postgraduate Course

33. Postgraduate Course
Current developments
Education
Access to research databases
REAs
Building a community

34. Evidence based management:
Future developments
34

35. Postgraduate Course
CEBMa Database of Evidence Summaries
Online learning modules
Accreditational bodies
Future developments: practice

36. Postgraduate Course
 Some move towards systematic review methods
in management
 Limited interest in teaching EBP in management
schools
 Management schools still a bit trapped: Cash
cows for universities; focus on ‘top quality’
academic research; few incentives to be relevant
Future developments: academia
36

37. Postgraduate Course
 Though issues around accountability, ethics,
corporate social responsibility could in part be
resolved by research and teaching EBP
 Need for a professional doctorate?
Future developments: academia

38. Evidence based management:
Challenges
38

39. In general
incentives to do non-EBP and punishment
(or no incentives) for doing EBP
39
Challenges
(same for practitioners and academics)

40. Incentives for managers
 Not rewarded for doing what ‘works’ – few evaluations
 Speed and action valued more highly than accuracy
and analysis
 Managing and understanding power and politics to get
things done more valued than understanding and
using evidence to make decisions
 It may be too late to change existing senior
managers…
40

41.  Barriers from academic / buss school context
• In publishing and research
• In teaching content and teaching style
 Barriers from organizational contexts
• Managers love fads and quick fixes
• Power and politics
41
Other challenges

42. In general: academics don’t like EBMgt

43.  Ambivalence about the value and applicability of
management research
 Few incentives to get involved
 Primary research (collecting new data) valued more highly
than secondary research (reviewing existing data)
 EBMgt not academics‘responsibility – this is about practice
not research
 Some concern that systematic reviews will expose the
limited nature of management research
 Some academics are like ‘gurus’ and feel that EBMgt
might show their claims to be untrue
Why don’t academics like EBMgt?

44. Espoused and more implicit goals of
management school educators
ESPOUSED GOALS
 To develop critical thinking
 To help students understand the
full body of knowledge
 To discuss gaps and limitations
of our knowledge
 To challenge thinking and
assumptions
 To educate
 To teach students how to think
for themselves
 To maintain quality standards
IMPLICIT GOALS
 To help students feel
successful
 To select those bits of that are
interesting or digestible or
‘cutting edge’ fads
 To reassure students that what
we’re teaching them is solid
 To make sure students are
satisfied
 To entertain (edutainment)
 Teach students what they need
to pass assessments
 To give ever-higher grades
44

45. ESPOUSED GOALS
 To advance scientific
understanding
 Using the best research
techniques
 Publishing all results and
replications – unbiased
 Focus on what’s important
 Being honest about existing
evidence
 To disseminate all our
evidence and make publically
available
 Collaboration & cooperation
IMPLICIT GOALS
 To advance career
 Use whatever techniques will
get you published
 Publishing (mostly) only
positive results, no replications
 Identifying ‘new’ or trendy
topics – creating empires
 Exaggerating how much we
know
 Locking up our evidence
behind publishers’ pay walls
 Competition for resources,
slots in journals, between
universities 45
Espoused and more implicit goals of
management school researchers

46. Implications for research
 Less focus on collecting new primary data
 More focus on systematic reviews and
understanding what we know and don’t know
 Focusing research efforts on what is
academically and/or practically important
without compromising ‘academic freedom’
 Broader-based research training –
management researchers highly and
narrowly specialized
46

47. WHAT’S STOPING US?In general: managers don’t like EBMgt

48.  Undermines formal authority
 They feel it constrains freedom to make
managerial decisions
 Speed valued and rewarded more than accuracy
 Feel they cannot use their own experience and
judgment (not true)
 Managers not necessarily rewarded for doing
what works (organizations rarely evaluate)
 THEY LOVE FADS & QUICK FIXES
Why don’t managers like EBMgt?

49. Huge (peer) pressure to adopt fads & quick fixes
Fads & quick fixes

51. Postgraduate Course
51

52. Postgraduate Course
How are fads a problem?
 “The main problem…is their lack of any solid
intellectual foundation. Implicit in each fad is a
cause effect statement that is rarely made
explicit and never properly supported.”
52(Donaldson & Hilmer, 1998)

53. 53

54. 54

55. 55
George BuckleyJames McNerney

56. QUICK FIXES
Because quick fixes / fads
 Can be career-enhancing for managers
 Speed is often valued over accuracy
 Do we crave quick and easy solutions?
So who needs or wants academic research?
So why do managers love quick fixes?

57. Implications for management practice
 Focus on accountability
 Focus on next generation
 Management = profession
 EBP > Professional standard >Accreditation
57

58. Think about this
How can a profession / domain be evidence-based
if it is not managed in an evidence-based way?
58
“No job is more vital to our society than that of a
manager. It is the manager who determines whether our
social institutions serve us well or whether they
squander our talents and resources.”
Henry Mintzberg