Sandra Macleod, Echo Research www.charitycomms.org.uk/events

1. Evaluating PR Effectiveness
21 July 2011, London
Web: www.charitycomms.org.uk
Twitter: @CharityComms

2. Evaluation PR & Measurement in Action
Charity Communication Presentation
Sandra Macleod, Group CEO, Echo Research
E. sandram@echoresearch.com
© echo
21 July 2011

3. Outline
1. About Echo
2. Setting PR Objectives
3. Measuring PR
4. Case Studies
5. Echoes of the future …
© echo | Page 3

4. Evaluation PR & Measurement in Action
About Echo
“The soul never
thinks without a
picture”
Aristotle

5. About Echo Research
22-year track record in communication research - Echo has supported 500 world-class clients incl:
media content analysis, stakeholder
evaluation and reputation measurement.
Now part of Ebiquity plc for integrated above
& below line ROI
and insight.
50 staff and 150 analysts
Offices in London, Paris,
New York, Singapore
Members of
UN Global Compact
All Echo research complies with the MRS Code of
Conduct and
ISO 9001:2008; Registered Expert Witness on
image and reputation. Winner of 89 industry
awards for best practice in communications
and reputation research
© echo | Page 5

6. Publicity-based Research
Echo shows IT plays major role in
corporate ‘green’ initiatives
© echo | Page 6

7. Thought Leadership
• Echo Chambers
(London, NY, Paris),
Annual Echo Summit
• White Papers (Web 2.0,
Communications
Research Wiki)
• Global reports on CSR
and Trust (freely
available to download
from our website)
© echo | Page 7

8. Evaluation PR & Measurement in Action
Setting PR Objectives

9. Making the Intangibles Tangible
The Organisation Activities Output
values, objectives, messages sent by messages received
strategies company by audience
media analysis
market research
Driver Analysis
Values and
expectations
Outcome
Audience choices / behaviours
Advocacy & 3rd party Outtake / Impact
endorsement audience perceptions
Recruitment / retention market research
Brand equity
Regulation
© echo | Page 9

10. What Are Your Objectives?
© echo | Page 10

11. What Are Your Objectives?
•Building awareness / visibility
•Fundraising
•Membership recruitment and retention
•Loyalty building
•Networking and connecting
•Thought Leadership
© echo | Page 11

12. Why Setting Objectives Is Key
OUTPUTS OUTTAKES
SETTING OBJECTIVES RESEARCH & PLANNING RESULTS
(CONTENT) (INTERPRETATION)
• Replicable • Links outputs to • Truthful/believable • Creates clear • Objectives set in the
outcomes brand/corporate planning stage were
• Clearly defined • Communicates differentiation achieved
audiences/ • Identifies audience clearly
stakeholders needs • Communicates • Measurable impact
• Flawless execution desired information on stakeholder
• Objectives support • Relevant/salient to relationships
• Addresses • Creates a positive
organisational goals target audiences
stakeholder needs image • Measurable impact
• Clearly stated • Research-based on behaviours
• Consistent
business objectives programme and/or attitudes
implementation
• Measurable • Quantifiable • Demonstrable ROI,
• Single minded
business objectives evaluation metrics including non-
• Integrated with financial indicators
• Identifies obstacles
other marketing such as increased
& barriers to launch
disciplines awareness
• Executable
programme
© echo | Page 12

13. New Priorities: Stakeholder Engagement
“Companies and industries with reputation problems are more likely to
incur the wrath of legislators, regulators and the public.
“To prepare for and respond to serious reputational threats, companies should
emphasize three priorities.
1.Develop a rich understanding of key stakeholders
2.Focus on the actions that matter most to stakeholders
3.Influence stakeholders through techniques that go beyond traditional PR
approaches, with an emphasis on two-way dialogue.”
Sheila Bonini, David Court, Alberto Marchi
The McKinsey Quarterly
© echo | Page 13

14. Reputation: for WHAT and among WHOM?
Communities Consumer
NGOs
groups
Customers
Regulators Partners
Potential
customers
Organisation The Media
Suppliers
Employees
Shareholders
Trade unions
Potential &
Government employees professional
Analysts bodies
© echo | Page 14

15. Reputation Drivers On Scorecards
Management
Performance
Ethics Financial Performance
Environmental Innovation
Performance
Marketing Workplace
© echo | Page 15

16. Measurement that matters
• Reputation affects attitudes (like satisfaction, commitment,
and trust) and drives behaviour (like loyalty, advocacy, support)
– THESE ARE THE THINGS THAT MUST BE MEASURED
• Reputation exists in the minds of a myriad of stakeholders and ‘influencers’
and is based on a summation of influences and
experiences over time through:
• Direct experiences
• Employee advocacy
• Direct Communication
• Indirect Communication – know who is influencing whom
© echo | Page 16

17. Evaluation PR & Measurement in Action
Measuring PR
© echo 17

18. Echo’s Integrated Reputation Research
Key stakeholder groups:
• Personnel *
Identity
• Customers*
• Opinion Formers
• Recruits*
• Investors
Image
• Suppliers
• NGO
• Community .....etc
Content
Analysis
• Media/web 2.0 Influence
© echo | Page 18 *current/potential/lapsed

19. Triangulation (I³ or I-Cubed)
© echo | Page 19

20. Influence of media messages
uptake of messages in the media compared to awareness among target audience
Audience awareness
© echo | Page 20

21. Identity vs Image
= Identity
Perception Gap = Image
© echo | Page 21

22. Media content evaluation
Develop
Media Lists
Analyse, Code Quality Control
& Rate Articles
Develop Issues
& Messages
Report
Production
Ongoing Updates Report
and Finetuning of Delivery &
issues & messages Review
© echo
© echo | Page 22

23. © echo | Page 23

24. © echo | Page 24

25. Image on the Internet vs Other Media
© echo | Page 25

26. Penetration of campaign messages
Penetration and favourability of messages in UK national media, against uptake of message in target audience.
© echo | Page 26

27. Visualisation: From Diagnosis to Solutions
Discover Address & Enhance
• Baseline
measures: how
are we • What is the
perceived? perceived ideal
• What do we do reputation for a • What should we do or
particularly well / group like ours? say differently?
poorly? • How should we • How should we
• How do we position and communicate and • How do we
compare to differentiate engage with key measure
competitors? ourselves? stakeholders? success?
• What does • How do we manage
‘success’ look the risks and
like? opportunities?
© echo | Page 27

28. Evaluation PR & Measurement in Action
Case Studies
© echo 28

29. Case Study: Aligning Research
• Echo has provided communications research support to
WWF for many years
Campaign evaluation – media content analysis to
evaluate effectiveness of PR and media outreach e.g.
Earth Hour – the global initiative to encourage people to
switch off their lights for one hour.
Journalist surveys to track awareness of WWF and its
activities and to drive improvements to the media
relations function.
Research audit of existing WWF brand research held
across the organisation. To help WWF to
align research to focus on three brand pillars
develop consistent brand measurement metrics
identify gaps in research
maximise the value of the research.
© echo | Page 29

30. Media Relations Scorecard
Positive Points Monthly Trend
by volume, favourability & rating
Negative Points
Rating rose from 53.5 to 56.0 due to a high 160 70
7 Negative comment was minimal
proportion of favourable coverage (54% in February Note: Qualitative analysis this month with just 1% of coverage
140
to 81% in March). This was very much boosted by started in Jan 03 65 rating under 50.0.
120
Fundraising campaign reviews and feedback from
100 7 Criticism came from the West
recipients. 60
# articles
rating
80 Highland Free Press, who
Volume remained virtually the same as last month.
60 55 commented on the agreement
Interviews scored very highly in March, with an
40 between the WWF and Lafarge –
article about WWF’s Deputy CEO Leslie Jones 50
20 Redland.
achieving an impressive 70.0 rating.
Once again the issue of the proposed Lignite mine 0
Nov-02 Dec-02 Jan-03 Feb-03 Mar-03
45
7Proactive coverage was also
was the focus of many articles in Ireland with Mr Favourable Unfavourable Neutral Rating
down on February from 25% to
Malachy Campbell quoted in the Northern and South 19%.
Derry Constitution, 1/3, “In Northern Ireland we have Leading Descriptors 7 Negative messages were down
fabulous rivers and they should be protected. Lignite • WWF (67 mentions, + 56%) this month although ‘an
mining should not go ahead”. • World Wildlife Fund (52 mentions, + 63%) organisation with unclear and
Message penetration remained the same at 1: 1.4 • Worldwide Fund for Nature (8 mentions, - unrealistic objectives’ did appear
articles. 20%) once in relation to the conflicting
• Charity (7 mentions, + 20%)
Campaigns proved very popular with Big Swim and Lafarge agreement.
Global Applause gaining good volume and high
ratings. Total Reach : 14.4 million (- 34%)
Monthly Ratio Leading Media
by volume by volume & rating
Neutral Publication Volume Rating Leading Bylines
18 (23%) Tonbridge Courier 4 52.5 by volume & rating
Scotland on Sunday 3 55.0
Unfavourable The Scotsman 3 51.6
1 (1%) Bridgnorth Journal 2 55.0
Byline Volume Rating
Clacton Gazette 2 57.5
Dawlish Post 2 52.5 Bryan Millin - Dunstable Gazette 2 55.0
Harwich & Manningtree Standard 2 62.5 Charles Glover - The Daily Telegraph 2 52.5
Rutland & Stamford Mercury 2 57.5
Teignmouth News 2 55.0 Ian Johnston - Scotland on Sunday 2 55.0
The Daily Telegraph 2 52.5 Sue Howlett - Clacton Gazette 2 65.0
The Times 2 50.0
Times & Citizen 2 60.0 With more than one article
Favourable
85 (76%) Woman's Weekly 2 57.5
With more than one article
© echo | Page 30

31. © echo | Page 31

32. Reputation Measurement Scorecard
(illustration only)
The latest data on stakeholder perceptions highlights inroads made with targets in Government
Perceptions and Supporter Perspective showing a continuing improvement on influence and action.
Of note, though, is the latest ‘Business & Industry’ survey indicates that overall perceptions are down
on the 2006 figures. Awareness is still low, and the ‘dip’ in perceptions of influence further
investigation.
Supporters Business & Industry Government
© echo | Page 32

33. Case Study: Influencing the Media Agenda
• PneumoADIP was formed by the Global Alliance of Vaccines and
Immunisation (GAVI) to speed up the provision of vaccines for
pneumococcal diseases to the world’s poorest countries.
• Initial research showed that despite killing more children than AIDS,
malaria and measles combined, there was a significant lack of
awareness of pneumococcal disease among health policymakers
and in the media.
• Echo helped PneumoADIP evaluate its progress in moving
pneumococcal disease up the media agenda.
• In addition, the findings were presented to an influential audience
of pneumococcal disease experts at two pneumococcal disease
congresses.
Gold AMEC Award winner -
“easy to read, good analysis
and good coverage” AMEC
Judge
© echo | Page 33

34. Case Study: Media Landscape Analysis
• Detailed analysis of the media landscape in 15 countries in the
East Asia and Pacific region (including North Korea, Papua New
Guinea, Myanmar and Timor Leste).
• To support UNICEF in its communication activities and media
outreach on emerging infectious diseases.
• Desk research plus over 50 in-depth interviews with media
experts.
• Identified the most influential medium and media outlets for
reaching different target audiences in each country.
• Also included information on media freedom and legislative,
cultural and historical factors together with new media trends.
© echo | Page 34

35. Case Study: Motivating Volunteers
• To identify current and potential volunteer motivations and to
uncover any disconnect between RNLI volunteers and staff to
ensure a cohesive organisation focused on the same goals.
• Echo applied its I3 (i-cubed) research approach:
IDENTITY - how the RNLI views itself - depth interviews with
senior executives, focus groups with staff and a review of RNLI
internal and external literature
IMAGE - how key audiences view the RNLI - quantitative
research with existing and potential volunteers, staff and
general public
INFLUENCE - how the media portrays the RNLI - media
content analysis benchmarked against other national charities.
• “The findings and recommendations will inform the future direction
of the RNLI’s relationship with volunteers for many years to come,
but there have been a number of ‘quick wins’ enabling us to show
volunteers that we have both listened to them and acted upon their
concerns.” Ginette Tessier, RNLI Market Research and
Development Manager
© echo | Page 35

36. Case Study: Advocacy
• To review reaction to RSPB advocacy delivery among
Whitehall and Westminster decision makers, with a view
to modifying or adapting its approach, as necessary, to
meet its objectives.
• Echo conducted 28 in-depth interviews with elected and
non-elected government officials, special advisers and
policymakers.
• Interviewees were considered, frank and thoughtful in
their responses thus providing the RSPB with rich
feedback.
• The findings enabled the RSPB to determine the most
effective way of working with these stakeholders in the
future.
© echo | Page 36

37. Case Study: Evaluating Partnership Activities
Integrated study to evaluate 2010
International Year of Biodiversity
Interviews with partner organisations
Media content analysis
Social media analysis
© echo | Page 37

38. Case Study: Event evaluation
• Evaluation of Holocaust
Memorial Day 2011.
• To evaluate the success of
the event and identify key
learnings for 2012.
• Qualitative and
quantitative interviews
with partners and
supporters including
teachers, local
authorities and faith
groups.
© echo | Page 38

39. Evaluation PR & Measurement in Action
Echoes of the future …

40. Guidelines for Success
Educate & involve Management
Link communications to goals & objectives of the organisation
Use measurement & evaluation in plans
(you-can’t-manage-it-if-you-can’t measure-it)
Focus on stakeholders and their influencers/influences
Quantify the financial contribution of your reputation
Set benchmarks – OWN THIS SPACE !
Be proactive
© echo | Page 40

41. Last words ….
“Everything that we do gets measured and my
bonus and remuneration are tied to my “You must challenge
scorecard, to my performance. I don't think constantly. But do it with
you can justify your seat at the top table unless substance.”
you are accountable, unless you can
demonstrate that you are contributing to the
business.”
“I don't think you should resist the notion that communications is a
deliverable and that there should be a constant quest to measure it and
evaluate it in similar terms to which other parts of the organisation are
also subject.”
© echo | Page 41

42. “This is an INSIGHT process. It makes tangible the
intangibles and gives us a precise roadmap of what we need
to focus on; it's about the business understanding this
and owning it. It takes the risk out of marketing
and communications”
Board Director
© echo | Page 42

43. Welcome to the
Journey ! Thank
you.
© echo | Page 43