1. Result-based communications:
Some examples taken from EU
public policy communication
George Ellis Ruano, Director
Sophie Mestchersky, Consultant
Lugano, 10 February 2009
2. ● The CHF10 question
How do you know if your
communication activities
are effective?
www.gellis.com
4. ● Measurement & evaluation: what’s in a word?
We measure an input, an output,
but…
We evaluate a result, an outcome, an impact.
www.gellis.com
5. ● A CHF100 question
How do you know if your communication activities
can be evaluated?
www.gellis.com
6. ● Requirement 1: Easy access to data and information
Example of impact - Vote, access to public services, etc.
Example of outcome - Level of knowledge about the policy
being promoted amongst target groups;
- Level of endorsement.
Example of result - Visits, downloads, comments left by
visitors
Example of output - A website (number of pages, number of
languages available, etc.)
Example of input - A webmaster, a budget line
www.gellis.com
7. ● Requirement 2: A coherent and logical presentation
www.gellis.com
8. ● Requirement 3: SMART objectives
• Example: a 15% increase of subscriptions to your
newsletter by the end of the programme.
• Example: Each year until the end of the
programme, secure at least two additional
sponsors for the annual conference
• Example: 50% awareness of your activities
amongst Decision Makers
www.gellis.com
9. ● Example (1/5)
1. Measuring inputs
• Evolution of the dedicated budget Y0 / Y-1 / Y-2, etc.
• Comparison with the budget of a similar organisation
www.gellis.com
10. ● Example (2/5)
2. Measuring outputs
Checklist:
Number of website updates;
Number of events and conferences;
Improvement of the corporate identity;
Number and quality of contacts made with
potential partners;
www.gellis.com
11. ● Example (3/5)
3. Evaluating results
• Quantitative results: Number of visits, number of
downloads, number of comments, average duration
of a visit, etc.
• Qualitative results: nature of comments, visitors’
patterns, preferred language version, etc.
www.gellis.com
12. ● Example (4/5)
4. Evaluating outcomes
• Quantitative outcomes: share of the target group
that is aware of the service being promoted, etc.
• Qualitative outcomes: target group’s opinion, etc.
www.gellis.com
13. ● Example: (5/5)
5. Evaluating impact
• Evolution of the market share, of citizens’ support for
a specific policy, etc.
www.gellis.com
14. ● Key success factors (1/3)
1. A broad skill set
• Cutting across different disciplines
– Communication
– Political science
– Organisational behaviour
– Sociology
– Psychology, etc.
www.gellis.com
16. ● Key success factors (3/3)
3. A participatory approach
• To get new ideas
• To facilitate data collection
• To distribute tasks fairly
• To secure commitment
• Etc.
www.gellis.com
17. ● Case studies (1/3)
Client: URBACT
Issue: Evaluation of the existing communications strategy
and design of a seven-year communication plan
Results:
−Evaluation of the URBACT website (including usability study, communication value study
and online user survey)
−Evaluation of the 2002-2006 communication strategy (desk research, stakeholder
interviews, benchmarking exercise, SWOT analysis)
−Design of the 2007-2013 communication plan (Identification of needs and opportunities,
formulation of SMART - Specific, Measurable, Attainable, Research based, Time-bound -
communication objectives, identification of target audiences, definition of strategy and
dissemination tactics, guidance on organisational structure, resources and appropriate
timing, drawing up of a 2008 action plan)
www.gellis.com
18. ● Case studies (2/3)
Client: European Commission, DG ENV
Issue: Market analysis in view of developing a new approach
for the ‘Environment for Young Europe’ website
Results:
−Review of the website, assessment of its continued relevance and identification of its
strengths and weaknesses;
−Comprehensive market analysis through surveys conducted in 10 EU Member States
with a view to assemble the teachers’ and students’ ideas on how the European
Commission could make better use of the Internet to communicate environmental issues
to young audiences;
−Identification of the appropriate target groups for the website and design of a
strategy to turn them into multipliers;
−Strategic recommendations about the optimal structure, content, features and visual
and verbal identity for the development of a new website.
www.gellis.com
19. ● Case studies (3/3)
Client: European Commission, DG Employment
Issue: Scoping study for communication and networking (CN)
activities under the PROGRESS programme
Results:
−Definition of new communication objectives;
−New Target Group Classification based on policy cycle;
−Need for more targeted messages;
−Need for a more ‘national approach’;
−Need to focus on most effective communication tools and channels;
−Need to rethink organisation and systems;
−Need for a Monitoring and Evaluation Plan
www.gellis.com
20. ● Gellis’ key assets
• A multi-national, multi-disciplinary team of 13 highly
qualified professionals;
• Value-adding partnerships;
• A roster of experts in a wide range of policy areas;
• Sound understanding of the policy-making process;
• Professional memberships;
• Strong project management skills;
• Proximity to EU institutions;
• Stability due to long-term contracts with EC
www.gellis.com
21. Thank you!
George Ellis Ruano, Director
george.ellis@gellis.com
Sophie Mestchersky, Consultant
sophie.mestchersky@gellis.com
www.gellis.com
