By: Donna Jean Forster-Gill, Tamarack-An Institute for Community Engagement

1. Vibrant Communities Canada:
Measuring Impact
Donna Jean Forster-Gill
Tamarack-An Institute for Community Engagement

2. Why Vibrant Communities?

3. The Impetus

4. The Complex Nature of Poverty
“Poverty is a complex issue. There is no single
cause and no one solution. Its successful
reduction, and ideally its eradication, require a
set of linked interventions undertaken by all
orders of government working in collaboration
with communities.”
Poverty Policy
Sherri Torjman,
Caledon Institute of Social Policy
October 2008

5. What is
Vibrant Communities?

6. A Pan-Canadian initiative exploring comprehensive,
multi-sector approaches to poverty reduction
Launched in 2002 by three national partners
• Tamarack: An Institute for Community Engagement
• The Caledon Institute of Social Policy
• The J.W. McConnell Family Foundation

7. Vibrant Communities
An experiment designed to test a specific
way to address the complex realities of
poverty through local level action.
Theory of Change:
Guided by 5 principles & assisted by extra
supports provided by national sponsors –
local organizations and leaders could
revitalize poverty reduction efforts in their
communities and generate significantly
improved outcomes.

8. We believed that using a comprehensive,
multi-sector approach communities can …
• Raise the local and national
profile of poverty
• Build a constituency for
change
• Encourage collaborative
ways of working
• Begin to shift the systems
underlying poverty
• Generate substantial
changes for a large number
of people living in poverty.

9. Who are the
Vibrant Communities?

10. 13 Vibrant Communities
Part One – Exploring Principles
+ Niagara Region

11. How does
Vibrant Communities work?

12. What are the 5 components of
Vibrant Communities?

13. Multi-Sector Collaboration
Business
Government
• Exper(se,
credibility
and
• Exper(se,
connec(ons
to
voice,
connec(ons,
funding
elected
officials,
funding
and
other
resources,
and
other
resources,
policy
leadership
change,
leadership
Nonprofit
Organiza2ons
Ci2zens
with
Lived
Experience
• Exper(se,
experience
on
• Exper(se
about
the
issues,
the
ground,
service
delivery,
prac(cal
and
relevant
ability
to
ramp
up
change
solu(on,
leadership,
connec(ons
to
other
efforts
ci(zens

14. Comprehensive Thinking & Action
address the interrelated root causes of
poverty rather than its various symptoms
Create
Community
Knowledge

15. Community Asset Building
building on community strengths
rather than focusing on deficits
Develop a Community Aspiration

16. Community Learning and Change
embracing a long-term process of learning and
change rather than simply undertaking a series
of specific interventions
Create a
Framework
for Change

17. Poverty Reduction
a focus on reducing poverty as opposed to
alleviating the hardships of living in poverty
Move towards Systemic Change

18. Impact of
Vibrant Communities?

19. Collective Impact
• Common agenda
• Shared measurement
systems
• Mutually reinforcing
activities
• Continuous
communication
• Backbone support
organization
– John Kania and Mark
Kramer, Winter 2011

21. Measuring the impact of
Vibrant Communities –
Shared Measurement

22. Original Approach
• Logic Models, Outcomes and Outcome
Tracking. (Traditional Evaluation methods)
• Used for first 2-3 years of project
SPEED BUMP:
• Approach did not fit with what was
happening on the local level
• Shifted to developmental evaluation
approach

23. Developmental Evaluation in Vibrant Communities
Stream One –each local community to articulate the ‘theory of change’
guiding its work and to reflect annually on how those ideas were playing
out in practice.
Stream Two –communities to prepare brief stories describing their
specific poverty reduction strategies: the challenge addressed, the
strategy employed and the outcomes anticipated. These stories helped
guide their outcome tracking efforts.
Stream Three –communities complete semi-annual statistical reports
focussed on two main targets: number of partners participating in their
work and number of low-income households benefitting from their
efforts. It also asked for an annual narrative report elaborating on the
overall development of each community’s initiative, including its efforts to
build community capacity and impact wider systems that contribute to
poverty.

24. What was captured:
• Stories and Lessons Learned
• Expected and unexpected outcomes
• Most significant change
• Statistical Reporting and Narrative reporting
focused on 5 types of outcomes initially.
• Used sustainable livelihoods model with 5
outcomes areas, 13 indicators and many sub-
indicators.
• After 5 years, this was revised to add outcomes
around community capacity building and policy
and systems change

25. Indicators of Community
Change
Policy and • Changes in public policy
• Changes in service and support systems
Systems • Changes in material resources
Change • Changes in community-level assets
Community • Convening capacity
• Multisectoral leadership
Capacity • Collaboration
Building • Community awareness
Individual and • Personal assets
• Physical assets
Household • Social assets
• Human assets
Assets • Financial assets

26. Inner resources Basic material
goods & services
• Self-awareness
• Self-esteem and self- • Emergency supports
confidence • Food, Housing
• Hope and motivation • Transportation
• Dependent care
Income, Savings Financial Social Relationships
• Employment income Assets Assets and Networks
• Non-employment
• Civic participation
income Human • Support networks
• Savings and financial Assets
assets
• Reduced debt/costs
Skills, knowledge,
education & health
• Health, Life skills
• Financial literacy
• Education
• Employment Skills

27. Why Developmental Evaluation
Worked for VC
At the local level: required reflecting on the
theory of change and upgrading it as required
to better achieve desired outcomes, rapid
response to a changing environment, and
ability to capture the emerging insights and
questions of participants.
At the national level: it is about mining the on-
the-ground experience of communities for
patterns and themes that helped us understand
the value of this approach to reducing poverty.

28. Challenges in measuring
the impact of the
Vibrant Communities
Initiative

29. Overall Challenges:
• The sheer scope of the work: Lots of time and energy
required
• ‘attribution issues’: whose efforts are responsible for
outcomes achieved or roles that different partners play.
• The guiding ideas are often left implicit making it hard to
assess their validity.
• Evaluation processes need to be highly flexible due to
emergent nature of the process.
• Outcomes are integral, but are often difficult to define and
measure.
• The long-term focus must be reconciled with the need to
track progress in the near- and mid-term.
• appropriate adjustments must be made along the way

30. Most significant challenge:
The evaluation needed to accommodate
substantial differences across the sites while
continuing to meet collaborative objectives.
National Level:
• Managing the volume of data
• Getting materials submitted on time
• Lack of evaluation advisory group to guide the
work
• More collaboration needed between evaluators
and local level
• Takes lots of time and concentrated effort

31. Local Level Challenges:
• Keeping up with evaluation demands
• Balance between hard numbers and story
• National Level questions did not always work
well at the community level
• Staff turnover, inadequate evaluation training
for the communities
• Often off the side of the desk rather than
designated staff person
• Takes lots of time and concentrated effort
• Not single organization but collaborations

32. What have we
learned?

33. How much more information do we need
to know that a hungry child will not do
well in school? Stop admiring the
problem and get on with the work.
Mark Chamberlain
CEO, Trivaris

34. Evaluating a comprehensive, multi-sector
approach to poverty reduction is …
1. Hard work, messy and
time consuming
2. Fluid, flexible and
requires adapting to
constantly changing
conditions
3. Requires outside eyes
to hold the evaluation
pieces.
4. Orgs can benefit from
training in evaluation
5. Everyone has to
develop the evaluation
framework

35. Common Success Factors
• Influential and credible convener(s)
• Cross-sector, connected leadership table
• Challenging community aspiration
• Clearly articulated purpose and approach
• High degree of resident mobilization
• Research which informs the work and
captures shared impact

36. Reporting to the
Community –
Continuous
Communication

37. Vibrant Communities (2002 - 2010) Evaluation Report
Reflecting on Vibrant Communities: 2002-2006
Understanding the Potential & Practice of Comprehensive,
Multi-sector Efforts to Reduce Poverty - The Preliminary
Experiences of the Vibrant Communities Trail Builders
To learn about the background of collecting the VC by
the Numbers reports:
http://tamarackcommunity.ca/downloads/vc/
VC_By_the_Numbers_FAQs_032511.pdf
In From the Field - Exploring the First Poverty
Reduction Strategies Undertaken by Trail Builders in the
Vibrant Communities Initiative

38. Where are we headed?
The Landscape has Changed…since 13 cities
began to experiment…
• Municipally: 84 collaborative poverty reduction roundtables have
connected to Vibrant Communities
• Provincially: 11 provinces and territories have or are developing
poverty reduction strategies
• Federally: A new all-party Roundtable has been formed to focus on
poverty, the Government of Canada – HUMA committee, Senate
Roundtable on Cities and Federation of Canadian Municipalities
have identified poverty as a critical issue

39. Our Aspiration:
Imagine…100 cities reducing poverty
TOGETHER