We shared an evaluation framework that we have developed for assessing the impact of a network strategy for leadership development programs. This is part of a project that LLC is doing in conjunction with network and leadership development experts for the Robert Wood Johnson Foundation.
2. • Network: Over 3,800 members
who fund, run, and study
leadership development
• Collaborative Research: Promoting
cutting edge models & innovation
• Application: Putting new models
into practice to scale leadership
impact on social justice issues
2
About LLC
3. How Can Leadership Programming
Contribute to a Culture of Health?
3
“
“A Culture of Health is a society where getting healthy and staying healthy
are fundamental and guiding social values that help define American
culture. It’s an America in which all the people— whatever their ethnic,
geographic, racial or socioeconomic circumstance happens to be—live
longer, healthier lives. It’s a nation where promoting health is as
important as treating illness. It’s a society in which high-quality health
care is available to everyone—where, when, and how they need it. It’s a
culture that values physical and mental wellness. And it’s a country where
the health of all of our children is a matter of fact and not a matter of
chance.”
4. RWJF New Leadership Programming
RWJF Multi-Sector Leaders for Health will develop, harness, and leverage the
collective power of individuals (and their networks) already in leadership positions
in traditional health and health care-related areas, as well as others engaged in a
broad range of community efforts that may influence a Culture of Health.
4
RWJF Interdisciplinary Culture of Health Research Leaders will engage
researchers from multiple disciplines, such as clinicians, public health
researchers, economists, sociologists, and others in and out of the health sector
to produce evidence that will drive change toward a Culture of Health.
RWJF Diversity in Health Policy Research will support doctoral students from
disadvantaged or underrepresented backgrounds and from different disciplines
to focus on health issues and policy.
RWJF (New) Clinical Scholars will develop an inter professional group of clinicians
to expand their understanding of, and influence on, the Culture of Health.
6. RWJF Project Objectives
6
Develop a comprehensive network
strategy to inform leadership
programming that:
• Informs program design, delivery,
content and network supports
• Incorporates an inclusion and
equity lens
• Offers a learning and evaluation
framework for evaluating network
impact
7. Project Team
7
Milano Harden Beth Kanter June Holley Kiara Nagel
Heather McLeod Grant Claire Reinelt Deborah Meehan
8. Why a Network Strategy?
8
• Building a Culture of Health will require work across silos and
divides to create systems change
• Networks weave relationships and increase the flow of ideas and
resources across sectors and institutions
• Networks increase innovation and accelerate learning
• Networks can scale what works to improve health outcomes and
conditions that create health
9. Culture of Health Ecosystem
9
Health
Non Profits
Media
Faith
Government
Education Arts
10. Chat Question 1
10
Can you think of
examples of networks
that are contributing to
Culture Change?
11. Network Leadership: Leveraging a CoH
• Growing
Alignment &
Action Across
Networks
• Accelerated
Learning &
Innovation Across
Networks
• Network competency &
scaffolding connects &
strengthens networks
• People,
organizations, and
networks,
connecting
Diverse
Connections
Network
Supports
Alignment &
Action
Learning &
Innovation
11
12. A Network Strategy for Leadership Development
12
Program Design Principles
Delivery Strategies
Competencies & Content
Engagement Strategies
Infrastructure
14. Network Capacity
• Network Leadership
Network Mindset/Behaviors
Network Skills
Alignment and Commitment around a
Shared Understanding of a CoH
• Network Strength
Stage of growth
Strong Core and Periphery
Flow of Information
• Network Infrastructure
Communication platforms
Leadership roles & structures
Resources
14
15. Network Action
o Forming cross-sector, cross-issue
partnerships
o Aligning strategic priorities
o Engaging community health
leadership
o Collecting and using data to set
direction and investment priorities
o Innovating and scaling what works
15
16. Societal Impact
o Shifts in public will around
health as a human right
o Policies that promote a CoH
o Improvements in health
outcomes
o Reduction of disparities in
health outcomes
o Changed environments for
healthy living
o Increased access to quality
care for all
16
17. Chat Question 2
17
Are you measuring a
societal level change in your
work?
18. Measuring Network Capacity
18
Resources
• Network Weaver
Handbook (Holley)
• Network Health
Assessment (Taylor and
Coffman)
19. Assessing Network Development
Networks often develop from fragmented clusters of people interested in a concept, to a
single hub-and-spoke when a leader pulls the clusters together, to multiple hubs where a
variety of people take responsibility, and finally to a core/periphery with a strongly connected
core and healthy periphery of people who bring new resources and ideas to the network.
Fragments Single-Hub Multi-Hub Core/Periphery
Patterns of networks development
23. Smart Growth (4/29/11, 184 vertices)
Walkability (4/28/11, 136 vertices)
Complete Streets (4/29/11, 233 vertices)
Healthy Food Access (4/29/11, 173 vertices)
Saveplay (4/29/11, 70 vertices)
Cross: user is bridging between clusters
RWJF: one of the RWJF related accounts
23
#Hashtag Conversations
Source: NodeXL, NetDraw
Date: 6/2/11
on Twitter
24. Measuring Societal Impact
24
Assessment Tools:
• County Health Rankings
• Community Commons
(communitycommons.org)
• Health Impact Assessments
25. #CHRankings Twitter Conversation
Magenta: most influential
accounts (healthyfinder,
gohealthypeople)
Orange: local, county health
Green: health advocacy
Blue: policy
Date: 4/2/11
Source: NodeXL
Number of participants (vertices): 339 25
26. Break Out Group Discussion Question
26
What ideas or questions do
you have about assessing
the contribution of a
network leadership strategy
to changes in health
outcomes?
Hinweis der Redaktion
UNetwork strategies intentionally weave relationships and foster partnerships across sectors, disciplines, and professions so that needed ideas, data, resources, and energy flow more freely; integrated solutions are developed, and local communities and counties learn and share with each other to scale what works to improve health outcomes and the conditions that create health.
Updated by DM 1/27/14
Overview of the process – focus on the evaluation
This is the third of series of conversations
To begin to create this framework
Domains
The first domain where we expect to see change is in network capacity. We are focused here on measuring changes mindsets and skillsets of leadership program participants so that they can be levers and catalysts so that they can support and reach others, in the strength of the networks they form across programs, and in the infrastructure that supports them to use network platforms to communicate, connect, learn, and collaborate
The second domain where we expect to see change is in network actions – Are networks well connected across sectors and issues? Are cross-sector, and cross-issue partnerships forming? Are organizations and networks communicating and working together in ways that amplify their message and create a sense of urgency? Is leadership engaged and connected in every community? Are networks collecting and using data to learn what is working to improve health outcomes and spread what works?
The third domain where we expect to see change is in society. Do we value everyone’s health equally? Are health outcomes improving for all social groups? Have we eliminated racial disparities in health outcomes? Are communities healthier places to live, work and play? Do families have access to good food, transportation, jobs, education? Is there affordable, accessible healthcare in every community?
I explained some of the thinking about this in the slide before. It also maps onto our recommendations that include paying attention to the network infrastructure needed to support graduates.
The first domain where we expect to see change is in network capacity, both in the mindsets and skillsets of leadership program participants, in the strength of the networks they form across programs, and in the infrastructure that supports them to use network platforms to communicate, connect, learn, and collaborate
There is a mix of verbs and nouns. Was trying unsuccessfully to make this more consistent. I feel like we really need to lift up the importance of relationships being formed across sectors, issues, etc. …the unusual suspects who can move the work outside traditional public health channels. Can you give more concrete examples of what we would look for and how we would measure it?
The second domain where we expect to see change is in network actions – Are networks well connected across sectors and issues? Are cross-sector, and cross-issue partnerships forming? Are organizations and networks communicating and working together in ways that amplify their message and create a sense of urgency? Is leadership engaged and connected in every community? Are networks collecting and using data to learn what is working to improve health outcomes and spread what works?
Are we saying these are changes we expect a network strategy to contribute to?... A reiteration of our assumptions or are we going to try and measure changes at this level. In the spirit of the latter I changed some of these to make them feel more measurable. It would be good to have examples.
The third domain where we expect to see change is in society. Do we value everyone’s health equally? Are health outcomes improving for all social groups? Have we eliminated racial disparities? Are communities healthier places to live, work and play? Do families have access to good food, transportation, jobs, education? Is there affordable, accessible healthcare in every community?
Are you measuring a change?
We are moving now to talk about tools and resources for measuring network capacity, network action, and societal impact of network leadership investments
There are a number of tools being developed to measure network leadership capacity.
For instance June Holley has developed a network weaver checklist which can be used by individuals to understand their strengths and weaknesses in various network roles. This tool can also be used to assess the network’s leadership capacity. Does the network have people who are actively engaged as weavers, connectors, facilitators, guardians – all the roles that are needed for the network to function at optimal capacity.
The Network Health assessment is a tool to assess network health – for example does the network have the resources it needs to sustain itself, are the internal systems and structures efficient, does the network have a common purpose and are they adding value to each other’s work
From “what Network Stage is Your Community?” Valdis Krebs and June Holly, Wiser Earth, September 2009
Another way to assess the network’s capacity is to map how the network is changing over time.
Identification of partners within the collaboration.
Record of the frequency of interactions.
Elements of the strength and quality of the interactions.
Measures of trust and value within the collaboration.
Network scores to report and illustrate changes to collaboration activity over time.
Outcomes measures related to success of the collaborative, products and programs developed, and change to relationships among members of the collaborative are provided.
RWJF project
You can understand network action by collecting and anlayzing twitter hashtag conversations.
There are many action-oriented health health tags that people are using to promote actions and conversations around an issue. Sometimes you can see high levels of synergy and activity around multiple hashtags like smart growth, walkability, complete streets. Can also locate who is bridging clusters
County Health Rankings
Community Commons is a mapping tool that identifies vulnerable communities that are at high risk for poor health outcomes based on poverty rate and high school graduation rates – two outcomes that are predictors of health and for which there is good data being collected at a neighborhood level
Health impact assessments (HIAs) are being used to advance equevidence-based analyses
that predict health benefits and risks of proposed laws,
regulations, programs, and projects
Health Impact Assessment (HIA) is an important tool for understanding the health implications of proposed policies, plans, or projects on communities. Policy Link and others have been focusing on bringing an equity lens to these assessments so that they can be useful in advancing equity in decision-making
Conversation with Marc 5/26 [incorporated these notes into slide on 5/27]
2011 conversation was a bit denser than 2010 conversation but foundation has not been able to recruit a spokesperson that is not them. If we do this map next year and there is another hub that’s not RWJF hub then that’s a good outcome. You could have a communications strategy that would actively recruit different hubs
+++++
Map created on 5/24/11. Discussion with Marc:
Hubs: healthfinder (magenta), gohealthypeople (magenta), amheartadvocacy (lime), rissalavizzo (orange)
Of the top 10 people talking about healthrankings, 6 of them work for RWJF. (see top users below)
More people than in 2010 (339 on this day and 183 in 2010, difference of 156 participants)
Over the year RWJF accounts have gained in followers in proportion to their followed but the three clusters remain the same, clusters have become a bit bigger, not that much more dense
+++
New Cluster Notes (Natalia 5/24/11)
Magenta (g4): mindofandre, chimoose, femelmed, healthyfinder, gohealthypeople , susannahfox, reportinghealth, cascadia
Orange (g2): nacchoalerts, chrankings, brenlizhen (Working to leave a positive legacy in my little corner of the world. Program Officer @ Robert Wood Johnson Foundation.), ncuih_ed (NCUIH is a nonprofit organization in Washington, D.C. that promotes quality health services for American Indians in the U.S.)
Schweitzerasf: Addressing health disparities by developing Leaders in Service who are skilled in, and committed to, meeting the health needs of underserved communities.
Green (g3): amheartadvocacy, misshealth (Interests: Public health, grassroots advocacy, youth empowerment, new media, activism, hunger, food revolution, tobacco control, fighting cancer!), pophealth (Improving Population Health is a blog that explores current thinking in policy, practice, and research in population health improvement. )
Blue (g1): healthpolicyhub, kathleenld (Likes: international affairs, WashDC, Global Pub Health, new media, the internet, cooking. Dislikes: mean ppl, pigeons, frisbees ), orleanscohealth (Empower county residents to achieve optimal health, wellness & safety,both individually & as a community, through innovative leadership, advocacy & education. ), capecodgurl (Politically motivated #obamaCRAT..underdog rooter..news junkie..aspiring hermit..loves Red Sox, OuterCape, community activism, nonfiction & documentaries &dogs. )
Cyan: howardhealth (Howard Health Counts is a one stop source of non-biased data & info. about community health in Howard County, MD. It is sponsored by The Horizon Foundation.)
++++
Top 10 Users [Natalia 5/25]
rwjf_pubhealth: magenta
rwjf_humancap: magenta
Healthfinder: magenta
Gohealthypeople: magenta
Rwjf: lime
Amheartadvocacy: lime
american_heart: lime
chrankings: orange
Risalavizzo: orange
Healthyamerica1: orange
++++
Workbook Metrics [we did not discuss this with Marc, this is just FYI]
Graph Type Directed
Vertices 339
Unique Edges 4436
Edges With Duplicates 544
Total Edges 4980
Self-Loops 0
Connected Components 3
Single-Vertex Connected Components 1
Maximum Vertices in a Connected Component 336
Maximum Edges in a Connected Component 4978
Maximum Geodesic Distance (Diameter) 4
Average Geodesic Distance 2.191143
Graph Density 0.041044841
NodeXL Version 1.0.1.164