Network Analysis (SNA/ONA) Methods for Assessment & Measurement
•Als PPTX, PDF herunterladen•
6 gefällt mir•3,001 views
Many nonprofits and foundations have been using social network analysis (SNA) and organizational network analysis (ONA) techniques in program assessment, planning, and measurement. This webinar will review a number of techniques that are being used and the ways that the results of network analysis are informing and supporting the ways that nonprofits are leveraging networks to achieve greater good by creating, facilitating, and weaving networks.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Ähnlich wie Network Analysis (SNA/ONA) Methods for Assessment & Measurement
Ähnlich wie Network Analysis (SNA/ONA) Methods for Assessment & Measurement (20)
Mehr von Leadership Learning Community
Mehr von Leadership Learning Community (20)
Network Analysis (SNA/ONA) Methods for Assessment & Measurement
- 1. Using Network Analysis to Assess and Measure Networks January 14, 2012 Patti Anklam With June Holley and Claire Reinelt
- 2. Webinar Goals Share current thinking about how network analysis is used in designing and evaluating nonprofit programs Provide examples of network analysis used in assessment and measurement contexts Stimulate thinking about correlating network analysis with measurement and evaluation outputs and outcomes 2
- 3. Network Thinking & Non-Profits
- 4. The Evolution of Network Thinking 4
- 5. What is Network Analysis? • Social network analysis (SNA) is a collection of techniques, tools, and methods to map and measure the relationships among people and organizations • Organizational network analysis (ONA) often refers to the use of SNA methods in the context of organization dynamics and development • In practice, we use these tools to map connections among people and ideas, issues, and other entities as well as the social and organizational connections 5
- 6. Network Analysis: The Method in a Nutshell Step Activities/Tools Design Identify boundaries Clarify and design questions Collect Data Surveys Interviews Facebook, LinkedIn Email logs Analyze data to generate (Netdraw/UCINET, NodeXL, Gephi … maps and metrics many others) Review data Validate; look for questions Prepare evaluation Match network results with context and stories Move into action Weaving & other interventions 6
- 7. Survey Example 7
- 8. Survey Example – Demographic Component 8
- 9. Survey Example – Affiliation Component 9
- 10. Survey Example – Network Questions 10
- 11. Network Questions Probe Relationships 11
- 12. Analyses Outputs: Map Patterns Multi-Hub Hub and Spoke Stove-piped (Siloed) Core/Periphery 12
- 13. Quick View: What an Analysis Can Tell • Overall very well connected • One region distinctly clustered with few connects to other regions • Staff are highly central • Identification of key connectors 13
- 14. Reasons for a Network Analysis: Examples 1. Assessment, Planning, & Weaving 2. Measure changes over time 3. Sense-making & story- finding 4. Positioning and working with individuals in the network 14
- 15. Assessment, Planning, & Weaving Strategic Purpose • Assess the network’s capacity for collaboration, information transfer, innovation • Identify key individuals • Establish goals for enhancing connectivity • Create an action plan 15
- 16. Assessment: Capacity for Collaboration Current Funder Interaction Network Future Funder Interaction Network When funders indicate with whom they would like to work in the near future, the network becomes more robust. Funders are saying they want to work more together. Source: Transcending Boundaries: Strengthening Impact. The Full Potential of a Justice Network (Research & Network-Building Project Report, April 2011, Criminal Justice Funders Network). Courtesy of June Holley. 16
- 17. Assessment: Affiliation Network Strategic Purpose • Identify potential relationships among people based on shared events, meetings, ideas, or areas of expertise • Nonprofits use this to see which organizations “attach” to different ideas • Forms the basis for network weaving 17
- 18. Drill Down Into Affiliation Network • Identify people with common interest – basis for building communities of practice • See which people share interest in multiple issues or topics • A way for the network to reveal itself and have rich conversations 18
- 19. Measuring Changes Over Time Very Well Boston Green & Healthy Building Network Well Somewhat 2005 2007 24 9 32 1 24 9 32 1 31 3 31 18 7 3 18 6 7 6 25 25 33 28 4 33 28 4 26 26 8 29 17 8 29 2 17 2 11 12 11 12 23 16 27 23 16 27 10 19 10 19 21 21 15 15 13 13 34 14 34 14 30 30 5 20 22 5 20 22 Source: Boston Green & Healthy Building Network, Beth Tener and Al Nierenberg, January 2008 Maps copyright © 2012 New Directions Collaborative 19
- 20. Analyses Outputs: Metrics Overall network metrics Individual position metrics • Look at the whole network • Look at positions of and its components: individuals in the network: – Overall cohesion – # of connections – Degrees of separation – Favorability of position • Good for comparing • Good for identifying groups within networks or people who are well for comparing changes in a positioned to influence the network over time network or to move information around 20
- 21. How the Metrics Enhance the Maps 2011 Year # Density Avg # ties 2009 55 2.2% 1.2 2010 90 2.7% 2.4 2011 85 5.3% 4.5 2012 82 8% 6.88 2010 2009 2012 21
- 22. Sense-Making & Emergence • Barr Foundation Fellows Program – See changes over time, but really to see how the network has supported emergence – Work to shift Barr staff from the center Pat Brandes Source: Networking a City, Marianne Hughes & Didi Goldenhar, Stanford Social Innovation Review, Summer 2012 22
- 23. Sense-Making: New School Development in Boston • An intentional network may “This person has helped me accomplish work-related tasks.” have no other purpose than to enable emergence • Maps that show the evolving relationships within a network help to identify powerful network stories Source: Networking a City, Stanford Social Innovation Review, Summer 2012 23
- 24. Positioning: The Individual View Node Betweenness Indegree OutDegree 62 792.67 26 30 80 660.48 17 32 • Centrality metrics 64 23 530.61 333.36 20 20 33 14 identify people with 71 321.42 21 20 56 316.42 20 18 the most ties (in- degree and out- degree) • Those positioned to move information around in the network or be in the know (betweenness) • Can identify people to lead task teams, to provide resources to, or to train as weavers 24
- 25. Tracking Individuals’ Changes I learned something from this person that made me a better leader. – 2009 2005 2007 2008 2009 2010 25
- 26. Tracking Individuals’ Changes I learned something from this person that made me a better leader. – 2011 26
- 27. Network Analysis & Measuring Outcomes
- 28. Summary – What We Know What We Can Measure and Show in an Analysis: • Measure the cohesion of the network overall: – High-level structure (stove-piped, core/periphery, highly clustered) – Average degree of separation – Average number of connections each person has • Identify individuals by their centrality to the network: – Core or periphery? How do you bring people in from the outside? – Broker? Connector? Facilitator? Bottleneck? – Number and diversity of connections • See changes over time 28
- 29. Things We Can Do With What We Know Ways to change patterns in Practices from the KM/OD Repertoire networks Weaving. Create intentional Convene. Make introductions through meetings and webinars, face-to- connections face events Increase the flow of knowledge Establish collaborative workspaces, install instant messaging systems, make existing knowledge bases more accessible and usable; implement social software or social network software Create awareness Provide expertise directories Connect disconnected clusters Weave: establish knowledge brokering roles; expand communication channels Create more trusted relationships Assign people to work on projects together Alter the behavior of individual nodes Create awareness of the impact of an individual’s place in a network; foster network literacy Increase diversity Add nodes; connect and create networks; encourage people to bring knowledge in from their networks in the world 29
- 30. Measurement Challenges • Maps area snapshot in time • Targets and thresholds – How much cohesion is “enough?” Is there a point at which increasing the number of ties makes the network less efficient? – Is it reasonable to set a target for the cohesion metric? • Tying Network Metrics to Outcomes – We have to think of the metrics as indicators and as correlates of Source: Dave Snowden, Cynefin Advanced Practitioner’s Course December 2012 other survey questions 30
- 31. Questions? 31
- 32. • patti@pattianklam.com http://www.pattianklam.com •claire@leadershiplearning.org http://www.leadershiplearning.org/ •June@networkweaving.com http://www.networkweaving.com Thank you. Question
Hinweis der Redaktion
- The last ten years have seen the emergence of network thinking and network analysis into the nonprofit world. LLC, the Packard foundation, and many individuals have built on the work that June did with Valdis Krebs in 2002. We’[ve been inspired by case studies like the RE-AMP case and have had the power of social media laid bare by Beth Kanter’s tireless teaching and evangelism.
- Check on the poll at this point…
- Here’s a quick example of a survey. I use ONAsurveys, a software program developed by colleagues at Optimice in Australia, specifically for network analysis. You can also use other survey software like SurveyMonkey, but the data format that comes out of Survey Monkey is not as easy to work with. June also has network survey software.The important things to capture in the survey are the demographic information, affiliation information, and relationship information.
- The demographic component is really important, as it helps to show things that divide people or might put up barriers to collaboration. This just shows a geographic question.
- We like to try to capture information about topics or issues that are particularly important to people in the network we are looking at. This is a fragment from a survey of the Network Weavers Facebook group that June and I have been working on with Ken Vance-Borland.
- Last, we ask relationship information. In this case, we just start by asking how people might know each other… how they have interacted. Responses to this question might give us an idea of whether this FB group has attracted a diverse set of people who could be more explicitly woven into a network. The topical information would help with that as well.
- What other relationship questions might you ask? In the programs we have worked with, some of the key questions we ask to understand not just how cohesive the network is, but how people are relating to each other. The more questions we ask at the same time, the more perspectives we might have on the network.
- When survey data is analyzed, the first views are the maps that show the connections and we can quickly see specific patterns. The patterns that come out of looking at the different questions or different values of response to a question, help us understand where to zero in and look for more context.
- My primary mantra in working with SNA is that a map doesn’t represent truth – it’s a snapshot that shows patterns that help us ask good questions.This is from a recent project that I did for a foundation that has funded over 100 grantees in a particular field and has been working on building the network among these grantees for over four years. They had not done an analysis previously. The color coding indicates geographic regions. The yellow dots are the foundation staff who work with the grantees. The size of the dots is a network measure of “betweenness” which is an indicator of how well positioned an individual is to move information around in the network. It’s easy to see quickly that there is one geographic area that is highly connected within itself but that is not integrated with the rest of the network. What will the foundation do? They are not planning to charge ahead and try to get these folks integrated. First, they need to think about what the value of that would be and whether it’s worth investing more effort in weaving. But the map was an good heads up.They also became very aware of the extent to which they, as program staff, were holding the network together. They will take steps to understand how to ensure that the network might survive without the focused attention of the foundation.Last, they saw that one of the key connectors was a person who is soon to retire and are taking steps to capture his expert knowledge and work to make sure that the connections are preserved in the network.
- Now, we’re going to walk through a set of examples that illustrate these ways that network analysis is being used – at the outset of network building in the assessment phase, as well as how network analysis can be used through the life cycle of a program to measure overall network changes over time, the positioning and movement of individuals, and the ways that network analysis helps make sense of networks and point to evidence for evaluation.
- Those of you familiar with June Holley’s work and that of other network builders know that the groundwork is really important. Understanding the capacity of the network for collaboration as well as the individual needs of members of the network. I’m going to let June tell you the story of how a network analysis provided input for designing a network approach in the criminal justice funders network.
- June’s script here.
- Thanks, June. In addition to looking at individual and people connections and potential, we can also map the network of what might make people want to connect – the very basis for weaving a network. In this case, we can ask people to identify the topics of interest or major concern and draw the map connecting people to topics. We show people as circles and the topics as squares. (Note you can do this in a spreadsheet or table, but sometimes things pop out more clearly in the map. Or people just like visuals!)
- So one thing you can do with these affliation networks (just as you can with any network map) is to drill down and see, for example, what people might be interested in two related topics or people who have cross-over expertise. Here we can see a few people who are interested in the really tightly connected topics of collaboration platforms and change management. Anyone who has ever tried to get a bunch of people to change how they work by introducing collaboration platforms knows what a change management challenge it is!
- So let’s shift into thinking about how to use network analysis when you are working with a network over time. At the outset, you may have a map (like the criminal justice map that June talked about) showing current connections. And you have an affiliation matrix that helps you to know which people might “do lunch” and this gives you the basis for weaving.So here’s an example from my colleague Beth Tener, who has worked with a number of nonprofit networks in New England. This network building project was funded by the Barr Foundation who brought together government agencies with nonprofits working on environmental issues and those working in health areas to coordinate their work to develop a new set of building codes for the city of Boston.A network analysis of those brought into the network shows, on the left, the connections as they existed when the network was first convened. Lines of different thicknesses show whether people know each other very well, well, or somewhat. Through the work of Beth’s team, the network grew more cohesive as they worked together and the result is what you see on the right. Just as important to this map, however, is the survey result that Two years after the network began, 95 percent of participants agreed that the network had helped their organization advance their mission.
- In addition to the network maps that can show us so much about connectivity, network analysis also produces metrics – quantitative information that reveals aspects of the network that might not be easy to see in the map, or that support what the maps show. The overall network metrics can tell us about the structure of the network – how well connected it is overall. This number is good for looking at changes over time. Individual position metrics show who the people who are most connected, either because of the sheer number of connections that they have or because they are connected to well connected people…. I have an example of individual position metrics later, but first let’s take a quick look at how the quantitative network metrics can enhance presentation of the maps.
- I depend on this person regularly for important advice and have worked with him/her on more than one projectThe Urban Sustainability Directors Network has been mapping its members since 2009. Here we see the sequence of the maps from this time. This network asks people to describe the extent of their relationships which can be at the minimal level of having been introduced to the highly connected level of working together. Here, percentages of connection are shown. Density is the overall cohesion level, that is the number of ties that exist out of a total possible. The average number of ties is what you might expect – it indicates on average how many people are connecting in the network.But metrics do not tell the whole story and can never be the whole purpose of doing an SNA. What we really need you to take away from this is that the maps and the metrics should be taken as indicators – clues of where to look for stories or to provide evidence of the success of network building.Two other points to make about metrics. We don’t know what a “good” number is. We tell by comparison year over year whether things are improving, but we don’t know when to expect the metrics to level off – when is a network saturated to the point where people are getting what they need, they have enough connections, and so on. At this point we might want to start to look at whether people change their connections --- spend more time with new people, less time with people they know. There are a lot of interesting questions in the area of metrics. [permission from Julia Parzen below]Hi, Patti, I did respond. I am sorry you did not receive a response. I would be happy to have you use this slide identifying it as USDN if you take out the names on the maps. I can't take any risk of having the names on the maps. I would be happy also to talk to you about my experience and to learn from you about your experiences. All the Best, Julia Julia ParzenCoordinatorwww.usdn.org773-288-3596773-315-7427 (Cell)
- For example, the Barr Foundation Fellows program has been doing SNA since its first leadership cohort in 2005. One goal of this program was to create a cohesive network of nonprofit Executive Directors in the Boston area. At the outset of the program, the Barr program director Pat Brandes was the connector, but by 2011 when the most recent survey was completed, we could see that the role of the foundation in maintaining the network was diminished.This is one example of a story that network maps can tell. Claire Reinelt, who has been working with the Barr Foundation Fellows program for a number of years, is going to show how she uses the maps in conjunction with telling stories about the network in her evaluation work.
- Claire’s script: [Important to note here the importance of other, qualitative assessment & evaluation.]
- Now let’s turn to another way of looking at individuals. The individual metrics we can look at in a map that are associated with people are those that can be counted or tracked. Indegree is the number of people who point to a given individual – perhaps in the case of “receive information from this person”. The Out-degree is the number of people that a person points to. There are dozens of metrics that calculate a person’s position in the network. One that is the most often used is the “betweenness” value, that is the extent to which a person is on paths across the network – how many people is somebody “between”. In this example, we’ve made the people nodes sized according to the betweenness value – see #62 and # 80. They also have high in-degree and out-degree values, but not the highest outdegree.We use these maps an analysis to understand what people might be most useful to enlist in helping build the network, work on task teams, or otherwise support weaving in one way or another. This is another kind of drilling-down we use in assessing and working to sustain a network.Individual analysis can also be helpful in looking at how an individual moves in the network over time. The last network analysis example shows how one foundation is using maps this way.
- Here’s an example from a leadership development program that has been doing network analysis since 2009. The leadership program is focused on helping individuals become more effective and a foundational technique is to bring the leaders together so they can learn from each other. One of the key questions that is asked in this network is, has your connection with this person made you a better leader? The nodes here are colored by cohort years. Here one individual is circled.
- Looking at the same individual two years later, you can see that this person has become more embedded in the network and has an increase in incoming ties as well as outgoing ties.2009 (in: 1, out 5) 2011 (in: 7 out 10)The fund believes that this change in network position is a direct result of an particularretreat . Validation of the importance of the face to face convenings in a retreat setting, and that they should continue to do them. Grantee confirmed this in her reports and she also went on to initiate meetings with other fellows.In another example, a fellow who was not a strong person in-person convenings also made a number of connections. Because he didn’t participate in the convenings, they thought he was peripheral. The mapping process showed that this person was more connected than they thought.Community reporting back helps in the decision-making, much more concrete than the individual perspectives of the fund managers.
- I realize that this a lot to digest and I hope you’ll take time to review the slides at some point and see what things you might see in the maps here. The important thing to remember is that a good analysis leads to really good questions – more places to probe, stories to find. And that brings me to some things I’d like to put out to the community for future conversations and work.
- In dealing with data for a network analysis, the danger is in thinking that we have something that gives us measures that count. Cohesion, centrality of individuals, their positions in the network and how they change over time.
- We use what we learn from probing in the maps and metrics to create a plan for building the network, for weaving. There are not a lot of new tricks out there – people like me who have worked in the knowledge management space, people who have worked with communities of practice, organizational development people – have all acquired a repertoire of ways to bring people together, to convene networks, provide collaborative spaces, and so on. I offer this list as a starting point. There are no answers. There is no “being right” about doing this. But the main question remains: does network analysis give us evidence to point to outcomes in our programs?
- Just to add a cautionary note about metrics. They are very appealing but we do have to use them within reason and, as I have suggested, as indicators. It would be good to start to collect and maintain a database of metrics that would describe networks of various sizes and types, by goals and desired outcomes, and compare the metrics over time. I do think that metric comparisons can help us learn how to accelerate network growth.But we do need to be careful that our measures don’t become the targets; as you see in these quotes, it would be dangerous to focus solely on getting numbers. Network building is about relationships and the focus of the work – the goal of network building – should always be on the outcomes.I think there is a lot of interesting work ahead to better understand how to tie network analyses to outcomes. For now we can do very well and be quite content using the metrics and maps as indicators of where to look for meaningful stories that support designing and evaluating networks.