Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie BBVT_Results Based Accountability_Presentation
Ähnlich wie BBVT_Results Based Accountability_Presentation (20)
Mehr von Lauren-Glenn Davitian
Mehr von Lauren-Glenn Davitian (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
BBVT_Results Based Accountability_Presentation
- 1. Results-Based Accountability Vermont Nonprofit Conference March 20, 2012
- 2. Goals for Today (aka How I hope you will better off) Basic understanding of Results-Based Accountability (RBA) •Why bother? •What is it? •How it can help you plan and manage and maximize •How it can help you communicate •Where to get more information & resources Experience working with Performance Measures Be ready to take next steps Benchmarks for a Better VT March 2012 2
- 3. Sources & Resources Results-Based AccountabilityTM (RBA) Trying Hard Is Not Good Enough: How to Produce Measurable Improvements for Customers and Communities Mark Friedman (2005) Results and Performance Accountability, Decision- Making, and Budgeting Fiscal Policy Studies Institute Santa Fe, New Mexico www.resultsaccountability.com www.raguide.org Benchmarks for a Better VT March 2012 3
- 4. Wait a Second… Results Accountability Performance Measures Haven’t we heard about this before? Everyone wants to: •Improve our programs & maximize impact •Leverage support for our programs Benchmarks for a Better VT March 2012 4
- 5. What gets in the way? • “Too little time” • “Other priorities” • “Too complicated” • “Too many hard decisions” • “We don’t have expertise – or $$ to hire someone” • “They don’t get it” • “Doesn’t feel relevant” • Overthinking…Trying too hard Benchmarks for a Better VT March 2012 5
- 6. Why Bother? Don’t I have enough to worry about already?? Benchmarks for a Better VT March 2012 6
- 7. Why Bother? Benchmarks for a Better VT March 2012 7
- 8. What is Results Accountability? Benchmarks for a Better VT March 2012 8
- 9. Shifting our Thinking Results Accountability is Results Accountability IS a : NOT: •behavior discipline •a product •framework for planning, doing, •a year-end evaluation and evaluating •software •process for moving from Talk •(just) a reporting tool to Action •cycle for continuous improvement •tool for communicating Benchmarks for a Better VT March 2012 9
- 10. Shifting our Thinking RBA is NOT: RBA is : •fancy •simple •new •the MAIN track •a side track RBA is : •asking an answering common sense questions (that we already think about all the time) Benchmarks for a Better VT March 2012 10
- 11. Common Sense Questions • What do we want? • How will we recognize it? • How do things look? • What will it take to get the results we want? • Who are the partners with a role to play? • What is our role in helping achieve the result? • How good of a job are we doing? Benchmarks for a Better VT March 2012 11
- 12. What’s the Big Deal? (1) Common Sense Questions •What do we want? Start with ENDS •How will we recognize it? •How do things look now? •What will it take to get the results we want? •Who are the partners with a role to play? •What is our role in helping achieve Then talk about the result? MEANS •How good of a job are we doing? Benchmarks for a Better VT March 2012 12
- 13. What’s the Big Deal? (2) Common Sense Questions •What do we want? COMMUNITY FOCUS •How will we recognize it? •How do things look now? Can look at different levels… •What will it take to get the results …with appropriate we want? accountability (more on that •Who are the partners with a role later) to play? •What is our role in helping achieve the result? PROGRAM FOCUS •How good of a job are we doing? Benchmarks for a Better VT March 2012 13
- 14. What’s the Big Deal? (2 continued) Common Sense Questions •What do we want? For each “thing” we do: •How will we recognize it? •How do things look now? •Who do we serve? •What will it take to get the results we •What do we do? want? •How good of a job are we •Who are the partners with a role to play? doing? •What is our role in helping •What works to achieve the result? improve/strengthen? •How good of a job are we doing? • What do we propose to do? Benchmarks for a Better VT March 2012 14
- 15. What’s the Big Deal? (3) Common Sense Questions •What do we want? Common Sense •How will we recognize it? •How do things look now? Clear Language •What will it take to get the results we want? •Who are the partners with a role No Jargon to play? •What is our role in helping achieve Tells a Story the result? •How good of a job are we doing? Benchmarks for a Better VT March 2012 15
- 16. A few words about LANGUAGE • Outcome, Result, Benchmark, Goal, Target • It doesn’t matter what you call it, as long as people know what you mean. You can call me anything you like, but just don’t call me late for dinner (or do I mean supper?). Benchmarks for a Better VT March 2012 16
- 17. Agreeing on Clear Language • What do we want? RESULT A condition of well-being for children, adults, families or communities as a whole • How will we recognize it? INDICATOR How we measure this • How do things look now? condition • What will it take? • Who are the partners? • What is our role? • How good of a job are we doing? Benchmarks for a Better VT March 2012 17
- 18. EXAMPLES RESULTS for All People of the State, County, City or Neighborhood o A Prosperous Economy A A Clean Environment A Healthy and Safe Communities Self-Sufficient Families S Children Ready for and Succeeding in School C Parents and Other Adults Healthy and Self-Sufficient P Elders Living with Dignity in Setting of Their Own Choice Benchmarks for a Better VT March 2012 18
- 19. Clear Language, continued • What do we want? • How will we recognize it? • How do things look now? • What will it take? • Who are the partners? • What is our role? What will STRATEGY we do to help achieve the An action taken to improve result? the RESULT • How good of a job are we PERFORMANCE doing? MEASURES How we know if strategies Benchmarks for a Better VT March 2012 are working 19
- 20. Strategy Examples What we do to help achieve the desired result: •Program/Service Delivery •Collaboration •System Change •Service Integration •Funding •Resource Development/Leveraging Benchmarks for a Better VT March 2012 20
- 21. Different Levels of Focus RESULTS & INDICATORS are about the well being of populations “All people are happy and healthy” Benchmarks for a Better VT March 2012 21
- 22. Different Levels of Focus STRATEGIES & PERFORMANCE MEASURES are about the well being of program participants / service recipients “Vaccinations for school children” Benchmarks for a Better VT March 2012 22
- 23. A quick test: RESULT, INDICATOR, or PERFORMANCE MEASURE? • Safe Community RESULT INDICATOR • Crime Rate PERF. MEASURE • Average Police Department response time RESULT • A community without graffiti INDICATOR • % of Montpelier buildings without graffiti RESULT • People have living wage jobs and income INDICATOR • % of people in Windsor County with living wage jobs and income PERF. MEASURE • % of participants in job training who get living wage jobs
- 24. Different Levels of Focus = Different Levels of Accountability 1. Population Level - Focus = well being of population - Collective accountability: Many partners are responsible for achieving the desired result 2. Program (Strategy) Level - Focus = well being of program participants/service recipients - Program accountability: Program is responsible for its performance Benchmarks for a Better VT March 2012 24
- 25. Who should be accountable? • Children in Brattleboro are ready for school • Children in grade 3 at St. Albans City School learn to read • Job training participants gaining new skills • Employment rate in Bennington County • Employment in Burlington’s New North End neighborhoods Benchmarks for a Better VT March 2012 25
- 26. What’s Appropriate Accountability? Who What Makes sense Many Partners All children are ready for school NO!!! Not appropriate! Program Makes sense Pre literacy skills in Head Start classroom Benchmarks for a Better VT March 2012 26
- 27. Bringing Two Levels Together See Handout “Results Accountability: The WHOLE PICTURE” Benchmarks for a Better VT March 2012 27
- 28. Benchmarks for a Better VT March 2012 28
- 29. Benchmarks for a Better VT March 2012 29
- 30. Benchmarks for a Better VT March 2012 30
- 31. The Accountability Boundary _____________________________________________ ** ACCOUNTABILITY BOUNDARY ** _____________________________________________ •Why is this so important? •OK to cross –but have to be AWARE Benchmarks for a Better VT March 2012 31
- 32. How does RBA fit with: • Strategic planning • Outcomes evaluation • Logic Model • SSML • SWOT • PDCA Benchmarks for a Better VT March 2012 32
- 33. Break for Lunch Burning questions? Share them with the facilitators and fax to Amy See you again at 12:45 sharp After lunch we will focus in on: Program Accountability & Performance Measurement Benchmarks for a Better VT March 2012 33
- 34. REMINDER Different Levels of Focus = Different Levels of Accountability 1. Population Level - Focus = well being of population - Collective accountability: Many partners are responsible for achieving the desired result 2. Program (Strategy) Level - Focus = well being of program participants/service recipients - Program accountability: Program is responsible for its performance Benchmarks for a Better VT March 2012 34
- 35. REMINDER What’s the Big Deal? Common Sense Questions For each “thing” we do: •What do we want? •How will we recognize it? •How do things look now? •Who do we serve? •What do we do? •What will it take to get the results we want? •How good of a job are we •Who are the partners with a role to play? doing? •What works to •What is our role in helping improve/strengthen? achieve the result? •What do we propose to do? •How good of a job are we doing? Benchmarks for a Better VT March 2012 35
- 36. Program Accountability PINK Handout • Who do we serve? • What do we do? • How good of a job are we doing? – How much do we do? – How well do we do it? – Is anyone better off? (Who? How so?) • What works to improve/strengthen? • What do we propose to do? Benchmarks for a Better VT March 2012 36
- 37. Program Accountability • Who do we serve? • What do we do? • How good of a job are we doing? 1. How much do we do? Performance 2. How well do we do it? Measures 3. Is anyone better off? (Who? How so?) • What works to improve/strengthen? • What do we propose to do? Benchmarks for a Better VT March 2012 37
- 38. Program Performance Measures COMMON EXAMPLES How much? •# People served •# Hours service offered •# Activities (by type) How well? •Participant satisfaction •Use of best practice •Qualifications/training •External review Anyone better off? •Change in skills, knowledge, behavior, circumstance, well being •Possible sources: program records, participant survey, external data Benchmarks for a Better VT March 2012 38
- 39. Program Performance Measures EXAMPLE High School How much? •# students How well? •Student teacher ratio Anyone better off? •NECAP scores •Graduation rates •% students entering higher education or employment after graduation Benchmarks for a Better VT March 2012 39
- 40. Program Performance Measures EXAMPLE Welfare to Work How much? •# people trained How well? •% child care and transportation needs met Anyone better off? •% employed 6 months after program completion Benchmarks for a Better VT March 2012 40
- 41. Program Performance Measures EXAMPLE Raising Children How much? •# children, # hours spent How well? •Parent training/education; use of best practice (child satisfaction??) Anyone better off? •Child health •Child learning achievement •Child happiness & well being Benchmarks for a Better VT March 2012 41
- 42. Performance Measurement Challenges How do we know if anyone is better off? Required measures Lack of data Excess data Inappropriate data Inappropriate expectations Benchmarks for a Better VT March 2012 42
- 43. Choosing Performance Measures (1) • Answer all 3 questions • Choose what YOU want to know – What do we need to know to plan? – Evaluate our work? – Make it better? • Don’t think (too much) about others • This is about the program Benchmarks for a Better VT March 2012 43
- 44. Choosing Performance Measures (2) • Whole staff activity – Central part of planning, service delivery, budgeting, communications, reporting • Map out what you will measure & why • Look at what you already have • TRACK OVER TIME • TRY something. Did that work/help? Try something else. Benchmarks for a Better VT March 2012 44
- 45. GROUP ACTIVITY Imaginary Program: Cavity Free Richmond Who do we serve? Residents of Richmond, all ages, who do not have access to affordable dental care (e.g., underinsured, uninsured) What do we do? •Free dental care in clinic (preventive care & treatment) •Presentations in schools about good dental care habits •Outreach to build awareness of services Benchmarks for a Better VT March 2012 45
- 46. GROUP ACTIVITY Imaginary Program: Cavity Free Richmond Your Task: Identify possible Performance Measures How could this program measure: •How much it does? •How well it does it? •If anyone is better off? 15 minutes for group work Report out from Newport, Rutland, St. Albans Benchmarks for a Better VT March 2012 46
- 47. More Examples Benchmarks for a Better VT March 2012 47
- 48. Example: Early Care & Education 1. How much ? # children served; # receiving financial assistance 2. How well? STARS rating Staff training & qualifications, best practices 3. Is anyone better off? Old measure: developmental status New measure: % needs addressed Benchmarks for a Better VT March 2012 48
- 49. Example: Youth Development 1. How much ? # and characteristics of youth served 2. How well? Licensing, external review, quality rating Staff training & qualifications, best practices, standards 3. Is anyone better off? Youth feel safe at the program. Youth feel connected to caring adults/role models. Youth for a Betterconnected to their community. Benchmarks feel VT 49 March 2012
- 50. Example: Residential Treatment Is anyone better off? How to define this? Complexity Simplicity What do we really need to know? Benchmarks for a Better VT 50 March 2012
- 51. WRAPPING UP PROGRAM ACCOUNTABILITY Benchmarks for a Better VT March 2012 51
- 52. POPULATION ACCOUNTABILITY YELLOW Handout A few key points: •Indicators = data •Don’t get lost in the data. Focus on RESULT. •How to pick good indicators? There’s a chapter for that. Let go of perfection. •When is Population Accountability useful? – Clarifying accountability – Showing relevance – Bringing together partners. Benchmarks for a Better VT March 2012 52
- 53. GROUP ACTIVITY Population Accountability Exercise Imaginary Problem: •Community upset about fires in Winooski •Fire Department budget questioned •Plan for community meeting Benchmarks for a Better VT March 2012 53
- 54. GROUP ACTIVITY Population Accountability Exercise What we want? RESULT All people in Winooski are safe from fire Your task today: Who are the partners with a role to play? 10 minutes Report out from Montpelier & Bennington (if time) Benchmarks for a Better VT March 2012 54
- 55. Population Accountability: Examples Benchmarks for a Better VT March 2012 55
- 56. Population Accountability : Burlington Truancy Project Started with a Problem: Burlington had highest drop out rate in VT. And it was going up. What will it take to turn this around? Who needs to be at the table? Benchmarks for a Better VT March 2012 56
- 57. Bur l i ngt on Tr uanc y Pr oj ec t Shar ed ac c ount abi l i t y & par t ner s hi p • Desired Result All Burlington students graduate from high school • Indicator High school drop out rate • Baseline Not good! Highest dropout rate in VT Dropout Rate for Burlington School District Students 10.0% 9.0% 8.0% 7.0% 6.0% 5.0% 4.0% 3.0% 2.0% 1.0% 0.0% 96 98 99 01 03 04 06 07 08 95 97 00 02 05 19 19 20 20 20 20 20 19 19 19 20 20 20 20 Benchmarks for a Better VT March 2012 57
- 58. Bur l i ngt on Tr uanc y pr oj ec t ( c ont . ) • What works? Address truancy to prevent drop out • Partners Non-profit service providers, school district, police department, school administrators, juvenile court judges, attorneys, child protection agencies, and United Way of Chittenden County • Strategies A coordinated response to truancy: - School District: new policies & procedures - Juvenile Court & SA : monthly court time - Non profits & state agencies: services that remove barriers Benchmarks for a Better VT 58 March 2012
- 59. The Tr uanc y Pr oj ec t : Key I ndi c at or Dropout Rate for Burlington School District Students 10.0% 9.0% 8.0% 7.0% 6.0% 5.0% 4.0% 3.0% 2.0% 1.0% 0.0% 95 96 97 99 01 02 03 04 05 06 07 08 98 00 19 19 19 19 20 20 20 20 20 20 20 20 20 19 Benchmarks for a Better VT 59 March 2012
- 60. Example: Truancy Task Force & the EVEN BIGGER Picture What do we want? Youth succeed in school How will we recognize it? School completion; Competencies What does it take? – Good educational opportunities – Youth attend school – Youth ready to learn Benchmarks for a Better VT March 2012 60
- 61. Example: Investing in Early Childhood (How UWCC fits in bigger picture) Result RESULT All children enter school ready to learn Indicator STRATEGY STRATEGY OTHER Kindergarten readiness 1 2 INFLUENCES UWCC Strategies (the roles we play) 1. Fund programs that provide parent support & education 2. Provide access to high quality early education 3. Volunteer programs supporting early literacy Benchmarks for a Better VT March 2012 61
- 62. RBA = Process What about Product? Benchmarks for a Better VT March 2012 62
- 64. 64
- 65. Reminder: Goals for Today aka How I hope you will better off Brief Intro Results-Based Accountability (RBA) •Why bother? •What is it? ese? •How it can help you plan e co ver th Did w •How can it help you manage…and maximize…and communication •Where to get more information & resources If nothing else… 65
- 66. Key Features of RBA 3 Performance Measure questions 2 Levels of focus (& accountability) 1 Simple framework to help focus Benchmarks for a Better VT March 2012 66
- 67. Remember: • Anyone can do this, at any time. Period. • This is a whole staff/whole organization process. All the time. • Don’t worry about perfection. Sketch it (don’t etch it). • Your program is not responsible for population level results. • You have a role to play in population level results. Know what that is. • Your program is responsible for helping its participants be better off. You have to know if and how. • Keep RBA questions in your pocket, bring them with you. Say, “How much? How well? Anyone better off?” Say, “Who is responsible for that? For this?”67
- 68. Feeling ready to give it a try? • Buy the book – “There’s a chapter for that!” • Check out the website • Apply for the Performance Institute • Start asking and answering the questions – Staff meeting – Board meeting – Coalition meeting • JUST DO IT. Benchmarks for a Better VT March 2012 68
- 69. Any questions? bbvt@marlboro.edu http://bbvt.marlboro.edu/ Benchmarks for a Better VT March 2012 69
Hinweis der Redaktion
- Start my piece by explaining that I’m not a trainer/consultant, But a USER and BIG FAN!
- Covering enough to give you a taste, decide how this might be useful for you now/in future. Share some important ideas about planning and decision-making and accountability for making the biggest difference, achieving the greatest impact for the work we do. Here’s where I say that the plan is not to do ALL of RBA – to hit highlights and point out where there are options to go deeper, using resources
- These are the resources we’re working with – we adjust a bit based on what works best, and that’s OK – same concepts, same approach (that’s how we explain that our Perf Account. Questions are in different order - not perfect match with book, but not at all inconsistent) Recommend the book – great read, honestly
- Wait a second…haven’t we heard this before? I did a lot of outcomes trainings – and talked about keeping eye on the prize: Improving program performance Leveraging support for program And let’s face it – everyone wants to do that, and everyone will agree that any process that makes those two things happen is worth doing, spending time on. BUT I came to realize that the problem is not the PRIZE, but the feeling that the prize is not attainable. “Yeah, sure.” Feels rigged. Carnival game. I don’t want to waste time trying to convince you that you want the prize – I should try to help you figure out how to “win” the game. See more on this in notes re: the BIG DEAL slides.
- Too complicated Too little time/other priorities (We don’t have the expertise – or $$ to hire someone) Hard decisions – don’t know how to choose Inappropriate expectations from others (who don’t get the point) Not relevant (what funders want) Overthinking…Trying too hard to think of what others want/expect. I have to address this and say that I’m going to try to focus on how PM can meet YOUR needs and therefore meet funders/others’ needs.
- Maybe feel too busy dealing with all the work you’re already doing – trying to figure out hard things like: What’s our mission, how can we achieve it, what should we be doing, how can we support that work, is it working? (read questions) How should I run my program(s)? How can I get the most out of limited resources? How much time/money should go into which activities? Should I take on this new activity? Who should I be talking to how to meet the needs of the families we see? How does what they’re doing relate to what we’re doing? How can I convince this funder to support my program? How can I spend less time tracking data I don’t need? How can I demonstrate value? Is what we’re doing making a difference? What else? These are pressing questions, and we talk about the time.
- Maybe feel too busy dealing with all the work you’re already doing – trying to figure out hard things like: What’s our mission, how can we achieve it, what should we be doing, how can we support that work, is it working? (read questions) How should I run my program(s)? How can I get the most out of limited resources? How much time/money should go into which activities? Should I take on this new activity? Who should I be talking to how to meet the needs of the families we see? How does what they’re doing relate to what we’re doing? How can I convince this funder to support my program? How can I spend less time tracking data I don’t need? How can I demonstrate value? Is what we’re doing making a difference? What else? These are pressing questions, and we talk about the time.
- Isn’t this just another hoop we have to jump through??? No! (Timing on this slide)
- Simple: keep it simple, not overthinking, getting side tracked. Here’s where I say: “It slices, it dices, it gets out any stain.” (no wait, there’s that icky feeling that it’s too good to be true!)
- This is the BIG view of RBA – shows how it starts at TOP level, big picture. But an important part of RBA is that down at the bottom is breaks out into a more specific set of questions that allow us to look very specifically at our role/our performance.
- This isn’t the ONLY way, but we’ve found it to be the best way for these reasons. Goal = common approach among all of us, and this does that VERY well. So that’s why we’re talking about RBA. (not because I work for RBA) Starting point with this work = ENDS (we need a common approach) and RBA = MEANS. ENDS first, then MEANS – emphasize this more, we talk about what we DO or our inputs, but it’s really about the ENDS, we should always start there! (And that’s the conversation boards, staff, RD people need to be having always, starting with ENDS) Should always START with ENDS, that’s our mission – our ENDS are why we’re doing this work. Have to be driven by the ENDS. A few points along this sequence – using DATA to help us look how things are, drawing on what we KNOW about what works, scoping out WHO should be involved/who can help this happen, THEN getting down to our role in this big picture. What I love is the sequence of questions (and answers) that get us from those ENDS to the MEANS. We’re all good at saying mission and activities – this framework builds the bridge between them. This helps us FOCUS our work, improve (hone??? Can’t read my notes) what we do.
- 2 levels of accountability (maybe save this for later slide showing two levels): helps us not misattribute/set up for failure.
- Prior slide = two levels embedded in the main set of questions, this is showing how the bottom two questions can be expanded/broken out. I removed the 3 PMs from this slide…think about that. This is the PM piece…MORE ABOUT THIS LATER
- Common sense : Anyone can get this. Anyone can DO this (that point may deserve its own slide at some point). This is something YOU do, as a program, don’t have to hire someone (careful with this point). Clear language/no jargon helps us cut through the crap, get focused, and be able to JUST DO IT. Creates common understanding – easy accessibility/no exclusion. Making it not so FOREIGN, off putting, isolating. (Example of seeing someone else’s “evaluation” with pages and pages and big words and long paragraphs and and and. Part of us thinks: boy, wish I had that, and part of us thinks, what’s the use of that????!!!! TRANSITION TO NEXT SLIDE: Clear language = last of my three “Big Deal” slides, but let’s get to that right now: AGREEING on clear lanuage (then we’ll come back to these other points in a minute)
- This isn’t the ONLY way, but we’ve found it to be the best way for these reasons. Goal = common approach among all of us, and this does that VERY well. So that’s why we’re talking about RBA. (not because I work for RBA) Starting point with this work = ENDS (we need a common approach) and RBA = MEANS. ENDS first, then MEANS – emphasize this more, we talk about what we DO or our inputs, but it’s really about the ENDS, we should always start there! (And that’s the conversation boards, staff, RD people need to be having always, starting with ENDS)
- Results are stated in positive terms and that’s a very important characteristic of results accountability. Most planning processes we have used in the past state with children’s problems or with unmet needs in the community. Now we have to talk about problems and unmet needs, but you don’t have to start there. We send a powerful message out into the community in the way we talk about this. And results should always be stated in positive, not negative, terms. Ideas about indicators used to track these Results?
- So STRATEGIES = what we do, and this can take many forms. Read through list And maybe say something quick about PMs – those would be the measures/data we look at to assess the performance, see if these strategies are working as intended. We’ll talk more about those MEASURES later – the main point I want here is about STRATEGIES. Data are important, but data are a way to see/track/understand something MORE important (ala Joy L’s comment on 3/8!) But remember strategy = MEANS to END (not the END in itself). Point of the strategy = contribute to the END/Result/OUTCOME Let’s look at this this idea of ENDS and MEANS…
- Another “big deal” I pointed out was the fact that the RBA framework includes different levels of focus – can fit BIG PICTURE as well as specific focus.
- Accountability: 2 Levels (why important to make distinction/see that line – common mistake of being held accountable in wrong way!!!) Stop here – or maybe sooner – and talk about ‘what does it mean to be “accountable?” Not solely responsible, but that this is how you’ll evaluate your performance as a program. “We are accountable for figuring out how to make this happen (this ABO). If it’s not happening, that means WE need to do something differently.” Need a good example here about how a POP Level indicator would NOT affect how you run your program. We can talk about Impact/Measurement/Evaluation at TWO different levels – and the implications of these two levels, in terms of what you measure, what you hope to achieve, and how you interpret the data, are very important 1. Community- level impact means looking at the well being of a community population Acknowledges that many partners, many strategies contribute to (are accountable for contributing to) community-level outcomes, and the well being of population This is the appropriate level of focus for a community collaborative, a group of partners working together to achieve a common goal This is not the appropriate level of focus for an individual program or strategy. Example: High school achievement (SAT scores for a specific school/community/area). Program level impact – focus = well being of people you serve Gathering data to see how program participants benefit from the program/service This is the appropriate level of focus for an individual program, agency, organization, strategy Example = teen tutoring program data might be the grades of the kids in the program (going up?) So important distinction = appropriate accountability SWITCH TO NEXT SLIDE
- This big picture shows the link between YOU and your mission. Between US and what we want. We all know our missions, and we all know what we DO (our activities). It’s the inbetweens that get so easily muddied and misunderstood. What can seem less clear is: Do we do it well? Should we do it differently? Are we doing the right things, the right way? Is this working? Making a difference? How can I demonstrate that? Convince others? Good leadership is being able to link those two things, balance both levels.
- What’s up with this? Why is this border so important? GIVE some examples of how we cross it all the time and don’t realize it – had a good one when Monica did dental example – program name = Cavity Free Richmond ZAP/BUZZER!! OK to cross – I’ve just spent quite a bit of time talking about how RBA is so lovely because we can do both sides/countries. But have to be AWARE.
- Hope to do this part right before lunch – if time. Should I say that I went to a great SP session yesterday, mention Jim & Marlboro, and here were some key points: SP is for the whole organization Good SPs make things clearer and simpler. Perfection is elusive, expensive, and unnecessary.
- Accountability: 2 Levels (why important to make distinction/see that line – common mistake of being held accountable in wrong way!!!) Stop here – or maybe sooner – and talk about ‘what does it mean to be “accountable?” Not solely responsible, but that this is how you’ll evaluate your performance as a program. “We are accountable for figuring out how to make this happen (this ABO). If it’s not happening, that means WE need to do something differently.” Need a good example here about how a POP Level indicator would NOT affect how you run your program. We can talk about Impact/Measurement/Evaluation at TWO different levels – and the implications of these two levels, in terms of what you measure, what you hope to achieve, and how you interpret the data, are very important 1. Community- level impact means looking at the well being of a community population Acknowledges that many partners, many strategies contribute to (are accountable for contributing to) community-level outcomes, and the well being of population This is the appropriate level of focus for a community collaborative, a group of partners working together to achieve a common goal This is not the appropriate level of focus for an individual program or strategy. Example: High school achievement (SAT scores for a specific school/community/area). Program level impact – focus = well being of people you serve Gathering data to see how program participants benefit from the program/service This is the appropriate level of focus for an individual program, agency, organization, strategy Example = teen tutoring program data might be the grades of the kids in the program (going up?) So important distinction = appropriate accountability SWITCH TO NEXT SLIDE
- Prior slide = two levels embedded in the main set of questions, this is showing how the bottom two questions can be expanded/broken out. I removed the 3 PMs from this slide…think about that. This is the PM piece…MORE ABOUT THIS LATER
- Do these questions sound familiar? Make sense? Good story line, right? Great grant application, good plannign document. Shouldn’t EVERYONE at your org be able to answer these, and in the same way? I won’t spend much time on the first two questions. You can do that easily! (But make sure all agree/realize) And I won’t spend too much time on the bottom 2 Qs – again, I think you’re good at that, but need to do it AFTER the other questions – main point is the SEQUENCE of these questions. You PLAN/DO only after you’ve answered the other questions. ALSO: Make the point that this is the CYCLE!!!
- This is where we ask if what we’re doing is working -- how well we’re performing.
- Look how well you did – for a program y 15 minutes max PLUS the report out 3 minutes max let’s here from Newport, Rutland, Mont/St. A (and save time for me to comment – 5-10 minutes) TOTAL 30-45 min. ou don’t even know, doesn’t even exist!
- Look how well you did – for a program y 15 minutes max PLUS the report out 3 minutes max let’s here from Newport, Rutland, Mont/St. A (and save time for me to comment – 5-10 minutes) TOTAL 30-45 min. ou don’t even know, doesn’t even exist!
- THEME: Program is right (not funder) Another example, from programs we partner with/fund Youth development programs funded by UWCC answer the 3 PM questions in this way: (read through slide) Only New data = survey, yearly, getting youth input (and that was a refinement of existing survey, so not totally new) Q examples from survey: Youth feel safe: Youth feel connected to caring adults: Youth feel connected to community: Can be used by individual program performance but also shared by programs so can look at Comm level because of uniform Qs # of matches Average length of match, satisfaction Perceived benefits (reported by youth, what else?) Can talk here about how UWCC’s investments contribute to community status on this TO
- THEME: Don’t need Longer Term Another example, from programs we partner with/fund Youth development programs funded by UWCC answer the 3 PM questions in this way: (read through slide) Only New data = survey, yearly, getting youth input (and that was a refinement of existing survey, so not totally new) Q examples from survey: Youth feel safe: Youth feel connected to caring adults: Youth feel connected to community: Can be used by individual program performance but also shared by programs so can look at Comm level because of uniform Qs # of matches Average length of match, satisfaction Perceived benefits (reported by youth, what else?) Can talk here about how UWCC’s investments contribute to community status on this TO
- THEME = don’t need to be fancy or scientific (use staff’s knowledge and case review activity – no EXTRA data collection) Maybe rework this example to be more general – did the client get what she/he needed? Example I heard about that I really liked… (I was not involved, but really liked the message)
- Segue to POP Account is about telling your story – PM is a big part of that, but you also need to be able to talk about where you fit in the big picture.
- Look how well you did – for a program y 15 minutes max PLUS the report out 3 minutes max let’s here from Newport, Rutland, Mont/St. A (and save time for me to comment – 5-10 minutes) TOTAL 30-45 min. ou don’t even know, doesn’t even exist!
- Look how well you did – for a program y 15 minutes max PLUS the report out 3 minutes max let’s here from Newport, Rutland, Mont/St. A (and save time for me to comment – 5-10 minutes) TOTAL 30-45 min. ou don’t even know, doesn’t even exist!
- POPULATION/COMMUNITY = BURLINGTON high school students Like many projects/programs, this one started with a problem: Problem : Burlington had highest drop-out rate in VT Problem : Burlington had highest drop-out rate in VT. And it was going up. Here you can see the Indicator, and the baseline (and you can get a sense of the forecast). Response : Truancy Task Force convened to address the school district’s high dropout rates They wanted to TURN THE CURVE
- So what happened?
- The results for the youth of Burlington have been spectacular! In the first 5 years, Burlington cut its drop-out rate from 10% to 3.5% . . . (I need to update this chart!) a measurable example of improving lives by mobilizing communities to create lasting changes in community conditions. Now, remember there were multiple strategies – and we could look at the relative impact of each if we looked at how kids did if they received/were affected by each strategy.
- Go through this example in detail – highlighting point about developmental data being a tricky performance measures. What is success in terms of developmental assessment? Esp. when programs are serving children at risk? When done with this example: I’ve been using the same terms so far, and I hope it is clear what I mean when I say RESULT or STRATEGY. There is no reason to use THESE exact terms, the important thing is that we (you) use the same terms and focus on the same indicators etc. Be ready to give your ELEVATOR SPEECH! Next slide< common language options
- Does this fit with what people expected? Anything else? Here’s where I say that the plan is not to do ALL of RBA – to hit highlights and point out where there are options to go deeper, using resources (next slide)
- Does this fit with what people expected? Anything else? Here’s where I say that the plan is not to do ALL of RBA – to hit highlights and point out where there are options to go deeper, using resources (next slide)
- Buy the book. “There’s a chapter for that!” Check out the website Apply for the Performance Institute I – I said you don’t need outside help, and you don’t, but BOY is it nice! Start asking and answering those questions JUST DO IT.