Hearth Academy Performance Measurement
•Download as PPT, PDF•
0 likes•646 views
On Wednesday, October 13, 2010, the Center for Capacity Building will host "Performance Improvement Strategies," the second pre-session webinar in the HEARTH Academy series. This webinar presented strategies to reduce length of stay in homelessness, new entries into homelessness, and repeat episodes of homelessness, all of which drive reductions in overall homelessness.
![[object Object],[object Object],[object Object],[object Object],[object Object],The HEARTH Act ,[object Object],[object Object],[object Object],The HEARTH Academy](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Hearth Academy Performance Measurement
Similar to Hearth Academy Performance Measurement (20)
More from Monarch Housing
More from Monarch Housing (10)
Recently uploaded
Recently uploaded (7)
Hearth Academy Performance Measurement
- 1. The National Alliance to End Homelessness presents The HEARTH Academy Training and tools to help your community achieve the goals of the HEARTH Act
- 3. Implementing proven strategies to end homelessness HEARTH Academy Implementation Clinic Participants in this 1.5 day clinic will assess the performance of their homelessness assistance and implement community-wide strategies to better achieve the goals of the HEARTH Act. Individualized Consulting The Alliance’s Center for Capacity Building and other expert consultants will be available to provide customized assistance. Webinars and Tools Webinars, tools, and training materials will help communities prepare for the Implementation Clinic and learn about and implement the strategies that help prevent and end homelessness.
- 4. Aisha Williams Center for Capacity Building National Alliance to End Homelessness [email_address] 202-942-8298 If you are interested in participating or would like more information, please contact: Note: The HEARTH Academy is not sponsored by or affiliated with the Department of Housing and Urban Development (HUD) or any other federal agency. The HEARTH Academy is a project of the National Alliance to End Homelessness.
- 5. The HEARTH Academy Performance Improvement Strategies HEARTH Academy Sep 2010 - Apr 2011
- 6. Performance Improvement Agenda Introduce the HEARTH performance measures Understand key measures, programs, and overall impact How to measure at a system level Successful performance improvement strategies Define performance measurement and improvement
- 8. Performance improvement requires systematic evaluation of your programs and system to determine their impact and guide efforts to improve results.
- 9. HEARTH Act Performance Measures Reduce the number of people who become homeless Reduce length of homelessness Reduce returns to homelessness Reduce overall homelessness Increase jobs and income Thoroughness in reaching homeless population Other accomplishments related to reducing homelessness
- 10. Core HEARTH measures Reduce new episodes of homelessness Reduce lengths of homeless episodes Reduce returns to homelessness
- 13. Measures of Homelessness Average number of homeless people at any time = Number of people who newly become homeless each day x Average length (days) of homeless episodes + Number of people who return to homelessness each day 25 1 40 1,040
- 14. Measures of Homelessness Average number of homeless people at any time = Number of people who newly become homeless each day x Average length (days) of homeless episodes + Number of people who return to homelessness each day 25 1 36 936
- 15. Homelessness Assistance Programs Program Program Program Program Program Program Evaluation
- 16. Homelessness Assistance S ystem Program Program Program Program Program Program Evaluation
- 17. System Measurement Challenges Program Program Program Program Program Program Program Program
- 19. Columbus: Performance on Key Outcomes 2009 Performance Families Singles Reductions in homelessness 6% 4% Diversion 32% 13% Length of Stay 59 49 Reentries 0% 1%
- 22. Columbus, Ohio: A Data Driven System Individual Outcome Performance Scores YES 90% or more or within 5% of numerical goal NO Less than 90% or not within 5% N/A New and Unassigned Goals System/Program Evaluation Ratings HIGH 75% overall outcomes or better MED 50% to 74% overall outcomes LOW Less than 50% overall outcomes
- 28. Measurement Tools & Strategies System Performance Measurement Tool Coming Soon…
Editor's Notes
- The HEARTH Act establishes an ambitious federal goal of “ ensuring that individuals and families who become homeless return to permanent housing within 30 days” The HEARTH Act also requires HUD to evaluate CoCs based on their performance on a number of outcomes, including Duration of homelessness episodes Returns to homelessness Number of people who become homeless Overall homelessness The Alliance’s HEARTH Academy is designed to help your community achieve these ambitious goals by Helping you assess the structure and performance of your current homelessness assistance Providing tools to help you implement proven strategies to reduce homelessness Helping you create an action plan to reshape your community’s response to homelessness so that it better achieves the outcomes set by the HEARTH Act The HEARTH Academy is not sponsored by or affiliated with the Department of Housing and Urban Development (HUD) or any other federal agency. The HEARTH Academy is a project of the National Alliance to End Homelessness.
- The HEARTH Academy has three major components: A 1.5 day clinic where 6-10 leaders from each of several communities will come together to assess their communities performance and create an action plan to implement cost effective strategies that will help their communities reduce homelessness and better meet the goals of the HEARTH Act A series of preparation and implementation webinars and tools covering a range of topics: An overview of the HEARTH Act and its implications Using performance measures to improve outcomes Designing a cost effective homelessness assistance system Individualized consulting for communities interested in receiving customized assistance
- The HEARTH Academy is not sponsored by or affiliated with the Department of Housing and Urban Development (HUD) or any other federal agency. The HEARTH Academy is a project of the National Alliance to End Homelessness. SURVEY???
- Thanks, Norm. Well. In the last Academy webinar you surveyed the viewers about what part of HEARTH they were most excited by and only 7% said it was the new performance measures. Well I am really excited about the inclusion of performance measures in the HEARTH Act because I think we have, for the first time, both the opportunity to really measure our impact as a field, and the tools with which to do it. I hope we’ll make some listeners a little excited too!
- Our agenda for this webinar is to first define what we mean by performance measurement and improvement Then we’ll learn about new performance measures that are embedded in the HEARTH Act. Next, we’ll understand the relationship between the key outcome measures, programs, and the overall impact on the goal of ending homelessness. We’ll talk about how to measure at a system level – and give a community example of this And we’ll conclude by present some successful performance improvement strategies, and then take your questions.
- Many of the ideas we will talk about today are discussed in the Alliance publication: What Gets Measured Gets Done, which is a toolkit that provides explanations and examples of many concepts that are important for creating a performance improvement strategy, including inputs, outputs, and outcomesrisk adjustment; and calculating system outcomes. The toolkit is accessible by visiting the Alliance website at http://www.endhomelessness.org and typing “What Get’s Measured Gets Done” into the search engine on the upper right hand corner of the screen. What Gets Measured Gets Done is a valuable companion to this presentation.
- Improving our performance first requires we evaluate our programs and the whole system in order to understand and then improve our outcomes. Putting this into practice at the community level means several things: First, we have to evaluate both our programs and the system — if we are going to achieve the goal of ending homelessness, we have to measure individual programs, but more importantly, we have to measure the entire homelessness assistance system. Secondly, we need to focus on evaluating outcomes and impact — For a long time we’ve been able to count people and services delivered but its been much harder to say what impact a single program, let alone a whole component of a system or the entire system are having on our ultimate goal of ending homelessness. Finally, we have to use the outcome data to improve results — Data about outcomes provides feedback to programs and systems about what is working and what needs to change. With this information, activities that don’t result in better outcomes can be retooled or ended and more effective program and system models can be replicated and expanded.
- These are the measures required by the HEARTH Act. They are part of the legislation and HUD is required to evaluate communities on these areas in its funding decisions. They include reducing the numbers of people who fall into homelessness over time, reducing how long people who do become homeless remain homeless, ensuring that people who have been homeless as far as possible don’t return to the state, increasing people who exit our systems with jobs or an appropriate source of income, Looking how effectively our system really reaches those in need of our assistance, and other accomplishments that HUD may choose to look at that are related to the ultimate reduction of homelessness.
- While all of the HEARTH Act measures are important, the intent of the Act seems to us to be summarized in three key goals and measures: 1) We need to reduce the number of people who newly become homeless—this includes people who use McKinney funded programs and those that don’t – we’re talking about impacting the overall flow into homelessness. 2) We need to try to eliminate as much as possible people who return to state of homelessness from having been homeless before, and 3) we need to reduce the time that people spend being homeless before they regain housing. The HEARTH act sets a national goal that no one will be homeless longer than 30 days. Stemming entries to homelessness in the first place and shortening the length of time homeless episodes last is the only way we can meet this ambitious national goals Other measures, such as our effectiveness at reaching people and our impact in helping them increase their incomes, play an important role in creating an effective system because they are linked to our ability to meet these core goals.
- This slide illustrates the relationship between the key performance measures and homelessness. The three measure factors that directly contribute to the number of people experiencing homelessness in your community and how our resources are occupied in helping them. New entries into homelessness are the source of most homelessness in our systems – more of them obviously means more homelessness Repeat episodes of homelessness mean our original interventions were unsuccessful – we did not achieve the goal of ending homelessness for that person or household, and thus must serve them again Finally, longer episodes of homelessness mean we spend more time and resources on that group of households, and are less able to stem the tide of new entries as a result
- So, successfully reducing homelessness requires investing in strategies that address these three factors. Our efforts to reduce new entries mean ensuring effective prevention and diversion programs are in place to reduce the number of people who become homeless. Housing stabilization services for households who exit homelessness but continue to be vulnerable can reduce their rates of return to homelessness. Prioritizing previously homeless people for prevention and diversion assistance can also help reduce reentries. A well developed rapid re-housing component is the primary strategy for reducing lengths of stay in homelessness. Getting people out sooner is going to reduce stays in homelessness. Permanent supportive housing programs can also reduce average lengths of homeless episodes, when they are targeted to people who are most likely to remain homeless. Looking at it the other way, if you want to measure the effectiveness of these strategies—prevention, diversion, stabilization, or rapid re-housing— or of any of your system strategies, you want to measure whether new entries, repeat episodes, and length of homeless episodes are increasing or declining.
- For people who like equations here’s another way to look at the same thing. The relationship between these factors can be expressed as an equation, the product of which is how many homeless people there are at a given time. The number of people who become homeless each day—including both people who become newly homeless and people who return to homelessness—times the average length of homeless episodes equals the number of people who are homeless at a given time. In this example, 25 new people become homeless each day and 1 person returns to homelessness. As these folks are staying homeless on average 40 days, the typical daily count of homelessness in going to be about 1,000 – those who newly entered, and those who continued being homeless.
- If you were to decrease the average length of homeless episodes in your community by 10 percent, say from 40 days to 36 days it would reduce the number of homeless people at a point in time by 10 percent. Similarly, if you reduce the number of people who become homeless in your community by 10 percent it would reduce your point in time count by 10 percent.
- Now we’ll look at another of the major shifts in the HEARTH Act—the shift from looking at homelessness assistance as a set of separate programs to looking at it as a system. In many communities, homelessness assistance is more of a collection of associated programs than a coordinated system. Providers and funders may work well together on new efforts or refer to one another and generally have good relations but such a relationship is not the same thing as a system. For example, in an uncoordinated system When people need homelessness assistance, they typically must contact and negotiate with individual providers or projects. The onus is on the person or household that needs shelter, to call l shelters until they find one with available space. Without a systematic approach, whether a participant is accepted into a program is determined mostly by the operator of that program and can be subjective. And, evaluation is typically focused on the results of individual programs for the people served by that program, but not on the overall impact of all programs working together on the problem. A disjointed system like this not only creates burden on clients, it is very hard to know what impact the entire system is having. It also can create data quality and performance measurement challenges. For example, one of the most important data elements under the HEARTH Act will be knowing when a household moves into permanent housing. If the system is not working together to ensure that this data is collected and reported in a uniform way, it will have a very difficult time reporting on impact. The rate of “don’t know” or missing answers can be a serious challenge to evaluating effectiveness.
- In a coordinated system, the focus of evaluation is on the impact of all the parts working together. Each part has a role to play but are working toward common ends – the ones we described before, reducing entries and reentries and shortening the overall length of time people spend homeless in our community. Some hallmarks of a more coordinated system include: A single front door or coordinated program entry so that when people need homeless assistance, they can contact a single entity facilitates their receiving any appropriate assistance. Typically, in a coordinated system whether a participant is accepted into a program is determined by shared criteria based on assessed factors designed to help match people with the programs and services that can most effectively and quickly meet their needs. And evaluation is focused on the whole system and its outcomes for every homeless or potentially homeless person in the community. A benefit of operating a cohesive system like this is that people experiencing homelessness are more quickly connected to the intervention that is going to get them to permanent housing. When the entire system is operating by common standards with a shared or transparent data system, evaluation of client outcomes is part of standard operations. Data is available about the impact of the whole system and its component parts, and performance can be improved for the entire system because programs and resource can be adjusted based on common goals and objective and readily available outcome data.
- There are many challenges to moving to a system-wide approach to homeless assistance, performance evaluation and performance improvement. In our next HEARTH Academy Webinar on October 27th, we will present more information on approaches to system design. Right now we’ll talk a little about challenges to system-wide measurement and evaluation. Even if you’ve got a more coordinated system with accountability for outcomes there will be some challenges: First of all, there are programs that are outside of the system in some way, and they may not participate in HMIS. Secondly, in most communities there are homeless individuals who are unsheltered and not attached to any program. Traditionally, such programs and people have just been left out of performance measurement and improvement efforts. But with the HEARTH Act’s new focus on system outcomes, they will have to be brought in. In addition, your community will have to make decisions about what to track and how to track it based on the HEARTH Act. Decisions will have to be made about data quality and sharing. Currently, programs track and report information independently of system goals and needs. They do these activities based on individual program goals and performance improvement needs. Moreover, many programs enter incomplete or incorrect data in to the HMIS, and they do not effectively share client information with partner agencies. This makes it impossible to track clients across your system. Under HEARTH, however, you will have to track people even as they move between programs, for example between different shelters, or as they are in multiple programs, for example receiving rapid re-housing while they are in shelter. Although these factors all present challenges, they can be overcome. Now Aisha is going to walk us through an example of a system that has overcome these challenges.
- Columbus OH has consistently performed well on outcomes included in the HEARTH Act. From 2007 to 2009, the community reduced homelessness in the midst of a nationwide recession with 6%, and 4% reductions in family and single adult homelessness, respectively. Also in 2009, the majority of homeless families exited the “front-door” emergency shelter for housing within 21 days and spent less than 60 days in shelter, on average. More than 60 percent of Columbus’ homeless families exited homelessness for housing with a lease in their name or homeownership. Only one percent of singles and zero percent of families who exited homelessness in 2009 returned to homelessness.
- In order to drive overall system improvement and reductions in homelessness, Columbus uses a combination approaches. The lead continuum agency, The Community Shelter Board, incentivizes a high level of performance by using performance-based contracts. Performance based contracts are a good way to encourage strong performance on predetermined goals and outcomes such as those outlined in the HEARTH Act. Performance based contracts encourage agencies to use proven strategies to move people out of homelessness quickly and stabilize them in housing. They also allow agencies enough flexibility to expand on what works and practice innovation, particularly when the reward is above and beyond the agency’s operating costs. The Community Shelter Board makes funding decisions and merit awards based on agencies’ annual evaluation and performance ratings, which means the agency only pays for programs that lead to reductions in homelessness. Sample performance-based contracts used in Columbus will be available on the HEARTH Academy website.
- Columbus uses contracts for other purposes related to performance improvement and measurement as well. One hundred percent of Columbus CoC service providers participate in the community’s HMIS, and the Community Shelter Board’s service contracts requires them to do so. Nearly 100 percent of all HUD-funded service providers who are not funded by CSB also participate in the HMIS. CSB tightly monitors participation in HMIS, and if a CSB-funded agency’s client files are inconsistent with the information entered into the community HMIS system, or data is entered incompletely or incorrectly, the agency is required to correct the data issues within one month. If the issues are still unresolved within one month, the agency is in breach of its contract and funding stops until the problem is resolved.
- Columbus’ CoC providers track more than 30 client- and program- level outcome measures. Some of the most important measures are: - Length of client stay and/or participation in a program - Cost per household served, per unit operated, and cost-efficiency - Employment and benefits status changes and income changes - Housing affordability and stability, and - Recidivism for prevention, diversion, shelter and permanent housing. These and other measures are folded into 15 system-wide measures, and progress on program- and system- level measures is reviewed quarterly. Evaluation in Columbus occurs on three distinct levels. First, the community assesses and “scores” each system and program performance goal as Achieved (Yes/Y), Not Achieved (No/N), or Not Applicable (N/A). An Achieved Goal is defined as 90% or better of a numerical goal or within 5 percentage points of a percentage goal. Not Applicable is assigned when a performance goal was not assigned or the program is too new to be measured. The key here is that systems have to develop system wide Second, each system and program is assessed as a High, Medium or Low performer based on overall achievement of system-level performance outcomes and overall program performance outcomes. High performing systems or programs are those that achieve at least 75% of the measure outcomes. Medium performers achieve at least 50% but less than 75% of the measured outcomes. Low performers achieve less than 50% of the measured outcomes At the third level of evaluation, Columbus’ system’s effectiveness as a whole is monitored and evaluated with combined data across providers and systems. You can view additional information about Columbus’ performance measurement and evaluation system on the HEARTH Academy website, where we’ve provided detailed summaries of the processes and tools used in Columbus. And, of course, we have Lianna Barbu from the Community Shelter Board on today’s webinar to answer specific questions about Columbus when we open up for Q&A. This information is also presented in even greater detail on Community Shelter Board’s website at www.csb.org The key points here are: Performance measurement and evaluation must be system wide All stakeholders operate according to the same goals, outcomes All stakeholders enter data using the same data management standards, and are consistent and accurate Program activities and client outcomes must be measured and evaluated across programs Evaluation that tracks client experiences and outcomes from system entry to exit and every subsequent interaction is required by the HEARTH and it is the only way to understand your current performance and determine how to improve your performance
- We will now shift to performance improvement. In Columbus, programs rated as “Low” or that experience repeated difficulty meeting performance goals are referred to CSB’s Quality Improvement Intervention Program (QII). During the QII process CSB helps poorly performing programs adjust their program activities to achieve better outcomes. The process uses targeted organizational support and development strategies to link system goals and program outcomes. The QII process includes: Assessment of the problem(s) Collaborative goal setting and improvement planning Regular check-ins regarding specific action steps and timelines If an entire component of Columbus' system is “scored” as low performing (e.g., the emergency shelter system), CSB works with all the providers in that system, collectively, to determine the source(s) of or barrier(s) to stronger performance. For example, the introduction of a new program, such as the Homelessness Prevention and Rapid Re-housing Program (HPRP), can adversely affect the performance of the subsystem that administers the program as providers go through a “learning” phase in the first few months of implementation.
- There are several tools and resources on the HEARTH Academy website related to Columbus’ approach to performance improvement and measurement as well as other tools that are already or will be coming available. These resources include detailed briefs on key components of Columbus’ system, blank templates from Columbus, and data management related resource documents. Watch the Alliance newsletter for tools that we put out.
- Going back to the initial goals from HEARTH. Here are ways that the HEARTH measures can be translated into measurable outcomes. These can be looked at over time and should be measured across the system rather than just for each individual program. Additionally, each individual program should be able to assess their performance relative to other similar programs.
- Most of us will encounter some significant challenges as we try to collect this data. Here are a few tips to overcome common barriers. Challenge: Impact of prevention hard to know Tips: Have point-in-time or entries to shelter gone down since prevention introduced/expanded? Look for returns to system for people provided assistance (see above on returns to homelessness). If possible, compare with data on people who are not assisted. Look at whether people served with prevention have similar “prior living situations” as those entering shelter. Challenge: Lengths of stay hard to determine, especially across multiples programs Tips: Definitely look at lengths of stay within single programs. Use HMIS to select persons with multiple program entries and track; Use a sample of multiple entries if difficult to get to all. AHAR gives idea of overlap between shelter and TH programs and of lengths of stay. Challenge: Rates of return to homelessness Tips: Select exits for a single month in previous year - see if the people who exited to permanent housing have a new entry. If your HMIS creates unique ID’s in chronological order, look for older numbers that have new entries.
- As part of the National Alliance HEARTH Academy we will be producing a data collection and evaluation tool that you can use to look specifically at the core HEARTH measures, including positive or quality exits, lengths of stay in homelessness and returns to homelessness. This tool will be published in November after our system design webinar and will include instruction on how to complete it to take a close look at system performance. Additional assistance with analyzing the data will be offered to communities that sign up for the Academy.
- But you don’t have to wait for us or for HUD to develop guidance. We already have several tools and strategies at our fingertips. We will be posting a document along with this presentation with some suggestions.
- First of all, HMIS systems have to be able to produce certain “canned reports” like the APR – while we tend to run them for programs or the whole system we can also run them for components – all shelters or all transitional housing – and look at how the combined component differs from individual programs or from the whole system. We can learn about where people enter from, exit to, what their characteristics are and how they differ. We can also run data quality reports that tell us about missing or partial data and evaluate where we need to work to get better answers. Other sources of information include HMIS Custom reports Point in time count data AHAR HPRP Reports
- So what does all this mean for you and your community? Analyze and understand your current performance. Gather data on new entries, reentries, and length of stay. Find out which programs are getting which outcomes. Begin examining your system goals, outcomes and activities in the context of the HEARTH Act, as well as performance measurement practices that need to be improved. Based on the data, what goals, outcomes, and activities seem inconsistent with the HEARTH Act? Determine the barriers to accurately tracking clients across programs and assessing client outcomes. Start planning
- More information about the HEARTH Academy can be found at: http://www.endhomelessness.org/content/article/detail/3335