SlideShare ist ein Scribd-Unternehmen logo
1 von 28
Evidence-Based Recommendations
     for Participation Station


          Marcie Timmerman
         Capstone Presentation
         MHA Candidate, 2012
       UK College of Public Health
Participation Station




MISSION STATEMENT: To provide a program-based
environment that educates, empowers and enhances
the lives of individuals moving toward recovery and
mental wellness.
The Problem(s)
• Need funding diversification: grant
  applications
• Need to “prove” their success to date
• Need to convince other agencies to
  collaborate for advocacy and mission
• Pursuing internal continuous quality
  improvement
Project Purpose
• Analysis of program evaluation and outcomes
  data collected Sept-Dec 2011
• Recommendations for improvement based on
  that analysis
Literature Review
• Gained understanding of models and
  ideologies which led to PS creation
• Learned vocabulary and theories behind tools
  questions
• Little to no available measurement tools,
  peer-reviewed “best practices”, etc. for
  services like PS in literature
Methods: FACIT
• Fidelity Assessment Common Ingredients Tool
  (FACIT) from SAMHSA with benchmark data
  – Structured Observation
  – Structured item-by-item questionnaire
  Evaluations could be internal or external.
     • Internal only
     • Interviewed 12 leaders, 75% of whom were peers
Example of FACIT Survey Question
1.1.3 “Hiring (1)Consumers are not involved in any
Decisions     hiring decisions
             (2) Consumers have some involvement in
             hiring decisions
             (3) Consumers are responsible for making
             most of the hiring decisions (50% or more).
             (4) Consumers are responsible for making
             all hiring decisions.
Methods: POP
• Peer Outcomes Protocol (POP)
  – Structured questionnaire with anchored scale
    responses for each participant
     •   1 for “Disagree”
     •   2 for “Somewhat Disagree”
     •   3 for “Somewhat Agree”
     •   4 for “Agree”
  – Last 10 questions “Coercion Scale”
     • Anchored 1-5 with “5” as best
  – Some questions worded so that the “4” is bad and
    “1” is good
Examples of POP Questions

1. Overall, I am satisfied with this peer support
program.

2. I am satisfied with the peer program facilities,
such as the condition and layout of the rooms and
building.

4. Overall, the program services are useful to me.
FACIT Results & Analysis
FACIT Results: Year One
                         Comparison of Domain Scores to National Benchmark
              50

              45
                                                                           Yr 1   Benchmark

              40

              35

              30
Total Score




              25

              20

              15

              10

               5

               0
                   Structure   EnvironmentBelief Systems
                                                       Peer Support Education     Advocacy
                                                     Domain
FACIT Analysis
    “Red” Analysis: 2+ below benchmark,
            Areas of deficiency
 “Yellow” Analysis: 1-1.99 below benchmark,
            Close, but not quite
         “Green” Analysis: Proficient
“Purple” Analysis: 1 or more above benchmark,
                   Excellence
"Red" Scored Data                      Year 1
        5
                                                              Benchmark
       4.5
        4
       3.5
        3
MEAN




       2.5
        2
       1.5
        1
       0.5
        0
              1.1.1 Board    1.1.4 Budget Control 1.1.3 Hiring Decisions
             Participation
                                    ITEM
Red Analysis & Recommendations
STRUCTURE DOMAIN:
• Board Participation
• Hiring Decisions
• Budget Control
“Yellow” Scored Data
  6

  5
                                                                                           Year 1   Benchmark


  4
MEAN




  3

  2

  1

  0
        1.3.2 Linkage to   3.3.1 Personal   1.1.2 Consumer Staff   3.5 Recovery   4.4.2 Informal Crisis
       Other Consumer-     Empowerment                                                 Prevention
       Operated Services

                                                  ITEM
Yellow Analysis & Recommendations
STRUCTURE DOMAIN
• Consumer Staff
• Linkages with other Consumer-Operated Srvcs
BELIEF SYSTEMS DOMAIN
• Personal Empowerment
• Recovery
PEER SUPPORT DOMAIN
• Informal Crisis Prevention
“Purple” Scored Data
       6
                                                                              Year 1
       5                                                                      Benchmark

       4
MEAN




       3

       2

       1

       0
           3.4 Choice 3.7 Spiritual Growth
                                     1.3.1 Linkage with TMHS of Coerciveness Other Service Agen
                                                    2.2.1 Lack
                                                             1.3.3 Linkage with
                                            ITEM
Purple Analysis & Recommendations
STRUCTURE DOMAIN
• Linkage with Other Service Agencies
• Linkage with Traditional Mental Health Services
ENVIRONMENT DOMAIN
• Lack of coerciveness
BELIEF SYSTEMS
• Choice
• Spiritual Growth
EDUCATION DOMAIN
• Formal curriculums in problem-solving and self-
  management
POP
Results & Analysis
POP Results
• Mean: Goals met in all categories except
  meeting employment needs of participants
• Median: “Staff threatens me with loss of
  housing” is irrelevant to services at PS, had a
  median indicating “not applicable”
• Mode: Coercion scale uniformly positive with
  “5” as mode – best score possible.
Project Limitations
FACIT
• Internal review only
• Performed over several weeks
• No internal comparison (1st time)

POP
• Interviewer error
• Small sample size
• No benchmark data available
Recommendations: FACIT
• “Red” Recommendations
  – Continue separation plan
  – Training
• “Yellow” Recommendations
  – Training
  – Formal crisis prevention process
• “Purple” Recommendations
  – Monitor environment
  – Reach out
Recommendations: POP
• Employment assistance referral plan
• Community Resources manual
• Monitoring
Implementation Plan
• Establish dedicated quality improvement team
  from peers
• Develop key indicators to monitor which are
  relevant to mission and applicable to funding
  requests
• Small cycles of change – path of least
  resistance to start
• Additional training on CQI
• Inform stakeholders of results
Benefits to Participation Station
•   Usable data for grant applications
•   Evidence-based practices
•   Training needs outlined
•   Small cycles of change start with the
    recommendations
Useful Courses
•   HA601           •   HA660
•   HA602           •   HA673
•   HA603           •   HA711
•   HA604           •   HA785
•   HA621          •    CPH758
•   HA623
•   HA642
Martha Riddell, MPH, DrPH, Chair
Andrew Johnson, PhD, Committee
 Sarah Wackerbarth, PhD, Reader
    David Riggsby, Practitioner
Questions/Comments

Weitere ähnliche Inhalte

Ähnlich wie Participation Station: Health Administration Capstone Presentation

Testing Your Testing Program
Testing Your Testing ProgramTesting Your Testing Program
Testing Your Testing ProgramJill Martay
 
CPHA 2014: Partnerships for Health System Improvement
CPHA 2014: Partnerships for Health System ImprovementCPHA 2014: Partnerships for Health System Improvement
CPHA 2014: Partnerships for Health System ImprovementHealth Evidence™
 
Impact of Work life balancing (WLB) practices on employee performance
Impact of Work life balancing (WLB) practices on employee performanceImpact of Work life balancing (WLB) practices on employee performance
Impact of Work life balancing (WLB) practices on employee performanceKajan Muneeswaran
 
Is foundations approach that assesse, evaluate and improve volunteering initi...
Is foundations approach that assesse, evaluate and improve volunteering initi...Is foundations approach that assesse, evaluate and improve volunteering initi...
Is foundations approach that assesse, evaluate and improve volunteering initi...Khaled Alanzi - 6 Sigma Green Belt
 
Performance Improvement Approach Orientation
Performance  Improvement Approach OrientationPerformance  Improvement Approach Orientation
Performance Improvement Approach OrientationIsaac Munene Ntwiga,MPH
 
Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010
Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010
Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010TEST Huddle
 
Positive Youth Development (PYD) Framework toolkit application modules - Dr B...
Positive Youth Development (PYD) Framework toolkit application modules - Dr B...Positive Youth Development (PYD) Framework toolkit application modules - Dr B...
Positive Youth Development (PYD) Framework toolkit application modules - Dr B...Khulisa Management Services
 
Training Program Evaluation
Training Program EvaluationTraining Program Evaluation
Training Program EvaluationLaura Pasquini
 
How to Implement Quality in Health Care Organizations.
How to Implement Quality in Health Care Organizations.How to Implement Quality in Health Care Organizations.
How to Implement Quality in Health Care Organizations.Healthcare consultant
 
Mastering the Huddle Office of Lean Transformation
Mastering the Huddle Office of Lean TransformationMastering the Huddle Office of Lean Transformation
Mastering the Huddle Office of Lean TransformationAmanda Gilmore
 
Management and managing aspects of a wellness service in a health system
Management and managing aspects of a wellness service in a health systemManagement and managing aspects of a wellness service in a health system
Management and managing aspects of a wellness service in a health systemSEJOJO PHAAROE
 
Value Summary Online Improvement Portal: Product Overview
Value Summary Online Improvement Portal: Product OverviewValue Summary Online Improvement Portal: Product Overview
Value Summary Online Improvement Portal: Product OverviewUniversity of Utah
 
ITFT Strategic Management
ITFT Strategic ManagementITFT Strategic Management
ITFT Strategic Managementarchan26
 
Linda Dulye IABC 2010 Global Conference Presentation
Linda Dulye IABC 2010 Global Conference PresentationLinda Dulye IABC 2010 Global Conference Presentation
Linda Dulye IABC 2010 Global Conference PresentationDulye
 
Running Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docx
Running Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docxRunning Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docx
Running Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docxtoltonkendal
 

Ähnlich wie Participation Station: Health Administration Capstone Presentation (20)

Testing Your Testing Program
Testing Your Testing ProgramTesting Your Testing Program
Testing Your Testing Program
 
CPHA 2014: Partnerships for Health System Improvement
CPHA 2014: Partnerships for Health System ImprovementCPHA 2014: Partnerships for Health System Improvement
CPHA 2014: Partnerships for Health System Improvement
 
Impact of Work life balancing (WLB) practices on employee performance
Impact of Work life balancing (WLB) practices on employee performanceImpact of Work life balancing (WLB) practices on employee performance
Impact of Work life balancing (WLB) practices on employee performance
 
306 - Performance Management the Death Of the Collegiate system in HE
306 - Performance Management the Death Of the Collegiate system in HE306 - Performance Management the Death Of the Collegiate system in HE
306 - Performance Management the Death Of the Collegiate system in HE
 
Monitoring and Evaluating Scale-Up: Methodological and Programmatic Challenges
Monitoring and Evaluating Scale-Up: Methodological and Programmatic ChallengesMonitoring and Evaluating Scale-Up: Methodological and Programmatic Challenges
Monitoring and Evaluating Scale-Up: Methodological and Programmatic Challenges
 
Vaf volunteer assessment foundation
Vaf volunteer assessment foundationVaf volunteer assessment foundation
Vaf volunteer assessment foundation
 
Is foundations approach that assesse, evaluate and improve volunteering initi...
Is foundations approach that assesse, evaluate and improve volunteering initi...Is foundations approach that assesse, evaluate and improve volunteering initi...
Is foundations approach that assesse, evaluate and improve volunteering initi...
 
Performance Improvement Approach Orientation
Performance  Improvement Approach OrientationPerformance  Improvement Approach Orientation
Performance Improvement Approach Orientation
 
Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010
Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010
Tafline Murnane - The Carrot or The Whip-What Motivates Testers? - EuroSTAR 2010
 
Positive Youth Development (PYD) Framework toolkit application modules - Dr B...
Positive Youth Development (PYD) Framework toolkit application modules - Dr B...Positive Youth Development (PYD) Framework toolkit application modules - Dr B...
Positive Youth Development (PYD) Framework toolkit application modules - Dr B...
 
Training Program Evaluation
Training Program EvaluationTraining Program Evaluation
Training Program Evaluation
 
model-of-human-occ-and-prac-dev-LLL event-London Region-heasman.pdf
model-of-human-occ-and-prac-dev-LLL event-London Region-heasman.pdfmodel-of-human-occ-and-prac-dev-LLL event-London Region-heasman.pdf
model-of-human-occ-and-prac-dev-LLL event-London Region-heasman.pdf
 
How to Implement Quality in Health Care Organizations.
How to Implement Quality in Health Care Organizations.How to Implement Quality in Health Care Organizations.
How to Implement Quality in Health Care Organizations.
 
Mastering the Huddle Office of Lean Transformation
Mastering the Huddle Office of Lean TransformationMastering the Huddle Office of Lean Transformation
Mastering the Huddle Office of Lean Transformation
 
Performance Appraisal
Performance AppraisalPerformance Appraisal
Performance Appraisal
 
Management and managing aspects of a wellness service in a health system
Management and managing aspects of a wellness service in a health systemManagement and managing aspects of a wellness service in a health system
Management and managing aspects of a wellness service in a health system
 
Value Summary Online Improvement Portal: Product Overview
Value Summary Online Improvement Portal: Product OverviewValue Summary Online Improvement Portal: Product Overview
Value Summary Online Improvement Portal: Product Overview
 
ITFT Strategic Management
ITFT Strategic ManagementITFT Strategic Management
ITFT Strategic Management
 
Linda Dulye IABC 2010 Global Conference Presentation
Linda Dulye IABC 2010 Global Conference PresentationLinda Dulye IABC 2010 Global Conference Presentation
Linda Dulye IABC 2010 Global Conference Presentation
 
Running Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docx
Running Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docxRunning Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docx
Running Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docx
 

Participation Station: Health Administration Capstone Presentation

  • 1. Evidence-Based Recommendations for Participation Station Marcie Timmerman Capstone Presentation MHA Candidate, 2012 UK College of Public Health
  • 2. Participation Station MISSION STATEMENT: To provide a program-based environment that educates, empowers and enhances the lives of individuals moving toward recovery and mental wellness.
  • 3. The Problem(s) • Need funding diversification: grant applications • Need to “prove” their success to date • Need to convince other agencies to collaborate for advocacy and mission • Pursuing internal continuous quality improvement
  • 4. Project Purpose • Analysis of program evaluation and outcomes data collected Sept-Dec 2011 • Recommendations for improvement based on that analysis
  • 5. Literature Review • Gained understanding of models and ideologies which led to PS creation • Learned vocabulary and theories behind tools questions • Little to no available measurement tools, peer-reviewed “best practices”, etc. for services like PS in literature
  • 6. Methods: FACIT • Fidelity Assessment Common Ingredients Tool (FACIT) from SAMHSA with benchmark data – Structured Observation – Structured item-by-item questionnaire Evaluations could be internal or external. • Internal only • Interviewed 12 leaders, 75% of whom were peers
  • 7. Example of FACIT Survey Question 1.1.3 “Hiring (1)Consumers are not involved in any Decisions hiring decisions (2) Consumers have some involvement in hiring decisions (3) Consumers are responsible for making most of the hiring decisions (50% or more). (4) Consumers are responsible for making all hiring decisions.
  • 8. Methods: POP • Peer Outcomes Protocol (POP) – Structured questionnaire with anchored scale responses for each participant • 1 for “Disagree” • 2 for “Somewhat Disagree” • 3 for “Somewhat Agree” • 4 for “Agree” – Last 10 questions “Coercion Scale” • Anchored 1-5 with “5” as best – Some questions worded so that the “4” is bad and “1” is good
  • 9. Examples of POP Questions 1. Overall, I am satisfied with this peer support program. 2. I am satisfied with the peer program facilities, such as the condition and layout of the rooms and building. 4. Overall, the program services are useful to me.
  • 10. FACIT Results & Analysis
  • 11. FACIT Results: Year One Comparison of Domain Scores to National Benchmark 50 45 Yr 1 Benchmark 40 35 30 Total Score 25 20 15 10 5 0 Structure EnvironmentBelief Systems Peer Support Education Advocacy Domain
  • 12. FACIT Analysis “Red” Analysis: 2+ below benchmark, Areas of deficiency “Yellow” Analysis: 1-1.99 below benchmark, Close, but not quite “Green” Analysis: Proficient “Purple” Analysis: 1 or more above benchmark, Excellence
  • 13. "Red" Scored Data Year 1 5 Benchmark 4.5 4 3.5 3 MEAN 2.5 2 1.5 1 0.5 0 1.1.1 Board 1.1.4 Budget Control 1.1.3 Hiring Decisions Participation ITEM
  • 14. Red Analysis & Recommendations STRUCTURE DOMAIN: • Board Participation • Hiring Decisions • Budget Control
  • 15. “Yellow” Scored Data 6 5 Year 1 Benchmark 4 MEAN 3 2 1 0 1.3.2 Linkage to 3.3.1 Personal 1.1.2 Consumer Staff 3.5 Recovery 4.4.2 Informal Crisis Other Consumer- Empowerment Prevention Operated Services ITEM
  • 16. Yellow Analysis & Recommendations STRUCTURE DOMAIN • Consumer Staff • Linkages with other Consumer-Operated Srvcs BELIEF SYSTEMS DOMAIN • Personal Empowerment • Recovery PEER SUPPORT DOMAIN • Informal Crisis Prevention
  • 17. “Purple” Scored Data 6 Year 1 5 Benchmark 4 MEAN 3 2 1 0 3.4 Choice 3.7 Spiritual Growth 1.3.1 Linkage with TMHS of Coerciveness Other Service Agen 2.2.1 Lack 1.3.3 Linkage with ITEM
  • 18. Purple Analysis & Recommendations STRUCTURE DOMAIN • Linkage with Other Service Agencies • Linkage with Traditional Mental Health Services ENVIRONMENT DOMAIN • Lack of coerciveness BELIEF SYSTEMS • Choice • Spiritual Growth EDUCATION DOMAIN • Formal curriculums in problem-solving and self- management
  • 20. POP Results • Mean: Goals met in all categories except meeting employment needs of participants • Median: “Staff threatens me with loss of housing” is irrelevant to services at PS, had a median indicating “not applicable” • Mode: Coercion scale uniformly positive with “5” as mode – best score possible.
  • 21. Project Limitations FACIT • Internal review only • Performed over several weeks • No internal comparison (1st time) POP • Interviewer error • Small sample size • No benchmark data available
  • 22. Recommendations: FACIT • “Red” Recommendations – Continue separation plan – Training • “Yellow” Recommendations – Training – Formal crisis prevention process • “Purple” Recommendations – Monitor environment – Reach out
  • 23. Recommendations: POP • Employment assistance referral plan • Community Resources manual • Monitoring
  • 24. Implementation Plan • Establish dedicated quality improvement team from peers • Develop key indicators to monitor which are relevant to mission and applicable to funding requests • Small cycles of change – path of least resistance to start • Additional training on CQI • Inform stakeholders of results
  • 25. Benefits to Participation Station • Usable data for grant applications • Evidence-based practices • Training needs outlined • Small cycles of change start with the recommendations
  • 26. Useful Courses • HA601 • HA660 • HA602 • HA673 • HA603 • HA711 • HA604 • HA785 • HA621 • CPH758 • HA623 • HA642
  • 27. Martha Riddell, MPH, DrPH, Chair Andrew Johnson, PhD, Committee Sarah Wackerbarth, PhD, Reader David Riggsby, Practitioner

Hinweis der Redaktion

  1. Participation Station (PS) is a peer-operated, drop-in mental health services center offering a variety of programming free of charge to the public. Their mission statement says they are there: To provide a program based environment that educates, empowers and enhances the lives of individuals moving toward recovery and mental wellness A funded program of the National Alliance for Mental Illness Lexington and is partially funded by Bluegrass Mental Health Mental Retardation Board Inc. Staffed by volunteers and part-time employees and short on resources as well as experience, they asked me to solve a problem.
  2. That problem was actually several problems.They needed to diversify funding to promote sustainability and growth, as well as gain legitimacy in the community.“Prove” success after nearly 2 years of workConvince other agencies to collaborate on advocacy issues and the shared mission of recoveryWanted to begin pursuing internal CQI before a non-quality culture was established
  3. Practices and places like Participation Station are relatively new. The variety of services offered by similarly minded organizations complicates attempts at traditional research. Oftentimes, researchers looking at similar venues would not be comparing apples to apples but would be comparing fruit salad to fruit salad. As a result, my literature review ended up focusing on the ideologies and ethics involved in the Recovery Movement and Consumer Movement – the foundations upon which PS was built. I was able to build a vocabulary and understand the technical jargon within the measurement tools which aided in my analysis.
  4. Structured individual interviewInterviewer must be a “peer”Responses recorded by participantProgram Satisfaction Module is one of many that are offered as part of the protocol; choosing one module is acceptable and this module offered the most appropriate questions for what they are measuring. 1-4 for the first 40 questionsLast 10 questions included a coercion scale measure that had been included in the POP with permission from its creator
  5. FACIT covers 6 domains of proficiency: Structure, Environment, Belief Systems, Peer Support, Education & AdvocacyPS was at or near benchmarks in 3:Environment, Belief Systems, Education and behind in 3 : Structure, Peer Support and Advocacy
  6. Calculated mean of responses from PSCompared those to the benchmarked responses for each itemUsing the standard deviation from the benchmark data as a constant,Established a color-coded analysis.Items labeled “red”…Items labeled “yellow”…Items labeled “green”…Items labeled “purple”…
  7. These deficiencies are universally related to the fact that Participation Station is a program of NAMI Lexington. They are technically overseen by the NAMI Lexington Board of Directors, meaning that their board is actually the NAMI Lex board – comprised of family members, practitioners, and other interested parties who are not “peers”. Hiring decisions & budget decisions are with NAMI Lexington’s management for liability reasons – they aren’t a separate entity and can’t have their own liability insurance, etc. so NAMI Lexington currently serves that role.
  8. The line items here each reflect the newness of the program, its relationship with NAMI Lexington, and its uniqueness in the state of Kentucky. Training deficiencies account for 4 of the 5 items listed here. Consumer Staff is a reflection of being a NAMI programThere were no other consumer-operated service centers at the time of the FACITPersonal Empowerment & Recovery reflect a need for training of volunteers & staff in these areas. Informal Crisis Prevention is a call for standardization which also translates to training first and foremost.
  9. Establishing excellence in some areas allows PS to shine on grant applications, influences program reviews by stakeholders, and expresses their benefit to the general community. In this instance, these are areas where internally the program’s leaders feel they are doing really well; they are areas to highlight, search for grants/awards in, and to continue tweaking so they can meet their goal of being a “flagship” peer support institution.
  10. Team decided to keep track of individual responses so they could look at them in the future for individual outcomes measures as well.12 individuals took the POP before January 1, 2012. All were peers and had been coming to PS for at least 3 visits.
  11. The overall results of the POP were positive. There were not many deviations from positive results on an individual basis.The Mean of all responses showed the employment issue.Median illustrated the fact that PS does not cover housing issuesIllustrates a clear lack of coercion on part of PS’s program leaders.
  12. Recommendation: Engage an external evaluator as soon as you can!Continue performing FACIT on an annual or bi-annual basis to keep data current.
  13. “Red” Recommendations Continue process of formally separating from NAMI Lexington, so peer leadership team is the top of org chartTrain leaders on more executive functions, so they are ready to take over “Yellow” RecommendationsTraining staff & volunteers in recovery principles, empowerment principles and engaging community in their missionEstablish a formal crisis prevention process“Purple” RecommendationsMonitor environment regularly to ensure non-coercionReach out to other community agenciesReach out to more traditional mental health practices
  14. Structure a plan for referral of those needing employment assistanceHave information available on other services in the communityContinue to monitor measures that relate to the mission and values of Participation Station