Running Head REPLY TO OPINION 5.1 FOR KIMBRILEE SCHMITZ 1REPLY.docx
Participation Station: Health Administration Capstone Presentation
1. Evidence-Based Recommendations
for Participation Station
Marcie Timmerman
Capstone Presentation
MHA Candidate, 2012
UK College of Public Health
2. Participation Station
MISSION STATEMENT: To provide a program-based
environment that educates, empowers and enhances
the lives of individuals moving toward recovery and
mental wellness.
3. The Problem(s)
• Need funding diversification: grant
applications
• Need to “prove” their success to date
• Need to convince other agencies to
collaborate for advocacy and mission
• Pursuing internal continuous quality
improvement
4. Project Purpose
• Analysis of program evaluation and outcomes
data collected Sept-Dec 2011
• Recommendations for improvement based on
that analysis
5. Literature Review
• Gained understanding of models and
ideologies which led to PS creation
• Learned vocabulary and theories behind tools
questions
• Little to no available measurement tools,
peer-reviewed “best practices”, etc. for
services like PS in literature
6. Methods: FACIT
• Fidelity Assessment Common Ingredients Tool
(FACIT) from SAMHSA with benchmark data
– Structured Observation
– Structured item-by-item questionnaire
Evaluations could be internal or external.
• Internal only
• Interviewed 12 leaders, 75% of whom were peers
7. Example of FACIT Survey Question
1.1.3 “Hiring (1)Consumers are not involved in any
Decisions hiring decisions
(2) Consumers have some involvement in
hiring decisions
(3) Consumers are responsible for making
most of the hiring decisions (50% or more).
(4) Consumers are responsible for making
all hiring decisions.
8. Methods: POP
• Peer Outcomes Protocol (POP)
– Structured questionnaire with anchored scale
responses for each participant
• 1 for “Disagree”
• 2 for “Somewhat Disagree”
• 3 for “Somewhat Agree”
• 4 for “Agree”
– Last 10 questions “Coercion Scale”
• Anchored 1-5 with “5” as best
– Some questions worded so that the “4” is bad and
“1” is good
9. Examples of POP Questions
1. Overall, I am satisfied with this peer support
program.
2. I am satisfied with the peer program facilities,
such as the condition and layout of the rooms and
building.
4. Overall, the program services are useful to me.
11. FACIT Results: Year One
Comparison of Domain Scores to National Benchmark
50
45
Yr 1 Benchmark
40
35
30
Total Score
25
20
15
10
5
0
Structure EnvironmentBelief Systems
Peer Support Education Advocacy
Domain
12. FACIT Analysis
“Red” Analysis: 2+ below benchmark,
Areas of deficiency
“Yellow” Analysis: 1-1.99 below benchmark,
Close, but not quite
“Green” Analysis: Proficient
“Purple” Analysis: 1 or more above benchmark,
Excellence
13. "Red" Scored Data Year 1
5
Benchmark
4.5
4
3.5
3
MEAN
2.5
2
1.5
1
0.5
0
1.1.1 Board 1.1.4 Budget Control 1.1.3 Hiring Decisions
Participation
ITEM
14. Red Analysis & Recommendations
STRUCTURE DOMAIN:
• Board Participation
• Hiring Decisions
• Budget Control
15. “Yellow” Scored Data
6
5
Year 1 Benchmark
4
MEAN
3
2
1
0
1.3.2 Linkage to 3.3.1 Personal 1.1.2 Consumer Staff 3.5 Recovery 4.4.2 Informal Crisis
Other Consumer- Empowerment Prevention
Operated Services
ITEM
16. Yellow Analysis & Recommendations
STRUCTURE DOMAIN
• Consumer Staff
• Linkages with other Consumer-Operated Srvcs
BELIEF SYSTEMS DOMAIN
• Personal Empowerment
• Recovery
PEER SUPPORT DOMAIN
• Informal Crisis Prevention
17. “Purple” Scored Data
6
Year 1
5 Benchmark
4
MEAN
3
2
1
0
3.4 Choice 3.7 Spiritual Growth
1.3.1 Linkage with TMHS of Coerciveness Other Service Agen
2.2.1 Lack
1.3.3 Linkage with
ITEM
18. Purple Analysis & Recommendations
STRUCTURE DOMAIN
• Linkage with Other Service Agencies
• Linkage with Traditional Mental Health Services
ENVIRONMENT DOMAIN
• Lack of coerciveness
BELIEF SYSTEMS
• Choice
• Spiritual Growth
EDUCATION DOMAIN
• Formal curriculums in problem-solving and self-
management
20. POP Results
• Mean: Goals met in all categories except
meeting employment needs of participants
• Median: “Staff threatens me with loss of
housing” is irrelevant to services at PS, had a
median indicating “not applicable”
• Mode: Coercion scale uniformly positive with
“5” as mode – best score possible.
21. Project Limitations
FACIT
• Internal review only
• Performed over several weeks
• No internal comparison (1st time)
POP
• Interviewer error
• Small sample size
• No benchmark data available
22. Recommendations: FACIT
• “Red” Recommendations
– Continue separation plan
– Training
• “Yellow” Recommendations
– Training
– Formal crisis prevention process
• “Purple” Recommendations
– Monitor environment
– Reach out
24. Implementation Plan
• Establish dedicated quality improvement team
from peers
• Develop key indicators to monitor which are
relevant to mission and applicable to funding
requests
• Small cycles of change – path of least
resistance to start
• Additional training on CQI
• Inform stakeholders of results
25. Benefits to Participation Station
• Usable data for grant applications
• Evidence-based practices
• Training needs outlined
• Small cycles of change start with the
recommendations
Participation Station (PS) is a peer-operated, drop-in mental health services center offering a variety of programming free of charge to the public. Their mission statement says they are there: To provide a program based environment that educates, empowers and enhances the lives of individuals moving toward recovery and mental wellness A funded program of the National Alliance for Mental Illness Lexington and is partially funded by Bluegrass Mental Health Mental Retardation Board Inc. Staffed by volunteers and part-time employees and short on resources as well as experience, they asked me to solve a problem.
That problem was actually several problems.They needed to diversify funding to promote sustainability and growth, as well as gain legitimacy in the community.“Prove” success after nearly 2 years of workConvince other agencies to collaborate on advocacy issues and the shared mission of recoveryWanted to begin pursuing internal CQI before a non-quality culture was established
Practices and places like Participation Station are relatively new. The variety of services offered by similarly minded organizations complicates attempts at traditional research. Oftentimes, researchers looking at similar venues would not be comparing apples to apples but would be comparing fruit salad to fruit salad. As a result, my literature review ended up focusing on the ideologies and ethics involved in the Recovery Movement and Consumer Movement – the foundations upon which PS was built. I was able to build a vocabulary and understand the technical jargon within the measurement tools which aided in my analysis.
Structured individual interviewInterviewer must be a “peer”Responses recorded by participantProgram Satisfaction Module is one of many that are offered as part of the protocol; choosing one module is acceptable and this module offered the most appropriate questions for what they are measuring. 1-4 for the first 40 questionsLast 10 questions included a coercion scale measure that had been included in the POP with permission from its creator
FACIT covers 6 domains of proficiency: Structure, Environment, Belief Systems, Peer Support, Education & AdvocacyPS was at or near benchmarks in 3:Environment, Belief Systems, Education and behind in 3 : Structure, Peer Support and Advocacy
Calculated mean of responses from PSCompared those to the benchmarked responses for each itemUsing the standard deviation from the benchmark data as a constant,Established a color-coded analysis.Items labeled “red”…Items labeled “yellow”…Items labeled “green”…Items labeled “purple”…
These deficiencies are universally related to the fact that Participation Station is a program of NAMI Lexington. They are technically overseen by the NAMI Lexington Board of Directors, meaning that their board is actually the NAMI Lex board – comprised of family members, practitioners, and other interested parties who are not “peers”. Hiring decisions & budget decisions are with NAMI Lexington’s management for liability reasons – they aren’t a separate entity and can’t have their own liability insurance, etc. so NAMI Lexington currently serves that role.
The line items here each reflect the newness of the program, its relationship with NAMI Lexington, and its uniqueness in the state of Kentucky. Training deficiencies account for 4 of the 5 items listed here. Consumer Staff is a reflection of being a NAMI programThere were no other consumer-operated service centers at the time of the FACITPersonal Empowerment & Recovery reflect a need for training of volunteers & staff in these areas. Informal Crisis Prevention is a call for standardization which also translates to training first and foremost.
Establishing excellence in some areas allows PS to shine on grant applications, influences program reviews by stakeholders, and expresses their benefit to the general community. In this instance, these are areas where internally the program’s leaders feel they are doing really well; they are areas to highlight, search for grants/awards in, and to continue tweaking so they can meet their goal of being a “flagship” peer support institution.
Team decided to keep track of individual responses so they could look at them in the future for individual outcomes measures as well.12 individuals took the POP before January 1, 2012. All were peers and had been coming to PS for at least 3 visits.
The overall results of the POP were positive. There were not many deviations from positive results on an individual basis.The Mean of all responses showed the employment issue.Median illustrated the fact that PS does not cover housing issuesIllustrates a clear lack of coercion on part of PS’s program leaders.
Recommendation: Engage an external evaluator as soon as you can!Continue performing FACIT on an annual or bi-annual basis to keep data current.
“Red” Recommendations Continue process of formally separating from NAMI Lexington, so peer leadership team is the top of org chartTrain leaders on more executive functions, so they are ready to take over “Yellow” RecommendationsTraining staff & volunteers in recovery principles, empowerment principles and engaging community in their missionEstablish a formal crisis prevention process“Purple” RecommendationsMonitor environment regularly to ensure non-coercionReach out to other community agenciesReach out to more traditional mental health practices
Structure a plan for referral of those needing employment assistanceHave information available on other services in the communityContinue to monitor measures that relate to the mission and values of Participation Station