The PACE Community is engaged in driving forward improvement for looked after children, and recently hosted a PACE Data Community Event, Milestones, Measures & Aims. How to incorporate the Data 'ask' into the system which looked at how data can practically be captured and analysed to drive and evidence improvement in achievement of early permanence for our looked after infants, children, and young people.
The aim of the session was to look specifically at the key milestones on the journey to permanence and the data issues that these pose. We learned from examples of how local authority partnership areas have sought to address and overcome particular data issues, and shared general learning from CELCIS resulting from implementation of the PACE programme.
Sequential and reinforcement learning for demand side management by Margaux B...
PACE Data Community Workshop. 23rd January 2019. Milestones, measures and aims: How to incorporate the Data ask into the system
1. celcis.org @CELCISTweets
Wednesday 23rd January 2019
CELCIS
Practice Exchange Workshop
Permanence and Care Excellence Programme (PACE)
Milestones, measures and aims:
How to incorporate the Data ‘ask’ into the system
6. www.celcis.org www.celcis.org
Key milestones for children/ young who are looked after and become
accommodated away from home :
Milestones - Data
Looked After
Start Date
Looked After &
Accommodated
Review
Recommendation
Date
Agency
Decision
Maker
Sign Off
Order
Granted
Date
Application
Lodged at
Court
Aim 2 Aim 3 Aim 4
Last
Accommodated
Date Away
from Home
7. www.celcis.org www.celcis.org
Aim 2
Children who become
Looked after & accommodated
will have a
recommendation for
Permanence (including a
permanent return home within
6 months of becoming looked
after and accommodated
Aim 3
Children who have had
a recommendation for
permanence away from
home will have the decision
ratified by the Agency
Decision Maker within 14
week of the Looked After
Review that
recommended permanence.
Aim 4
Children whose plan has
been approved by the
Agency Decision their
application for a legal
order lodged
within 3 weeks of
Agency Decision Maker
agreement with the
recommendation
Aim 1
Children who
have been looked
after at home for
more than 2 years
will have a 24 month
review looking at their
whole period of time
being looked after at
home on a CSO.
National Aims
8. celcis.org @CELCISTweets
Exercise:
Construct a measureable Aim
What definitional issues are there?
• Are the dates accessible?
• How and where is it recorded?
How would you measure the aim?
• Does it apply to all looked after children?
• Does the aim cover all placement types?
9. celcis.org @CELCISTweets
Date child/young person became looked after (start of current episode of care)
Date child was first accommodated away from home (first placement in current episode)
Date child was last accommodated away from home (start of latest period of
accommodation)
LAC review (or other meeting) recommends pursuing permanence at home or away from
home for child
Date of Fostering/Adoption/Permanence/Kinship Panel
Agency Decision Maker approval of decision to pursue permanence for child
Date of move to (intended) final placement (even if not yet legally secured)
Date of application submission to court
Date that legal permanence is secured for child, including termination of order when
permanence at home is achieved
Data Requirements - Definitions
10. celcis.org @CELCISTweets
Author:
Slide:
Administration 10
First Accommodated Date:
Home FC FC Residential
In this example it would be the date relating to the first FC placement
Last Accommodated Date:
Home FC Home FC Residential
In this example the date for last accommodated would be the second FC
placement
Accommodated Date Example
13. celcis.org @CELCISTweets
Examples from Practice
‘Capture & Extraction’
Nicola Milne
Performance & Evaluation Support Officer Education & Social Care
Moray Council
Stuart Osborne
Performance & Business Analyst
City of Edinburgh Council
15. Aim 1:
By 30 September 2018, 100% of children
looked after at home for more than two
years from 1 June 2018, will undergo a
peer review that will be reported to the
Service & Team Managers.
16. AIM 1
By 30 September 2018, 100% of children looked after at home for
more than 2 years from 1 June 2018, will undergo a peer review
that will be reported to the Service & Team Managers.
Date CSO at
home granted
18 Month
Reminder
2 YEAR
DEADLINE
Date CSO at
home
terminated
12-Jun-18 11-Dec-19 11-Jun-20
18-Jun-18 17-Dec-19 17-Jun-20
16-Oct-18 15-Apr-20 15-Oct-20
28-Jun-18 27-Dec-19 27-Jun-20 17-Oct-18
17. Aim 2:
By 1 July 2019, 90% of looked after &
accommodated children, accommodated
on or after 1 June 2018, will have a
recommendation for permanence within
30 weeks.
18. AIM 2
By 1 July 2019, 90% of looked after & accommodated children, accommodated on or
after 1 June 2018, will have a recommendation for permanence within 30 weeks.
KEY
MILESTONE
Date became
Accommodate
d
DEADLINE
Date Permanence Decision
due
RED - Overdue
AMBER - Due in next 30
days
GREEN - Decision made
KEY MILESTONE
Date of
permanence
recommendation
Weeks from
Accommodation to
recommendation
for Permanence
01-May-18 30-Oct-18 24-Oct-18 25
04-Jan-18 05-Jul-18
30-Jul-18 28-Jan-19
19. Aim 3:
By 1 July 2019, 60% of children who have
had a recommendation for permanence
away from home from 1 June 2018, will be
presented to the Fostering & Adoption
Panel within 15 weeks.
20. AIM 3
By 1 July 2019, 60% of children who have had a recommendation for permanence
away from home from 1 June 2018, will be presented to the Fostering & Adoption
Panel within 15 weeks.
DEADLINE
Date should be
presented to
Panel
RED - Overdue
AMBER - Due in
next 30 days
GREEN - Been to
Panel
Weeks
overdue
to Panel
Proposed
Permanence
Plan
KEY MILESTONE
Panel Date
(or LAC Review date
where decision
made that Panel no
longer appropriate -
in YELLOW)
Weeks from
Permanence
Decision to
Plan being
presented to
Panel
KEY
MILESTONE
Date ADM
Approved
Plan
Panel Admin
KEY
MILESTONE
Weeks to
ADM
approval from
date of
recommendat
ion for
Permanence
09-Jun-09 494 06-May-09 7 19-May-09 9
02-Dec-11 367 02-May-12 33 16-May-12 35
21. Aim 4:
By 1 July 2019, 60% of children from 1 June
2018, who have an agency approved
permanence plan to be secured via a
Permanence Order, will have the order
lodged within 16 weeks of the agency
decision date.
22. AIM 4
By 1 July 2019, 60% of children from 1 June 2018, who have an agency approved
permanence plan to be secured via a Permanence Order, will have the order lodged
within 16 weeks of the agency decision date.
Date
ADM
approv
ed
Match
DEADLINE
Date should
be in final
placement
RED -
Overdue
AMBER -
Due in next
30 days
GREEN - In
placement
KEY
MILEST
ONE
Date in
Final
Placeme
nt
KEY
MILEST
ONE
Weeks to
Final
Placeme
nt from
ADM
approval
KEY
MILESTONE
Weeks from
Accommoda
tion to Final
Placement
KEY
MILESTO
NE -
Date
Lodged
in Court
KEY
MILESTO
NE
Date
legal
order
granted
KEY
MILESTO
NE
Weeks
from
Lodging
in Court
to Order
being
granted
Final
Placeme
nt Type
07-Feb-19
07-Feb-19
19-Jun-
17
05-Oct-17 21-May-
15
-71 0 17-Mar-17 18-Apr-18 56 Permane
nt
Fostering
23. Issues
• Data spread throughout department
• Two separate departments in one
• Interrogating systems - sheer size of systems
• Culture change
• Lack of consistency
24. Plus points
• Better understanding of our processes
• Can try new ideas on a small scale
• Focus on process helps improve it overall
• Better understanding of the importance of
evidencing what we do
26. City of Edinburgh Council
Recording of permanence recommendation
915 Looked After and Accommodated
376 LAC at home
single review team (12 FTE) managing all LAAC reviews and CPCCs
SWIFT
LAAC review types - core three:
'1st review’ six weeks from accommodation
'2nd review’ three months from 1st (and so 4.5 months from accommodation)
'subsequent review’ three months from 2nd (and so 7.5 months from accommodation) or
ongoing six month or annual
Initial thoughts were to record recommendation in 'Outcome' field of review for 1st,
2nd and any subsequent review until recommendation made.
Also only for those in focus for PACE in Edinburgh, i.e. those under 12, and from a
point in time forward.
27. City of Edinburgh Council
Recording of permanence recommendation
Discussion with review team regarding purpose, process, outcome options, guidance
Six options for Outcome field agreed:
• Further assessment required
• Decision previously made
• Not required, other order in place
• Required, referral to panel
• Required, no referral to panel
• Rehabilitation
Conclusion was that, for a number of reasons (reporting, DQ, consistency) the Outcome
field would be populated for every LAAC review going forward
AND a one off exercise to populate the appropriate review for every current LAAC where
the recommendation had been made
Exercise on 22 November 2017, full day commitment from team, just over 1,000 cases
Overall this has been successful and provided a fairly solid base for moving forward
with PACE work
30. Sometimes called trend or time-
series charts
X axis: Sequential occurrence
of event
Y axis: Measure
Median: Mid-point of data
What is a run chart?
32. Let’s make one by hand…
(me first, then your turn,
- there is a prize for the best)
33. Understand variation in data
Simple way to make performance
visible
Show the impact of change:
- is change an improvement?
- is improvement sustained?
Why use a run chart?
34. Data for improvement, not data
for accountability
Use for:
individual tests of change
effects of tests of change
Why use a run chart?
35. Use your eyes, what can you see?
Probability based non-random
signals of change
Four rules:
• A shift
• A trend
• Too many or too few runs
• An astronomical data point
Reading a run chart
41. 5 or more consecutive points increasing or decreasing
63
0
20
40
60
80
100
3/12/2015
5/12/2015
5/12/2015
5/24/2015
6/2/2015
6/15/2015
6/29/2015
6/29/2015
10/27/2015
2/15/2016
6/30/2016
8/31/2016
11/2/2016
11/2/2016
11/2/2016
1/30/2017
1/30/2017
2/9/2017
2/9/2017
6/9/2017
6/9/2017
6/13/2017
6/26/2017
9/7/2017
9/7/2017
6/29/2018
Date of permanence recommendation
Last accommodated to permanence recommendation
Median
Weeks
Trend
42. Too many or too few crossings of the median line is non-random
2 crossings here = 3 runs
28.5
0
20
40
60
80
100
3/12/2015
5/12/2015
5/12/2015
5/24/2015
6/2/2015
6/15/2015
6/29/2015
6/29/2015
10/27/2015
2/15/2016
6/30/2016
8/31/2016
11/2/2016
11/2/2016
11/2/2016
1/30/2017
1/30/2017
2/9/2017
2/9/2017
6/9/2017
6/9/2017
6/13/2017
6/26/2017
9/7/2017
9/7/2017
6/29/2018
Date of permanence recommendation
Last accommodated to permanence recommendation
Median
Weeks
Number of runs
43. Unusually large or small data point (wtf!)
39.5
0
50
100
150
200
250
3/12/2015
5/12/2015
5/12/2015
5/24/2015
6/2/2015
6/15/2015
6/29/2015
6/29/2015
10/27/2015
2/15/2016
6/30/2016
8/31/2016
11/2/2016
11/2/2016
11/2/2016
1/30/2017
1/30/2017
2/9/2017
2/9/2017
6/9/2017
6/9/2017
6/13/2017
6/26/2017
9/7/2017
9/7/2017
6/29/2018
Date of permanence recommendation
Last accommodated to permanence recommendation
Median
Weeks
Astronomical data point
57. celcis.org @CELCISTweets
What we’ve learned – Q&A session
• Feedback to the Government
• New Information Management System
• Creation of Data Community
59. celcis.org @CELCISTweets
PACE Data Community
Would it be useful?
What could you share?
What could you learn from each other?
How could you support each other?
Any topics PACE Data Analysts can cover?
60. celcis.org @CELCISTweets
PACE Data Community
• Purpose
• Infrastructure, functionality
• Moderator
• Social Media Platform
• What is focus
Instructions:
Everyone stand up,
Go to someone you don’t know
Introduce yourself
Ask Question 1
After 2/3 mins can you bring your conversation to a close
Next to go to someone else you do not know
Introduce yourself
Ask Q2, Q3,
Ask everyone to take seat
Q3. What does everyone want to get out of today’s event?
Record on Flipchart the above
So what are Key Milestones for Accommodated Population = Key dates in child’s journey to permanence
One of reason we focus on the key milestones is that these dates are already collated by local authorities for the yearly CLAS return
In groups
Group 1 – Aim 2
Group 2 – Aim 4
Print A3 sheets
Data point of view start and end date
What definitional issues are there?
Are the dates accessible?
How and where is it recorded?
How would you measure the aim?
Prom
Does it apply to all looked after children?
Does the aim cover all placement types?
Outcome: Construct an aim that could be measured
Group Feedback on capturing the milestone dates to defining the Aim.
Capture on flipchart
Reminder:
NO food or drinks allowed in Computer LAB
Median is 50th percentile. Splits data in two
5 is based on experience
Should be between 9 and 19 for 26 data points (only count those not on median)
Way too far…
How we use run charts for PACE. First try and establish a baseline to get a sense of how long things currently take. Need 10 or more data points to calculate median if we want to use probability based rules. Common error is to continue adding data points and not recalculate median.
Add data points as more kids pass milestones. Any signals of change in the data?
11 runs, should be between 9 and 19. Median reduced from 51 to 40. Taken 9 weeks off process, let’s go home…
Shouldn’t really compare medians in this way. They are for really for use within the chart, rather than to compare between them
First shift disappears. Trend still there, and now an additional shift
We are trying to measure things that are in the process of happening. Completely intuitive to order data by start of process, because you’re looking at the next child that comes along. However some of the processes we are looking at are very long, so we capture those passing key milestones as they happen, rather than waiting to look back. Looks like improvement, but it’s an incomplete picture
They’ve dealt with some cases hanging around in system quite recently, so these appear at right of chart
Example from another LA where they’ve set an aim with an accommodatated from date, and 3 to 4 months after that date one child has had a recommendation
New Information Management System
In addition to the main systems( Carefirst, SWIFT, Mosiac, Eclipse) there are other in-house Information Management Systems operating across the 32 Local Authorities.
System Supplier OLM Acquired Northgate which was SWIFT which as of soon they will no longer be supporting.
OLM’s new platform is Eclipse and they are looking to move Carefirst customers as well as SWIFT customers to their new platform Eclipse.
But this requires tendering process so are up against
Liquidlogic - Liquidlogic Children’s System (LCS) and Corelogic - Mosiac.
Liquidlogic as we know has broken into the Scottish market with winning the tender for Falkirk and East Ayrshire.
Q. So what challenges are Local Authorities are facing?
Q What opportunities does potential system change present/ give you?
PACE Data Community - Q. Would it be useful? What could you share? What could you learn from each other? How could you support each other?
Aim: expand their professional knowledge base and share their invaluable insights
Purpose
Determine the goals and objectives that the community of practice hopes to achieve through knowledge sharing.
What is the purpose of creating this data community of practice?
Infrastructure
How will communicate, when and how - hold meetings, etc.
Functionality
How will the group function and what is expected of everyone
Moderator
Appoint someone to help guide discussions.
Social Media Platform
Social media sites as communication tools which can even be used to share files, images, and videos.
What is Focus
How do we decide what we want to focus on, the core discussions - carry out initial survey?