The document presents a framework for understanding statistical performance. It outlines operational and strategic drivers for managing statistics about performance. Operationally, performance is important to outcomes like safety and education. Strategically, there is more data and emphasis on using data for decisions. The framework provides a macro level overview of the data, analysis, insight, and product cycle. It also details an analytical level approach involving a snapshot, trend, benchmark, and target analysis to understand a measure over time and versus comparisons. The goal is to extract insight from data to improve performance.
RSA Conference Exhibitor List 2024 - Exhibitors Data
A Framework for Statistical Performance
1. A Framework for Understanding
Statistical Performance
Paul Askew
CONFERENCE
2-5 SEPTEMBER 2013
NEWCASTLE
2. Outline
1. Introduction
2. Framework – the “Why”
Operational Drivers
Current Strategic Drivers
3. Framework – the “How”
Macro level
Analytical level
3. 1. Introduction
1. Scope….A framework for
Managing Statistics about performance
(rather than performance of statistical techniques)
2. Operational Origins
•
More about practical drivers and process
•
Utility….target setting, performance improvement
3. Distilling application and development across sectors….
•
Criminal justice, regulation, education, health
•
It really matters….safety, housing, education….
5. 1. Introduction
1. Scope….A framework for
Managing Statistics about performance
(rather than performance of statistical techniques)
2. Operational Origins
•
More about practical drivers and process
•
Utility….target setting, performance improvement
3. Distilling application and development across sectors….
•
Criminal justice, regulation, education, health
•
It really matters….safety, housing, education….
6.
7. Outline
1. Introduction
2. Framework – the “Why”
Operational Drivers
Current Strategic Drivers
3. Framework – the “How”
Macro level
Analytical level
8.
9. 2. Why - Operational Drivers
1. It actually matters to people – safety, home, education
2. Performance Regime – broad scope, high profile, deep drill down
3. “Multi-multi” dimensional – both of measures and assessments
4. Statistics meaning – datum, summary, technique
5. Targets - legal, audited, collaborative!
6. Performance Pantomime
7. Less about techniques, more about process
8. Operational Delivery – police, health, regulation…
10. “Burglary is down compared to last month”
“Yes but it’s up compared the same month last year”
“Yes but it’s down overall for the financial year to date”
“Yes but its’ up for the calendar year so far”
“Yes but we’re still less better than our neighbours”
“Yes but they are reducing faster than we are this year”
“Yes but
we’re still under (over) target”.
etc………….
11. 2. Why - Operational Drivers
1. It actually matters to people – safety, home, education
2. Performance Regime – broad scope, high profile, deep drill down
3. “Multi-multi” dimensional – both of measures and assessments
4. Statistics meaning – datum, summary, technique,
5. Targets - legal, audited, collaborative!
6. Performance Pantomime
7. Less about techniques, more about process
8. Operational Delivery – police, health, regulation…
12. Smoothed Data
or Real Data
Smoothed Data
Smoothed Data – 12 month rolling average
This smoothed data is derived
from any of these underlying
raw data examples.
Example Real Data
Two month step
Three month step
Increasing
Decreasing
Decreasing convergence
High and low
Six month step
Increasing convergence
Highs and lows
Notes: Real data for 12 months, previous 12 months is exactly the same, to create 12 month rolling average (mean).
13. 2. Why - Current and Strategic Drivers
1. Data, Evidence, Decisions… Impact, Value.
2. Big & Open & Now data
3. Tactical vs. Strategic focus
4. Key Strategies…Communication emphasis - ONS, RSS…
5. Underlying Numeracy and statistical literacy
6. Policy Perception Gap
7. Data Science – Shakespeare review, Open Data, UKSA…
8. Austerity World - Effective (right thing) & Efficient (right way)
17. 2. Why - Current and Strategic Drivers
1. Data, Evidence, Decisions… Impact, Value.
2. Big & Open & Now data
3. Tactical vs. Strategic focus
4. Key Strategies…Communication emphasis - ONS, RSS…
5. Underlying Numeracy and statistical literacy
6. Policy Perception Gap
7. Data Science – Shakespeare review, Open Data, UKSA…
8. Austerity World - Effective (right thing) & Efficient (right way)
18. % Adults at GCSE+ Levels
The numeracy challenge is big and getting bigger…
• Literacy Improving
while Numeracy
declining
Numeracy
• 26% to 22% (7.5m
adults) with GCSE+
• 17m adults at
primary school level
Skills for Life Survey 2011 (England)
Department for Business Innovation and Skills
19. A Framework for Understanding
Statistical Performance
Paul Askew
20. 2. Why - Current and Strategic Drivers
1. Data, Evidence, Decisions… Impact, Value.
2. Big & Open & Now data
3. Tactical vs. Strategic focus
4. Key Strategies…Communication emphasis - ONS, RSS…
5. Underlying Numeracy and statistical literacy
6. Policy Perception Gap
7. Data Science – Shakespeare review, Open Data, UKSA…
8. Austerity World - Effective (right thing) & Efficient (right way)
21. Outline
1. Introduction
2. Framework – the “Why”
Operational Drivers
Current Strategic Drivers
3. Framework – the “How”
Macro level
Analytical level
22. 3. How - Macro
DATA
- inputs -
INSIGHT
ANALYSIS
- outcomes -
- process -
PRODUCTS
- outputs -
27. 3. How – Analytical Level
0.
Snapshot
1.
Trend
2.
Benchmark
Time
Periods
Comparitors
Time
Periods
3.
Target
28. 3. How – Analytical Level
0.
Snapshot - we have a number which is important to us
1. Trend - what’s happening to our measure over time
2. Benchmark – how this compares to others
2a. Trend for the comparison to others
3. Target – the trajectory for our measure
3a. – the comparison trajectory
0.
1.
2.
3.
29. 3. How – Analytical Level
0.
Snapshot - we have a number which is important to us
1. Trend - what’s happening to our measure over time
2. Benchmark – how this compares to others
2a. Trend for the comparison to others
3. Target – the trajectory for our measure
3a. – the comparison trajectory
0.
1.
2.
3.
30. 3. How – Analytical Level
0.
Snapshot - we have a number which is important to us
1. Trend - what’s happening to our measure over time
2. Benchmark – how this compares to others
2a. Trend for the comparison to others
3. Target – the trajectory for our measure
3a. – the comparison trajectory
0.
1.
2.
3.
31. 3. How – Analytical Level
0.
Snapshot - we have a number which is important to us
1. Trend - what’s happening to our measure over time
2. Benchmark – how this compares to others
2a. Trend for the comparison to others
3. Target – the trajectory for our measure
3a. – the comparison trajectory
0.
1.
2.
3.
32. 3. How – Analytical Level
0.
Snapshot - we have a number which is important to us
1. Trend - what’s happening to our measure over time
2. Benchmark – how this compares to others
2a. Trend for the comparison to others
3. Target – the trajectory for our measure
3a. – the comparison trajectory
0.
1.
2.
3.
33. 3. How – Analytical Level
0.
Snapshot - we have a number which is important to us
1. Trend - what’s happening to our measure over time
2. Benchmark – how this compares to others
2a. Trend for the comparison to others
3. Target – the trajectory for our measure
3a. – the comparison trajectory
0.
1.
2.
3.
34. 0. Snapshot – we have a number which is important to us
Value
160
140
120
100
80
60
40
20
0
t-9
t-8
t-7
t-6
t-5
t-4
t-3
t=2
t-1 t=now t+1
t+2
t+3
t+4
Time
35. 1. Trend – what’s happening over time
Value
160
140
120
100
80
60
40
20
0
t-9
t-8
t-7
t-6
t-5
t-4
t-3
t=2
t-1 t=now t+1
t+2
t+3
t+4
Time
36. 2. Benchmark – how this measures compares to others
Value
160
140
120
100
80
60
40
20
0
t-9
t-8
t-7
t-6
t-5
t-4
t-3
t=2
t-1 t=now t+1
t+2
t+3
t+4
Time
37. 2a. Trend for the comparison to others
Value
160
140
120
100
80
60
40
20
0
t-9
t-8
t-7
t-6
t-5
t-4
t-3
t=2
t-1 t=now t+1
t+2
t+3
t+4
Time
38. 3. Target - the trajectory for our measure
Value
160
140
120
100
80
60
40
20
0
t-9
t-8
t-7
t-6
t-5
t-4
t-3
t=2
t-1 t=now t+1
t+2
t+3
t+4
Time
39. 3a. Target - Trajectory for the comparison to others
Value
160
140
120
100
80
60
40
20
0
t-9
t-8
t-7
t-6
t-5
t-4
t-3
t=2
t-1 t=now t+1
t+2
t+3
t+4
Time
40. Outline
1. Introduction
2. Framework – the “Why”
Operational Drivers
Current Strategic Drivers
3. Framework – the “How”
Macro
Analytical
41. A Framework for Understanding
Statistical Performance
Paul Askew
Thank You
CONFERENCE
2-5 SEPTEMBER 2013
NEWCASTLE