Quality reviews in Statistics Norway – Setup and experiences

1. Quality reviews in Statistics Norway – Setup and
experiences
Hans Viggo Sæbø and Jan Byfuglien
Statistics Norway
• Purpose
• Setup and implementation of quality reviews
• Results
• Follow up of improvement plans
• The way forward

2. Purpose
• Improve quality and efficiency:
– Discover weaknesses and propose improvements
– Raise awareness about quality issues
• Part of general internal control system
• Follow up earlier peer review and provide basis for new
reviews
During 2011 and 2012 18 statistics have been reviewed

3. The European Statistics Code of Practice
Institutional
environment
Statistical
processes Statistical output Users
Professional Independence
Mandate for Data Collection
Adequacy of Resources
Quality Commitment
Statistical Confidentiality
Impartiality and Objectivity
Sound Methodology
Appropriate Statistical
Procedures
Non-excessive Burden on
Respondents
Cost effectiveness
Relevance
Accuracy and Reliability
Timeliness and
Punctuality
Coherence and
Comparability
Accessibility and Clarity
A common European quality framework

4. 4
Audits based on
Code of practice

5. Self assessment questionnaire - Example
Grad
samsvar Spesifiser - styrker, svakheter, kommentarer
3 Det finnes prosedyrer for planlegging og
overvåking av kvaliteten på prosessen
for utarbeiding av statistikk.
B Det foreligger en kvalitetsrapport for datasamarbeidet mot
oppgavegiveren
4 Produktkvaliteten overvåkes og vurderes
regelmessig med tanke på mulige
kompromisser, og den rapporteres i
henhold til kvalitetskriteriene for
europeisk statistikk.
B Gjennom samarbeidet med TAD overvåkes kvaliteten med hensyn til
avleverte oppgaver. Utarbeider ikke en fast løpene kvalitetsrapport.
Eurostat skal komme med forslag til en slik for EFTA-landene.
4 Personalet har tilgang til retningslinjer
og veiledninger om statistisk
konfidensialitet ved utarbeiding og
formidling. Disse retningslinjene er
skriftlige og offentlig tilgjengelige.
B Vi praktiserer passiv undertrykking i tråd med hva nesten alle andre
land gjør og etter FN's anbefalinger. Trenger imidlertid noe mer
skriftlig dokumentasjon. Generelle retningslinjer er beskrevet i "Om
statistikken"
5 Fysiske,teknologiske og
organisatoriske tiltak iverksettes for å
ivareta sikkerheten og integriteten til
statistiske databaser.
A Det er et skille mellom databaser tilgjengelig for seksjonen og
andre, og konfidensialitet er iveretatt.
6
Selvvurdering
4. Kvalitets-
bevissthet
5. Konfidensialitet
Prinsipp
Compliance with Code of Practice indicators is indicated by:
A – Full
B – Partly
C – No
X – Not relevant

6. Process mapping
• Purpose:
– To get overview of involved partners, key routines, data flows,
processes and elapse of time
– Uncover bottlenecks, inefficient routines, lacking documentation,
risks for errors etc.
• Method:
– The process drawn on a paper on the wall in a meeting between
the team and key experts responsible for the product – and later
refined in printable form
– Supplemented by available documentation
• Results, for example:
– Revealing unnecessary and delaying routines
– Revealing unclear responsibilities
– Revealing lacking documentation of routines

7. An example: Statistics on primary eduction from
administrative authorities and an IT service
provider

8. More than 60 percent of the resources are used
for process 4 and 5 for the selected statistics
0 5000 10000 15000 20000 25000 30000
1. Specify needs
2. Develop and design
3. Build
4. Collect
5. Process
6. Analyse
7. Disseminate
8. Quality management
Hours

9. Improvement areas ranked

10. Follow up of improvement proposals
• About 140 improvement measures were planned
• 60 remain, for 9 of 18 statistics all measures were
completed by March 2013
• Improvements within documentation, dissemination and
production processes
• Some statistics will be abolished, one will be outsourced
• More remains where the responsible division depends on
others (such as ssb.no, IT, standardized solutions)

11. Assessment of the setup
• Useful, cost-efficient
• Important with external reviewers
• Have contributed to identify problematic areas
• Focus groups have given new knowledge
• Self-assessments have been
conscientiously completed
without use of much resources

12. The way forward
• Adapt to Lean plans
• Themes comprising clusters of statistics, but must be
possible to handle with limited use of resources
• External trade and population statistics first
• Need for renewal of review team
• Good basis for European peer reviews