Brief, introductiory slides for second Dallas Data Brewery meetup. Topic: Data Quality Perception.

1. Data Quality
Perception
data brewery
Dallas Data Brewery, June 2013

2. Topic
■ What is "high quality data"?
■ What are data quality expectations?
you, people or businesses you know have
■ Business issues and data quality
How to you deal with it?
■ What happens when you ignore it?

3. What is data quality ?

4. Dimensions
■ completeness – data provided
■ accuracy – reflecting real world
■ credibility – regarded as true
■ timeliness – up-to-date
■ consistency – matching facts across datasets
■ integrity – valid references between datasets
... and there are more

5. Fallacies
■ “good data are error-free and valid”
■ “improving quality means cleansing”
■ “it is IT problem”
■ “it can be fixed”

6. Short Story:
Completeness
Open Public Procurements

7. from this...

8. ... to this:
http://tendre.sme.sk

9. 0%
25%
50%
75%
100%
2005-3
2005-5
2005-7
2005-92005-112006-1
2006-3
2006-5
2006-7
2006-92006-112007-1
2007-3
2007-5
2007-7
2007-92007-112008-1
2008-3
2008-5
2008-7
2008-92008-112009-1
2009-3
2009-5
2009-7
2009-92009-112010-1
2010-3
2010-5
2010-7
2010-9
better
have it all
none
Quality measure
completeness: 55%
how many % of the field is filled and
successfully processed?

10. type 1 type 2
+

11. how many % of the field is filled
and successfully processed?
0%
25%
50%
75%
100%
2005-3
2005-5
2005-7
2005-92005-102005-122006-3
2006-5
2006-7
2006-92006-112007-1
2007-3
2007-5
2007-7
2007-92007-102007-122008-3
2008-5
2008-7
2008-92008-112009-1
2009-3
2009-5
2009-7
2009-92009-112010-1
2010-3
2010-5
2010-7
2010-9
Quality measure
completeness: 88%
better
have it all
none

12. What does that mean:
“high quality data?”
?

13. 85% ?

14. Conclusion

15. appropriate for given
purpose

16. Data Project
■ define data quality requirements
■ measure during development
■ provide data quality report

17. More topics
■ Data quality measurement
indicators, probes
■ Data quality management
roles, processes, impact
■ Data cleansing

18. Thank You
stefan@freshdata.sk ￭ Stiivi
www.meetup.com/dallas-data-brewery