The document discusses challenges in obtaining ground truth data in business applications. It notes that ground truth is difficult to acquire for recent data and can be expensive. However, ground truth statistics are valuable for measuring accuracy and performance. The document explores techniques for obtaining approximate ground truth values through methods like wisdom of the crowds, combining multiple independent labels, and data stream algorithms that don't explicitly store ground truth labels.

1. Ground Truth Problems
in
Business
Andrés Corrada-Emmanuel
Data Engines
github.com/andrescorrada/ground-truth-problems-in-business

2. github.com/andrescorrada/ground-truth-problems-in-business
github.com/andrescorrada/ground-truth-problems-in-business

3. github.com/andrescorrada/ground-truth-problems-in-business

4. No Ground Truth
because
data is recent
github.com/andrescorrada/ground-truth-problems-in-business

5. github.com/andrescorrada/ground-truth-problems-in-business

6. No Ground Truth
because
it is expensive to get
github.com/andrescorrada/ground-truth-problems-in-business

7. github.com/andrescorrada/ground-truth-problems-in-business

8. The journey of ground truth solutions
• Measurement
• Diagnosis
• Solution
github.com/andrescorrada/ground-truth-problems-in-business

9. Statistics of Ground Truth
Statistic(answer key, student exam) = grade
Statistic(labels, classifications) = accuracy
github.com/andrescorrada/ground-truth-problems-in-business

10. The value
of
ground truth statistics
in
business
github.com/andrescorrada/ground-truth-problems-in-business

11. github.com/andrescorrada/ground-truth-problems-in-business

12. github.com/andrescorrada/ground-truth-problems-in-business

13. github.com/andrescorrada/ground-truth-problems-in-business

14. github.com/andrescorrada/ground-truth-problems-in-business
Original Binarized

15. github.com/andrescorrada/ground-truth-problems-in-business
Original Binarized

16. github.com/andrescorrada/ground-truth-problems-in-business

17. github.com/andrescorrada/ground-truth-problems-in-business

18. github.com/andrescorrada/ground-truth-problems-in-business

19. github.com/andrescorrada/ground-truth-problems-in-business

20. github.com/andrescorrada/ground-truth-problems-in-business

21. github.com/andrescorrada/ground-truth-problems-in-business

22. github.com/andrescorrada/ground-truth-problems-in-business

23. github.com/andrescorrada/ground-truth-problems-in-business
Ground Truth Wisdom of the Crowd

24. github.com/andrescorrada/ground-truth-problems-in-business

25. github.com/andrescorrada/ground-truth-problems-in-business

26. github.com/andrescorrada/ground-truth-problems-in-business
Ground Truth Best Three

27. github.com/andrescorrada/ground-truth-problems-in-business
Ground TruthWisdom of the Crowd Best of Three
88% reduction in errors

28. github.com/andrescorrada/ground-truth-problems-in-business

29. The Ground Truth is out there,
it’s just too large
github.com/andrescorrada/ground-truth-problems-in-business

30. Data Stream Algorithms - refusing
to remember ground truth
github.com/andrescorrada/ground-truth-problems-in-business

31. github.com/andrescorrada/ground-truth-problems-in-business

32. github.com/andrescorrada/ground-truth-problems-in-business

33. github.com/andrescorrada/ground-truth-problems-in-business

34. github.com/andrescorrada/ground-truth-problems-in-business

35. github.com/andrescorrada/ground-truth-problems-in-business

36. github.com/andrescorrada/ground-truth-problems-in-business

37. github.com/andrescorrada/ground-truth-problems-in-business
• Compressive
• Private

38. github.com/andrescorrada/ground-truth-problems-in-business
Nov-23-00
Nov-24-00
Nov-25-00
Nov-26-00
Nov-27-00
Nov-28-00
Nov-29-00
Nov-30-00
Dec-01-00
Dec-02-00
Dec-03-00
Dec-04-00
Dec-05-00
Dec-06-00
Nov- 23- 00
Nov- 24- 00
Nov- 25- 00
Nov- 26- 00
Nov- 27- 00
Nov- 28- 00
Nov- 29- 00
Nov- 30- 00
Dec- 01- 00
Dec- 02- 00
Dec- 03- 00
Dec- 04- 00
Dec- 05- 00
Dec- 06- 00
First Arrival "Learning" IP Self Match Rate: AppNexus RTB
0
0.2
0.4
0.6
0.8

39. Grading an exam
without
the answer key
github.com/andrescorrada/ground-truth-problems-in-business

40. github.com/andrescorrada/ground-truth-problems-in-business
• “Do you need ground truth to calculate a
statistic of it?”
a. “Yes”
b. “No”
A question from a multiple-choice exam

41. github.com/andrescorrada/ground-truth-problems-in-business

42. github.com/andrescorrada/ground-truth-problems-in-business

43. github.com/andrescorrada/ground-truth-problems-in-business

44. github.com/andrescorrada/ground-truth-problems-in-business

45. github.com/andrescorrada/ground-truth-problems-in-business

46. github.com/andrescorrada/ground-truth-problems-in-business

47. github.com/andrescorrada/ground-truth-problems-in-business
•Compressive
•Private

48. github.com/andrescorrada/ground-truth-problems-in-business

49. github.com/andrescorrada/ground-truth-problems-in-business

50. github.com/andrescorrada/ground-truth-problems-in-business

51. github.com/andrescorrada/ground-truth-problems-in-business

52. github.com/andrescorrada/ground-truth-problems-in-business

53. github.com/andrescorrada/ground-truth-problems-in-business

54. The Opposite
github.com/andrescorrada/ground-truth-problems-in-business

55. Why 4,000 Questions?
• Correlations!
• Small ground truth size or non-independent
classifiers require a different model .... Our
secret sauce.
github.com/andrescorrada/ground-truth-problems-in-business

56. Ground Truth for Regressors

57. Ground Truth for Regressors

58. Ground Truth for Regressors

59. Low High
Elevations
Sample DEM Fused DEM
Blunders are removed and noise is reduced

60. github.com/andrescorrada/ground-truth-problems-in-business
github.com/andrescorrada/ground-truth-problems-in-business
Thank You!