This document discusses how human and machine intelligence can work together in data science. It addresses key areas like data, models, and decisions. For data, it emphasizes representativeness and avoiding biases. For models, it advocates bringing the best of human and machine through techniques like human-in-the-loop machine learning. For decisions, it notes the importance of considering costs of mistakes, knowing limits of models, and prototyping systems before full deployment. The overall message is that close collaboration between human and AI experts can help ensure systems are effective, fair and accountable.

1. PEOPLE + AI
“Ta” Virot Chiraphadhanakul
virot.ch@skooldio.com
Human intelligence meets machine intelligence

2. Virot Chiraphadhanakul
วิโรจน์ จิรพัฒนกุล (ต้า)
B.Eng. (Computer), Chulalongkorn
Ph.D. in Operations Research, MIT
ex-Data Scientist, Facebook
Google Developers Expert in Machine Learning
Lecturer @ Chula Business School
Managing Director @ Skooldio

3. www.skooldio.com

4. “I believe in a world that will
have an abundance of artificial
intelligence, but what will be
scarce is real intelligence and
human qualities, like empathy”
— Satya Nadella, Microsoft CEO

5. Human-Centered Data Science

6. Human-Centered
Data Science

7. Source: www.google.com

8. Do we actually need machine learning?

9. Source: https://newsfeed.fb.com/customization

11. Source: Once more with feeling: Supportive responses to social sharing on Facebook (Burke and Develin, 2016)

12. Source: https://medium.com/google-design/human-centered-machine-learning-a770d10562cd

13. Source: https://www.facebook.com/editor

14. Source: https://medium.com/google-design/human-centered-machine-learning-a770d10562cd

15. Describe ways human experts
might solve the task today

16. data models decisions

17. DATA

18. Representativeness
Garbage in, garbage out.

19. Source: http://www.google.org/flutrends/

20. “The problems may be due to widespread
media coverage of this year’s severe US flu
season […] The press reports may have
triggered many flu-related searches by
people who were not ill.”
When
Google got
flu wrong
Source: http://www.nature.com/news/when-google-got-flu-wrong-1.12413

21. Bias

22. Source: http://www.streetbump.org/

23. Source: Machine Learning and Human Bias (Google / Youtube)

24. Avoid Self-Fulfilling Prophecies

25. “the decisions you make today
affect the training data you collect
tomorrow. Once your machine
learning system embeds biases
into its model, it can continue
generating new training data that
reinforces those biases.”
Source: https://medium.com/@dtunkelang/10-things-everyone-should-know-about-machine-learning-15279c27ce96

27. Collect more data, if needed

28. MODEL

29. Human-in-the-Loop Machine Learning
Bring the best of both worlds

30. Source: https://facebook.github.io/prophet/

31. Source: Forecasting at Scale (Sean J Taylor and Benjamin Letham)
capacity
changepoints
seasonality and holidays
smoothing parameters

33. Source: www.slideshare.net/BradKlingenberg/making-fashion-recommendations-with-humanintheloop-machine-learning
Styling at Stitch Fix: personalized recommendations
Inventory
Algorithmic
recommendations
Machine learning

34. Source: www.slideshare.net/BradKlingenberg/making-fashion-recommendations-with-humanintheloop-machine-learning
Styling at Stitch Fix: personalized recommendations
Inventory
Algorithmic
recommendations
Machine learning
Styling at Stitch Fix: expert human curation
Human curation
Algorithmic
recommendations

35. Improvement through a Feedback Loop

36. Source: www.slideshare.net/BradKlingenberg/making-fashion-recommendations-with-humanintheloop-machine-learning
Learning through feedback

39. Source: https://medium.com/google-design/human-centered-machine-learning-a770d10562cd

40. Human Evaluation

42. Source: https://webmasters.googleblog.com/2015/11/updating-our-search-quality-rating.html

43. DECISION

44. Costs of Mistakes

45. • Consider the following scenarios
• A customer is wrongly alerted
(false positive)
• A customer is wrongly charged for
a fraudulent transaction (false
negative)
Unequal
Costs and
Benefits

46. ROC Curve
Source: https://stripe.com/radar/guide
Receiver Operating
Characteristic

47. Know Your Limits

48. Source: https://hbr.org/2016/05/when-to-trust-robots-with-decisions-and-when-not-to

49. Prototyping
Wizard of Oz

50. Source: ‘Mystery Van’ Helping Self-Driving Cars Communicate with People (Ford Motor Company / Youtube)

51. Source: Natural Language Generation at Google Research (Google Cloud / Youtube)

52. Test. Test. Test.

53. data models decisions
We choose We determine
We’re affected

54. PEOPLE + AI
“Ta” Virot Chiraphadhanakul
virot.ch@skooldio.com
Human intelligence meets machine intelligence