Ece Kamar Researcher, Adaptive Systems and Interaction Group Microsoft Research

1. Humans to the Rescue: Troubleshooting
AI Systems with Human-in-the-loop
Ece Kamar
Senior Researcher, Microsoft Research AI
eckamar@microsoft.com

2. Exciting Times

3. AI and the Crowd
training data
accuracy
test data

4. Power of Data
[Banko&Brill, 2001]

5. In the Wild

6. In the Wild

7. Hybrid Intelligence
Human
Intelligence
AI Systems

8. AI Applied to Critical Domains

9. Power of the Hybrid
[Courtesy of Murray Campbell]

10. Troubleshooting of ML Systems
training data
accuracy
test data
query
system
response
execution
data
In the lab
In the wild
What is the performance in the wild?
How does the system fail?
Why does the system fail?
How the system can be improved?

11. Biases in ML
[Lakkaraju, K., Caruana, Horvitz; AAAI 2017]

12. Biases in ML
[Lakkaraju, K., Caruana, Horvitz; AAAI 2017]

13. Biases in ML
[Lakkaraju, K., Caruana, Horvitz; AAAI 2017]

14. Where do Blind Spots Come From?
M
cats
dogs
cat
(conf = 0.96)
Unknown unknowns: Data points with confident but incorrect predictions.
Blind-spots: Feature spaces with high concentration of unknown unknowns

15. Blind-spots Detection
execution data
Beat the Machine
[Attenberg, Ipeirotis, Provost, 2011]
Exploration of Unknown Unknowns
[Lakkaraju, K., Caruana, Horvitz, 2011]
Step 1:
Descriptive
Space
Partitioning
execution data
Step 2:
Multi-armed
Bandit
based
Exploration

16. Troubleshooting Complex Systems

17. Challenge
Possible fixes
for each
component
Limited development time
Where to invest
development time for
biggest impact?

18. Human-assisted troubleshooting methodology
system
outputComponent
1
Component
2
Component
3
I/OI/O
Evaluation
Failures
Fixes
[Nushi, K., Kossmann, Horvitz, 2011]

19. Complex Issues
Fairness Biases
TransparencyResponsibility
Good
vs. Bad
Policy & Law

20. Complex challenges
require collective efforts
No AI is perfect