In recent years, machine learning has undoubtedly been hugely successful in driving progress in AI applications. However, as we will explore in this talk, even state-of-the-art systems have "blind spots" which make them generalize poorly out of domain and render them vulnerable to adversarial examples. We then suggest that more unsupervised learning settings can encourage the development of more robust systems. We show positive results on two tasks: (i) text style and attribute transfer, the task of converting a sentence with one attribute (e.g., sentiment) to one with another; and (ii) solving SAT instances (classical problems requiring logical reasoning) using end-to-end neural networks.

1. Pushing the Limits of Machine Learning
Percy Liang
AI Frontiers Conference — November 9, 2018

2. Image classification
[figure credit: EFF]
1

3. Image classification
[Szegedy+ 2014, Goodfellow+ 2015]
2

4. Image classification
[Szegedy+ 2014, Goodfellow+ 2015]
[Sharif+ 2016] [Evtimov+ 2017]
2

5. SQuAD: 100,000+ Questions for Machine Comprehension of Text
(EMNLP 2016; best resource paper award)
Pranav Rajpurkar Jian Zhang Konstantin Lopyrev
3

6. Reading Comprehension
[with Pranav Rajpurkar et al; 2016]
4

7. 5

8. Adversarial Evaluation of Reading Comprehension
(EMNLP 2017; outstanding paper award)
Robin Jia
6

9. Reading comprehension
Individual Huguenots settled at the Cape of Good Hope from as early as 1671 with the arrival of Francois Villion (Viljoen). The
first Huguenot to arrive at the Cape of Good Hope was however Maria de la Queillerie, wife of commander Jan van Riebeeck (and
daughter of a Walloon church minister), who arrived on 6 April 1652 to establish a settlement at what is today Cape Town. The
couple left for the Far East ten years later. On 31 December 1687 the first organised group of Huguenots set sail from the Netherlands
to the Dutch East India Company post at the Cape of Good Hope. The largest portion of the Huguenots to settle in the Cape arrived
between 1688 and 1689 in seven ships as part of the organised migration, but quite a few arrived as late as 1700; thereafter, the
numbers declined and only small groups arrived at a time.
The number of new Huguenot colonists declined after what year?
BERT
1700
[with Robin Jia; EMNLP 2017]
7

10. Reading comprehension
Individual Huguenots settled at the Cape of Good Hope from as early as 1671 with the arrival of Francois Villion (Viljoen). The
first Huguenot to arrive at the Cape of Good Hope was however Maria de la Queillerie, wife of commander Jan van Riebeeck (and
daughter of a Walloon church minister), who arrived on 6 April 1652 to establish a settlement at what is today Cape Town. The
couple left for the Far East ten years later. On 31 December 1687 the first organised group of Huguenots set sail from the Netherlands
to the Dutch East India Company post at the Cape of Good Hope. The largest portion of the Huguenots to settle in the Cape arrived
between 1688 and 1689 in seven ships as part of the organised migration, but quite a few arrived as late as 1700; thereafter, the
numbers declined and only small groups arrived at a time. The number of old Acadian colonists declined after the year 1675.
The number of new Huguenot colonists declined after what year?
BERT
[with Robin Jia; EMNLP 2017]
7

11. Reading comprehension
Individual Huguenots settled at the Cape of Good Hope from as early as 1671 with the arrival of Francois Villion (Viljoen). The
first Huguenot to arrive at the Cape of Good Hope was however Maria de la Queillerie, wife of commander Jan van Riebeeck (and
daughter of a Walloon church minister), who arrived on 6 April 1652 to establish a settlement at what is today Cape Town. The
couple left for the Far East ten years later. On 31 December 1687 the first organised group of Huguenots set sail from the Netherlands
to the Dutch East India Company post at the Cape of Good Hope. The largest portion of the Huguenots to settle in the Cape arrived
between 1688 and 1689 in seven ships as part of the organised migration, but quite a few arrived as late as 1700; thereafter, the
numbers declined and only small groups arrived at a time. The number of old Acadian colonists declined after the year 1675.
The number of new Huguenot colonists declined after what year?
BERT
1675
[with Robin Jia; EMNLP 2017]
7

12. Results on SQuAD models
Model Original F1 Adversarial F1
BERT 93.2 70.7
SLQA+ 88.6 64.2
r-net+ 88.5 63.4
ReasoNet-E 81.1 49.8
SEDT-E 80.1 46.5
BiDAF-E 80.0 46.9
Mnemonic-E 79.1 55.3
Ruminating 78.8 47.7
jNet 78.6 47.0
Mnemonic-S 78.5 56.0
8

13. Results on SQuAD models
Model Original F1 Adversarial F1
Humans 92.6 89.2
BERT 93.2 70.7
SLQA+ 88.6 64.2
r-net+ 88.5 63.4
ReasoNet-E 81.1 49.8
SEDT-E 80.1 46.5
BiDAF-E 80.0 46.9
Mnemonic-E 79.1 55.3
Ruminating 78.8 47.7
jNet 78.6 47.0
Mnemonic-S 78.5 56.0
8

14. Humans versus machines
9

15. Outline
Probing
Extrapolation
Language
10

16. Understanding Black-Box Predictions via Influence Functions
(ICML 2017; best paper award)
Pang Wei Koh
11

17. Why does the model make this prediction?
dog
12

18. Why does the model make this prediction?
dog
• What inputs maximally activate these neurons? [Girshick+
2014]
• Which part of the input was most responsible? [Zeiler &
Fergus 2013; Simonyan+ 2013; Li+ 2016; Shrikumar+ 2017]
• What is a simpler model that locally approximates the model?
[Ribeiro+ 2016; Bastani+ 2017]
12

19. Understanding via influence functions
dog
13

20. Understanding via influence functions
...
fish
dog
...
Training
dog
The training data holds the deep reason for model behavior.
13

21. Understanding via influence functions
...
fish
dog
...
Training
dog
The training data holds the deep reason for model behavior.
13

22. Adversarial training examples
14

23. Adversarial training examples
Change one training example causes 16/30 test examples to be misclassified
14

24. Certified Defenses Against Adversarial Examples (ICLR 2018)
Aditi Raghunathan Jacob Steinhardt
15

25. Setup
Defender: train a scoring function f (classify positive if f(x) > 0)
16

26. Setup
Defender: train a scoring function f (classify positive if f(x) > 0)
Attacker: given input x, find ˜x such that f(x) is large and ˜x − x ∞ ≤
16

27. Cat-and-mouse game
17

28. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
17

29. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
[Goodfellow+ 2015]: Adversarial training (AT) against FGSM
17

30. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
[Goodfellow+ 2015]: Adversarial training (AT) against FGSM
[Papernot+ 2015]: defensive distillation
17

31. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
[Goodfellow+ 2015]: Adversarial training (AT) against FGSM
[Papernot+ 2015]: defensive distillation
[Calini & Wagner 2016]: distillation is not secure
17

32. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
[Goodfellow+ 2015]: Adversarial training (AT) against FGSM
[Papernot+ 2015]: defensive distillation
[Calini & Wagner 2016]: distillation is not secure
[Papernot+ 2017]: better distillation
17

33. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
[Goodfellow+ 2015]: Adversarial training (AT) against FGSM
[Papernot+ 2015]: defensive distillation
[Calini & Wagner 2016]: distillation is not secure
[Papernot+ 2017]: better distillation
[Carlini & Wagner 2017]: All detection strategies fail
17

34. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
[Goodfellow+ 2015]: Adversarial training (AT) against FGSM
[Papernot+ 2015]: defensive distillation
[Calini & Wagner 2016]: distillation is not secure
[Papernot+ 2017]: better distillation
[Carlini & Wagner 2017]: All detection strategies fail
[Madry+ 2017]: AT against PGD, informal argument about optimality
17

35. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
[Goodfellow+ 2015]: Adversarial training (AT) against FGSM
[Papernot+ 2015]: defensive distillation
[Calini & Wagner 2016]: distillation is not secure
[Papernot+ 2017]: better distillation
[Carlini & Wagner 2017]: All detection strategies fail
[Madry+ 2017]: AT against PGD, informal argument about optimality
[Lu+ July 12 2017]: ”NO Need to Worry about Adversarial Examples in Object Detection in
Autonomous Vehicles”
17

36. Cat-and-mouse game
[Szegedy+ 2014]: first discover adversarial examples
[Goodfellow+ 2015]: Adversarial training (AT) against FGSM
[Papernot+ 2015]: defensive distillation
[Calini & Wagner 2016]: distillation is not secure
[Papernot+ 2017]: better distillation
[Carlini & Wagner 2017]: All detection strategies fail
[Madry+ 2017]: AT against PGD, informal argument about optimality
[Lu+ July 12 2017]: ”NO Need to Worry about Adversarial Examples in Object Detection in
Autonomous Vehicles”
[Athalye & Sutskever July 17 2017]: break defense with AT on PGD with transformed examples
17

37. Verification
Can we get robustness against all attacks?
18

38. Attacks
Key: use convex relaxation to upper bound all attacks
19

39. Results on MNIST
Key: obtain certificate of robustness
20

40. Summary
• Influence functions help understand (and attack) models
• Convex relaxation provides provable guarantees of security
21

41. Outline
Probing
Extrapolation
Language
22

42. Style / attribute transfer in natural language (NAACL 2018)
Juncen Li Robin Jia He He
23

43. Task setup
Train (review ⇒ sentiment):
very tasty burritos, and cheap too! ⇒ positive
found hair in my soup, would never go back again ⇒ negative
... ...
Test (negative review ⇒ positive review):
great food but very rude workers ⇒ great food and very friendly staff
24

44. Deletion-based model
Step 1: extract attributes
25

45. Deletion-based model
Step 1: extract attributes Step 2: delete + predict
25

46. Deletion-based model
Step 1: extract attributes Step 2: delete + predict
Inductive bias: attribute/style is localized in the text
25

47. Datasets
[Shen+ 2017; Fu+ 2018; Gan+ 2017]
26

48. Results
Human evaluation: grammatical, preserve content, has target attribute
[Shen+ 2017; Fu+ 2018]
27

49. Results
Source: we sit down and we got some really slow and lazy service .
28

50. Results
Source: we sit down and we got some really slow and lazy service .
CrossAligned: we went down and we were a good , friendly food .
StyleEmbedding: we sit down and we got some really slow and prices suck .
MultiDecoder: we sit down and we got some really and fast food .
28

51. Results
Source: we sit down and we got some really slow and lazy service .
CrossAligned: we went down and we were a good , friendly food .
StyleEmbedding: we sit down and we got some really slow and prices suck .
MultiDecoder: we sit down and we got some really and fast food .
Delete: we sit down and we got some great and quick service .
Delete+Retrieve: we got very nice place to sit down and we got some service .
Locality inductive bias helps!
28

52. SAT solving with neural networks
Daniel Selsam Matt Lamm Benedikt Bunz Leonardo de Moura David Dill
29

53. SAT solving
(x1 ∨ x2) ∧ (¬x1 ∨ x3) ⇒ x1 = 0, x2 = 1, x3 = 1
x1 ∧ ¬x1 ⇒ unsat
• Lots of applications to scheduling and verification
• Intractable to solve exactly
• Can solve large instances in practice with lots of heuristics
30

54. Model
(x1 ∨ x2) ∧ (¬x1 ∨ ¬x2)
Captures inductive bias of survey propagation
31

55. Predicting satisfiability
Train: random instances of sat/unsat minimal pairs
(x1 ∨ x2) · · ·
sat
⇒ 1 (¬x1 ∨ x2) · · ·
unsat
⇒ 0
Test: random instances (same distribution)
32

56. Predicting satisfiability
Train: random instances of sat/unsat minimal pairs
(x1 ∨ x2) · · ·
sat
⇒ 1 (¬x1 ∨ x2) · · ·
unsat
⇒ 0
Test: random instances (same distribution)
Test accuracy: 85%
32

57. Decoding satisfying assignments
33

58. Decoding satisfying assignments
Can decode 70% of instances where model predicts sat — extrapolation!
33

59. Extrapolation to larger instances, more iterations
34

60. Summary
• Extrapolation: evaluate on structurally unseen task
• Strong inductive bias permits this unsupervised learning
35

61. Outline
Probing
Extrapolation
Language
36

62. learning language
37

63. learning language
37

64. learning language
Key: use language to communicate inductive bias directly
37

65. Learning From Definitions (ACL 2017)
Sida Wang Sam Ginn Chris Manning
38

66. Interpreting natural language commands
add two chairs 5 spaces apart
39

67. Interpreting natural language commands
add two chairs 5 spaces apart
(:s (:s (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select))))) (:loop (number 3) (:for (call adj left this)
(: select)))) (:s (:s (:s (:s (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select))))) (:loop (number 3) (:for
(call adj back this) (: select)))) (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select)))))) (:loop (number 3)
(:for (call adj right this) (: select)))) (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select)))))))
39

68. Interpreting natural language commands
add two chairs 5 spaces apart
Provide definitions:
add chair, move 5 left, add chair
(:s (:s (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select))))) (:loop (number 3) (:for (call adj left this)
(: select)))) (:s (:s (:s (:s (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select))))) (:loop (number 3) (:for
(call adj back this) (: select)))) (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select)))))) (:loop (number 3)
(:for (call adj right this) (: select)))) (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select)))))))
39

69. Interpreting natural language commands
add two chairs 5 spaces apart
Provide definitions:
add chair, move 5 left, add chair
(:s (:s (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select))))) (:loop (number 3) (:for (call adj left this)
(: select)))) (:s (:s (:s (:s (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select))))) (:loop (number 3) (:for
(call adj back this) (: select)))) (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select)))))) (:loop (number 3)
(:for (call adj right this) (: select)))) (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select)))))))
39

70. Interpreting natural language commands
add two chairs 5 spaces apart
Provide definitions:
add 4 legs, add chair base, add chair back, ...
(:s (:s (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select))))) (:loop (number 3) (:for (call adj left this)
(: select)))) (:s (:s (:s (:s (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select))))) (:loop (number 3) (:for
(call adj back this) (: select)))) (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select)))))) (:loop (number 3)
(:for (call adj right this) (: select)))) (:blkr (:s (:loop (number 3) (:s (: add brown here) (:for (call adj top this) (: select)))) (:loop (number 3) (:for (call adj bot this) (: select)))))))
39

71. Interface for definitions
40

72. Results
3 days, 42 users, 230 structures, 64075 utterances, 2495 definitions
add chair, move 5 left, add chair
add 4 legs, add chair base, add chair back, ...
41

73. Outline
Probing
Extrapolation
Language
42

74. Parting questions
What are the limits?
43

75. Parting questions
What are the limits?
How can we robustify?
43

76. Worksheets
Robin Jia Pranav Rajpurkar Sida Wang Sam Ginn Chris Manning
Pang Wei Koh Jacob Steinhardt Aditi Raghunathan Jian Zhang Konstantin Lopyrev
Daniel Selsam Matt Lamm Benedikt Bunz Leonardo de Moura David Dill
OpenPhil DARPA NSF Facebook Microsoft Intuit Tencent
Thank you!
44