Smart Reply: Learning a Model of Conversation from Data: Smart Reply is a text assistance feature that was recently introduced to Inbox by Gmail. Given an incoming email message, the Smartreply system analyzes its contents and suggests complete responses that the recipient can send with just one tap. This talk will cover how we built Smartreply using a combination of deep learning and semantic clustering, as well as what we learned along the way and why we think it shows promise for the future of dialogue models.

1. Confidential + Proprietary
Smart Reply: Learning a Model of
Conversation from Data
Anjuli Kannan
Software Engineer, Google Brain

2. Problem

3. Confidential + Proprietary
Can you do Tuesday or Wednesday?
Phil Sharp

4. Confidential + Proprietary
Tuesday Wednesday
Can you do Tuesday or Wednesday?
Phil Sharp

5. Smart Reply feature
● Provide text assistance for email
reply composition
● Targeted at mobile
● Responses can be sent on their
own or extended

6. Smart Reply feature predicts email responses
Smart Reply
Incoming
email
Response
email

7. Why is this task hard?
● extracting meaning from previous message
● generating language
● grammatical transformations between call and response
● matching style/tone

8. Why is this solution interesting?
● Model is learned fully from data

9. Model

10. Confidential + Proprietary
Neural network
Is a 4
Is a 5
...
...
Image: Wikipedia

11. Confidential + Proprietary
Neural network
Neuron
Is a 4
Is a 5

12. Confidential + Proprietary
Basic building block is the neuron
Greg Corrado

13. Confidential + Proprietary
Neural network
Is a 4
Is a 5
...
...

14. Confidential + Proprietary
Learn a function from one space to another
f(.)x ∈ Rn
y ∈ Rm

15. Confidential + Proprietary
Smartreply feature predicts email responses
Smartreply
Incoming
email
Response
email

16. Confidential + Proprietary
Recurrent neural networks handle sequences of input
Diagram by Felix Gers

17. Confidential + Proprietary
Recurrent neural networks handle sequences of input
Diagram by Felix Gers

18. Confidential + Proprietary
Recurrent neural networks handle sequences of input

19. Confidential + Proprietary
Reading a word into a feed-forward neural network
cat
output

20. Confidential + Proprietary
Reading a sequence of words into an RNN
That

21. Confidential + Proprietary
Reading a sequence of words into an RNN
That is

22. Confidential + Proprietary
Reading a sequence of words into an RNN
That is good

23. Confidential + Proprietary
Reading a sequence of words into an RNN
That is good !

24. Confidential + Proprietary
Reading a sequence of words into an RNN
That is good !
output

25. Sequence-to-sequence model
Sutskever et al, NIPS 2014

26. Sequence-to-sequence model
encoder decoder

27. Sequence-to-sequence model
Ingests incoming message Generates reply message

28. Inference

29. Reading a sequence of words into an RNN
How

30. Reading a sequence of words into an RNN
How are

31. Reading a sequence of words into an RNN
How are you

32. Reading a sequence of words into an RNN
How are you ?

33. Encoder ingests the incoming message
How are you ?
Internal state is a fixed
length encoding of the
message

34. Decoder is initialized with final state of encoder
How are you ? __

35. Decoder is initialized with final state of encoder
How are you ? __

36. Decoder predicts next word
How are you ? __

37. Decoder predicts next word
How are you ? ____ I

38. Smartreply model
How
Message

39. Smartreply model
How are
Message

40. Smartreply model
How are you
Message

41. Smartreply model
How are you ?
Message

42. Smartreply model
How are you ? __
I
Message
Response

43. Smartreply model
How are you ? __ I
I am
Message
Response

44. Smartreply model
How are you ? __ I am
I am great
Message
Response

45. Smartreply model
How are you ? __ I am great
I am great !
Message
Response
Vinyals & Le, ICML DL 2015

46. Inference
● Resulting model is fully generative
● Output distribution can be used to determine the most likely responses using a
beam search

47. Training

48. Training
● Training data is a corpus of email-reply pairs
● Both encoder and decoder are trained together (end-to-end)

49. Training
● Training data is a corpus of email-reply pairs
● Both encoder and decoder are trained together (end-to-end)

50. Confidential + Proprietary
Key points about model
● Everything is learned from data, even features
● Neural network smooths across language variation

51. Smart Reply in Production

52. Deployment & coverage
● Deployed in Inbox by Gmail
● Used to assist with more than 10% of all mobile replies

53. Examples

54. Quality
● How do we ensure that the response options are always high quality in content
and language?
○ Avoid incorrect grammar and mechanics, misspellings e.g., your the best
○ Avoid inappropriate, offensive responses. e.g., Leave me alone.
○ Deal with wide variability, informal language. e.g., got it thx
● Restricting model vocabulary is not sufficient!
Solution: Restrict to a fixed set of valid responses, derived automatically from data.

55. Most frequently used clusters

56. Confidential + Proprietary
What the model can do

57. Confidential + Proprietary
What the model can't do
● Match every user's tone and style

58. Confidential + Proprietary
What the model can't do
● Match every user's tone and style
● Ensure diverse options

59. Confidential + Proprietary
What the model can't do
● Match every user's tone and style
● Ensure diverse options
● Access and update any kind of state or knowledge base

60. Conclusions

61. Conclusions
● Sequence-to-sequence produces plausible email replies in many common
scenarios, when trained on an email corpus
● Smart Reply is deployed in Inbox by Gmail and generates more than 10% of
mobile replies

62. Confidential + Proprietary
Conclusions
● A conversation model learned entirely from data is very powerful
● A data-driven approach can be complementary to hand-crafted rules and
scenarios

63. Confidential + Proprietary
Collaborators
- Greg Corrado, Oriol Vinyals (Google Brain)
- Balint Miklos, Tobias Kaufman, Laszlo Lukacs, and Karol Kurach (GMail)
- Sujith Ravi (Google Research)

64. Confidential + Proprietary
Thank you!

65. Extra slides

66. Example

67. Unique cluster and suggestion usage

68. Ranking experiments