Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems presented by Hongkyu Lim

1. 2022. 06. 10
Mem2Seq: Effectively Incorporating Knowledge Bases into
End-to-End Task-Oriented Dialog Systems
Andrea Madotto, Chien-Sheng Wu, Pascale Fung
ACL 2018
Hongkyu Lim

2. Contents
• Overview
• Introduction
• Model Description
• Memory Encoder
• Memory Decoder
• Sentinel
• Memory Content
• Experimental Results
• Analysis and Discussion
• Conclusion

3. 3
Overview
• In Task Oriented Dialog system, it is hard to combine Knowledge
base(KB).
• Struggling to combine KB to RNN hidden states
• Time consumption : using attention mechanism
• Mem2Seq is a solution to solve the issues.
• Mem2Seq is a model that combines pointer network and attention.

4. 4
Introduction
• Task oriented dialog system is used to conduct particular objectives.
• It is essential to generate query with KB.
• Currently(2018), RNN based on hidden states has yielded good
performances.
•  But, there are still problems
• It is hard to comprehend KB and RNN hidden states
• Takes too long to process long sequences with attention

5. 5
Introduction
• MemNN
• A Recurrent attention model to utilize large external memory
• Reports embedding to the external memory
• Reads the memory repeatedly with query vectors
• This approach enables…
• Remembers KB for longer than before
• Encodes long sequential dialog fast
• However…
• MemNN only chooses from the pool.
• It does not generate answers.

6. 6
Model Description
• Mem2Seq
• Solves the limitations of MemNN
• Mem2Seeq relates concepts of pointer network to multi-hop attention mechanism.
• Mem2Seq copies words directly from KB
• Mem2Seq learns generating dynamic query to access to memory.

7. 7
Model Description
• Mem2Seq(architecture)
• Composed of MemNN Encoder and memory decoder
• MemNN Encoder makes vectors for dialog reports
• Memory Decoder generates responses by reading and copying memory

8. 8
Model Description
• Terms & Equations
• Sequence Tokens for dialog records
• $ is a special sign of token to generate words from memory content
• Tuple for Knowledge Base
• Concat of X and B

9. 9
Model Description
• Memory Encoder
• 𝑈 is a word wise concatenation of dialog and sentinel token.
• The memory of MemNN is represented as
• 𝐶 is a vector mapped with token used in reading query vectors.
• Repeated for K hops.
• For each memory sequence, the model calculates attention weights at hop k.

10. 10
Model Description
• Memory Encoder
• pk is responsible for memory selector to assign relations between memory
queries.
• The model reads memory ok through the sum of weights
• The result of the encoder is ok and it is the input of the decoder of Mem2Seq.

11. 11
Model Description
• Memory Decoder
• Uses both dialog records and KB
• GRU modules receives previously generated words and query to generate new
queries every time step t.
• Query h0 is the result of the Encoder
• In every step, the decoder computes vocabulary distributions and memory
contents distributions
• The decoder generates tokens at the memory by pointing the input words.

12. 12
Model Description
• Sentinel
• If memory has no required words, memory content distribution yields sentinel
words.
• Memory Content
• Dialog record is saved in the memory with respect to words.
• Speakers and time are added to each token.
• When saving KB, the token is based on subject, relations, and objects.
• KB is only used to consult on particular conversations.

13. 13
Experimental Results

14. 14
Analysis and Discussion
• Memory Attention
• As shown in the picture, the
distribution of weights is very clear.

15. 15
Conclusion
• Mem2Seq is a memory to sequence model for task—oriented dialog
system in end-to-end framework.
• Mem2Seq is combining multi-hop attention mechanism of end-to-end
memory network with pointer network.
• They validated the performance of Mem2Seq with experiments.

16. Thank you