Natural human to machine communication / Geektime Devfest 2016 By: Alon eirew

1. Natural Human to
Machine
Communication
By Alon Eirew

2. Who Am I
• Software Architect at Intel Multimodal Language
Understanding Group
• Smart Home, Robotics, Smart Car, Mobile & Wearables
• Lead Developer at Intel New Devices Group
• Personal Assistant for Mobile and Wearables
• Lead Developer Telmap (Navigation App)
• Navigation Assistant for Mobile
https://il.linkedin.com/in/aloneirew

3. 3
Agenda
 What is Natural Communication?
 Why is it interesting?
 Overview the components and see some
examples

4. 4
Human to Human

5. Natural Communication Feature

7. Change the way we are communicating with our smart devices

8. 8
Technologies that can’t support conventional input sources

9. The Era Of Conversational UI
How can I help
you?

10. 10
Building Blocks
Hardware
Recognition
Understanding
1
2
3

11. 11
Hardware and Recognition Modules
Automatic Sound
Recognition
Speech To Text
Computer Vision
Object Recognition
Person Recognition
Gesture Recognition

12. Intent Extraction (NLU)
12
Please turn
off the lights
in the living
room at
4PM
Location: living room
Time: 4PM
Object: lights
Action: turn off
Subject: (you)

13. World Knowledge
13

14. User Knowledge
14

15. 15
Knowledge Use Cases

16. Semantic Fusion
Object [1] [N-Best]
Label Prob Location
Chair 0.7
x1, y1, z1
Stool 0.4
Table 0.2
Label Prob Location
Table 0.8 x4, y4, z4
Stool 0.3 x4, y4, z4
Object 0.2 x4, y4, z4
Object [2] [N-Best]
Gesture List
Object Rec 1
Object Rec 2
End Point
Clean
next to
this ????
????
Word Prob
School Desk 0.7
Ship Deck 0.5
Sport Deko 0.2
ASR [N-Best]
Object [3] [N-Best]
Label Prob Location
Plant 0.7
x7, y7, z7
Tree 0.4
Object 0.2

17. Dialog Features
17

18. Thank You!