Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
To organize the world's information and make it universally accessible and useful.
Google Home/Mobile device
The Google Assistant —
Actions on Google —
Assistant App
Well, it’s kind of cold outside, so I'd like
something to warm me up, like a hot
soup, and I want it fast.
I have some chi...
Assistant app
{ conversation
API request }
{ conversation
API response }
user input
app
response
Well, it’s kind of cold outside, so I'd like
something to warm me up, like a hot
soup, and I want it fast.
I have some chi...
“
Intent Matching —
Entity Extraction —
@
const App = require('actions-on-google').ApiAiApp;
exports.yourApp = (request, response) => {
const app = new App({request...
const app = new ApiAiApp({request: request, response: response});
const WELCOME_INTENT = 'input.welcome'; // the action na...
if (app.hasSurfaceCapability(app.SurfaceCapabilities.SCREEN_OUTPUT)) {
app.ask(app.buildRichResponse()
.addSimpleResponse(...
g.co/actionswebsim
Hey! This is ___
Welcome to ___
Ready to play ____
Hi! ___ here.
Hello. I’m ___
Greetings, human.
Welcome back to ___
Hey ...
<speak> <!-- Must be at the start of the string -->
<say-as interpret-as="characters">SSML</say-as>
<break time="3s"/>.
<a...
function welcome (app) {
return isPreviousUser(app.getUser().userId).then((userHasVisited) => {
if (userHasVisited) {
app....
function simpleResponse (app) {
app.ask({
speech: 'Howdy! I can tell you fun facts about ' +
'almost any number, like 42. ...
function suggestionChips (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse({
speech: 'Howdy! I can tell you fun f...
function basicCard (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse('Math and prime numbers it is!')
.addBasicCa...
let permission = app.SupportedPermissions.DEVICE_COARSE_LOCATION;
app.askForPermission('To find bookstores near you', perm...
Ok Google, ask Personal Chef for a hot soup recipe
Trigger Phrase App Name
Developer Specified
Action
Preposition
Action P...
Conversation Design
GUI :: VUI What’s the difference?
Top 3 Design Tips
Create a persona.1
Persona is conveyed through:
Tone
Word and phrase choices
Functional design
Style
Technique
Voice
...and based on:
Your us...
Think outside the box. Literally.2
Confidential and proprietary
code code code code code
code code code code
code code “ dialog string” code
code code code
c...
Confidential and proprietary
code code code code code
code code code code
code code “ dialog string” code
code code code
c...
Confidential and proprietary
DIALOG + STRUCTURE
In conversations, there are no “errors”.3
real life
user
conditions{
“I don’t understand you”
When a so-called “error” occurs in a
conversation, it should be treated simply as
a new turn in the dialog, only with diff...
Users have intent
developers.google.com/actions
goo.gl/PMdr7N
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Building Conversational Experiences with Actions on Google
Nächste SlideShare
Wird geladen in …5
×

Building Conversational Experiences with Actions on Google

324 Aufrufe

Veröffentlicht am

Since we first started using computers, we have dreamt of interacting with them in a more natural, human-like way using spoken language. Advances in machine learning and artificial intelligence have spiked an interest in natural language processing, and building conversational interfaces no longer is a dream, but a reality every developer can use in their own applications.

The problem of recognising spoken input has been largely solved, and the next challenge is how to build compelling user experiences that are modelled after natural human conversation.

In this talk, I will give an introduction to the Google Assistant and how to use Actions on Google to build conversational interfaces for devices with and without screens. We will take a look at how API.AI makes implementing natural language understanding feasible even for non-linguists, and discuss how you can integrate agents with existing backends. In conversation, it's all about being natural, so we will also look at how to create agents that feel like real characters.

Veröffentlicht in: Software
  • Als Erste(r) kommentieren

Building Conversational Experiences with Actions on Google

  1. 1. To organize the world's information and make it universally accessible and useful.
  2. 2. Google Home/Mobile device The Google Assistant — Actions on Google —
  3. 3. Assistant App
  4. 4. Well, it’s kind of cold outside, so I'd like something to warm me up, like a hot soup, and I want it fast. I have some chicken, and also canned tomatoes.
  5. 5. Assistant app { conversation API request } { conversation API response } user input app response
  6. 6. Well, it’s kind of cold outside, so I'd like something to warm me up, like a hot soup, and I want it fast. I have some chicken, and also canned tomatoes.
  7. 7. “ Intent Matching — Entity Extraction — @
  8. 8. const App = require('actions-on-google').ApiAiApp; exports.yourApp = (request, response) => { const app = new App({request, response}); console.log('Request headers: ' + JSON.stringify(request.headers)); console.log('Request body: ' + JSON.stringify(request.body)); // Fulfill action business logic function responseHandler (app) { // Complete your fulfillment logic and send a response app.ask('Hello, World!'); } const actionMap = new Map(); actionMap.set('<API.AI_action_name>', responseHandler); app.handleRequest(actionMap); };
  9. 9. const app = new ApiAiApp({request: request, response: response}); const WELCOME_INTENT = 'input.welcome'; // the action name from the API.AI intent const NUMBER_INTENT = 'input.number'; // the action name from the API.AI intent const NUMBER_ARGUMENT = 'input.mynum'; // the action name from the API.AI intent function welcomeIntent (app) { app.ask('Welcome to action snippets! Say a number.'); } function numberIntent (app) { let number = app.getArgument(NUMBER_ARGUMENT); app.tell('You said ' + number); } let actionMap = new Map(); actionMap.set(WELCOME_INTENT, welcomeIntent); actionMap.set(NUMBER_INTENT, numberIntent); app.handleRequest(actionMap);
  10. 10. if (app.hasSurfaceCapability(app.SurfaceCapabilities.SCREEN_OUTPUT)) { app.ask(app.buildRichResponse() .addSimpleResponse(`Here's a fact for you. ${fact} Which one ` + `do you want to hear about next, Google's history or headquarters?`) .addBasicCard( app.buildBasicCard('Google is an amazing company.') .setImage(GOOGLE_LOGO_SRC)) .addSuggestions(['History', 'Headquarters'])); } else { app.ask(`Here's a fact for you. ${fact} Which one ` + `do you want to hear about next, Google's history or headquarters?`); }
  11. 11. g.co/actionswebsim
  12. 12. Hey! This is ___ Welcome to ___ Ready to play ____ Hi! ___ here. Hello. I’m ___ Greetings, human. Welcome back to ___ Hey again. ___ Let’s play ___ Here’s your ___ Brought to you by ___ Hi there, ___ Let’s get started. Ready for your ___ ___, here to… Live from ___ This is ___ What’s up, ___ and more...
  13. 13. <speak> <!-- Must be at the start of the string --> <say-as interpret-as="characters">SSML</say-as> <break time="3s"/>. <audio src="https://example.com/file.mp3"></audio> <say-as interpret-as="cardinal">10</say-as>. <say-as interpret-as="ordinal">10</say-as> <say-as interpret-as="characters">10</say-as>. <sub alias="World Wide Web">WWW</sub>. <p><s>This is one.</s><s>This is two.</s></p> </speak> <!-- Must be at the end of the string --> “S S M L” [3 second pause] [audio file plays] “Ten” “Tenth” “One Oh” World Wide Web [two sentences]
  14. 14. function welcome (app) { return isPreviousUser(app.getUser().userId).then((userHasVisited) => { if (userHasVisited) { app.ask(`Welcome to Number Genie!...`, NO_INPUT_PROMPTS); } else { app.ask(`Hey you're back...`, NO_INPUT_PROMPTS); } }); }
  15. 15. function simpleResponse (app) { app.ask({ speech: 'Howdy! I can tell you fun facts about ' + 'almost any number, like 42. What do you have in mind?', displayText: 'Howdy! I can tell you fun facts about ' + 'almost any number. What do you have in mind?' }); }
  16. 16. function suggestionChips (app) { app.ask(app.buildRichResponse() .addSimpleResponse({ speech: 'Howdy! I can tell you fun facts about ' + 'almost any number like 0, 42, or 100. What number do you have ' + 'in mind?', displayText: 'Howdy! I can tell you fun facts about almost any ' + 'number. What number do you have in mind?' }) .addSuggestions(['0', '42', '100', 'Never mind']) ); }
  17. 17. function basicCard (app) { app.ask(app.buildRichResponse() .addSimpleResponse('Math and prime numbers it is!') .addBasicCard( app.buildBasicCard(`42 is an even composite number. It ` + `is composed of three distinct prime numbers multiplied together. It ` + `has a total of eight divisors. 42 is an abundant number, because the ` + `sum of its proper divisors 54 is greater than itself. To count from ` + `1 to 42 would take you about twenty-one…`) .setTitle('Math & prime numbers') .addButton('Read more') .setImage('https://example.google.com/42.png', 'Image alternate text') ) ); }
  18. 18. let permission = app.SupportedPermissions.DEVICE_COARSE_LOCATION; app.askForPermission('To find bookstores near you', permission);
  19. 19. Ok Google, ask Personal Chef for a hot soup recipe Trigger Phrase App Name Developer Specified Action Preposition Action Phrase Developer Specified Ok Google, let’s talk to Personal Chef Trigger Phrase App Name Developer Specified
  20. 20. Conversation Design
  21. 21. GUI :: VUI What’s the difference?
  22. 22. Top 3 Design Tips
  23. 23. Create a persona.1
  24. 24. Persona is conveyed through: Tone Word and phrase choices Functional design Style Technique Voice ...and based on: Your user population and their needs The imagery & qualities associated with your brand
  25. 25. Think outside the box. Literally.2
  26. 26. Confidential and proprietary code code code code code code code code code code code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE + CODE
  27. 27. Confidential and proprietary code code code code code code code code code code code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE + CODE
  28. 28. Confidential and proprietary DIALOG + STRUCTURE
  29. 29. In conversations, there are no “errors”.3
  30. 30. real life user conditions{
  31. 31. “I don’t understand you”
  32. 32. When a so-called “error” occurs in a conversation, it should be treated simply as a new turn in the dialog, only with different conditions.
  33. 33. Users have intent
  34. 34. developers.google.com/actions
  35. 35. goo.gl/PMdr7N

×