Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Future of the Web with Conversational Interface

Using voice commands has been ubiquitous nowadays, as more mobile phone users use voice assistants like Siri, as well as devices like Amazon Echo and Google Home have been invading our living rooms. You can interact with the intelligent assistant without leaving your couch. At the same time, chatbots have gained popularity, and services like Slack and Facebook Messenger let you achieve tasks without leaving the client- you can schedule a meeting, order some pizza, call a taxi, etc.

Historically in web development, we have been relying on various UI elements to interact with users. Now with the new technologies, you can develop rich applications with the natural user interactions with a minimal visual interface. This enables countless use cases for richer and more accessible web applications.

In this talk, Tomomi Imura talks about the trend of the conversational interface, and what and how you can build with JavaScript.

The talk is given at:
- TokyoJS (shorter ver) July 28, 2017
- ModernWeb Taiwan Aug 10, 2017

  • Loggen Sie sich ein, um Kommentare anzuzeigen.

Future of the Web with Conversational Interface

  1. 1. Future of the Web with Conversational Interface BOTS, AI, AND JAVASCRIPT Tomomi Imura (@girlie_mac) #ModernWeb2017
  2. 2. @ girlie_mac ● advocate open web & technology ● code JavaScript & Node.js ● write & speak about tech ● hack useless stuff ● dev relations at Slack ● live in foggy San Francisco tomomi imura
  3. 3. "Bots are like new applications that you can converse with. " -- Satya Nadella, Microsoft
  4. 4. @ girlie_mac “We will evolve in computing from a mobile first to an AI first world.” -- Sundar Pichai, Google
  5. 5. @ girlie_mac CC BY-SA: nextdayblinds.com
  6. 6. @ girlie_mac Traditional Web & App Interactions
  7. 7. @ girlie_mac Modern Web & Apps with Social Interactions
  8. 8. @ girlie_mac Conversational User Interactions: Siri and Alexa (Voice Assistants) Alexa, how is the weather? Hey Siri In various form-factors
  9. 9. @ girlie_mac Conversational User Interactions for Kids - with Voice CogniToys Dino - https://cognitoys.com
  10. 10. @ girlie_mac Conversational User Interactions in a robot shape - with Voice
  11. 11. @ girlie_mac
  12. 12. @ girlie_mac Conversational User Interactions: Goole Assistant (Voice & Text)
  13. 13. @ girlie_mac Conversational User Interactions: SlackBots (Text) !
  14. 14. Slack Integrations & Bots for Better Productivity + + +
  15. 15. Graphic Interface to Conversational Interface
  16. 16. @ girlie_mac Deliver me a large margherita pizza! What kind of crust do you want? 1. Regular crust 2. Thin crust 3. Gluten free crust Thin crust
  17. 17. Natural user interactions with a minimal visual interface
  18. 18. No UI Clutter Less Time Spent
  19. 19. @ girlie_mac Where is the address of the place? It’s 325 Pearl Street. Come on now! Request a ride
  20. 20. @ girlie_mac Yes, get me a ride now Your driver, Sam will pick you up in 6 minutes. Look for the red Toyota Prius!
  21. 21. Old concepts. Technology caught up with the ideas.
  22. 22. @ girlie_mac “I'm sorry Dave, I'm afraid I can't do that”
  23. 23. @ girlie_mac “I'm sorry Dave, I'm afraid I can't do that”
  24. 24. Conversational Interface is:
  25. 25. ● Intuitive ● Accessible ● Productive
  26. 26. @ girlie_mac Messaging Platforms ● Slack ● Facebook Messenger ● Telegram ● WeChat ● Kik ● Viver ● LINE etc.
  27. 27. Messaging + Bots for More Interactive Communications
  28. 28. Slackbots at
  29. 29. Slackbots at basuke@sony.com
  30. 30. TacoBot by Taco Bell https://www.tacobell.com/feed/tacobot
  31. 31. @ girlie_mac Chatbot Building APIs ● API.ai (Google) ● Wit.ai (Facebook) ● Microsoft Bot Framework ● Motion.ai ● Converse AI ● Chatbots.io ● Recast.ai ● Flow XO ● ManyChat ● Fluent.ai (Voice UI) etc.
  32. 32. AIaaS Motion.ai API.ai Artificial Intelligence as a Service
  33. 33. @ girlie_mac Hi, I want to book a flight! Yes, from SFO. Where are you flying to? Taipei Hi Linda! Are you flying from San Francisco, as usual? An airline customer service bot Linda may not know she is talking to a bot!
  34. 34. @ girlie_mac NLP & Conversational Platforms / APIs Natural Language Processing & Cognitive platforms: ● IBM Watson ● Google Cloud Natural Language API ● Microsoft LUIS ● Amazon Lex ● Baidu UNIT
  35. 35. Build Your Own Conversational Interface
  36. 36. ...with JavaScript
  37. 37. @ girlie_mac { REST } JSSDK The APIs are mostly accessible with JS
  38. 38. @ girlie_mac Example: API.ai + FB Messenger const app = require('apiai')(CLIENT_ACCESS_TOKEN); function sendMessage(event) { let sender = event.sender.id; let text = event.message.text; let ai = app.textRequest(text, { sessionId: SESSION_STRING }); ai.on('response', (response) => { // Got a response. Let's POST to Facebook Messenger }); ai.end(); } API.ai Node.js SDK
  39. 39. @ girlie_mac Example: IBM Watson + Slack
  40. 40. @ girlie_mac
  41. 41. @ girlie_mac Example: IBM Watson + Slack The bot workflow: 1. Read each message on a Slack channel 2. Send the message to IBM Watson for examination 3. If the likelihood of an emotion is above the given confidence threshold post the most prominent emotion
  42. 42. @ girlie_mac Example: IBM Watson + Slack Slackbot Config: 1. Set up a bot user 2. Set OAuth permission scopes 3. Subscribe a bot event, so the bot can post messages on Slack channels IBM Bluemix Config for Watson
  43. 43. @ girlie_mac Example: IBM Watson + Slack app.post('/events', (req, res) => { let q = req.body; if (q.type === 'event_callback') { if(!q.event.text) return; analyzeTone(q.event); } }); Use Slack Events API to grab the text when a user post a message Pass the text data to Watson to analyze HTTP POST using ExpressJS
  44. 44. @ girlie_mac Example: IBM Watson + Slack const watson = require('watson-developer-cloud'); let tone_analyzer = watson.tone_analyzer({ username: process.env.WATSON_USERNAME, password: process.env.WATSON_PASSWORD, }); const confidencethreshold = 0.55; tone_analyzer.tone({text: text}, (err, tone) => { tone.document_tone.tone_categories.forEach((tonecategory) => { if(tonecategory.category_id === 'emotion_tone'){ tonecategory.tones.forEach((emotion) => { if(emotion.score >= confidencethreshold) { postEmotionOnSlackChannel(emotion); }}); }}); }); Returns emotions score in 0 to 1 Just initializing it w/ your API credentials Post the result on Slack
  45. 45. @ girlie_mac Example: IBM Watson + Slack + Raspberry Pi (for fun) function colorEmotion(emotion) { if (emotion.tone_id === 'anger') { setLED(red); } else if(emotion.tone_id === 'joy') { setLED(yellow); } else if(emotion.tone_id === 'fear') { setLED(purple); } else if(emotion.tone_id === 'disgust') { setLED(green); } else if(emotion.tone_id === 'sadness') { setLED(blue); } } Change the LED color to match the emotion
  46. 46. @ girlie_mac Ack, this sucks! I want my money back! An angry customer detected. Connect the customer with a human!
  47. 47. Conversational Interface with Voice in Browser?
  48. 48. @ girlie_mac Hello! 1 2 Howdy 3rd Party NLP API for artificial conversations Voice command: Voice -> Text 3 Text -> Synthetic Voice Text
  49. 49. @ girlie_mac Web Speech API Speech Recognition & Speech Synthesis http://caniuse.com/#feat=speech-recognition http://caniuse.com/#feat=speech-synthesis 1 3
  50. 50. @ girlie_mac Web Speech API: Speech Recognition const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition; const recognition = new SpeechRecognition(); Get an instance of the SpeechRecognition, the controller interface In the current Chromium, it is still prefixed
  51. 51. @ girlie_mac Web Speech API: Speech Recognition (Cont’d) recognition.lang = 'en-US'; recognition.interimResults = false; recognition.start(); recognition.addEventListener('result', (e) => { let last = e.results.length - 1; let text = e.results[last][0].transcript; }); Some properties Methods: start(), stop(), abort() Events: onresult, onerror, onaudiostarted, onaudioend, etc.
  52. 52. @ girlie_mac Web Speech API: Speech Synthesis const synth = window.speechSynthesis; const utterance = new SpeechSynthesisUtterance(); utterance.text = 'I am a robot'; utterance.pitch = 1.5; utterance.voice = 'Alex'; synth.speak(utterance); No vendor prefix Properties of the SpeechSynthesisUtterance interface Get available voices with synth.getVoices() method
  53. 53. @ girlie_mac
  54. 54. Demo on Chrome https://webspeech.herokuapp.com/
  55. 55. @ girlie_mac https://www.smashingmagazine.com/2017/08/ai-chatbot-web-speech-api-node-js/
  56. 56. @ girlie_mac
  57. 57. The future was here already. Long Live Clippy! 1997 - 2007
  58. 58. @ girlie_mac 謝謝 ! @girlie_mac girliemac.com github.com/girliemac slideshare.net/tomomi
  59. 59. @ girlie_mac Attribution: Open Emoji by Emoji-One (CC-BY 4.0)

×