Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Embedding Intelligence in Everyday Objects with TJBot

946 Aufrufe

Veröffentlicht am

TJBot is a DIY open source kit that allows you to build your own programmable cardboard robot powered by Watson. It consists of a cardboard cutout (which can be 3D printed or laser cut), Raspberry Pi and a variety of add-ons – including a RGB LED light, a microphone, a servo motor, and a camera. This presentation provides an overview of how Watson Cognitive Services are leveraged to create capabilities within TJBot, and how to build simple applications for TJBot using Node.js.

Veröffentlicht in: Geräte & Hardware
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Embedding Intelligence in Everyday Objects with TJBot

  1. 1. IBM Research 1 IBM Research Victor Dibia Embedding Intelligence in Everyday Objects with TJBot. An open source DIY project powered by Watson Cognitive Services. Human – Agent Collaboration Lab, IBM Research dibiavc@us.ibm.com @vykthur | github.com/victordibia Feb 20, 2017
  2. 2. IBM Research 2 TJBot : What and Why? - Open source DIY project to get you engaged with Watson Services
  3. 3. IBM Research 3 What is TJBot? - A cardboard robot - Simple, approachable - Open Source (design, code) - Cognitive (IBM Watson services) - Extensible (prototyping platform) Components: Raspberry Pi, LED, Camera, Microphone, Speaker, Servo.
  4. 4. IBM Research 4 3D Print Laser cut
  5. 5. IBM Research 5 ibm.biz/mytjbot
  6. 6. IBM Research 6 Recipes. Step by step instructions + Code (node.js) to help you prototype capabilities for TJBot powered by Watson services. http://www.instructables.com/member/TJBot/
  7. 7. IBM Research 7 Project Goals How can we make it easier to engage a community of enthusiasts experimenting with embodied cognition – the idea of embedding intelligence in everyday objects within the physical world?
  8. 8. IBM Research 8 Project Goals Design principle – Approachable Design - Use of familiar material (cardboard) that can be altered with ease. - Simplified part assembly: no soldering or adhesive required. - Simplified programming model and language interface (JavaScript).
  9. 9. IBM Research 9 Project Outcome A prototyping platform to help democratize Embodied Cognition. Target communities: - Makers - Developers - Students (Education and Learning)
  10. 10. IBM Research 10 How Does Watson Enable TJBot? Listen Watson Speech to Text service converts spoken speech to text that can be analyzed Speak Watson Text to Speech service service converts text to sound using various voices. Understand Emotions Watson Tone Analyzer service can infer the emotion within text. E.g.. it can tell if a message contains emotions like happy , sad, angry Understand Conversations Watson Conversation Service can respond to users in a way that simulates a conversation between humans. See Watson Visual Recognition service can understand the content of an image and describe it.
  11. 11. IBM Research 11TJBot Sensors Example Capabilities Example Watson Services Example Use cases LED Speakers CameraServo Motor Arm Microphone Listen Speak Shine Show emotion Wave See Speech to text Tone Analyzer Vision Recognition Conversation Text to speech Sentiment Analysis Virtual Agents (eldercare, home care) Education (language learning)
  12. 12. IBM Research 12 Demo. - Watson Services - Speech to text - Text to speech - Conversation - Visual Recognition
  13. 13. IBM Research 13 Overview of Watson Services
  14. 14. IBM Research 14 IBM Watson Cognitive Take your first step into the cognitive era with our variety of smart services. Services. - Natural interaction - Semi-structured data processing - Trained and continuously improved via machine learning and deep learning. - Restful API services with SDKs for node.js, java, python.
  15. 15. IBM Research 15 Language
  16. 16. IBM Research 16 Speech Vision Data Insights
  17. 17. IBM Research 17 Speech Vision Data Insights
  18. 18. IBM Research 18 Speech to Text Converts audio voice into written text. • Transcription • Voice-controlled applications: allows for custom models https://speech-to-text-demo.mybluemix.net/
  19. 19. IBM Research 19 Text To Speech Converts written text into natural sounding audio in a variety of languages and voices. • Customize and control the pronunciation of specific words to deliver a seamless voice interaction that catered s to your audience. • Interactive voice based applications. https://text-to-speech-demo.mybluemix.net/
  20. 20. IBM Research 20 Tone Analyzer Uses linguistic analysis to detect three types of tones in written text: emotions, social tendencies, and writing style. • Understand emotional context in conversations or communications • Taylor interaction based on sentiment. https://tone-analyzer-demo.mybluemix.net/
  21. 21. IBM Research 21 Visual Recognition Understands the contents of images - visual concepts tag the image, find human faces, approximate age and gender, and find similar images in a collection. • Train the service by creating your own custom concepts. Use Visual Recognition to detect a dress type in retail, identify spoiled fruit in inventory, and more. https://visual-recognition-demo.mybluemix.net/
  22. 22. IBM Research 22 AlchemyLanguage Analyzes text to help you understand its concepts, entities, keywords, sentiment, and more. • Additionally, you can create a custom model for some APIs to get specific results that are tailored to your domain. https://alchemy-language-demo.mybluemix.net/
  23. 23. IBM Research 23 Conversation Quickly build, test and deploy a bot or virtual agent across mobile devices, messaging platforms like Slack or even on a physical robot. • Visual dialog builder to help you create natural conversations between your apps and users, without any coding experience required. https://conversation-demo.mybluemix.net/
  24. 24. IBM Research 24 Programming TJBot - Getting started - Tying stuff together
  25. 25. IBM Research 25 Steps. - Wifi Setup, - Raspberry Pi Software update - Hardware setup : LED, Servo, Microphone, etc - Credential Setup : Bluemix - Recipe software setup : Clone Github repo - Ready to go.
  26. 26. IBM Research 26 http://www.instructables.com/member/TJBot/ Instructions
  27. 27. IBM Research 27 Libraries Used Depends on several npm packages. - RGB LED – ws281x library - Servo – pigpio software PWM library - Microphone – mic library - Speaker – aplay library - Camera – raspistill wrapper
  28. 28. IBM Research 28 Code Walk through: Control LED on TJBot using voice. - http://www.instructables.com/i d/Use-Your-Voice-to-Control-a- Light-With-Watson/ - Code Walk through
  29. 29. IBM Research 29 TJBot Library [Beta] - Experimental work to encapsulate basic functions of the bot. - https://github.com/ibmtjbot/tjbotlib
  30. 30. IBM Research 30 The TJBot Library Encapsulate basic functions for TJBot such as listening, speaking, led color change, waving, seeing.
  31. 31. IBM Research 31 The TJBot Library tj.listen(transcript callback) tj.speak(“text”) tj.converse() tj.see() tj.shine(“red”)
  32. 32. IBM Research 32 Code Walk through: Control LED using the TJBot library. - https://github.com/ibmt jbot/recipes - Code Walk through
  33. 33. IBM Research 33 Open Issues - Improving accuracy - Bot “Interruptibility” - Gracefully managing latency
  34. 34. IBM Research 34 Improving Accuracy How do we improve interaction (voice) accuracy? Improving Speech-to-Text models may not be enough! - Customized language models? - Intent Matching? - Multi-turn conversations?
  35. 35. IBM Research 35 Bot “Interruptibility” When and how should the robot be interrupted (while performing an activity like speaking, waving etc.)? - Vision? (monitoring a user’s facial expression, raised hand) - Hardware button or sensor?
  36. 36. IBM Research 36 Latency Tolerance Latency can severely degrade quality of interaction. How do we minimize its effect? - Managing and ordering service responses - Leverage cues to provide additional information - Balancing capabilities – cloud vs local processing.
  37. 37. IBM Research 37 Next Steps
  38. 38. IBM Research 38 Next Steps 3 pronged - Conduct basic research that address open issues. - Make TJBot simpler and easier to use (tjbot library, visual programming tool) - Build and sustain the TJBot community.
  39. 39. IBM Research 39 Learn more? - Ibm.biz/mytjbot - http://www.instructables.com/ member/TJBot/
  40. 40. IBM Research 40 Thank You!