Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

The Convergence of Robotics, the Web, and the IoT

598 Aufrufe

Veröffentlicht am

Principal Engineer Michael McCool discusses the convergence of Robotics, the web, and the Internet of Things.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

The Convergence of Robotics, the Web, and the IoT

  1. 1. The Convergence of Robotics, the Web, and the Internet of Things Michael McCool, Principal Engineer, Intel IoT Dev Fest, June 4, 2017
  2. 2. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Abstract What do the web, the IoT, and robotics have in common? For one, they can all benefit from the network effect, where the whole is greater than the sum of the parts. For the Web, the network allows distribution of information between agents anywhere in the world. For the IoT, the focus is on the remote monitoring and coordination of physical sensors and actuators, and the analysis of data to provide actionable insight. Robots can be defined as autonomous cyberphysical systems that can sense and manipulate their environment. Although not essential to the definition, robots are often connected to the internet as well, and can use it to enhance or enable their function in various ways: supporting, for example, telepresence on the web, remote sensing integrated with the IoT, or coordination between multiple robots and IoT devices, or just to offload computation. In the past these three ecosystems had their own software frameworks and standards but there are indications they are converging: IoT systems often integrate with cloud services that have web APIs. Web API standards are being used to define the data models for IoT devices. Machine learning models can be trained with data from the Web and used for real-time perception in robots. IoT gateways can be used to command robots by voice. Robots can serve their own graphical user interfaces over the Web. Robots are adopting service-oriented distributed software architectures and programming models originally developed for web servers. IoT devices and radio networks can be used a navigation beacons and even raw sensory data for robots. In this talk, I will talk about a number of these convergences and what the implications are. 2
  3. 3. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Outline and Agenda • Ecosystem Overview • Web, IoT, and Robotics • Requirements and technologies • Commonalities and differences • Trends and Technologies • Embedded HPC and AI • Remote compute offload • Real-time (sub)systems • Semantic data modelling • Emerging Standards • OSRF*/ROS2* and OMG*/DDS* • OCF* and W3C* Web of Things • OpenFog* 3 Web IoT Robotics
  4. 4. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice The World-Wide Web • “Distributed information space with resources identified by URLs” • Hyperlinked documents • Delivered by Internet • Displayed in browsers • Many, many extensions… • Search engines • Web service RESTful APIs • Applications as content • Semantic markup 4 https://en.wikipedia.org/wiki/World_Wide_Web This Wikipedia and Wikimedia Commons image is from the user Chris 73 and is freely available at //commons.wikimedia.org/wiki/File:WorldWideWebAroundWikipedia.png under the creative commons cc-by-sa 3.0 license.
  5. 5. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Technologies Engendered by the Web • Web Services • RESTful: Set of properties that allows an online service to scale gracefully • Stateless interface, small set of verbs, structured data • Metadata: RAML, Swagger/OpenAPI • Cloud Services: X-aaS, SD-X, virtualization… • “Software applications as content” • Containers and Microservices • Docker, Snaps, etc. • Compose services out of loose collections of other services • Flexible structured data formats (and now efficient binary encodings…) • JSON  CBOR; metadata: JSON-Schema • XML  EXI; metadata: XSI 5
  6. 6. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Semantic Web To enable Web 3.0:  Allow the meaning (semantics) of data to be understood Technologies to facilitate data interchange:  RDF: Resource Description Framework  OWL: Web Ontology Language  SPARQL: Protocol and RDF Query Language  Syntaxes: XML, N3, JSON-LD, etc.  Ontologies (Vocabularies): Haystack, Schema.org, etc. 6/37
  7. 7. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice RDF: Resource Description Framework 7/37  General approach for describing data as sets of “triples”  Subject-Predicate-Object  RDF triples are stored in a distributed fashion  Federated database  Subsumes relational databases  Standard terms and relations are defined in ontologies (“vocabularies”), including RDF and OWL Socrate s Man Is-a Mortal IsIs
  8. 8. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Internet of Things Network of devices, each capable of communication and associated with a set of sensors and/or actuators. • Enable numerous applications • Multiple verticals: home, industrial, city, building, vehicle, etc. • Typically rely on backend systems for data analytics and services (eg web services) • Edge + “infrastructure” nodes… 8 https://commons.wikimedia.org/wiki/File:Nest_Learning_Thermostat_(cropped).JPG
  9. 9. Optimization Notice Internet of Things (IoT) Systems Gateway Gateway Analysis (Server) Human Interface Wi-Fi* LTE Wired Endpoint Endpoint
  10. 10. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Robots • Cyberphysical systems with the capability to autonomously interact with their environments • Ideally capable of autonomous • Perception • Planning • Mobility • Manipulation • Human-Robot Communication • Note: typical “Industrial Robots” may only be capable of a subset of these 10
  11. 11. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Typical Robot Software Stack Servos Wheel Motors RGB Camera Depth Camera Motion Touch Mic Array Discrete SensorsCamera Drivers Motor Drivers SensorFusion Point Cloud 2DMap Body Tracking Face Tracking Object Recognition RGBD+IMU / SLAM Speech Recognition Behavior Control Reflex Goals Interact Speech Output Dialog Control User/ World Knowledge Gesture / Action World View Robot State Command Inputs Behavior Arbitrator Path Planning Movement External Device Control Voice Command External Controller Gesture Input Environment Stimulus Behavior Behavior Behaviors Applications Acoustics OptionalInputs Cloud offload & control IR Ultra- Sonic Sense Plan Act Sensor Hardware and Drivers Actuators
  12. 12. • Sense a potentially dynamic environment • Determine current location • Plan path and then navigate from one location to another • Avoid obstacles • Interact with humans as needed (avoid, follow, respond to commands, etc.) Mobile Robotics 12
  13. 13. • Speech recognition • Face recognition • Body pose understanding • Object recognition • Dialogue management • Context awareness • Emotional state recognition • Social relationship understanding Social Robotics 13
  14. 14. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice 14
  15. 15. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Robotics and the IoT Robots are… Cyberphysical systems with the capability to autonomously interact with their environments  Not just passive sensors If connected to the internet, robots may also participate in the IoT ecosystem  Share experiences via the network  Use sensor data from other IoT devices  Offload work to external systems  Access large amounts of data for training data  Coordinate motion across multiple robots/devices Next-generation connected robots are explicitly leveraging IoT technologies and middleware But they also need to be autonomous 15
  16. 16. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Comparative Requirements 16 Web IoT Robots Scalability Connectivity Scalability Low cost Low power Functional safety Connectivity Functional safety High performance Real time Reliability Availability Flexibility Security Reliability Availability Flexibility Security Reliability Availability Flexibility Security
  17. 17. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Emerging Trends and Technologies Artificial Intelligence • Machine Learning • Perception • Semantic Processing Embedded HPC (especially for Robotics) • Computer Vision and Speech • Localization and Mapping Distributed computing • Remote (cloud, fog) compute offload • Service-oriented computing Real-time (sub)systems • Control systems • Functional safety 17
  18. 18. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Case Studies ROS*: Robot Operating System • Microservices architecture, Web integration, ROS2, OMG’s DDS, embedded perception OCF*: IoT standardization effort • Microservices architecture using CoAP/CBOR, IoTivity implementation of Smart Home example system W3C* Web of Things: IoT standardization effort • Standardized metadata for microservices architecture OpenFog*: Edge computing standardization effort • Infrastructure support for microservices architecture 18
  19. 19. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice ROS*: Robot Operating System • Supports distributed system based on a graph of communicating nodes • Both pub-sub and request- response patterns • Large community with many existing nodes for sensors, planning, navigation, etc. • Various visualization tools and simulators are also provided
  20. 20. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Autonomous Navigation ROS* Node Graph
  21. 21. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Rviz: Robot Visualization
  22. 22. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Robot Web Tools* and ROSWeb* http://robotwebtools.org/ and https://github.com/EESC-LabRoM/rosweb 22 ROSWeb Robot Manager • Manage/control ROS robot via web console • Uses rosbridge, which bridges ROS to a JSON- over-Websocket protocol • Javascript library can then access ROS functionality either from Node.js or a browser • Can also use WebGL, TLS, D3S, Google maps, etc.
  23. 23. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice ROS2* Designed to resolve a number of issues with current ROS architecture: • Real-time support • Quality of service control in communications • Distributed node management • Lifecycle management • Determinism  Built on top of OMG’s DDS standards, a middleware framework for industrial IoT systems, that provides real-time pub-sub https://github.com/ros2/ros2/wiki and http://www.omg.org/spec/DDS/
  24. 24. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice SAWR Project Simple Autonomous Wheeled Robot • Demonstration system for autonomous robotics • Can be built using either • Intel® RealSense™ Robotic Development Kit, which combines Aaeon’s UP* Board and an Intel® RealSense R200 camera • Intel® Joule™ compute module and an Intel® RealSense ZR300 camera • OSH design: laser-cut frame/BOM https://github.com/01org/sawr
  25. 25. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Robotis* Turtlebot 3 25 Modular Robotics Research and Training Platform • Intel® Joule™ compute module • Intel® RealSense R200 camera • Laser triangulation 360 degree rangefinder • Real-time Arduino-based subsystem http://turtlebot3.robotis.com/en/latest/hardware.html
  26. 26. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Intel Confidential Intel® RealSense™ SDK: Perceptual Capabilities Objects Database Library of pre- trained or customer specific objects Object Tracking Track objects of interest, even when robot and/or object are moving Object Localization Determine where objects are in the scene Object Segmentation Pixels that are part of the object for manipulation or removal
  27. 27. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice
  28. 28. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Person Interaction
  29. 29. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice OCF* & IoTivity* Specification Open Source Coordination Business (Marketing, Strategy, Membership) Open Source Project Reference Implementation of OCF (and Non-Spec Related Code) Sponsored (funded) by OCF Separate Governance Coordination April 2017 Open Connectivit y Foundation Public 29 Certification Data Modelling Specs & Open Source ready simultaneously
  30. 30. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice OCF – Conceptual Framework 30 April 2017 Core Framework Profiles Consumer Enterprise Industrial Auto Education Health Security, Identity & Permissions Discovery Data Transmission Data Management Device Management Transports (Smart) Remote Access Cloud Resource Model Interaction / Data Model Open Connectivit y Foundation Public
  31. 31. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice OCF – Resource Model 31 April 2017 Resource URI rt: Resource Type if: Resource Interface prop: Policy n: Resource Name … … Common Properties Resource Specific Identifies the type of resource List of interfaces associated with the resource Policy associated with resource: discoverable, observable, secure, etc Friendly name § The resource model, coupled with the common data model, provides the base interoperability of OCF § Any physical or software artifact on a device that needs to be manipulated or made visible across the network can be described via the resource model § A resource has a URI and a collection of Properties links: Other Resource URI Resources can include links to other resources Open Connectivit y Foundation Public
  32. 32. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Resource Model - examples 32 April 2017 14 Resource URI :/a/light1 rt: oic.ex.light if: oic.if.rw prop: discoverable, observable n: myHallWayLight State: 0 (OFF) Dim Level: 0 Resource URI :/a/fan1 rt: oic.ex.fan if: oic.if.rw prop: discoverable n: myKitchenFan State: 1 (ON) Speed: 10 Open Connectivit y Foundation Public
  33. 33. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice CREATE  Create a new Resource on the Server RETRIEVE  Get the current State or representation of a Resource from a Server UPDATE  Request a partial or complete update to the information stored in a Resource DELETE  Remove a Resource from the Server NOTIFY  Request asynchronous notification of state changes in a Resource  Special-cased to support OBSERVE for publish/subscribe support Common Interaction Model – Transport Agnostic April 201733 Open Conne ctivity Found ation Public
  34. 34. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Let’s look deeper at a Resource … 34 Server - DevA /dimming /oic/d /oic/res Main Living Room Light /switch /oic/d /oic/p oic.r.switch.binary oic.wk.d oic.wk.p false100 /switch true /dimming oic.r.light.brightness /oic/p LED-Light-Co, 40W Equivalent Bulb
  35. 35. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice There can be more Properties … 35 /oic/p mnml mnmn pi http://ledlight.com/gg3-40 Led-Light-Co GoGlow III - 40W mnmo GG3-40
  36. 36. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Resource Type - Data Modeling 36 JSON Schema* • Describes the Properties and Representation ReSTful API Modelling Language (RAML)* • Describes the Request and Response *JSON Schema and RAML are normative and also can be used for code generation "oic.r.switch.binary": { "type": "object", "properties": { "value": { "type": "boolean", "description": "Status of the switch" } } } get: responses : 200: body: application/json: schema: | { } Defined by: (see OIC_Resource_Type_Specification)
  37. 37. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice OCF/IoTivity Smart Home Demo https://github.com/01org/SmartHome-Demo 37 Prototype Smart Home • Example architecture for Smart Home • Based on OSS IoTivity implementation of OCF • Combines cloud services, gateways, and sensor nodes • Technologies: Docker containers, Node.js, Zephyr RTOS, HTTPS, CoAP, WebGL, RAML, JSON/CBOR
  38. 38. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice W3C Web of Things https://www.w3.org/WoT/ and https://github.com/w3c/wot 38 Thing API A Applications Thing API B Applications Interaction Model Interaction Model Protocol Mappings Protocol Mappings CoAP OneM2M CoAP OCF CoAP BACnetBridging Gateway Thing Description HTTP W3C Working Group • Standardize descriptions of data and interaction models for IoT devices (“Things”) • Support semantic interoperability via linked data, JSON- LD, RDF, IoT ontologies • Bridge multiple other standards
  39. 39. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice https://www.openfogconsortium.org/ 39 Description • Reference architecture for distributed “fog” computation • Fog: cloud computing, but supported throughout the network, both in data centers and at the edge
  40. 40. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Summary and Conclusions • Web, IoT, and Robotics ecosystems to interoperate, but not merge • Several key differences mean completely shared technology base is not feasible • However, many core components can and should be shared • Many key “IoT” services will depend on these systems interoperating • New opportunities and challenges • Moving cloud services to the edge: • Embedded microservice architectures, fog computing, containerization • Using web technologies and infrastructure in IoT and Robotics: • RESTful APIs, cloud services, semantic interoperability • Software development challenges: • Data-centric, distributed computing, microservices, scaling, security 40
  41. 41. Copyright © 2017, Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.Optimization Notice Legal Disclaimer & Optimization Notice INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS”. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. Copyright © 2014, Intel Corporation. All rights reserved. Intel, Pentium, Xeon, Xeon Phi, Core, VTune, Cilk, and the Intel logo are trademarks of Intel Corporation in the U.S. and other countries. Optimization Notice Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision #20110804 41

×