This webinar presentation discussed frontiers in visual and tactile sensing for robotic design. It covered advances in computer vision that have enabled perception capabilities for robots in unconstrained environments. Examples of embedded vision systems in automobiles and challenges in implementing computer vision on devices were presented. The presentation concluded with a discussion of the future potential for biomimetic tactile sensing solutions to allow robots to perform delicate human tasks through sensitive touch.

1. Robotic Design: Frontiers in
Visual and Tactile Sensing

2. Thank You To Our Sponsors

3. Before We Start
q This webinar will be available afterwards at
www.designworldonline.com & via email
q Q&A at the end of the presentation
q Hashtag for this webinar: #DWwebinar

4. Moderator Presenters
Paul Heney
Design World
Gerald Loeb
Biomed Concepts
SynTouch
Goksel Dedeoglu
PercepTonic

5. Robotic Design: Frontiers in
Visual and Tactile Sensing
presented at the
December 4, 2014
Goksel Dedeoglu, Ph.D.
Founder, PercepTonic, LLC

6. Robotics Then…
Moravec, 1980
SRI’s Shakey, 1967
The perception needs of robots have
motivated much of the research in
Computer Vision over the past 50 years.
© PercepTonic, LLC

7. Moravec, 1980
SRI’s Shakey, 1967
thanks to
50 years
of consumer
electronics
Pelican Imaging
HTC M8
Subaru
XBOX Kinect
Robotics Today
Prox Dynamics
MAST
Brain Corp.
© PercepTonic, LLC

8. The Promise of Computer Vision
Computer Vision has advanced from factory-floor
automation to real-world problem solving
in unconstrained environments
Embedded
Ever-Ready
Everywhere
© PercepTonic, LLC

9. Computer Vision: Images In, Information Out
LENS
INFORMATION
• Size and shape
• 3-­‐‑D position &
orientation
• Identity, object class
• Expression, gesture
• Location
• Motion
• Illumination,
weather, …
CAMERAS
Computer Vision algorithms analyze images
to extract information about the world.
This enables machines that can perceive.
© PercepTonic, LLC

10. How Does a Robot “See”?
Contextual understanding:
Who, What, Where?
Real-time operation with visual feedback (do
until …)
Learning and adaptation
TeraDeep
OMRON
© PercepTonic, LLC

11. Tools That Help Robots “See”
Vision algorithms customized for tracking, segmentation and
recognition
Power- and cost-optimized processors with expert
partitioning between edge and cloud loads
Sensors attuned to the particular needs of the computer
vision task Intel
OmniVision
© PercepTonic, LLC

12. Case Study: 3D Depth Perception
Example: Subaru’s EyeSight
© PercepTonic, LLC

13. How Stereo Vision Creates
3D Depth Perception
input output
+
left image
right image
disparity image
(inverse depth)
stereo
algorithm
• Works by triangulation: the closer an object is, the greater its
parallax between the left-right views.
• Stereo is a passive depth sensing method. Active methods
include structured light (Kinect) and Time-of-Flight sensors.
© PercepTonic, LLC

14. Application: Automotive Safety
Obstacles detected via stereo depth Stereo Depth/Disparity output
Source: Texas Instruments
for more information, see hap://goksel-­‐‑dedeoglu.com/3d-­‐‑stereo-­‐‑vision.html
© PercepTonic, LLC

15. Challenges in Embedded Vision
Computer Vision has the potential to fuel the next wave of
smart and affordable robots
but…
• The algorithms are pixel-intensive and computationally expensive
• The solutions are still evolving –new algorithms every year!
• The processors and sensors are really optimized for multimedia
© PercepTonic, LLC

16. Embedded Vision Architectures
Our best guess for power-efficient vision computation is
heterogeneous and multi-core architectures.
• Mobile SoCs: Qualcomm Snapdragon, NVIDIA Tegra K1
• Established DSPs and FPGAs, Movidius’ Myriad
• Vision IP: CogniVue, videantis, CEVA, adapteva
• Early examples of fixed-function hardware for vision
• Power-efficient server design starting to adopt the same typical block diagram
of modern SoCs
© PercepTonic, LLC

17. The Future of Computer Vision:
Embedded, Ever-ready, and Everywhere
From proof-­‐‑of-­‐‑concept prototypes to products
“PC World”
unlimited CPU, GPU, …
“Embedded World”
select processors & sensors
“Product World”
finalized hardware
© PercepTonic, LLC

18. We Have Come a Long Way…
Moravec, 1980
SRI’s Shakey, 1967
Consumer Electronics is now driving significant
investments in computer vision HW and SW,
helping the robotics vision wish-list come true.
© PercepTonic, LLC

19. References and Resources
Case studies and tutorials at the PercepTonic web site
http://www.PercepTonic.com/case-studies.html
The Embedded Vision Alliance (2011-present)
http://www.embedded-vision.com
http://embeddedvisionsummit.com
The IEEE Embedded Vision Workshops (2005-present)
http://cvisioncentral.com/evw2014/
© PercepTonic, LLC

20. Thank You!
slides will be available at
http://www.PercepTonic.com/slides/DesignWorld
© PercepTonic, LLC

21. The Future of Machine Touch®
Gerald E. Loeb, M.D., CEO

22. Touch is essential for dexterity and perception.
Simple tasks become impossible without it.

23. The Absence of Machine Touch
Precise
Expensive
Insensate
Rigid
Dangerous

24. Robots Doing Human Jobs
Need Biomimetic Design
Soft
Sensitive
Adaptive
• Works in unstructured environs
• Safe near humans
• Performs a variety of tasks
• Does not need retooling

25. BioTac® Biomimetic Tactile Sensors
Supple Fingertips
Multi-Modal Sensing
High Bandwidth
Durable
Field Repairable
Scalable

26. Live LabView GUI Demo

27. EPFL Lausanne
Allegro Hand
Kuka Arm
BioTacs
Dexterous
Autonomous
Robots

28. Contact Detection Reflex Enables Dexterous Use of a Myoelectric
Prosthetic Hand
The Today Show: Popular Mechanics Breakthrough Award

29. Blaine Matulevich,
Vikram Pandit,
Jeremy Fishel
Biomimetic
Inhibitory Reflex for
Fragile Grasp
0 1 2 3 4 5 6
10
8
6
4
2
0
Preamplified Net EMG Signal (V)
Motor Command (V)
Contact
No Contact

30. Performance
on cracker task Speed
Accuracy

31. Part Misalignment Failures
error
Contact
Forces

32. Part Misalignment Failures
Displacement
Ejection

33. Solution with Tactile Feedback
Contact Detection

34. NumaTac®
“airbags for robots”

35. Companies Want to Understand
Consumer’s Haptic Perception of Their Products
Focus Groups and Sensory Advisory
Panels are expensive, slow and prone to
“noise”.
Product releases have
$Billions at stake.
BioTac texture robot provides
objective data about human
dimensions of product “feel”.

36. Bayesian Action&Perception:
Representing the World in the Brain
G.E. Loeb and J.A. Fishel
Frontiers in Neuroscience
vol. 8 (October, 2014)

37. Tactile Sensing for Humans-in-the-Loop:
Telerobots for Hazardous Environments
• Tactile sensing is essential in low visibility environments.
• Fast reflex responses are more useful
than conscious perception.
• Government Applications for Dexterous Telerobots:
Ø DOD – ordnance
Ø DARPA – drones
Ø NASA – space
Ø NOAA – underwater
Ø NRC – nuclear reactors

38. “The Sense of
Touch”
Jusepe di
Ribera
ca. 1615
Norton Simon Gallery,
Pasadena, CA

39. The Standard in Machine Touch®
www.SynTouchLLC.com
Schunk Dexterous Hand
Allegro Hand
Willow Garage PR2
Shadow Hand
HUBO Robot
Robotiq Gripper
Barrea Hand
Kinova JACO Arm

40. Questions?
Paul Heney
Design World
pheney@wtwhmedia.com
Twitter: @DW_Editor
Gerald Loeb
SynTouch/Biomed Concepts
geloeb@gmail.com
Phone: 213.944.2283
Goksel Dedeoglu
PercepTonic
goksel@perceptonic.com
Phone: 214.356.4640

41. Thank You
q This webinar will be available at
designworldonline.com & email
q Tweet with hashtag #DWwebinar
q Connect with Design World
q Discuss this on EngineeringExchange.com