Bio-inspired robotic touch

1. Artiﬁcial Touch L. Ascari Artiﬁcial Touch Introduction Towards a new approach in prosthetics? The tactile system Modelling L. Ascari Validation Conclusions and Future HENESIS S.R.L. Options References Parma - February 22nd, 2012 — All the activity described in the presentation has been carried on while post-doc at Scuola Superiore Sant’Anna, Pisa (I)

2. Outline Artiﬁcial Touch 1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions 2 The tactile system and Future Options Hardware References Software 3 Modelling 4 Validation 5 Conclusions and Future Options

7. Outline Artiﬁcial Touch 1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Touch in Robotics Approach Prosthetics SoA Approach The pick and lift task The pick and lift task Bioinspiration Bioinspiration The tactile system 2 The tactile system Modelling Hardware Validation Software Conclusions and Future 3 Modelling Options References 4 Validation 5 Conclusions and Future Options

8. Touch in Robotics I Artiﬁcial Touch L. Ascari Introduction Robots are now very complex and sophisticated systems. Touch in Robotics Higher computational requirements. Prosthetics SoA Approach The pick and lift task Automation robots: very high performing and reliable Bioinspiration machines. The tactile system Outside the factory ﬂoor: limited interaction with humans, Modelling specially in terms of autonomous behavior and of friendly Validation HMIs1 , Conclusions and Future despite a huge market is expected to develop rapidly2 . Options References

9. Touch in Robotics II Artiﬁcial Touch L. Ascari Tactile sensing can provide information about mechanical properties such as compliance, friction, and mass. Introduction Touch in Robotics Knowledge of these parameters is essential if robots are to Prosthetics SoA Approach reliably handle unknown objects in unstructured The pick and lift task Bioinspiration environments. For interaction, localization of the stimulus The tactile is essential3 . system Modelling Validation Conclusions 1 and Future J. Ayers et al. Neurotechnology for biomimetic robots. MIT Press, Options 2002. References 2 WorldRobotics. World Robotics 2006. International Federation of Robotics, Statistical Department, 2006. url: http://www.worldrobotics-online.org/. 3 R. D. Howe. “Tactile sensing and control of robotic manipulation”. In: Journal of Advanced Robotics 8 (1994), pp. 245–261.

10. For what? Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach Interaction The pick and lift task Bioinspiration The tactile system Modelling Validation Autonomy Conclusions and Future Options References Locomotion

14. SOA in robotic skins? Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References

15. Open Issues Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA wiring Approach The pick and lift task robustness Bioinspiration The tactile stretchability system Modelling bandwidth Validation processing Conclusions and Future Options References

20. Touch in Prosthetics - Commercial SoA Artiﬁcial Touch More advanced: myoelectric control L. Ascari I-Limb Ultra from Touch Bionics Introduction Ultra from BeBionics Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References

21. Touch in Prosthetics - Commercial SoA Artiﬁcial Touch More advanced: myoelectric control L. Ascari I-Limb Ultra from Touch Bionics Introduction Ultra from BeBionics Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References Often refused by patients!

22. Touch in Prosthetics - Commercial SoA Artiﬁcial Touch Classical prosthesis, cable actuated L. Ascari Otto bock grippers Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References

23. Touch in Prosthetics - Commercial SoA Artiﬁcial Touch Classical prosthesis, cable actuated L. Ascari Otto bock grippers Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References Not sensorized. Higher user acceptance. Why?

24. Contemporary prosthetics: directions and open issues4 Artiﬁcial Directions Touch autonomous control of low level tasks L. Ascari higher spatial resolution of the sensing system Introduction Touch in Robotics neural control (prototypes exist) Prosthetics SoA Approach feedback to the patient (preliminary results) The pick and lift task Bioinspiration The tactile system Open issues Modelling connection with tactile nerves Validation dexterity Conclusions and Future sensitivity Options References CONTROL (myo-electrical vs neural) feedback to the patient 4 R.G.E. Clement et al. “Bionic prosthetic hands: A review of present technology and future aspirations”. In: The Surgeon 9.6 (12/2011),

25. Contemporary prosthetics: directions and open issues4 Artiﬁcial Directions Touch autonomous control of low level tasks L. Ascari higher spatial resolution of the sensing system Introduction Touch in Robotics neural control (prototypes exist) Prosthetics SoA Approach feedback to the patient (preliminary results) The pick and lift task Bioinspiration The tactile system Open issues Modelling connection with tactile nerves Validation dexterity Conclusions and Future sensitivity Options References CONTROL (myo-electrical vs neural) feedback to the patient 4 R.G.E. Clement et al. “Bionic prosthetic hands: A review of present technology and future aspirations”. In: The Surgeon 9.6 (12/2011),

26. Basic questions Artiﬁcial Touch Some fundamental questions L. Ascari What is the main issue with advanced prosthesis? Introduction Touch in Robotics Prosthetics SoA Is feedback to the user essential for this? Approach The pick and lift task Bioinspiration “Solved” Issues The tactile system low level control with many signals (here) Modelling parallel but portable processing (here) Validation Conclusions mechanics (single ﬁngers, underactuation, . . . ) and Future Options References

27. Basic questions Artiﬁcial Touch Some fundamental questions L. Ascari What is the main issue with advanced prosthesis? Object Introduction Slippage and Grasp force control Touch in Robotics Prosthetics SoA Is feedback to the user essential for this? Approach The pick and lift task Bioinspiration “Solved” Issues The tactile system low level control with many signals (here) Modelling parallel but portable processing (here) Validation Conclusions mechanics (single ﬁngers, underactuation, . . . ) and Future Options References

28. Basic questions Artiﬁcial Touch Some fundamental questions L. Ascari What is the main issue with advanced prosthesis? Object Introduction Slippage and Grasp force control Touch in Robotics Prosthetics SoA Is feedback to the user essential for this? No! Approach The pick and lift task Bioinspiration “Solved” Issues The tactile system low level control with many signals (here) Modelling parallel but portable processing (here) Validation Conclusions mechanics (single ﬁngers, underactuation, . . . ) and Future Options References

29. Basic questions Artiﬁcial Touch Some fundamental questions L. Ascari What is the main issue with advanced prosthesis? Object Introduction Slippage and Grasp force control Touch in Robotics Prosthetics SoA Is feedback to the user essential for this? No! Approach The pick and lift task Bioinspiration “Solved” Issues The tactile system low level control with many signals (here) Modelling parallel but portable processing (here) Validation Conclusions mechanics (single ﬁngers, underactuation, . . . ) and Future Options References

30. Bio-inspired approach Artiﬁcial Touch Why and to what extent? L. Ascari Ultimate model: man Man Larger dimensions, Introduction Touch in Robotics Infinite Complexity: higher densities Prosthetics SoA sensors and processing Approach The pick and lift task Technological, Bioinspiration wiring, processing The tactile limitations system Modelling Model and Simplification Principle Validation Lower complexity validation Conclusions Innovative approach sensory systems and Future Options •Technology References •Processing •Scalability Star-nosed mole

31. Bio-inspired approach Artiﬁcial Touch Why and to what extent? L. Ascari Ultimate model: man Man Larger dimensions, Introduction Touch in Robotics Infinite Complexity: higher densities Prosthetics SoA sensors and processing Approach The pick and lift task Technological, Bioinspiration wiring, processing The tactile limitations system Touch sense Modelling Model and Simplification Principle Validation Lower complexity validation Conclusions Innovative approach and Future sensory systems Options •Technology References •Processing •Scalability Star-nosed mole

32. The human hand: tactile structure Artificial Touch L. Ascari Human hand touch Structure of the skin Introduction 3 major groups of afferent Touch in Robotics (tactile afferents, joint Prosthetics SoA Approach mechanoreceptors, spindles) The pick and lift task Bioinspiration The glabrous skin has 17.000 The tactile system tactile units Modelling 4 main types of Validation mechanoreceptors (Ruffini, Conclusions and Future Pacini, Merkel, Meissner) for Options intensity, pressure, acceleration References stimuli

33. The human hand: tactile structure Artificial Touch L. Ascari Human hand touch Structure of the skin Introduction 3 major groups of afferent Touch in Robotics (tactile afferents, joint Prosthetics SoA Approach mechanoreceptors, spindles) The pick and lift task Bioinspiration The glabrous skin has 17.000 The tactile system tactile units Modelling 4 main types of Validation mechanoreceptors (Ruffini, Conclusions and Future Pacini, Merkel, Meissner) for Options intensity, pressure, acceleration References stimuli from Johansson and Westling (“Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects”)

34. Sensors performance... Artiﬁcial Touch ... in engineering terms L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References

35. The pick and lift task Artiﬁcial Two aspects are crucial for a stable grasp: Touch L. Ascari the ability of the HW/SW system to avoid object slip Introduction to control in real-time the grasping force. Touch in Robotics Prosthetics SoA Approach Human physiology of the task The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References

36. The pick and lift task Artiﬁcial Two aspects are crucial for a stable grasp: Touch L. Ascari the ability of the HW/SW system to avoid object slip Introduction to control in real-time the grasping force. Touch in Robotics Prosthetics SoA Approach Human physiology of the task The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References

37. On the need for feedback Artiﬁcial Touch L. Ascari Evidence Where? Introduction Johansson measured Touch in Robotics 50-60ms of reaction Prosthetics SoA Approach time The pick and lift task Bioinspiration incompatible with The tactile system propagation time to the Modelling motor cortex Validation evidence of circuit Conclusions and Future closed at subcortical Options level (olivo-cerebellar References system and thalamus). from Johansson and Westling (“Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects”)

40. Biological vs Robotic worlds Artiﬁcial Touch Do we have these limitations (signaling speed) in robots? L. Ascari Man Introduction Biological models for the Touch in Robotics design of biomimetic robots Prosthetics SoA Approach The pick and lift task Nerves Brain Limbs Bioinspiration The tactile system Interfacing Bio and Modelling Robotics Validation Robot Conclusions and Future Options • Robots as physical platforms for validating biological models References Artificial Electric Artificial Brain wires limbs 3

41. Biological vs Robotic worlds Artiﬁcial Touch Do we have these limitations (signaling speed) in robots? L. Ascari Man Introduction Biological models for the Touch in Robotics design of biomimetic robots Prosthetics SoA Approach The pick and lift task Nerves Brain Limbs Bioinspiration The tactile system Interfacing Bio and Modelling Robotics Validation Robot Conclusions and Future Options • Robots as physical platforms for validating biological models References Artificial Electric Artificial Brain wires limbs 3 No, but other constraints exist. Ex: computational power

42. Biological vs Robotic worlds Artiﬁcial Touch Do we have these limitations (signaling speed) in robots? L. Ascari Ultimate model: man Introduction Man Larger dimensions, Touch in Robotics Infinite Complexity: higher densities Prosthetics SoA sensors and processing Approach The pick and lift task Technological, Bioinspiration wiring, processing The tactile limitations Touch sense system Model and Modelling Simplification Principle Lower complexity validation Validation Innovative approach sensory systems Conclusions •Technology and Future Options •Processing References •Scalability Star-nosed mole No, but other constraints exist. Ex: computational power

43. Biological vs Robotic worlds Artiﬁcial Touch Do we have these limitations (signaling speed) in robots? L. Ascari Man Ultimate model: man Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Touch sense Modelling Validation Lower complexity Innovative approach sensory systems Conclusions and Future •Technological Options •Processing References •Scalability Star-nosed mole No, but other constraints exist. Ex: computational power

44. Multidisciplinarity — The animal model (touch) Artiﬁcial Touch L. Ascari Condylura Cristata A nose to see / Eimer Introduction 12 mobile appendages Touch in Robotics covered with more than Prosthetics SoA Approach 25.000 tactile receptors The pick and lift task Bioinspiration (Eimer organs) The tactile system Structure of the Eimer Modelling organ: a sort of pillar with Validation 3 nervous terminations Conclusions (for constant pressures, and Future Options vibrations, ﬁne surface References details); foveated tactile vision. from Catania and Kaas (“Somatosensory Fovea in the Star-Nosed Mole: Behavioral Use of the Star in Relation to Innervation Patterns

47. Multidisciplinarity — The animal model (vision) Artiﬁcial Touch L. Ascari Honeybee Fixed yet good eye Introduction Non-mobile compound Touch in Robotics eyes (ommatidia); Prosthetics SoA Approach The pick and lift task 3000-4000 facets each eye Bioinspiration ( = 64x64 pixel array); The tactile system spatial resolution = 1/60 Modelling of the human eye; Validation No distance information Conclusions and Future from stereo vision; Options References Center facets larger than the peripheral sensors. yet: high performance

52. Multidisciplinarity — The animal model (vision) Artificial Touch L. Ascari Honeybee Fixed yet good eye Introduction Non-mobile compound Touch in Robotics eyes (ommatidia); Prosthetics SoA Approach The pick and lift task 3000-4000 facets each eye Bioinspiration ( = 64x64 pixel array); The tactile system spatial resolution = 1/60 Modelling of the human eye; Validation No distance information Conclusions and Future from stereo vision; Options References Center facets larger than the peripheral sensors. optical flow balance yet: high performance motion detection (Flicker effect)

53. Multidisciplinarity — The computational model Artiﬁcial Touch L. Ascari Cellular non linear networks Parallel topological Introduction CNN is a massive parallel architecture Touch in Robotics computing paradigm deﬁned Prosthetics SoA Approach in discrete N-dimensional The pick and lift task Bioinspiration spaces. The tactile system A CNN is an N-dimensional Modelling regular array of elements Validation (cells); Conclusions and Future Cells are multiple input-single Options output analog processors, all References described by one or just some few parametric functionals. from Chua and Roska (Cellular Neural Networks and Visual Computing: Foundations and Applications)

54. Multidisciplinarity — The computational model Artiﬁcial Touch L. Ascari Cellular non linear networks Parallel topological Introduction CNN is a massive parallel architecture Touch in Robotics computing paradigm deﬁned Prosthetics SoA Approach in discrete N-dimensional The pick and lift task Bioinspiration spaces. The tactile system A CNN is an N-dimensional Modelling regular array of elements Validation (cells); Conclusions and Future Cells are multiple input-single Options output analog processors, all References described by one or just some few parametric functionals. from Chua and Roska (Cellular Neural Networks and Visual Computing: Foundations and Applications)

55. CNN characteristics I Artiﬁcial Touch L. Ascari Locality of the connections between the units: in fact the Introduction Touch in Robotics main diﬀerence between CNN and other Neural Networks Prosthetics SoA Approach paradigms is the fact that information are directly The pick and lift task exchanged just between neighbouring units. Of course this Bioinspiration The tactile characteristic allows also to obtain global parallel system processing. Modelling Validation A cell is characterized by an internal state variable, Conclusions sometimes not directly observable from outside the cell and Future Options itself; References More than one connection network can be present;

56. CNN characteristics II Artificial Touch L. Ascari A CNN dynamical system can operate both in continuous Introduction (CT-CNN) or discrete time (DT-CNN), with analogical Touch in Robotics Prosthetics SoA signals from different sources; Approach The pick and lift task CNN data and parameters are typically real values; Bioinspiration The tactile CNN operate typically with more than one iteration, i.e. system they are recurrent networks; It is a Universal Machine Modelling (CNN-UM); Validation Conclusions It offers Stored programmability; and Future Options a Hardware implementation exists. References

57. CNN core: the template Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References

58. Template meaning Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation State-out Conclusions and Future Options References in

59. Features of the ACE4K (16K) chip — 3TOps Artiﬁcial Touch System Desktop PC, PC-104 industrial PC, Windows NT, 2000 L. Ascari Bus PCI, 33 MHz, 32 bit data width; Visual Microprocessor type ACE4k, 64x64 processor array Introduction Grayscale image download (64x64) 2688 frame/sec 372 !s Touch in Robotics Prosthetics SoA Grayscale image readback (64x64) 3536 frame/sec (compensated through look-up table); 283!s Approach Binary image download (64x64) 44014 frame/sec; 22.72 !s The pick and lift task Bioinspiration Binary image readback (64x64) 23937frame/sec; 41.78 !s Array operation (64x64) 9 !s + N*100ns The tactile Logical operation (64x64) 3.8 !s system DSP type Texas TMS320C6202; 250MHz, 1600 MIPS operation Modelling Memory 16MB, SDRAM 125 MHz; 2Mbyte FLASH (bootable) Validation Serial Ports 3 Other features Watch Dog, Timer Conclusions and Future Options Programmability C language, native languages References Image processing library Several image processing functions optimized for CVM Application Program Interface (API) Integrate the Aladdin systerm into different environments

60. Recall Artiﬁcial Touch FINAL GOAL COMPUTATIONAL PLATFORM L. Ascari Introduction ROBOTIC Touch in Robotics PLATFORM Prosthetics SoA Approach The pick and lift task Tactile Bioinspiration system SW The tactile system Modelling TASK CONTROLLER Validation Tactile system HW Conclusions and Future Options References TASK, PHYSIOLOGICAL STRATEGY

61. Outline Artiﬁcial Touch 1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Hardware Software Bioinspiration Modelling Validation 2 The tactile system Conclusions Hardware and Future Options Software References 3 Modelling 4 Validation 5 Conclusions and Future Options

62. The MEMS mechanoreceptor Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Rpu Vc Modelling R1 R2 Validation V13 V24 Conclusions R3 R4 and Future Options References 0

63. The array — Fabrication steps Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References

64. The whole system — HW Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References From L Ascari et al. “A miniaturized and ﬂexible optoelectronic sensing system for tactile skin”. In: Journal of Micromechanics and Microengineering 17.11 (11/2007), pp. 2288–2298. issn: 0960-1317. doi: 10.1088/0960-1317/17/11/016. url: http://ejournals.ebsco.com/direct.asp?ArticleID= 4A9A98E0B7D16F0C429C

65. The whole system — from HW to SW Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References From L. Ascari et al. “Bio-inspired grasp control in a robotic hand with massive sensorial input”. In: Biological Cybernetics 100.2 (2009), p. 109. doi: 10.1007/s00422-008-0279-0

66. Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP References

71. Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP References Where is information? What kind of spatial and temporal patterns? How to recognize and prevent slippage?

72. Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP References Where is information? What kind of spatial and temporal patterns? How to recognize and prevent slippage? We need to learn the tactile “alphabet”

73. The task controller — FSM Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References

74. The task controller — FSM Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References

75. The task controller — Features Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References

77. The slip effect in robotic grasp Artificial Touch Slip as vibrations. “Catch and snap” effect on the rubber L. Ascari (60Hz stable + initial 10Hz component). Recall FAII human mechanoreceptors. Introduction The tactile system Modelling Validation Conclusions and Future Options References Holweg et al., “Slip detection by tactile sensors: algorithms and experimental results”

78. Deﬁnition of Tactile Events of interest Artiﬁcial Touch L. Ascari Variations, oscillations, vibrations Introduction Time is divided in periods of The tactile duration T ∗ s system Modelling Variation change in signal Validation larger than σ in Conclusions same period and Future Options Oscillation seq. of 2 subsequent References variations of opposite sign in same T ∗ . (m,n) Vibration seq. of 2 oscillations in 2 adjacent periods σ = 2% dynamic range

Bio-inspired robotic touch

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (14)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Bio-inspired robotic touch