The document discusses research on robotics and software engineering for robotics. It describes challenges with robot heterogeneity, including different sensors, actuators, computing capabilities, drivers and protocols. It presents PhaROS as a middleware to address these challenges. It discusses using robots as guides and visual programming with Phratch. The document also covers more challenges like resource constraints, hardware/software integration, self-adaptation and safety; and proposes methodology for safe test-driven development. In conclusion, it mentions increasing investments in robotics by big companies and argues for using Smalltalk to build the robotics future.

1. Noury Bouraqadi, Luc Fabresse, Jannik Laval 1

2. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Pomodoro
2

3. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Pomodoro
3

4. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Pomodoro
3
pomodorotechnique.com

5. Small Talk with Robots
Noury Bouraqadi, Luc Fabresse, Jannik Laval
http://car.mines-douai.fr

6. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Research on Software Engineering for
5

7. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Research on Software Engineering for
6

8. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Research on Software Engineering for
6
Very long-term goal !

9. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Current robots
7

10. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Robot at our lab
8

11. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Infrastructure
9
Physical)Environment)
Robot)
Actuator)Actuator)
Computer)
Sensor)Sensor)
Sensor)
Actuator)Actuator)
Actuator)
So9ware)So9ware)So9ware)

12. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Infrastructure
10
Physical)Environment)
Robot)
Actuator)Actuator)
Computer)
Sensor)Sensor)
Sensor)
Actuator)Actuator)
Actuator)
So9ware)So9ware)

13. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Multiple kinds of robots
11

14. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Multiple kinds of robots
12

15. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Multiple kinds of robots
13

16. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Heterogeneity challenge
14

17. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Heterogeneity challenge
14
Different sensors
and actuators

18. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Different
computing
capabilities
Heterogeneity challenge
14
Different sensors
and actuators

19. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Different drivers
and protocols
Different
computing
capabilities
Heterogeneity challenge
14
Different sensors
and actuators

20. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Different drivers
and protocols
Different
computing
capabilities
Heterogeneity challenge
14
Different sensors
and actuators
Different low-level
algorithms

21. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Different drivers
and protocols
Different
computing
capabilities
Heterogeneity challenge
14
Different sensors
and actuators
Different shapes and
sizes
Different low-level
algorithms

22. Noury Bouraqadi, Luc Fabresse, Jannik Laval 15
implementations
exist

23. Noury Bouraqadi, Luc Fabresse, Jannik Laval 16
Physical)Environment)
Robot)
Actuator)Actuator)
Computer)
Sensor)Sensor)
Sensor)
Actuator)Actuator)
Actuator)
Algo)
Algo)
C,#C++#
Python#
)
)
)

24. Noury Bouraqadi, Luc Fabresse, Jannik Laval 17
Physical)Environment)
Robot)
Actuator)Actuator)
Computer)
Sensor)Sensor)
Sensor)
Actuator)Actuator)
Actuator)
Algo)
Algo)
C,)C++)
Python)
?" )
)
)

25. Noury Bouraqadi, Luc Fabresse, Jannik Laval 18
Physical)Environment)
Robot)
Actuator)Actuator)
Computer)
Sensor)Sensor)
Sensor)
Actuator)Actuator)
Actuator)
Middleware)
Algo)
Algo)
C,)C++)
Python)
)
)
)

26. Noury Bouraqadi, Luc Fabresse, Jannik Laval 19
Physical)Environment)
Robot)
Actuator)Actuator)
Computer)
Sensor)Sensor)
Sensor)
Actuator)Actuator)
Actuator)
Algo)
Algo)
C,)C++)
Python)
)
)
)

27. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Our work is PhaROS
20
Physical)Environment)
Robot)
Actuator)Actuator)
Computer)
Sensor)Sensor)
Sensor)
Actuator)Actuator)
Actuator)
Algo)
Algo)
C,)C++)
Python)
)
)
)

28. (small) talking to Robots

29. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Robot guide using
22
- Museum
- Hospital
- Shopping Mall
- …

30. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Robot guide using
23
https://vimeo.com/109125625

31. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Laser scans > Map
24

32. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Annotated map
25

33. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Robot guide using
26
Motor%Odometer% Motor%Odometer%Rangefinder%Rangefinder%Rangefinder%
Proprietary%So4ware%

34. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Robot guide using
27
Motor%Odometer% Motor%Odometer%Rangefinder%Rangefinder%Rangefinder%
Proprietary%So4ware%
Path%%
Planner%
Localiza/on%
System%

35. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Robot guide using
28
Motor%Odometer% Motor%Odometer%Rangefinder%Rangefinder%Rangefinder%
Proprietary%So4ware%
Path%%
Planner%
Localiza/on%
System%
App#
Bridge#

36. Noury Bouraqadi, Luc Fabresse, Jannik Laval 29
Talking(to(
with%

37. Noury Bouraqadi, Luc Fabresse, Jannik Laval 30
Talking(to(
with%

38. Noury Bouraqadi, Luc Fabresse, Jannik Laval 31
Talking(to(
with%

39. visual programing for robotics

40. Noury Bouraqadi, Luc Fabresse, Jannik Laval
You know phratch !
33

41. Noury Bouraqadi, Luc Fabresse, Jannik Laval
You know phratch !
34
on top of
Pharo
https://code.google.com/p/scat/

42. Noury Bouraqadi, Luc Fabresse, Jannik Laval
You know phratch !
34
on top of
Pharo
https://code.google.com/p/scat/

43. Noury Bouraqadi, Luc Fabresse, Jannik Laval
You know phratch !
34
on top of
Pharo
phratch.com
sponsored by
https://code.google.com/p/scat/

44. Noury Bouraqadi, Luc Fabresse, Jannik Laval
What is new ?
35
phratch.com
Jetstorm -
robotics
Jenkins
smart icons

45. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Add-ons
36

46. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Code accessibility
37

47. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Code accessibility
38

48. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Code accessibility
39

49. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Code accessibility
39

50. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Code accessibility
40

51. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Code accessibility
41

52. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Code accessibility
42

53. Noury Bouraqadi, Luc Fabresse, Jannik Laval
High extensibility !
43

54. Noury Bouraqadi, Luc Fabresse, Jannik Laval
High extensibility !
44

55. Noury Bouraqadi, Luc Fabresse, Jannik Laval
High extensibility !
44

56. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Used for teaching robotics!
45
https://vimeo.com/103126982

57. More challenges

58. Noury Bouraqadi, Luc Fabresse, Jannik Laval 47

59. Noury Bouraqadi, Luc Fabresse, Jannik Laval 48
Resource(
Constraints(
(RAM,(CPU,(
Ba5ery...)(
Changing'
Condi*ons'
Hardware'
+'So+ware'
Integra0on'

60. Noury Bouraqadi, Luc Fabresse, Jannik Laval 49
Resource(
Constraints(
(RAM,(CPU,(
Ba5ery...)(
Hardware'
+'So+ware'
Integra0on'
Self%
Adapta+on%
Changing'
Condi*ons'

61. Noury Bouraqadi, Luc Fabresse, Jannik Laval 50
Hardware'
+'So+ware'
Integra0on'
Self%
Adapta+on%
Changing'
Condi*ons'
Smalltalk'
+'FPGA'
Resource(
Constraints(
(RAM,(CPU,(
Ba5ery...)(

62. Noury Bouraqadi, Luc Fabresse, Jannik Laval 51
•Build a Pharo robotic
application
•Identify critical parts
•Project the critical parts
on FPGA
•Evaluation of performance
gain/loss
EXPERIMENT

63. Noury Bouraqadi, Luc Fabresse, Jannik Laval 52
http://www.youtube.com/watch?v=e1Hzt4soQIc

64. Noury Bouraqadi, Luc Fabresse, Jannik Laval 53
Camera RGB →HSV
HSV filtrer
Laser
sensor
Object
detector
Motors

65. Noury Bouraqadi, Luc Fabresse, Jannik Laval 54
Camera RGB →HSV
HSV filtrer
Laser
sensor
Object
detector
Critical part !
Motors

66. Noury Bouraqadi, Luc Fabresse, Jannik Laval 55
Smalltalk application
HW/SW Partitioning
SW HW
PharoVM FPGA
Interface

67. Noury Bouraqadi, Luc Fabresse, Jannik Laval 56
Self%
Adapta+on%
Changing'
Condi*ons'
Smalltalk'
+'FPGA'
Resource(
Constraints(
(RAM,(CPU,(
Ba5ery...)( TDD#
Hardware'
+'So+ware'
Integra0on'

68. Noury Bouraqadi, Luc Fabresse, Jannik Laval 57
Physical)Environment)
Robot%
Actuator%Actuator%
Computer%
Sensor%Sensor%
Sensor%
Actuator%Actuator%
Actuator%
So2ware%So2ware%So2ware%

69. Noury Bouraqadi, Luc Fabresse, Jannik Laval 58
Physical)Environment)
Robot%
Actuator%Actuator%
Computer%
Sensor%Sensor%
Sensor%
Actuator%Actuator%
Actuator%
So2ware%So2ware%So2ware% Safety)

70. Noury Bouraqadi, Luc Fabresse, Jannik Laval 59
Simula'on*
Safety*

71. Noury Bouraqadi, Luc Fabresse, Jannik Laval 60
http://www.youtube.com/watch?v=oNrvuaASwrU

72. Noury Bouraqadi, Luc Fabresse, Jannik Laval 61
37#
Simula*on#
Noise#
Resources#
Failure#
Asynchrony#
Precision#

73. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Methodology for safe TDD
62
Ensure emergency stop working!

74. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Methodology for safe TDD
63
Ensure emergency stop working!
Order tests
- Sensors before actuators
- Controlled environments before open ones
- Static environments before dynamic ones

75. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Methodology for safe TDD
64
Ensure emergency stop working!
Order tests
- Sensors before actuators
- Controlled environments before open ones
- Static environments before dynamic ones
Run test only if prerequisites pass

76. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Methodology for safe TDD
65
Ensure emergency stop working!
Order tests
- Sensors before actuators
- Controlled environments before open ones
- Static environments before dynamic ones
Run test only if prerequisites pass
BoTest'
Framework'

77. Noury Bouraqadi, Luc Fabresse, Jannik Laval 66
the$
Future
is
Robo-cs

78. Noury Bouraqadi, Luc Fabresse, Jannik Laval 67
10#000#000+#

79. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Big players invest in robotics
68
Amazon drones for Parcel Delivery

80. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Big players invest in robotics
69
Google self-driving car

81. Noury Bouraqadi, Luc Fabresse, Jannik Laval
Big players invest in robotics
70
Google self-driving car
Google&Bought&8&
Robo.cs&Companies&in&
2013!&

82. Noury Bouraqadi, Luc Fabresse, Jannik Laval 71
the$
Future
is
Robo-cs

83. Noury Bouraqadi, Luc Fabresse, Jannik Laval 72
the$
Future
is
Robo-cs
the$
Future
is
Robo-cs
Let’s$$make$it$with$
Smalltalk

84. Noury Bouraqadi, Luc Fabresse, Jannik Laval 72
the$
Future
is
Robo-cs
the$
Future
is
Robo-cs
Let’s$$make$it$with$
Smalltalk

85. Douai loves Argentina !

86. Noury Bouraqadi, Luc Fabresse, Jannik Laval 74

87. Noury Bouraqadi, Luc Fabresse, Jannik Laval 74

88. Noury Bouraqadi, Luc Fabresse, Jannik Laval 74

89. Noury Bouraqadi, Luc Fabresse, Jannik Laval 74
Douai

90. Small Talk with Robots
Noury Bouraqadi, Luc Fabresse, Jannik Laval
http://car.mines-douai.fr