Public lecture at Microsoft Research about the convergence of robots and artificial intelligence. This talk presented the development and how to approach generic software architectures for robots.

1. Presented by:
Alexander Braendle
Andreas Heil
Robots in Human Environments
Date
12 Dec 2005

2. Introduction
"Men Are From Mars, Robots Are From Mitsubishi“
Financial Times (12/09/05) P. 9; Pincock, Stephen
As Carnegie Mellon roboticist Daniel Wilson outlines in his book, "How to Survive a Robot
Uprising," the field of robotics has taken off in recent years, with researchers around the world
developing robotic applications to do everything from vacuuming to exploring space alongside
man. Several Japanese companies are developing robots that can serve as in-home
assistants, link up to the Internet to respond to questions, and serve as a kind of companion.
Toyota recently released its Partner Robot with the ability to play the trumpet through lips
endowed with sensitivity and fingers with human-like dexterity. The convergence of robotics
and artificial intelligence has enabled researchers to develop devices such as Sony's Qrio, a
small robot that knows to hold out its arms if it is falling, and can pick itself up from the
ground. Embodied intelligence seeks to equip robots with such cognitive abilities, though
ingraining many basic components of intelligence still eludes roboticists. Robots still cannot
understand what gives an object its properties, though roboticists are working to give their
creations the ability to learn by experience. The international group of researchers working
together in the RobotCub project is trying to create a child-sized robot that can learn from
interactions with its environment, just as people do. Many researchers feel that people's
perceptions of the role of robots must change if they will ever be accepted as legitimate
companions. Mitsubishi has begun taking orders for its Wakamaru robot, an in-home personal
assistant that wakes you up in the morning, reports the weather and the headlines, and then
greets you in the evening with any telephone messages. "We have tried to create a robot you
can have a relationship with," said Mitsubishi's Ken Onishi.

3. What do we think about?
• Increasing the intelligence of environments
• Thinking future daily objects, appliances, architecture.
• Personal(ized) Devices
• R-H, H-R, H-R-H, R-E Interaction (Relationship)
• Software & Tools

4. Robots in human environments
Future Applications
• Support & Care, Education, Entertainment
Personal Robotics
• Human-robotics Interaction
• Mediator between digital/real world
• Natural and affective interaction (speech, gestures,
emotions)
From nature to
software models
New programming paradigms
• Body inspired software architecture (Pisa)
• Meta-programming models (Berlin)
What is needed that robots could be successfully
integrated in our everyday life? What are key
technical issues?

5. Robots!
Already many commercial robots
available
• Aibo
• Asimo
• Hoap
• fischertechnik
• Lego
• …
How to program them?
• Different tools
• Different programing paradigm
• Different programing languages

6. Robots, too?
Software
• MS Agent
• Internet Explorer
• Media Player
Additional Hardware
• Smartphone

7. Goal
One language for all robots?
One tool to rule them all?

8. VRDK
Building a compelling & engaging
programmable environment to play &
learn for children
Innovative, Very Easy, flexible
programming environment for Robot
control applications
That is accessible to non technical
market (children, nurse, elderly, machine
operator)
That enables a ‘path’ from very simple
(beginner, child) to professional (using
visual studio)
Abstracting complexity on different levels
Source Code Visual
Andreas

9. First Attempt
Build a framework (middleware)
Abstract from the underlying hardware
• i.e. classes for sensors, actuators,
and communication
Unique programming model C o n t r o l a p p l ic a t io n
• Can use every concept the
runtime provides R o b o t AP I
Use the .NET CLR and API
C LR
Sen s o rs an d actuato rs c o n tro ller

10. This wont always work!
Things that work fine on a PC don’t scale down
• Thread switches on a smart-phone or PDA are costly
• FischerTechnik controller doesn’t support threads at
all
Hardware not capable enough for a middleware
• CPU, memory, battery, …
Compact Framework: no serialization, no remoting, …
• Even though PDAs are fairly powerful, CF has some
severe restrictions
Now think of even smaller devices
• Embedded controllers, wrist-watches, sensor
networks, …

11. Bridging the Gap
Model-Driven Architecture
Platform Independent Model
• High-level model of the application
• Describes desired behaviour P IM
Platform Specific Model
• Implementation of an application for model transformation
a specific target platform (i.e. fischertechnik)
PS M
Executable
• The artifact that is eventually executed generation
on the robots controller
exec utable

12. Model-driven Development
P l a t f o r m -in d e p e n d e n t m o d e l o f t h e p r o g r a m Same programming model
(V R D K L a n g u a g e ) for different devices
transformation transformation
Platform-specific
LEGO RC X F is c h e r T e c h n ik Smartph o n e realisation of the programm
generation generation
Executable or interpreter
exec utable exec utable exec utable
on target platform

13. VRDK & MDA
Executable !!
Platform Independent
Model
Model
Transformation
Platform
Proc1 Proc2 Proc1 Proc2 Handler
Specific
Models
•CF.NET •.NET •RoboCode Simulator
•static scheduling •runtime •multi threaded
•single threaded scheduling •heavy usage of
•multi threaded event handlers
C# & VS.NET Project Other

14. VRDK Architecture • Rapid Prototyping
• Program can be
launched directly Application logic
from the editor is executed on
the PC.
VRDK runs
VRDK
Editor Interpreter
C# Code Generation
uses
runs
VRDK Robots.NET
uses
Code Generation API
Application logic can
be executed on:
• PC
• PDA
• Robot Hardware

15. Full Tool Chain
Code generation
VRDK Editor VS.NET
Deploy
and run
Deploy
and run
the gentle shortcut
target devices

16. Demo

17. Distribution and Communication
VRDK has parallelism and communication
• Have multiple robots/devices to interact
• First class concept
Use communication of underlying platform
• i.e. .NET Sockets, LEGO Infrared, Indigo, …
PC
future

18. VRDK API Structure
(F is c h e r T e c h n ik )
Co n t r o l l e r
Ac t u a t o r s Sen s o rs
An a l o g S e n s o r D ig it a l S e n s o r
(e .g . M o t o r s )
(e .g . H e a t S e n s o r )
(e.g. Touch Sensor)

19. Robotics4.Net

20. Interacting with Humans
Considering affects and
emotions as base
components for designing
interaction Sensing Human
Social, verbal,
Human The Emotive Response
affective
Human User
Improving interaction in a Social
Environment Communication
novel social environment
(humans + Computers + Emotional Context-
Aware application on
Robots) Mobile devices Recognizing humans
and provide behavioural
Response Pattern
How could robots facilitate Interaction with
the access to technology and Computers, Robots,
Humans
provide supportive services
Computing Computing
Applications Support
And Knowledge
Understanding and
Modelling Affects in
Social contexts

21. Questions