Programming robots using visualisation, concurrency and coordination, using the Microsoft Robotics Developer Studio and the Visual Robotic Development Kit prototype held for the British Computer Society (BCS) in Cambridge, UK in 2006.

1. ERO Technology Group
Presented by
Programming Robots
Andreas Heil
v-aheil@microsoft.com
Date
Programming with Visualisation, 08. November 2006
Concurrency and Coordination

3. Definition: Robot
A robot is an electro-mechanical device that can perform autonomous or
preprogrammed tasks. A robot may act under the direct control of a human (eg. the
Canadarm on the space shuttle) or autonomously under the control of a
programmed computer.
Wikipedia
1. Mechanical device programmed to perform tasks
any machine that can be programmed to carry out instructions and
perform particular duties, especially one that can take over tasks
normally done by people.
2. Imaginary machine like human
a machine that resembles a human in appearance and can function like a
human, especially in science fiction
3. Person like a machine
somebody who works or behaves mechanically, showing little or no
emotion and often responding to orders without question
4. Traffic light
South Africa a set of automatic traffic light
Encarta Dictionary

4. Definition: Robot
A robot is a device, hard- or
software with the capability
of sensing and (re)acting.

5. Robotics Market Potential
Service and consumer markets just emerging
• Remote assistance/presence
• Assistive Worldw ide Robotics Market Grow th
• Facilities maintenance
• Security
• Education
• Entertainment 70000000 Home
Mediacal/Welfare
More applications are required to improve the 60000000
Public Sector
market 50000000 Bio-Industrial
Market Size $1,000s
Manufacturing
40000000
• A lack of reusability keeps robot
developers endlessly re-solving the same 30000000
software problems 20000000
10000000
• Concentrating on the mechanics of 0
robotics rather than the science of robotics 1995 2000 2005 2010 2025
Year
• Concurrency and complexity plague all
software engineering and especially
robotics * Source Japan Robotics Association
• Development requires too much effort and
knowledge

6. Robotics Market Potential
Worldw ide Robotics Market Grow th
70000000 Home
Mediacal/Welfare
60000000
Public Sector
50000000 Bio-Industrial
Market Size $1,000s
Manufacturing
40000000
30000000
20000000
10000000
0
1995 2000 2005 2010 2025
Year
* Source Japan Robotics Association

7. 3
2
1
20
05
Real vacuum cleaner
Filtering unnecessary Information
Household Devices (Microwave, Fridge)
20
Robot with integrated Communication Capabilities1
07
Robot loading Dishwasher2
20
10
Tourguide3
Development Trend
Robot adapting to individual Needs
Gaming Robots
Learning about new Environments by giving a Tour to it, after that it can re-give that tour.
20
Robot helping handicapped People
15
Increasing Software Challenges
Ironing Robot
Proactive Robots
Robots understanding human Activities
20
20
24h Assistant
Companion for elderly People
* Source EURON Workshop 2005

8. Microsoft & Robotics
Experience Groups
Systems Microsoft Research
• PC Ecosystem (desktop, web, • Enabling Technologies
mobile, home) • Human Robot Interaction
• XP, XP Embedded, CE (real-time) • Personal Robotics
• Educational Robotics
Development environment and tools • Programming Environments
• Programming IDE Microsoft Robotics Groups
• Debugging and Opmization
• Microsoft Robotics Group
Existing applicable Technologies Redmond
• Center for Innovative Robotics
• Speech SDK Carnegie Mellon University
• Real-time Communications SDK • External Research Office
• RFID SDK Cambridge

9. New Requirements
New Requirements
• Ultra Heterogeneity
• Distributed environments
• Dynamic Configuration
• Context-Awareness
• Personalization
• Extensibility
• Reliability
• Security
• Privacy protection
• Usability
• Autonomy
• …
Ubiquitous computing environments should be deployed incrementally
• Living space is not a demonstration room
• We like to replace existing objects
How can software infrastructures help to decrease the complexities?

10. Challenges to Address
Future Applications
• Support & Care, Education, Entertainment
Personal Robotics
• Human-robotics Interaction
• Mediator between digital/real world
• Natural and affective interaction (speech,
gestures, emotions) From nature to
software models
New programming paradigms
• Body inspired software architecture (Pisa)
• Meta-programming models (Berlin, Stuttgart)
What is needed that robots could be
successfully integrated in our everyday life?
What are key technical issues?

11. A typical first Approach
• Build a (another) middleware
• Abstract from the underlying hardware (HAL)
> i.e. classes for sensors, actuators, and communication
• Unique programming model Control Application
> Can use every concept the
runtime provides
Robot API
• Use the .NET CLR and API
CLR
Sensors and Actuators Controller

12. This wont always work!
Things that work fine on a PC don’t scale down
• Thread switches on a smart-phone or PDA are costly
• Some hardware controller don’t support threads at all
• Hardware not capable enough for a middleware
(CPU, memory, battery, …)
• Even though PDAs/Smartphones are fairly powerful, CF has some
several restrictions (good for some reasons, but not for us)
Now think of even smaller devices
• Embedded controllers, wrist-watches, sensor networks, smart dust

13. VRDK – A Research Prototype
• Building a compelling & engaging
programmable environment
• Innovative, very easy, flexible
programming environment for Robot
control applications
• Accessible to non technical audience
(children, nurse, elderly, machine
operator)
• Enables a ‘Path’ from very simple
(beginner, child) to professional (using
Visual Studio)
Abstracting complexity on different levels
Source Code Visual

14. Model-driven Development Approach
Platform-independent Model of the Program Same programming model
(VRDK Language) for different devices
Transformation Transformation
Platform-specific
.NET .NET CF XYZ realization of the program
Generation Generation
Executable or interpreter
Executable Executable Executable
on target platform

15. VRDK API Structure
Controller
(e.g., fischertechnik)
Actuators Sensors
Analog Sensor Digital Sensor
(e.g., Motors)
(e.g., Heat Sensor)
(e.g., Touch Sensor)

16. Full Tool Chain
Code Generation
VRDK Editor VS.NET
Deploy
and Run
Deploy
and Run
The gentle Shortcut
Target Devices

17. Demo

18. Domain Specific Language
Visual N#
Users can switch between
• Domain-specific both notations
graphical language
• Extension of VRDK
• Easy to design
Code Translator
N#
• Domain-specific
textual language
• Easy to read and write
C# / VB.NET
• General purpose
programming language
• Don’t write code
anymore

19. N# - A Textual Notation for Visual N#
ambient MyAmbient @ Person
where filter ($1.Company == „Microsoft Research“),
filter($1.Location == „Cambridge"),
filter($1.SecurityLevel > 2) {
discover Lights @ Light
where distance(a),
filter($1.Color == „green");
process OnLampAdd @ l = Lights.Added {
l.On();
}
process OnLampRemove @ l = Lights.Removed {
l.Off();
}
a = 100;
}

20. Demo

21. The Microsoft Robotics Studio
A lightweight concurrency and services oriented runtime
• Handling of sensory input and controlling actuators
• Based on synchronous message passing
• Decentralized System Services (DSS) facilitating tasks and
basic services such as debugging, logging, monitoring,
security, discovery, and data persistence
Authoring/development tools
• Visual programming editor
• Simulation
• Message debugging
Technology libraries and basic algorithms
• Code samples and documentation

22. Open Platform
Supported manufacturers
• Robotics Connection Applications and tools
• Mobile Robots
• Coroware
• KUKA
• LEGO
Authoring Tools
• fischertechnik
• Robosoft Visual Studio Technology libraries
Robot models
• WhiteBox Robotics Device services
• Lynxmotions Runtime Environment (OEM, ISV, IRV)
• Phidgets Robotic Application
• iRobot Roomba
• … Runtime
• Community platforms

23. Development Model
Application model
Application Model • Build complex systems from smaller, simpler
decentralized services
Distribution Distribution model
Model
• Applications are a collection of distributed
services
• Discovery of functionality via contract and
categories
Programming • Self-organizing
Model • Interaction via Messaging
Programming model
• Coordination of messages
• No manual creation of threads, locks,
semaphores

24. Robotics Studio Architecture
Orchestration
Orchestration Application
Decentralized System Services Robotic Services Services
Concurrency and
Coordination
Device Services Library Services
Activation Discovery Robot Model
Runtime
Device 1 Vision
Services
Diagnostics Storage Vis/Sim Device 2 SLAM
… …
Terminal UX
Messaging Transport
Signal Processing
SPL
Hardware Abstraction Layer
HAL

25. Layered and Distributed Application
“Learns” and begins to favors specific behavioral
characteristics. Interacts with the orchestration layer
to achieve the favored patterns.
Behavior Control Layer
Defines and facilitates communication patterns that
coordinate information processing amongst lower
level services.
Orchestration Layer
Provides functionality abstraction, computational
encapsulation, failure isolation, distributed and
concurrent execution, via software services .
Services Layer
Executes algorithms that require near real-time
computation and deterministic time control.
Signal Processing Layer
Directly interfaces with the physical robotics
hardware - Sensors and Actuators.
Hardware Abstraction Layer

26. Service Composition
• Services aren’t just for devices
• Services can be compose to form other services
• Services can represent passive or non-existent devices
> Fused sensory data can be re-exposed as an independent service
Trajectory
Drive
Pose
Motor Encoder
Services
Steering Servo
Robot Model Service
Robot Model
Wheel Base Geometry

27. Runtime Environment
An application is a composition of loosely-coupled services concurrently executing across
scopes
• Interactions described using service contacts
• Components are networked
• Provides isolation between components
> Reliability – ability to restart independently, replaceable/updatable
> Parallelism
y
UI
un dar
Main Port
n Bo
io
icat
A ppl Main Port
Service
Private Port
Code
State Main Port Service
Main Port
Code
Main Port
Private Port State
Private Port
Service
Service Code Service
Code State Code
State State
H/W H/W
* Example: Model car + sensors

28. Communication & Coordination
Port<int> p = new Port<int>();
• Typed Ports p.post(42) ;
• Messages sent to ports
• Arbiter handle messages
> Persistent
> Batch
> Choice
> Join
> …
Activate (
Arbiter.CreateReceiver(false, p, delegate(int i)
{
Console.WriteLine(i) ;
}
) );

29. Runtime Reliability
• Reliability
> Reservations
> Guarantueed CPU bandwith
• Number of fixed threads in dispatcher
> Set during creation of dispatcher
> Not using CLR threadpool
> Less overhead
• No Priorities for threads
> Hard to persuade developer not to use priorities
> So, priorities introduced for dispatcher
• FIFO Ports
> 90,000 SOAP messages/s within a node (=multiple services)
> > 3,000 SOAP messages/s among nodes

30. Service Development vs. Application Development
Runtime Environment
Main Port
Private Port
Decentralized System
Service
Services
Code
Concurrency and
Coordination State
Runtime
• Done only once
• Driver related
• Done by manufacturer
• Distribute & reuse

31. Authoring Environment
Standalone & Visual Studio
integrated
Scales with programmer skill Visual Studio
• Helps beginners with drag/drop
programming
IDE
• Assists advanced developer with
service orchestration Robot Package
Robot Model Designer Code Generator
• Conceptually consistent with
conventional programming
Tool Box
Application Designer Debugging services
languages
Packages
Visualization and
• Context based toolbox Service Contract Designer Simulation
Support Rapid Application Deployment Designer Code Downloader
Development via visual
programming, simulation and
visualization.
Supports programming of
distributed scenarios
Extensible to support other
hardware

32. Visual Programming Language
Robots as first class objects
Data flow and orchestration
• Easy for novices
• Rapid prototyping for experts Activity
Message link
Activity
• Implicit parallelism
• Generates C# code
• Formal checking of protocol
interactions
• Deadlock detection, state
exploration

33. Demo

34. Simulation Capabilities

35. Supported KUKA robot

36. Community: Robot Swarm Heading Target Area

37. Community: Robot Swarm Simulation

38. Community: Maze Simulator

39. Outlook
• VRDK
> Research prototype – not developed anymore
• Microsoft Robotics Studio
> Available as CTP (November CTP, today at 5.00 am)
> Improved visualisation
> Package based deployment
> Wrap up for final version
> CCR / DSS maybe available as separate DL
> http://msdn.microsoft.com/robotics/
• N#
> Ongoing research
> Publications etc.
> Visual Studio integration in progress

40. Thank you or your attention.