2. So far
• Reactive and deliberative distributed
algorithms
• Formal models describing sub-sets of the
systems
• Deterministic models for deliberative
algorithms
• Convex cost functions and feedback control
for reactive systems
3. Problems so far
• How to model
– Sensor uncertainty (localization, vision, range)
– Communication uncertainty
– Actuation uncertainty (e.g. wheel-slip)
• Deterministic algorithms break
• Reactive algorithm become unpredictable
4. Problem Statement
• Predict the performance of a
system given
– Problem
– Algorithms
– Capabilities/Uncertainty
• Find most suitable coordination
scheme / set of resources
4
Robot 1 Robot 2
Task
A
Task
B
Problem
Algorithms
Random Deliberative
Centralized
Decentralized
Collaborative Greedy
N. Correll. Coordination schemes for distributed boundary coverage with a
swarm of miniature robots: synthesis, analysis and experimental validation.
EPFL PhD thesis #3919, 2007.
Capabilities / Probabilistic Behavior
Navigation Localization Communication
5. Master equations and Markov
Chains
5
• The system state ({robot states} X {environment state}) is finite
• Non-deterministic elements of the system follow a known statistical distribution
: Conditional probability to be in state w
when in state w’ a time-step before
Transition probability from w’ to w in a Markov Chain
: Probability for the system to be in state w at time k
w’ w
6. From Master to Rate Equations
6
: probability to be in state x at time k
x can be a robot’s or a system’s state
: Total number of robots
Average number of robots in state x:
7. Example 1: Collision Avoidance
Two states, search and avoid
N0 robots
State duration of avoid
– Probabilistic
– Deterministic
Possible implementations:
Obstacle
“Proximal”
Obstacle
“180o turn”
What are the parameters
of this system and what
are their distributions?
How to get them?
8. Parameters
Encountering probability pR
– Probability to encounter another robot per time
step
Interaction time Ta
– Average time a collision lasts
– Geometric distribution or Dirac pulse
9. Interaction time
Average time Ta constant regardless whether
probabilistic or deterministic
Distribution Ta is different depending on
– Controller
– Model abstraction level
Model is only an approximation!
14. Example 2: Collaboration
Two states: search and wait
N0 robots M0 collaboration sites
State duration of wait
– probabilistic: robots wait a random time
– deterministic: robots wait a fixed time
robot
site
15. Parameters
Encountering probability ps
– Probability to encounter one site
Interaction time Tw
– (Average) time a robot waits for collaboration
before moving on
Robot-Robot collisions are ignored in this example
18. Summary: Memory-less systems
Systems with no or little memory (Time-outs),
essentially reactive
Master equation for a single robot allows
estimation of population dynamics
How to deal with deliberative systems
that use memory?
19. Example 3: Task Allocation
Scenario: 2 robots, 2 tasks A and B
Robots prefer task A over task B
Global metric requires solution of both tasks
Task evaluation subject to noise, robots
choose the wrong task with probability p
A B
1-p p
A B
1-p 1-p
“Greedy” “Coordinated”
20. Example 3a: Task Allocation
Non-Collaborative, Greedy
Both robots will go for task A, then B
Expected time: 2 time-steps
Noise! Effective outcomes might be AA, AB,
BB, BA
There is a possibility to complete in one time-
step (due to noise): AB or BA
What is the state transition diagram of this system?
22. Example 3b: Task Allocation
Collaborative
Robots will allocate the tasks among them
Robot 1 will go for task A, robot 2 go for task B
If only one task is left, both try to accomplish
it
Effective outcomes AA, AB, BB, BA
Expected time to completion 1 time-step for:
AB and BA
24. Summary
Master equation: change of probability to be
in state x
Enumerate all possible states of a system
Calculate all possible state transition
probabilities
Solve difference equations (numerically,
analytically, Lyapunov, …)
Useful for analyzing dominant collaboration
dynamics of a system
I would like to demonstrate my methodology using a simple example where two robots – robot 1 and robot 2 – have to solve two tasks, A and B.
For this problem you can think of a series of possible algorithms that we can use, from simple random allocation of tasks to centralized, optimal allocation.
We also know that our robots key capabilities navigation, localization and communication are subject to uncertainty.
What we can now do is to write down all possible states of the system – robot 1 doing task A, robot 2 doing task B, both robots doing task A and so forth. We can then write down all possible state transitions. For example for a random algorithm all initial states are equally likely. For a deterministic algorithm, state transitions are given by the uncertainty of the robots subystems, for example the likelihood that a coordination message got lost.
In this example, it is easy to calculate the expected value for time to completion analytically for a series of coordination mechanisms and assumptions on sensor and actuator reliability.
More formally, we associate a probability with each state of the system that varies over time as well probabilities for each possible state transition. You can visualize that using a Markov chain, here showing a two-state system with the time varying probability for the system to change from state omega prime to omega.
The rate with which the probability to be in a certain state changes per time step is then given by the Master equation from physics – here in discrete time. The probability gets decreased by all probabilistic state transitions leading away from the system and it gets increased by all probabilistic state transitions leading into the system.
