Talk in Workshop on Conflict Resolution in Decision Making (COREDEMA '13)

1. Outline Discovery Strategy Promotion Techniques Results Conclusions
Strategies for Cooperation Emergence
in Distributed Service Discovery
E. del Val M. Rebollo V. Botti
Univ. Politècnica de València (Spain)
COREDEMA ’13
Salamanca, May 2013
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

2. Outline Discovery Strategy Promotion Techniques Results Conclusions
Promoting Cooperation
Motivation
There are scenarios in decentralized systems in which cooperation
plays a central role
agents connected in networks
bounded rationality
heterogeneous, self-interested agents
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

3. Outline Discovery Strategy Promotion Techniques Results Conclusions
Our Proposal
The challenge
Obtain an emergent, cooperative global behavior even when
cooperators are a minority, from local decisions.
What is done. . .
a network structure that ensures navigation and efficiency
structural changes to isolate undesired agents
variable incentives to promote cooperation
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

4. Outline Discovery Strategy Promotion Techniques Results Conclusions
Outline
1 Outline
2 Discovery Strategy
3 Isolated Cooperation Promotion Techniques
4 Combined Cooperation Model
5 Results
6 Conclusions
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

5. Outline Discovery Strategy Promotion Techniques Results Conclusions
Agent Network Model
A = {1, ..., n} a set of agents connected in a
undirected network G, where N(i) denotes the neighbors of
agent i
each agent plays a role ri and offers a service si
agents have an initial behavior: cooperative (c) or not
cooperative (nc)
each agent has an initial budget b
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

6. Outline Discovery Strategy Promotion Techniques Results Conclusions
Service Discovery
Purpose
Locate in the network a similar enough service offer by a concrete
role
qt
i = {stg , rtg , TTL, ε, {}}
stg required semantic service description
rtg organizational role the target agent should play
TTL: time to live
ε similarity threshold in [0, 1]
{} participant list (initially empty)
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

7. Outline Discovery Strategy Promotion Techniques Results Conclusions
Discovery Process
i
Ri = {r1}
Si = {s1}
k
CH(k, t) = 0.5
j
CH(j, t) = 0.5
n
CH(n, t) = 0.15
A S R |N|
k Sk Rk = {r1} 5
n Sn Rn = {r2} 5
j Sj Rj = {r1} 4
v
Rt = {r5}
St = {s6}
m
Rm = {r7}
Sm = {s7}
each agent knows its
direct neighbors
query qt
i is redirected to
the most promising
neighbor
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

8. Outline Discovery Strategy Promotion Techniques Results Conclusions
Similarity Measure
FNi (tg) = argmaxj∈Ni
P( j, tg )
For each neighbor j, P( j, tg ) determines the probability that the
neighbor j redirects the search to the nearest network community
where there are more probabilities of finding the agent tg.
P( j, tg ) = 1 −





1 −





CH(j, tg)
k∈Ni
CH(k, tg)










kj
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

9. Outline Discovery Strategy Promotion Techniques Results Conclusions
Social Plasticity
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 10 20 30 40 50 60
Probabilitytomaintainthelink
Number of queries that were forwarded to other links
n = 2
n = 4
n = 6
rewiring action λ to avoid
non-cooperative agents
decay function using a
sigmoid
d parameter establishes
benevolence of the agent
Pdecay (rqij) = 1
1+e
−(rqij −d)
n
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

10. Outline Discovery Strategy Promotion Techniques Results Conclusions
Social Plasticity Effects
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

11. Outline Discovery Strategy Promotion Techniques Results Conclusions
Social Plasticity Effects
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

12. Outline Discovery Strategy Promotion Techniques Results Conclusions
Incentives Effect
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

13. Outline Discovery Strategy Promotion Techniques Results Conclusions
Social Plasticity and Incentives
When a neighbor j receives a query qt
i , it has a set of possible
actions Ac = {ρ, ∞, 1, 2, ..., ki , ∅, λ}, where:
ρ is asking for a service
∞ is providing the service
{1, ..., ki } is forwarding the query to one of its neighbors ∈ Ni
∅ is doing nothing
λ rewiring a link
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

14. Outline Discovery Strategy Promotion Techniques Results Conclusions
Action Selection
Action Condition
at
i = ∞ if |CH(i, tg)| ≥ ε
at
i = ∅ if |CH(i, tg)| < ε ∧
at−1
j = ∅, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = j if |CH(i, tg)| < ε ∧
at−1
j = 0, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = λ if at−1
i = j ∧
at
j = ∅ ∧ |coop| < σ, coop ⊆ Ni (g)|j is a coop.
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

15. Outline Discovery Strategy Promotion Techniques Results Conclusions
Action Selection
Action Condition
at
i = ∞
if |CH(i, tg)| ≥ ε
at
i = ∅ if |CH(i, tg)| < ε ∧
at−1
j = ∅, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = j if |CH(i, tg)| < ε ∧
at−1
j = 0, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = λ if at−1
i = j ∧
at
j = ∅ ∧ |coop| < σ, coop ⊆ Ni (g)|j is a coop.
Do the task if agent knows how to do it
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

16. Outline Discovery Strategy Promotion Techniques Results Conclusions
Action Selection
Action Condition
at
i = ∞ if |CH(i, tg)| ≥ ε
at
i = ∅
if |CH(i, tg)| < ε ∧
at−1
j = ∅, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = j if |CH(i, tg)| < ε ∧
at−1
j = 0, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = λ if at−1
i = j ∧
at
j = ∅ ∧ |coop| < σ, coop ⊆ Ni (g)|j is a coop.
Do nothing if the agent guess that the most promising neighbor
will no cooperate
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

17. Outline Discovery Strategy Promotion Techniques Results Conclusions
Action Selection
Action Condition
at
i = ∞ if |CH(i, tg)| ≥ ε
at
i = ∅ if |CH(i, tg)| < ε ∧
at−1
j = ∅, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = j
if |CH(i, tg)| < ε ∧
at−1
j = 0, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = λ if at−1
i = j ∧
at
j = ∅ ∧ |coop| < σ, coop ⊆ Ni (g)|j is a coop.
Forward the query if the agent guess that the most promising
neighbor will cooperate
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

18. Outline Discovery Strategy Promotion Techniques Results Conclusions
Action Selection
Action Condition
at
i = ∞ if |CH(i, tg)| ≥ ε
at
i = ∅ if |CH(i, tg)| < ε ∧
at−1
j = ∅, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = j if |CH(i, tg)| < ε ∧
at−1
j = 0, j ∈ argmax(CHt−1
1 , ..., CHt−1
ki
)
at
i = λ
if at−1
i = j ∧
at
j = ∅ ∧ |coop| < σ, coop ⊆ Ni (g)|j is a coop.
Rewire some links with a probability Pdecay if the agent is
surrounded by non-coop agents
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

19. Outline Discovery Strategy Promotion Techniques Results Conclusions
Costs of the Actions
ut
i (at
i ) =



−β if at
i = ρ
p if at
i = ∞
−c if at
i ∈ {1, 2, ..., ki }
0 if at
i = ∅ ∧ t ≤ t : at
i ∈ {1, 2, ...ki }
α if at
i = ∅ ∧ ∃t ≤ t : at
i ∈ {1, 2, ..., ki } ∧ ∃j ∈ A : at
j = ∞
−γ if at
i = λ
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

20. Outline Discovery Strategy Promotion Techniques Results Conclusions
Incentives Policy
uniformly distributed
System the system provides incentives
Fixed the agent that request the service pays for it
base on a criterion
Path depends on the length of the path
SimDg the more similar the higher reward
InvSimDg the less similar the higher reward
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

21. Outline Discovery Strategy Promotion Techniques Results Conclusions
Experimental Parameters
network size: 1 000 agents
average degree of connection: 2.5
similarity threshold ε = 0.75
TTL = 100
initial budget: 100
40 % cooperative - 60 % non cooperative
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

22. Outline Discovery Strategy Promotion Techniques Results Conclusions
Budget Distribution
Incentives
0
200
400
600
800
1000
1200
1400
1600
2 4 6 8 10 12 14 16 18 20
budget
degree of connection
Fixed Path Sim InvSim
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

23. Outline Discovery Strategy Promotion Techniques Results Conclusions
Budget Distribution
Incentives
0
200
400
600
800
1000
1200
1400
1600
2 4 6 8 10 12 14 16 18 20
budget
degree of connection
Fixed Path Sim InvSim
Incentives + Social Plasticity
0
200
400
600
800
1000
1200
1400
1600
2 4 6 8 10 12 14 16 18 20
budget degree of connection
Fixed Path Sim InvSim
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

24. Outline Discovery Strategy Promotion Techniques Results Conclusions
Cooperative Behavior Rate
Incentives
0
200
400
600
800
1000
2 4 6 8 10 12 14 16 18
coop
snapshot
Fixed
Path
Sim
InvSim
System
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

25. Outline Discovery Strategy Promotion Techniques Results Conclusions
Cooperative Behavior Rate
Incentives
0
200
400
600
800
1000
2 4 6 8 10 12 14 16 18
coop
snapshot
Fixed
Path
Sim
InvSim
System
Incentives + Social Plasticity
0
200
400
600
800
1000
2 4 6 8 10 12 14 16 18
coop snapshot
Fixed
Path
Sim
InvSim
System
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

26. Outline Discovery Strategy Promotion Techniques Results Conclusions
Success Rate
Incentives
0
20
40
60
80
100
2 4 6 8 10 12 14 16 18
%successfulsearches
snapshot
Fixed Path Sim InvSim System
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

27. Outline Discovery Strategy Promotion Techniques Results Conclusions
Success Rate
Incentives
0
20
40
60
80
100
2 4 6 8 10 12 14 16 18
%successfulsearches
snapshot
Fixed Path Sim InvSim System
Incentives + Social Plasticity
0
20
40
60
80
100
2 4 6 8 10 12 14 16 18
%successfulsearches snapshot
Fixed
Path
Sim
InvSim
System
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

28. Outline Discovery Strategy Promotion Techniques Results Conclusions
Path Length
Incentives
0
20
40
60
80
100
2 4 6 8 10 12 14 16 18
steps
snapshot
Fixed Path Sim InvSim System
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

29. Outline Discovery Strategy Promotion Techniques Results Conclusions
Path Length
Incentives
0
20
40
60
80
100
2 4 6 8 10 12 14 16 18
steps
snapshot
Fixed Path Sim InvSim System
Incentives + Social Plasticity
0
20
40
60
80
100
2 4 6 8 10 12 14 16 18
steps snapshot
Fixed
Path
Sim
InvSim
System
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

30. Outline Discovery Strategy Promotion Techniques Results Conclusions
Num. of Broken Links (Rewired)
0
200
400
600
800
1000
2 4 6 8 10 12 14 16 18
budget
snapshot
Fixed Path Sim InvSim System
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery

31. Outline Discovery Strategy Promotion Techniques Results Conclusions
Conclusions
What we’ve done
To combine structural changes (social plasticity) with different
incentives policies in a decentralized service discovery scenario with
local search.
What we’ve got
variable incentives work better than homogenous ones
combination of mechanisms promotes cooperation in scenarios
in which |nc| > |c|
it increases the performance of the agents
reduces the average path length
increases the success rate
M. Rebollo et al. (UPV) COREDEMA’13
Strategies for Cooperation Emergence in Distributed Service Discovery