This work was presented in 14th International Symposium on Stabilization, Safety, and Security of Distributed Systems (SSS 2012) Toronto, Canada (1-4 October 2012). Abstract. End-to-end communication over the network layer (or data link in overlay networks) is one of the most important communication tasks in every communication network, including legacy communication networks as well as mobile ad hoc networks, peer-to-peer networks and mash networks. We study end-to-end algorithms that exchange packets to deliver (high level) messages in FIFO order without omissions or duplications. We present a self-stabilizing end-to-end algorithm that can be applied to networks of bounded capacity that omit, duplicate and reorder packets. The algorithm is network topology independent, and hence suitable for always changing dynamic networks with any churn rate.

1. 14th International Symposium on Stabilization, Safety, and Security of
Distributed Systems (SSS 2012)
Toronto, Canada (1-4 October 2012)
Self-Stabilizing
End-to-End Communication
in
Bounded Capacity, Omitting, Duplicating and non-FIFO
Dynamic Networks
Shlomi Dolev1, Ariel Hanemann1, Elad M. Schiller2, and
Shantanu Sharma1
1 Department of Computer Science, Ben-Gurion University of the Negev, Israel
2 Department of Computer Science and Engineering, Chalmers University of
Technology, Sweden

2. Outline
• Introduction
• Problem Specification
• The Algorithm
• Key Features
• Conclusion
2 of 27

3. Introduction
• End-to-end communication is an essential task in
communicating systems
Omission of packets
capacity in
Duplication by the channel
• Intromission capacityReorder “channel”
of errorsof message transmission is
of the sender
common
• Dynamic networks have a greater chance of having
errors, like, omission, duplication, reordering, than
R
conventional networks
SS
R R
S
• Network has known capacity constant, which is an
upper bound on number of packets in the network
capacity packets

4. Introduction
• Self-Stabilization
– A self-stabilizing system is
a
system
that
can
automatically
recover
following the occurrence
of transient faults
– e.g.-token based mutual
exclusion algorithm
4 of 27

5. Outline
• Introduction
• Problem Specification
• The Algorithm
• Key Features
• Conclusion

6. Problem Specification
• Design an algorithm for a bounded capacity
non-FIFO network which
– Guarantee exactly one copy of packet delivery
– Tolerate corruption, omission, duplication of messages by
the “channel” and the sender
– Deliver messages in the same order they were sent
– Applied to topological independent network
– Employed to any starting configuration
5 of 27

7. Problem Specification
• Why new end-to-end communication protocol?
– TCP is not self-stabilizing
6 of 27

8. Outline
• Introduction
• Problem Specification
• The Algorithm
• Key Features
• Conclusion

9. The Algorithm-Motivation
• Dolev et al* has presented self-stabilizing data link
algorithm for reliable FIFO packet delivery over
bounded non-FIFO and non-duplicating channel
• The sender sends a message infinitely often until it
gets ‘3capacity + 2‘ identical acknowledgements
• The receiver delivers a message when it gets identical
‘capacity + 1‘ messages
*Shlomi Dolev, Swan Dubois, Maria Potop-Butucaru, and Sébastien Tixeuil.
Stabilizing data-link over non-fifo channels with optimal fault-resilience. Inf.
7 of
Process. Lett., 111(18):912–920, 2011.
27

10. The Algorithm-Motivation
1
2
Last delivered alternating
capacity acks for message in bit
Alternating bit
the channel from sender to = 1
last_delivered 0
ab = 0
ack(m, 0)
ack(m,receiver
0)
capacity messages
capacity+1 acks for the
capacity
message sent by sender
capacity capacity acks for message in
messages
capacity+1
m, 0 m, 0the channel from receiver to
m, 0
sender
But, the sender sends new message, gets 3capacity+2 bit
Now, the sender fetchesmessage until itchangesonce
Now, the receiver delivers the 1 ack for bad initialization
message m only alternating all
acknowledgements. However,
to 1 changes last_delivered = 0 the receiver acknowledges
and and repeat the whole process (due to same
incoming message but do not store
3capacity+1 last_delivered=0)
S
3capacity+2
Acknowledgments
R
The channel may containcapacity+2(or zero)
Hence, the sender waits 3 capacity
messages sendercontainfrom receiver zero)
The channel may sends capacity messages
acknowledgement messages
Hence, thein the channel capacity +1(or to sender
messages in the
to the receiver. channel from sender to receiver
m
m
m
1st message
2ed message
3rd message
capacity+1
message
Messages
m
8 of 27

11. The Algorithm-Motivation
• Disadvantages Dolev et al*:
– Explicit synchronization of the sender and the
receiver
– One packet delivery requires lots of packet flow 
overburden network
– Does not consider duplication by the channel
*Shlomi Dolev, Swan Dubois, Maria Potop-Butucaru, and Sébastien Tixeuil.
Stabilizing data-link over non-fifo channels with optimal fault-resilience. Inf.
9 of
Process. Lett., 111(18):912–920, 2011.
27

12. S2E2C Algorithm
• S2E2C (Self-Stabilizing End-to-End Communication)
Algorithm
– Overcomes all the cons of Dolev et al (presented
previously)
– Sends n messages (packets) infinitely
instead of one message at a time
often
– Consider the duplication by the channel
– Message to packet transformation
10 of 27

13. S2E2C Algorithm
Packet_set
1,0,m
1,1,m
• With only one message
Alternating Index
AltIndexs= 1
S
LastDeliveredIndexr = 0
0,
(1,(1, capacity, m)
capacity, m)
capacity+1,
(1, 1, m) m) m)
(1,2,
n-1,
3,m)
(2,2,m) (1,2,m)
1,2,m3
2,2,d
(1,3,m) (2,3,m)
1,3,m
Correct2,3,m
messages
R
m
1,capacity,m
1,capacity+1,m
Capacity
Alternating
Data
Label
Index
Now, the delivery of the correct messages, the receiver deliver it
the receiver selects majority of the message s and
After
acknowledges the sender.
Transmission of messages from the sender to the receiver.
1,n-1,m
11 of 27

14. The Algorithm-The Idea
• Distinct Labels with each outgoing packets
– prevent duplication and guarantee in-order delivery of messages as
they were sent
• Alternating index 0, 1 or 2
– avoids explicit synchronization between the sender and the receiver
• The receiver accumulates packets of ‘n‘ distinct
labels with identical alternating index before the
delivery of messages
• The sender waits for capacity+1 distinct labels
acknowledgements before the next fetch
12 of 27

15. S2E2C Algorithm
Packet_set
Correct1,0,derrors
Messagesmessages
with 1
st message
1st 1,1,d2
ACK_set
0
1
Alternating Index
AltIndexs= 1
S
LastDeliveredIndexr = 0
0, 3
(1,(1, capacity,c)dcc+1)
capacity, d
capacity+1,
(1, 1, d1)dn-1 d )
(1,2, 2
n-1,
3, 4
capacity,11
1 ,1
0,
edmessage
(2,2,d3) (1,2,d3)
2ed1,2,d33
2,2,d
rd message
(1,3,d4) (2,3,d4)
2,3,d
3rd 1,3,d44
R
1,capacity,dc
capacity
1,capacity+1,dc+1
th
plth message
Capacity
Alternating
Data
Label
Index
Now, the delivery of the correct messages,which have identical
the receiver selects n distinct labels the receiver
After
alternating index except LastDeliveredIndex.
acknowledges the sender.
1,n-1,dn-1
Transmission of acknowledgements from the receiver to the sender.
Transmission of messages from the sender to the receiver.
13 of 27

16. The Algorithm at Sender
ml
1st
message
2ed
message
3rd
message
The sender fetches pl messages of
ml length
plth
message
14 of 27

17. The Algorithm at Sender
n > ml
lbl
1
AltIndex ai
ml
1st
message
2
ai
3
ai
ml
ai
n-1
ai
n
ai
Consider all the bits of each pl
messages
2ed
message
3rd
message
Error
Correcting
Encoding
plth
message
1st bit 2ed bit 3rd bit
mlth bit
1st Packet
3rd Packet
redundant bits ed Packet pl
2 to each
The sender appends
message which results in message length n (n>ml) Error Correcting Bits
15 of 27
nth Pacet

18. The Algorithm at Sender
• The transmitting packet has the form of
ai, lbl, dat
ai is alternating index
lbl is labels
dat is data which is formed by ith bit of all messages
• The sender waits for capacity + 1 distinct label
acknowledgements before next fetch
• The sender changes its alternating index x to x+1
mod 3, at every fetch
16 of 27

19. The Algorithm at Receiver
• The receiver collects packets of n distinct labels
which have alternating index different from last
delivered index
– capacity distinct labels may be from the channel
– n – capacity distinct label packets must be from the sender
• The receiver forms
messages form all incoming
packets
ith message is formed by
the ith bit of every packet
1
ai
2
ai
3
ai
ml
ai
n-1
ai
n
ai

20. The Algorithm at Receiver
• After delivery of the message
– Resets data structure
– changes LastDeliveredIndex to the value of delivered
message alternating index
• The receiver acknowledges the sender with
’capacity + 1’ distinct labels and the value of
LastDeliveredIndex
18 of 27

21. Outline
• Introduction
• Problem Specification
• The Algorithm
• Key Features
• Conclusion

22. Key Features
• Handles capacity corruptions in the messages
• Does one time delivery of message with handling of
omission and reorder, i.e., simulate reliable FIFO
message delivery over the non-FIFO network
• No explicit synchronization between the sender and
the receiver
• No need of denotative reset operation at the sender
and at the receiver
• Topological independence
19 of 27

23. Key Features
• Handles
capacity
messages
• The
intromission
of
error correcting bits is
capable
to
prevent
capacity errors or loss
of capacity packets
• capacity corruption is
equivalent to loss of
distinct capacity labels
packets
corruptions
1
ai
2
ai
3
ai
in
the
ml
ai
n-1
ai
n
ai
20 of 27

24. Key Features
• Handles capacity corruptions in the messages
• Does one time delivery of message with handling of
omission and reorder, i.e., simulate reliable FIFO
message delivery over the non-FIFO network
• No explicit synchronization between the sender and
the receiver
• No need of denotative reset operation at the sender
and at the receiver
• Topological independence

25. Key Features
• Does one time delivery of message with
handling of omission and reorder
– The omission of packets is prevented by error
correcting bits
– The duplication of messages is done by either
channel or the sender
• The channel can duplicate up to capacity packets
• The sender duplicates each packets infinity often
– Distinct labels ensures one time order delivery of
all the messages
21 of 27

26. Key Features
• Handles capacity corruptions in the messages
• Does one time delivery of message with handling of
omission and reorder, i.e., simulate reliable FIFO
message delivery over the non-FIFO network
• No explicit synchronization between the sender and
the receiver
• No need of denotative reset operation at the sender
and at the receiver
• Topological independence

27. Key Features
• No explicit synchronization between the sender and
the receiver
– Synchronization of alternating index is necessary due to
arbitrary initialization
– If the sender and the receiver are having same alternating
index, then the receiver does not deliver incoming messages.
But, it acknowledges the sender by capacity + 1 distinct
labels
– On reception of capacity + 1 distinct labels, the sender
changes alternating index and fetches new messages
22 of 27

28. Key Features
• No explicit synchronization between the sender and
the receiver
ACK_set
– Why alternating bit 0 and 1 is not useful in implicit
synchronization?
Packet_set
0,1
0,0
1,1
1,0
Alternating Index
Alternating Index
AltIndexs=s= 1
AltIndex 0
S
0
LastDeliveredIndex r = 1
(0, 2, *) *)
1,
(0,Capacity-1,1
n-2,
0,
1,1
0,1 *)
R
1,0,*
0,0,*
0,1,*
1,1,*
1,2,*
0,2,*
capacity-1,1
capacity-1,0
This situation is same as initial starting configuration,
where the sender and the receiver are having
acknowledges is with
Such situationthe successfully
resets and Now, from
Now,sender gets acknowledgementweAltIndexs 0handled by 1
the sender sends bit 0. changes are r to 1 alternating bit
The receiveralterntingLastDeliveredIndexhavingand
changes packets sender the channel
s
resets Packet_set alternating index 0, 1, 2
at both the end
capacity ,0
1,n -2,*
0,n
1,n-1,*
23
of
27

29. Key Features
• No explicit synchronization between the sender and
the receiver
– Use of alternating index 0, 1 and 2
Packet_set
ACK_set
0,1
1,1
Alternating Index
AltIndexs= 2
1
S
0, 1
1
LastDeliveredIndex r = 0
(1, 2, *) *)
(1, 0, *)
1,
n-2,
R
0,0,*
1,0,*
1,1,*
0,1,*
0,2,*
1,2,*
capacity-1,1
capacity ,1
resets sender channel
Now,sender Now, packet form thefrom thechannel
the sender acknowedgement and the 1to 2
gets sends packets with AltIndex 0 and
1 and
gets the and changes
The receiver changes LastDeliveredIndexr toreceiver
resets Packet_set
synchronized
are
1,n
0,n -2,*
1,n-1,*
0,n-1,*
24
of
27

30. Key Features
• Handles capacity corruptions in the messages
• Does one time delivery of message with handling of
omission and reorder, i.e., simulate reliable FIFO
message delivery over the non-FIFO network
• No explicit synchronization between the sender and
the receiver
• No need of denotative reset operation at the sender
and at the receiver
• Topological independence

31. Key Features
• No need of denotative reset operation at the
sender and at the receiver
– In arbitrary initialization, the sender and the receiver may
have some garbage in its data structure then both do reset
– The reset process for garbage items is only required at
arbitrary stages
25 of 27

32. Key Features
• Handles capacity corruptions in the messages
• Does one time delivery of message with handling of
omission and reorder, i.e., simulate reliable FIFO
message delivery over the non-FIFO network
• No explicit synchronization between the sender and
the receiver
• No need of denotative reset operation at the sender
and at the receiver
• Topological independence
26 of 27

33. Outline
• Introduction
• Problem Specification
• The Algorithm
• Key Features
• Conclusion

34. Conclusion
• A self-stabilizing end-to-end communication algorithm
• Topological independent, fits for dynamic networks
• Implicit sender/receiver synchronization
• Simulate perfect FIFO behavior – in-order delivery,
avoid omission, duplication by the sender, duplication
by the channel - over non-FIFO networks
Interested audience may refer the paper with the same title for details and proofs.
27 of 27

35. Presentation is available at
http://www.cs.bgu.ac.il/~sharmas/publication.html
Shlomi Dolev1, Ariel Hanemann1, Elad M. Schiller2, and
Shantanu Sharma1
1 Department of Computer Science, Ben-Gurion University of the
Negev, Israel
{dolev,hanemann,sharmas}@cs.bgu.ac.il
2 Department of Computer Science and Engineering, Chalmers University of
Technology, Sweden
elad@chalmers.se