MULTIMEDIA COMMUNICATION & NETWORKS

1. 13PIT101
Multimedia Communication & Networks
UNIT - I
Dr.A.Kathirvel
Professor & Head/IT - VCEW

2. Unit - I
Open Data Network Model – Narrow Waist Model of the
Internet - Success and Limitations of the Internet – Suggested
Improvements for IP and TCP – Significance of UDP in
modern Communication – Network level Solutions – End to
End Solutions – Best Effort service model – Scheduling and
Dropping policies for Best Effort Service model

3. Open Data Network Models

4. LAYERED TASKS
We use the concept of layers in our daily life. As an example, let us consider
two friends who communicate through postal mail. The process of sending a
letter to a friend would be complex if there were no services available from
the post office.
Topics discussed in this section:
Sender, Receiver, and Carrier
Hierarchy

5. Figure 1 Tasks involved in sending a letter

6. THE OSI MODEL
Established in 1947, the International Standards Organization (ISO) is a
multinational body dedicated to worldwide agreement on international
standards. An ISO standard that covers all aspects of network communications
is the Open Systems Interconnection (OSI) model. It was first introduced in the
late 1970s.
Topics discussed in this section:
Layered Architecture
Peer-to-Peer Processes
Encapsulation

7. Note
ISO is the organization.
OSI is the model.

8. Figure 2 Seven layers of the OSI model

9. Figure 3 The interaction between layers in the OSI model

10. Figure 4 An exchange using the OSI model

11. LAYERS IN THE OSI MODEL
In this section we briefly describe the functions of each layer in
the OSI model.
Topics discussed in this section:
Physical Layer
Data Link Layer
Network Layer
Transport Layer
Session Layer
Presentation Layer
Application Layer

12. Figure 5 Physical layer

13. Note
The physical layer is responsible for movements of
individual bits from one hop (node) to the next.

14. Figure 6 Data link layer

15. Note
The data link layer is responsible for moving
frames from one hop (node) to the next.

16. Figure 7 Hop-to-hop delivery

17. Figure 8 Network layer

18. Note
The network layer is responsible for the
delivery of individual packets from
the source host to the destination host.

19. Figure 9 Source-to-destination delivery

20. Figure 10 Transport layer

21. Note
The transport layer is responsible for the delivery
of a message from one process to another.

22. Figure 11 Reliable process-to-process delivery of a message

23. Figure 12 Session layer

24. Note
The session layer is responsible for dialog
control and synchronization.

25. Figure 13 Presentation layer

26. Note
The presentation layer is responsible for translation,
compression, and encryption.

27. Figure 14 Application layer

28. Note
The application layer is responsible for
providing services to the user.

29. Figure 15 Summary of layers

30. TCP/IP PROTOCOL SUITE
The layers in the TCP/IP protocol suite do not exactly match those
in the OSI model. The original TCP/IP protocol suite was defined
as having four layers: host-to-network, internet, transport, and
application. However, when TCP/IP is compared to OSI, we can
say that the TCP/IP protocol suite is made of five layers: physical,
data link, network, transport, and application.
Topics discussed in this section:
Physical and Data Link Layers
Network Layer
Transport Layer
Application Layer

31. Figure 16 TCP/IP and OSI model

32. ADDRESSING
Four levels of addresses are used in an internet employing the TCP/IP
protocols: physical, logical, port, and specific.
Topics discussed in this section:
Physical Addresses
Logical Addresses
Port Addresses
Specific Addresses

33. Figure 17 Addresses in TCP/IP

34. Figure 18 Relationship of layers and addresses in TCP/IP

35. Example 1
In Figure 19 a node with physical address 10 sends a frame to a
node with physical address 87. The two nodes are connected by
a link (bus topology LAN). As the figure shows, the computer
with physical address 10 is the sender, and the computer with
physical address 87 is the receiver.

36. Figure 19 Physical addresses

37. Example 2
Most local-area networks use a 48-bit (6-byte) physical
address written as 12 hexadecimal digits; every byte (2
hexadecimal digits) is separated by a colon, as shown below:
07:01:02:01:2C:4B
A 6-byte (12 hexadecimal digits) physical address.

38. Example 3
Figure 20 shows a part of an internet with two routers connecting
three LANs. Each device (computer or router) has a pair of
addresses (logical and physical) for each connection. In this case,
each computer is connected to only one link and therefore has
only one pair of addresses. Each router, however, is connected to
three networks (only two are shown in the figure). So each router
has three pairs of addresses, one for each connection.

39. Figure 20 IP addresses

40. Example 4
Figure 21 shows two computers communicating via the Internet.
The sending computer is running three processes at this time
with port addresses a, b, and c. The receiving computer is
running two processes at this time with port addresses j and k.
Process a in the sending computer needs to communicate with
process j in the receiving computer. Note that although physical
addresses change from hop to hop, logical and port addresses
remain the same from the source to destination.

41. Figure 21 Port addresses

42. Note
The physical addresses will change from hop to hop,
but the logical addresses usually remain the same.

43. Example 5
A port address is a 16-bit address represented by one decimal
number as shown.
753
A 16-bit port address represented
as one single number.

44. Narrow waist Model of the
Internet

45. Fundamental Goal
• “technique for multiplexed utilization of existing
interconnected networks”
• Multiplexing (sharing)
– Shared use of a single communications channel
• Existing networks (interconnection)

46. Fundamental Goal: Sharing
Packet Switching
• No connection setup
• Forwarding based on destination address in packet
• Efficient sharing of resources
Tradeoff: Resource management potentially more difficult.

47. Type of Packet Switching: Datagrams
• Information for forwarding traffic is contained in destination address of
packet
• No state established ahead of time (helps fate sharing)
• Basic building block
• Minimal assumption about network service
Alternatives
• Circuit Switching: Signaling protocol sets up entire path out-of-band. (cf.
the phone network)
• Virtual Circuits: Hybrid approach. Packets carry “tags” to indicate path,
forwarding over IP
• Source routing: Complete route is contained in each data packet

48. An Age-Old Debate
Circuit Switching
• Resource control, accounting, ability to “pin” paths, etc.
Packet Switching
• Sharing of resources, soft state (good resilience properties),
etc.
It is held that packet switching was one of the Internet’s greatest
design choices.
Of course, there are constant attempts to shoehorn the best aspects of
circuits into packet switching.
Examples: Capabilities, MPLS, ATM, IntServ QoS, etc.

49. Stopping Unwanted Traffic is Hard
February 2000
March 2006

50. Research: Stopping Unwanted Traffic
• Datagram networks: easy for anyone to send traffic to anyone
else…even if they don’t want it!
cnn.com
Possible Defenses
• Monitoring + Filtering: Detect DoS attack and install filters to
drop traffic.
• Capabilities: Only accept traffic that carries a “capability”

51. The Design Goals of Internet, v1
• Interconnection/Multiplexing (packet switching)
• Resilience/Survivability (fate sharing)
• Heterogeneity
– Different types of services
Decreasing
– Different types of networks
Priority
• Distributed management
• Cost effectiveness
“This set of goals might seem to be nothing more
than a checklist of all the desirable network features.
• Ease of attachment
It is important to understand that these goals are in
• Accountability
order of importance, and an entirely different
network architecture would result if the order
were changed.”
These goals were prioritized for a military network.
Should priorities change as the network evolves?

52. Fundamental Goal: Interconnection
• Need to interconnect many existing networks
• Hide underlying technology from applications
• Decisions:
– Network provides minimal functionality
– “Narrow waist”
email WWW phone...
SMTP HTTP RTP...
Applications
TCP UDP…
IP
ethernet PPP…
CSMA async sonet...
copper fiber radio...
Technology
Tradeoff: No assumptions, no guarantees.

53. The Internet Protocol Suite
FTP
HTTP
DNS
TCP
TFTP
UDP TCP
UDP
IP
Applications
Waist
Data Link
Ethernet
SONET
802.11
Physical
The Hourglass Model
The waist facilitates interoperability
53

54. The “Curse of the Narrow Waist”
• IP over anything, anything over IP
– Has allowed for much innovation both above and below the
IP layer of the stack
– An IP stack gets a device on the Internet
• Drawback: very difficult to make changes to IP
– But…people are trying
– NSF GENI project: http://www.geni.net/

55. Interconnection: “Gateways”
• Interconnect heterogeneous networks
• No state about ongoing connections
– Stateless packet switches
• Generally, router == gateway
• But, we can think of your home router/NAT as also performing the function
of a gateway
192.168.1.51
Home
Network
68.211.6.120:50878
192.168.1.52
68.211.6.120:50879
Internet

56. Network Address Translation
• For outbound traffic, the gateway:
– Creates a table entry for computer's local IP address and port number
– Replaces the sending computer's non-routable IP address with the
gateway IP address.
– replaces the sending computer's source port
• For inbound traffic, the gateway:
– checks the destination port on the packet
– rewrites the destination address and destination port those in the table
and forwards traffic to local machine

57. NAT Traversal
• Problem: Machines behind NAT not globally addressable or routable.
Can’t initiate inbound connections.
• One solution: Simple Traversal of UDP Through NATs
– STUN client contacts STUN server
– STUN server tells client which IP/Port the NAT mapped it to
– STUN client uses that IP/Port for call establishment/incoming
messages
Home
Network 1
Relay node
More next time.
Home
Network 2

58. Goal #2: Survivability
• Network should continue to work, even if some devices fail, are
compromised, etc.
• Failures on the Abilene (Internet 2) backbone network over the course of 6
months
Thanks to Yiyi Huang
How well does the current Internet support
survivability?

59. Goal #2: Survivability
Two Options
• Replication
– Keep state at multiple places in the network, recover when nodes crash
• Fate-sharing
– Acceptable to lose state information for some entity if the entity itself is
lost
Reasons for Fate Sharing
• Can support arbitrarily complex failure scenarios
• Engineering is easier
Some reversals of this trend:
NAT, Routing Control Platform

60. Goal #3: Heterogeneous Services
• TCP/IP designed as a monolithic transport
– TCP for flow control, reliable delivery
– IP for forwarding
• Became clear that not every type of application would need
reliable, in-order delivery
– Example: Voice and video over networks
– Example: DNS
– Why don’t these applications require reliable, in-order
delivery?
– Narrow waist: allowed proliferation of transport protocols

61. Topic: Voice and Video over Networks
• Deadlines: Timeliness more important than 100% reliability.
• Propagation of errors: Some losses more devastating than
others
Loss i
A chor Fra e (I-Frame)
Propagates to Depe de t Fra es
(P and B-Frames)

62. Goal #3b: Heterogeneous Networks
• Build minimal functionality into the network
– No need to re-engineer for each type of network
• “Best effort” service model.
– Lost packets
– Out-of-order packets
– No quality guarantees
– No information about failures, performance, etc.
Tradeoff: Network management more difficult

63. Research: Network Anomaly Detection
• Operators want to detect when a traffic flow from ingress to
egress generates a “spike”.
• Problem: Today’s protocols don’t readily expose this
information.
• Management/debuggability not initially a high priority!

64. Goal #4: Distributed Management
Many examples:
• Addressing (ARIN, RIPE, APNIC, etc.)
– Though this was recently threatened.
• Naming (DNS)
• Routing (BGP)
No single entity in charge.
Allows for organic growth, scalable management.
Tradeoff: No one party has visibility/control.

65. No Owner, No Responsible Party
“Some of the most significant problems with the Internet today relate to
lack of sufficient tools for distributed management, especially in the area of
routing.”
• Hard to figure out who/what’s causing a problem
• Worse yet, local actions have global effects…

66. Local Actions, Global Consequences
“…a glitch at a small ISP… triggered a major outage in Internet access across
the country. The problem started when MAI Network Services...passed bad
router information from one of its customers onto Sprint.”
-- news.com, April 25, 1997

67. Goal #5: Cost Effectiveness
• Packet headers introduce high overhead
• End-to-end retransmission of lost packets
– Potentially wasteful of bandwidth by placing burden on the
edges of the network
Arguably a good tradeoff. Current trends are to exploit
redundancy even more.

68. Goal #6: Ease of Attachment
• IP is “plug and play” Anything with a working IP stack can
connect to the Internet (hourglass model)
• A huge success!
– Lesson: Lower the barrier to innovation/entry and people
will get creative (e.g., Cerf and Kahn probably did not
think about IP stacks on phones, sensors, etc.)
• But….
Tradeoff: Burden on end systems/programmers.

69. Goal #7: Accountability
• Note: Accountability mentioned in early papers on TCP/IP, but
not prioritized
• Datagram networks make accounting tricky.
– The phone network has had an easier time figuring out
billing
– Payments/billing on the Internet is much less precise
Tradeoff: Broken payment models and incentives.

70. Success and Limitations of the Internet
• Success of Internet
– e-com, Internet Marketing etc..
• The quality of information resources might not always be
reliable and accurate.
• Searching of information can be very tedious.
• Internet is definetly not 100% secure.
• Performance and speed are the main limitations to today's
Internet

71. Transport Protocols
• Provide logical communication between
application
application processes running on
transport
network
different hosts
data link
physical
• Run on end hosts
– Sender: breaks application messages
into segments,
and passes to network layer
– Receiver: reassembles segments into
messages, passes to application layer
• Multiple transport protocol available to
applications
– Internet: TCP and UDP
network
data link
physical
network
data link
physical
network
data link
physical
network
data link
physical
network
data link
physical
application
transport
network
data link
physical
71

72. Internet Transport Protocols
• Datagram messaging service (UDP)
– No-frills extension of “best-effort” IP
• Reliable, in-order delivery (TCP)
– Connection set-up
– Discarding of corrupted packets
– Retransmission of lost packets
– Flow control
– Congestion control (next lecture)
• Other services not available
– Delay guarantees
– Bandwidth guarantees
72

73. Multiplexing and Demultiplexing
• Host receives IP datagrams
– Each datagram has source and
destination IP address,
– Each datagram carries one
transport-layer segment
– Each segment has source and
destination port number
• Host uses IP addresses and port
numbers to direct the segment to
appropriate socket
32 bits
source port #
dest port #
other header fields
application
data
(message)
TCP/UDP segment format
73

74. Unreliable Message Delivery Service
• Lightweight communication between processes
– Avoid overhead and delays of ordered, reliable delivery
– Send messages to and receive them from a socket
• User Datagram Protocol (UDP)
– IP plus port numbers to support (de)multiplexing
– Optional error checking on the packet contents
SRC port
DST port
checksum
length
DATA
74

75. Why Would Anyone Use UDP?
• Finer control over what data is sent and when
– As soon as an application process writes into the socket
– … UDP will package the data and send the packet
• No delay for connection establishment
– UDP just blasts away without any formal preliminaries
– … which avoids introducing any unnecessary delays
• No connection state
– No allocation of buffers, parameters, sequence #s, etc.
– … making it easier to handle many active clients at once
• Small packet header overhead
– UDP header is only eight-bytes long
75

76. Popular Applications That Use UDP
• Multimedia streaming
– Retransmitting lost/corrupted packets is not worthwhile
– By the time the packet is retransmitted, it’s too late
– E.g., telephone calls, video conferencing, gaming
• Simple query protocols like Domain Name System
– Overhead of connection establishment is overkill
– Easier to have application retransmit if needed
“Address for www.cnn.com?”
“12.3.4.15”
76

77. Transmission Control Protocol (TCP)
• Connection oriented
– Explicit set-up and tear-down of TCP session
• Stream-of-bytes service
– Sends and receives a stream of bytes, not messages
• Reliable, in-order delivery
– Checksums to detect corrupted data
– Acknowledgments & retransmissions for reliable delivery
– Sequence numbers to detect losses and reorder data
• Flow control
– Prevent overflow of the receiver’s buffer space
• Congestion control
– Adapt to network congestion for the greater good
77

78. An Analogy: Talking on a Cell Phone
• Alice and Bob on their cell phones
– Both Alice and Bob are talking
• What if Alice couldn’t understand Bob?
– Bob asks Alice to repeat what she said
• What if Bob hasn’t heard Alice for a while?
– Is Alice just being quiet?
– Or, have Bob and Alice lost reception?
– How long should Bob just keep on talking?
– Maybe Alice should periodically say “uh huh”
– … or Bob should ask “Can you hear me now?” 
78

79. Some Take-Aways from the Example
• Acknowledgments from receiver
– Positive: “okay” or “ACK”
– Negative: “please repeat that” or “NACK”
• Timeout by the sender (“stop and wait”)
– Don’t wait indefinitely without receiving some response
– … whether a positive or a negative acknowledgment
• Retransmission by the sender
– After receiving a “NACK” from the receiver
– After receiving no feedback from the receiver
79

80. Challenges of Reliable Data Transfer
• Over a perfectly reliable channel
– All of the data arrives in order, just as it was sent
– Simple: sender sends data, and receiver receives data
• Over a channel with bit errors
– All of the data arrives in order, but some bits corrupted
– Receiver detects errors and says “please repeat that”
– Sender retransmits the data that were corrupted
• Over a lossy channel with bit errors
– Some data are missing, and some bits are corrupted
– Receiver detects errors but cannot always detect loss
– Sender must wait for acknowledgment (“ACK” or “OK”)
– … and retransmit data after some time if no ACK arrives
80

81. TCP Support for Reliable Delivery
•
•
•
81
Checksum
– Used to detect corrupted data at the receiver
– …leading the receiver to drop the packet
Sequence numbers
– Used to detect missing data
– ... and for putting the data back in order
Retransmission
– Sender retransmits lost or corrupted data
– Timeout based on estimates of round-trip time
– Fast retransmit algorithm for rapid retransmission

82. TCP Segments
82

83. TCP “Stream of Bytes” Service
Host A
Host B
83

84. …Emulated Using TCP “Segments”
Host A
Segment sent when:
TCP Data
1.
2.
3.
TCP Data
Host B
84
Segment full (Max Segment Size),
Not full, but times out, or
“Pushed” by application.

85. TCP Segment
IP Data
TCP Data (segment)
TCP Hdr
IP Hdr
• IP packet
– No bigger than Maximum Transmission Unit (MTU)
– E.g., up to 1500 bytes on an Ethernet
• TCP packet
– IP packet with a TCP header and data inside
– TCP header is typically 20 bytes long
• TCP segment
– No more than Maximum Segment Size (MSS) bytes
– E.g., up to 1460 consecutive bytes from the stream
85

86. Sequence Numbers
Host A
ISN (initial sequence number)
Sequence
number = 1st
byte
TCP Data
TCP
HDR
TCP Data
Host B
86
ACK sequence
number = next
expected byte
TCP
HDR

87. Initial Sequence Number (ISN)
• Sequence number for the very first byte
– E.g., Why not a de facto ISN of 0?
• Practical issue
– IP addresses and port #s uniquely identify a connection
– Eventually, though, these port #s do get used again
– … and there is a chance an old packet is still in flight
– … and might be associated with the new connection
• So, TCP requires changing the ISN over time
– Set from a 32-bit clock that ticks every 4 microseconds
– … which only wraps around once every 4.55 hours!
• But, this means the hosts need to exchange ISNs
87

88. TCP Three-Way Handshake
88

89. Establishing a TCP Connection
A
B
Each host tells its
ISN to the other
host.
• Three-way handshake to establish connection
– Host A sends a SYN (open) to the host B
– Host B returns a SYN acknowledgment (SYN ACK)
– Host A sends an ACK to acknowledge the SYN ACK
89

90. TCP Header
Source port
Destination port
Sequence number
Flags:
SYN
FIN
RST
PSH
URG
ACK
Acknowledgment
HdrLen
0
Flags
Advertised window
Checksum
Urgent pointer
Options (variable)
Data
90

91. Step 1: A’s Initial SYN Packet
A’s port
B’s port
A’s Initial Sequence Number
Flags: SYN
FIN
RST
PSH
UR
G
ACK
Acknowledgment
20
0
Flags
Advertised window
Checksum
Urgent pointer
Options (variable)
A tells B it wants to open a connection…
91

92. Step 2: B’s SYN-ACK Packet
B’s port
A’s port
B’s Initial Sequence Number
Flags:
SYN
FIN
RST
PSH
URG
ACK
A’s ISN plus 1
20
0
Checksum
Flags
Advertised window
Urgent pointer
Options (variable)
B tells A it accepts, and is ready to hear the next byte…
… upon receiving this packet, A can start sending data
92

93. Step 3: A’s ACK of the SYN-ACK
A’s port
B’s port
Sequence number
Flags:
SYN
FIN
RST
PSH
URG
ACK
B’s ISN plus 1
20
0
Flags
Advertised window
Checksum
Urgent pointer
Options (variable)
A tells B it wants is okay to start sending
… upon receiving this packet, B can start sending data
93

94. What if the SYN Packet Gets Lost?
• Suppose the SYN packet gets lost
– Packet is lost inside the network, or
– Server rejects the packet (e.g., listen queue is full)
• Eventually, no SYN-ACK arrives
– Sender sets a timer and wait for the SYN-ACK
– … and retransmits the SYN-ACK if needed
• How should the TCP sender set the timer?
– Sender has no idea how far away the receiver is
– Hard to guess a reasonable length of time to wait
– Some TCPs use a default of 3 or 6 seconds
94

95. SYN Loss and Web Downloads
• User clicks on a hypertext link
– Browser creates a socket and does a “connect”
– The “connect” triggers the OS to transmit a SYN
• If the SYN is lost…
– The 3-6 seconds of delay may be very long
– The user may get impatient
– … and click the hyperlink again, or click “reload”
• User triggers an “abort” of the “connect”
– Browser creates a new socket and does a “connect”
– Essentially, forces a faster send of a new SYN packet!
– Sometimes very effective, and the page comes fast
95

96. TCP Retransmissions
96

97. Automatic Repeat reQuest (ARQ)
• Automatic Repeat Request
– Receiver sends acknowledgment
(ACK) when it receives packet
– Sender waits for ACK and timeouts if it
does not arrive within some time period
Sender
Timeout
• Simplest ARQ protocol
– Stop and wait
– Send a packet, stop and wait until ACK
arrives
Receiver
Time
97

98. Packet lost
98
Timeout
Timeout
Timeout
Timeout
Timeout
Timeout
Reasons for Retransmission
ACK lost
DUPLICATE
PACKET
Early timeout
DUPLICATE
PACKETS

99. How Long Should Sender Wait?
• Sender sets a timeout to wait for an ACK
– Too short: wasted retransmissions
– Too long: excessive delays when packet lost
• TCP sets timeout as a function of the RTT
– Expect ACK to arrive after an RTT
– … plus a fudge factor to account for queuing
• But, how does the sender know the RTT?
– Can estimate the RTT by watching the ACKs
– Smooth estimate: keep a running average of the RTT
• EstimatedRTT = a * EstimatedRTT + (1 –a ) * SampleRTT
– Compute timeout: TimeOut = 2 * EstimatedRTT
99

100. Example RTT Estimation
RTT: gaia.cs.umass.edu to fantasia.eurecom.fr
350
RTT (milliseconds)
300
250
200
150
100
1
8
15
22
29
36
43
50
57
64
71
time (seconnds)
SampleRTT
100
Estimated RTT
78
85
92
99
106

101. A Flaw in This Approach
• An ACK doesn’t really acknowledge a transmission
– Rather, it acknowledges receipt of the data
• Consider a retransmission of a lost packet
– If you assume the ACK goes with the 1st transmission
– … the SampleRTT comes out way too large
• Consider a duplicate packet
– If you assume the ACK goes with the 2nd transmission
– … the Sample RTT comes out way too small
• Simple solution in the Karn/Partridge algorithm
– Only collect samples for segments sent one single time
101

102. Yet Another Limitation…
• Doesn’t consider variance in the RTT
– If variance is small, the EstimatedRTT is pretty accurate
– … but, if variance is large, the estimate isn’t all that good
• Better to directly consider the variance
– Consider difference: SampleRTT – EstimatedRTT
– Boost the estimate based on the difference
• Jacobson/Karels algorithm
– See Section 5.2 of the Peterson/Davie book for details
102

103. TCP Sliding Window
103

104. Motivation for Sliding Window
•
•
104
Stop-and-wait is inefficient
– Only one TCP segment is “in flight” at a time
– Especially bad when delay-bandwidth product is high
Numerical example
– 1.5 Mbps link with a 45 msec round-trip time (RTT)
• Delay-bandwidth product is 67.5 Kbits (or 8 KBytes)
– But, sender can send at most one packet per RTT
• Assuming a segment size of 1 KB (8 Kbits)
• … leads to 8 Kbits/segment / 45 msec/segment  182 Kbps
• That’s just one-eighth of the 1.5 Mbps link capacity

105. Sliding Window
• Allow a larger a ou t of data i flight
– Allow sender to get ahead of the receiver
– … though ot too far ahead
105

106. Receiver Buffering
• Window size
– Amount that can be sent without acknowledgment
– Receiver needs to be able to store this amount of data
• Receiver advertises the window to the receiver
– Tells the receiver the amount of free space left
– … and the sender agrees not to exceed this amount
Window Size
Data ACK’d
106
Outstanding
Un-ack’d data
Data OK
to send
Data not OK
to send yet

107. TCP Header for Receiver Buffering
Source port
Destination port
Sequence number
Flags: SYN
FIN
RST
PSH
URG
ACK
Acknowledgment
HdrLen
0
Flags
Advertised window
Checksum
Urgent pointer
Options (variable)
Data
107

108. Fast Retransmission
108

109. Timeout is Inefficient
• Timeout-based retransmission
– Sender transmits a packet and waits until timer expires
– … and then retransmits from the lost packet onward
109

110. Fast Retransmission
• Better solution possible under sliding window
– Although packet n might have been lost
– … packets n+1, n+2, and so on might get through
• Idea: have the receiver send ACK packets
– ACK says that receiver is still awaiting nth packet
• And repeated ACKs suggest later packets have arrived
– Sender can view the “duplicate ACKs” as an early hint
• … that the nth packet must have been lost
• … and perform the retransmission early
• Fast retransmission
– Sender retransmits data after the triple duplicate ACK
110

111. Effectiveness of Fast Retransmit
• When does Fast Retransmit work best?
– Long data transfers
• High likelihood of many packets in flight
– High window size
• High likelihood of many packets in flight
– Low burstiness in packet losses
• Higher likelihood that later packets arrive successfully
• Implications for Web traffic
– Most Web transfers are short (e.g., 10 packets)
• Short HTML files or small images
– So, often there aren’t many packets in flight
– … making fast retransmit less likely to “kick in”
– Forcing users to like “reload” more often… 
111

112. Tearing Down the Connection
112

113. Tearing Down the Connection
B
A
time
• Closing the connection
– Finish (FIN) to close and receive remaining bytes
– And other host sends a FIN ACK to acknowledge
– Reset (RST) to close and not receive remaining bytes
113

114. Sending/Receiving the FIN Packet
• Sending a FIN: close()
– Process is done sending
data via the socket
– Process invokes
“close()” to close the
socket
– Once TCP has sent all of
the outstanding bytes…
– … then TCP sends a FIN
114
• Receiving a FIN: EOF
– Process is reading data
from the socket
– Eventually, the attempt
to read returns an EOF

115. Suggested improvement for
IP and TCP
2.115

116. •
•
•
•
•
•
The TCP/IP data path has improved pathlength and scalability, and it provides
virtual storage constraint relief. Communications Server does the following:
Reduces extended common storage area (ECSA) consumption for TCP/IP
workloads
Communications Server housed portions of inbound datagrams in ECSA, and in
certain circumstances, system outages caused by ECSA usage spikes could occur.
Communications Server does not use ECSA to hold inbound IP traffic.
Reduces system pathlength for the TCP/IP data path. This results in more efficient
TCP/IP communications (potentially lower utilization of the LPAR), and can lead to
improved network response time if the z/OS image is currently MIPs-constrained.
Improves scalability.
The UDP layer is enhanced to enable more efficient processing of incoming
datagrams when an application has multiple threads concurrently reading datagrams
from the same datagram socket. With this enhancement, the UDP layer now wakes
up only a single thread to process an incoming datagram, which reduces overhead
by avoiding the unnecessary resumption and suspension of multiple threads for
every incoming datagram.

117. Significance of UDP in
modern communication
2.117

118. •
•
•
•
In situations where your really want to get a simple answer to another server
quickly, UDP works best. In general, you want the answer to be in one response
packet, and you are prepared to implement your own protocol for reliability or
resends. DNS is the perfect description of this use case. The costs of connection
setups are way to high (yet, DNS does support a TCP mode as well).
Another case is when you are delivering data that can be lost because newer data
coming in will replace that previous data/state. Weather data, video streaming, a
stock quotation service (not used for actual trading), or gaming data come to mind.
Another case is when you are managing a tremendous amount of state and you want
to avoid using TCP because the OS cannot handle that many sessions. This is a rare
case today. In fact, there are now user-land TCP stacks that can be used so that the
application writer may have finer grained control over the resources needed for that
TCP state. Prior to 2003, UDP was really the only game in town.
One other case is for multicast traffic. UDP can be multicasted to multiple hosts
whereas TCP cannot do this at all.

119. Telecommunications
• Tele (Far) + Communications
• Early telecommunications
– smoke signals and drums
– visual telegraphy (or semaphore in 1792)
• Telegraph and telephone
– Telegraph (1839)
– Telephone (1876)
• Radio and television
• Telephony
– Voice and Data

120. Communications and Networks
• Data Communications
– Transmission of signals
• Encoding, interfacing, signal integrity, multiplexing
etc.
• Networking
– Topology & architecture used to interconnect devices
• Networks of communication systems

121. Network Trends (1980-Present)
Voice, Image, Data, Video
Microcontroller
Microcontroller
Networking
Wireless
Integrated Systems!

122. Communication Systems
•
•
•
Process describing transfer of information, data, instructions between one or more
systems through some media
– Examples
• people, computers, cell phones, etc.
• Computer communication systems
Signals passing through the communication channel can be Digital, or analog
– Analog signals: continuous electrical waves
– Digital signals: individual electrical pulses (bits)
Receivers and transmitters: desktop computers, mainframe computers, etc.
Communication channel
Communication
media
R R
R X X
X
T
X
Amp/Adaptor

123. Communication Systems

124. Communications Components
• Basic components of a
communication system
– Communication technologies
– Communication devices
– Communication channels
– Communication software

125. A Communications Model

126. Communications Tasks
Transmission system utilization
Addressing
Interfacing
Routing
Signal generation
Recovery
Synchronization
Message formatting
Exchange management
Security
Error detection and correction
Network management
Flow control

127. Data Communications Model

128. Communication Technology Applications
voice mail
instant
messaging
e-mail
newsgroups
collaboration
Twitter
telephony
groupware
chat rooms
videoconferencing
global positioning
system (GPS)

129. Communication Technologies - Applications
• Different technologies allowing us to communicate
– Examples: Voice mail, fax, email, instant message, chat rooms,
news groups, telephony, GPS, and more
• Voice mail: Similar to answering machine but digitized
• Fax: Sending hardcopy of text or photographs between computers
using fax modem
• Email: electronic mail – sending text, files, images between different
computer networks - must have email software
– More than 1.3 billion people send 244 billion messages monthly!
• Chat rooms: Allows communications in real time when connected to
the Internet

130. Communication Technologies – Applications
(cont)
• Telephony: Talking to other people over the Internet (also called VoIP)
– Sends digitized audio signals over the Internet
– Requires Internet telephone software
• Groupware: Software application allowing a group of people to
communicate with each other (exchange data)
– Address book, appointment book, schedules, etc.
• GPS: consists of receivers connected to satellite systems
– Determining the geographical location of the receiver
– Used for cars, advertising, hiking, tracking, etc.

131. Communication Devices
• Any type of hardware capable of transmitting data,
instructions, and information between devices
– Functioning as receiver, transmitter, adaptor, converter
– Basic characteristics: How fast, how far, how much data!
• Examples: Dial-up modems, ISDN, DSL modems, network
interface cards

132. Communication Devices(Cont)
– Dial-up modem: uses standard phone lines
• Converts digital information into analog
• Consists of a modulator and a demodulator
• Can be external, internal, wireless
– ISDN and DSL Modem: Allows digital communication between networks and
computers
• Requires a digital modem
• Digital is better than analog – why?
– Cable modem: a modem that transmits and receives data over the cable
television (CATV) network
• Also called broadband modem (carrying multiple signals)
• The incoming signal is split
• Requires a cable modem
– Network interface cards: Adaptor cards residing in the computer to transmit
and receiver data over the network (NIC)
• Operate with different network technologies (e.g., Ethernet)

133. Communication Software
•
Examples of applications (Layer 7) take advantage of the transport (Layer 4)
services of TCP and UDP
– Hypertext Transfer Protocol (HTTP): A client/server application that
uses TCP for transport to retrieve HTML pages.
– Domain Name Service (DNS): A name-to-address translation application
that uses both TCP and UDP transport.
– Telnet: A virtual terminal application that uses TCP for transport.
– File Transport Protocol (FTP): A file transfer application that uses TCP
for transport.
– Trivial File Transfer Protocol (TFTP): A file transfer application that
uses UDP for transport.
– Network Time Protocol (NTP): An application that synchronizes time
with a time source and uses UDP for transport.
– Border Gateway Protocol (BGP): An exterior gateway routing protocol
that uses TCP for transport. BGP is used to exchange routing information
for the Internet and is the protocol used between service providers.

134. Communication Channels
• A channel is a path between two communication devices
• Channel capacity: How much data can be passed through the channel
(bit/sec)
– Also called channel bandwidth
– The smaller the pipe the slower data transfer!
• Consists of one or more transmission media
– Materials carrying the signal
– Two types:
• Physical: wire cable
T1
T1
lines
• Wireless: Air
destinatio
lines
n network
server
T3
lines
T1
lines

135. Physical Transmission Media
• A tangible media
– Examples: Twisted-pair cable, coaxial cable, Fiber-optics, etc.
• Twisted-pair cable:
– One or more twisted wires bundled together (why?)
– Made of copper
• Coax-Cable:
– Consists of single copper wire surrounded by three layers of insulating
and metal materials
– Typically used for cable TV
• Fiber-optics:
– Strands of glass or plastic used to transmit light
– Very high capacity, low noise, small size, less suitable to natural
disturbances

136. Physical Transmission Media
twisted-pair cable
woven or
braided
metal
plastic outer
coating
copper wire
insulatin
g
material
optical fiber
core
glass cladding
protective
coating
twisted-pair wire

137. Wireless Transmission Media
• Broadcast Radio
– Distribute signals through the air over long
distance
– Uses an antenna
– Typically for stationary locations
– Can be short range
• Cellular Radio
– A form of broadcast radio used for mobile
communication
– High frequency radio waves to transmit voice or
data
– Utilizes frequency-reuse

138. Wireless Transmission Media
• Microwaves
– Radio waves providing high speed
transmission
– They are point-to-point (can’t be
obstructed)
– Used for satellite communication
• Infrared (IR)
– Wireless transmission media that sends
signals using infrared light- waves - Such
as?

139. Physical Transmission Media
Wireless channel capacity:
100 Mbps is how many bits per sec?
Which is bigger:
10,000 Mbps, 0.01Tbps or 10Gbps?

140. Networks
•
•
•
•
•
Collection of computers and devices connected together
Used to transfer information or files, share resources, etc.
What is the largest network?
Characterized based on their geographical coverage, speed, capacities
Networks are categorized based on the following characteristics:
– Network coverage: LAN, MAN, WAN
– Network topologies: how the computers are connected together
– Network technologies
– Network architecture

141. Network coverage
•
•
•
Local Area Networks:
– Used for small networks (school, home, office)
– Examples and configurations:
• Wireless LAN or Switched LAN
• ATM LAN, Frame Ethernet LAN
• Peer-2-PEER: connecting several computers together (<10)
• Client/Server: The serves shares its resources between different clients
Metropolitan Area Network
– Backbone network connecting all LANs
– Can cover a city or the entire country
Wide Area Network
– Typically between cities and countries
– Technology:
• Circuit Switch, Packet Switch, Frame Relay, ATM
– Examples:
• Internet P2P: Networks with the same network software can be
connected together (Napster)

142. LAN v.s WAN
LAN - Local Area Network a group of
computers connected within a building or a
campus (Example of LAN may consist of
computers located on a single floor or a
building or it might link all the computers in a
small company.
WAN - A network consisting of
computers of LAN's connected
across a distance WAN can cover
small to large distances, using
different topologies such as
telephone lines, fiber optic cabling,
satellite transmissions and
microwave transmissions.

143. Network Topologies
• Configuration or physical arrangement in which devices are connected
together
• BUS networks: Single central cable connected a number of devices
– Easy and cheap
– Popular for LANs
• RING networks: a number of computers are connected on a closed loop
– Covers large distances
– Primarily used for LANs and WANs
• STAR networks: connecting all devices to a central unit
– All computers are connected to a central device called hub
– All data must pass through the hub
– What is the problem with this?
– Susceptible to failure

144. Network Topologies
personal
computer
personal
computer
personal
computer
personal
computer
personal
computer
personal computer
personal computer
personal computer
personal computer
host
compute
r
printer
file server

145. Network Architecture
•
•
Refers to how the computer or devices are designed in a network
Basic types:
– Centralized – using mainframes
– Peer-2-Peer:
• Each computer (peer) has equal responsibilities, capacities, sharing hardware,
data, with the other computers on the peer-to-peer network
• Good for small businesses and home networks
• Simple and inexpensive
– Client/Server:
• All clients must request service from the server
• The server is also called a host
• Different servers perform different tasks: File server, network server, etc.
client
laser
printer
client
serve
r
client

146. P2P vs Client-Server
Peers make a portion of their resources, such
as processing power, disk storage or network
bandwidth, directly available to other
network participants, without the need for
central coordination by servers or stable hosts
Peer-to-Peer
Examples

147. (Data) Network Technologies
• Vary depending on the type of devices we use for interconnecting
computers and devices together
• Ethernet:
– LAN technology allowing computers to access the network
– Susceptible to collision
– Can be based on BUS or STAR topologies
– Operates at 10Mbps or 100Mbps, (10/100)
– Fast Ethernet operates at 100 Mbps /
– Gigabit Ethernet (1998 IEEE 802.3z)
– 10-Gigabit Ethernet (10GE or 10GbE or 10 GigE)
• 10GBASE-R/LR/SR (long range short range, etc.)
• Physical layer
– Gigabit Ethernet using optical fiber, twisted pair cable, or balanced
copper cable
Project
Topic

148. (Data) Network Technologies
• Token Ring
– LAN technology
– Only the computer with the token can transmit
– No collision
– Typically 72-260 devices can be connected together
• TCP/IP and UDP
– Uses packet transmission
• 802.11
– Standard for wireless LAN
– Wi-Fi (wireless fidelity) is used to describe that the device is in
802.11 family or standards
– Typically used for long range (300-1000 feet)
– Variations include: .11 (1-2 Mbps); .11a (up to 54 Mbps); .11b (up
to 11 Mbps); .11g (54 Mbps and higher
Project
Topic

149. (Data) Network Technologies
• 802.11n
– Next generation wireless LAN technology
– Improving network throughput (600 Mbps compared to 450 Mbps) –
thus potentially supporting a user throughput of 110 Mbit/s
• WiMAX
– Worldwide Interoperability for Microwave Access
– Provides wireless transmission of data from point-to-multipoint links to
portable and fully mobile internet access (up to 3 Mbit/s)
– The intent is to deliver the last mile wireless broadband access as an
alternative to cable and DSL
– Based on the IEEE 802.16(d/e) standard (also called Broadband
Wireless Access)
http://www.broadcom.com/collateral/wp/802_11n-WP100-R.pdf
Project
Topic

150. Network Technologies
• Personal area network (PAN)
– A low range computer network
– PANs can be used for communication among
the
personal devices themselves
– Wired with computer buses such as USB
and FireWire.
• Wireless personal area network (WPAN)
– Uses network technologies such as IrDA, Bluetooth, UWB, Z-Wave
and ZigBee
• Internet Mobile Protocols
– Supporting multimedia Internet traffic
– IGMP & MBONE for multicasting
– RTP, RTCP, & RSVP (used to handle multimedia on the Internet)
• VoIP
RTP: Real-time Transport Protocol
Project
Topic

151. Network Technologies
•
•
•
•
•
Zigbee
– High level communication protocols using small, low-power digital radios based on the
IEEE 802.15.4
– Wireless mesh networking proprietary standard
Bluetooth
– Uses radio frequency
– Typically used for close distances (short range- 33 feet or so)
– Transmits at 1Mbps
– Used for handheld computers to communicate with the desktop
IrDA
– Infrared (IR) light waves
– Transfers at a rate of 115 Kbps to 4 Mbps
– Requires light-of-sight transmission
RFID
– Radio frequency identification
– Uses tags which are places in items
– Example: merchandises, toll-tags, courtesy calls, sensors!
WAP
– Wireless application protocol
– Data rate of 9.6-153 kbps depending on the service type
– Used for smart phones and PDAs to access the Internet (email, web, etc)
Project
Topic

152. Network Examples
•
•
•
•
IEEE 802.15.4
– Low-rate wireless personal area networks (LR-WPANs)
– Bases for e ZigBee, WirelessHART, and MiWi specification
– Also used for 6LoWPAN and standard Internet protocols to build a Wireless
Embedded Internet (WEI)
Intranets
– Used for private networks
– May implement a firewall
• Hardware and software that restricts access to data and information on a
network
Home networks
– Ethernet
– Phone line
– HomeRF (radio frequency- waves)
– Intelligent home network
Vehicle-to-Vehicle (car2Car) - http://www.car-to-car.org/
– A wireless LAN based communication system to guarantee European-wide
inter-vehicle operability
Car2Car Technology: http://www.youtube.com/watch?v=8tFUsN3ZgR4
Project
Topic

153. Network Examples
• Interplanetary (Internet) Network
http://www.ece.gatech.edu/research/labs/bwn/deepspace/
Project
Topic

154. Network Example: Telephone Networks
•
•
•
•
•
•
•
•
•
Called the Public Switched Telephone Network (PSTN)
World-wide and voice oriented (handles voice and data)
Data/voice can be transferred within the PSTN using different technologies (data transfer rate
bps)
Dial-up lines:
– Analog signals passing through telephone lines
– Requires modems (56 kbps transfer rate)
Switching Technologies:
ISDN lines:
Technologies:
– Integrated Services Digital Network
•Circuit Switching
– Digital transmission over the telephone lines
•Packet Switching
– Can carry (multiplex) several signals on a single line
•Message Switching
DSL
•Burst Switching
– Digital subscribe line
– ADSL (asymmetric DSL)
• receiver operated at 8.4 Mbps, transmit at 640 kbps
T-Carrier lines: carries several signals over a single line: T1,T3
Frame Relay
ATM:
– Asynchronous Transfer Mode
– Fast and high capacity transmitting technology
– Packet technology
Project
Topic

155. Network Example: Optical Networks
• Fiber-to-the-x
– Broadband network architecture that
uses optical fiber to replace copper
– Used for last mile
telecommunications
– Examples: Fiber-to-the-home
(FTTH); Fiber-to-the-building
(FTTB); Fiber-to-the premises
(FTTP)
• Fiber Distribution Network (reaching
different customers)
– Active optical networks (AONs)
– Passive optical networks (PONs)
Project
Topic

156. Network Example
• Smart Grid
– Delivering electricity from suppliers
to consumers using digital
technology to save energy
• Storage Area Networks
• Computational Grid Networks
http://rekuwait.wordpress.com/2009/06/18/smart-electric-grid/
Project
Topic

157. Network Example:
Telephone Networks

158. Network Examples

159. Network Examples
Public Telephone
Network
T-Carrier
ATM
Dedicated
Lines
DSL
What about Cable Internet Services?
Dail-up
ISDN

160. solutions
2.160

161. Cluster-based Storage Systems
Ethernet: 1-10Gbps
Client
Commodity
Ethernet
Switch
Round Trip Time (RTT):
100-10us
Servers

162. Cluster-based Storage Systems
Synchronized Read
1
R
R
R
R
2
3
Client
1
Switch
2
3
4
4
Client now sends
next batch of requests
Storage
Servers
Data Block
Server
Request Unit
(SRU)

163. Synchronized Read Setup
• Test on an Ethernet-based storage cluster
• Client performs synchronized reads
• Increase # of servers involved in transfer
– Data block size is fixed (FS read)
• TCP used as the data transfer protocol

164. TCP Throughput Collapse
Cluster Setup
Collapse!
1Gbps Ethernet
Unmodified TCP
S50 Switch
1MB Block Size
• TCP Incast
• Cause of throughput collapse:
coarse-grained TCP timeouts

165. Solution: µsecond TCP + no minRTO
Throughput
(Mbps)
Our solution
Unmodified TCP
more servers 
High throughput for up to 47 servers
Simulation scales to thousands of servers

166. Overview
• Problem: Coarse-grained TCP timeouts (200ms) too expensive
for datacenter applications
• Solution: microsecond granularity timeouts
– Improves datacenter app throughput & latency
– Also safe for use in the wide-area (Internet)

167. Outline
• Overview
• Why are TCP timeouts expensive?
• How do coarse-grained timeouts affect apps?
• Solution: Microsecond TCP Retransmissions
• Is the solution safe?

168. TCP: data-driven loss recovery
Seq #
1
2
3
Ack 1
4
Ack 1
5
Ack 1
Ack 1
3 duplicate ACKs for 1
(packet 2 is probably lost)
Retransmit packet 2
immediately
In datacenters
data-driven recovery
in µsecs after loss.
2
Ack 5
Sender
Receiver

169. TCP: timeout-driven loss recovery
Seq #
1
2
3
Timeouts are expensive (msecs
to recover after loss)
4
5
Retransmission
Timeout
(RTO)
Retransmit packet
1
Ack 1
Sender
Receiver

170. TCP: Loss recovery comparison
Timeout driven recovery is
slow (ms)
Data-driven recovery is
super fast (µs) in datacenters
Seq #
1
2
3
4
5
Seq #
1
2
3
4
5
Retransmission
Timeout
(RTO)
1
Sender
Retransmit
2
Sender
Ack 1
Receiver
Ack 1
Ack 1
Ack 1
Ack 1
Ack 5
Receiver

171. RTO Estimation and Minimum Bound
• Jacobson’s TCP RTO Estimator
– RTOEstimated = SRTT + (4 * RTTVAR)
• Actual RTO = max(minRTO, RTOEstimated)
• Minimum RTO bound (minRTO) = 200ms
– TCP timer granularity
– Safety (Allman99)
– minRTO (200ms) >> Datacenter RTT (100µs)
– 1 TCP Timeout lasts 1000 datacenter RTTs!

172. Outline
• Overview
• Why are TCP timeouts expensive?
• How do coarse-grained timeouts affect apps?
• Solution: Microsecond TCP Retransmissions
• Is the solution safe?

173. Single Flow TCP Request-Response
R
Data
Data
Data
Client
Switch
Response
sent
Request
sent
Server
Response
resent
time
Response
dropped
200ms

174. Apps Sensitive to 200ms Timeouts
• Single flow request-response
– Latency-sensitive applications
• Barrier-Synchronized workloads
– Parallel Cluster File Systems
• Throughput-intensive
– Search: multi-server queries
• Latency-sensitive

175. Link Idle Time Due To Timeouts
Synchronized Read
1
R
R
R
R
2
4
Client
1
3
Switch
2
3
4
4
Req.
sent
Rsp.
sent
4 dropped
1 – 3 done
Link Idle!
Server
Request Unit
(SRU)
Response
resent
time

176. Client Link Utilization
Link Idle!
200ms

177. 200ms timeouts  Throughput Collapse
Cluster Setup
Collapse!
1Gbps Ethernet
200ms minRTO
S50 Switch
1MB Block Size
• [Nagle04] called this Incast
• Provided application level solutions
• Cause of throughput collapse: TCP timeouts
• [FAST08] Search for network level solutions to TCP Incast

178. Results from our previous work
(FAST08)
Network Level Solutions
Increase Switch Buffer Size
Results / Conclusions
 Delays throughput collapse
Throughput collapse inevitable
Expensive

179. Results from our previous work (FAST08)
Network Level Solutions
Increase Switch Buffer Size
Alternate TCP Implementations
(avoiding timeouts, aggressive datadriven recovery, disable slow start)
Results / Conclusions
 Delays throughput collapse
Throughput collapse inevitable
Expensive
Throughput collapse inevitable
because timeouts are inevitable
(complete window loss a common
case)

180. Results from our previous work (FAST08)
Network Level Solutions
Increase Switch Buffer Size
Alternate TCP Implementations
(avoiding timeouts, aggressive datadriven recovery, disable slow start)
Ethernet Flow Control
Results / Conclusions
 Delays throughput collapse
Throughput collapse inevitable
Expensive
Throughput collapse inevitable
because timeouts are inevitable
(complete window loss a common
case)
 Effective
Limited effectiveness (works for
simple topologies)
head-of-line blocking

181. Results from our previous work (FAST08)
Network Level Solutions
Increase Switch Buffer Size
Alternate TCP Implementations
(avoiding timeouts, aggressive datadriven recovery, disable slow start)
Ethernet Flow Control
Reducing minRTO (in simulation)
Results / Conclusions
 Delays throughput collapse
Throughput collapse inevitable
Expensive
Throughput collapse inevitable
because timeouts are inevitable
(complete window loss a common
case)
 Effective
Limited effectiveness (works for
simple topologies)
head-of-line blocking
 Very effective
Implementation concerns (µs timers
for OS, TCP)
Safety concerns

182. Outline
• Overview
• Why are TCP timeouts expensive?
• How do coarse-grained timeouts affect apps?
• Solution: Microsecond TCP Retransmissions
– and eliminate minRTO
• Is the solution safe?

183. µsecond Retransmission Timeouts (RTO)
RTO = max( minRTO, f(RTT) )
200ms
RTT tracked in
milliseconds
200µs?
Track RTT in µsecond
0?

184. Lowering minRTO to 1ms
• Lower minRTO to as low a value as possible without changing
timers/TCP impl.
• Simple one-line change to Linux
• Uses low-resolution 1ms kernel timers

185. Default minRTO: Throughput Collapse
Unmodified TCP
(200ms minRTO)

186. Lowering minRTO to 1ms helps
1ms minRTO
Unmodified TCP
(200ms minRTO)
Millisecond retransmissions are not enough

187. Requirements for µsecond RTO
• TCP must track RTT in microseconds
– Modify internal data structures
– Reuse timestamp option
• Efficient high-resolution kernel timers
– Use HPET for efficient interrupt signaling

188. Solution: µsecond TCP + no minRTO
microsecond TCP
+ no minRTO
1ms minRTO
more servers
• High throughput for up to 47 servers
Unmodified TCP
(200ms minRTO)

189. Simulation: Scaling to thousands
Block Size = 80MB, Buffer = 32KB, RTT = 20us

190. Synchronized Retransmissions At Scale
Simultaneous retransmissions  successive timeouts
Successive RTO = RTO * 2backoff

191. Simulation: Scaling to thousands
Desynchronize retransmissions to scale further
Successive RTO = (RTO + (rand(0.5)*RTO) ) * 2backoff
For use within datacenters only

192. • Overview
Outline
• Why are TCP timeouts expensive?
• The Incast Workload
• Solution: Microsecond TCP Retransmissions
• Is the solution safe?
– Interaction with Delayed-ACK within datacenters
– Performance in the wide-area

193. Delayed-ACK (for RTO > 40ms)
Seq #
Seq #
Seq #
1
2
1
1
2
Ack 2
Ack 0
40ms
Ack 1
Sender
Receiver
Sender
Receiver
Sender
Receiver
Delayed-Ack: Optimization to reduce #ACKs sent

194. µsecond RTO and Delayed-ACK
RTO < 40ms
RTO > 40ms
Seq #
Seq #
1
1
1
40ms
Timeout
Retransmit packet
Ack 1
Ack 1
Sender
Receiver
Sender
Receiver
Premature Timeout
RTO on sender triggers before Delayed-ACK on receiver

195. Impact of Delayed-ACK

196. Is it safe for the wide-area?
• Stability: Could we cause congestion collapse?
– No: Wide-area RTOs are in 10s, 100s of ms
– No: Timeouts result in rediscovering link capacity (slow down the rate
of transfer)
• Performance: Do we timeout unnecessarily?
– [Allman99] Reducing minRTO increases the chance of premature
timeouts
• Premature timeouts slow transfer rate
– Today: detect and recover from premature timeouts
– Wide-area experiments to determine performance impact

197. Wide-area Experiment
BitTorrent
Seeds
BitTorrent
Clients
Microsecond TCP
+
No minRTO
Standard TCP
Do microsecond timeouts harm wide-area throughput?

198. Wide-area Experiment: Results
No noticeable difference in throughput

199. Best Effort Service Model –
scheduling and policy
2.199

200. Question to the Class?
5 Mbps
A
10 Mbps
B
C
D
Cross Traffic
E
F
• Flow AD requires b/w, delay, loss guarantees
• Cross traffic is unpredictable
• Can IP provide this?
• What modifications are necessary to accomplish this?
200

201. Limitations of IP
• IP provides only best effort service
• IP does not participate in resource management
– Cannot provide service guarantees on a per flow
basis
– Cannot provide service differentiation among traffic
aggregates
• Early efforts
– Tenet group at Berkeley
– ATM
• IETF efforts
– Integrated services initiative
– Differentiated services initiative
201

202. So, what is required?
• Flow differentiation
– Simple FIFO scheduling will not work!
• Admission control
• Resource reservation
• Flow specification
202

203. Integrated Services Internet
• Enhance IP’s service model
– Old model: single best-effort service class
– New model: multiple service classes, including best-effort and
QoS classes
• Create protocols and algorithms to support new service models
– Old model: no resource management at IP level
– New model: explicit resource management at IP level
• Key architecture difference
– Old model: stateless
– New model: per flow state maintained at routers
• used for admission control and scheduling
• set up by signaling protocol
203

204. Integrated Services Network
• Flow or session as QoS
abstractions
• Each flow has a fixed or
stable path
• Routers along the path
maintain the state of the
flow
204

205. Integrated Services Example
• Achieve per-flow bandwidth and delay guarantees
– Example: guarantee 1MBps and < 100 ms delay to a flow
Receiver
Sender
205

206. Integrated Services Example
• Allocate resources - perform per-flow admission control
Receiver
Sender
206

207. Integrated Services Example
• Install per-flow state
Receiver
Sender
207

208. Integrated Services Example
• Install per flow state
Receiver
Sender
208

209. Integrated Services Example: Data Path
• Per-flow classification
Receiver
Sender
209

210. Integrated Services Example: Data Path
• Per-flow buffer management
Receiver
Sender
210

211. Integrated Services Example
• Per-flow scheduling
Receiver
Sender
211

212. How Things Fit Together
RSVP
Admission
Control
Forwarding Table
Per Flow QoS Table
Control Plane
Routing
RSVP
messages
Data Plane
Routing
Messages
Data In
Route Lookup
Classifier
Scheduler
Data Out
212

213. Service Classes
• Service can be viewed as a contract between network and
communication client
– end-to-end service
– other service scopes possible
• Three common services
– best-effort (“elastic” applications)
– hard real-time (“real-time” applications)
– soft real-time (“tolerant” applications)
213

214. Hard Real Time: Guaranteed Services
• Service contract
– network to client: guarantee a deterministic upper bound on
delay for each packet in a session
– client to network: the session does not send more than it
specifies
• Algorithm support
– admission control based on worst-case analysis
– per flow classification/scheduling at routers
214

215. Soft Real Time: Controlled Load Service
• Service contract:
– network to client: similar performance as an unloaded besteffort network
– client to network: the session does not send more than it
specifies
• Algorithm Support
– admission control based on measurement of aggregates
– scheduling for aggregate possible
215

216. Improving QOS in IP Networks
Thus far: “making the best of best effort”
Future: next generation Internet with QoS guarantees
– RSVP: signaling for resource reservations
– Differentiated Services: differential guarantees
– Integrated Services: firm guarantees
• simple model
for sharing and
congestion
studies:

217. Principles for QOS Guarantees
• Example: 1MbpsI P phone, FTP share 1.5 Mbps link.
– bursts of FTP can congest router, cause audio loss
– want to give priority to audio over FTP
Principle 1
packet marking needed for router to distinguish between different
classes; and new router policy to treat packets accordingly

218. Principles for QOS Guarantees (more)
• what if applications misbehave (audio sends higher than declared rate)
– policing: force source adherence to bandwidth allocations
• marking and policing at network edge:
– similar to ATM UNI (User Network Interface)
Principle 2
provide protection (isolation) for one class from others

219. Principles for QOS Guarantees (more)
• Allocating fixed (non-sharable) bandwidth to flow:
inefficient use of bandwidth if flows doesn’t use its
allocation
Principle 3
While providing isolation, it is desirable to use resources
as efficiently as possible

220. Principles for QOS Guarantees (more)
• Basic fact of life: can not support traffic demands beyond
link capacity
Principle 4
Call Admission: flow declares its needs, network may
block call (e.g., busy signal) if it cannot meet needs

221. Summary of QoS Principles
Let’s next look at mechanisms for achieving this ….

222. Scheduling And Policing Mechanisms
• scheduling: choose next packet to send on link; allocate link capacity and
output queue buffers to each connection (or connections aggregated into
classes)
• FIFO (first in first out) scheduling: send in order of arrival to queue
– discard policy: if packet arrives to full queue: who to discard?
• Tail drop: drop arriving packet
• priority: drop/remove on priority basis
• random: drop/remove randomly

223. Need for a Scheduling Discipline
• Why do we need a non-trivial scheduling discipline?
• Per-connection delay, bandwidth, and loss are determined by the
scheduling discipline
– The NE can allocate different mean delays to different connections by
its choice of service order
– it can allocate different bandwidths to connections by serving at least a
certain number of packets from a particular connection in a given time
interval
– Finally, it can allocate different loss rates to connections by giving them
more or fewer buffers

224. FIFO Scheduling
• Disadvantage with strict FIFO scheduling is that the scheduler
cannot differentiate among connections -- it cannot explicitly
allocate some connections lower mean delays than others
• A more sophisticated scheduling discipline can achieve this
objective (but at a cost)
• The conservation law
– “the sum of the mean queueing delays received by the set
of multiplexed connections, weighted by their fair share of
the link’s load, is independent of the scheduling discipline”

225. Requirements
• A scheduling discipline must satisfy four requirements:
– Ease of implementation -- pick a packet every few microsecs; a
scheduler that takes O(1) and not O(N) time
– Fairness and Protection (for best-effort connections) -- FIFO does
not offer any protection because a misbehaving connection can
increase the mean delay of all other connections. Round-robin
scheduling?
– Performance bounds -- deterministic or statistical; common
performance parameters: bandwidth, delay (worst-case, average),
delay-jitter, loss
– Ease and efficiency of admission control -- to decide given the
current set of connections and the descriptor for a new connection,
whether it is possible to meet the new connection’s performance
bounds without jeopardizing the performance of existing
connections

226. Schedulable Region

227. Designing a scheduling discipline
• Four principal degrees of freedom:
– the number of priority levels
– whether each level is work-conserving or non-work-conserving
– the degree of aggregation of connections within a level
– service order within a level
• Each feature comes at some cost
– for a small LAN switch -- a single priority FCFS scheduler or at most
2-priority scheduler may be sufficient
– for a heavily loaded wide-area public switch with possibly
noncooperative users, a more sophisticated scheduling discipline may
be required.

228. Work conserving and non-work conserving
disciplines
• A work-conserving scheduler is idle only when there is no packet awaiting
service
• A non-work-conserving scheduler may be idle even if it has packets to
serve
– makes the traffic arriving at downstream switches more predictable
– reduces buffer size necessary at output queues and the delay jitter
experienced by a connection
– allows the switch to send a packet only when the packet is eligible
– for example, if the (k+1)th packet on connection A becomes eligible for
service only i seconds after the service of the kth packet, the
downstream swicth receives packets on A no faster than one every i
secs.

229. Eligibility times
• By choosing eligibility times carefully, the output from a switch can be
mode more predictable (so that bursts won’t build up in the n/w)
• Two approaches: rate-jitter and delay-jitter
• rate-jitter: peak rate guarantee for a connection
– E(1) = A(1); E(k+1) = max(E(k) + Xmin, A(k+1)) where Xmin is the
time taken to serve a fixed-sized packet at peak rate)
• delay-jitter: at every switch, the input arrival pattern is fully reconstructed
– E(0,k) = A (0,k); E(i+1, k) = E(i,k) + D + L where D is the delay bound
at the previous switch and L is the largest possible delay on the link
between switch i and i+1

230. Pros and Cons
• Reduces delay jitter: Con -- we can remove jitter at endpoints with an
elasticity buffer; Pro--reduces buffers(expensive) at the switches
• Increases mean delay, problem?: pro--for playback applications, which
delay packets until the delay-jitter bound, increasing mean delay does not
affect the perceived performance
• Wasted bandwidth, problem?: pro--It can serve best-effort packets when
there are no eligible packets to serve
• Needs accurate source descriptors -- no rebuttal from the non-work
conserving camp

231. Priority Scheduling
transmit highest priority queued packet
• multiple classes, with different priorities
– class may depend on marking or other header info, e.g. IP
source/dest, port numbers, etc..

232. Priority Scheduling
• The scheduler serves a packet from priority level k only if
there are no packets awaiting service in levels k+1, k+2, …, n
• at least 3 levels of priority in an integrated services network?
• Starvation? Appropriate admission control and policing to
restrict service rates from all but the lowest priority level
• Simple implementation

233. Round Robin Scheduling
• multiple classes
• cyclically scan class queues, serving one from each class (if available)
• provides protection against misbehaving sources (also guarantees a
minimum bandwidth to every connection)

234. Max-Min Fair Share
• Fair Resource allocation to best-effort connections?
• Fair share allocates a user with a “small” demand what it wants, and evenly
distributes unused resources to the “big” users.
• Maximize the minimum share of a source whose demand is not fully
satisfied.
– Resources are allocated in order of increasing demand
– no source gets a resource share larger than its demand
– sources with unsatisfied demand s get an equal share of resource
• A Generalized Processor Sharing (GPS) server will implement max-min
fair share

235. Weighted Fair Queueing
• generalized Round Robin (offers differential service to
each connection/class)
• each class gets weighted amount of service in each cycle

236. Policing Mechanisms
Goal: limit traffic to not exceed declared parameters
Three common-used criteria:
• (Long term) Average Rate: how many pkts can be sent per unit
time (in the long run)
– crucial question: what is the interval length: 100 packets
per sec or 6000 packets per min have same average!
• Peak Rate: e.g., 6000 pkts per min. (ppm) avg.; 1500 ppm
peak rate
• (Max.) Burst Size: max. number of pkts sent consecutively
(with no intervening idle)

237. Traffic Regulators
• Leaky bucket controllers
• Token bucket controllers

238. Policing Mechanisms
Token Bucket: limit input to specified Burst Size and Average Rate.
• bucket can hold b tokens
• tokens generated at rate r token/sec unless bucket full
• over interval of length t: number of packets admitted less than or equal to
(r t + b).

239. Policing Mechanisms (more)
• token bucket, WFQ combine to provide guaranteed upper bound on
delay, i.e., QoS guarantee!
arriving
token rate, r
traffic
bucket size, b
per-flow
rate, R
WFQ
D = b/R
max

240. Queries