1. Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
2.2 Web and HTTP with TCP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with UDP
SMTP, POP3, IMAP 2.9 Building a Web
2.5 DNS server
2: Application Layer 1
Chapter 2: Application Layer
Our goals: learn about protocols
conceptual, by examining popular
implementation application-level
aspects of network protocols
application protocols HTTP
transport-layer FTP
service models SMTP / POP3 / IMAP
DNS
client-server
paradigm programming network
peer-to-peer applications
paradigm socket API
2: Application Layer 2
1
2. Some network apps
E-mail Internet telephone
Web Real-time video
Instant messaging conference
Remote login Massive parallel
P2P file sharing computing
Multi-user network
games
Streaming stored
video clips
2: Application Layer 3
Creating a network app
Write programs that application
transport
run on different end network
data link
systems and physical
communicate over a
network.
e.g., Web: Web server
software communicates
with browser software
little software written for
devices in network core application
application
transport
network core devices do transport
network
network
data link
not run user application data link
physical
physical
code
application on end systems
allows for rapid app
development, propagation
2: Application Layer 4
2
3. Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
2.2 Web and HTTP with TCP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with UDP
SMTP, POP3, IMAP 2.9 Building a Web
2.5 DNS server
2: Application Layer 5
Application architectures
Client-server
Peer-to-peer (P2P)
Hybrid of client-server and P2P
2: Application Layer 6
3
4. Client-server architecture
server:
always-on host
permanent IP address
server farms for scaling
clients:
communicate with
server
may be intermittently
connected
may have dynamic IP
addresses
do not communicate
directly with each other
2: Application Layer 7
Pure P2P architecture
no always-on server
arbitrary end systems
directly communicate
peers are intermittently
connected and change IP
addresses
example: Gnutella
Highly scalable but
difficult to manage
2: Application Layer 8
4
5. Hybrid of client-server and P2P
Skype
Internet telephony app
Finding address of remote party: centralized server(s)
Client-client connection is direct (not through server)
Instant messaging
Chatting between two users is P2P
Presence detection/location centralized:
• User registers its IP address with central server when it
comes online
• User contacts central server to find IP addresses of
buddies
2: Application Layer 9
Processes communicating
Process: program running Client process: process
within a host. that initiates
within same host, two communication
processes communicate Server process: process
using inter-process that waits to be
communication (defined contacted
by OS).
processes in different Note: applications with
hosts communicate by P2P architectures have
exchanging messages client processes &
server processes
2: Application Layer 10
5
6. Sockets
process sends/receives
host or host or
server server
messages to/from its
socket controlled by
app developer
socket analogous to door process process
sending process shoves socket socket
message out door TCP with TCP with
Internet
sending process relies on buffers, buffers,
variables variables
transport infrastructure
on other side of door which
brings message to socket controlled
by OS
at receiving process
API: (1) choice of transport protocol; (2) ability to fix
a few parameters (lots more on this later)
2: Application Layer 11
Addressing processes
to receive messages,
process must have
identifier
host device has
unique32-bit IP
address
Q: does IP address of
host on which process
runs suffice for
identifying the
process?
2: Application Layer 12
6
7. Addressing processes
to receive messages, identifier includes both
process must have IP address and port
identifier numbers associated with
host device has process on host.
unique32-bit IP Example port numbers:
address HTTP server: 80
Q: does IP address of Mail server: 25
host on which process to send HTTP message
runs suffice for to gaia.cs.umass.edu web
identifying the server:
process? IP address: 128.119.245.12
Answer: NO, many Port number: 80
processes can be running
more shortly…
on same host
2: Application Layer 13
App-layer protocol defines
Types of messages Public-domain protocols:
exchanged, defined in RFCs
e.g., request, response allows for
Message syntax: interoperability
what fields in messages & e.g., HTTP, SMTP
how fields are delineated
Proprietary protocols:
Message semantics
meaning of information in e.g., KaZaA
fields
Rules for when and how
processes send &
respond to messages
2: Application Layer 14
7
8. What transport service does an app need?
Data loss Bandwidth
some apps (e.g., audio) can some apps (e.g.,
tolerate some loss multimedia) require
other apps (e.g., file minimum amount of
transfer, telnet) require bandwidth to be
100% reliable data
“effective”
transfer
other apps (“elastic
Timing apps”) make use of
some apps (e.g., whatever bandwidth
Internet telephony, they get
interactive games)
require low delay to be
“effective”
2: Application Layer 15
Transport service requirements of common apps
Application Data loss Bandwidth Time Sensitive
file transfer no loss elastic no
e-mail no loss elastic no
Web documents no loss elastic no
real-time audio/video loss-tolerant audio: 5kbps-1Mbps yes, 100’s msec
video:10kbps-5Mbps
stored audio/video loss-tolerant same as above yes, few secs
interactive games loss-tolerant few kbps up yes, 100’s msec
instant messaging no loss elastic yes and no
2: Application Layer 16
8
9. Internet transport protocols services
TCP service: UDP service:
connection-oriented: setup unreliable data transfer
required between client and between sending and
server processes receiving process
reliable transport between does not provide:
sending and receiving process connection setup,
flow control: sender won’t reliability, flow control,
overwhelm receiver congestion control, timing,
or bandwidth guarantee
congestion control: throttle
sender when network
overloaded Q: why bother? Why is
does not provide: timing, there a UDP?
minimum bandwidth
guarantees
2: Application Layer 17
Internet apps: application, transport protocols
Application Underlying
Application layer protocol transport protocol
e-mail SMTP [RFC 2821] TCP
remote terminal access Telnet [RFC 854] TCP
Web HTTP [RFC 2616] TCP
file transfer FTP [RFC 959] TCP
streaming multimedia proprietary TCP or UDP
(e.g. RealNetworks)
Internet telephony proprietary
(e.g., Vonage,Dialpad) typically UDP
2: Application Layer 18
9
10. Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
app architectures with TCP
app requirements
2.8 Socket programming
2.2 Web and HTTP with UDP
2.4 Electronic Mail 2.9 Building a Web
SMTP, POP3, IMAP server
2.5 DNS
2: Application Layer 19
Web and HTTP
First some jargon
Web page consists of objects
Object can be HTML file, JPEG image, Java
applet, audio file,…
Web page consists of base HTML-file which
includes several referenced objects
Each object is addressable by a URL
Example URL:
www.someschool.edu/someDept/pic.gif
host name path name
2: Application Layer 20
10
11. HTTP overview
HTTP: hypertext
transfer protocol HT
PrT
eq u
Web’s application layer PC running HT est
TP
protocol Explorer r es
pon
se
client/server model
client: browser that st
ue
requests, receives, eq
T Pr nse Server
“displays” Web objects HT spo running
P re
server: Web server HTT Apache Web
server
sends objects in
response to requests
Mac running
HTTP 1.0: RFC 1945 Navigator
HTTP 1.1: RFC 2068
2: Application Layer 21
HTTP overview (continued)
Uses TCP: HTTP is “stateless”
client initiates TCP server maintains no
connection (creates socket) information about
to server, port 80 past client requests
server accepts TCP
connection from client aside
Protocols that maintain
HTTP messages (application- “state” are complex!
layer protocol messages) past history (state) must
exchanged between browser be maintained
(HTTP client) and Web
if server/client crashes,
server (HTTP server)
their views of “state” may
TCP connection closed be inconsistent, must be
reconciled
2: Application Layer 22
11
12. HTTP connections
Nonpersistent HTTP Persistent HTTP
At most one object is Multiple objects can
sent over a TCP be sent over single
connection. TCP connection
HTTP/1.0 uses between client and
nonpersistent HTTP server.
HTTP/1.1 uses
persistent connections
in default mode
2: Application Layer 23
Nonpersistent HTTP
(contains text,
Suppose user enters URL references to 10
www.someSchool.edu/someDepartment/home.index jpeg images)
1a. HTTP client initiates TCP
connection to HTTP server
1b. HTTP server at host
(process) at
www.someSchool.edu waiting
www.someSchool.edu on port 80
for TCP connection at port 80.
“accepts” connection, notifying
client
2. HTTP client sends HTTP
request message (containing
URL) into TCP connection 3. HTTP server receives request
socket. Message indicates message, forms response
that client wants object message containing requested
someDepartment/home.index object, and sends message
into its socket
time
2: Application Layer 24
12
13. Nonpersistent HTTP (cont.)
4. HTTP server closes TCP
connection.
5. HTTP client receives response
message containing html file,
displays html. Parsing html
file, finds 10 referenced jpeg
objects
time 6. Steps 1-5 repeated for each
of 10 jpeg objects
2: Application Layer 25
Non-Persistent HTTP: Response time
Definition of RRT: time to
send a small packet to
travel from client to
server and back. initiate TCP
connection
Response time: RTT
one RTT to initiate TCP request
connection file
time to
RTT
one RTT for HTTP transmit
file
request and first few file
bytes of HTTP response
received
to return time time
file transmission time
total = 2RTT+transmit time
2: Application Layer 26
13
14. Persistent HTTP
Nonpersistent HTTP issues: Persistent without pipelining:
requires 2 RTTs per object client issues new request
OS overhead for each TCP only when previous
connection response has been received
browsers often open parallel one RTT for each
TCP connections to fetch referenced object
referenced objects Persistent with pipelining:
Persistent HTTP default in HTTP/1.1
server leaves connection client sends requests as
open after sending response soon as it encounters a
subsequent HTTP messages referenced object
between same client/server as little as one RTT for all
sent over open connection the referenced objects
2: Application Layer 27
HTTP request message
two types of HTTP messages: request, response
HTTP request message:
ASCII (human-readable format)
request line
(GET, POST, GET /somedir/page.html HTTP/1.1
HEAD commands) Host: www.someschool.edu
User-agent: Mozilla/4.0
header Connection: close
lines Accept-language:fr
Carriage return,
line feed (extra carriage return, line feed)
indicates end
of message
2: Application Layer 28
14
15. HTTP request message: general format
2: Application Layer 29
Uploading form input
Post method:
Web page often
includes form input URL method:
Input is uploaded to Uses GET method
server in entity body Input is uploaded in
URL field of request
line:
www.somesite.com/animalsearch?monkeys&banana
2: Application Layer 30
15
16. Method types
HTTP/1.0 HTTP/1.1
GET GET, POST, HEAD
POST PUT
HEAD uploads file in entity
body to path specified
asks server to leave
in URL field
requested object out of
response DELETE
deletes file specified in
the URL field
2: Application Layer 31
HTTP response message
status line
(protocol
status code HTTP/1.1 200 OK
status phrase) Connection close
Date: Thu, 06 Aug 1998 12:00:15 GMT
Server: Apache/1.3.0 (Unix)
header
Last-Modified: Mon, 22 Jun 1998 …...
lines
Content-Length: 6821
Content-Type: text/html
data, e.g., data data data data data ...
requested
HTML file
2: Application Layer 32
16
17. HTTP response status codes
In first line in server->client response message.
A few sample codes:
200 OK
request succeeded, requested object later in this message
301 Moved Permanently
requested object moved, new location specified later in
this message (Location:)
400 Bad Request
request message not understood by server
404 Not Found
requested document not found on this server
505 HTTP Version Not Supported
2: Application Layer 33
Trying out HTTP (client side) for yourself
1. Telnet to your favorite Web server:
telnet cis.poly.edu 80 Opens TCP connection to port 80
(default HTTP server port) at cis.poly.edu.
Anything typed in sent
to port 80 at cis.poly.edu
2. Type in a GET HTTP request:
GET /~ross/ HTTP/1.1 By typing this in (hit carriage
Host: cis.poly.edu return twice), you send
this minimal (but complete)
GET request to HTTP server
3. Look at response message sent by HTTP server!
2: Application Layer 34
17
18. Let’s look at HTTP in action
telnet example
Ethereal example
2: Application Layer 35
User-server state: cookies
Many major Web sites Example:
use cookies Susan access Internet
Four components: always from same PC
She visits a specific e-
1) cookie header line of
commerce site for first
HTTP response message
time
2) cookie header line in
When initial HTTP
HTTP request message
requests arrives at site,
3) cookie file kept on site creates a unique ID
user’s host, managed by and creates an entry in
user’s browser backend database for
4) back-end database at ID
Web site
2: Application Layer 36
18
19. Cookies: keeping “state” (cont.)
client server
Cookie file usual http request msg server n e
da try i
usual http response + creates ID tab n b
as ac
e ke
ebay: 8734 Set-cookie: 1678 1678 for user nd
Cookie file
usual http request msg
amazon: 1678 cookie: 1678 cookie-
ss
ebay: 8734 specific acce
usual http response msg action
one week later:
s s
ce
ac
usual http request msg
Cookie file cookie-
cookie: 1678
amazon: 1678 spectific
ebay: 8734 usual http response msg action
2: Application Layer 37
Cookies (continued)
aside
What cookies can bring: Cookies and privacy:
authorization cookies permit sites to
shopping carts learn a lot about you
recommendations you may supply name
and e-mail to sites
user session state
(Web e-mail) search engines use
redirection & cookies
to learn yet more
advertising companies
obtain info across
sites
2: Application Layer 38
19
20. Web caches (proxy server)
Goal: satisfy client request without involving origin server
user sets browser: Web origin
accesses via cache server
browser sends all HTTP HT Proxy
requests to cache T Pr server es t
eq u e qu
clientHTTP est T Pr se
object in cache: cache HT pon
r es
pon r es
returns object se TP
HT
st
else cache requests ue
req n se
object from origin TP po
HT r es
server, then returns TP
object to client HT
client
origin
server
2: Application Layer 39
More about Web caching
Cache acts as both client Why Web caching?
and server Reduce response time for
Typically cache is installed client request.
by ISP (university, Reduce traffic on an
company, residential ISP) institution’s access link.
Internet dense with caches
enables “poor” content
providers to effectively
deliver content (but so
does P2P file sharing)
2: Application Layer 40
20
21. Caching example
origin
Assumptions
servers
average object size = 100,000
bits public
Internet
avg. request rate from
institution’s browsers to origin
servers = 15/sec
1.5 Mbps
delay from institutional router
access link
to any origin server and back
to router = 2 sec institutional
network
10 Mbps LAN
Consequences
utilization on LAN = 15%
utilization on access link = 100%
total delay = Internet delay + institutional
access delay + LAN delay cache
= 2 sec + minutes + milliseconds
2: Application Layer 41
Caching example (cont)
origin
Possible solution
servers
increase bandwidth of access
link to, say, 10 Mbps public
Internet
Consequences
utilization on LAN = 15%
utilization on access link = 15%
10 Mbps
Total delay = Internet delay + access link
access delay + LAN delay
institutional
= 2 sec + msecs + msecs
network
often a costly upgrade 10 Mbps LAN
institutional
cache
2: Application Layer 42
21
22. Caching example (cont)
origin
Install cache servers
suppose hit rate is .4 public
Consequence Internet
40% requests will be
satisfied almost immediately
60% requests satisfied by
origin server 1.5 Mbps
access link
utilization of access link
reduced to 60%, resulting in institutional
negligible delays (say 10 network
10 Mbps LAN
msec)
total avg delay = Internet
delay + access delay + LAN
delay = .6*(2.01) secs +
.4*milliseconds < 1.4 secs institutional
cache
2: Application Layer 43
Conditional GET
Goal: don’t send object if cache server
cache has up-to-date cached HTTP request msg
version
object
If-modified-since:
cache: specify date of <date>
not
cached copy in HTTP request modified
If-modified-since: HTTP response
HTTP/1.0
<date> 304 Not Modified
server: response contains no
object if cached copy is up-
HTTP request msg
to-date: If-modified-since:
HTTP/1.0 304 Not <date> object
Modified modified
HTTP response
HTTP/1.0 200 OK
<data>
2: Application Layer 44
22
23. Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
2.2 Web and HTTP with TCP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with UDP
SMTP, POP3, IMAP 2.9 Building a Web
2.5 DNS server
2: Application Layer 45
FTP: the file transfer protocol
FTP file transfer
FTP FTP
user client server
interface
user
at host local file remote file
system system
transfer file to/from remote host
client/server model
client: side that initiates transfer (either to/from
remote)
server: remote host
ftp: RFC 959
ftp server: port 21
2: Application Layer 46
23
24. FTP: separate control, data connections
TCP control connection
FTP client contacts FTP port 21
server at port 21, specifying
TCP as transport protocol
TCP data connection
Client obtains authorization FTP port 20 FTP
over control connection client server
Client browses remote
directory by sending Server opens a second TCP
commands over control data connection to transfer
connection. another file.
When server receives a Control connection: “out of
command for a file transfer, band”
the server opens a TCP data FTP server maintains “state”:
connection to client current directory, earlier
After transferring one file, authentication
server closes connection.
2: Application Layer 47
FTP commands, responses
Sample commands: Sample return codes
sent as ASCII text over status code and phrase (as
control channel in HTTP)
USER username 331 Username OK,
PASS password password required
125 data connection
LIST return list of file in
already open;
current directory
transfer starting
RETR filename retrieves 425 Can’t open data
(gets) file connection
STOR filename stores 452 Error writing
(puts) file onto remote file
host
2: Application Layer 48
24
25. Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
2.2 Web and HTTP with TCP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with UDP
SMTP, POP3, IMAP 2.9 Building a Web
2.5 DNS server
2: Application Layer 49
Electronic Mail outgoing
message queue
user mailbox
user
Three major components: agent
user agents mail
user
server
mail servers agent
simple mail transfer SMTP mail
protocol: SMTP server user
SMTP agent
User Agent
a.k.a. “mail reader” SMTP
mail user
composing, editing, reading agent
server
mail messages
e.g., Eudora, Outlook, elm, user
Netscape Messenger agent
user
outgoing, incoming messages agent
stored on server
2: Application Layer 50
25
26. Electronic Mail: mail servers
user
Mail Servers agent
mailbox contains incoming mail
user
messages for user server
agent
message queue of outgoing
SMTP
(to be sent) mail messages mail
server user
SMTP protocol between mail
servers to send email SMTP agent
messages SMTP
client: sending mail mail user
agent
server server
“server”: receiving mail
user
server agent
user
agent
2: Application Layer 51
Electronic Mail: SMTP [RFC 2821]
uses TCP to reliably transfer email message from client
to server, port 25
direct transfer: sending server to receiving server
three phases of transfer
handshaking (greeting)
transfer of messages
closure
command/response interaction
commands: ASCII text
response: status code and phrase
messages must be in 7-bit ASCII
2: Application Layer 52
26
27. Scenario: Alice sends message to Bob
1) Alice uses UA to compose 4) SMTP client sends Alice’s
message and “to” message over the TCP
bob@someschool.edu connection
2) Alice’s UA sends message 5) Bob’s mail server places the
to her mail server; message message in Bob’s mailbox
placed in message queue 6) Bob invokes his user agent
3) Client side of SMTP opens to read message
TCP connection with Bob’s
mail server
1 mail
mail
server user
user server
2 agent
agent 3 6
4 5
2: Application Layer 53
Sample SMTP interaction
S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM: <alice@crepes.fr>
S: 250 alice@crepes.fr... Sender ok
C: RCPT TO: <bob@hamburger.edu>
S: 250 bob@hamburger.edu ... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection
2: Application Layer 54
27
28. Try SMTP interaction for yourself:
telnet servername 25
see 220 reply from server
enter HELO, MAIL FROM, RCPT TO, DATA, QUIT
commands
above lets you send email without using email client
(reader)
2: Application Layer 55
SMTP: final words
SMTP uses persistent Comparison with HTTP:
connections
HTTP: pull
SMTP requires message
(header & body) to be in 7- SMTP: push
bit ASCII both have ASCII
SMTP server uses command/response
CRLF.CRLF to determine interaction, status codes
end of message
HTTP: each object
encapsulated in its own
response msg
SMTP: multiple objects
sent in multipart msg
2: Application Layer 56
28
29. Mail message format
SMTP: protocol for
exchanging email msgs header
blank
RFC 822: standard for text
line
message format:
header lines, e.g.,
To:
body
From:
Subject:
different from SMTP
commands!
body
the “message”, ASCII
characters only
2: Application Layer 57
Message format: multimedia extensions
MIME: multimedia mail extension, RFC 2045, 2056
additional lines in msg header declare MIME content
type
From: alice@crepes.fr
MIME version To: bob@hamburger.edu
Subject: Picture of yummy crepe.
method used MIME-Version: 1.0
to encode data Content-Transfer-Encoding: base64
Content-Type: image/jpeg
multimedia data
type, subtype, base64 encoded data .....
parameter declaration .........................
......base64 encoded data
encoded data
2: Application Layer 58
29
30. Mail access protocols
SMTP SMTP access user
user
agent protocol agent
sender’s mail receiver’s mail
server server
SMTP: delivery/storage to receiver’s server
Mail access protocol: retrieval from server
POP: Post Office Protocol [RFC 1939]
• authorization (agent <-->server) and download
IMAP: Internet Mail Access Protocol [RFC 1730]
• more features (more complex)
• manipulation of stored msgs on server
HTTP: Hotmail , Yahoo! Mail, etc.
2: Application Layer 59
POP3 protocol S: +OK POP3 server ready
C: user bob
authorization phase S: +OK
C: pass hungry
client commands: S: +OK user successfully logged on
user: declare username
C: list
pass: password S: 1 498
server responses S: 2 912
S: .
+OK
C: retr 1
-ERR S: <message 1 contents>
transaction phase, client: S: .
C: dele 1
list: list message numbers C: retr 2
retr: retrieve message by S: <message 1 contents>
number S: .
C: dele 2
dele: delete
C: quit
quit S: +OK POP3 server signing off
2: Application Layer 60
30
31. POP3 (more) and IMAP
More about POP3 IMAP
Previous example uses Keep all messages in
“download and delete” one place: the server
mode. Allows user to
Bob cannot re-read e- organize messages in
mail if he changes folders
client IMAP keeps user state
“Download-and-keep”: across sessions:
copies of messages on names of folders and
different clients mappings between
message IDs and folder
POP3 is stateless
name
across sessions
2: Application Layer 61
Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
2.2 Web and HTTP with TCP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with UDP
SMTP, POP3, IMAP 2.9 Building a Web
2.5 DNS server
2: Application Layer 62
31
32. DNS: Domain Name System
People: many identifiers: Domain Name System:
SSN, name, passport # distributed database
Internet hosts, routers: implemented in hierarchy of
many name servers
IP address (32 bit) -
used for addressing
application-layer protocol
host, routers, name servers to
datagrams
communicate to resolve names
“name”, e.g., (address/name translation)
ww.yahoo.com - used by
note: core Internet
humans
function, implemented as
Q: map between IP application-layer protocol
addresses and name ? complexity at network’s
“edge”
2: Application Layer 63
DNS
DNS services Why not centralize DNS?
Hostname to IP single point of failure
address translation traffic volume
Host aliasing distant centralized
Canonical and alias database
names maintenance
Mail server aliasing
Load distribution doesn’t scale!
Replicated Web
servers: set of IP
addresses for one
canonical name
2: Application Layer 64
32
33. Distributed, Hierarchical Database
Root DNS Servers
com DNS servers org DNS servers edu DNS servers
pbs.org poly.edu umass.edu
yahoo.com amazon.com
DNS servers DNS serversDNS servers
DNS servers DNS servers
Client wants IP for www.amazon.com; 1st approx:
Client queries a root server to find com DNS
server
Client queries com DNS server to get amazon.com
DNS server
Client queries amazon.com DNS server to get IP
address for www.amazon.com
2: Application Layer 65
DNS: Root name servers
contacted by local name server that can not resolve name
root name server:
contacts authoritative name server if name mapping not known
gets mapping
returns mapping to local name server
a Verisign, Dulles, VA
c Cogent, Herndon, VA (also Los Angeles)
d U Maryland College Park, MD k RIPE London (also Amsterdam,
g US DoD Vienna, VA Frankfurt)
h ARL Aberdeen, MD i Autonomica, Stockholm (plus 3
j Verisign, ( 11 locations) other locations)
m WIDE Tokyo
e NASA Mt View, CA
f Internet Software C. Palo Alto,
CA (and 17 other locations)
13 root name
servers worldwide
b USC-ISI Marina del Rey, CA
l ICANN Los Angeles, CA
2: Application Layer 66
33
34. TLD and Authoritative Servers
Top-level domain (TLD) servers: responsible
for com, org, net, edu, etc, and all top-level
country domains uk, fr, ca, jp.
Network solutions maintains servers for com TLD
Educause for edu TLD
Authoritative DNS servers: organization’s
DNS servers, providing authoritative
hostname to IP mappings for organization’s
servers (e.g., Web and mail).
Can be maintained by organization or service
provider
2: Application Layer 67
Local Name Server
Does not strictly belong to hierarchy
Each ISP (residential ISP, company,
university) has one.
Also called “default name server”
When a host makes a DNS query, query is
sent to its local DNS server
Acts as a proxy, forwards query into hierarchy.
2: Application Layer 68
34
35. Example root DNS server
2
Host at cis.poly.edu 3
TLD DNS server
wants IP address for 4
gaia.cs.umass.edu
5
local DNS server
dns.poly.edu
7 6
1 8
authoritative DNS server
dns.cs.umass.edu
requesting host
cis.poly.edu
gaia.cs.umass.edu
2: Application Layer 69
Recursive queries root DNS server
recursive query:
2
puts burden of name 3
resolution on 6
7
contacted name TLD DNS server
server
heavy load?
local DNS server
iterated query: dns.poly.edu 5 4
contacted server 1 8
replies with name of
server to contact authoritative DNS server
“I don’t know this dns.cs.umass.edu
requesting host
name, but ask this cis.poly.edu
server”
gaia.cs.umass.edu
2: Application Layer 70
35
36. DNS: caching and updating records
once (any) name server learns mapping, it caches
mapping
cache entries timeout (disappear) after some
time
TLD servers typically cached in local name
servers
• Thus root name servers not often visited
update/notify mechanisms under design by IETF
RFC 2136
http://www.ietf.org/html.charters/dnsind-charter.html
2: Application Layer 71
DNS records
DNS: distributed db storing resource records (RR)
RR format: (name, value, type, ttl)
Type=A Type=CNAME
name is hostname name is alias name for some
value is IP address “canonical” (the real) name
www.ibm.com is really
Type=NS
servereast.backup2.ibm.com
name is domain (e.g.
value is canonical name
foo.com)
value is hostname of Type=MX
authoritative name
value is name of mailserver
server for this domain
associated with name
2: Application Layer 72
36
37. DNS protocol, messages
DNS protocol : query and reply messages, both with
same message format
msg header
identification: 16 bit #
for query, reply to query
uses same #
flags:
query or reply
recursion desired
recursion available
reply is authoritative
2: Application Layer 73
DNS protocol, messages
Name, type fields
for a query
RRs in response
to query
records for
authoritative servers
additional “helpful”
info that may be used
2: Application Layer 74
37
38. Inserting records into DNS
Example: just created startup “Network Utopia”
Register name networkuptopia.com at a registrar
(e.g., Network Solutions)
Need to provide registrar with names and IP addresses of
your authoritative name server (primary and secondary)
Registrar inserts two RRs into the com TLD server:
(networkutopia.com, dns1.networkutopia.com, NS)
(dns1.networkutopia.com, 212.212.212.1, A)
Put in authoritative server Type A record for
www.networkuptopia.com and Type MX record for
networkutopia.com
How do people get the IP address of your Web site?
2: Application Layer 75
Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
app architectures with TCP
app requirements
2.8 Socket programming
2.2 Web and HTTP with UDP
2.4 Electronic Mail 2.9 Building a Web
SMTP, POP3, IMAP server
2.5 DNS
2: Application Layer 76
38
39. P2P file sharing
Alice chooses one of
Example the peers, Bob.
Alice runs P2P client File is copied from
application on her Bob’s PC to Alice’s
notebook computer notebook: HTTP
Intermittently While Alice downloads,
connects to Internet; other users uploading
gets new IP address from Alice.
for each connection Alice’s peer is both a
Asks for “Hey Jude” Web client and a
transient Web server.
Application displays
other peers that have All peers are servers =
copy of Hey Jude. highly scalable!
2: Application Layer 77
P2P: centralized directory
Bob
original “Napster” design centralized
1) when peer connects, it directory server
1
informs central server: peers
IP address 1
content
1 3
2) Alice queries for “Hey
2
Jude” 1
3) Alice requests file from
Bob
Alice
2: Application Layer 78
39
40. P2P: problems with centralized directory
Single point of failure file transfer is
Performance decentralized, but
bottleneck locating content is
Copyright highly centralized
infringement
2: Application Layer 79
Query flooding: Gnutella
fully distributed overlay network: graph
no central server edge between peer X
public domain protocol and Y if there’s a TCP
many Gnutella clients connection
implementing protocol all active peers and
edges is overlay net
Edge is not a physical
link
Given peer will
typically be connected
with < 10 overlay
neighbors
2: Application Layer 80
40
41. Gnutella: protocol
File transfer:
Query message HTTP
sent over existing TCP
connections
Query
peers forward QueryHit
Query message
y
QueryHit Qu
er
it
Qu
ery
sent over eryH
Qu
reverse
path Query
QueryHit
Scalability: Qu
er
y
limited scope
flooding
2: Application Layer 81
Gnutella: Peer joining
1. Joining peer X must find some other peer in
Gnutella network: use list of candidate peers
2. X sequentially attempts to make TCP with peers
on list until connection setup with Y
3. X sends Ping message to Y; Y forwards Ping
message.
4. All peers receiving Ping message respond with
Pong message
5. X receives many Pong messages. It can then
setup additional TCP connections
Peer leaving: see homework problem!
2: Application Layer 82
41
42. Exploiting heterogeneity: KaZaA
Each peer is either a
group leader or assigned
to a group leader.
TCP connection between
peer and its group leader.
TCP connections between
some pairs of group
leaders.
Group leader tracks the
content in all its ordinary peer
children. group-leader peer
neighoring relationships
in overlay network
2: Application Layer 83
KaZaA: Querying
Each file has a hash and a descriptor
Client sends keyword query to its group
leader
Group leader responds with matches:
For each match: metadata, hash, IP address
If group leader forwards query to other
group leaders, they respond with matches
Client then selects files for downloading
HTTP requests using hash as identifier sent to
peers holding desired file
2: Application Layer 84
42
43. KaZaA tricks
Limitations on simultaneous uploads
Request queuing
Incentive priorities
Parallel downloading
For more info:
J. Liang, R. Kumar, K. Ross, “Understanding KaZaA,”
(available via cis.poly.edu/~ross)
2: Application Layer 85
Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
2.2 Web and HTTP with TCP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with UDP
SMTP, POP3, IMAP 2.9 Building a Web
2.5 DNS server
2: Application Layer 86
43
44. Socket programming
Goal: learn how to build client/server application that
communicate using sockets
Socket API socket
introduced in BSD4.1 UNIX, a host-local,
1981 application-created,
explicitly created, used, OS-controlled interface
released by apps (a “door”) into which
client/server paradigm application process can
two types of transport both send and
service via socket API: receive messages to/from
another application
unreliable datagram
process
reliable, byte stream-
oriented
2: Application Layer 87
Socket-programming using TCP
Socket: a door between application process and end-
end-transport protocol (UCP or TCP)
TCP service: reliable transfer of bytes from one
process to another
controlled by
controlled by process application
application process
developer
developer socket socket
TCP with TCP with controlled by
controlled by
buffers, operating
operating buffers, internet system
system variables variables
host or host or
server server
2: Application Layer 88
44
45. Socket programming with TCP
Client must contact server When contacted by client,
server process must first server TCP creates new
be running socket for server process to
server must have created communicate with client
socket (door) that allows server to talk with
welcomes client’s contact multiple clients
Client contacts server by: source port numbers
used to distinguish
creating client-local TCP
clients (more in Chap 3)
socket
specifying IP address, port application viewpoint
number of server process
TCP provides reliable, in-order
When client creates
transfer of bytes (“pipe”)
socket: client TCP
between client and server
establishes connection to
server TCP
2: Application Layer 89
Stream jargon
A stream is a sequence of
characters that flow into
or out of a process.
An input stream is
attached to some input
source for the process,
e.g., keyboard or socket.
An output stream is
attached to an output
source, e.g., monitor or
socket.
2: Application Layer 90
45
46. Socket programming with TCP
keyboard monitor
Example client-server app:
1) client reads line from
inFromUser
standard input (inFromUser input
stream
stream) , sends to server via Client
socket (outToServer Process
process
stream)
2) server reads line from socket
3) server converts line to
uppercase, sends back to
inFromServer
outToServer
client
output input
stream stream
4) client reads, prints modified
line from socket client TCP
clientSocket
(inFromServer stream) socket TCP
socket
to network from network
2: Application Layer 91
Client/server socket interaction: TCP
Server (running on hostid) Client
create socket,
port=x, for
incoming request:
welcomeSocket =
ServerSocket()
TCP create socket,
wait for incoming
connection request connection setup connect to hostid, port=x
connectionSocket = clientSocket =
welcomeSocket.accept() Socket()
send request using
read request from clientSocket
connectionSocket
write reply to
connectionSocket read reply from
clientSocket
close
connectionSocket close
clientSocket
2: Application Layer 92
46
47. Example: Java client (TCP)
import java.io.*;
import java.net.*;
class TCPClient {
public static void main(String argv[]) throws Exception
{
String sentence;
String modifiedSentence;
Create
input stream BufferedReader inFromUser =
new BufferedReader(new InputStreamReader(System.in));
Create
client socket, Socket clientSocket = new Socket("hostname", 6789);
connect to server
Create DataOutputStream outToServer =
output stream new DataOutputStream(clientSocket.getOutputStream());
attached to socket
2: Application Layer 93
Example: Java client (TCP), cont.
Create BufferedReader inFromServer =
input stream new BufferedReader(new
attached to socket InputStreamReader(clientSocket.getInputStream()));
sentence = inFromUser.readLine();
Send line
to server outToServer.writeBytes(sentence + 'n');
Read line modifiedSentence = inFromServer.readLine();
from server
System.out.println("FROM SERVER: " + modifiedSentence);
clientSocket.close();
}
}
2: Application Layer 94
47
48. Example: Java server (TCP)
import java.io.*;
import java.net.*;
class TCPServer {
public static void main(String argv[]) throws Exception
{
String clientSentence;
Create String capitalizedSentence;
welcoming socket
ServerSocket welcomeSocket = new ServerSocket(6789);
at port 6789
while(true) {
Wait, on welcoming
socket for contact Socket connectionSocket = welcomeSocket.accept();
by client
BufferedReader inFromClient =
Create input new BufferedReader(new
stream, attached InputStreamReader(connectionSocket.getInputStream()));
to socket
2: Application Layer 95
Example: Java server (TCP), cont
Create output
stream, attached DataOutputStream outToClient =
to socket new DataOutputStream(connectionSocket.getOutputStream());
Read in line
from socket clientSentence = inFromClient.readLine();
capitalizedSentence = clientSentence.toUpperCase() + 'n';
Write out line
outToClient.writeBytes(capitalizedSentence);
to socket
}
}
} End of while loop,
loop back and wait for
another client connection
2: Application Layer 96
48
49. Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
2.2 Web and HTTP with TCP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with UDP
SMTP, POP3, IMAP 2.9 Building a Web
2.5 DNS server
2: Application Layer 97
Socket programming with UDP
UDP: no “connection” between
client and server
no handshaking
sender explicitly attaches application viewpoint
IP address and port of
destination to each packet UDP provides unreliable transfer
of groups of bytes (“datagrams”)
server must extract IP
between client and server
address, port of sender
from received packet
UDP: transmitted data may be
received out of order, or
lost
2: Application Layer 98
49
50. Client/server socket interaction: UDP
Server (running on hostid) Client
create socket, create socket,
port=x, for clientSocket =
incoming request: DatagramSocket()
serverSocket =
DatagramSocket()
Create, address (hostid, port=x,
send datagram request
using clientSocket
read request from
serverSocket
write reply to
serverSocket
specifying client read reply from
host address, clientSocket
port number close
clientSocket
2: Application Layer 99
Example: Java client (UDP)
keyboard monitor
inFromUser
input
stream
Client
Process
Input: receives
process
packet (recall
Output: sends thatTCP received
packet (recall “byte stream”)
receivePacket
sendPacket
that TCP sent UDP
packet
UDP
packet
“byte stream”)
client UDP
clientSocket
socket UDP
socket
to network from network
2: Application Layer 100
50
51. Example: Java client (UDP)
import java.io.*;
import java.net.*;
class UDPClient {
public static void main(String args[]) throws Exception
{
Create
input stream BufferedReader inFromUser =
new BufferedReader(new InputStreamReader(System.in));
Create
client socket DatagramSocket clientSocket = new DatagramSocket();
Translate
hostname to IP InetAddress IPAddress = InetAddress.getByName("hostname");
address using DNS byte[] sendData = new byte[1024];
byte[] receiveData = new byte[1024];
String sentence = inFromUser.readLine();
sendData = sentence.getBytes();
2: Application Layer 101
Example: Java client (UDP), cont.
Create datagram
with data-to-send, DatagramPacket sendPacket =
length, IP addr, port new DatagramPacket(sendData, sendData.length, IPAddress, 9876);
Send datagram clientSocket.send(sendPacket);
to server
DatagramPacket receivePacket =
new DatagramPacket(receiveData, receiveData.length);
Read datagram
clientSocket.receive(receivePacket);
from server
String modifiedSentence =
new String(receivePacket.getData());
System.out.println("FROM SERVER:" + modifiedSentence);
clientSocket.close();
}
}
2: Application Layer 102
51
52. Example: Java server (UDP)
import java.io.*;
import java.net.*;
class UDPServer {
public static void main(String args[]) throws Exception
Create {
datagram socket
DatagramSocket serverSocket = new DatagramSocket(9876);
at port 9876
byte[] receiveData = new byte[1024];
byte[] sendData = new byte[1024];
while(true)
{
Create space for
received datagram DatagramPacket receivePacket =
new DatagramPacket(receiveData, receiveData.length);
Receive serverSocket.receive(receivePacket);
datagram
2: Application Layer 103
Example: Java server (UDP), cont
String sentence = new String(receivePacket.getData());
Get IP addr
InetAddress IPAddress = receivePacket.getAddress();
port #, of
sender int port = receivePacket.getPort();
String capitalizedSentence = sentence.toUpperCase();
sendData = capitalizedSentence.getBytes();
Create datagram
DatagramPacket sendPacket =
to send to client new DatagramPacket(sendData, sendData.length, IPAddress,
port);
Write out
datagram serverSocket.send(sendPacket);
to socket }
}
} End of while loop,
loop back and wait for
another datagram
2: Application Layer 104
52
53. Chapter 2: Application layer
2.1 Principles of 2.6 P2P file sharing
network applications 2.7 Socket programming
app architectures with TCP
app requirements
2.8 Socket programming
2.2 Web and HTTP with UDP
2.4 Electronic Mail 2.9 Building a Web
SMTP, POP3, IMAP server
2.5 DNS
2: Application Layer 105
Building a simple Web server
handles one HTTP after creating server,
request you can request file
accepts the request using a browser (e.g.,
IE explorer)
parses header
see text for details
obtains requested file
from server’s file
system
creates HTTP response
message:
header lines + file
sends response to client
2: Application Layer 106
53
54. Chapter 2: Summary
Our study of network apps now complete!
Application architectures specific protocols:
client-server HTTP
P2P FTP
hybrid SMTP, POP, IMAP
DNS
application service
requirements: socket programming
reliability, bandwidth,
delay
Internet transport
service model
connection-oriented,
reliable: TCP
unreliable, datagrams: UDP
2: Application Layer 107
Chapter 2: Summary
Most importantly: learned about protocols
typical request/reply
control vs. data msgs
message exchange:
in-band, out-of-band
client requests info or
centralized vs. decentralized
service
server responds with stateless vs. stateful
data, status code reliable vs. unreliable msg
transfer
message formats:
“complexity at network
headers: fields giving edge”
info about data
data: info being
communicated
2: Application Layer 108
54