20CS2021 DISTRIBUTED COMPUTING

20CS2021
DISTRIBUTED COMPUTING
Module – I
Dr.A.Kathirvel, Professor, Computer Science and Engg.
Karunya Institute of Technology and Sciences, CBE

20CS2021 DISTRIBUTED COMPUTING
Module I - Introduction
Introduction to Distributed systems - Examples of
distributed systems, resource sharing and the
web, challenges - System model - introduction -
architectural models - fundamental models -
Introduction to inter-process communications -
API for Internet protocol - external data.
Dr.A.Kathirvel, Professor,
DCSE, KITS
kathirvel@karunya.edu

Module I
Introduction to Distributed systems -
Examples of distributed systems, resource
sharing and the web, challenges - System
model - introduction - architectural models
- fundamental models - Introduction to
inter-process communications - API for
Internet protocol - external data.
Book Referred: Coulouris, Dollimore, Kindberg and Blair
Distributed Systems:Concepts and Design Edition 5, Addison-Wesley 2012

Agenda
Syllabus
Text Book
Why Distributed Computing?
Applications

Course Objectives: Enable the student to
1.describe the concepts of resource sharing, system modeling, inter-process
communication and file systems in distributed environment
2.apply the skill sets in name services and global states, distributed transaction
and concurrency control model, distributed shared memory and distributed
multimedia system.
3.compare the replication techniques and shared memory schemes in
distributed environment
Course Outcomes: The student will be able to
1.describe the distributed system models.
2.Classify inter-process communication mechanisms and their application
scenarios.
3.illustrate a transaction and concurrency control scheme for a real time
application.
4.compare the various file system architectures used in distributed systems
5.construct a real time distributed system with suitable IPC, event coordination,
file management, name service, transaction and concurrency control
mechanisms.
6.evaluate an efficient distributed system and its qualities

Module I - Introduction:
Introduction to Distributed systems - Examples of distributed systems, resource sharing and the
web, challenges - System model - introduction - architectural models - fundamental models -
Introduction to inter-process communications - API for Internet protocol - external data.
Module II - Distributed objects and file systems:
Introduction - Communication between distributed objects - Remote procedure call - Events and
notifications - case study - Operating system support - introduction - operating system layer -
protection - process and threads - communication and invocation - architecture - Introduction to
DFS - File service architecture - Sun network file system - Andrew file system - Enhancements
and future developments.
Module III - Name services and global states:
Introduction to Name Services- Name services and DNS - Directory services - case study -
Introduction to peer-to-peer systems - Napster and its legacy - Peer-to-peer middleware - Routing
overlays - case study - Introduction to time and global states - Clocks, Events and Process states -
Synchronizing physical clocks - Logical time and logical clocks - Global states - Distributed
debugging.
Module IV - Distributed transaction and concurrency control:
Introduction to coordination and agreement - Distributed mutual exclusion - elections - multicast
communication - consensus and related problems - Introduction to transaction and concurrency
control - Transactions - Nested transaction - Locks - Optimistic concurrency control - Timestamp
ordering - Comparison of methods for concurrency control - Introduction to distributed
transactions - Flat and nested distributed transactions - Atomic commit protocols - Concurrency
control in distributed transactions - Distributed deadlocks - Transaction recovery.

Module V - Replication, distributed shared memory: Introduction to Replication -
System model and group communications - Fault tolerant services - Highly available
services - Transactions with replicated data - Distributed shared memory - design and
implementation issues - sequential consistency and Ivy - release consistency and Munin
- other consistency models-
Module 6 - distributed multimedia system
Distributed multimedia systems - characteristics of multimedia data - quality of service
management - resource management - stream adaptation - case study.
Text Books:
1. George Coulouris, Jean Dollimore, and Tim Kindberg, “Distributed Systems
Concepts and Design”,Addison-Wesley, Fifth edition, 2011, ISBN: 0132143011.
2. Andrew S. Tanenbaum and Maartenvan Steen, “Distributed Systems: Principles and
Paradigms”,Prentice Hall, Second edition, 2006, ISBN: 0132392275.
Reference Books:
1. William Buchanan, “Distributed Systems and Networks”, McGraw-Hill, 2001.
2. Pradeep K. Sinha, “Distributed Operating Systems: Concepts and Design”, Wiley- IEEE,1996.
3. Kai Hwang, Jack Dongarra & Geoffrey C. Fox, “Distributed and Cloud Computing: Clusters,
Grids, Clouds, and the Future Internet (DCC)”, 2012,ISBN: 0262313367
4. Ajay D. Kshemkalyani, MukeshSinghal, “Distributed Computing: Principles, Algorithms, and
Systems”, Cambridge University Press, 2011, ISBN-13 : 978-0521189842

1. Distributed System Types
Fully
Distributed
Processors
Control
Fully replicated
Not fully replicated
master directory
Local data,
local directory
Master-slave
Autonomous
transaction based
Autonomous
fully cooperative
Homog.
special
purpose
Heterog.
special
purpose
Homog.
general
purpose
Heterog.
general
purpose
9

1. What is a Distributed Computing?
Definition: A distributed computing system is one in
which components located at networked computers
communicate and coordinate their actions only by
passing messages. This definition leads to the
following characteristics of distributed systems:
 Concurrency of components
 Lack of a global „clock‟
 Independent failures of components 10

Centralized System Characteristics
One component with non-autonomous parts
Component shared by users all the time
All resources accessible
Software runs in a single process
Single point of control
Single point of failure
11

Distributed System Characteristics
Multiple autonomous components
Components are not shared by all users
Resources may not be accessible
Software runs in concurrent processes on different
processors
Multiple points of control
Multiple points of failure
12

Applications
Most people know about the Internet (a
computer network) through applications
World Wide Web
Email
Online Social Network
Streaming Audio Video
File Sharing
Instant Messaging
…
13

Examples of Distributed Systems
Local Area Network and Intranet
Database Management System
Automatic Teller Machine Network
Internet/World-Wide Web
Mobile and Ubiquitous Computing
14
14

Local Area Network
the rest of
email server
Web server
Desktop
computers
File server
router/firewall
print and other servers
other servers
print
Local area
network
email server
the Internet
15
15

Database Management System
16
16

Automatic Teller Machine Network
17
17

Internet
intranet
ISP
desktop computer:
backbone
satellite link
server:
%
network link:
%
%
%
18
18

Fig1.6: Growth of the Internet
(computers and web servers)
Date Computers Web servers Percentage
1993, July 1,776,000 130 0.008
1995, July 6,642,000 23,500 0.4
1997, July 19,540,000 1,203,096 6
1999, July 56,218,000 6,598,697 12
2001, July 125,888,197 31,299,592 25
42,298,371
2003, July
2005, July
~200,000,000
353,284,187 67,571,581
21
19
19

Web Servers and Web Browsers
Internet
Browsers
Web servers
www.google.com
www.uu.se
www.w3c.org
Protocols
Activity.html
http://www.w3c.org/Protocols/Activity.html
http://www.google.comlsearch?q=lyu
http://www.uu.se/
File system of
www.w3c.org
21
21

Mobile and Ubiquitous Computing
Laptop
Mobile
Printer
Camera
Internet
Host intranet Home intranet
GSM/GPRS
Wireless LAN
phone
gateway
Host site
22
22

23
Selected application domains and
associated networked applications
Finance and commerce eCommerce e.g. Amazon and eBay, PayPal, online banking
and trading
The information society Web information and search engines, ebooks, Wikipedia; social
networking: Facebook and MySpace.
Creative industries and
entertainment
online gaming, music and film in the home, user-generated
content, e.g. YouTube, Flickr
Healthcare health informatics, on online patient records, monitoring
patients
Education e-learning, virtual learning environments; distance learning
Transport and logistics GPS in route finding systems, map services: Google Maps,
Google Earth
Science The Grid as an enabling technology for collaboration between
scientists
Environmental
management
sensor technology to monitor earthquakes, floods or tsunamis

Common Characteristics
What are we trying to achieve when we construct a
distributed system?
Certain common characteristics can be used to assess
distributed systems
Heterogeneity
Openness
Security
Scalability
Failure Handling
Concurrency
Transparency
24
24

1 Heterogeneity
Variety and differences in
Networks
Computer hardware
Operating systems
Programming languages
Implementations by different developers
Middleware as software layers to provide a programming
abstraction as well as masking the heterogeneity of the
underlying networks, hardware, OS, and programming
languages (e.g., CORBA, COM).
Mobile Code to refer to code that can be sent from one
computer to another and run at the destination (e.g., Java
applets and Java virtual machine). 25
25

2 Openness
Openness is concerned with extensions and
improvements of distributed systems.
Detailed interfaces of components need to be
published.
New components have to be integrated with
existing components.
Differences in data representation of interface
types on different processors (of different vendors)
have to be resolved.
26
26

3 Security
 Confidentiality
 Integrity
 Availability
 In a distributed system, clients send requests to access data managed
by servers, resources in the networks:
 Doctors requesting records from hospitals
 Users purchase products through electronic commerce
 Security is required for:
 Concealing the contents of messages: security and privacy
 Identifying a remote user or other agent correctly (authentication)
 New challenges:
 Denial of service attack
 Security of mobile code
27
27

4 Scalability
 Adaptation of distributed systems to
accommodate more users
respond faster (this is the hard one)
 Usually done by adding more and/or faster processors.
 Components should not need to be changed when scale of a system
increases.
 Design components to be scalable!
 Design Challenges
Controlling the cost pf physical resource
Controlling the performance loss
Preventing software resources running out
Avoiding performance bottlenecks
28
28

5 Failure Handling (Fault Tolerance)
Hardware, software and networks fail!
Distributed systems must maintain availability
even at low levels of hardware/software/network
reliability.
Fault tolerance is achieved by
Detecting Failure
Masking Failures
Tolerating Failures
recovery
redundancy 29
29

6 Concurrency
Components in distributed systems are executed
in concurrent processes.
Components access and update shared resources
(e.g. variables, databases, device drivers).
Integrity of the system may be violated if
concurrent updates are not coordinated.
Lost updates
Inconsistent analysis
30
30

7 Transparency
Distributed systems should be perceived by users
and application programmers as a whole rather
than as a collection of cooperating components.
Transparency has different aspects.
These represent various properties that
distributed systems should have.
31
31

7.1 Access Transparency
Enables local and remote information objects to
be accessed using identical operations.
Example: File system operations in NFS.
Example: Navigation in the Web.
Example: SQL Queries
32
32

7.2 Location Transparency
Enables information objects to be accessed
without knowledge of their location.
Example: File system operations in NFS
Example: Pages in the Web
Example: Tables in distributed databases
33
33

Network Transparency
Access Transparency
Location Transparency
34

7.3 Concurrency Transparency
Enables several processes to operate
concurrently using shared information objects
without interference between them.
Example: NFS
Example: Automatic teller machine network
Example: Database management system
35
35

7.4 Replication Transparency
Enables multiple instances of information objects
to be used to increase reliability and performance
without knowledge of the replicas by users or
application programs
Example: Distributed DBMS
Example: Mirroring Web Pages.
36
36

7.5 Failure Transparency
Enables the concealment of faults
Allows users and applications to complete their
tasks despite the failure of other components.
Example: Database Management System
37
37

7.6 Mobility Transparency
Allows the movement of information objects
within a system without affecting the operations of
users or application programs
Example: NFS
Example: Web Pages
38
38

7.7 Performance Transparency
Allows the system to be reconfigured to improve
performance as loads vary.
Example: Distributed make.
39
39

7.8 Scaling Transparency
Allows the system and applications to expand in
scale without change to the system structure or
the application algorithms.
Example: World-Wide-Web
Example: Distributed Database
40
40

Transparencies
Access transparency: enables local and remote resources to be accessed using identical
operations.
Location transparency: enables resources to be accessed without knowledge of their
physical or network location (for example, which building or IP address).
Concurrency transparency: enables several processes to operate concurrently using
shared resources without interference between them.
Replication transparency: enables multiple instances of resources to be used to increase
reliability and performance without knowledge of the replicas by users or application
programmers.
Failure transparency: enables the concealment of faults, allowing users and application
programs to complete their tasks despite the failure of hardware or software
components.
Mobility transparency: allows the movement of resources and clients within a system
without affecting the operation of users or programs.
Performance transparency: allows the system to be reconfigured to improve
performance as loads vary.
Scaling transparency: allows the system and applications to expand in scale without
change to the system structure or the application algorithms.
41

Fig2.2:Communicating entities and
communication paradigms
43

Fig2.3:Clients invoke individual servers
Server
Client
Client
invocation
result
Server
invocation
result
Process:
Key:
Computer:
44

Fig2.4a:Peer-to-peer architecture
45

Fig2.4b: A service provided by multiple
servers
Server
Server
Server
Service
Client
Client
46

Fig2.5:Web proxy server
Client
Proxy
Web
server
Web
server
server
Client
47

Fig2.6: Web applets
a) client request results in the downloading of applet code
Web
server
Client
Web
server
Applet
Applet code
Client
b) client interacts with the applet
48

Fig2.7:Software and hardware
service layers in distributed Systems
Applications, services
Computer and network hardware
Platform
Operating system
Middleware
49

Fig2.8:Two-tier & three-tier architectures
50

Fig2.9:AJAX example: soccer score
updates
new Ajax.Request('scores.php? game=Arsenal:Liverpool‟,
{onSuccess: updateScore});
function updateScore(request) {
.....
( request contains the state of the Ajax request including the
returned result.
The result is parsed to obtain some text giving the score, which
is used to update the relevant portion of the current page.)
....
}
51

Fig2.10:Thin clients and compute
servers
Thin
Client
Application
Process
Network computer or PC
Compute server
network
52

Fig2.11:The web service architectural
pattern
53

Fig2.12: Categories of middleware
54

Fig2.13:Real-time ordering of events
send
receive
send
receive
m1 m2
2
1
3
4
X
Y
Z
Physical
time
A
m3
receive receive
send
receive receive receive
t1 t2 t3
receive
receive
m2
m1
55

Fig2.14:Processes and channels
process p process q
Communication channel
send
Outgoing message buffer Incoming message buffer
receive
m
56

Fig2.15: Omission and arbitrary failures
Class of failure Affects Description
Fail-stop Process Process halts and remains halted. Other processes may
detect this state.
Crash Process Process halts and remains halted. Other processes may
not be able to detect this state.
Omission Channel A message inserted in an outgoing message buffer never
arrives at the other end‟s incoming message buffer.
Send-omission Process A process completes a send,but the message is not put
in its outgoing message buffer.
Receive-omissionProcess A message is put in a process‟s incoming message
buffer, but that process does not receive it.
Arbitrary
(Byzantine)
Process or
channel
Process/channel exhibits arbitrary behaviour: it may
send/transmit arbitrary messages at arbitrary times,
commit omissions; a process may stop or take an
incorrect step.
57

Fig2.11:Timing failures
Class of FailureAffects Description
Clock Process Process’s local clock exceeds the bounds on its
rate of drift from real time.
Performance Process Process exceeds the bounds on the interval
between two steps.
Performance Channel A message’s transmission takes longer than the
stated bound.
58

Fig2.17:Objects and principals
Network
invocation
result
Client
Server
Principal (user) Principal (server)
Object
Access rights
59

Fig2.18: The enemy
Communication channel
Copy of m
Process p Process q
m
The enemy
m’
60

Fig2.19: Secure channels
Principal A
Secure channel
Process p Process q
Principal B
61

Fig3.1:Network performance
km
62

Fig3.2: Conceptual layering of
protocol software
Layer n
Layer 2
Layer 1
Message sent Message received
Communication
medium
Sender Recipient
63

Fig3.3: Encapsulation as it is applied
in layered protocols
Presentation header
Application-layer message
Session header
Transport header
Network header
64

Fig3.4:Protocol layers in the ISO Open
Systems Interconnection (OSI) model
Application
Presentation
Session
Transport
Network
Data link
Physical
Message sent Message received
Sender Recipient
Layers
Communication
medium
65

Fig3.5: OSI protocol summary
Layer Description Examples
Application Protocols that are designed to meet the communication requirements of
specific applications, often defining the interface to a service.
HTTP, FTP , SMTP,
CORBA IIOP
Presentation Protocols at this level transmit data in a network representation that is
independent of the representations used in individual computers, which may
differ. Encryption is also performed in this layer, if required.
Secure Sockets
(SSL),CORBA Data
Rep.
Session At this level reliability and adaptation are performed, such as detection of
failures and automatic recovery.
Transport This is the lowest level at which messages (rather than packets) are handled.
Messages are addressed to communication ports attached to processes,
Protocols in this layer may be connection-oriented or connectionless.
TCP, UDP
Network Transfers data packets between computers in a specific network. In a WAN
or an internetwork this involves the generation of a route passing through
routers. In a single LAN no routing is required.
IP, ATM virtual
circuits
Data link Responsible for transmission of packets between nodes that are directly
connected by a physical link. In a WAN transmission is between pairs of
routers or between routers and hosts. In a LAN it is between any pair of hosts.
Ethernet MAC,
ATM cell transfer,
PPP
Physical The circuits and hardware that drive the network. It transmits sequences of
binary data by analogue signalling, using amplitude or frequency modulation
of electrical signals (on cable circuits), light signals (on fibre optic circuits)
or other electromagnetic signals (on radio and microwave circuits).
Ethernet base- band
signalling, ISDN
66

Fig3.6:Internetwork layers
Underlying network
Application
Network interface
Transport
Internetwork
Internetwork packets
Network-specific packets
Message
Layers
Internetwork
protocols
Underlying
network
protocols
67

Fig3.7:Routing in a wide area network
Hosts
Links
or local
networks
A
D E
B
C
1
2
5
4
3
6
Routers
68

Fig3.8:Routing tables for the network
in Fig3.7
Routings from D Routings from E
To Link Cost To Link Cost
A
B
C
D
E
3
3
6
local
6
1
2
2
0
1
A
B
C
D
E
4
4
5
6
local
2
1
1
1
0
Routings from A Routings from B Routings from C
To Link Cost To Link Cost To Link Cost
A
B
C
D
E
local
1
1
3
1
0
1
2
1
2
A
B
C
D
E
1
local
2
1
4
1
0
1
2
1
A
B
C
D
E
2
2
local
5
5
2
1
0
2
1
69

Fig3.9:Pseudo-code for RIP routing
algorithm
Send: Each t seconds or when Tl changes, send Tl on each non-faulty outgoing
link.
Receive: Whenever a routing table Tr is received on link n:
for all rows Rr in Tr {
if (Rr.link | n) {
Rr.cost = Rr.cost + 1;
Rr.link = n;
if (Rr.destination is not in Tl) add Rr to Tl;
// add new destination to Tl
else for all rows Rl in Tl {
if (Rr.destination = Rl.destination and
(Rr.cost < Rl.cost or Rl.link = n)) Rl = Rr;
// Rr.cost < Rl.cost : remote node has better route
// Rl.link = n : remote node is more authoritative
}}}
70

Fig3.10: Simplified view of part of a
university campus network
file
compute
dialup
hammer
henry
hotpoint
138.37.88.230
138.37.88.162
bruno
138.37.88.249
router/
sickle
138.37.95.241
138.37.95.240/29
138.37.95.249
copper
138.37.88.248
firewall
web
138.37.95.248/29
server
desktop computers 138.37.88.xx
subnet
subnet
Eswitch
138.37.88
server
server
server
138.37.88.251
custard
138.37.94.246
desktop computers
Eswitch
138.37.94
hub
hub
Student subnet
Staff subnet
other
servers
router/
firewall
138.37.94.251
☎
1000 Mbps Ethernet
Eswitch: Ethernet switch
100 Mbps Ethernet
file server/
gateway
printers
Campus
router
Campus
router
138.37.94.xx
71

Fig3.11:Tunnelling for IPv6 migration
A B
IPv6 IPv6
IPv6 encapsulated in IPv4 packets
Encapsulators
IPv4 network
72

Fig3.12: TCP/IP layers
Messages (UDP) or Streams (TCP)
Application
Transport
Internet
UDP or TCP packets
IP datagrams
Network-specific frames
Message
Layers
Underlying network
Network interface
73

Fig3.13:Encapsulation in a message
transmitted via TCP over an Ethernet
Application message
TCP header
IP header
Ethernet header
Ethernet frame
port
TCP
IP
74

Fig3.14:The programmer's conceptual
view of a TCP/IP Internet
IP
Application Application
TCP UDP
75

Fig3.15: Internet address structure,
showing field sizes in bits
7 24
Class A: 0 Network ID Host ID
14 16
Class B: 1 0 Network ID Host ID
21 8
Class C: 1 1 0 Network ID Host ID
28
Class D (multicast): 1 1 1 0 Multicast address
27
Class E (reserved): 1 1 1 1 unused
0
28
76

Fig3.16:Decimal representation of
Internet addresses
octet 1 octet 2 octet 3
Class A: 1 to 127
0 to 255 0 to 255 1 to 254
Class B: 128 to 191
Class C: 192 to 223
224 to 239
Class D (multicast):
Network ID
Network ID
Network ID
Host ID
Host ID
Host ID
Multicast address
0 to 255 0 to 255 1 to 254
0 to 255 0 to 255 0 to 255
0 to 255 0 to 255 0 to 255
0 to 255 0 to 255 1 to 254
240 to 255
Class E (reserved):
1.0.0.0 to
127.255.255.255
128.0.0.0 to
191.255.255.255
192.0.0.0 to
223.255.255.255
224.0.0.0 to
239.255.255.255
240.0.0.0 to
255.255.255.255
Range of addresses
77

Fig3.17:IP packet layout
data
IP address of destination
IP address of source
header
up to 64 kilobytes
Figure 3.19 IPv6 header layout
Source
address
(128 bits)
Destination
address
(128 bits)
Version (4 bits) Traffic class (8 bits) Flow label (20 bits)
Payload length (16 bits) Hop limit (8 bits)
Next header (8 bits)
78

Fig3.18:A typical NAT-based home
network
79

Fig3.20: The MobileIP routing
mechanism
Sender
Home
Mobile host MH
Foreign agent FA
Internet
agent
First IP packet
addressed to MH
Address of FA
returned to sender
First IP packet
tunnelled to FA
Subsequent IP packets
tunnelled to FA
80

Fig3.21:Firewall configurations
Internet
Router/
Protected intranet
a) Filtering router
Internet
b) Filtering router and bastion
filter
Internet
R/filter
c) Screened subnet for bastion R/filter Bastion
R/filter Bastion
web/ftp
server
web/ftp
server
web/ftp
server
81

Fig3.22: IEEE 802 network standards
IEEE No. Name Title Reference
802.3 Ethernet CSMA/CD Networks (Ethernet) [IEEE 1985a]
802.4 Token Bus Networks [IEEE 1985b]
802.5 Token Ring Networks [IEEE 1985c]
802.6 Metropolitan Area Networks [IEEE 1994]
802.11 WiFi Wireless Local Area Networks [IEEE 1999]
802.15.1 Bluetooth Wireless Personal Area Networks [IEEE 2002]
802.15.4 ZigBee Wireless Sensor Networks [IEEE 2003]
802.16 WiMAX Wireless Metropolitan Area Networks [IEEE 2004a]
82

Fig3.23:Ethernet ranges and speeds
10Base5 10BaseT 100BaseT 1000BaseT
Data rate 10 Mbps 10 Mbps 100 Mbps 1000 Mbps
Max. segment lengths:
Twisted wire (UTP) 100 m 100 m 100 m 25 m
Coaxial cable (STP) 500 m 500 m 500 m 25 m
Multi-mode fibre 2000 m 2000 m 500 m 500 m
Mono-mode fibre 25000 m 25000 m 20000 m 2000 m
83

Fig3.24: Wireless LAN configuration
LAN
Server
Wireless
LAN
Laptops
Base station/
access point
Palmtop
radio obstruction
A B C
D
E
84

Fig3.25: Bluetooth frame structure
SCO packets (e.g. for voice data) have a 240-bit payload containing 80
bits of data triplicated, filling exactly one timeslot.
bits: 72 18 18 18 0 - 2744
Access code Header
copy 1
Header
copy 2
Header
copy 3
Data for transmission
bits: 3 1 1 1 4 8
Destination Flow Ack Seq Type Header checksum
Address within
Piconet
= ACL, SCO,
poll, null
Header
85

Designing Distributed Systems
Google Case Study
86

Fig21.1:Outline architecture of the
original Google search engine
87

Fig21.2: Example Google applications
88

Fig21.3:Organization of the Google
physical infrastructure
(To avoid
clutter the
Ethernet
connection
s are
shown
from only
one of the
clusters to
the
external
links)
89

Fig21.4:The scalability problem-Google
90

Fig21.5:The overall Google systems
architecture
91

Fig21.6: Google infrastructure
92

Fig21.7: Protocol buffers example
93

Fig21.8a:Summary of design choices
related to communication-paradigmspart
94

Fig21.8b:Summary of design choices
related to communication paradigms-part2
95

Fig21.9:Overall architecture of GFS
96

Fig21.11:Overall architecture of Chubby
98

Fig21.12: Message exchanges in
Paxos (in absence of failures) - step 1
99

Fig21.12:Message exchanges in Paxos
(in absence of failures) - step 2
100

Fig21.12:Message exchanges in
Paxos (in absence of failures)-step 3
101

Fig21.13:The table abstraction in Bigtable
102

Fig21.14:Overall architecture of Bigtable
103

Fig21.15:The storage architecture-Bigtable
104

Fig21.16:The hierarchical indexing
scheme adopted by Bigtable
105

Fig21.17:Summary of design choices
related to data storage and coordination

Fig21.18:Examples of the use of MapReduce
107

Fig21.19:The overall execution of a
MapReduce program
108

Fig21.20:The overall execution of a
Sawzall program
109

Fig21.21:Summary of design choices
related to distributed computation
110

Fig4.1: Middleware layers
INTERPROCESS COMMUNICATION
111

Fig4.2:Sockets and ports
message
agreed port
any port socket
socket
Internet address = 138.37.88.249
Internet address = 138.37.94.248
other ports
client server
112

UDP client sends a message to the
server and gets a reply
import java.net.*;
import java.io.*;
public class UDPClient{
public static void main(String args[]){
// args give message contents and server hostname
DatagramSocket aSocket = null;
try {
aSocket = new DatagramSocket();
byte [] m = args[0].getBytes();
InetAddress aHost = InetAddress.getByName(args[1]);
int serverPort = 6789;
DatagramPacket request = new DatagramPacket(m, m.length(), aHost, serverPort);
aSocket.send(request);
byte[] buffer = new byte[1000];
DatagramPacket reply = new DatagramPacket(buffer, buffer.length);
aSocket.receive(reply);
System.out.println("Reply: " + new String(reply.getData()));
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e){System.out.println("IO: " + e.getMessage());}
}finally {if(aSocket != null) aSocket.close();}
} }
113

UDP server repeatedly receives a
request and sends it back to the client
import java.net.*;
import java.io.*;
public class UDPServer{
DatagramSocket aSocket = null;
try{
aSocket = new DatagramSocket(6789);
while(true){
DatagramPacket request = new DatagramPacket(buffer, buffer.length);
aSocket.receive(request);
DatagramPacket reply = new DatagramPacket(request.getData(),
request.getLength(), request.getAddress(), request.getPort());
aSocket.send(reply);
}
}catch (IOException e) {System.out.println("IO: " + e.getMessage());}
}finally {if(aSocket != null) aSocket.close();}
}}
114

TCP client makes connection to server,
sends request and receives reply
import java.net.*;
import java.io.*;
public class TCPClient {
public static void main (String args[]) {
// arguments supply message and hostname of destination
Socket s = null;
try{
int serverPort = 7896;
s = new Socket(args[1], serverPort);
DataInputStream in = new DataInputStream( s.getInputStream());
DataOutputStream out =new DataOutputStream( s.getOutputStream());
out.writeUTF(args[0]); // UTF is a string encoding see Sn 4.3
String data = in.readUTF();
System.out.println("Received: "+ data) ;
}catch (UnknownHostException e){ System.out.println("Sock:"+e.getMessage());
}catch (EOFException e){System.out.println("EOF:"+e.getMessage());
}catch (IOException e){System.out.println("IO:"+e.getMessage());}
}finally {if(s!=null) try {s.close();}catch (IOException
e){System.out.println("close:"+e.getMessage());}}
}}
115

TCP server makes a connection for each
client and then echoes the client‟s request
import java.net.*;
import java.io.*;
public class TCPServer {
public static void main (String args[]) {
try{ int serverPort = 7896;
ServerSocket listenSocket = new ServerSocket(serverPort);
while(true) {
Socket clientSocket = listenSocket.accept();
Connection c = new Connection(clientSocket); }
} catch(IOException e) {System.out.println("Listen :"+e.getMessage());}
}
}
// this continues on the next slide
116

class Connection extends Thread {
DataInputStream in;
DataOutputStream out;
Socket clientSocket;
public Connection (Socket aClientSocket) {
try {
clientSocket = aClientSocket;
in = new DataInputStream( clientSocket.getInputStream());
out =new DataOutputStream( clientSocket.getOutputStream());
this.start();
} catch(IOException e) {System.out.println("Connection:"+e.getMessage());}
}
public void run(){
try { // an echo server
String data = in.readUTF();
out.writeUTF(data);
} catch(EOFException e) {System.out.println("EOF:"+e.getMessage());
} catch(IOException e) {System.out.println("IO:"+e.getMessage());}
} finally{ try {clientSocket.close();}catch (IOException e){/*close failed*/}}
}
}
117

CORBA CDR for constructed types
Type Representation
sequence length (unsigned long) followed by elements in order
string length (unsigned long) followed by characters in order (can also
can have wide characters)
array array elements in order (no length specified because it is fixed)
struct in the order of declaration of the comp onents
enumerated unsigned long (the values are specified by the order declared)
union type tag followed by the selected m emb er
118

CORBA CDR message
The flattened form represents a Person struct with value: {„Smith‟, „London‟, 1984}
0–3
4–7
8–11
12–15
16–19
20-23
24–27
5
"Smit"
"h___"
6
"Lond"
"on__"
1984
index in
sequence of bytes 4 bytes
notes
on representation
length of string
‘Smith’
length of string
‘London’
unsigned long
119

Indication of Java serialized form
The true serialized form contains additional type markers; h0 and h1 are handles
Serialized values
Person
3
1984
8-byte version number
int year
5 Smith
java.lang.String
name:
6 London
h0
java.lang.String
place:
h1
Explanation
class name, version number
number, type and name of
instance variables
values of instance variables
120

XML definition of the Person structure
<person id="123456789">
<name>Smith</name>
<place>London</place>
<year>1984</year>

</person >
use of namespace in the Person structure
<person pers:id="123456789" xmlns:pers="http://www.cdk5.net/person">
<pers:name> Smith </pers:name>
<pers:place> London </pers:place >
<pers:year> 1984 </pers:year>
</person>
121

An XML schema for the Person structure
<xsd:schema xmlns:xsd = URL of XML schema definitions >
<xsd:element name= "person" type ="personType" />
<xsd:complexType name="personType">
<xsd:sequence>
<xsd:element name = "name" type="xs:string"/>
<xsd:element name = "place" type="xs:string"/>
<xsd:element name = "year" type="xs:positiveInteger"/>
</xsd:sequence>
<xsd:attribute name= "id" type = "xs:positiveInteger"/>
</xsd:complexType>
</xsd:schema>
Representation of a remote object reference
Internet address
port number time object number
interface of
remote object
32 bits 32 bits 32 bits 32 bits
122

Multicast peer joins a group and sends
and receives datagrams
import java.net.*;
import java.io.*;
public class MulticastPeer{
// args give message contents & destination multicast group (e.g. "228.5.6.7")
MulticastSocket s =null;
try {
InetAddress group = InetAddress.getByName(args[1]);
s = new MulticastSocket(6789);
s.joinGroup(group);
byte [] m = args[0].getBytes();
DatagramPacket messageOut =
new DatagramPacket(m, m.length, group, 6789);
s.send(messageOut);
// this figure continued on the next slide
123

// get messages from others in group
for(int i=0; i< 3; i++) {
DatagramPacket messageIn =
new DatagramPacket(buffer, buffer.length);
s.receive(messageIn);
System.out.println("Received:" + new String(messageIn.getData()));
}
s.leaveGroup(group);
}catch (IOException e){System.out.println("IO: " + e.getMessage());}
}finally {if(s != null) s.close();}
}
}
124

Types of overlay
table continues on the next slide
125

Skype overlay architecture
127

Fig:4.17 An overview of point-to-point
communication in MPI
128

Selected send operations in MPI

20CS2021 DISTRIBUTED COMPUTING

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie 20CS2021 DISTRIBUTED COMPUTING

Ähnlich wie 20CS2021 DISTRIBUTED COMPUTING (20)

Mehr von Kathirvel Ayyaswamy

Mehr von Kathirvel Ayyaswamy (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

20CS2021 DISTRIBUTED COMPUTING