Introduction to Distributed Systems
8/16/2022 1
Dr.Mohammed Ibrahim

• Why do we develop distributed systems?
–availability of powerful yet cheap
microprocessors (PCs, workstations),
continuing advances in communication
technology,
8/16/2022 2
Dr.Mohammed Ibrahim

What is a distributed system?
• A distributed system is a collection of independent computers
that appear to the users of the system as a single system.
• Examples:
– Network of workstations
– Distributed manufacturing system (e.g., automated
assembly line)
– Network of branch office computers
8/16/2022 3
Dr.Mohammed Ibrahim

Hardware Concepts
MIMD (Multiple-Instruction Multiple-Data)
8/16/2022 4
Dr.Mohammed Ibrahim

Software Concepts
• Software more important for users
• Three types:
1. Network Operating Systems
2. Distributed Systems
3. Multiprocessor Time Sharing
8/16/2022 5
Dr.Mohammed Ibrahim

Advantages of Distributed Systems
over Centralized Systems
• Economics: a collection of microprocessors offer a better
price/performance than mainframes. Low price/performance ratio:
cost effective way to increase computing power.
• Speed: Enhanced performance through load distributing.
• Inherent distribution: Some applications are inherently distributed.
Ex. a supermarket chain.
• Reliability: If one machine crashes, the system as a whole can still
survive. Higher availability and improved reliability.
• Incremental growth: Computing power can be added in small
increments.
• Another deriving force: the existence of large number of personal
computers, the need for people to collaborate and share
information.
8/16/2022 6
Dr.Mohammed Ibrahim

Advantages of Distributed Systems
over Independent PCs
– Data sharing: allow many users to access to a common
data base
– Resource Sharing: expensive peripherals like color printers
– Communication: enhance human-to-human
communication, e.g., email, chat
– Flexibility: spread the workload over the available
machines
8/16/2022 7
Dr.Mohammed Ibrahim

Design Issues of Distributed Systems
• Transparency
• Flexibility
• Reliability
• Performance
• Scalability
8/16/2022 8
Dr.Mohammed Ibrahim

1 petaFLOPS = 10^15 flops or 1000 teraflops.
No computer has achieved this performance yet.
PETA FLoating point OPerations per Second
One quadrillion floating point operations per second
As of 2008, the fastest PC processors (quad-core)
perform over 70 GFLOPS (Intel Core i7 965 XE)
8/16/2022 9
Dr.Mohammed Ibrahim

8/16/2022 10
Dr.Mohammed Ibrahim

Distributed computing deals with hardware and software
systems containing more than one processing element or
storage element, concurrent processes, or multiple
programs, running under a loosely or tightly controlled
regime.
In distributed computing a program is split up into parts
that run simultaneously on multiple computers
communicating over a network. Distributed computing is
a form of parallel computing
Common Distributed Computing
Model
8/16/2022 11
Dr.Mohammed Ibrahim

In distributed computing a program is split up into parts that run simultaneously on
multiple computers communicating over a network
TASK 1
T
A
S
K
5
TASK 5
T 2
T4
T1 T3 T5
TASK 4
TASK 2
8/16/2022 12
Dr.Mohammed Ibrahim

Consider If There Are n Systems Connected In A
Network, Then We Can Split One Program Instruction
Into n Different Tasks And Compute Them
Concurrently.
8/16/2022 13
Dr.Mohammed Ibrahim

Why we need Distributed Computing?
8/16/2022 14
Dr.Mohammed Ibrahim

•Computation requirements are ever increasing
• development in networking technology.
•The parallel processing technology is mature and is being exploited
commercially.
8/16/2022 15
Dr.Mohammed Ibrahim

Speedup achieved by distributed computing
Speedup = log2(no. of processors)
8/16/2022 16
Dr.Mohammed Ibrahim

8/16/2022 17
Dr.Mohammed Ibrahim

Organizing the interaction between
the computers that execute distributed
computations is of prime importance.
In order to be able to use the widest
possible variety of computers, the
protocol or communication channel
should be universal.
Software Portability
Motivation Factor
The human brain consists of a
large number (more than a
billion) of neural cells that
process information. Each cell
works like a simple processor and
only the massive interaction
between all cells and their
parallel processing makes the
brain's abilities possible.
8/16/2022 18
Dr.Mohammed Ibrahim

There are many different types of distributed computing systems and
many challenges to overcome in successfully designing one. The main
goal of a distributed computing system is to connect users and
resources in a transparent, open, and scalable way.
8/16/2022 19
Dr.Mohammed Ibrahim

Processor
A
Processor
A
Processor
A
M
E
M
.
B
us
M
E
M
.
B
us
M
E
M
.
B
us
Memory
System A
Memory
System A
Memory
System A
8/16/2022 20
Dr.Mohammed Ibrahim

Possible ways to Implement Distributed Computing
Architectures
8/16/2022 21
Dr.Mohammed Ibrahim

Various hardware and software architectures are used for distributed computing.
lower level,
it is necessary to interconnect multiple CPUs with some sort of network, regardless of
whether that network is printed onto a circuit board or made up of loosely-coupled
devices and cables.
higher level,
it is necessary to interconnect processes running on those CPUs with some sort of
communication system.
8/16/2022 22
Dr.Mohammed Ibrahim

Client-server — Smart client code contacts the server for data, then
formats and displays it to the user.
3-tier architecture — Three tier systems move the client intelligence to
a middle tier so that stateless clients can be used. Most web applications
are 3-Tier.
N-tier architecture — N-Tier refers typically to web applications which
further forward their requests to other enterprise services. This type of
application is the one most responsible for the success of application
servers.
clustered— refers typically to a cluster of machines that closely work
together, running a shared process in parallel.
Peer-to-peer — architecture where there is no special machine or
machines that provide a service or manage the network resources.
Instead all responsibilities are uniformly divided among all machines,
known as peers. Peers can serve both as clients and servers.
Client Server
Architecture
3-Tier
Architecture
N- Tier
Architecture
Tightly
Coupled
Peer To Peer
8/16/2022 23
Dr.Mohammed Ibrahim

Distributed computing implements a kind of concurrency.
8/16/2022 24
Dr.Mohammed Ibrahim

Multiprocessor systems
A multiprocessor system is simply a computer that has more than one CPU on its motherboard.
Multicore Systems
Intel CPUs from the late Pentium 4 era (Northwood and Prescott cores) employed a technology called
Hyper-threading that allowed more than one thread (usually two) to run on the same CPU.
Computer Clusters
A cluster consists of multiple stand-alone machines acting in parallel across a local high speed
network.
Grid computing
A grid uses the resources of many separate computers, loosely connected by a network (usually the
Internet), to solve large-scale computation problems.
8/16/2022 25
Dr.Mohammed Ibrahim

Language that Use or make a distributed system and projects that been implemented
8/16/2022 26
Dr.Mohammed Ibrahim

Remote procedure calls distribute operating system
commands over a network connection. Systems like CORBA,
Microsoft DCOM, Java RMI and others, try to map object
oriented design to the network.
Loosely coupled systems communicate through
intermediate documents that are typically human readable
(e.g. XML, HTML).
8/16/2022 27
Dr.Mohammed Ibrahim

• The Common Object Request Broker
Architecture (CORBA) is a standard defined by
the Object Management Group (OMG)
designed to facilitate the communication of
systems that are deployed on diverse
platforms.
• CORBA enables collaboration between
systems on different operating
systems, programming languages, and
computing hardware.
8/16/2022 Dr.Mohammed Ibrahim 28

• CORBA uses an interface definition
language (IDL) to specify the interfaces that
objects present to the outer world.
• CORBA then specifies a mapping from IDL to a
specific implementation language
like C++ or Java.
8/16/2022 Dr.Mohammed Ibrahim 29

Implemented Distributed Computing
8/16/2022 30
Dr.Mohammed Ibrahim

• Distributed Computing has become a reality:
– Threads concept utilized everywhere.
– Clusters have emerged as popular data centers and
processing engine:
• E.g., Google search engine.
• The emergence of commodity high-performance
CPU, networks, and OSs have made parallel
computing applicable to enterprise applications.
– E.g., Oracle {9i,10g} database on Clusters/Grids.
8/16/2022 31
Dr.Mohammed Ibrahim

Assignment 1
Writ about MIMD?
Next week
8/16/2022 Dr.Mohammed Ibrahim 32