1. Software Architecture
School of Computer Science University of Oviedo
University of Oviedo
Software Architecture
Part II. Software Architecture Taxonomies
Lesson 8: Components and connectors
2014 Jose Emilio Labra Gayo
2. Software Architecture
School of Computer Science University of Oviedo
Styles based on behaviour
Runtime behaviour
Components and connectors
3. Software Architecture
School of Computer Science University of Oviedo
Main classification
Data flow
Job management
Interaction
Repository
Invocation
Distributed systems
Adaptable systems
4. Software Architecture
School of Computer Science University of Oviedo
Batch
Pipes & Filters
Pipes & Filters with uniform interface
5. Software Architecture
School of Computer Science University of Oviedo
Batch
Independent programs are executed sequentially
Data is passed from one program to the next
Stage
Note
Batch style can be considered the grandfather of
software architectural styles
Write port
Stage
Stage
Stage Connector
Read port
6. Software Architecture
School of Computer Science University of Oviedo
Batch
Elements:
Independent executable programs
Constraints
Output of one program is linked to input of the next
A program usually waits for the previous one to
finish its execution
7. Software Architecture
School of Computer Science University of Oviedo
Batch
Advantages
Low coupling between components
Re-configurability
Debugging
It is possible to debug each input independently
8. Software Architecture
School of Computer Science University of Oviedo
Batch
Challenges
It does not offer interactive interface
Requires external intervention
No support for concurrency
Low throughput
High latency
9. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters
Data flows through pipes and is processed by filters
Filter
Filter
Filter
Pipe
Filter
Filter Filter
Write port
Read port
10. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters
Elements
Filter: component that transforms data
Filters can be executed concurrently
Pipe: Takes output data from one filter to the input of
another filter
Properties: buffer size, data format, interaction protocol
11. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters
Constraints
Pipes connect outputs from one filter to inputs of other
filters
Filters must agree on the exchange format they admit
12. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters
Advantages
Better understanding of global system
Total behavior = sum of each filter behavior
Reusability:
Filters can be recombined
Evolution and extensibility:
It is possible to create/add new filters
It is possible to substitute old filters by new ones
Independence
Independent verification of each filter
Performance
It enables concurrent execution of filters
13. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters
Challenges
Possible delays in case of long pipes
It may be difficult to pass complex data structures
Non interactivity
A filter can not interact with its environment
14. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters
Applications
Unix
who | wc -l
Java
Clases java.io (PipedReader, PipedWriter)
Yahoo Pipes
15. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters - uniform interface
Variant of Pipes & Filters where filters have the
same interface
Elements
The same as in Pipes & Filters
Constraints
Filters must have a uniform interface
16. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters - uniform interface
Advantages:
Independent development of filters
Re-configurability
Facilitates system understanding
Challenges:
Performance can be affected if data have to be
converted to the uniform interface
Marshalling
17. Software Architecture
School of Computer Science University of Oviedo
Pipes & Filters - uniform interface
Examples:
Unix operating system
Programs with a text input (stdin) and 2 text outputs
(stdout y stderr)
Web architecture: REST
19. Software Architecture
School of Computer Science University of Oviedo
Master-Slave
Master divides work in sub-tasks
Assigns each sub-task to different nodes
The computational result is obtained as the
combination of the slaves results results
Master
Slave 1 Slave 2
. . . Slave N
Problem
task 1 task 2 task N
Solution
result 1 result 2 result N
20. Software Architecture
School of Computer Science University of Oviedo
Master-Slave
Elements
Master: Coordinates execution
Slave: does a task and returns the result
Constraints
Slave nodes are only in charge of the computation
Control is done by the Master node
21. Software Architecture
School of Computer Science University of Oviedo
Master-Slave
Advantages
Parallel computation
Fault tolerance
Challenges
Difficult to coordinate work between slaves
Dependency on Master node
Dependency on physical configuration
22. Software Architecture
School of Computer Science University of Oviedo
Master-Slave
Applications:
Process control systems
Embedded systems
Fault tolerant systems
Search systems
23. Software Architecture
School of Computer Science University of Oviedo
MVC: Model - view - controller
MVC variants
PAC: Presentation - Abstraction - Control
24. Software Architecture
School of Computer Science University of Oviedo
MVC
MVC: Model - View - Controller
Proposed by Trygve Reenskaug (end of 70's)
Solution for GUI
Controller separates model from the view
"Mental model" offered through views
25. Software Architecture
School of Computer Science University of Oviedo
MVC
Elements
Model: represents business logic and state
View: Offers state representation to the user
Controller: Coordinates interaction, views and model
Mental
Model
Controller
Model
View 1
View 2
User
26. Software Architecture
School of Computer Science University of Oviedo
MVC
Constraints
Controller processes user events
Creates/removes views
Handles interaction
Views only show values
Models are independent of controllers/views
27. Software Architecture
School of Computer Science University of Oviedo
MVC
Advantages
It supports multiple views of the same model
Views synchronization
Separation of concerns
Interaction (controller), state (model)
It is easy to create new views and controllers
Easy to modify look & feel
Creation of generic frameworks
28. Software Architecture
School of Computer Science University of Oviedo
MVC
Challenges
Increases complexity of GUI development
Coupling between controllers and views
Controllers/Views should depend on a model
interface
Some difficulties for GUI tools
29. Software Architecture
School of Computer Science University of Oviedo
MVC
Applications
Lots of web frameworks follow MVC
Ruby on Rails, Spring MVC, Play, etc.
Some variants
Push: controllers send orders to views
RoR
Pull: controllers receive orders from views
Play
30. Software Architecture
School of Computer Science University of Oviedo
MVC variants
PAC
Model-View-Presenter
Model View ViewModel
...
31. Software Architecture
School of Computer Science University of Oviedo
PAC
PAC: Presentation-Abstraction-Control
Hierarchy of agents
Each agent contains 3 components
PAC Agent
Abstraction Control Presentation
PAC Agent
Abstraction Control Presentation
PAC Agent
Abstraction Control Presentation
32. Software Architecture
School of Computer Science University of Oviedo
PAC
Elements
Agents with
Presentation: visualization aspects
Abstraction: data model of an agent
Control: connects presentation and abstraction
components and enables communication between
agents
Hierarchical relationship between agents
33. Software Architecture
School of Computer Science University of Oviedo
PAC
Constraints
Each agent is in charge of some functionality
There is no direct communication between
abstraction and presentation in each agent
Communication through the control component
34. Software Architecture
School of Computer Science University of Oviedo
PAC
Advantages
Separation of concerns
Identifies functionalities
Support for changes and extensions
It is possible to modify an agent without affecting others
Multitask
Agents can reside in different threads, processes or
machines
35. Software Architecture
School of Computer Science University of Oviedo
PAC
Challenges
Complexity of the system
Too many agents can generate a complex structure
which can be difficult tom maintain
Complexity of control components
Control components handle communication
Quality of control components is important for the
whole quality of the system
Performance
Communication overload between agents
36. Software Architecture
School of Computer Science University of Oviedo
PAC
Applications
Network monitoring systems
Mobile robots
Drupal is based on PAC
Relationships
This patterns is related with MVC
MVC has no agent hierarchy
This pattern was re-discovered as Hierarchical MVC
38. Software Architecture
School of Computer Science University of Oviedo
Shared data
Independent components access the same state
Applications based on centralized data repositories
Shared
Data
Component Component
Component
...
39. Software Architecture
School of Computer Science University of Oviedo
Shared data
Elements
Shared data
Database or centralized repository
Components
Processors that interact with shared data
Shared
Data
Component Component
Component
...
40. Software Architecture
School of Computer Science University of Oviedo
Shared data
Constraints
Components interact with the global state
Components don't communicate between each other
Only through shared state
Shared repository handles data stability and consistency
41. Software Architecture
School of Computer Science University of Oviedo
Shared data
Advantages
Independent components
They don't need to be aware of the existence of other
components
Data consistency
Centralized global state
Unique Backup of all the system state
42. Software Architecture
School of Computer Science University of Oviedo
Shared data
Challenges
Unique point of failure
A failure in the central repository can affect the whole
system
Distributing the central data can be difficult
Possible bottleneck
Inefficient communication
Problems for scalability
Synchronization to access shared memory
43. Software Architecture
School of Computer Science University of Oviedo
Shared data
Applications
Lots of systems use this approach
Some variants
This style is also known as:
Shared Memory, Repository, Shared data, etc.
Blackboard
Rule based systems
44. Software Architecture
School of Computer Science University of Oviedo
Blackboard
Complex problems which are difficult to solve
Knowledge sources solve parts of the problem
Each knowledge source aggregates partial solutions to
the blackboard
Knowledge
Source
Knowledge
Source
Blackboard
Knowledge
Source ...
Control
45. Software Architecture
School of Computer Science University of Oviedo
Blackboard
Elements
Blackboard: Central data repository
Knowledge source: solves part of the problem and
aggregates partial results
Control: Manages tasks and checks the work state
Fuente
conocimiento
Fuente
conocimiento
Blackboard
Fuente
Conocimiento ...
Control
46. Software Architecture
School of Computer Science University of Oviedo
Blackboard
Constraints
Problem can be divided in parts
Each knowledge source solves a part of the problem
Blackboard contains partial solutions that are
improving
47. Software Architecture
School of Computer Science University of Oviedo
Blackboard
Advantages
Can be used for open problems
Experimenting
Facilitates strategy changes
Knowledge source can be reused
Support for fault tolerance
48. Software Architecture
School of Computer Science University of Oviedo
Blackboard
Challenges
It may be difficult to debug
No warranty that the right solution will be found
It may be difficult to establish a control strategy
Low performance
Sometimes it may need to review the incorrect
hypothesis
High development cost
Implementation of parallelism
It is necessary to synchronize blackboard access
49. Software Architecture
School of Computer Science University of Oviedo
Blackboard
Applications
Some speech recognition systems
HEARSAY-II
Pattern recognition
Weather forecasts
Games
Analysis of molecular structure
Crystalis
50. Software Architecture
School of Computer Science University of Oviedo
Rule based systems
Variant of shared memory
Shared memory = Knowledge base
Contains rules and facts
Knowledge base
Rules + facts
Inference
Engine
User
Interface
51. Software Architecture
School of Computer Science University of Oviedo
Rule based systems
Elements:
Knowledge base: Rules and facts about some
domain
User interface: Queries/modifies knowledge base
Inference engine: Answers queries from data and
knowledge base
Knowledge base
Rules + facts
Inference
Engine
User
Interface
52. Software Architecture
School of Computer Science University of Oviedo
Rule based systems
Constraints:
Domain knowledge captured in knowledge base
Limit imperative access to knowledge base
It is based on rules like:
IF antecedents THEN consequent
Limits expressiveness with regards to imperative
languages
53. Software Architecture
School of Computer Science University of Oviedo
Rule based systems
Advantages
Declarative solution
It may be easy to modify the knowledge base
Specially tailored to be modified by domain experts
Separation of concerns
Algorithm
Domain knowledge
Reusability
54. Software Architecture
School of Computer Science University of Oviedo
Rule based systems
Challenges
Rules Debugging
Performance
Rules creation and maintenance
Introspection
Automatic rule learning
Runtime update of rules
55. Software Architecture
School of Computer Science University of Oviedo
Rule based systems
Applications
Expert system
Production systems
Rules libraries in Java
JRules, Drools, JESS
Declarative, rule based languages
Prolog (logic programming)
56. Software Architecture
School of Computer Science University of Oviedo
Call-return
Client-Server
Event based architectures
Publish-Subscribe
Actor models
57. Software Architecture
School of Computer Science University of Oviedo
Call-return
A component calls another component and waits
for the answer
call
Component A Component B
return
58. Software Architecture
School of Computer Science University of Oviedo
Call-return
Elements
Component that does the call
Component that sends the answer
Constraints
Synchronous communication:
The caller waits for the answer
call
Componente A Componente B
return
59. Software Architecture
School of Computer Science University of Oviedo
Call-return
Advantages
Easy to implement
Challenges
Problems for concurrent computation
If component is blocked waiting for the answer
It can be using unneeded resources
Distributed environments
Little utilization of computational capabilities
60. Software Architecture
School of Computer Science University of Oviedo
Client-Server
Variant of layers
2 layers physically separated (2-tier)
Functionality is divided in several servers
Clients connect to services
Interface query/answer
Network
query
answer
client
server
61. Software Architecture
School of Computer Science University of Oviedo
Client-Server
Elements
Server: offers services through a query/answer
protocol
Client: does queries and process answers
Network protocol: communication management
between clients and servers
Network
query
answer
client
server
62. Software Architecture
School of Computer Science University of Oviedo
Client-Server
Constraints
Clients communicate with servers
Not the other way
Clients are independent from other clients
Servers don't have knowledge about clients
Network protocol establishes some communication
warranties Network
query
answer
client
server
63. Software Architecture
School of Computer Science University of Oviedo
Client-Server
Advantages
Servers can be distributed
Separation of functionality between clients/servers
Independent development
Scalability
General functionality available for all clients
Although not all the servers need to offer all the
functionality
64. Software Architecture
School of Computer Science University of Oviedo
Client-Server
Challenges
Each server can be a single point of failiure
Server attacks
Unpredictable performance
Dependency on the system and the network
Problems when servers belong to other organizations
How can quality of service be warranted?
65. Software Architecture
School of Computer Science University of Oviedo
Client-Server
Variants
Stateless
Replicated server
With cache
66. Software Architecture
School of Computer Science University of Oviedo
Network
Client-Server stateless
Constraint
Server does not store information about clients
Same query implies same answer
query
answer
client
server
67. Software Architecture
School of Computer Science University of Oviedo
Client-Server stateless
Advantages
Scalability
Challenges
Application state management
Client must remember requests
Some strategies can be established to handle
information between requests
68. Software Architecture
School of Computer Science University of Oviedo
Network
Replicated server
Constraint
Several servers offer the same service
Offer the client the appearance that there is only one
server
query
answer
client
server
server
server
Abstract
Server
69. Software Architecture
School of Computer Science University of Oviedo
Replicated server
Advantages
Better answer times
Less latency
Fault tolerance
Challenges
Consistency management between replicated
servers
Synchronization
70. Software Architecture
School of Computer Science University of Oviedo
Client-server with cache
Cache = mediator between client/server
Stores copies of previous answers to the server
When a query is received it return the cached answer
without asking the original server
Network
query
answer
Cache
client server
71. Software Architecture
School of Computer Science University of Oviedo
Client-server with cache
Elements:
Intermediate cache nodes
Constraints
Some queries are directly answered by the cache
node
Cache node has a policy for answer management
Expiration time
72. Software Architecture
School of Computer Science University of Oviedo
Client-server with cache
Advantages:
Less network overload
Lots of repeated requests can be stored in the cache
Less answer time
Cached answers arrive earlier
73. Software Architecture
School of Computer Science University of Oviedo
Client-server with cache
Challenges
Complexity of configuration
Expiration policy
Not appropriate for certain domains
If high fidelity of answers is needed
Example: real time systems
74. Software Architecture
School of Computer Science University of Oviedo
Event based
EDA (Event-Driven-Architecture)
Event
Producer
Event
Processor
Event
Consumer
75. Software Architecture
School of Computer Science University of Oviedo
Event based
Elements:
Event:
Something that has happened (≠ request)
Event producer
Event generator (sensors, systems, ...)
Event consumer
DB, applications, scorecards, ...
Event processor
Transmission channel
Filters and transforms events
76. Software Architecture
School of Computer Science University of Oviedo
Event based
Constraints:
Asynchronous communication
Producers generate events at any moment
Consumers can be notified of events at any moment
Relationship one-to-many
An event can be sent to several consumers
77. Software Architecture
School of Computer Science University of Oviedo
Event based
Advantages
Decoupling: event producer does not depend on the
consumer, nor vice versa.
Timelessness: events are published without any
need to wait for the termination of any cycle
Asynchronous: In order to publish an event there is
no need to finish any process
78. Software Architecture
School of Computer Science University of Oviedo
Event based
Challenges
Non sequential execution
Difficult to debug
79. Software Architecture
School of Computer Science University of Oviedo
Event based
Applications
Event processing networks
Event-Stream-Processing (ESP)
Complex-event-processing
Variants
Publish-subscribe
Actor models
80. Software Architecture
School of Computer Science University of Oviedo
Publish-subscribe
Components that subscribe to a channel to receive
message from other components
Component
Component Component
Event Bus
Subscribe
Port
Publish
Port
Component
Component
81. Software Architecture
School of Computer Science University of Oviedo
Publish-subscribe
Elements:
Component:
Component that subscribes to a channel
Publication port
It is registered to publish messages
Subscription port
It is registered to receive some kind of messages
Event bus (message channel):
Transmits messages to subscribers
82. Software Architecture
School of Computer Science University of Oviedo
Publish-subscribe
Constraints:
Separation between subscription/publication port
A component may have both ports
Non-direct communication
Asynchronous communication in general
Components delegate communication responsibility
to the channel
83. Software Architecture
School of Computer Science University of Oviedo
Publish-subscribe
Advantages
Communication quality
Improves performance
Debugging
Low coupling between components
Consumers do not depend on publishers
...nor vice versa...
84. Software Architecture
School of Computer Science University of Oviedo
Publish-subscribe
Challenges
It adds a new indirection level
Direct communication may be more efficient in some
domains
Complex implementation
It may require COTS
85. Software Architecture
School of Computer Science University of Oviedo
Actor models
Used for concurrent computation
Actors instead of objects
There is no shared state between actors
Asynchronous message passing
Theoretical developments since 1973 (Carl Hewitt)
86. Software Architecture
School of Computer Science University of Oviedo
Actor models
Elements
Actor: computational entity with state
It communicates with other actors sending messages
It process messages one by one
Messages
Addresses: Identify actors (mailing address)
87. Software Architecture
School of Computer Science University of Oviedo
Actor models
Constraints
An actor can only:
Send messages to other actors
Messages are immutable
Create new actors
Modify how it will process next message
Actors are decoupled
Receiver does not depend on sender
88. Software Architecture
School of Computer Science University of Oviedo
Actor models
Constraints (2)
Local addresses
An actor can only send messages to known addresses
Because they were given to it or because he created them
Parallelism:
All actions are in parallel
No shared global state
Messages can arrive in any order
89. Software Architecture
School of Computer Science University of Oviedo
Actor models
Challenges
Message sending
How to handle arriving messages
Actor Coordination
Non-consistent systems by definition
Transparency and scalability
Internal vs external addresses
Non-local actor models
Web Services
Multi-agent systems
90. Software Architecture
School of Computer Science University of Oviedo
Actor models
Implementations
Erlang (programming language)
Akka (library)
Applications
Reactive systems
Examples: Ericsson, Facebook, twitter
92. Software Architecture
School of Computer Science University of Oviedo
Broker
Intermediate node that manages communication
between a client and a server
Client stub Broker skeleton Server
bridge
Client stub Broker skeleton Server
93. Software Architecture
School of Computer Science University of Oviedo
Broker
Elements
Broker
It manages communication
Client: Sends requests
Client Proxy: stub
Server: Returns answers
Server Proxy: skeleton
Bridge: Can connect brokers
Client stub Broker skeleton Server
bridge
94. Software Architecture
School of Computer Science University of Oviedo
Broker
Advantages
Separation of concerns
Delegates low level communication aspects to the
broker
Separate maintenance
Reusability
Servers are independent from clients
Portability
Broker = low level aspects
Interoperability
Using bridges
95. Software Architecture
School of Computer Science University of Oviedo
Broker
Challenges
Performance
It adds an indirection layer
Direct communication can be quicker
It can generate a high coupling between
components
Broker = single point of failure
96. Software Architecture
School of Computer Science University of Oviedo
Broker
Applications
CORBA and distributed systems
Android uses a variation of Broker pattern
97. Software Architecture
School of Computer Science University of Oviedo
Peer-to-Peer
Equal and autonomous nodes (peers) that
communicate between them.
Network
Peer
Peer
Peer
Peer
Peer
Peer
98. Software Architecture
School of Computer Science University of Oviedo
Peer-to-Peer
Elements
Computational nodes: peers
They contain their own state and control thread
Network protocol
Constraints
There is no main node
All peers are equal
99. Software Architecture
School of Computer Science University of Oviedo
Peer-to-Peer
Advantages
Decentralized information and control
Fault tolerance
There is no single point of failure
A failure in one peer does not compromise the whole
system
100. Software Architecture
School of Computer Science University of Oviedo
Peer-to-Peer
Challenges
Keeping the state of the system
Complexity of the protocol
Bandwidth Limitations
Network and protocol latency
Security
Detect malicious peers
101. Software Architecture
School of Computer Science University of Oviedo
Peer-to-Peer
Popular applications
Napster, BitTorrent, Gnutella, ...
This architecture style is not only to share files
e-Commerce (B2B)
Collaborative systems
Sensor networks
...
Variants
Super-peers
102. Software Architecture
School of Computer Science University of Oviedo
MapReduce
Proposed by Google
Published in 2004
Internal implementation by Google
Goal: big amounts of data
Lots of computational nodes
Fault tolerance
Write-once, read-many
Style composed of:
Master-slave
Batch Big
Data
Map Reduce
Result
103. Software Architecture
School of Computer Science University of Oviedo
MapReduce
Elements
Master node: Controls execution
Node table
It manages replicated file system
Slave nodes
Execute mappers, reducers
Contain replicated data blocks
Big
Data
Map Reduce
Result
104. Software Architecture
School of Computer Science University of Oviedo
MapReduce - Scheme
INspired by functional programming
2 components: mapper and reducer
Data are divided for their processing
Each data is associated with a key
Transforms [(key1,value1)] to [(key2,value2)]
c1
Input:
[(key1,value1)]
v1
c1 v1
c1 v1
Output:
[(key2,value2)]
c2 v2
c2 v2
c2 v2
c2 v2
MapReduce
106. Software Architecture
School of Computer Science University of Oviedo
MapReduce - Merge and sort
System merges and sorts intermediate results
according to the keys
k1 v1
k2 v2
k1 v3
k3 v4
k1 v5
k1 v6
k3 v7
k1 v1 v3 v5 v6
k2 v2
k3 v4 v7
Merge
and
sort
107. Software Architecture
School of Computer Science University of Oviedo
MapReduce - Reducer
Type:
(clave2, [valor2]) (clave2,valor2)
k1 v1 v3 v5 v6
k2 v2
k3 v4 v7
reduce
r
reduce
r
reduce
r
vf1
vf2
vf3
k1
k2
k3
108. Software Architecture
School of Computer Science University of Oviedo
MapReduce - general scheme
c1 vi1
c1 vi1
mapper
c1 vi1 mapper
reducer vf1
reducer
reducer
vf2
vf3
k1
k2
k3
mapper
k1 v1
k2 v2
k1 v3
k3 v4
k1 v5
k1 v6
k3 v7
k1 v1 v3 v5 v6
k2 v2
k3 v4 v7
Merge
and
sort
MapReduce
109. Software Architecture
School of Computer Science University of Oviedo
MapReduce - count words
d1 a b
d2 a c a
d3 a c
4
1
2
a
b
c
a 1
b 1
a 1
c 1
a 1
a 1
c 1
mapper
mapper
mapper
a 1 1 1 1 reducer
reducer
reducer
b 1
c 1 1
Mezcla
y
ordena
MapReduce
// return each work with 1
mapper(d,ps) {
for each p in ps:
emit (p, 1)
}
// sum the list of numbers of each word
reducer(p,ns) {
sum = 0
for each n in ns { sum += n; }
emit (p, sum)
}
110. Software Architecture
School of Computer Science University of Oviedo
MapReduce - execution environment
Execution environment is in charge of:
Planning: Each job is divided in tasks
Placement of data/code
Each node contains its data locally
Synchronization:
reduce tasks must wait map phase
Error and failure handling
High tolerance to computational nodes failures
111. Software Architecture
School of Computer Science University of Oviedo
MapReduce - File system
Google developed a distributed file system - GFS
Hadoop created HDFS
Files are divided in chunks
2 node types:
Namenode (master), datanodes (data servers)
Datanodes store different chunks
Block replication
Namenode contains metadata
Where is each chunk
Direct communication between clients and datanodes
113. Software Architecture
School of Computer Science University of Oviedo
MapReduce - Inverted index
Given a number of documents, obtain a list of
words associating each word with the document
where it appears
Order documents according to number of word
appearance
Documento 1
En un lugar de la Mancha , de
cuyo nombre no quiero
acordarme no ha mucho tiempo
que vivía un hidalgo de los de
lanza en astillero, adarga antigua,
rocín flaco y galgo corredor. Una
olla de algo más…
Inverted
Index
lugar doc16, doc21, doc23, doc45
Mancha doc22, doc2, doc4, doc9, doc11
Quijote doc22, doc1, doc2, doc7
. . .
114. Software Architecture
School of Computer Science University of Oviedo
MapReduce - Inverted index
Basic scheme of a search engine
P1
P1
CaPc1hed
documents
Web
crawling
Web
Indexing
Query
Search
Index
word1 doc1, doc23, doc4,…
word2 doc54,doc23
word3 doc1,doc7,d1oc9,doc5...
word4 doc7,doc9,doc10
…
115. Software Architecture
School of Computer Science University of Oviedo
MapReduce - Inverted index
d1 a b
d2 a c a
d3 a c
d2
d1
d2
a
b
c
a d1
b d1
a d2
c d2
a d2
a d3
c d3
mapper
mapper
mapper
a d1 d2 d2 d3 reducer
reducer
reducer
b d1
c d2 d3
Mezcla
y
ordena
MapReduce
// return each word with its document
mapper(d,ps) {
for each p in ps:
emit (p, d)
}
// sort list of documents
reducer(p,ds) {
ds1 = sort(ds)
emit (p, ds1)
}
d1 d3
d3
116. Software Architecture
School of Computer Science University of Oviedo
MapReduce - Popular songs
Given the listening logs of a music server, obtain
the number of times that each song is listened
Inspired by last.fm
2/3/2010 9:41 Ana C1
2/3/2010 9:42 Dani C2
2/3/2010 9:44 Ana C2
2/3/2010 10:01 Luis C1
2/3/2010 10:10 Ana C3
2/3/2010 10:15 Ana C2
2/3/2010 10:20 Dani C2
2/3/2010 10:21 Luis C4
2/3/2010 10:24 Luis C2
2/3/2010 10:26 Luis C4
2/3/2010 10:27 Ana C4
Analyzer
C1 2 listeners, 2 times
C2 3 listeners, 5 times
C3 2 listeners, 2 times
C4 2 listeners, 3 times
117. Software Architecture
School of Computer Science University of Oviedo
MapReduce: popular songs
9:41 A C1
reducer
reducer
reducer
reducer
MapReduce
C1 2
C2 3
C3 1
C4 2
9:42 D C2
9:44 A C2
10:01 L C1
10:10 A C3
10:15 A C2
10:20 D C2
10:21 L C4
10:24 L C2
10:26 L C4
2
5
3
10:27 A C4
mapper
mapper
mapper
mapper
C1 A
C2 D
C2 A
C1 L
C3 A
C2 A
C2 D
C4 L
C2 L
C4 L
C4 A
C1 A C
C2 D
C3 A
C4 L L
Mezcla
y
ordena
A A D L
A
118. Software Architecture
School of Computer Science University of Oviedo
MapReduce: Friends in common
Find the list of friends in common
Inspired by Facebook
Ana Dani Juan Luis
Dani Ana Juan Luis Mar
Juan Ana Dani Luis Mar
Luis Ana Dani Juan Mar
Mar Dani Juan Luis
Dani
Juan
Ana
Luis
Mar
If Ana visits Juan's page, system should show Dani, Luis
119. Software Architecture
School of Computer Science University of Oviedo
MapReduce: Friends in common
A D J L
D A J L M
J A D L M
reducer
reducer
reducer
MapReduce
L A D J M
M D J L
A D D J L
A J D J L
A L D J L
A D A J L M
D J
D L
D M
mapper
mapper
mapper
mapper
A J L M
A J L M
A J L M
A J A D L M
D J
J L
J M
A D L M
A D L M
A D L M
A L A D J M
D L
J L
L M
A D J M
A D J M
A D J M
D M D J L
mapper J M D J L
L M D J L
reducer
reducer
reducer
reducer
reducer
reducer
A D J L
A J D L
A L D J
D J A L M
D L A J M
D M J L
J L A D M
J M D L
L M D J
A D D J L A J L M
A J D J L A D L M
A L D J L A D J M
D J A J L M A D L M
D L A J L M A D J M
D M A J L M D J L
J L A D L M A D J M
J M A D L M D J L
L M A D J M D J L
Mezcla
y
ordena
120. Software Architecture
School of Computer Science University of Oviedo
MapReduce: Similarity
From the listening logs, find the number of songs
in common between 2 users
Inspired by Amazon (similar books), Facebook
(possible friends), etc.
It is usual to do several mapReduce jobs
121. Software Architecture
School of Computer Science University of Oviedo
MapReduce2
MapReduce: Similarity
9:41 A C1
9:42 D C2
9:44 A C2
10:01 L C1
10:10 A C3
10:15 A C2
10:20 D C2
10:21 L C4
10:24 L C2
10:26 L C4
10:27 A C4
A C C1, 1
A D C2, 2
A L C2, 1
A L
mapper
mapper
C4, 2
A C C1, 1
A D C2, 2
A L C2 1 C4 2
Mezcla
y
ordena
reducer
reducer
reducer
A C 1
A D 2
A L 3
C1 A C
C2 D
C3 A
A A D L
C4 L L
A
MapReduce1
122. Software Architecture
School of Computer Science University of Oviedo
MapReduce: Applications
Lots of applications:
Google, 2007, 20petabytes/day, around 100,000
mapreduce jobs/day
PageRank algorithm can be implemented as MapReduce
Success stories:
Automatic translation, similarity, sorting, ...
Other companies: last.fm, facebook, Yahoo!, twitter, etc.
123. Software Architecture
School of Computer Science University of Oviedo
MapReduce: Implementations
Google (internal)
Hadoop (open source)
CloudMapReduce (based on Amazon services)
Aster Data (SQL)
Greenplum (SQL)
Disco (Python/Erlang)
Holumbus (Haskell)
124. Software Architecture
School of Computer Science University of Oviedo
MapReduce: Libraries/languages
Hive (Hadoop): query language inspired by SQL
Pig (Hadoop): specific language that can define
data flows
Cascading: API that can specify distributed data
flows
Flume Java (Google)
Dryad (Microsoft)
125. Software Architecture
School of Computer Science University of Oviedo
Plugins
Microkernel
Reflection
Interpreters and DSL
Mobile code
Code on demand
126. Software Architecture
School of Computer Science University of Oviedo
Plugin
Plugin
Plugins
It allows to extend the system using plugins that
add new functionality
Runtime
engine
Base system
Plugin
127. Software Architecture
School of Computer Science University of Oviedo
Plugins
Elements
Base system:
System that allows plugins
Plugins: Components that can be added/removed
dynamically
Runtime engine:
Starts, localizes, initializes, executes, and stops plugins
Plugin
Plugin
Runtime
engine
Base system
Plugin
128. Software Architecture
School of Computer Science University of Oviedo
Plugins
Constraints
Runtime engine manages plugins
System can add/remove plugins
Some plugins can depend on other plugins
The plugin must declare dependencies and the
exported API
Plugin
Plugin
Runtime
engine
Base system
Plugin
129. Software Architecture
School of Computer Science University of Oviedo
Plugins
Advantages
Extensibility
Application can get new functionalities in some ways
that were not foreseen by the original developers
Personalization
Application can have a small kernel that is extended
on demand
Plugin
Plugin
Runtime
engine
Base system
Plugin
130. Software Architecture
School of Computer Science University of Oviedo
Plugins
Challenges
Consistency
Plugins must be added to the system in a sound way
Performance
Delay searching/configuring plugins
Security
Plugins made by third parties can compromise
security
Plugin management and dependencies
131. Software Architecture
School of Computer Science University of Oviedo
Plugins
Examples
Eclipse
Firefox
Technologies
Component systems: OSGi
132. Software Architecture
School of Computer Science University of Oviedo
Microkernel
Identify minimal functionality in a microkernel
Extra functionality is added using internal servers
External server handles communication with
other systems
Adapter Microkernel
Servidor
inSteerrvniodor
inItnetrenronal
server
External
server Client
System
133. Software Architecture
School of Computer Science University of Oviedo
Microkernel
Elements
Microkernel: Minimal functionality
Internal server: Extra functionality
External server: Offers external API
Client: External application
Adapter: Component that establish communication
with external server
Adapter Microkernel
Servidor
inSteerrvniodor
inItnetrenronal
server
External
server Client
System
134. Software Architecture
School of Computer Science University of Oviedo
Microkernel
Constraints:
Microkernel implements only minimal functionality
The rest of the functionality is implemented using
internal servers
Communication with clients by external servers
135. Software Architecture
School of Computer Science University of Oviedo
Microkernel
Advantages
Portability
It is only needed to port the kernel
Flexibility and extensibility
Adding new functionality with new internal servers
Security and reliability
Critical parts of the system are encapsulated
Errors in external parts don't affect the microkernel
136. Software Architecture
School of Computer Science University of Oviedo
Microkernel
Challenges
Performance
A monolithic can be more efficient
Design complexity
Identify components in the microkernel
It may be difficult to separate parts to internal servers
Unique point of failure
If microkernel fails, the whole system may fail
137. Software Architecture
School of Computer Science University of Oviedo
Microkernel
Applications
Operating systems
Games
Editors
138. Software Architecture
School of Computer Science University of Oviedo
Reflection
It allows to change the structure and behavior of
an application dynamically
Systems that can modify themselves
Metalevel
Base level
Meta-object
Protocol
139. Software Architecture
School of Computer Science University of Oviedo
Reflection
Elements
Base level: Implements application logic
Metalevel: Aspects that can be modified
Metaobject protocol: Interface that can modify the
metalevel
Metalevel
Base level
Meta-object
Protocol
140. Software Architecture
School of Computer Science University of Oviedo
Reflection
Constraints
Base level uses metalevel aspects for its behavior
At runtime, it is possible to modify the metalevel
using the metaobject protocol
Metalevel
Base level
Meta-object
Protocol
141. Software Architecture
School of Computer Science University of Oviedo
Reflection
Advantages
Flexibility
The system can adapt to changing conditions
It is not necessary to modify source code or to stop the
execution to change the running system
Metalevel
Base level
Meta-object
Protocol
142. Software Architecture
School of Computer Science University of Oviedo
Reflection
Challenges
Implementation
Not all the languages enable meta-programming
More difficult to combine with static type systems
Performance
It may be necessary to do some optimizations to limit
reflection
Security:
Consistency maintenance
Metalevel
Base level
Meta-object
Protocol
143. Software Architecture
School of Computer Science University of Oviedo
Reflection
Applications
Most dynamic languages support reflection
Scheme, CLOS, Ruby, Python, ....
Intelligent systems
Self-modifiable code
Metalevel
Base level
Meta-object
Protocol
144. Software Architecture
School of Computer Science University of Oviedo
Interpreters and DSLs
Include a domain specific language (DSL) that is
interpreted by the system
Interpreter
Context
DSL
program
Application
User
145. Software Architecture
School of Computer Science University of Oviedo
Interpreters and DSLs
Elements
Interpreter: Module that executes the program
Program: Written in the DSL
DSL can be designed so the end user can write programs
Context: Environment where the program is executed
Intérprete
Contexto
Programa
en DSL
Aplicación
Usuario
146. Software Architecture
School of Computer Science University of Oviedo
Interpreters and DSLs
Constraints
Interpreter runs the program interacting with the
context
It is necessary to define a DSL
Syntax (grammar, parsing,...)
Semantics (behavior)
147. Software Architecture
School of Computer Science University of Oviedo
Interpreters and DSLs
Advantages
Flexibility
Adapt application behavior to user needs
Usability
End users can write their own programs
Adaptability
Easy to adapt to unforeseen situations
148. Software Architecture
School of Computer Science University of Oviedo
Interpreters and DSLs
Challenges
Design of the DSL
Complexity of implementation
Interpreter
Separation of context/interpreter
Performance
Possible programs may be not optimal
Security
Handle wrong programs
149. Software Architecture
School of Computer Science University of Oviedo
Interpreters and DSLs
Variants:
Embedded DSLs
150. Software Architecture
School of Computer Science University of Oviedo
Embedded DSLs
Embedded DSLs
Domain specific languages that are embedded in
general purpose host languages
This technique is popular in soma languages like
Haskell, Ruby, Scala, etc.
Advantages:
Reuse of host language syntax
Access to libraries and IDEs of host language
Challenges
Separation between DSL and host language
151. Software Architecture
School of Computer Science University of Oviedo
Mobile code
Code that is transferred from one machine to another
System A sends a program to be run by system B
System B must contain an interpreter for the language
in which the program is written
Network
System
A
System
B
Interpreter
program
152. Software Architecture
School of Computer Science University of Oviedo
Mobile code
Elements
Interpreter: Runs the code
Program: Program that is transferred
Network: Transfers the program
Network
System
A
System
B
Interpreter
program
153. Software Architecture
School of Computer Science University of Oviedo
Mobile code
Constraints
The program must be run in the receiver system
The network protocol transfers the program
Network
System
A
System
B
Interpreter
program
154. Software Architecture
School of Computer Science University of Oviedo
Mobile code
Advantages
Flexibility and adaptability to new environments
Parallelism
Challenges
Complexity of implementation
Security
Network
System
A
System
B
Interpreter
program
155. Software Architecture
School of Computer Science University of Oviedo
Mobile code
Variants
Code on demand
Remote evaluation
Mobile Agents
156. Software Architecture
School of Computer Science University of Oviedo
Query Server
Network
Code on demand
Code is downloaded and run by the client
Combination between mobile code and client-server
Example:
ECMAScript
Client
Program
Interpreter
Answer
157. Software Architecture
School of Computer Science University of Oviedo
Code on demand
Elements
Client
Server
Code that is transferred from server to client
Constraints
Code resides or is generated by the server
It is transferred to the client when it asks for it
It is run by the client
Client must have an interpreter for the corresponding
language
158. Software Architecture
School of Computer Science University of Oviedo
Code on demand
Advantages
Improves user experience
Extensibility: Application can add new
functionalities that were not foreseen
No need to install or download a whole application
Always Beta
Adaptability to client environment
159. Software Architecture
School of Computer Science University of Oviedo
Code on demand
Challenges
Security
Coherence
It may be difficult to ensure an homogeneous behavior
in different types of clients
Client can even decide not to run the program
Reminder: Responsive design
160. Software Architecture
School of Computer Science University of Oviedo
Code on demand
Applications:
RIA (Rich Internet Applications)
HTML5 standardizes a lot of APIs
Improves coherence between clients
Variants
AJAX
Initially: Asynchronous Javascript and XML
The program that is running at the client side sends
asynchronous requests to the server without
stopping its running
161. Software Architecture
School of Computer Science University of Oviedo
Remote evaluation
System A sends program to system B to be run
and obtain its results
System
A
System
B
Query
Answer
(Result)
Interpreter
Program
Network
162. Software Architecture
School of Computer Science University of Oviedo
Remote evaluation
Elements
Sender: Does the query including the program
Receiver: Runs the program and returns the results
Constraints
Receiver runs the program
It must contain some interpreter of the program
language or the program could be in machine code
Network protocol transfers program and results
163. Software Architecture
School of Computer Science University of Oviedo
Remote evaluation
Advantages
Exploits capabilities of third parties
Computational capabilities, memory, resources, etc.
Challenges
Security
Untrusted code
Virus = variant of this style
Configuration
164. Software Architecture
School of Computer Science University of Oviedo
Remote evaluation
Example:
Volunteer computation
SETI@HOME
It was the basis for the BOINC system
Berkeley Open Infrastructure for Network Computing
Other projects: Folding@HOME, Predictor@Home,
AQUA@HOME, etc.
165. Software Architecture
School of Computer Science University of Oviedo
Mobile agents
Code and data can move from one machine to
another to be run
The process takes its state from machine to machine
Code can move autonomously
Systema B
Program Interpreter
System A
Program Interpreter
166. Software Architecture
School of Computer Science University of Oviedo
Mobile agents
Elements
Mobile agent: Program that travels and is run from
one machine or another autonomously
System: Execution environment where the mobile
agents are run
Network protocol: transfers state between agents
Systema B
Program Interpreter
System A
Program Interpreter
167. Software Architecture
School of Computer Science University of Oviedo
Mobile agents
Constraints
Systems host and run mobile agents
Mobile agents can decide to change its running from
one system to another
They can communicate with other agents
Systema B
Program Interpreter
System A
Program Interpreter
168. Software Architecture
School of Computer Science University of Oviedo
Mobile agents
Advantages
It can reduce network traffic
Code blocks that are run are transmitted
Implicit parallelism
Fault tolerance to network failures
Agents can be conceptually simple
Agent = independent unit of execution
It is possible to create mobile agent systems
Adaptability to environtment changes
Reactive and learning systems
169. Software Architecture
School of Computer Science University of Oviedo
Mobile agents
Challenges
Complexity of configuration
Security
Malicious or incorrect code
Systema B
Program Interpreter
System A
Program Interpreter
170. Software Architecture
School of Computer Science University of Oviedo
Mobile agents
Applications
Information retrieval
Web crawlers
Peer-to-peer systems
Telecommunications
Remote control and monitoring
Systems:
JADE (Java Agent DEvelopment framework)
IBM Aglets