Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.
In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.
Read articles based on the presentation here: http://plainoldobjects.com/2012/05/31/decomposing-applications-for-deployability-and-scalability-part-1/
Ensuring Technical Readiness For Copilot in Microsoft 365
Decomposing applications for deployability and scalability #gluecon
1. Decomposing applications for
deployability and scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
crichardson@vmware.com
1
2. Presentation goal
How decomposing
applications improves
deployability and scalability
and
How Cloud Foundry helps
2
11. Traditional web application architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
InventoryService Database
Shipping
Service
Simple to Tomcat
develop
test
deploy
scale 11
13. Users expect a rich, dynamic and
interactive experience on mobile
devices and desktop
h
oug
d en
oo
’t g
HTTP Request
e isn
r
ctu
Java Web
Browser
e
hit
HTML/Javascript Application
c
I ar
ty le U
s
Old
Real-time web ≅ NodeJS
13
14. Obstacle to frequent deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often
e.g. Makes A/B testing UI really difficult 14
16. Scaling development
WAR
StoreFrontUI
Accounting
Scalable
!=
InventoryService development
Shipping
Forces teams to synchronize development efforts
Teams need to coordinate updates
16
17. Long-term commitment to a single technology stack
Let’s suppose you go with the JVM
• Some polyglot support
• Many (but not all) JVM languages interoperate easily
• e.g. Scala, Groovy modules within a Java application
• But you can’t use non-JVM languages
Application depends on a particular:
• Framework - e.g. Spring or Java EE
• Container - e.g. Tomcat
Switching technology stack touches entire application
• Painful
• Rarely done
17
18. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
18
19. 3 dimensions to scaling
Y axis -
functional
decomposition
Scale by
s
ng
splitting
ila g
n
hi
g s oni
rt
different things
iti
rt
im
pa
ta
in
da
ittl
s-
sp
i
ax
by
Z
ale
X axis - horizontal duplication
Sc
Scale by cloning
19
20. Y axis scaling - functional partitioning
Splits monolithic application into a set of services
Each service implements a set of related functionality
Partitioning schemes:
• Partition functionality by noun or by verb
• Single Responsibility Principle
• Unix utilities - do one focussed thing well
20
21. Y axis scaling - application level
billing web application
Accounting
Service
Store front web application inventory web application
Store Front InventoryService
shipping web application
ShippingService
Apply X axis cloning and/or Z axis partitioning to each service
21
22. Real world examples
http://techblog.netflix.com/
Between 100-150 services are accessed to
build a page.
http://highscalability.com/amazon-architecture
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
22
23. There are drawbacks
Complexity:
• Architectural
• Development
• Deployment
• Application management
Deciding when to use it
• In the beginning: you don’t need it and it will slow you down
• When you do need it: refactoring existing code is painful
See Steve Yegge’s Google
Platforms Rant re Amazon.com
23
24. But there are many benefits
Scales development: focussed two pizza devops teams
Deploy services independently
Scale services independently
Improves fault isolation
Enforces well defined interfaces between components
Eliminates long-term commitment to a single technology
stack
Modular, polyglot applications
24
25. Two levels of architecture
System level
• Defines the inter-service glue: interfaces and
communication mechanisms
• Slow changing
Versus
Service level
• Defines the internal architecture of each service
• Far fewer constraints on technology
• Each service could use a different technology stack
• Rapid evolving
25
26. If services are small...
Regularly rewrite using a better technology stack
Pick the best developers rather than best <pick a
language> developers polyglot culture
Adapt system to changing requirements and better
technology without a total rewrite
Fred George
“Developer Anarchy”
26
27. Moreover: you are not the same you ...
Cell lifetimes: Can we build software
• hours - some white blood cells systems with these
characteristics?
• days - stomach lining cells
• years - bone cells Too much technical debt
• lifetime - brain cells component death?
50 to 70 billion of your cells die each day
Yet you (the system) remains intact
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf
http://dreamsongs.com/Files/WhitherSoftware.pdf
27
28. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
28
29. Inter-service communication options
Multiple collaborating services need a communication
mechanism
Many choices:
• Synchronous asynchronous
• Transports: HTTP, AMQP, ...
• Formats: JSON, XML, Protocol Buffers, Thrift, ...
• Even via the database
Distributed application error handling strategies
Asynchronous is preferred
JSON is fashionable but binary
format is more efficient
29
30. Asynchronous message-based communication
wgrus-billing.war
Accounting
Service
wgrus-store.war
RabbitMQ wgrus-inventory.war
StoreFrontUI (Message
MySQL
Broker) Widget
InventoryService
wgrus-inventory.war
InventoryService
30
31. Benefits and drawbacks
Benefits
• Decouples caller from server
• Caller unaware of server’s coordinates (URL)
• Message broker buffers message when server is down/slow
Drawbacks
• Additional complexity of message broker
• RPC using messaging is more complex
31
33. Composable futures
Problem:
• Service A needs to call services B and C and then D
• Makes sense to call B and C parallel
• Yet most concurrency APIs are low-level, error-prone etc
Solution:
• Use Akka composable futures = really nice abstraction
val futureB = callB() Two calls execute in parallel
val futureC = callC()
val futureD = for {
b <- futureB.mapTo[SomeType]
c <- futureC.mapTo[SomeType]
d <- callD(b, c) And then invokes D
} yield d
val result = Await.result(futureD, 1 second).
asInstanceOf[SomeOtherType]
Get the result of D
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
http://en.wikipedia.org/wiki/Futures_and_promises
33
34. Spring Integration
Builds on Spring framework
Implements EAI patterns = high-level of abstraction for
building message based applications
Provides the building blocks for a pipes and filters
architecture
• Pipes = message channels
• Filters = endpoints that filter, transform, route messages and
integrate with external messaging infrastructure
Enables development of components that are
• loosely coupled
• insulated from underlying messaging infrastructure
34
35. Handling failure
Errors happen (especially in distributed systems)
Use timeouts and retries
• Never wait forever
• Some errors are transient so retry
Use per-dependency bounded thread pool with bounded queue
• Limits number of outstanding requests
• Fails fast if service is slow or down
Use circuit breaker
• High rate of errors stop calling temporarily
• Avoids calling service that has issues
On failure
• Returned cached or default data
• Invoke custom error handler
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
35
36. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
36
38. NodeJS is the fashionable technology
Many JavaScript client frameworks have a NodeJS counterpart
e.g. socket.io
38
39. NodeJS isn’t the only game in town
JVM-based http://vertx.io/
39
40. A modern web application
Service 1
RESTful WS Node JS
Browser
HTML 5 Server Application
Application Service 2
Events
Socket.io Socket.io
client server
...
40
41. NodeJS - using RESTful WS and AMQP
REST
Service
REST
Requests
Node JS
Events
socket.io
AMQP AMQP
RabbitMQ Service
41
42. Updating the UI is easy
Update the UI independently of rest of system
Easily run A/B tests
Enables fast iteration of the UI
http://theleanstartup.com/principles
42
43. But coordination with backend changes required
Let’s imagine that you are deploying an advanced search
feature:
• Enhancements to search service
• Enhancements to UI
Before
• Deploy new war
Now:
• Some coordination required
• Deploy updated backend service
• Deploy updated NodeJS and browser code
• Enable feature using feature switch
43
44. Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
44
46. Developing modular apps is more difficult
Many more moving parts to manage
• Platform services: SQL, NoSQL, RabbitMQ
• Application services: your code
Who is going to setup the environments:
• the developer sandbox?
• ...
• QA environments?
But Cloud Foundry
helps...
46
47. Easy polyglot application deployment
and service provisioning
OSS community
vFabric
Postgres
Ap
p lica
Data Private
'o
Services Clouds
n
S
erv
ice
In
vFabric
ter
fac
RabbitMQTM Msg Public
e
Services Clouds
Micro
Other Clouds
Services
Additional partners
services …
48. Creating a platform service instance
$ vmc create-service mysql --name mysql1
Creating Service: OK
$ vmc services
......
=========== Provisioned Services ============
+-------------+---------+
| Name | Service |
+-------------+---------+
| mysql1 | mysql |
+-------------+---------+
50. Multi-application manifest - part 2
store/target: Path to application
name: store
url: cer-store.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services: Required platform services
si-mongo:
type: :mongodb
si-rabbit:
type: :rabbitmq
50
51. One command to create platform services and deploy
application
$ vmc push
Would you like to deploy from the current directory? [Yn]:
Pushing application 'inventory'...
Creating Application: OK
Creating Service [si-rabbit]: OK
Binding Service [si-rabbit]: OK
Creating Service [si-mongo]: OK
Binding Service [si-mongo]: OK
Creating Service [si-redis]: OK
Binding Service [si-redis]: OK
Uploading Application:
Checking for available resources: OK
Processing resources: OK
Packing application: OK vmc push:
Uploading (12K): OK
Push Status: OK •Reads the manifest file
Staging Application 'inventory': OK
Starting Application 'inventory': OK •Creates the required platform services
Pushing application 'store'...
Creating Application: OK
•Deploys all the applications
Binding Service [si-mongo]: OK
Binding Service [si-rabbit]: OK
Uploading Application:
Checking for available resources: OK
Processing resources: OK
Packing application: OK
Uploading (5K): OK
Push Status: OK
Staging Application 'store': OK
51
Starting Application 'store': ...
52. Micro Cloud Foundry: new developer sandbox
App Instances Services
Open source Platform as a Service project
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
52
53. Using Caldecott…
$ vmc tunnel
1: mysql-135e0
2: mysql1
Which service to tunnel to?: 2
Password: ********
Stopping Application: OK
Redeploying tunnel application 'caldecott'.
Uploading Application:
Checking for available resources: OK
Packing application: OK
Uploading (1K): OK
Push Status: OK
Binding Service [mysql1]: OK
Staging Application: OK
Starting Application: OK
Getting tunnel connection info: OK
Service connection info:
username : uMe6Apgw00AhS
password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.
1: none
2: mysql
Which client would you like to start?: 2
54. …Using Caldecott
Launching 'mysql --protocol=TCP --host=localhost --port=10000 --
user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ
d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 10944342
Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL),
Release 12.5, Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights
reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input
statement.
mysql>
55. Running JUnit test with Caldecott
Configure your test code to use port + connection info
55
56. Summary
Monolithic applications are simple to develop and deploy
BUT applying the scale cube
Decomposes your application into services
Enables scaling for transactions and data volumes
Tackles application complexity
Enables scaling for development
Enables frequent, independent deployments
Make it easy to leverage other technologies
AND
Cloud Foundry simplifies the development and deployment of
modular, service-based applications
56