Boost PC performance: How more available memory can improve productivity
Decomposing applications for deployability and scalability (cfopentour india)
1. Decomposing applications for
deployability and scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
crichardson@vmware.com
http://plainoldobjects.com/
1
2. Presentation goal
How decomposing
applications improves
deployability and
scalability
2
3. vmc push About-Chris
Developer Advocate for
CloudFoundry.com
Signup at http://cloudfoundry.com
cfopentour2012
3
6. Traditional web application architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
InventoryService Database
Shipping
Service
Simple to Tomcat
develop
test
deploy
scale 6
7. But there are problems with
a monolithic architecture
7
8. Users expect a rich, dynamic
and interactive experience
h
oug
en
ood
’tg
HTTP Request
isn
ure
ect
Java Web
Browser
it
HTML/Javascript Application
Ia rch
ty le U
s
Old
Real-time web ≅ NodeJS
8
9. Obstacle to frequent deployments
§Need to redeploy everything to change one component
§Interrupts long running background (e.g. Quartz) jobs
§Increases risk of failure
Fear of change
§Updates will happen less often
§e.g. Makes A/B testing UI really difficult 9
17. The scale cube
Y axis -
functional
decomposition
Scale by
im ing
splitting
g s on
r
ila
tin iti
different things
lit art
p
gs y ata
th ale - d
sp
i s
ax
in b
Z
X axis - horizontal
Sc
duplication
17
18. Y-axis scaling - application level
WAR
StoreFrontUI
Accounting
Service
InventoryService
Shipping
Service
18
19. Y-axis scaling - application level
accounting web application
Accounting
Service
Store front web application inventory web application
InventoryService
StoreFrontUI
shipping web application
Shipping
Service
Apply X axis cloning and/or Z axis partitioning to each service 19
20. Partitioning strategies
§Partition by verb, e.g. shipping service
§Partition by noun, e.g. inventory service
§Single Responsibility Principle
§Unix utilities - do one focussed thing well
Something of an art
20
21. Real world examples
http://techblog.netflix.com/
Between 100-150 services are accessed to build a
page.
http://highscalability.com/amazon-architecture
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
21
25. When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful 25
26. But there are many benefits
§Scales development: develop, deploy and scale
each service independently
§Update UI independently
§Improves fault isolation
§Eliminates long-term commitment to a single
technology stack
Modular, polyglot, multi-
framework applications
26
27. Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving
27
28. If services are small...
§Regularly rewrite using a better technology stack
§Adapt system to changing requirements and
better technology without a total rewrite
§Pick the best developers rather than best <pick
a language> developers polyglot culture
Fred George
“Developer Anarchy”
28
32. Can we build software systems
with these characteristics?
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf
http://dreamsongs.com/Files/WhitherSoftware.pdf
32
34. Inter-service communication options
§Synchronous HTTP asynchronous AMQP
§Formats: JSON, XML, Protocol Buffers, Thrift, ...
§Even via the database
Asynchronous is preferred
JSON is fashionable but binary
format is more efficient
34
35. Asynchronous message-based communication
wgrus-billing.war
Accounting
Service
wgrus-store.war wgrus-inventory.war
RabbitMQ
StoreFrontUI (Message InventoryService MySQL
Broker)
wgrus-shipping.war
ShippingService
35
36. Benefits
§Decouples caller from server
§Caller unaware of server’s coordinates (URL)
§Message broker buffers message when server is
down/slow
36
42. Using Akka futures
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ... Two calls execute in parallel
val future = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
And then invokes D
} yield d
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html Get the result of D
42
43. Spring Integration
§Implements EAI patterns
§Provides the building blocks for
a pipes and filters architecture
§Enables development of
application components that are
•loosely coupled
•insulated from messaging
infrastructure
§Messaging defined declaratively
43
44. Handling failure
Service A Service B
Errors happen
in distributed systems
44
45. About Netflix
> 1B API calls/day
1 API call average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
45
46. Use timeouts and retries
Never wait forever
Errors can be transient retry
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
46
47. Use per-dependency bounded thread pool
Service A
Runnable 1 Task 1
Runnable 2 Task 2
Service B
Runnable ... Task ...
bounded queue bounded thread pool
Fails fast if Limits number of
service is slow or down outstanding requests
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 47
48. Use a circuit breaker
High error rate stop calling temporarily
Down wait for it to come back up
Slow gives it a chance to recover
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 48
49. On failure
Return cached data
Avoid
Failing
Return default data
Fail fast
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 49
52. Apply the scale cube
§Modular, polyglot, and
scalable applications
§Services developed,
Y axis -
functional
decomposition
ng
deployed and scaled
independently
i
on
iti
rt
pa
ta
da
is-
ax
Z
X axis - horizontal duplication
52