Latest version of the decomposing applications for scalability and deployability talk. From Cloud Foundry July/August India tour.

1. Decomposing applications for
deployability and scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
crichardson@vmware.com
http://plainoldobjects.com/
1

2. Presentation goal
How decomposing
applications improves
deployability and scalability
and
How Cloud Foundry helps
2

3. About Chris
3

4. dsply ‘About Chris’;
RPG
4

5. (About Chris)
5

6. About Chris()
6

7. About Chris
7

8. About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
8

9. vmc push About-Chris
Developer Advocate for
CloudFoundry.com
Signup at http://cloudfoundry.com
9

10. Agenda
 The (sometimes evil) monolith
 Decomposing applications into services
 How do services communicate?
 Presentation layer design
 How Cloud Foundry helps
10

11. Let’s imagine you are building an e-
commerce application
11

12. Traditional web application architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
InventoryService Database
Shipping
Service
Simple to Tomcat
develop
test
deploy
scale 12

13. But there are problems
13

14. Users expect a rich, dynamic and
interactive experience on mobile
devices and desktop
h
oug
d en
oo
’t g
HTTP Request
e isn
r
ctu
Java Web
Browser
e
hit
HTML/Javascript Application
c
I ar
ty le U
s
Old
Real-time web ≅ NodeJS
14

15. Obstacle to frequent deployments
 Need to redeploy everything to change one component
 Interrupts long running background (e.g. Quartz) jobs
 Increases risk of failure
Fear of change
 Updates will happen less often
 e.g. Makes A/B testing UI really difficult 15

16. Overloads your IDE and container
Slows down development 16

17. Obstacle to scaling development
WAR
StoreFrontUI
Accounting
Scalable
!=
InventoryService development
Shipping
 Forces teams to synchronize development efforts
 Teams need to coordinate updates
17

18. Requires long-term commitment to a technology stack
 Switching technology stack touches entire application
 Painful and rarely done
 Makes it difficult to incrementally adopt newer technologies
 You end up being stuck with the technology you started with
18

19. Agenda
 The (sometimes evil) monolith
 Decomposing applications into services
 How do services communicate?
 Presentation layer design
 How Cloud Foundry helps
19

20. 3 dimensions to scaling
Y axis -
functional
decomposition
Scale by
s
ng
splitting
ila g
n
hi
g s oni
rt
different things
iti
rt
im
pa
ta
in
da
ittl
s-
sp
i
ax
by
Z
ale
X axis - horizontal duplication
Sc
Scale by cloning
20

21. Y axis scaling - functional partitioning
 Splits monolithic application into a set of services
 Each service implements a set of related functionality
 Partitioning schemes:
• Partition functionality by noun or by verb
• Single Responsibility Principle
• Unix utilities - do one focussed thing well
Something of an art
21

22. Y axis scaling - application level
WAR
StoreFrontUI
Accounting
Service
InventoryService
Shipping
Service
22

23. Y axis scaling - application level
billing web application
Accounting
Service
Store front web application inventory web application
StoreFrontUI InventoryService
shipping web application
Shipping
Service
Apply X axis cloning and/or Z axis partitioning to each service 23

24. Real world examples
http://techblog.netflix.com/
Between 100-150 services are accessed to
build a page.
http://highscalability.com/amazon-architecture
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
24

25. There are drawbacks
 Complexity: development, deployment, etc.
 Multiple databases transaction management challenges
 Deciding when to use it
• In the beginning: you don’t need it and it will slow you down
• When you do need it: refactoring existing code is painful
See Steve Yegge’s Google
Platforms Rant re Amazon.com
25

26. But there are many benefits
 Scales development: develop, deploy and scale each service
independently
 Improves fault isolation
 Enforces well defined interfaces between components
 Eliminates long-term commitment to a single technology
stack
Modular, polyglot, multi-
framework applications
26

27. Two levels of architecture
 System level
• Defines the inter-service glue: interfaces and
communication mechanisms
• Slow changing
Versus
 Service level
• Defines the internal architecture of each service
• Far fewer constraints on technology
• Each service could use a different technology stack
• Pick the best tool for the job
• Rapid evolving
27

28. If services are small...
 Regularly rewrite using a better technology stack
 Adapt system to changing requirements and better
technology without a total rewrite
 Pick the best developers rather than best <pick a
language> developers polyglot culture
Fred George
“Developer Anarchy”
28

29. The human body as a system
29

30. 50 to 70 billion of your cells die
each day
30

31. Yet you (the system) remain you
31

32. Can we build software systems with
these characteristics?
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf
http://dreamsongs.com/Files/WhitherSoftware.pdf
32

33. Agenda
 The (sometimes evil) monolith
 Decomposing applications into services
 How do services communicate?
 Presentation layer design
 How Cloud Foundry helps
33

34. Inter-service communication options
 Multiple collaborating services need a communication
mechanism
 Many choices:
• Synchronous asynchronous
• Transports: HTTP, AMQP, ...
• Formats: JSON, XML, Protocol Buffers, Thrift, ...
• Even via the database
 Distributed application error handling strategies
Asynchronous is preferred
JSON is fashionable but binary
format is more efficient
34

35. Asynchronous message-based communication
wgrus-billing.war
Accounting
Service
wgrus-store.war wgrus-inventory.war
RabbitMQ
StoreFrontUI (Message InventoryService MySQL
Broker)
wgrus-shipping.war
ShippingService
35

36. Benefits and drawbacks
 Benefits
• Decouples caller from server
• Caller unaware of server’s coordinates (URL)
• Message broker buffers message when server is down/slow
 Drawbacks
• Additional complexity of message broker
• RPC using messaging is more complex
36

37. Writing code that calls services
37

38. Composable futures
 Problem:
• Service A needs to call services B and C and then D
• Makes sense to call B and C parallel
• Yet most concurrency APIs are low-level, error-prone etc
 Solution:
• Use Akka composable futures = really nice abstraction
val futureB = callB() Two calls execute in parallel
val futureC = callC()
val futureD = for {
b <- futureB.mapTo[SomeType]
c <- futureC.mapTo[SomeType]
d <- callD(b, c) And then invokes D
} yield d
val result = Await.result(futureD, 1 second).
asInstanceOf[SomeOtherType]
Get the result of D
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
http://en.wikipedia.org/wiki/Futures_and_promises
38

39. Spring Integration
 Builds on Spring framework
 Implements EAI patterns = high-level of abstraction for
building message based applications
 Provides the building blocks for a pipes and filters
architecture
• Pipes = message channels
• Filters = endpoints that filter, transform, route messages and
integrate with external messaging infrastructure
 Enables development of components that are
• loosely coupled
• insulated from underlying messaging infrastructure
39

40. Handling failure
Service A Service B
Errors happen
in distributed systems
40

41. Use timeouts and retries
Never wait forever
Errors can be transient retry
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
41

42. Use per-dependency bounded thread pool
Service A
Runnable 1 Task 1
Runnable 2 Task 2
Service B
Runnable Task ...
bounded queue bounded thread pool
Fails fast if Limits number of
service is slow or down outstanding requests
42
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

43. Use a circuit breaker
High error rate stop calling temporarily
Avoids calling service that has issues
Down wait for it to come back up
Slow gives it a chance to recover
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
43

44. On failure - don’t fail too
Return cached data
Return default data
Invoke custom error handler
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
44

45. Agenda
 The (sometimes evil) monolith
 Decomposing applications into services
 How do services communicate?
 Presentation layer design
 How Cloud Foundry helps
45

46. Modular application
Choice of presentation layer technology
+
Redeploy UI frequently/independently
46

47. NodeJS is the fashionable technology
Many JavaScript client frameworks have a NodeJS counterpart
e.g. socket.io
47

48. NodeJS example Handle
HTTP
Handle file
request
var http = require('http'); read
var fs = require("fs");
http.createServer(function (req, res) {
fs.readFile('somefile.txt', function (err, data) {
if (err) throw err;
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end(data);
});
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
48

49. NodeJS isn’t the only game in town
JVM-based http://vertx.io/
49

50. A modern web application
Service 1
RESTful WS Node JS
Browser
HTML 5 Server Application
Application Service 2
Events
Socket.io Socket.io
client server
...
50

51. NodeJS - using RESTful WS and AMQP
REST
Service
REST
Requests
Node JS
Events
socket.io
AMQP AMQP
RabbitMQ Service
51

52. Updating the UI is easy
 Update the UI independently of rest of system
 Easily run A/B tests
 Enables fast iteration of the UI
http://theleanstartup.com/principles
52

53. Agenda
 The (sometimes evil) monolith
 Decomposing applications into services
 How do services communicate?
 Presentation layer design
 How Cloud Foundry helps
53

54. Traditional tools: monolithic applications
54

55. Developing modular apps is more difficult
 Many more moving parts to manage
• Platform services: SQL, NoSQL, RabbitMQ
• Application services: your code
 Who is going to setup the environments:
• the developer sandbox?
• ...
• QA environments?
But Cloud Foundry
helps...
55

56. Easy polyglot application deployment
and service provisioning
OSS community
vFabric
Postgres
Ap
p lica
Data Private
'o
Services Clouds
n
S
erv
ice
In
vFabric
ter
fac
RabbitMQTM Msg Public
e
Services Clouds
Micro
Other Clouds
Services
Additional partners
services …

57. Creating a platform service instance
$ vmc create-service mysql --name mysql1
Creating Service: OK
$ vmc services
......
=========== Provisioned Services ============
+-------------+---------+
| Name | Service |
+-------------+---------+
| mysql1 | mysql |
+-------------+---------+

58. Multi-application manifest - part 1
---
applications:
inventory/target: Path to application
name: inventory
url: cer-inventory.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services:
Required platform services
si-rabbit:
type: :rabbitmq
si-mongo:
type: :mongodb
si-redis:
type: :redis
58

59. Multi-application manifest - part 2
store/target: Path to application
name: store
url: cer-store.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services: Required platform services
si-mongo:
type: :mongodb
si-rabbit:
type: :rabbitmq
59

60. One command to create platform services and deploy
application
$ vmc push
Would you like to deploy from the current directory? [Yn]:
Pushing application 'inventory'...
Creating Application: OK
Creating Service [si-rabbit]: OK
Binding Service [si-rabbit]: OK
Creating Service [si-mongo]: OK
Binding Service [si-mongo]: OK
Creating Service [si-redis]: OK
Binding Service [si-redis]: OK
Uploading Application:
Checking for available resources: OK
Processing resources: OK
Packing application: OK vmc push:
Uploading (12K): OK
Push Status: OK •Reads the manifest file
Staging Application 'inventory': OK
Starting Application 'inventory': OK •Creates the required platform services
Pushing application 'store'...
Creating Application: OK
•Deploys all the applications
Binding Service [si-mongo]: OK
Binding Service [si-rabbit]: OK
Uploading Application:
Checking for available resources: OK
Processing resources: OK
Packing application: OK
Uploading (5K): OK
Push Status: OK
Staging Application 'store': OK
60
Starting Application 'store': ...

61. Micro Cloud Foundry: new developer sandbox
App Instances Services
Open source Platform as a Service project
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
61

62. Using Caldecott to tunnel into your services
62

63. Caldecott = TCP over HTTP
native
native
protocol HTTP
Service Caldecott Caldecott protocol
Service
client Port gem application
NNN
Your computer Cloud Foundry
63

64. Using Caldecott…
$ vmc tunnel
1: mysql-135e0
2: mysql1
Which service to tunnel to?: 2
Password: ********
Stopping Application: OK
Redeploying tunnel application 'caldecott'.
Uploading Application:
Checking for available resources: OK
Packing application: OK
Uploading (1K): OK
Push Status: OK
Binding Service [mysql1]: OK
Staging Application: OK
Starting Application: OK
Getting tunnel connection info: OK
Service connection info:
username : uMe6Apgw00AhS
password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.
1: none
2: mysql
Which client would you like to start?: 2

65. …Using Caldecott
Launching 'mysql --protocol=TCP --host=localhost --port=10000 --
user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ
d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 10944342
Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL),
Release 12.5, Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights
reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input
statement.
mysql>

66. Running JUnit test with Caldecott
Configure your test code to use port + connection info
66

67. Summary
 Monolithic applications are simple to develop and deploy
BUT have significant drawbacks
Applying the scale cube
 Decomposes your application into services
 Enables scaling for transactions and data volumes
 Tackles application complexity
 Enables scaling for development
 Enables frequent, independent deployments
 Make it easy to leverage other technologies
AND
 Cloud Foundry simplifies the development and deployment of
modular, service-based applications 67

68. @crichardson crichardson@vmware.com
http://slideshare.net/chris.e.richardson/
Questions?
www.cloudfoundry.com