Slaying Monoliths with Node and Docker

1. November 2016
Slaying Monoliths with
&
Yunong Xiao
Principal Engineer
@yunongx
http://yunong.io

3. #netflixeverywhere

4. Subscriber Growth
20M
33M
46M
59M
72M
85M
2011 2012 2013 2014 2015 2016

5. API Evolution

6. So You Want to Watch Netflix

7. So You Want to Watch Netflix

8. Watch Anywhere

9. In The Beginning…

10. Java Web Server
❖ Java based web server
❖ Renders UI
❖ Accesses data
❖ Individual clients for each service
❖ Different behavior for each client
Java Web server
Route A
Route B
Route C
Route D
Route N
…
Client
Library A
Client
Library B
Client
Library C
Client
Library N
…
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…

11. Spot the Monolith
Java Web server
Route A
Route B
Route C
Route D
Route N
…
Client
Library A
Client
Library B
Client
Library C
Client
Library N
…
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…
M
O
N
O
LITH

12. New Devices

13. API Evolution

14. Java Web Server
Java Web server
Route A
Route B
Route C
Route D
Route N
…
Client
Library A
Client
Library B
Client
Library C
Client
Library N
…
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…
M
O
N
O
LITH

15. REST API
REST API
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…

16. REST API
❖ Inflexible: waiting for weeks between API changes.
❖ Inefficient: multiple round trips
❖ Complex API: hard to maintain

17. API Evolution

18. Design for Innovation
❖ Rapid innovation
❖ More AB tests and devices
❖ Customized API
❖ Performance matters

19. REST API
REST API
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…

20. API.NEXT
API Server
Script A
Script B
Script C
Script D
Script N
…
Client
Library A
Client
Library B
Client
Library C
Client
Library N
…
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…
M
O
N
O
LITH

21. Scale
❖ 42.5 billion hours watched in 2015
❖ “Massive” RPS: Billions/day
❖ 1000s of scripts active in prod, 10000s in test
❖ 100s of changes/day
❖ 100s of AB tests with many variants/test

22. All Scripts Live in One Process
❖ Vertical Scale: Running out of headroom
❖ Memory
❖ I/O
❖ Instance cost: Largest instances $ can buy

23. HappySad Together?
❖ Resource contention
❖ 1 bad script takes out everyone
❖ Conflicting dependencies

24. API Server
Script A
Script B
Script C
Script D
Script N
…
Client
Library A
Client
Library B
Client
Library C
Client
Library N
…
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…
Developer Ergonomics
UI Engineering Systems Engineering

25. API Evolution

26. Requirements
❖ Scalability
❖ Availability
❖ Developer productivity

27. Runtime Scalability & Availability
❖ Process isolation
❖ Separation of data access scripts and API
servers
❖ Reduce infrastructure costs
❖ Horizontally scalable architecture
❖ Faster startup times
❖ Immutable deployment artifacts

28. Developer Productivity
❖ JS to rule them all
❖ Run and debug scripts locally, set
breakpoints, step through code
❖ Fast, incremental builds
❖ As closely mirrors production as possible

29. API Evolution
API Server
Script A
Script B
Script C
Script D
Script N
…
Client
Library A
Client
Library B
Client
Library C
Client
Library N
…
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…
M
O
N
O
LITH

30. API Server
Script A
Script B
Script C
Script D
Script N
…
Client
Library A
Client
Library B
Client
Library C
Client
Library N
…
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…
Natural Separation
UI Engineering Systems Engineering

31. Next Generation Data Access API
TV
iOS
Android
Windows
Browsers
Remote
Service
Layer
Search
MAP
GPS
Playback
…
Clients Node API Edge API Backend Services

32. Node API Platform
❖ Set of JS data access scripts
❖ Running Node.js + restify
❖ Inside of a Docker
/browse
/search
/account
/signup
Unified
Remote
Service
Layer
/bootstrap
/search
/account
/login
Unified
Remote
Service
Layer

33. Evolutionary Traits
❖ Runtime platform
❖ Application management
❖ Container infrastructure
❖ Developer tools
“Production”

34. Evolutionary Traits
❖ Runtime platform
❖ Application management
❖ Container infrastructure
❖ Developer tools
“Production”

35. -Twitter
“A full-stack developer is one who can
add technical debt to any layer of the
application”

36. Aim: Paved Path for Data Access Apps
❖ Metrics
❖ Alerts
❖ Autoscaling
❖ Load balancing
❖ Discovery
❖ Analytics

37. Node Runtime: Platform as a Service
❖ Production ready Node platform
❖ Just bring JS business logic
❖ Everything else is “free”
❖ No servers/infrastructure to manage
nf-iso-
properties
Properties Discovery RPC
nf-eureka-
client
reactive-
datasource
Insight
nf-atlas-
client
bunyan-
suro
(data-
pipeline)
bunyan
(logging)
nf-salp
Web serverRuntime
reactive-
socket-lb
HTTP
Client

38. Evolutionary Traits
❖ Runtime platform
❖ Application management
❖ Container infrastructure
❖ Developer tools
“Production”

39. Aim: Simple App Management
❖ Versioning
❖ Deployment
❖ Operational Insights

40. Versioning: Current Problems
❖ APIs change all the time
❖ 100000s different versions
❖ 1000s live in prod

41. Versioning: Inconsistency
api.netflix.com/tvui/1469577600021
api.netflix.com/web/6dbd361
api.netflix.com/ios/1.3.2
api.netflix.com/android/1234
Build Timestamp
Git sha
App version
Integer

42. Aim: Consistent Versions & Reproducible
Builds

43. Solution: Use SemVer

44. Versioning: Node API Index

45. Routing
api.netflix.com/tvui/1469577600021
api.netflix.com/web/6dbd361
api.netflix.com/ios/1.3.2
api.netflix.com/android/1234
Build Timestamp
Git sha
App version
Integer

46. Problem: API Upgrades
api.netflix.com/ios/1.3.2 1.3.2
1.3.3
Path immutably baked into client

47. Solution: SemVer Routing
api.netflix.com/ios/^1.0.0
1.3.2
1.3.3
1.4.0
1.6.5
nq.netflix.com
api.netflix.com/ios/1.3.2
^1.0.0
^1.0.0
1.3.2 1.3.2

48. Operational Insights
❖ List and view deployed apps and
routes
❖ Deployment history
❖ Metrics: RPS, latency, errors, …
❖ Analytics

49. Generated Dashboards

50. Evolutionary Traits
❖ Runtime platform
❖ Application management
❖ Container infrastructure
❖ Developer tools
“Production”

51. Titus: Container Management & Scheduling
Fenzo

52. Evolutionary Traits
❖ Runtime platform
❖ Application management
❖ Container infrastructure
❖ Developer tools
“Production”

53. Aim: Developer Productivity
❖ Run and debug scripts locally
❖ Fast, incremental builds
❖ Local “prod” environment

54. Local Development: Builds are Slow
Build deps
Commit to
SCM
Document
JS NQ
Scripts
Build
Docker
Image
Tens of Minutes

55. Rapid Local Development: Debug in Seconds
Developer Laptop (Mac OSX)
Virtual Box (Linux)
Running Docker Host
Docker Server
Container
Running MyApp Image MyApp Image
MyApp scripts & config
NodeQuark Image
Prana Image
NodeJS Image
Ubuntu Image

56. Recap: Containers
❖ Process isolation
❖ Layered dependency management
❖ Portability across environments:
prod->test
❖ Fast deployment
❖ Single deployment artifact: Docker
image

57. Recap: Node.js
❖ JS everywhere: client & server
❖ Performant
❖ Lightweight & efficient: run
locally
❖ Non blocking
❖ Superb ecosystem (npm)
❖ Built for the web

58. Recap: Node Platform
❖ Developer productivity
❖ Fast incremental builds
❖ Run, debug, and test locally
❖ Local prod like environment
❖ Scalability & availability
❖ Monolith -> micro-services
❖ Process isolation: better availability
❖ Horizontally scalable architecture
❖ Immutable deployment artifacts
Unified
Remote
Service
Layer
Backend Service
A
Backend Service
B
Backend Service
C
Backend Service
N
…

59. Thanks!
❖ Interested? is hiring!
❖ @yunongx
❖ yunong@netflix.com
❖ yunong.io