Presented at QCon NYC 2016.
The Netflix API is the front-door for almost all device/UI requests from 1000+ device types to the Netflix backends. It serves everything from movie and show recommendations, profile, sign-up, and A/B test related functionality, to bookmarks and licenses for playback.
Because all devices use this API, and because Netflix runs on devices of widely varying sizes and interaction models, it has served us well to enable a platform against which device teams write server-side scripts. Using Netflix as an example, the goal of this talk is to explore situations in which server-side scripting is a good solution for applications. I will describe our first approach, which uses Groovy scripts. I will detail how the scripts are uploaded and can make use of shared modules. This approach allows for high flexibility and performance as well as high developer velocity, at the expense of added risk of injecting scripts into running servers. I will then dive into a new approach that will isolate the scripts into their own containers without compromising the original goals and will allow teams to write scripts in node.js, a language that is more natural for them.
2. The risks of modifying running production servers
Problem identified: new
servers aren’t coming up
healthy!
Ugh! There’s a problem.
Errors from API are up.
13. Js
(mostly)
java
Client A
Client B
Client C
Client A
Client Y
Client Z
...
...
Netflix
Microservices
scripts
scripts
scripts
scripts
...
scripts
scripts
scripts
scripts
Network
boundary
API Server JVM
~700 active
Today’s architecture
groovy
19. Js
(mostly)
java
Client A
Client B
Client C
Client A
Client Y
Client Z
...
...
Netflix
Microservices
script
script
...
script
script
Network
boundary
API Server JVM
Resiliency in today’s system
Strong
resiliency
with Hystrix
What about
resiliency on
this side?
groovy
22. Js
(mostly)
java
Client A
Client B
Client C
Client A
Client Y
Client Z
...
...
Netflix
Microservices
script
script
...
script
script
Network
boundary
API Server JVM
1-2 years ago
few, small
scripts
fewer uploads
groovy
23. Js
(mostly)
java
Client A
Client B
Client C
Client A
Client Y
Client Z
...
...
Netflix
Microservices
script/app
script/app
script/app
script/app
...
script/app
script/app
script/app
script/app
Network
boundary
API Server JVM
Today
script/app
script/app
~700 more
complex
scripts/apps,
10-50 uploads
per day
groovy
30. Js
(mostly)
java
Client A
Client B
Client C
Client A
Client Y
Client Z
...
...
Netflix
Microservices
node script
node script
...
node script
node script
Network
boundary API Server JVM
The (near) future
node.js
process
isolation
38. Run-time debugging/optimization (today)
Js
(mostly)
java
Client A
Client B
Client C
Client A
Client Y
Client Z
...
...
Netflix
Microservices
script
script
script
script
...
script
script
script
script
Network
boundary
API Server JVM
script
script
Problems hard to
root cause, hard to
measure/optimize
performance
groovy
39. Script → API interaction (today)
API
device
server-side
script
device client
40. Script → API interaction (future)
API
device
server-side
script
48. Client A
Client B
Client C
Client E
Netflix
Microservices
node script
Network
boundary API Server JVM
First end-to-end implementation and shadow traffic
49. Client A
Client B
Client C
Client E
Netflix
Microservices
node script
Network
boundary API Server JVM
Problem isolation (ex: memory leak)
Memory leak makes
RSL blow up.
Clearer idea of
where the problem
is.
50. node.js
Client A
Client B
Client C
Client E
Netflix
Microservices
node script
Network
boundary API Server JVM
Problem isolation (ex: memory leak)
Same with node script.
52. Js
(mostly)
Client A
Client B
Client C
Client A
Client Y
Client Z
...
...
Netflix
Microservices
node script
node script
...
node script
node script
Network
boundary API Server JVM
node.js
How not to compromise what we’re good at
62. Js
(mostly)
java
Client A
Client B
Client C
Client A
Client Y
Client Z
...
...
Netflix
Microservices
node script
node script
...
node script
node script
Network
boundary API Server JVM
node.js
process
isolation