Increasing Microservice Availability by teaching best practices to developer teams. Presentation given at South Bay Site Reliability Engineering (SRE) Meetup June 20th, 2016.

1. @jonahhorowitz
Production Ready Services
Increasing Microservice Availability
at Netflix

2. @jonahhorowitz
Netflix Architecture

3. @jonahhorowitz

4. @jonahhorowitz
Determining Microservice Availability
Service A Service B
?

5. @jonahhorowitz
Microservice Availability Report

6. @jonahhorowitz
Microservice Availability Report

7. @jonahhorowitz
Production Ready

8. @jonahhorowitz
Characteristics of a Production Ready Service
● Reliable
○ Chaos Monkey Enabled
○ Automated Deployment Pipelines
○ Automated Canary Analysis

9. @jonahhorowitz
Characteristics of a Production Ready Service
● Reliable
○ Chaos Monkey Enabled
○ Automated Deployment Pipelines
○ Automated Canary Analysis
● Scalable
○ Proactive autoscaling
○ Reactive autoscaling

10. @jonahhorowitz
Characteristics of a Production Ready Service
● Reliable
○ Chaos Monkey Enabled
○ Automated Deployment Pipelines
○ Automated Canary Analysis
● Scalable
○ Proactive autoscaling
○ Reactive autoscaling
● Performant
○ Automated Canary Analysis
○ Proper Instance Type
○ Well-Tuned GC
○ Well-Tuned connections, threads
(ezconfig)

11. @jonahhorowitz
Characteristics of a Production Ready Service
● Reliable
○ Chaos Monkey Enabled
○ Automated Deployment Pipelines
○ Automated Canary Analysis
● Scalable
○ Proactive autoscaling
○ Reactive autoscaling
● Performant
○ Automated Canary Analysis
○ Proper Instance Type
○ Well-Tuned GC
○ Well-Tuned connections, threads
(ezconfig)
● Monitored
○ High Quality Alerts
■ Upstream Failure %
■ Downstream Failure %
○ Dashboards
■ Monitoring Releases
■ Troubleshooting

12. @jonahhorowitz
Team Engagements

13. @jonahhorowitz
Drill Sergeant
Automated Checks
● Deployment Pipelines
● Operating System
● Canary Analysis
● Chaos Monkey
Manual Checks
● Alerts
● Dashboards
● Autoscaling Rules (for now)
● Java/Tomcat/Apache Tuning (for
now)

14. @jonahhorowitz
Jonah Horowitz
Senior Site Reliability Engineer
@jonahhorowitz
https://netflix.github.io/
https://jobs.netflix.com/
Velocity Ignite Talk tomorrow:
7:00pm – 8:30pm Santa Clara Convention Center