CoreOS Meetup talk at New Relic

1. #nginx #nginxplus
Load Balancing Applications with NGINX in a
CoreOS Cluster
1
Kevin Jones - Engineer, NGINX, Inc. @webopsx
Michael Pleshakov - Engineer, NGINX, Inc. @plshkv

2. #nginx #nginxplus2
Links
https://goo.gl/DY0QIt

3. #nginx #nginxplus3
• Quick Overview of Our Deployment Plan
• Quick Overview of CoreOS, etcd and fleet
• Discuss Using etcd as a Service Discovery Tool
• Discuss Using fleet as Application Deployment Scheduler
• Discuss Using NGINX as a Software Load Balancer
• Tie It All Together With…. A Live Demo!
Our Objectives… wait CoreOS party bus?!

4. #nginx #nginxplus4
What Exactly is Service Discovery?
https://www.nginx.com/blog/service-discovery-in-a-microservices-architecture/
• Used for tracking dynamic assigned IP addresses
and port numbers of services
• Tracking credentials, protocols, version or
environment details of services

5. #nginx #nginxplus5
The Sidekick Service Discovery Model

6. #nginx #nginxplus6
CoreOS Clustered Deployment

7. #nginx #nginxplus7

8. #nginx #nginxplus8
• Open Source Project
• Easy to Setup
• Run Services as Containers
• Stable & Reliable Update System
• Secure
Why we like CoreOS…

9. #nginx #nginxplus9
• Built in Cluster
Management! (fleet)
• Built in Service
Discovery Tool! (etcd)
Most Importantly…

10. #nginx #nginxplus10
fleet

11. #nginx #nginxplus11
What is fleet?
• Present your CoreOS cluster as a single init system
• Schedule deployment units across a cluster
• Deploy containers on arbitrary hosts
• Distribute services as ephemeral units across a cluster
of CoreOS machines
• Maintain set number of instances and re-schedule on
failure
• Remote configuration using fleetctl

12. #nginx #nginxplus12
core@core-01 ~ $ fleetctl list-machines
MACHINE IP METADATA
20f5eff1... 172.17.8.103 -
23a36596... 172.17.8.102 -
6ee835cb... 172.17.8.101 -
fd546c18... 172.17.8.104 -
Use fleetctl To Manage Your Cluster
https://coreos.com/fleet/docs/latest/using-the-client.html

13. #nginx #nginxplus13
core@core-01 ~/unit-files $ cat backend@.service
[Unit]
Description=Backend Service
After=docker.service
Requires=docker.service
[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill backend
ExecStartPre=-/usr/bin/docker rm backend
ExecStartPre=/usr/bin/docker pull nginx
ExecStart=/usr/bin/docker run --name backend -p 8080:80 nginxdemos/hello
ExecStop=/usr/bin/docker stop backend
[X-Fleat]
Conflicts=backend@*.service
Unit Files
https://coreos.com/fleet/docs/latest/launching-containers-fleet.html

14. #nginx #nginxplus14
Conflict - Tells fleet not to schedule more than one Unit on the same machine.
EnvironmentFile - Imports the CoreOS environment variables from a specified file.
ExecStart - Executes a command at the launch of the Unit.
MachineOf - Tells fleet to schedule the Unit on the the same server of the specified Unit.
BindsTo - Links the two Units so they stop at the same time.
ExecStop - Executes a command and the stop of the Unit.
After - Tells fleet to schedule deployment after the specified Unit.
Unit File Configurations Used…

15. #nginx #nginxplus15

16. #nginx #nginxplus
What is etcd?
16
• Reliable distributed key/value storage
• Written in Go
• Simple interface (HTTP+JSON)
• Secure (SSL client cert auth)
• Fast
https://github.com/coreos/etcd

17. #nginx #nginxplus17
core@core-01 ~/unit-files $ etcdctl ls /services/backend
/services/backend/172.17.8.104:8080
/services/backend/172.17.8.103:8080
/services/backend/172.17.8.102:8080
Use etcdctl To Manage Your Key Store

18. #nginx #nginxplus18
Or…. A REST API!! YAY!
core@core-01 ~/unit-files $ curl 127.0.0.1:2379/v2/keys/services/backend
{"action":"get","node":{"key":"/services/backend","dir":true,"nodes":
[{"key":"/services/backend/
172.17.8.104:8080","value":"server","expiration":"2016-02-27T22:28:24.021550
862Z","ttl":72,"modifiedIndex":4438,"createdIndex":4438},{"key":"/services/
backend/
172.17.8.103:8080","value":"server","expiration":"2016-02-27T22:28:29.225960
364Z","ttl":77,"modifiedIndex":4463,"createdIndex":4463},{"key":"/services/
backend/
172.17.8.102:8080","value":"server","expiration":"2016-02-27T22:29:12.812515
389Z","ttl":120,"modifiedIndex":4569,"createdIndex":4569}],"modifiedIndex":
1010,"createdIndex":1010}}

19. #nginx #nginxplus19

20. #nginx #nginxplus20
confd - http://www.confd.io/
Method 1: Configuration Management
confd is a lightweight configuration management tool focused on:
• keeping local configuration files up-to-date using data stored in etcd,
consul, dynamodb, redis, vault, zookeeper or env vars and processing
template resources.
• reloading applications to pick up new config file changes

21. #nginx #nginxplus21
Method 2: NGINX Plus API
core@core-01 ~/unit-files $ curl 127.0.0.1:8081/upstream_conf?upstream=backend
server 172.17.8.103:8080; # id=2
server 172.17.8.102:8080; # id=7
server 172.17.8.104:8080; # id=8
Dynamic Reconfiguration API (upsteam_conf) - HTTP based API to manage NGINX upstream
servers

22. #nginx #nginxplus22
How Can NGINX Plus Help?
Active Health Checks - Ability to perform regular expression match against the body of the
response, specific HTTP status code and specific header response
location @healthcheck {
internal;
proxy_pass http://backend;
proxy_connect_timeout 1s;
proxy_read_timeout 1s;
health_check interval=1s;
}

23. #nginx #nginxplus23
Well How About Open Source NGINX?
Passive Health Checks - If the response from a particular server fails with an error, NGINX will
mark this server as failed, and will try to avoid selecting this server for subsequent inbound
requests for a while.
• fail_timeout - Sets the time during which the specified number of failed attempts should
happen and still consider the server unavailable. In other words, the server is unavailable for
the interval set by fail_timeout. (default 10 seconds)
• max_fails - sets the number of failed attempts that should happen during the specified time
to still consider the server unavailable. (default 1)

24. #nginx #nginxplus
Live Demo!
24

25. #nginx #nginxplus25
https://goo.gl/DY0QIt
Thank you for coming!