Presentation I gave at Nextbuild 2016. Gives an overview of how Consul can be used in microservice architecture. Accompanying examples and demo can be found here: https://github.com/josdirksen/next-build-consul

1. Service Discovery in a
Architecture
using
Microservice
Consul

2. Who Am I
Currently doing Devops, Scala stu
At Equeris, lean startup within Equens
Docker, Consul, Scala, Cloud and other buzzwords.
Email me at:
I write at:
Twitter:
jos.dirksen@gmail.com
http://www.smartjava.org
@josdirksen

3. Follow along
Sources, presentation etc:
Demo heavy session
https://github.com/josdirksen/next-build-consul

4. Microservices?

5. What are microservices?
Small, ne-grained easy to replace components.
Organized around capabilities.
Di erent languages and backends (whatever ts best).
Fault tolerant, resiliant, automated deployements.
"Small Autonomous services that work together",
Sam Newman

6. From three tier to Microservices

7. Running Microservices is hard
Where is my other service or database?
Am I healthy, is the other one healthy?
Where do I store con guration?
How do I handle redundancy and failover?
...
, says everyone"Distributed systems are hard"

8. Service
Discovery

9. Basic approach
Hardcoded IP Address or LookupDNS
Lookup is nice!
Requires managing names (con g les), DNS Server
How to handle failover?
DNS

10. Now with failover
Point to a loadbalancerDNS
Works nicely with !
How to check health and register services?
Programmatic access to LB?
DNS

11. What would be nice
Does : Lightweight ( , REST) support failover
Has exible and manages
Lookups DNS
Health checking Con guration

12. Consul
"Consul [..] provides an opinionated framework for
service discovery and eliminates the guess-work
and development e ort. Clients simply register
services and then perform discovery using a DNS
or HTTP interface. Other systems require a home-
rolled solution." - consul.io

13. Main Features
Service discovery through REST and DNS
Simple registration using REST API
Distributed KV store for con guration
Provides extensive health checking
All in one package

14. Good to know
Multi DC-ready
API for distributed locks
Easy HA Setup
Event system
Consul = Zookeeper + Nagios + DNSMasq +
Scriptings + ...

15. Consul Architecture

16. Service Registration Flow
1. Servicecalls Consul Agentwith registration message:
.
2. Agentcommunicates registration with Consul Server
3. Agentchecks health of Service.
4. If check succeeds mark as Healthy, if not mark as Unhealthy,
communicate results with Server
5. When a lookup for Serviceoccurs, only return Healthyservices
http://agent_host/v1/agent/service/register

17. Sample: registration message
{
"Name": "service1",
"address": "10.0.0.12",
"port": 8080,
"Check": {
"http": "http://10.0.0.12:8080/health",
"interval": "5s"
}
}
Send when a new service starts up
Check types: script, http, tcp, TTL, Docker

18. Sample: DNS Lookup
$dig @nb-consul.local backend-service.service.consul
; <<>> DiG 9.8.3-P1 <<>> @nb-consul.local backend-service.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27716
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;backend-service.service.consul. IN A
;; ANSWER SECTION:
backend-service.service.consul. 0 IN A 10.0.9.3
backend-service.service.consul. 0 IN A 10.0.9.2
backend-service.service.consul. 0 IN A 10.0.9.4
Consul provides a DNS Server
Works great with Docker ( : will show in Demo)teaser

19. Sample: REST Lookup
$ curl -s http://192.168.99.106:8500/v1/catalog/service/backend-service
[{
"Node": "cf2f293e423c",
"Address": "192.168.99.111",
"ServiceID": "backend-service",
"ServiceName": "backend-service",
"ServiceAddress": "10.0.9.2",
"ServicePort": 8080
},{
"Node": "072b4ea1abc1",
"Address": "192.168.99.112",
"ServiceID": "backend-service",
"ServiceName": "backend-service",
"ServiceAddress": "10.0.9.3",
"ServicePort": 8080
}]

20. DEMO

21. Closer look: DNS Lookup
In code:
resp, err := http.Get("http://backend-service:8081/")
if err != nil {
fmt.Println(err)
} else {
defer resp.Body.Close()
body, _ := ioutil.ReadAll(resp.Body)
w.Header().Set("Content-Type",resp.Header.Get("Content-Type"))
w.Write(body)
}
For docker:
dns: 192.168.99.106
dns_search: service.consul

22. Consul
Ecosystem

23. Consul template
Render template based on Consul state
Setup reverse proxy: Nginx, Apache, haproxy
global
daemon
maxconn {{key "service/haproxy/maxconn"}}
defaults
mode {{key "service/haproxy/mode"}}{{range ls "service/haproxy/timeouts"}}
timeout {{.Key}} {{.Value}}{{end}}
listen http-in
bind *:8000{{range service "release.web"}}
server {{.Node}} {{.Address}}:{{.Port}}{{end}}

24. Demo

25. Envconsul
"III. Con g Store con g in the environment",
http://12factor.net/
More settings >More complexity
$ envconsul
-consul demo.consul.io
-prefix redis/config
redis-server [opts...]
Vault: help in managing secrets

26. DEMO ENVCONSUL

28. Prometheus & Grafana

29. More information
links:
This presentation, sources and docker stu :
https://www.consul.io/
https://github.com/hashicorp/consul-template
https://github.com/hashicorp/envconsul
https://github.com/josdirksen/next-build-consul

30. Thank You!