I would like to speak about what I am actually doing at InfluxData. Sharing with you some ideas about how an orchestrator should work. We will start from a bit of history about distributed system, containers, runtime and so on. Hoping to have a good chat about the future of scheduling and orchestrator.

1. InfluxCloud
Craft Container Orchestrator
Milan 21st Sept 2017 at the CNCFItaly

2. Gianluca Arbezzano
SRE at InfluxData
@gianarb
gianluca@influxdb.com

3. I am building

5. InfluxCloud
The as a service solution to get a fully managed
TICK stack

6. Very High level
¨ Cloud Provider (AWS)
¨ Containers (Docker)
¨ CoreOS
¨ Etcd
¨ Some code (Golang)
¨ A lot of code actually (Golang)

7. Master Master
Master
us-east-1 eu-west-2

8. Master Master
Master
us-east-1 eu-west-2
Can I have a new
cluster in us-east-1?

9. Master Master
Master
us-east-1 eu-west-2
Data
Meta
sg-21412
E
L
B
E
L
B
Route53
Web
Worker

10. Master Master
Master
us-east-1 eu-west-2
Data
Meta
sg-21412
E
L
B
E
L
B
Route53
Web
Worker
sg-21412
E
L
B
Can I have a new
cluster in
us-west-2?

11. Data
Meta
sg-21412
E
L
B
E
L
B
Route53
Worker
E
L
B
Route53
Addon
etcd +
skydns

12. Master
We use cloud-init to start the worker.
The worker ask for the cluster
configuration in order to start the right
containers

13. We use cloud-init to start the
worker.

14. “The worker ask for the cluster
configuration in order to start the right
containers

15. Cloud Provider (AWS)
¨ EC2
¨ EBS
¨ ELB. Moving to ELBv2
¨ Security Groups, VPC bla bla
¨ Route53
¨ Massive use of CloudInit

16. This looks like a EC2/Container
orchestrator/scheduler
Infrakit, k8s, Swarm, Mesos, Openshift...

17. A good amount of Go and testing
Infrakit, k8s, Swarm, Mesos, Openshift...

18. Why?
¨ We have the right use case
¨ Make stuff simple for support team
¨ Write the right amount of code
¨ Build a use case for the TICK Stack

19. The History of life
It’s time for Cloud v2

20. Advanced networking ideas about v2
Switch from Security groups isolation to
Security Groups plus subnet
Master Master
Master
us-east-1
sg-21412
E
L
B
Web

21. Interface between Master and Slave ideas about v2
Master
CRI from k8s can be a good gRPC
interface between Master and Worker
● Worker will be simpler (it will get ONLY
what to do directly from the master)
● We can reuse cri-o, docker-cri,
contaierd-cri what ever! (runtime are
bored!

22. ¨ Join worker and master under the
same repository to make it easy to test
and improve.
¨ First step for a lot of things like
¨ Advantage container networking.
¨ Scheduling based on cluster
topology and stull file labels.
¨ If you don’t need Raft you are
wrong!
Raft and membership ideas about v2
image credits Martin Kleppmann

23. I have no idea yet. That’s why I am here!
¨ Efficient and easy to manage in-cluster
networking.
¨ More secure with something like BPF
scanning or Cilium.
Advanced container networking ideas about v2

24. Stability and HealthCheck ideas about v2
Gossip protocol based on SWIM paper.
¨ Fast propagation
¨ Small and constant network load
¨ We need to check instances and
containers from outside and inside
https://youtu.be/CDQaqiRhTtk?t=12m15s

25. ¨ Raft should allow us to distribute the requests to the right datacenter (the one where
the cluster is).
¨ Will improve isolation and security.
¨ It will make the code a lot easy.
Master Master
us-east-1 eu-west-2
Master
Master Master
Master
Improve multi datacenters distribution ideas about v2

26. ¨ Recover EC2 from failure
¨ Recover Containers
¨ Secure and Stable update/termination from S1 to S2
Achieve and maintain desired state ideas about v2

27. Monitoring - that’s easy!

28. ¨ In InfluxData we think that distributed
tracing can be the right decision
¨ We can trace our a requests moves
between:
¨ Frontend
¨ Queue System
¨ Master and Slave
¨ AWS
¨ Runtime
¨ We would like to validate if the tracing
model can work with InfluxDB and if it can
be really helpful. We are the guinea pig.
Distributed Tracing ideas about v2

30. Thank you
gianluca@influxdb.com
@gianarb