Making cloud native platform by kubernetes

1. Making cloud native
platform by kubernetes
the hard way
이어형
al.lee@linecorp.com

2. DEMO
http://nucleo.linecorp-dev.com

3. Nucleo

4. Nucleo - Cloud Native
Platform
Disposable
Continuous Delivery & Continuous Integration
Automation, DevOps
Events
Micro Service Architecture , Stateless , Container ,
Serverless Computing , Service Discovery

5. Closing the distance
Nucleo is a platform that help developers can
stay focused on development.
and Nucleo will take over the rest things such
as Resource pooling, Continuous delivery,
Rollback, Scaling, Auto scaling, Logging,
Metering, Alerting, DNS, SSL, Event, Service
Discovery...

6. Nucleo Architecture
https://goo.gl/AKpG8L

7. Nucleo Architecture

8. Kubernetes
Production-Grade Container Orchestration
Automated container deployment, scaling, and
management
https://kubernetes.io/

9. Read manual rst if you can
but, tl;dr
http://leoh0.github.io/images/kubernetes-
documents.pdf

10. So today I speak only about...
ingress
persistent volume
custom resource de nition

11. before speak ingress(L7)
service (L4) publishing types
ClusterIP <- internal ip
NodePort <- all nodes need to open that port
LoadBalancer <- need to cloud provider
ExternalName <- CNAME

12. ingress

13. auto con gured con g in
ingress nginx controller
upstream kube-system-dashboard-80 {
#(...configs...)
server 192.168.63.221:8080 max_fails=0;
server 192.168.68.26:8080 max_fails=0;
}
server {
server_name dashboard-kube-system-nucleo.local.com;
listen [::]:80;
location / {
#(...configs...)
proxy_pass http://kube-system-dashboard-80;
}
}

14. ingress node deploy types

15. ingress node
use hostnetwork for xed port or port-ranges
scheduling only for ingress controller pods

16. scheduling control pattern

17. nodeselector

18. taints and tolerations

19. nodeselector & taints and tolerations

20. how to make ingress node
1. setup ingress node rst just like other nodes
2. setup labels to ingress node (enable scheduling
ingress pods)
3. setup taints to ingress node (disable scheduling
other pods)
4. install nginx-ingress-controller by hostnetwork
daemonset with node selector and tolerations

21. persistent volume

22. volume
volume
persistent volume
persistent volume claim

24. rbd
rook (alpha stage) <- for develop
https://github.com/rook/rook
prepared ceph <- for production

25. rook

26. rook operator & agent

27. prepared ceph
controller manager doesn't have rbd command
use external storage provisioner
with system:controller:persistent-volume-binder
role

28. rbd provisioner

29. custom resource de nition
A built-in API that offers a simple way to create
custom resources .
ThirdPartyResource(TPR) ->
CustomResourceDe nition(CRD)
TPR is deprecated at k8s < 1.8.0
CRD is begin at k8s > 1.7.0

30. resources
A resource is an endpoint in the Kubernetes API
that stores a collection of API objects of a certain
kind.
DaemonSet , Deployment , Job , Pod , ReplicaSet ,
ReplicationController , StatefulSet , Ingress , Service ,
ConfigMap , Secret , PersistentVolumeClaim ,
StorageClass , Volume , Event , HorizontalPodAutoscaler ,
PodPreset , ...

31. custom resources
make your own resource dynamically
and use k8s database, API, authentication like the
other resource objects
support CRUD functions e.g. Create , Update , Delete
, Get , and List

32. custom controllers
The controller interprets the structured data as a
record of the user’s desired state, and continually
takes action to achieve and maintain that state.
e.g. kube-controller-manager , rbd-provisioner ,
operator , ...
can work with any kind of resource, but they are
especially effective when combined with
custom resources .

33. custom resource and controller

34. feel free to contact me
facebook (kubernetes Korea Group)
https://www.facebook.com/groups/169159038
4455961/
slack (Kubernetes #kr-users)
https://kubernetes.slack.com/messages/kr-
users/
email
al.lee@linecorp.com