Cloud Native Night, April 2018, Mainz: Workshop led by Jörg Schad (@joerg_schad, Technical Community Lead / Developer at Mesosphere)
Join our Meetup: https://www.meetup.com/de-DE/Cloud-Native-Night/
PLEASE NOTE:
During this workshop, Jörg showed many demos and the audience could participate on their laptops. Unfortunately, we can't provide these demos. Nevertheless, Jörg's slides give a deep dive into the topic.
DETAILS ABOUT THE WORKSHOP:
Kubernetes has been one of the topics in 2017 and will probably remain so in 2018. In this hands-on technical workshop you will learn how best to deploy, operate and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. You will learn how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow and more) on any infrastructure.
This workshop best suits operators focussed on keeping their apps and services up and running in production and developers focussed on quickly delivering internal and customer facing apps into production.
You will learn how to:
- Introduction to Kubernetes and DC/OS (including the differences between both)
- Deploy Kubernetes on DC/OS in a secure, highly available, and fault-tolerant manner
- Solve operational challenges of running a large/multiple Kubernetes cluster
- One-click deploy big data stateful and stateless services alongside a Kubernetes cluster
2. Jörg Schad
Technical Community
Lead / Developer
● Core Mesos
developer at
Mesosphere
● Passions are deep
learning, distributed
data systems, and
data analytics
3. Chris Gaun
PMM at Mesosphere /
CNCF Ambasador
● Previous to that
Gartner analyst
covering public IaaS
● @gaunetes
4. First, Some Questions (Show of
Hands)
• Are you a developer?
• Are you in operations?
• Are you using K8s (in production)?
• Are you using DC/OS (in production)?
14. The premier platform to run data-driven,
containerized applications on any infrastructure
North American
Banks
Worldwide Telco
Companies
Highest-Valued
Startups
Automotive
Technology Efforts
Represents organizations using Mesosphere technologies
5 of Top
10
30% of 7 of Top
12
5 of Top
10
6
Leading
28. (µ)Services
Service A
Stable Cluster-wide IP
Cluster-wide DNS name
Layer 3 Load Balancer
Updates continuously
app = portal
tier = frontend
version = v1
SELECTOR
A A A
29. (µ)Services
Service A
New Pod added
automatically based
on its label selector
app = portal
tier = frontend
version = v1
SELECTOR
A A A A
35. ETCD 01
MASTER 01
etcd
LoadBalancer
WORKER 01
API Server
Scheduler
Controller
Manager
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
36. ETCD 01
MASTER 01
LoadBalancer
WORKER 01
API Server
Scheduler
Controller
Manager
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
• Etcd is a distributed, consistent
key-value store
• Uses the RAFT consensus
algorithm for leader election
• Supports revisions and event
streams
• Primary store for all K8s API
objects (single source of truth)
• The only storage backend
currently supported by
Kubernetes
etcd
37. ETCD 01
MASTER 01
etcd
LoadBalancer
WORKER 01
API Server
Scheduler
Controller
Manager
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
38. ETCD 01
MASTER 01
LoadBalancer
WORKER 01
Scheduler
Controller
Manager
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
etcd
API Server
• Entry point to the system
• CRUD server for all the API
objects in Kubernetes
• Processes REST requests and
updates the objects in etcd
• Performs authentication /
authorization
MORE: https://goo.gl/KL8WfQ
39. ETCD 01
MASTER 01
LoadBalancer
WORKER 01
Scheduler
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
etcd
API Server
• Daemon process that
implements the control loops
built into Kubernetes
• E.g. … see NEXT slide
MORE: https://goo.gl/NJyRP3
Controller
Manager
40. ETCD 01
MASTER 01
LoadBalancer
WORKER 01
Scheduler
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
etcd
API Server
Types of Controllers in Kubernetes:
● ReplicaSet - desired state of pods
● ReplicationController - deprecated
● Deployments - updates
● StatefulSets - data services
● DaemonSet
● GarbageCollection - kills orphans
● Jobs
● CronJobs
Controller
Manager
41. ETCD 01
MASTER 01
LoadBalancer
WORKER 01
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
etcd
API Server
Controller
Manager
• Answers: Where should this pod
run?
• Based on policy, available
resources, affinity, labels, Qos
requirements, etc.
• Binds unscheduled pods to
nodes
EXPERT MODE:
● Node Affinity/Anti-Affinity
● Taints and Tolerations
● Pod Affinity/Anti-Affinity
https://goo.gl/nvLDE9Scheduler
42. ETCD 01
MASTER 01
etcd
LoadBalancer
WORKER 01
API Server
Scheduler
Controller
Manager
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
43. ETCD 01
MASTER 01
LoadBalancer
WORKER 01
Kubectl
Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
etcd
API Server
Controller
Manager
Scheduler
• Agent that runs on each node
• Recieves a set of PodSpecs and
insures all pods are running and
healthy
• Interacts with container runtime
(Docker, CRI-O)
• Performs some node-level health
checks
MORE: https://goo.gl/FEKN43
Kubelet
44. ETCD 01
MASTER 01
LoadBalancer
WORKER 01
Kubectl
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
etcd
API Server
Controller
Manager
Scheduler
Kubelet
• Network proxy and load balancer
for Services
• Used to be a user-space proxy
• Currently uses iptables
MORE: https://goo.gl/ph4sAs
Kube Proxy
45. ETCD 01
MASTER 01
etcd
LoadBalancer
WORKER 01
API Server
Scheduler
Controller
Manager
Kubectl
Kubelet Kube Proxy
Container Engine (Docker,
CRI-O)
Pod Pod Pod
HTTPS
HTTPS
HTTPS
HTTPS
57. PHYSICAL INFRASTRUCTURE
MICROSERVICES, CONTAINERS, & DEV TOOLS
VIRTUAL MACHINES PUBLIC CLOUDS
DATA SERVICES, MACHINE LEARNING, & AI
Security &
Compliance
Application-Aware
Automation
Multitenancy
Hybrid Cloud
Management
100+
MORE
DatacenterEdge
Datacenter and Cloud as a Single Computing Resource
Powered by Apache Mesos
20+
MORE
Unified hybrid cloud operations
Securely manage cloud, datacenter, and edge
infrastructures from a single control plane
4
Mesosphere DC/OS
Intelligent resource pooling
Optimize workload density for highest utilization with
resource guarantees
3
Broad workload coverage
Run today & tomorrow’s applications including traditional
J2EE, containers, analytics & ML
1
Application-aware automation
Automate workload-specific operating procedures to
“as-a-Service” anything from Kubernetes to data services
2
74. Kubernetes with DC/OS 1.11 Features
Simple HA
Cluster
Provisioning
Robust API
Server Auth
Non-disruptive
Upgrades
Automated
self-healing
Scalable
Kubernetes
Transport Layer
Security
Cloud Native
Services
75. Why Kubernetes on DC/OS: Automation
Management
Ability to easily create & scale pure play Kubernetes, data services,
and OSS components anywhere
Lifecycle
Ability to install/upgrade/patch every component of the
solution stack (data, K8s, CI/CDs and ML)
as-a-Service
Ability for each LOB to choose components and
manage resources and lifecycle according to its will
Security 30% Fortune 50 | CNI | TLS | Authentication
76. Simple HA Cluster
Provisioning
Kubernetes with DC/OS 1.11 Features
Robust API
Server Auth
Non-disruptive
Upgrades
Automated
self-healing
Scalable
Kubernetes
Transport Layer
Security
Cloud Native
Services
77. Dead Simple Secure & HA Provisioning
1. Prerequisites
2. Installing the Client Tools
3. Provisioning Compute Resources
4. Provisioning the CA and Generating TLS Certificates
5. Generating Kubernetes Configuration Files for Authentication
6. Generating the Data Encryption Config and Key
7. Bootstrapping the etcd Cluster… 3x for HA
10. Bootstrapping the Kubernetes Control Plane… 3x for HA
13. Bootstrapping the Kubernetes Worker Nodes
14. Configuring kubectl for Remote Access
15. Provisioning Pod Network Routes
16. Deploying the DNS Cluster Add-on… Deploying other Add-ons
20. Smoke Test
21. Cleaning Up
We turn this (21 steps)*….
$ dcos package install
kubernetes
Into this….
* Kubernetes the Hard Way by Kelsey Hightower
78. What’s Kubernetes Done Right?
● Automated - Kubernetes operations should be automated
● Evergreen - You want to always run the latest version of Kubernetes
● Included - Kubernetes is a commoditized standard that is included in a solution
● Open source - Market standard is pure OSS Kubernetes
● Unified - Kubernetes is not an island, you’ll have other workloads
79. Enhanced Data Security (EE)
● Secure Authentication, Authorization and
In-Transit Data Encryption
● Enable Authentication of Users, Apps
(Client-Server) & Inter-Service
Communication (Server to Server)
● One-Click Configuration of Transport
Security (TLS)
● Significantly Reduce Operational Overhead Non-locked down DIY Kubernetes
clusters target of hidden malware
80. Enabling Developer Agility:
Kubernetes is one part of a holistic CI/CD pipeline
Source Code
Control
Build and Test Release
Deploy, Monitor
and Log
Automatically trigger CI/CD
pipeline based on code
check-in.
Start automated build and test,
including functional, security
and performance tests.
Update artifact repository with
latest successful code artifacts
and pull newest images
Deploy applications to container
orchestration and watch with
monitoring and logging
CONTINUOUS INTEGRATION (CI) CONTINUOUS DEPLOYMENT (CD)
Marathon
98. Zero Touch Self-Healing
Robust API
Server Auth
Non-disruptive
Upgrades
Scalable
Kubernetes
Transport Layer
Security
Cloud Native
Services
Simple HA
Cluster
Provisioning
Automated
self-healing
99. ServerServer Server
Application Aware Scheduling
Automated Self Healing
Server Server Server Server Server
Server Server Server
KubernetesZero
Touch
20+
MORE
Cloud Native
Services
104. One-Click Scalability and Upgrades
Robust API
Server Auth
Transport Layer
Security
Cloud Native
Services
Simple HA
Cluster
Provisioning
Automated
self-healing
Scalable
Kubernetes
Non-disruptive
Upgrades