AWS Container, Kubernetes on AWS – 김광영(AWS 솔루션즈 아키텍트)

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kwangyoung Kim
July 2019
Container, Kubernetes on AWS
Journey to modern application

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Containers are the best on ramp
towards modern applications

Why are enterprises
adopting containers?
• Accelerate software development
• Build modern applications
• Automate operations at web scale
© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Make AWS the BEST PLACE to run ANY
containerized applications
© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential

Why customers love AWS container services
Containers are a first-class citizen of the AWS Cloud
Deeply integrated
with AWS
Security and Compliance
Broad selection of compute instances
and IAM security, VPC networking,
load balancing, and autoscaling
ISO, HIPPA, PCI, SOC1, SOC2, SOC3
Infocomm Media Development Auth.
DevOps Workflow
Best place to build and operate
a complete DevOps workflow for
containers—AWS DevTools and Cloud9
DEV OPS

Typical use cases
• Microservices: Java, Node.js, Go, Web Apps, etc.
• Continuous Integration and Continuous Deployment (CICD)
• Batch Processing and ETL jobs
• Common PaaS Stack for Application Deployment
• Legacy Application Migration to the Cloud
• Hybrid Workloads
• AI/ML
• Scale Testing
• Backend for IoT use cases
• Datahub for SAP

Why containers?

Application environment components
Runtime Engine Code
Dependencies Configuration

Different environments
Local Laptop Staging / QA Production On-Premises

It worked on my machine, why not in prod?
Local Laptop Staging / QA Production On-Prem
v6.0.0 v7.0.0 v4.0.0 v7.0.0

Containers to the rescue
Runtime Engine
Code
Dependencies

Docker
Lightweight container virtualization platform.
Tools to manage and deploy your applications.
Licensed under the Apache 2.0 license.
First released March 2013
Built by Docker, Inc.

Docker Image
Read only image that is used as a
template to launch a container.
Start from base images that have
your dependencies, add your
custom code.
Docker file for easy, reproducible
builds.
bootfs
kernel
Base image
Image
Image
W
ritable
Container
add
nginx
add
nodejs
U
buntu
References
parent
image

Four environments, same container
Local Laptop Staging / QA Production On-Prem

Container Implementation
Server (Host)
Hypervisor
Guest OS
Bins/Libs
App 2
Guest OS
Bins/Libs
App 3
Guest OS
Bins/Libs
App 1
Server (Host)
Hypervisor
Guest OS
App 2
Guest OS
App 3
Guest OS
App 1
Guest OS / Docker Engine
Bins/Libs Bins/LibsBins/Libs
Server (Host)
Operating System (OS)
Guest OS Guest OSGuest OS Libraries
App 1, 2, 3
Bare Metal Virtual Machine Containers

Container & Docker Benefits
Portable application artifact that runs reliably everywhere
Run different applications or application versions with different
dependencies simultaneously
Better resource utilization by running multiple lightweight
containers per host

Docker Engine

Container registries
• A Docker registry stores Docker images
• Docker Hub is a public registry that anyone can use, and Docker is
configured to look for images on Docker Hub by default.
• https://hub.docker.com/
• Amazon EC2 Container Registry (Amazon ECR)
• Fully managed Docker container registry
• Makes it easy for developers to store, manage, and deploy
container images

Docker Architecture
2

Docker Compose
Docker Compose allows you to define your multi-container application with all of its dependencies
in a single file, then spin your application up in a single command
$ sudo curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-
`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose
$ docker-compose --version

Docker Compose : HAProxy
$ docker run -d --name web1 tutum/hello-
world
$ docker run -d --name web2 tutum/hello-
world
$ docker run -d -p 80:80 --link web1:web1 --
link web2:web2 tutum/haproxy
$ vi docker-compose.yml
proxy:
image: tutum/haproxy
links:
- webapp
ports:
- "80:80"
- "1936:1936"
webapp:
image: tutum/hello-world
$ docker-compose up -d
$ docker-compose scale webapp=3
$ docker-compose ps
$ docker-compose stop
$ docker-compose rm

How do we make this work at scale?

We need to
• start, stop, and monitor lots of containers running on
lots of hosts
• decide when and where to start or stop containers
• control our hosts and monitor their status
• manage rollouts of new code (containers) to our hosts
• manage how traffic flows to containers and how
requests are routed

Common Questions
• How do I deploy my containers to hosts?
• How do I do zero downtime or blue green deployments?
• How do I keep my containers alive?
• How can my containers talk to each other?
• Linking? Service Discovery?
• How can I configure my containers at runtime?
• What about secrets?
• How do I best optimize my "pool of compute”?

Container Orchestration
Instance Instance Instance
OS OS OS
Container Runtime Container Runtime Container Runtime
App Service App App Service Service

myJob: {
Cpu: 10
Mem: 256
}
Orchestrator
Schedule
Run “myJob”

Instance/OS Instance/OS Instance/OS
Service Management
Scheduling
Resource Management
OrchestrationService Management
§Availability
§Lifecycle
§Discovery

Service Management
Scheduling
Resource Management
Orchestration
Scheduling
§Placement
§Scaling
§Upgrades
§Rollbacks

Service Management
Scheduling
Resource Management
Orchestration
Resource Management
§ Memory
§ CPU
§ Ports

What are container orchestration tools?

Amazon Container Services Landscape
MANAGEMENT
Deployment, Scheduling,
Scaling & Management of
containerized applications
HOSTING
Where the containers run
Amazon Elastic
Container Service
Amazon Elastic
Container Service
for Kubernetes
Amazon EC2 AWS Fargate
IMAGE REGISTRY
Container Image
Repository
Amazon Elastic
Container Registry

Open source container
management platform
Helps you run
containers at scale
Gives you primitives
for building modern
applications
What is Kubernetes?

Community, contribution, choice

Kubernetes Cluster Architecture
API Server Scheduler
Etcd Controller Manager
kubelet
kube-proxy pod

Kubernetes Cluster Master
The cluster master includes the following core Kubernetes components:
• kube-apiserver - The API server is how the underlying Kubernetes APIs are exposed. This component
provides the interaction for management tools, such as kubectl or the Kubernetes dashboard.
• etcd - To maintain the state of your Kubernetes cluster and configuration, the highly available etcd is a key
value store within Kubernetes.
• kube-scheduler - When you create or scale applications, the Scheduler determines what nodes can run the
workload and starts them.
• kube-controller-manager - The Controller Manager oversees a number of smaller Controllers that perform
actions such as replicating pods and handling node operations.

Kubernetes Cluster Node
The cluster node includes the following core Kubernetes components:
• The kubelet is the Kubernetes agent that processes the orchestration requests from the cluster master and
scheduling of running the requested containers.
• Virtual networking is handled by the kube-proxy on each node. The proxy routes network traffic and
manages IP addressing for services and pods.
• The container runtime is the component that allows containerized applications to run and interact with
additional resources such as the virtual network and storage.
• Kubernetes uses pods to run an instance of your application. Pods are typically ephemeral, disposable
resources, and individually scheduled pods miss some of the high availability and redundancy features
Kubernetes provides. Instead, pods are usually deployed and managed by Kubernetes Controllers, such as
the Deployment Controller.

Pods
• Define how your containers should run
• Allow you to run 1 to n containers together
Containers in pods have
• Shared IP space
• Shared volumes
• Shared scaling (you scale pods not
individual containers)
When containers are started on our cluster,
they are always part of a pod.
(even if it’s a pod of 1)
IP
Container A
Container B

Services
One of the ways traffic gets to your containers.
• Internal IP addresses are assigned to each container
• Services are connected to containers
and use labels to reference which containers
to route requests to
IP
IP
IP
Service
IP

Deployments
Services work with deployments to manage
updating or adding new pods.
Let’s say we want to deploy a new version of our
web app as a ‘canary’ and see how it handles
traffic.

Deployments
The deployment creates a new replication set
for our new pod version.

Deployments
Only after the new pod returns a healthy
status to the service do we add more new
pods and scale down the old.

But where you run Kubernetes matters
Quality of the
cloud platform
Quality of the
applications
Your users

—CNCF survey

What is Amazon EKS?
• Amazon Elastic Kubernetes Service (Amazon EKS) is a managed service
that makes it easy for you to run Kubernetes on AWS without needing to
stand up or maintain your own Kubernetes control plane. Kubernetes is
an open-source system for automating the deployment, scaling, and
management of containerized applications.

Kubernetes on AWS
Managed Kubernetes on
AWS
Highly
available
Automated
version
upgrades
Integration
with other
AWS services
Etcd
Master
Managed
Kubernetes
control
plane CloudTrail,
CloudWatch, ELB,
IAM, VPC, PrivateLink

Kubernetes on AWS
3x Kubernetes masters for HA

Availability
Zone 1
Master Master
Availability
Zone 2
Availability
Zone 3
Master
Workers Workers Workers
Customer Account
AWS Managed

EKS Architecture
mycluster.eks.amazonaws.com
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl

EKS Architecture
EKS VPCCustomer VPC
Worker Nodes
EKS-Owned
ENI
Kubernetes
API calls
Exec, Logs,
Proxy
Internet

EKS Control Plane

Kubernetes Control Plane
Highly available and single tenant
infrastructure
All “native AWS” components
Fronted by an NLB
VPC
API Server ASG
Etcd ASG
NLB
AZ-1 AZ-2 AZ-3
ELB
Instances
Instances

Kubernetes Control Plane
Master Node
Scheduler
Controller
Manager
Cloud
Controller
Manager
API Server
etcd
Kubectl

What happens when I run ‘kubectl create –f pods.yaml’?

IAM Authentication
Kubectl
3) Authorizes AWS Identity with RBAC
K8s API
1) Passes AWS Identity
2) Verifies AWS Identity
4) K8s action
allowed/denied
AWS Auth

kubectl configuration
# [...]
users:
- name: aws
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: aws-iam-authenticator
args:
- "token"
- "-i"
- "CLUSTER_ID"
- "-r"
- "ROLE_ARN"
# no client certificate/key needed here!

Cluster Authentication and Authorization
• User or IAM role who creates EKS cluster gains Admin privileges
• This {“super”} user/role can then add additional users or IAM roles
and configure RBAC permissions
• To add, configure aws-auth Configmap
kubectl edit -n kube-system configmap/aws-auth

aws-auth configuration
apiVersion: v1
data:
mapRoles: |
- rolearn: arn:aws:iam::555555555555:role/devel-worker-nodes-NodeInstanceRole-74RF4UBDUKL6
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
mapUsers: |
- userarn: arn:aws:iam::555555555555:user/admin
username: admin
groups:
- system:masters
- userarn: arn:aws:iam::555555555555:user/john
username: john
groups:
- pod-admin # k8s RBAC group

EKS Data Plane

Kubernetes Data Plane
Worker Node
kube-dnsKubelet
aws-
node
Container runtime
Control Plane
API
kube-
proxy

EKS Networking & Load Balancing

AWS VPC CNI Plugin
ENI
Secondary IPs:
10.0.0.1
10.0.0.2
10.0.0.1
10.0.0.2
ENI
10.0.0.20
10.0.0.22
Secondary IPs:
10.0.0.20
10.0.0.22
ec2.associateaddress()
VPC Subnet –
10.0.0.0/24
Instance 1 Instance 2
VPC

AWS VPC CNI plugin – understanding IP
allocation
Primary CIDR range è RFC 1918 addresses è 10/8, 172.16/12, 192.168/16
Used in EKS for:
• Pods
• X-account ENIs for (masters à workers) communication (exec, logs,
proxy etc.)
• Internal Kubernetes services network (10.100/16 or 172.20/16 –
chosen based on your VPC range)
Setup:
• EKS cluster creation è provide list of subnets (in at least 2 AZs!) è
tagging

Load Balancing
All three AWS Elastic Load Balancing products are supported
NLB and CLB supported by Kubernetes Service type=LoadBalancer
Internal and External Load Balancer support

Load Balancing
Want to use an Internal Load Balancer? Use annotation:
service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
Want to use an NLB? Use annotation:
service.beta.kubernetes.io/aws-load-balancer-type: nlb

ALB Ingress Controller
Production-Ready 1.0 Release
Supported by Amazon EKS Team
Open Source Development: https://github.com/kubernetes-
sigs/aws-alb-ingress-controller
Customers are using it in production today!

ALB Ingress Controller
AWS Resources
Kubernetes Cluster
Node Node
Kubernetes
API Server ALB Ingress
Controller
Node
HTTP ListenerHTTPS Listener
Rule: /cheesesRule: /charcuterie
TargetGroup:
Green (IP Mode)
TargetGroup:
Blue (Instance
Mode)
NodePort NodePort
Ingress
Resource
Creation via
Kubectl or API

Container Services Roadmap
https://github.com/aws/containers-roadmap

Updates

Endpoint Access Control

Thank you!

AWS Container, Kubernetes on AWS – 김광영(AWS 솔루션즈 아키텍트)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Mehr von Amazon Web Services Korea

Mehr von Amazon Web Services Korea (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

AWS Container, Kubernetes on AWS – 김광영(AWS 솔루션즈 아키텍트)