This deck covers top 3 reasons why Google Kubernetes engine is best suited to run containerized workloads. The reasons covered are Security, Observability and Maturity.

1. Top 3 reasons why you should run your
Enterprise workloads on GKE
Sreenivas Makam
Partner Engineer @Google Cloud

2. Agenda
● Why Containers, Docker and Kubernetes
● GKE Value-Add
● GKE differentiators for Enterprises - Security, Observability
and Openness
● Demo

3. “Keeping our
infrastructure perfectly
homogenous is
giving me nightmares”
“It ran fine on
MY machine”
Problem: Deployments and Ops
are Hard
“We want to get the
best utilization of
our infrastructure”
“Keeping our
infrastructure perfectly
homogenous is
giving me nightmares”
“It ran fine on
MY machine”
“My developers aren’t
as productive as they
should be. Deployments
are slowing us down”

4. Bare Metal, VM and Container
Virtual machine
Kernel
Dependencies
Application Code
Hardware + hypervisor
Dedicated server
Kernel
Dependencies
Application Code
Hardware
Container
Kernel +
Container Runtime
Dependencies
Application Code
Hardware
Deployment ~mins (sec)
Portable
Very Efficient
Deployment ~months
Not portable
Low utilization
Deployment ~days (mins)
Hypervisor specific
Low isolation, Tied to OS

5. Why Containers
● Self contained
● Portability
● Decoupling from machine
● Image immutability
● Faster development
● Faster deployment
Virtual machine
Container
ImageMagick
6.4.90
Container
ImageMagick
7.0.28
Payments
application
Rendering
application
Linux distribution
Hardware

6. But they introduce a new set of
challenges
“Where should I run
my containers?”
“If we run our
containers on VMs,
I don’t want to
manage anything”
“How do I get my
containers to talk
to one another?”
“How do we ensure
our containers are
running smoothly?”
“We don’t want to
be locked into one
cloud provider”

7. Why Kubernetes
● Decoupling from infra
● Autoscaling
● Autohealing
● Automated rollout and rollbacks
● Abstractions that are cloud native and microservices friendly
● Extensible
● Open-source
● Integrates well with other Devops tools

8. Kubernetes cluster
Worker node
Master node
Worker node
Docker
Kubelet
Control
Plane
Docker
Kubelet
Deployment
Pod
Contain
er
Container
Pod
Contain
er
Container
Node pool
Deployment
Pod
Contain
er
Container
Pod
Contain
er
Container
Pod
Contain
er
Container
Pod
Contain
er
Container
Deployment
Pod
Contain
er
Container
Pod
Contain
er
Container
Pod
Contain
er
Container
Pod
Contain
er
Container
service
calls
Service
A
Service
B
kubectl
cmd
Kubernetes Architecture

9. Kubernetes control plane
Kubernetes Control Plane
API Server
etcd
Scheduler
Controller Manager
Kubernetes Master
API Server
etcd
Scheduler
Controller Manager
Kubernetes Nodes
Kubelet
Container Runtime
Kube-Proxy
Container Network

10. Kubernetes Abstractions
Pod
Deployment
Service
Config &
Secrets
Volume
Stateful set
Jobs

11. Google Kubernetes Engine (GKE)
gcloud
cmd
Kubernetes cluster
Worker node
Master node
Worker node
Runtime
E.g.Docker
Kubelet
Control
Plane
Runtime
E.g.Docker
Kubelet
Deployment
Pod
Containe
r
Container
Pod
Containe
r
Container
Node pool
Deployment
Pod
Containe
r
Container
Pod
Containe
r
Container
Pod
Containe
r
Container
Pod
Containe
r
Container
Deployment
Pod
Containe
r
Container
Pod
Containe
r
Container
Pod
Containe
r
Container
Pod
Containe
r
Container
service
calls
Service A
Service B
kubectl
cmd
GKE with Kubernetes

12. GKE Value Add
● Master management including master redundancy, upgrade, replication and
backup
● Worker node lifecycle management
● IAM integration for security and authentication
● Get all benefits of Google compute engine including Networking and Storage
● Integration with other Google cloud services like load balancer, storage, big
data, analytics
● Pod and cluster autoscale
● Integrated logging and monitoring with Stackdriver
● 99.5% SLA

13. Observability
Security
Openness
GKE for Enterprises - Top 3 reasons

14. 14
GKE for Enterprises
- Security

15. Container Security pillars
Software supply chain
Is my container image secure
to build and deploy?
Infrastructure security
Is my infrastructure secure
for developing containers?
Container runtime security
Is my container
secure to run?
Application security
Platform security
Are my applications secure?
Is my (cloud provider’s) infrastructure secure?
● IAM, RBAC, Pod access policy
● Shared VPC
● Private cluster
● Network control policy
● Image scanning
● Binary authorization
● Container OS
● Node OS(CoS)
● Cloud security command center
● Tie-up - Aquasec, Capsule8,
Stackrox, Sysdig,
Twistlock

16. Confidential & Proprietary
Google Kubernetes Engine patches you to the latest
version, automatically. This keeps you up to date with
security patches and with new features
Google Kubernetes Engine provides common best
practices with security by default. There will always be
app-level hardening and tuning to do
Google Kubernetes Engine provides the best of Google
Cloud Platform security features, with integrations with
IAM, Audit Logging, VPC, and more
1
2
3
Why GKE security?

17. Kubernetes Engine: Use RBAC and IAM
RBAC is enabled by default for GKE 1.8+ clusters
Use IAM to manage users and permissions
at the project-level, including API access,
service accounts, and quotas.
Use RBAC at the cluster and namespace
level to set permissions.
Infra sec

18. Kubernetes: RBAC Example
RBAC is enabled by default for Kubernetes 1.8+ clusters
e.g., give the ‘blue team’ user ‘cluster admin’ rights in the ‘blue’
namespace
$ cat blue-binding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: blue-dev-binding
namespace: blue
subjects:
- kind: User
name: blue-team-dev@kube-pw.iam.gsa.com
roleRef:
kind: ClusterRole
name: admin
apiGroup: rbac.authorization.k8s.io
Infra sec

19. Kubernetes: PodSecurityPolicy
apiVersion: extensions/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
spec:
privileged: false # Don't allow privileged pods!
allowPrivilegeEscalation: false # Don’t allow privilege
escalation
runAsUser: # Require the container to run without root
privileges
rule: ‘MustRunAsNonRoot’
supplementalGroups: # Forbid adding to the root group
rule: ‘MustRunAs’
ranges:
- min: 1
max: 65535
Infra sec

20. Network control policy
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: hello-allow-from-product
spec:
policyTypes:
- Ingress
podSelector:
matchLabels:
app: reviews
ingress:
- from:
- podSelector:
matchLabels:
app: productpage
Product Reviews
Details Ratings
Infra sec

21. Private Clusters
Trusted
Virtual Private Cloud (VPC)
Kubernetes Engine Cluster
Node Node Node
Google Kubernetes Engine
Kubernetes Master
Trusted
On-prem
Host HostVPN
Untrusted
Internet
Infra sec

22. Shared VPC
Sharing of
common
resources
More granular
billing
Isolation of data
and support for
multi-tenant
workloads
Security and
separation of
roles
Infra sec

23. Shared VPC
Organization
Apps
project
Kubernetes
Engine clusters
Apps
team
Shared VPC network
subnet-1 subnet-2
Network
admin
Host
Project
DB
project
Kubernetes
Engine clusters
DB
team
Infra
project
Kubernetes
Engine clusters
Infra
team
Private IP
connectivity
Infra sec

24. GKE: Minimal OS
Container-optimized OS (COS) based on Chromium OS, and maintained by Google
● Built from source: Since COS is based on Chromium OS, Google maintains all
components and is able to rebuild from source if a new vulnerability is discovered
and needs to be patched
● Smaller attack surface: Container-Optimized OS is purpose-built to run containers,
has a smaller footprint, reducing your instance's potential attack surface
● Locked-down by default: Firewall restricts all TCP/UDP except SSH on port 22, and
prevents kernel modules. Root file system is mounted read-only
● Automatic Updates: COS instances automatically download weekly updates in the
background; only a reboot is necessary to use the latest updates. Google provides
patches and maintenance
https://cloud.google.com/container-optimized-os/
Image sec

25. GCR: Vulnerability Scanning (Alpha)
● Scans all images in your private Google Container
Registry for known Common Vulnerabilities and
Exposures (CVEs)
● Examines images and packages
● Works for: Debian, Ubuntu and Alpine images
● Images are scanned when:
○ An image is added to the registry
○ There is an update to the vulnerability database
https://cloud.google.com/container-registry/docs/vulnerability-scanning
Image sec

26. To use,
● Enable the
Container
Analysis API
● Enable
Vulnerability
Scanning
GCR: Vulnerability Scanning (Alpha)
https://cloud.google.com/container-registry/docs/vulnerability-scanning
Image sec

27. 27
Launch
container
Requirements met?YES Requirements
Grafeas
Binary Authorization policyNO
Block launch
Attestations
Code
Signed by:
* Builder * Analysis tool
Must be built by
myphotos.com
Binary authorization (Alpha)
Image sec

28. Container security
Runtime
security
Sandboxing
Verification
with vTPM
Isolate a
workload at the
pod level using
hypervisor-like
technology
Bind information
to a node and
verify integrity of
a workload using
TPMs
Runtime
detection
agent
Monitor, detect,
and react to
common
container
attacks
Runtime sec

29. Runtime security partners in Cloud SCC
Cloud Security Command Center
5 partner integrations
Runtime sec

30. 30
0GKE for Enterprises
- Observability

31. Microservices
Kubernetes makes it easy to
break monolithic applications
into independently scalable
microservices
More pieces to monitor
and operate
Stackdriver - Rethinking monitoring with Kubernetes
Abstracted Infrastructure
Kubernetes offers a lot of
flexibility, with many
constructs that support and
make building your app easier
Increased observability across
your entire Kubernetes
environment becomes
necessary
Highly Dynamic Environment
Your environment scales and
adapts as needed, changing as
it reschedules and restarts
components
Keep track of your
applications, which may be
constantly moving

32. Multi-cluster
monitoring with
support for
Kubernetes Engine on
GCP and Kubernetes
on-prem in a single
place
Hybrid, multi-cluster Kubernetes monitoring

33. 33
Stackdriver
logging
● Review, monitor and alert on audit
logs centrally
● “jamie@myphotos.com deployed
a new frontend version @ time T”
● Runtime metrics gathered
and exported
● “Photo book creation latency in
the last 10 minutes was 1.3s”
K8s
Application logs
audit logs
Stackdriver
monitoring
Prometheus
GKE: Monitoring & logging

34. VPC Flow Logs for Kubernetes Engine
BigQuery
Cloud Pubsub
Stackdriver
Logging
Captures all flows in VPC
Integration with a host of partners
Optimize network usage and egress
Network Forensics & Security Analysis
Real-time Security Analysis

35. Kubernetes Load Balancing - Suboptimal
Two levels of load balancing
Inaccurate cloud-level health
checks
Multiple network hops

36. GKE Load balancing with Network Endpoint Group
Containers are “just another
endpoint”
Accurate cloud-level health
checks and load balancing
No extra network hops; direct
connection from load balancer to
container

37. Region: US West
Kubernetes
Engine
Alice
California
Google
Edge
myapp.com 120.1.1.1
Chao
Singapore
Google
Edge
myapp.com 120.1.1.1
Region: Asia East
Kubernetes
Engine
Bob
London
Google
Edge
myapp.com 120.1.1.1
Region: Europe West
Kubernetes
Engine
kind: Ingress Google Global HTTP(S) Load Balancing
Multi-region clusters

38. 38
GKE for
Enterprises
- Open & Mature

39. Each week, Google launches more than four billion
containers across its data centers around the world. These
containers house the full range of applications Google
runs, including user-facing applications such as Search,
Gmail, and YouTube.
Kubernetes was directly inspired by Google’s cluster
manager, internally known as Borg. Borg allows Google to
direct hundreds of thousands of software tasks across
vast clusters of machines numbering in the tens of
thousands — supporting seven businesses with over one
billion users each. Borg and Kubernetes are the
culmination of Google’s experience deploying resilient
applications at scale.
Containers at Google

40. GA for 3 years

41. Marquee customers
Kubernetes Engine (GKE) marquee customers

42. ...and many more

43. Cloud services platform(Hybrid cloud solution)

44. GKE Extended into your Datacenter
Google Cloud Platform
Serverless add-on for GKE
Google Kubernetes Engine
Istio add-on for GKE
Service Marketplace
Stackdriver +
Prometheus
Serverless add-on for GKE
Google Kubernetes Engine On-Prem
Istio add-on for GKE
Service Marketplace
Stackdriver +
Prometheus
Your Datacenter
Single-pane-of-glass UI
Policy Syncing
Aggregated Logging
CI/CD
Service Discovery
Multi-cluster Ingress

45. DEMO TIME!

46. Book review App

47. References
● Container Security - Blog series
● GKE NEXT18 sessions