For building project Kyma - we wanted to use every cool new buzzword - that's how we ended up sitting on top of Kubernetes, Istio and Knative. And we really liked the experience, which lead to our subsequent involvement in the Knative project.

1. Krasimir Semerdzhiev / @evilyeti
SAP SE
Practical experience with
Kubernetes, Istio and Knative

2. 2© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Krasimir Semerdzhiev (aka Krassi aka @evilyeti)
Gamers’ Workshop
Magazine
ProSyst GmbH
SAP Hybris à SAP Customer Experience
InQMy Labs
SAP Labs Bulgaria
SAP SE
1999
2016-…

3. Using Kubernetes: …2.5 years later

4. 4© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
KUBE-APISERVER
API
CLI
Kubernetes – basic architecture
Apache 2.0 licensed
ETCD
KUBE-SCHEDULER
CLOUD-CONTROLLER MANAGER
KUBE-CONTROLLER MANAGER
CLOUD PROVIDER
KUBERNETES MASTER
NODE
KUBE PROXY
KUBELET
POD
CONTAINER
NODE
POD
CONTAINER
NODE
POD
CONTAINER
CONTAINER REGISTRY
PLUGIN NETWORKOPERATOR/
DEVELOPER
KUBE PROXY
KUBELET
KUBE PROXY
KUBELET
USERS
…

5. 5© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
• Project “Seven” or “The better Borg”
• Large developer ecosystem
• Declarative state
• Portability – de-facto standard
• “Run any workload”
• Platform for building platforms
Noteworthy
10+ years of experience in
running “borglets” and “omletes”!

6. 6© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
• Mind your base image and keep it small! Don’t use :latest tag.
• One process per container!
• Immutable containers!
• Don’t use root users!
• Make your file system read-only!
• Never restart – crash cleanly instead! Use readiness and liveness probes
• Stdout and stderr
• Use ExternalName services
• Helm charts for installation
Noteworthy

7. 7© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Anybody else running it at SAP?

8. 8© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Spinning Up Kubernetes ... the Usual Way
Master
Master
Master
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
HA
Master
Master
Master
Worker/
Minion
Worker/
Minion
Worker/
Minion
HA
Master
Master
Master
Worker/
Minion
Worker/
Minion
HA
Master
Master
Master
Worker/
Minion
HA
Master
Master
Master
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
HA
Worker/
Minion
Worker/
Minion
Master
Worker/
Minion
Control plane (often in HA and on separate
hardware, usually quite underutilized)
The actual workload (managed by Kubernetes,
usually pretty well utilized)
Worker/
Minion
Master
Worker/
Minion
Worker/
Minion
Master
Worker/
Minion

9. 9© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Spinning Up Kubernetes...the Gardener Way
Seed Cluster
Master
Master
Master
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
HA
Worker/
Minion
Shoot Clusters
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
Worker/
Minion
manages
Worker/Minion
Think outside the box /
Move outside the box!
ETCD
API Server
Scheduler Controller Mgr
API Server
ETCD
SchedulerController Mgr
…
ETCD
Worker/
Minion
Worker/
Minion
Auto-scaling via native
hyperscale provider
service
or controller on bare
metal
Gardener

10. 10© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ

11. Using Istio: … 8 minor releases later

12. 12© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Basic micro-service challenges
• Each service has a well-defined purpose and data store
• Many loosely-coupled services with remote dependencies
• Significant “integration” overhead for every service (log
collection, APM, client SDKs) + updates
• Any technology (Go, node.js, Java, Vert.x, etc.)
From Monoliths to Microservices
AppServer + Frameworks
• Coherent application server instance, providing all
framework SDKs/clients
• Tight coupling and non-transparent dependencies
• Minimal “overhead” for the individual app
• Limited technology choice (Java + JVM-based)

13. 13© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Official mission statement:
Istio is an open platform for providing a uniform way to integrate
microservices, manage traffic flow across microservices, enforce policies
and aggregate telemetry data. It provides an abstraction layer over the
underlying cluster management platform, such as Kubernetes, Mesos, etc.
Community members/contributors:
• Google + IBM
• + Envoy from Lyft
Real use forward:
Istio will replace significant part of the traditional Java application server
layer + the SPI/SDK interaction patterns, established over the last 15
years. It’ll allow micro-service developers to FOCUS on their stuff, solving
the plumbing challenge with the underlying infrastructure.
With all the 15+ years of
WebSphere experience!
Istio
What is Istio?

14. 14© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
To be a network for running (not only micro!) services
§ Visibility & Discovery
§ Resiliency & Efficiency
§ Traffic Control
§ Security
§ Policy Enforcement
WHAT ISTIO FOCUSES ON?

15. 15© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Outbound features:
• Service authentication
• Load balancing
• Retry and circuit breaker
• Fine-grained routing
• Telemetry
• Request Tracing
• Fault Injection
Inbound features:
• Service authentication
• Authorization
• Rate limits
• Load shedding
• Telemetry
• Request Tracing
• Fault Injection
WHAT IS A SIDE-CAR AND HOW DOES ISTIO USE IT?

16. 17© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Extremely convenient for micro-service developers
Automatic security token handling
Service roles align with RBAC in Kubernetes
Metrics + requests tracing out of the box to Prometheus
Mutual-TLS via the local side-car
No need for cross-team “adaptation projects”
Breaks the strong technology coupling for services
Noteworthy

17. Kyma: … taking it all together

18. 19© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Business Events
API calls
Any
Open Service Broker
(OSB) compatible
service provider
Business services
ApplicationConnectivity
ServiceCatalog
Flows
Microservices
API Exposure
Functions
EventBus
Service
Management
Knative
Kubernetes
Third Party System
Customer System
§ Legacy
§ On Premise
§ External System
§ Cloud Solution
SAP® C/4HANA
SAP Marketing Cloud
SAP Commerce Cloud
SAP Sales Cloud
SAP Service Cloud
SAP Customer Data Cloud
SAP S/4HANA®
Service
Consumption
On -
Premise
On -
Premise
Kyma – extensibility in a box
Apache 2.0 licensed, kyma-project.io

19. 20© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Kubernetes
Istio
Prometheus
FluentD/Bit
Serverless
Eventing
Cloud Native
Developer
Experience
Service Mesh
Service Broker /
Catalog
Micro Frontends
Application
Integration
Customization /
Integration Toolbox
Lightweight PaaS
Smart Scaling
BuildPacks
Deployment
Management
Automation
Golang
Resource Efficiency (CPC)
CloudEvents
Knative Kyma
Zipkin
Jaeger
Nats /
Nats Streaming /
Message routing
Lifecycle
Management
API exposure
Open Tracing
Kyma next to Knative
Wormholes

20. 21© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
What to do, what to do?

21. 22© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Knative launch + Kyma launch on the same stage
• SAP involvement in Knative
• Kyma open sourced (kyma-project.io)
Separate session to outline SAP OSS projects (video):
• Project Kyma as a cloud native extension
framework developed by SAP
• Gardener as the Kubernetes clusters as a service
implementation from SAP
• SAPmachine.io the friendly OpenJDK fork from SAP
with free maintenance releases
Go all in! J à Jul ‘18: Kyma launch at Google Cloud Next ‘18

22. Getting involved in Knative: … as we speak

23. 24© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
What is Knative

24. 25© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
What is Knative
Function
App
Container
Registry

25. 26© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
“Serverless is more than
functions!”
~ Knative team

26. 27© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Anyone using it?

27. 28© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Scale to zero – true pay-per-use
Traffic splitting via dynamic routes (80/5/5/5/5)
Integrates networking and service mesh automatically
Reasonable object model
Batteries included (logging and monitoring), but pluggable
Auto-scaler is interchangeable
In-cluster build
Noteworthy

28. 29© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
“Knative will almost certainly
become the standard plumbing
for functions-as-a-service on
Kubernetes”
-- James Governor, RedMonk

29. 30© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Enhance Knative with our nats.io eventing
mechanism – ongoing.
• Will lower resource requirements
• Stick to the CNCF technology stack
• Pull request
Enhance the Event source catalog
Our needs
Service Broker / Catalog
Micro Frontends
Application Integration
Customization / Integration
Toolbox
Knative
Kyma

30. 31© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
STAY UP TO DATE and GET INVOLVED!
• kubernetes.io
• istio.io
• knative.dev
• kyma-project.io
• gardener.cloud
• sapmachine.io
… AND UPDATE YOUR OSS DEPENDENCIES! J
References:

31. Krasimir Semerdzhiev / @evilyeti
Thank you!