This document compares deploying Cloud Foundry on traditional infrastructure using VMs versus deploying it on Kubernetes. It notes some pain points of the traditional approach like requiring an IaaS and difficult installation. It then introduces Kubernetes as an alternative and shows how Cloud Foundry components can be containerized and installed on Kubernetes more easily using Helm. It provides a hands-on example of deploying Cloud Foundry on an AWS Kubernetes cluster.

1. Cloud Foundry on Kubernetes
Yun Sang Jun
April 5, 2018
chris.sj.yun@gmail.com

2. Cloud Foundry Pain Point
● IaaS(Hypervisor) required
○ Impossible to install on baremetal
● Difficult to install
○ Leaning Curve about Bosh
● High Cost
○ Cloud Foundry have many components such as UAA, GoRouter..

3. Kubernetes?
● Container orchestration open source (powered by Google)
● Docker container lifecycle Management
● Easily deploy & upgrade application using Helm(package manager)
● Possible to install on baremetal

4. Comparison in infrastructure
VM x N
IaaS(Hypervisor)
Go
Router
UAA
Diego
Cell
Traditional Cloud Foundry Cloud Foundry on Kubernetes
...
Kubernetes Cluster
Baremetal or VM
Go
Router
UAA
Diego
Cell
...
● IaaS(Hypervisor) required ● Any Infrastructure with Kubernetes
installed

5. Comparison in installation

6. Comparison in cost
Component Traditional Cloud Foundry Cloud Foundry on Kubernetes
api
diego
loggregator
nats
router
uaa
...
At least 2GB memory
ecommened
At least 256MB memory
recommened
Total ≅ 60GB(with 16GB Cell, not HA) ≅ 30GB(with 16GB Cell, not HA)
● Traditional Cloud Foundry requires more memory due to VM’s guest OS

7. How?
● Cloud Foundry components are containerized
○ https://github.com/SUSE/scf
● Easily install using Helm
○ https://github.com/SUSE/scf/wiki/How-to-Install-SCF

8. Hands-on Exercise

9. 1. Prepare kubernetes cluster
● Deployed on AWS kubernetes cluster
● 1 Master Node
○ t2.large(2 vCPU, 8GB Memory)
● 3 Worker Node
○ t2.xlarge(4 vCPU, 16GB Memory)

10. 2. Configuring the deployment
<scf-config-values.yaml>
env:
CLUSTER_ADMIN_PASSWORD: changeme
DOMAIN: cf-dev.io
UAA_ADMIN_CLIENT_SECRET: uaa-admin-client-secret
UAA_HOST: uaa.cf-dev.io
kube:
external_ip: 192.168.77.77
storage_class:
persistent: persistent
auth: rbac

11. 3. Deploy UAA
# helm install helm/uaa-opensuse
--namespace uaa
--values scf-config-values.yaml
NAME READY STATUS RESTARTS AGE
mysql-0 1/1 Running 0 4d
uaa-f697656cb-ssh8b 1/1 Running 0 4d
# CA_CERT="$(kubectl get secret secret --namespace uaa -o jsonpath="{.data['internal-ca-cert']}" | base64
--decode -)"

12. 4. Deploy CF
NAME READY STATUS RESTARTS
api-0 1/1 Running 0
blobstore-0 1/1 Running 0
cc-clock-66947dfdc-rpftx 1/1 Running 0
cc-uploader-789cd6f9d6-qdjrs 1/1 Running 0
cc-worker-598984c7cd-qtwd7 1/1 Running 0
cf-usb-64b67b45f6-q2h6f 1/1 Running 0
diego-access-97864fdd6-g42hz 1/1 Running 0
diego-api-56c499d88b-h8bxh 1/1 Running 0
diego-brain-b44897599-rvxxx 1/1 Running 0
diego-cell-0 1/1 Running 0
diego-cell-1 1/1 Running 0
diego-locket-69d48ㅇ 1/1 Running 0
doppler-57f5b85497 1/1 Running 0
NAME READY STATUS RESTARTS
loggregator-5cd488 1/1 Running 0
mysql-0 1/1 Running 0
mysql-proxy-6c9b5 1/1 Running 1
nats-0 1/1 Running 0
nfs-broker-6f48f5fc 1/1 Running 0
router-bd866b695 1/1 Running 1
router-bd866b695 1/1 Running 0
routing-api-0 1/1 Running 0
syslog-adapter-55d 1/1 Running 0
syslog-rlp-66f79d4 1/1 Running 0
syslog-scheduler 1/1 Running 1
tcp-router-85f49d 1/1 Running 0
# helm install helm/cf-opensuse --namespace scf --values scf-config-values.yaml
--set "env.UAA_CA_CERT=${CA_CERT}"

13. 5. Expose endpoint
For the items below, change service type from ClusterIP to NodePort
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
uaa-public NodePort 10.3.0.71 xxx.xxx.xxx 2793:30266/TCP
diego-access-public NodePort 10.3.0.222 xxx.xxx.xxx 2222:30938/TCP
router-public NodePort 10.3.0.43 xxx.xxx.xxx 80:32102/TCP,443:31250/TCP,4443:31407/TCP

14. 6. Configure Load Balancer

15. Bonus #1 Cloud Foundry Console
Cloud Foundry Console
1. git clone
https://github.com/cl
oudfoundry-incubato
r/stratos
2. cf push -b
nodejs-buildpack
3. Possible to modify
branding

16. Bonus #2 Ops Manager
Kubernetes Dashboard
1. Monitoring Host, CF
Components status

17. Bonus #2 Ops Manager
Kubernetes Dashboard
2. Easily Scale-out CF
Components