Containers make it easy to build and deploy applications by abstracting away the underlying operating system. But how do you build secure and compliant containerized applications in a distributed environment, and without direct access to the operating system your code is running on? In this session, hear how Amazon Elastic Container Service for Kubernetes (Amazon EKS) is integrated into a large-scale regulated enterprise in the areas of network, security, CI/CD, and monitoring to cater to the needs of various business units. We cover the basics in each of these areas in Amazon EKS, and we hear from Fidelity on how it is driving its cloud strategy with Amazon EKS in the heavily regulated finance sector. We also share best practices and common architectures for building containerized application in highly regulated industries.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building PaaS with EKS for large-
scale highly regulated enterprise
Madhuri Peri
Sr. IoT Solution Architect
Amazon Web Services
C O N 3 0 9
Amr Abdelhalem
Head of Cloud Architecture
Fidelity Investment

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
• Amazon EKS – Concepts overview
• Building blocks AWS Cloud offers
• How Fidelity used these services to achieve business needs maintaining compliance
• Fidelity – PaaS strategy & regulatory guidelines
• Fidelity
• Amazon EKS integration Security
• Amazon EKS integration DevOps & Helm
• Amazon EKS integration Monitoring
• Future & Next steps

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Common financial regulatory challenges
• Traceability of changes
• Ownership
• Accountability
• Change control
• Sensitive data security
• Data accessibility
• Only owner of the data can create/update/delete it
• Audit and tracking
• Consumer laws

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
xyz.eks.amazonaws.com
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
kubectl
Amazon EKS Architecture

5. Components of K8 with Amazon EKS
5
Amazon
EKS
Master
endpoint
ALB/NLB
ConnectivitytoCluster
Monitoring,Metrics
Amazon EKS Nodes
Storage & Policy ( Amazon EBS storage
classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Case for PaaS
Open-source
Flexibility tooling
Powerful feature sets
Amazon
EKS [k8s]
App dev
needs
Complex
to govern,
manage,
scale
Polyglot languages
Choice of tools
Agility

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS – Concepts overview

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS Security - Auth & AuthZ (A&A)
• K8s uses RBAC
• AWS IAM Authenticator – Bridge
between IAM/RBAC
• K8 mechanisms –
• Namespaces
• Service accounts
• User accounts

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
kubectl
3) Authorizes AWS Identity with RBAC
K8s API
1) Passes AWS Identity
2) Verifies AWS Identity
4) K8s action
allowed/denied
AWS IAM
Auth
Amazon EKS Security - AWS IAM A & A

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS / Amazon EKS & K8 Network controls
• VPC Security groups
• VPC Subnet NACL
• Pod level implement the
network policy
• Network segmentation
• Tenant isolation
• Network policies similar to
AWS security groups
• Assigned to pods using pod
selectors and labels

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS - Network layer
• K8s you have 3 layers of IP addresses:
• K8 Cluster level
• K8 Pod level
• AWS VPC layer
• CNI plugin
• L-IPAM daemon, attach & assign & maintain IP addresses to ENI’s
• CNI plugin wiring host network, and adding correct interface to pod namespace
• Deployed as a daemon set on each node
• Provides the IP addresses for the node based on the Amazon EC2 instance type launched.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Typical 3-tier application : Traffic flow constraints
Web server pods Application server pods
Reporting server pods DB instance read replica
❌
✅✅✅
✅ ✅ ✅
❌
Hello app Users

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sample policy
kubectl create -f - <<EOF
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: access-appserver
namespace: sample-policy
spec:
podSelector:
matchLabels:
run: app-server
ingress:
- from:
- podSelector:
matchLabels:
run: web-server
EOF

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ENI
Secondary IPs:
10.0.0.1
10.0.0.2
10.0.0.3
Pod: Nginx
Veth IP: 10.0.0.1
Namespace: web-server
ENI
Secondary IPs:
10.0.0.20
10.0.0.22
10.0.0.23
VPC Subnet – 10.0.0.0/24
EC2 Instance 1
Instance 2
Pod: Java
Veth IP: 10.0.0.2
Namespace: - app-server
Pod: Java
Veth IP: 10.0.0.3
Namespace: - app-server
Pod: Java
Veth IP: 10.0.0.22
Namespace: - app-server
Pod: Nginx
Veth IP: 10.0.0.21
Namespace: web-server
Pod: report
Veth IP: 10.0.0.23
Namespace: - report API
Calico : Network policy enforcement

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS & K8s - CI/CD Controls
• AWS CodeCommit
• AWS CodeBuild
• Amazon ECR
• AWS CodePipeline
• K8s Helm package
management
• Opensource / partner
software to scan images
• coreos/clair

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS – Helm/Package management
• Helm helps you manage K8s apps via Helm charts
• K8s Application Helm charts
• Define
• Install
• Upgrade
• Create, version, share and publish – Important for regulation!
• Reproduce builds of K8s
• Runs on CI/CD or dev laptops

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS – Helm/Package management
• Functionality through
• Client – helm
• Server – Tiller
• Charts
• Tiller runs in K8 cluster, manages installations of helm charts
• Charts are helm packages
• Description of package
• One or more templates of K8 manifests
• For example, mysql helm package would create below
• All required Service accounts, secrets, service, configMaps, pvc, deployment, etc required for
running mysql pods in the cluster

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Helm mysql example
$ helm install --name my-release -f values.yaml stable/mysql
$ helm install --name my-release --set
mysqlRootPassword=secretpassword,mysqlUser=my-user,mysqlPassword=my-
password,mysqlDatabase=my-database stable/mysql
Source:
https://github.com/helm/charts/blob/master/stable/mysql/README.md

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Visibility & Monitoring
• Amazon CloudWatch
Metrics
• VPC / ALB / Amazon EC2 / ASG /
Amazon EKS Control plane
• Custom metrics
• CloudWatch Logs
• VPC / ALB /Amazon EC2 / ASG /
Amazon EKS Control plane
• AWS CloudTrail
• K8s Scaling metrics
• HPA
• Cluster auto-scaler
• Cluster wide metrics

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fidelity

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cost Savings
Utilization based chargeback
Avoid developing non-differentiating capabilities
Faster time to market.
Focus on business logic
and value not
underlying plumbing.
Innovation
Launch new products
leveraging innovative
cloud capabilities.
Enable New business
opportunities in a
Digital API marketplace
On-demand elastic
compute.
Multi-availability zones
and data centers for
redundancy and HA
Full transparency and
traceability on usage,
access, assets,
deployment, issues.
PaaS Business Objectives
A Native Cloud Strategy is critically important to being competitive as a Digital Business.
As organizations progress along their Cloud Journey, the question is how to maximize business value while balancing risk
& complexity

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A Native Cloud Strategy is critically important to being competitive as a Digital Business.
As organizations progress along their Cloud Journey, the question is how to maximize business value while balancing risk
& complexity
Leveraging Open
Source technologies
Managed CSP services
Multi-tenancy
Common deployment
pipelines should be
leveraged to enforce
security controls &
policies.
Adopting and aligning
to industry standards
safeguards
Common set of
standards, controls and
policies
.
Innovation Tenets

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
IaaS
Applications
DevOps
Securit
y
Insights
Platform overview
Business
Insight
Monitor
Logging
Cloud
Analytic
Traces
IaaS Health
Certificate
Vulnerability
Auth
Secret
DDoS
Encryption
CI/CD
Config Mgmt
App Registry
IDE
Code Repo
Software
Store
API
Routing LB Content
UX Mobile IoT
Service Mesh
Micro Service
ServerlessContainers
ComputeStorageGlobal Network
Data
Databases Documents Key/ValueAnalytic
AI &ML
Shared
Services
Batch Services
Cache Services
Search
Messaging
Bot Services
Cognitive
Services
ML Services

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS Platform overview
24
Tools
Managed.
Apps
Routing
Amazon
EKS Control
Plane
Ingress
Connect
Telemetry
Amazon EKS Nodes
CI/CD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
SRESRE
Users
EKS Admin
DevOps
Stakeholder
Engineering

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Platform Structure
Platform
Cluster 1
Namespace
Group X
Name
Space A
PODs PODs PODs
Name
Space B
Namespace
Group Y
Name
Space C
Name
Space D
Namespace
Group Z
Name
Space C
Name
Space D
Admin
Namespace
Daemon
sets
Tools
Cluster 2 Cluster 3 .... Cluster N
Teams Access Point
Applications
Admins Access Point

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cluster Access
EKS Cluster
Worker Node
AWS IAM
Roles
GRP
A
GRP
B
GRP
C
GRP
ADM
EKS RBAC
GRP A
GRP B
GRP C
GRP
ADM
Namespace Access level Cluster Access level
Worker NodeWorker Node
Namespace 1
Namespace 2
Cluster Management Namespace
Namespace 3
AD

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EKS Cluster
Availability
Zone 1
Cluster Architecture
EKS Node 1
CNIKubletEngine
Availability
Zone 2
Availability
Zone 3
EKS Node 2
CNIKubletEngine
EKS Node 3
CNIKubletEngine
Cluster Mgmt. Name Space
Name Space A
Name Space B
External
DNS
Ingress
controller
Helm-tiller
Certificate
Mgmt
Network
policy
Monitor
daemon set
Bootstrap
Controller
Helm-tiller
Service
Mesh
Vault
Injection
Horizontal
AutoScaler Kube Bench Serverless
ALB
Kube Ark
Future
Service
Core Service
PODs PODs PODs PODs
Helm-tiller PODs PODs PODs PODs
AWS IAMAWS IAMAWS IAM
Node Level
Service
PODs
PODs
PODs
PODs
PODs
PODs
PODs
PODs

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Platform Access
EKSConnect
Kubectl
aws-iam-
authenticator
IAM
EKS Master
API server
authenticato
r server
Kubernetes
RBAC
ETCD
1
2
3
4
5
8
9,10
6
7
EKSConnect
Config
Helm
0
EKS Master
Router
Connect
Telemetry
EKS Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
DevOps
Retrieve
Token
JWT
Token
Authenticate
Invoke
Pull
Secret
EKS Connect

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
$ ./eksconnect
username [user123]:
password:
domain [default]:
region [us-east-1]:
Choose from the below list of available roles
[ 0 ] ----> arn:aws:iam::****:role/Cluster1_EKSMaster
[ 1 ] ----> arn:aws:iam::****:role/Cluster2_NameSpaceGroup1_namespaceX_API
[ 2 ] ----> arn:aws:iam::****:role/Cluster2_NameSpaceGroup1_namespaceY_UI
Role No [0]: 1
profile [default]:
Writing credentials ....
set http_proxy
set https_proxy
set no_proxy
*************************************************************************************
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
kv-sidecar-69667c9cb9-6nhp2 0/2 Terminating 0 10d
kv-sidecar-69667c9cb9-zrlfc 0/2 Terminating 0 10d
postgres-7ff9df5765-2xhjn 1/1 Running 0 11d
Platform AccessEKS
Connect
EKS Master
Router
Connect
Telemetry
EKS Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
DevOps

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS
Control Plane
Ingress
Connect
Telemetry
Amazon EKS Nodes
CICD
Infrastructure as code
Storage & Policy ( Amazon EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
SRESRE
Users
EKS Admin
DevOps
Application
Management
Stability
Platform Stakeholders
Management

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Platform Build
EKS Master
Router
Connect
Telemetry
EKS Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
EKS
Admin
31
Cluster
YAML
CLI
Cluster Bootstrapping
• Setup IAM roles
• Setup security group.
• Provision Control Plane.
• Provision Cluster Nodes.
• Setup Admin group, AWS IAM and RBAC.
NS Management
Bootstrapping
• Setup Ingress Controller.
• Setup Namespace Bootstrap
controller.
• Setup Secret Management
Controller.
• Setup AWS IAM Controller.
apiVersion: eksManager.fidelity.com
kind: Cluster
metadata:
name: <Cluster1>
description: ”reinvent Cluster"
labels:
key: value
spec:
cloudId: AWS
envType: dev
appOwner: <userA>
appEmail: <userA>@ourplatfrom.com
rbac:
- accessLevel:
adgroup: <…>
iamRole: <..>
members:
- ”<userB>"
- ”<userC>"
nodes:
………
secrets:
………
monitor:
…….
EKS Manager

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Platform Onboarding
EKS Master
Router
Connect
Telemetry
EKS Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
EKS
Admin
3
Cluster
YAML
CLI
• Setup AD group , AWS IAM roles and RBAC.
• Create Namespace(s).
• Setup Helm-Teller.
• Setup Ingress controller.
apiVersion: nsbootstrapoperator.fidelity.com
kind: nsgroup
metadata:
name: <GroupX>
description: ”Name Space Group X"
namespace: default
labels:
key: value
spec:
cloudId: AWS
envType: dev
appOwner: <userA>
appEmail: <userA>@ourplatfrom.com
rbac:
- accessLevel:
adgroup: <…>
iamRole: <..>
members:
- ”<userB>"
- ”<userC>"
- accessLevel: readonly
adgroup:<…>
iamRole: <..>
members:
- "<userC>"
- "<userD>"
namespaces:
- name: <Cluster>_<NamespaceGroup>_<namespace>_<UI>
- name: <Cluster>_<NamespaceGroup>_<namespace>_<API>
- name:
<Cluster>_<NamespaceGroup>_<namespace>_<BACKEND>
………

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Interfaces
Application Life Management EKS Master
Router
Connect
Telemetry
EKS Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
DevOps
Security Service
4
5 7
1
2
3
Audit
8
9
6
10
Development
Security
Scan
Build
Secret
Promote
Images
Deploy
Helm
Replicate
ECR
Deploy
Helm
Pull
images
Configure
R53
Pull
Secret

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Secret Management
Image Management
Release Management EKS
Master
Router
Connect
Telemetry
Amazon EKS Nodes
CI/C/D
Infrastructure as code
Storage & Policy ( Amazon EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
EKS
Admi
n
Release 1.0
Images
Data
App
Config
Secret
repos
Release 2.0
Images
Data
App
Config
Secret
repos
AppVault Init
container
1
2
3
1
2
3
Helm
charts
Helm
charts
AWSAWS
AWS
IAM
AWS
IAM
Message Message

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EKSWatcher - Telemetry EKS
Master
Router
Connect
Telemetry
EKS Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
SRESRE
Infrastructure
•Compute
•Network
•Storage
Management
Tier
•Events
•Resource consumptions
Events
•Availability
•Health checks
•auto scaling events
Applications
•Response Times
•Application logs
•Performance matrixes
Metrics
Logs
Health
DataMassage
Performance
dashboards
Applications
Analytic
Operation
Alerts
Automated Recovery Actions
AIOps(Recovery/Actionmanager)

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Future Tools - Concept EKS Master
Router
Connect
Telemetry
EKS Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
DevOps
 Cost to run container per hour.
 Cost to run application.
 Containers density.
Resource
Utilization
 EC2 utilization
 IP Utilization
 Storage Utilization
 Vulnerability scanning
 Penetration testing

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Service Mesh - Concept
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops
EKSMaster
Router
Connect
Telemetry
EKS
Nodes
CICD
Infrastructure as code
Storage & Policy ( EBS storage classes )
Storage Compute & Policy ( Auto Scaling)
Network & Policy ( Calico )
Dev
Ops

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amr Abdelhalem
https://www.linkedin.com/in/amrhalem/
Madhuri Peri
https://www.linkedin.com/in/mpericloud/

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.