BPF (Berkeley Packet Filter) is becoming the fastest growing technology in the Linux kernel and is revolutionizing networking, security and tracing. At the same time, the rise of container-based orchestration platforms such as Kubernetes is creating demand for routing, load-balancing & security infrastructure that is highly scalable, application-aware, and resilient. This talk introduces the open source project Cilium - a modern networking and security platform for microservices. Cilium is built on top of BPF and provides Linux native networking and security services with application protocol awareness. Cilium works hand in hand with application proxies such as Envoy and the services management orchestration layer Istio to provide infrastructure services in a transparent manner and with minimal overhead. This talk will discuss the challenges of exposing services via APIs and the solution that Cilium provides to enforce least privilege security.

1. Application-Aware Security
for Microservices via BPF
Cynthia Thomas, Technology Evangelist
@_techcet_
Seattle Kubernetes MeetUp
December 12th, 2017
Open Source Cloud Native Security

2. Application
Architectures
Delivery Frequency
Operational Complexity
Single Server
App
Yearly
Low
Evolution of Application Design & Delivery Frequency

3. Application
Architectures
Delivery Frequency
Operational Complexity
Single Server
App
Yearly
Low
3-Tier App
Monthly
Moderate
Evolution of Application Design & Delivery Frequency

4. Application
Architectures
Delivery Frequency
Operational Complexity
Single Server
App
Yearly
Low
Distributed
Microservices
10-100 x’s / day
Extreme
3-Tier App
Monthly
Moderate
Evolution of Application Design & Delivery Frequency

5. Network Security
has barely evolved
$ iptables -A INPUT -p tcp
-s 15.15.15.3 --dport 80
-m conntrack --ctstate NEW
-j ACCEPT
The world still runs on iptables
matching IPs and ports:

6. Your HTTP ports be like …

7. Network Security
for Microservices
Gordon the intern has
a brilliant idea…

8. Gordon wants to build a service
to tweet out all job offerings.
We’re Hiring!
Tweet
Service

9. GET /healthz
GET /jobs/{id}
GET /applicants/{job-id}
POST /jobs
API
GET /jobs/{id}
Jobs API
Service
Tweet
Service
The Jobs API service has all the
data Gordon needs.

10. GET /healthz
GET /jobs/{id}
GET /applicants/{job-id}
POST /jobs
API
GET /jobs/331
GET /jobs/{id}
Jobs API
Service
Tweet
Service
Gordon uses the GET /jobs/ API call

11. GET /healthz
GET /jobs/{id}
GET /applicants/{job-id}
POST /jobs
API
GET /jobs/331
GET /jobs/{id}
TLS Jobs API
Service
Tweet
Service
Developer etiquette.
Super simple stuff.
Gordon uses mutual TLS Auth
Good thinking Gordon

12. L3/L4
GET /healthz
GET /jobs/{id}
GET /applicants/{job-id}
POST /jobs
API
GET /jobs/331
The security team has L3/L4 network security
in place for all services
GET /jobs/{id}
Jobs API
Service
Tweet
Service
TLS
iptables -s 10.1.1.1
-p tcp --dport 80
-j ACCEPT

13. Gordon could
POST /jobs or GET /applicants
(mistakenly or haphazardly).
POTUS job available!
Tweet
Service

14. Jobs API
Service
L3/L4
GET /healthz
GET /jobs/{id}
GET /applicants/{job-id}
POST /jobs
API
exposed
exposed
exposed
GET /jobs/331
Large parts of the API are still
exposed unnecessarily
Tweet
Service
GET /jobs/{id}
TLS
iptables -s 10.1.1.1
-p tcp --dport 80
-j ACCEPT

15. Not exactly
least privilege
Security

16. GET /healthz
GET /jobs/{id}
GET /applicants/{job-id}
POST /jobs
API
GET /jobs/331
Back to the drawing board…
GET /jobs/{id}
TLS Jobs API
Service
Tweet
Service

17. L3/L4
GET /healthz
GET /jobs/{id}
GET /applicants/{job-id}
POST /jobs
API
GET /jobs/331
Least privilege security for microservices
GET /jobs/{id}
FROM “TurtleTweets”
ALLOW “GET /jobs/”
TLS Jobs API
Service
Tweet
Service

18. We demand
a demo

19. BPF - The
Superpowers
inside Linux

21. Kubernetes Integration

22. Kubernetes Integration
NetworkPolicy
Standard Resources
L3, L4 policy (ingress only in k8s 1.7)

23. Kubernetes Integration
NetworkPolicy
Services
Standard Resources
L3, L4 policy
ClusterIP, NodePort, LoadBalancer

24. Kubernetes Integration
NetworkPolicy
Services
Standard Resources
L3, L4 policy
Pods Pod Labels to specify policy on
ClusterIP, NodePort, LoadBalancer

25. Kubernetes Integration
NetworkPolicy
Services
Standard Resources
L3, L4 policy
Nodes
Pods Pod Labels to specify policy on
ClusterIP, NodePort, LoadBalancer
NodeIP to Node CIDR mapping

26. Kubernetes Integration
NetworkPolicy
CiliumNetworkPolicy
Services
Standard Resources
Custom Resource Definitions (CRD)
L3, L4 policy
L3 (Labels/CIDR), L4, L7 (ingress & egress)
Nodes
Pods Pod Labels to specify policy on
ClusterIP, NodePort, LoadBalancer
NodeIP to Node CIDR mapping

27. Should I encapsulate or not?
Node 1
Node 2
Node 3
Mode I: Overlay

28. Should I encapsulate or not?
Node 1
Node 2
Node 3
Mode I: Overlay
Name NodeIP Node CIDR
Node 1 192.168.10.1 10.0.1.0/24
Node 2 192.168.10.8 10.0.2.0/24
Node 3 192.168.10.9 10.0.3.0/24
Kubernetes Node resources table:
Installation
Run the kube-controller-manager with
the --allocate-node-cidrs
option

29. Should I encapsulate or not?
Mode I: Overlay Mode II: Native Routing
Node 1
Node 2
Node 3
L3
Network
Use case:
• Run your own routing daemon
• Use the cloud provider’s router
Use case:
• Simple
• “Just works” on Kubernetes
Node 1
Node 2
Node 3

30. L3 Policy (Labels Based)
Metadata
Allow from
pods
Pods the policy
applies to…
From Pod
To Pod

31. L3 Policy (CIDR)
Metadata
Allow to
IP 8.8.8.8/32
Pods the policy
applies to…
To CIDR
From Pod

32. L4 Policy
Metadata
Policy applies
to pods …
Allow incoming
on port 80
Pod
To Port

33. L4 Policy
Rule 2:
Allow PUT
If header is set
Rule 1:
Allow “GET /v/1”
L7 Policy – Only allow “GET /v1/”
Allowed
API
Calls

34. How are these policies enforced?

35. How are these policies enforced?
• L3 & L4: BPF in the kernel

36. How are these policies enforced?
• L3 & L4: BPF in the kernel
• L7: Sidecar proxy or KProxy / BPF

37. Node 2Node 1
ServiceService HTTP Request
What is a sidecar proxy?

38. Node 1
Service
Sidecar
Proxy
What is a sidecar proxy?
Node 2
Service
Sidecar
Proxy

39. Node 1
Service
Sidecar
Proxy
What is a sidecar proxy?
Node 2
Service
Sidecar
Proxy

40. Node 2Node 1
ServiceService
HTTP RequestSidecar
Proxy
Sidecar
Proxy
What is a sidecar proxy?

41. Node 2Node 1
ServiceService
HTTP RequestSidecar
Proxy
Sidecar
Proxy
What is a sidecar proxy?
Provides L7 functionality
• Routing / Load balancing
• Retries
• Circuit breaking
• Metrics
More info? Google is your friend “sidecar” / “service mesh”

42. Node 2Node 1
Service
Operating
System
Service
Network
Sidecar
Proxy
Sidecar
Proxy
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
• 3x Socket memory requirement
• 3x TCP/IP stack traversals
• 3x Context switches
• Complexity
Networking Path with a Sidecar
Network

43. Can we turn
the sidecar
into a racecar?

44. Node 2Node 1
Task
Operating
System
Kernel Proxy
Task
Network
Socket
KProxy
with
BPF
TCP/IP
Socket
TCP/IP
KProxy
with
BPF
kTLS kTLS
Sidecar
Proxy
Sidecar
Proxy
Network

45. Socket Redirect
Task
Socket Socket
Task
TCP/IP TCP/IP
Loopback

46. Socket Redirect
Task
Socket Socket
Task
TCP/IP TCP/IP
Loopback

47. Socket Redirect – Performance?
More info: https://www.cilium.io/blog/istio

48. Node 2Node 1
Service
Operating
System
Service
Network
Sidecar
Proxy
Sidecar
Proxy
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
Socket
TCP/IP
The Before and After
Network

49. Node 1 Node 2
Service
Operating
System
Service
Network
Socket
TCP/IP
The Before and After
KProxy
Socket
TCP/IP
KProxy
Network

50. Cilium Summary
• Kubernetes, Mesos, Docker
• CNI / libnetwork
• Networking: Overlay or Native Routing
• Network Security (ingress/egress)
• L3 (Identity or CIDR), L4
• L7: HTTP (0.11), Kafka (0.12), gRPC (0.12)
• Load Balancing (XDP / BPF)
• Dependencies: kvstore (etcd / consul)

51. Application-Aware Security
for Microservices via BPF

52. @ciliumproject
Star Us on GitHub! http://github.com/cilium/cilium
Thank You! Questions?
Tutorial / Getting Started:
http://cilium.io/try