What do you remember about the Equifax? Something about someone forgetting to patch Struts, and then the bad guys were able to get in and steal all the data? What actually happened was much more nuanced, and there's much to learn by diving into the details.
2. 2
Adrian who?
10 years as security practitioner (all the hats)
5 years as a security consultant (pen tester and PCI
QSA)
3 years as an industry analyst
2 years building my own company and working for vendors
Founded several local cybersecurity community groups in
East Tennessee
Old enough to remember Mastercard’s SDP before PCI DSS
was a thing
Now: cybersecurity product reviews at Security Weekly
Labs
6. 6
Common traits across breaches
Attacks aren’t ‘one-and-done’ - they have multiple
phases and take time
“Attackers only need to succeed once,
defenders need to get it right every time”
“Defenders only need to detect attackers
once, attackers have to evade every time”
7. 7
Common traits across breaches
• Vulnerabilities exploited in less than 3% of breaches
- The vulns that ARE exploited are OLD – over 91% are over a year
old!
• Malware isn’t always used
- But when it is, it’s in the middle and end stages of the attack
• Most attacks require four or more steps
- Especially system intrusions
- Webapp attacks have the shortest number of steps – think SQLi
or open S3 bucket
Source: 2021 Verizon data Breach Investigations Report
8. 8
Common traits across breaches
Most look pretty much like every pen test ever (ouch)
1. Phish an employee; exploit a vuln or misconfiguration
2. Steal creds
3. Log in via [technology]
4. Dump admin credentials
5. Pivot using newfound creds, maybe sprinkle some
malware
6. Own everything
10. 10
Struts vuln announced: CVE-2017-
5638
March 7th: struts vuln
announced
March 9th: Equifax’s GTVM
group urges everyone to
patch within 48 hours!
OMG, patch
struts now!
11. 11
Struts? What struts?
March 10th: First evidence of
struts exploits on Equifax
systems
March 14th: Emerging Threats
team releases faulty Snort
rule; Countermeasures team
installs it (neither team
tests it)
March 15th: Employees scan for
Struts, finding nothing
(McAfee Vuln Mgr)
March 16th: GTVM holds a
special meeting on this
Struts vulnerability
I got
nothin’
12. 12
Attackers break in
May 13th: 2 months after the
special Struts vuln
meeting, attackers exploit
Struts and drop web shells
Attackers hang out, explore
systems, run queries,
exfiltrate data – all
undetected for another 76
days
13. 13
Additional context
• The system attacked (ACIS)
was ancient, originally
built in the 70s
- Few understood it
- It was not well
documented
- J2EE
- Still, it was exposed
to the public Internet
• Equifax had a lot of
ground to cover
• Scanning tools seemed
ineffective and improperly
15. 15
Equifax discovers hijinks
July 29th,2017 at 9pm:
Countermeasures team
updates 67 SSL certificates
on SSL Visibility (SSLV)
device. They had been
expired since January 2016.
July 29th at 10pm:
Suspicious connections from
China are spotted
July 30th at 12:41pm: The
hacked system is taken
offline
July 30th at 1:30pm: CSO is
notified of the incident
16. 16
The initial response? (once they
knew, of course)
The Countermeasures team
identified the attack,
contained it and pulled in
key stakeholders in ~15.5
hours.
If the SSL Visibility
device was properly
maintained…
17. 17
Struts vuln: found at last
July 31st: The vulnerability
assessment team scans the
ACIS WAR file and finds
vulnerable Struts
July 31st: The team scans
more ACIS related systems
and finds more vulnerable
Struts not being inspected
by the SSLV IDS system
18. 18
Communication breakdown
July 31st: CSO suspects PII
is compromised, but doesn’t
tell CIO
August 2nd: CIO goes on
vacation for 2 weeks
August 2nd: Equifax calls
outside counsel, who then
calls Mandiant
August 3rd: Mandiant gets to
work
September 7th: Public notice
goes out
The public notice also
19. 19
Don’t skip those PR classes, CEOs
“…three weeks before Equifax publicly announced the
breach, Smith boasted Equifax was managing ‘almost 1,200
times’ the amount of data held in the Library of
Congress every day.”
22. 22
Equifax Process and Control
Failures
1. No asset inventory (CSC01
2. No software inventory (CSC02)
3. No file integrity monitoring
4. No network segmentation
5. Neglected SSL Visibility (SSLV)
Appliance
6. Neglected SSLV failed open
7. SSLV lacked certs for key systems
8. SAST failed to find Struts due to
misconfig
9. No anomaly detection on web
servers
10. Custom snort rule didn’t work
11. Custom snort rule wasn’t tested.
12. Network scanner didn’t find
Struts
16. Least privilege principles not
followed for database access
17. Ad-hoc DB queries not restricted
18. No DB anomaly monitoring
19. No field-level encryption in DBs
20. No data exfiltration detection
21. DAST scanning failed to detect
vulns
22. Ineffective IR plan/procedures
23. No owners assigned to apps or DBs
24. Comms issues due to corp
structure
25. Lack of accountability in
processes
26. No post-patching validation
27. Old audit findings were not
23. 23
Equifax Process and Control
Failures
1,12 26,28 29
2,8,21,23 26 3,9,13,14
4,5,6,7,1
6
10,11,20
15,23 16,17,19 17,18,20
24 25,26,27 22
Applicat
ions
Devices
Network
Data
People
Ident
ify
Recove
r
Respon
d
Detec
t
Prote
ct
Proces
s
Technolo
gy
People
Degree of
Dependence on
People, Process,
Technology
https://cyberdefensematrix.com/ - created by Sounil Yu - @sounilyu
24. 24
Summarizing the worst
control failures
Summarizing only the most egregious control failures:
Tech-oriented control failures: Zero
People-oriented control failures: One
Process-oriented control failures: Eight
Special thanks to Sounil Yu, the cyber defense matrix creator, for reviewing my work and making some
suggestions and corrections!
25. 25
References
House oversight report
Senate subcommittee report
Chinese military hacker indictment
Me, live-tweeting my way through the house
oversight report
Talos’s day 2 post reporting exploit activity
The original Struts vulnerability security
bulletin
The headline isn’t wrong, but as an answer to what question?
Why did attackers get an initial foothold into Equifax?
Why did Equifax fail to detect and respond to the attack in a reasonable timeframe?
Ultimately, breaches are almost never due to a single control failure, and the failure to patch here wasn’t because Equifax was slow or not paying attention.
Why did I pick the Equifax breach? Mainly because it’s an excellent example of a large, complex environment, heavy with tech debt. Also, because SO MANY details were made public that we can really have a meaningful conversation about what went wrong at each stage.
2017/03/07 - The struts vulnerability is announced by Apache
2017/03/08 - Talos blogs that they’re seeing live exploitation attempts (meaning the opportunistic scanning has begun)
2017/03/09 – Equifax’s Global threat and Vulnerability Management (GTVM) team forwards a USCert notification internally, noting the issue must be fixed within 48 hours!
2017/03/15 – Metasploit module released
So the way this likely happened was that attackers were just blasting out scans for vulnerable systems across the whole Internet. This is likely why we see some early exploitation, but nothing comes of it. Might not even be the same actor.
All this stuff about “we know patching is hard, but we’ve got to do better” punditry assumed that Equifax was aware of the need to patch, but their corporate wheels spun too slowly.
The reality was somewhat worse – they didn’t know they even had instances of Struts exposed to the public Internet that needed to be patched!
Also, why are they holding a special meeting about struts 5 days after the deadline for everyone to fix it? Because they suspect no one had?
There was seemingly no significant monitoring on this legacy system. No FIM, no anomaly detection.
Used to exfiltrate massive amounts of data (placed in webroot and retrieved with wget), without any detection
Further, the system broke many security policies:
having more access to systems than necessary (only needed 3 databases, had access to 51)
Storage of cleartext creds
He had to have known the breach investigation was well underway at this point, suggesting he was either
1. clueless about the severity of the breach, or
2. clueless about how tone deaf his statement was, given the circumstances
Okay, it wasn’t an intern, but I couldn’t pass up that title and it’s not far off the mark.
Also, recognition for a true American hero: the Monopoly cosplayer photobombing this hearing like crazy
Check on specifically what happened with #15
What the cyber defense matrix tells us:
1.
Put another way, all the technology did its job, but people or processes failed to keep it in a functional state.