1.
Adversary Driven Defense
in the Real World
Shannon Lietz - @devsecops
James Wickett - @wickett
1

2.
MY pseudo JOURNEY LINE... HOW I SPEND MY DAYS...WHAT MAKES ME HUMAN...
Shannon Lietz (@devsecops)
Sug l
fa e
DEV
SEC
OPS
DSO
RGD
1984
1989
1996
2001
2011
COMICS
#HACKERGIRL
2

3.
James Wickett (@wickett)
● Head of Research @ Signal Sciences
● Signal Sciences provides a NextGen
WAF and a RASP
● Author of DevOps and DevSecOps
courses at Linkedin Learning
lnkd.in/JamesWickett
● Organizer for DevOps Days Austin
@devsecops || @wickett 3

4.
“Companies are spending a great deal on security, but we
read of massive computer-related attacks. Clearly
something is wrong. The root of the problem is twofold:
1) we’re protecting the wrong things, and
2) we’re hurting productivity in the process.”
@devsecops || @wickett
- Steven Bellovin, Thinking Security
4

5.
5
“we’re protecting the wrong things”
@devsecops || @wickett

6.
6
NIST
800-53
@devsecops || @wickett
PCI DSS
OWASP
Top 10
GDPR
ISO
27000
CSC
Top 20
GLBA ...

7.
The Problem… Are we chasing the right issues?
1. How are the current issues the “right” issues?
2. Is what we are testing driving us towards the “right” issues?
3. Are we using the “right” tools?
How will we know?
@devsecops || @wickett 7

8.
Adversary Perspective
@devsecsops || @wickett8
Motives
OpportunitiesMeans

9.
@devsecops || @wickett 9

10.
Tools used in this Research
HONEY SCANNERS DETECTION
@devsecops || @wickett 10

11.
OWASP Top Ten is just
the most recognized part
of the Problem
You Can’t Secure
New App Tech w/
Legacy AppSec
Account Takeover
Direct Object Reference
Forceful Browsing
Feature Abuse
Evasion Techniques
Subdomain Takeover
Misconfiguration
• Legacy WAFs focus on the
same threats as 15 years ago
• False positives result from generic
signatures without context
• Rarely used in blocking mode
OWASP Injection
Attacks
Real-World Problems
11

12.
OWASP vs. Real World
OWASP Top 10
Advanced Adversaries
%
Perceived
Success
Number of
Adversaries
+ IPs
Scanners
Researchers
Paid Noise
@devsecops || @wickett 12

13.
Automated Scanners
● Continuously running on a
schedule
● Scanners run for good and/or bad
purpose
● Cost of running vs. Cost of
information discovered
@devsecops || @wickett 13

14.
Researchers
● Commonly apply their efforts to
get paid through bug bounties
● More likely to use common tools
and standards
● Time spent must be worth effort
@devsecops || @wickett 14

15.
Paid Noise
● Running when other attacks occur
● Used to outrun automated
detection and AI/ML
● Cost of running must be low
enough to allow for profit
@devsecops || @wickett 15

16.
Advanced Adversaries
● Commonly low and slow
● Leverages more human assisted
automation schemes
● Investment must not be easy to
disrupt
@devsecops || @wickett 16

17.
17
“we’re hurting productivity in the process”
@devsecops || @wickett

18.
OWASP TOP 10 App Sec Risks Real-World Top 10 Attacks
1 Injection Direct Object Reference
2 Broken Authentication Forceful Browsing
3 Sensitive Data Exposure Null Byte Attack
4 XML External Exposures (XXE) Command Injection
5 Broken Access Control Feature Abuse
6 Security Misconfiguration Evasion Techniques
7 Cross Site Scripting Subdomain Takeover
8 Insecure Deserialization Misconfiguration
9 Using Components with Known Vulnerabilities Cross Site Scripting
10 Insufficient Logging/Monitoring SQL Injection
Top 10 Comparison
@devsecops || @wickett 18
Less Guessing...

19.
Being Adversary Driven
changes the conversation
19 @devsecops || @wickett

20.
How do we change the game?
@devsecops || @wickett20

21.
DevSecOps Observability
and Measurement Insight
21
- Correlate anomalies and
attacks alongside regular
app flows
- Ex: HTTP 400s + ATO
events + address changes
- But… it varies! (by app, by
vertical, by lang)
@devsecops || @wickett

22.
@devsecops || @wickett22

23.
@devsecops || @wickett23

24.
Measurements
1) How often do adversaries return? Return Rate
2) How often do adversaries change their tactics? Rate of Change
3) How confident is the adversary? Cost of fix
4) How long do they have to find an issue? Mean Time to Identification
@devsecops || @wickett24

25.
Some interesting insights...
Bad guys:
● like to use scanning signatures to whitelist themselves
● don’t use commercial scanners except for noise or whitelisting
● have a few “goto” TTPs because they just work
● don’t underestimate the value of cryptocurrency mining
○ labs.signalsciences.com/using-signal-sciences-to-defend-apache-struts-cve-2018-11776
● are not afraid of AI/ML
● hide in lots of noise
@devsecops || @wickett 25

26.
How do we correct continuously?
•Everyone knows Maslow…
•If you can remember 5 things,
remember these ->
“Apps & data are as safe as where
you put it, what’s in it, how you
inspect it, who talks to it, and how
its protected…”
@devsecops || @wickett 26

27.
Call to Action
27
• Crawl
• Assess your attack surface
• Instrument and collect telemetry
• Determine basic patterns in your data
• Walk
• Examine telemetry data and determine the characteristics for your application’s adversaries
• Can you say who your top adversary or attack is?
• Run
• Understand how to forecast the most important issues to fix
• Be able to measure and report on defects fixed ahead of adversaries
@devsecops || @wickett

28.
28
What do we need
help with?
We are writing a book along with
Ernest Mueller and John Willis on
DevSecOps.
We are looking for stories of
DevSecOps transformations,
journeys, successes and failures.
book@devsecops.org