1.
Cybraics Confidential 2016 1
Man & Machine –
Working Together to Solve
the Cyber Security Problem
Pete Nicoletti
Chief Information Security Officer, Cybraics, Inc.

2.
• What is the Cybersecurity Problem?
• Top Contributing Factors
• How to Leverage AI/ML to Target and Reduce Contributing Factors
• How to Test and Measure Success with AI/ML Projects
• The Future
2
Agenda

3.
The Cybersecurity Problem
3
Figure 1 – Publicly Reported Cyber Breaches in North America 2005 - 2017
Number of breaches and records exposed per year is escalating despite billions of
dollars of investment in cyber tools and the best efforts of security professionals
• “Up and to the
right” - NOT success
on a cybersecurity
breach graphic
• Breaches in North
America have
almost increased by
an order of
magnitude in the
last 10 years.

4.
The Cybersecurity Problem
4
First, what is the cybersecurity problem? The cybersecurity problem is best explained by Figure 1, which illustrates the number of breaches reported in North
America over the last 12 years. The number of breaches and records exposed per year is escalating despite billions of dollars of investment in cyber tools
and the best efforts of security professionals.
Figure 1 – Publicly Reported Cyber Breaches in North America
“Up and to the right” is not
what one considers success
on a cybersecurity breach
graphic! Breaches in North
America have almost
increased by an order of
magnitude in the last 10
years.
4

5.
Top Contributors To “The Problem:”
5
• Conventional tools only combat
known attacks
• Incredible increasing volume of data
• Alert Overload
• Lack of Security Skills and Security
Professionals
• Efficacy of Conventional Security
tools averages only 85%
• Funding/Budget: There is never
enough
• Too many tool options and they don’t
talk to each other
• Lack of Effort on Security Framework
• Statistics Stink:
o Average time to fully contain a
breach is lengthening
o Cost of a breach remains very
expensive
o Length of time to identify a breach is
barely improving

6.
Conventional Tools Only Combat Known Attacks
6
• Most conventional security tools use:
o signature matching
o baselining
o rule matching
o threshold levels
• They have helped analysts better
understand attack surface activity
• However, tools only understand
known threats
They cannot identify or prevent
unknown attacks

7.
Advanced Tools Can Combat Unknown Attacks
7
Behavior Severity Rating
Analytic Confidence
Network Priority
Asset Priority
SOC Feedback
82
Asset Score
70
90%
• Cybraics’ AI/ML based analytics
scoring engine performs:
o Hunting
o Detecting
o Identifying anomalous behaviors
• Identifies known and unknown
attacks
• No:
o signatures need be updated
o rules created
o configuration efforts are necessary.
• SOC or orchestration engine make
immediate decisions
• Provides configurable features

8.
Incredible Increasing Volume of Data
• Volume of log/security/application
data collected growing exponentially
• Insights derived from this data is
fundamentally limited
• Data experts and tools are expensive
and in high demand
• Insights required by business
outstripping the small group of
trained experts
• No experts available in small
companies
8

9.
AI/ML – Deals with Huge Volumes of Data
9
• Humans can not work
at Thousands/Million/
Billions/Trillions scale
• Humans can not make
connections between
disparate systems and
events
• Humans can not
“scale” well
Cases (should)
be in parts per
billion

10.
Alert Overload
10
• Advanced threat detection -- point solutions scrutinizing network traffic in
traditional ways
o signature matching
o baselining
• Alerts generated -- thousands or 10s of thousands daily
• Alerts overwhelming cybersecurity analysts
• Analysts struggle to validate and escalate
• Almost half of security operation managers report
o receiving over 5,000 alerts per day
o over 50% of alerts = false positives
o average time to research each alert ~ 20 minutes
o average 4.35 days for Mean time to Respond for fully resolving cases
(IBM/Ponemon 2018)
o rule matching
o threshold breaches

11.
Lack of Skills/Security Professionals
11
• Huge problem
• Predictions of 3.5 million
unfilled positions by 2021.
• ~6 months to fill positions
• ~8 months to train
• 25% change organizations
within 2 years
(CSO Mag “Cybersecurity skills
shortage getting worse”)

12.
12
Empower your current Security Analysts!
Cybraics Confidential 2016
• Evolve from “Alert” to “Case” Capability
• Turn Level 1 Analysts into Level 2
• Machine Learning does the “Hunting”
• Know what cases are important using Network and Server Context
• Add appropriate 3rd Party Information
• Threat feeds, IOC Info, Articles
• Give Recommended Remediation Steps
• Turn Level 2 Analysts into Level 3
• Automatically consolidate all Entity Associated Logs
• Auto-create search strings for faster follow-up searching
• Support searching through all logs used for analytics and context
• Make White Listing Faster And: MAKE THE JOB MORE FUN AND REWARDING!

13.
Difficult to Implement a Security Framework
13
• Security experts strongly advocate adopting cybersecurity framework
• Most common
frameworks:
o PCI
o CIS: Critical
Security Controls
o NIST
o ISO 27000
o FINRA
• Frameworks designed to reduce risks but most small and medium
companies challenged to implement and maintain

14.
14
Get Started on a Framework:
Cybraics Confidential 2016
• Get Executive Sponsorship and Budget Commitment First
• Determine all the use cases for a compliance framework
• Self Audit to start
• Call in the Expensive Consultant since no one will believe you
• Engage the business
• Report on progress
• Hold everyone accountable
• Brag Everywhere… but remember: Every large Breached Company subscribed to
a compliance Framework
• Compliance Does Not Equal REAL SECURITY

15.
Efficacy of Conventional Security Tools ~85%
15
• 1 of every 130 emails contain malware
• Distributed workforce and BYOD
• Connections to unprotected business
partners
• Applications and Servers not maintained or
patched
• Firewall Configuration failures
• Signature based Tools not updated or
Signature Arrives too late
• No centralized monitoring
• Tools don’t work together
= 85% efficiency (Verizon NSS Labs Reports)
Virtually no way to keep all security tools updated and managed to protect a global
enterprise with data center and cloud deployments.

16.
More Funding for Cybersecurity Needed
16
Typical Challenges:
• Anything “new” must
replace something
• Executive relationships
with vendors
• No Breach = no
additional budget
Organizations spending more than ever on security
• 7 in 10 want at least 25% more $
• 17% want a 50% increase
• ~12% believe will receive budget increase >25%

17.
17
Don’t Waste Money on More Tools
Cybraics Confidential 2016
• Make your existing tools more efficient with ML Intelligence
• Consolidate logs with SIEM to see issues across all sources and
platforms
• Log the right stuff
• Usually no need for “Verbose”
• Check logging levels for each source
• Ensure that logs enable analytics and analysis
and forensics
• Oh yeah…and compliance
• Acquire logs from Data Center to Cloud
• Don’t use two separate systems
• Get more Life out of Current Firewalls and End Point solutions

18.
Breach Statistics Stink
18
Average time to contain a
breach is lengthening
• Average -- 66 days
(Verizon Breach Report 2018)
Average Cost per Breach
Cost = very expensive
• Average cost ~ $3-4M
• Lost and stolen records
cost ~ $140 -$150 per
record in 2017
• Average number of
compromised records
per breach - 24,000
(IBM/Ponemon)
Length of time to identify a
breach is barely improving
• 2018 - 191 days; a 5%
improvement from 2016
(Verizon Breach Report 2018)

19.
Use Real Data to avoid being a Statistic
Cybraics Confidential - Subject to NDA
19Cybraics Confidential 2018
19

20.
Too Many Tools & They Don’t Talk to Each Other
20
• Over 1600 vendors
• 70 Tools at average
large company
• No coordination of
tools in small/med
companies
• Very difficult to chair
swivel / head swivel
between tools
• Difficult to have one
pane of glass

21.
21
Tie your tools together into a Security Analytics & AI Platform
• Ensure Full coverage of threat space
• Get feeds from all sources
• Leverage custom analytics focused on cyber
• Benefits to a Fully-managed platform
Human Analysts in a Cyber Threat Center

22.
Historical
Data Pool
AI/ML
Model
Training
Models
Updated in
Platform
Process Live
Customer
Data
CTC Implicitly
Labels Data
(driven by
AI/ML)
CTC* implicitly labels data by
making decisions on model
results in real environments
(driven by AI/ML)
Labeled data is added back to historical pool Data pool is updated and made available to
analytics core
AI/ML models are trained
on available historic data
and open source data
Models are programmatically
and continuously updated in
platform
Models are applied to live data;
results are delivered to CTC* by the
Machine Analyst
Detection Engine: Analytics Core (AI/ML)
*CTC = Cyber Threat Center
22

23.
Leverage Machine Learning
Endpoint
AD
Firewall
DNS
Proxy
Raw Data
Examples
Analytics Core
• Machine Learning
• Artificial Intelligence
• Statistical Models
• Natural Language
Processing
Phishing
Malware
Scanning
DGA
DLP
Detection
Engine
Behavior
Examples
JSmith
Malware, DLP
10.1.1.1
Scanning
10.1.2.1
Phishing, DGA
Findings
Aggregation
90
25
67
Scoring Engine & Machine Analyst
Scoring engine result.
Scale = 1 – 100 based on priority (to
investigate or remediate)
Mathematical equation that weights multiple inputs and
provides a score ranging from 0 – 100, with a maximum
score of 100, that corresponds to malicious activity on the
highest priority asset at the organization.
Alerts
23

24.
Machine Learning Outlier Detection Details
Cybraics Confidential - Subject to NDA
24
• Illustration of outlier
distribution looks for
statistically significant
deviations representing
the most interesting
IPs/users
• Cybraics uses outlier
detection algorithms to
isolate significant
anomalies on the “tails”
of the curve – drives false
positives down/out

25.
25
The Process – How Does it Work….?
Ingestion Analysis Scoring Context Remediation
Environmental Logs
NetFlow
Active Directory
Firewalls
IDS/IPS
Web Proxy
DNS Servers
Secure Gateway
Web App Firewalls
OT/IOT Device Logs
Threat Intel Feeds
Anti-Virus
No custom sensors
or agents required
Multiple Factors
Behavior Severity
Rating
Analytic Confidence
Network Priority
Asset Priority
SOC Feedback
Business Priorities
Customer
Configurable
And Trainable
82
Multi-Modal AI/ML
40+ Algorithms
Ecosystem Baseline
Biased vs Unbiased
Decision Models
Anomaly Detection
“Normal” Deviations
Behavior Triggers
Known Threats/IOC
sAutomated,
advanced detection
Evidence Case Files
Summary Data
Risk Guidance
Associated Entities
Supporting Evidence
Outlier Summary
Entity Details
External Sources
Previous Instances
Associated IPs
Action Guidance
Remove Host
Block IP/Domain/URL
IP Access Blocked
Forensic Investigation
Exfiltration Analysis
Remove P2P App
User/Service Acct
Investigation
Acceptable Use
Guidance
Activity Validation
Activity
Incident Escalation
Credential Cancellation

26.
AI/ML Findings and Case Details
26
Wire Transfer PC
communicating with
Latvia, Russia, Cyprus

27.
Case Details: Associated Entities
27
Outliers
automatically
grouped into
Entities

28.
Case Details: Risk Assessment Created
28
Risk Assessment Value Popped up
when Analytics fired
Time Line of Machine
learning scores changing
and Analyst efforts are
all automatically created

29.
Case Details: Outliers Analytics Details
29
Details on
unauthorized C&C
communication to
one of the 4 C&C
servers

30.
Case Details: Risk Score
30
Easy to understand multifaceted Spider Web Risk Visual

31.
Case Details: External Case Information
CTC adds external commentary
offering research and context
31

32.
Case Details: Remediation & Recommendation Details
32
Remediation
Details
Recommendation
Details

33.
Case Details: Log Details & Global Search Capability
33
Log Search Function finds all related logs easily

34.
Case Details: Log GEO IP Details
34
GEOIP shows Latvia C&C
Location
Connections were made to 4
other locations of dubious
reputation

35.
35
Test your AI/ML Based System
• Determine use cases, compliance requirements
• Architect integration requirements
• Validate data transfer
• Document log lifecycle process
• Validate network, devices and hosts impact within acceptable
operational ranges
• Validate entire lifecycle of representative test events
• Validate findings of system with appropriate tools and processes
• Validate use cases
• Determine ROI for time savings, tool savings, response time reduction

36.
36
Tools to Estimate and Validate Impact

37.
• Leverage combination of unsupervised ML and behavioral
analytics to identify previously unknown threats
• Use ML techniques, coupled with AI to provide
context and automation to the findings and workflow
• ML and AI together will find threats and related
info faster and enable SOC analyst efficiency and help
reverse the alarming trend in data breaches
37
The Future: Fix the Security Problem!
Trust Me! I command You!

38.
Breaking the Ransomware Lifecycle
by using the
World’s Most Advanced Analytics Platform
July 2019

39.
Criminal Enterprise Set up
Domain and WWW site Created
Malware delivery method selected
Bitcoin Account established
Anonymous Email Account Set up
The ‘Net
Emails sent
Cloud &
Premises Based
User Clicks On Email URL
or other infection method
Firewall
Proxy
DNS
Outbound Connections
The ‘Net
Criminals
Criminal
WWW Site
Firewall
Proxy
IPS
End User Computer
Infected
Email Delivered to Inbox
The ‘Net
Criminal
WWW Site
Outbound
FAILURE 1
Email Passes Cloud
and Traditional
Protection
FAILURE 2
End User
Training Fails
FAILURE 3
Desk Top AV
Fails
FAILURE 4
DNS Allows
Resolution FAILURE 7
IPS Allows
Traffic
FAILURE 6
Firewall Allows
Traffic
FAILURE 5
Proxy Does Not
Block
FAILURE 10
Desk Top AV Fails
Again
FAILURE 9
Firewall Allows
Traffic
FAILURE 8
Proxy Does
Not Block FAILURE 11
East West Traffic
Not Inspected
FAILURE 14
Host Based
AV/HIPS Fail
FAILURE 13
No E-W Firewall
Stopping attacks
FAILURE 15
Outbound FW
Traffic Not
Stopped
FAILURE 12
Unpatched
Servers
Vulnerable
FAILURE 17
IPS Allows
Traffic
Microsoft ATP
And All others
Email Protections:
FAILURE 20
SOC Analysts
have Alert
Overload
FAILURE 19
SIEM’s log but
don’t prioritize
Network Attached Users &
Servers Scanned, Attacked, Encrypted
FAILURE 18
Proxy Does Not
Block
FAILURE 16
DNS Allows
Resolution
Servers Are Encrypted,
Backups Corrupted, Logs Deleted
Ransomware
Interrupts
Business
Pay Criminals
Bitcoin for Key
Happy
Criminals
Funded and
Emboldened Criminals
Attack More Users
Worldwide
The Lifecycle of Successful Ransomware
Traditional Defenses Have Multiple Failure Points

40.
FAILURE 20
SOC Analysts
have Alert
Overload
FAILURE 19
SIEM’s log but
don’t prioritize
BREAK the Lifecycle of Successful Ransomware
Empower Your Traditional Defenses with nLighten!Criminal Enterprise Set up
Domain and WWW site Created
Malware delivery method selected
Bitcoin Account established
Anonymous Email Account Set up
The ‘Net
Emails sent
Cloud &
Premises Based
User Clicks On Email URL
or other infection method
Firewall
Proxy
DNS
Outbound Connections
The ‘Net
Criminals
Criminal
WWW Site
Firewall
Proxy
IPS
End User Computer
Infected
Email Delivered to Inbox
The ‘Net
Criminal
WWW Site
Outbound
• Analytics Find & Alert on Unauthorized Activities in DNS,
Firewall, Proxy, AV, IPS & AD Logs
• Security Analysts Alerted - can see every stage of attack;
typically stop attack at End User Level before costly
damage
FAILURE 1
Email Passes
Protection
Logs Created
Logs Created
FAILURE 2
End User
Training Fails
FAILURE 3
Desk Top AV
Fails
FAILURE 4
DNS Allows
Resolution
FAILURE 7
IPS Allows
Traffic
FAILURE 6
Firewall Allows
Traffic
FAILURE 5
Proxy Does Not
Block
FAILURE 10
Desk Top AV Fails
Again
FAILURE 9
Firewall Allows
Traffic
FAILURE 8
Proxy Does
Not Block
Logs Created
FAILURE 11
East West Traffic
Not Inspected
FAILURE 14
Host Based
AV/HIPS Fail
FAILURE 13
No E-W Firewall
Stopping attacks
FAILURE 15
Outbound FW
Traffic Not
Stopped
FAILURE 12
Unpatched
Servers
Vulnerable
SUCCESS:
Stop Ransomware before
business interruptions / costly
recovery efforts
FAILURE 17
IPS Allows
Traffic
Logs Created
Microsoft ATP
And all others
Email Protections:
Network Attached Users &
Servers Scanned, Attacked, Encrypted
FAILURE 18
Proxy Does Not
Block
FAILURE 16
DNS Allows
Resolution
Logs Created
Logs Created

41.
Pete Nicoletti pete@Cybraics.com
• CISO for Hertz Global, Virtustream/RSA/EMC/DELL, VP
Security Engineering Terremark
• Gartner’s “most secure cloud design” #1 and #2
• Whitehouse.gov, FBI.Gov, DOT.gov, VA, Library of Congress
and more Federal Projects
• Managed two clouds through FEDRAMP and eventually to
EAL 5
• Book Author/Contributor: “An Intel Reference Design for
Secure Cloud”
• 20 years Security, Red Team leader, Incident Response leader
• Chief Data Officer at Cybraics focused on Security Operations,
NIST 800-53 certification and product development
• Secret Service Miami Electronic Crime Task Force, FBI
Infragard Contributor
• Awarded Top 100 Global CISO in 2017
Pete has 31 years of impressive success and responsibility in the deployment, marketing, sales, product
development, engineering design, project implementation and operation of information technology,
IaaS/SaaS/PaaS, cloud, data center operations, the entire spectrum of security technologies, compliance
frameworks, Global Security Deployments and operations and Managed Security Service Provider services
and operations.
Prior Experience:Skills
IR, Product Management, Cyber Security, MSSP, Operations
Education/Certifications/Presentations
BS University of Tennessee
CCSK, CISA,CISSP,SANS GIAC, FCNSP, CCSE
“Opensource Security Concerns” Security Mag. 11/17
“How to Sell Cybersecurity to your Team” CSO Mag.
“Not Obscured by Clouds, Forensics and Cloud Visibility,”
Netscout Global Conference, April, 2017
“Cloud Forensics, A practitioners View,” CS World 2016
“Cloud Security, Latest Developments,” ISSA Conf 2016
“Auditing the Cloud Challenges,” ISACA Conference, 2017
“Best Practices and the Latest Advances in Cloud Security,”
THANK YOU!