In security monitoring, sensors collecting suspicious events and SIEM (Security Information & Event Manager) detecting alerts by combining these events are getting a lot of attention. However, in actual operation, investigations and risk management after detection are more important tasks. Because the workload of these tasks is usually higher. In order to reduce this workload, products and services called SOAR (Security Orchestration, Automation and Response) have begun to emerge as a framework for automating the collection of threat information and investigation of compromised resources that are required after alert detection. By using such a framework, it becomes easier to respond to changes in business and security situations. To archive the structure, it is important not to divide a development team of infrastructure and an operation team. In this talk, I will introduce the approach of security monitoring work efficiency improvement based on DeepAlert, which is a SOAR framework that the speaker himself built. DeepAlert is abstracted into three steps: investigation of alert related information, risk evaluation of alerts, and response to alerts, and we will introduce how each contributes to reduction of load in monitoring work. Also, when utilizing SOAR, it will be possible to expect more flexible and quick responses by integrating the infrastructure and operations as much as possible. I would like to discuss “SecOps”, that is incorporating the culture of software development into security operations, and the future of cybersecurity countermeasure teams.

1. Effective Security Monitoring by
SOAR and SecOps
Cookpad Inc
Tech Department, Security Team Leader
Masayoshi Mizutani (水谷 正慶)
2020.10.29 (Thu)
CODE BLUE 2020

2. Today s Agenda
•What is SOAR?
•SOAR Case Study
•Security by Software Engineering
2

3. Who are you?
•Ph.D.（Media and Governance）
•2011.4 ∼ IBM Japan
• Research and development of security product/service (e.g. SIEM)
• Analyst of SOC (Security Operation Center)
•2017.11∼ Cookpad Inc
• Security Team Leader
• Development and operation of internal security infrastructure
• CSIRT, Information Security Committee
3
Masayoshi Mizutani (水谷正慶)

4. Background
4

5. Background of our CSIRT
•Limited human resources
• Only 2 active members
• Working for not only incident response but also company risk management
•Focusing on Security Monitoring
• Guardrail, not Gatekeeper
• Prevent to brake business activities by security rules
• Capability of detecting security breach and investigation
5

6. Components of Security Monitoring
6
Logging Alerting
Anti Virus
EDR (Endpoint
Detection & Response )
Network-based IDS
(intrusion Detection System)
System / Service Logs
WAF (Web Application Firewall)
Audit Logs
Network Logs
Sensor
Manager
Log Manager SIEM (Security Information & Event Manager)

7. Cycle of Security Monitoring
7
Observation Detection Response
Gathering security
events from own
system/environment
Detecting potential
of security breach by
observed event(s)
Judging severity of
the detected alert by
investigation
• Preserving evidences
• Mitigating damage
Sensors
SIEM
😫 Need big manpower
Triage
(Assessment)

8. SOAR
(Security Orchestration, Automation and Response)
8

9. (2) Detection
(3) Triage
(4) Response
SOAR (Security Orchestration, Automation and Response)
9
SOAR
Alerting Product
(e.g. SIEM)
Other Products/
Services
Other Products/
Services
Security
Alert
Information
related the alert
Change
configuration
•Automation for Triage and Response Phases in Security
Monitoring with Various Products and Services
• Originally coined by Gartner in their report (Nov, 2017)
• Definition is slightly different by each security vendor

10. SOAR (very simple) Use Case
10
SIEM
IP address
reputation service
AWS
Detect suspicious activities in
a EC2 instance on AWS
One of IP addresses is
Command & Control Server!!
Take snapshot and
quarantine the instance
Inquiry about communicated
IP addresses
Various
Logs Playbook, Runbook, etc.
SOAR

11. Benefit of SOAR
•Saving Labor workload
• Full automated triage + response for known alerts
• Reducing workload of triage for unknown alerts
•Advantages
• Helpful for a busy analyst by making alert handling easy
• Do not hesitate to monitor new alerts
11

12. SOAR Case Study:
Security Monitoring Automation
12

13. Technical Challenges of SOAR implementation
•Flexibility
• How can we describe complicated workflow?
• Proprietary language does NOT work
• Eco-system
• Maintenanceability
•Extensibility
• Many various orchestration services and products
• Integration with internal services and products
•Abstraction
• Workflow models
• Data models
13

14. DeepAlert
•Our original SOAR
framework
• Implement with AWS CDK
• Three main steps
• 1) Inspection: Enrich parameters
• 2) Review: Evaluate severity
• 3) Emit: Take action
• Started development at early
2018
14
https://github.com/deepalert/deepalert

15. DeepAlert Stack
[Step1] Inspection: Enrich parameters
•From external data source
• Reputation services (VirusTotal, Hybrid Analysis, urlscan.io, etc)
•From internal data source
• Service and system logs (G Suite, AzureAD, AWS, Internal servers)
• Endpoint logs (CrowdStrike Falcon, osquery)
15
SNS
(Notification Service)
SQS
(Queue Service)
VirusTotal
Stack
urlscan.io
Stack
CloudTrail
Stack
Alert

16. [Step2] Review: Evaluate severity
•Evaluation policy written in Go on AWS Lambda
• See not only security alert but also enriched data
• Eventually, choice severity from Safe, Unclassified and Urgent
•Policy is managed in GitHub Enterprise
• Reviewable code
• Testable code
• Change history management
16
Security as Code

17. Review Policy Changes
17
•Change by PR (Pull Request)
• On GitHub Enterprise
• Reviewed by a team member
• Can comment to code
• Can approve of changes
• Change management
In the same manner as
modern software development
Comment
Change history

18. Test Policy Changes
18
•Test by General Framework of Go
• Review Lambda Function is implemented
with simple input/output
• Input: Alert and Data
• Output: Severity
• Easy to write unit test by simple I/O
Original Security Alert
An enriched parameter
from internal data source
(Also) In the same manner as
modern software development
Run unit test

19. [Step3] Emit: Take action
•Notification
• Slack, PagerDuty (Obsoleted)
•Incident Ticket Creation
• GitHub Enterprise
•Quarantine and Evidence Preservation (Optional)
• Shutdown network by endpoint security service
• Shutdown network and take snapshot by cloud provider s function
• We do not quarantine for now because low frequent
19

20. Improve Security Monitoring Operation by SOAR
•Reducing a number of triage/response
• Over 50% cases are closed automatically (2019 results)
•Reducing time to triage
• An analyst starts triage phase with enriched alert information
•Low cost
• Average < $0.5/day
20

21. Security by
Software Engineering
21

22. SOAR is Good Example of Software Engineering
•Security monitoring system must be updated continuously
• Changing status of organization
• Changing attacker s trend
• Changing capability of sensor and alert detector
•For continuous reinforcement loop of SOAR…
• Development of new features
• Update, fix and reinforce policies
• Continuous Integration
• Continuous Delivery
22
Modern DevOps
techniques are appliable

23. SecOps
•Apply DevOps principles to Security systems
• Like MLOps (apply DevOps principles to ML systems)
• DevSecOps means security check in CI pipeline in general
•Build and operate your security system by your own
self
• To archive scalability, extensibility, agility and capability
• For monitoring, compliance, risk management, etc
• Prior examples: Netflix, Capital One, etc
23

24. Case Study: SRE
•Site Reliability Engineering (Engineer)
• A discipline that incorporates aspects of software engineering and
applies them to infrastructure and operations problems (from Wikipedia)
•From Google Book
• Keeping operational work (i.e., toil) below 50% of each SRE’s time
• Reducing toil and scaling up services is the ‘Engineering’ ”
• They continuously improves their works by software engineering
All principles of SRE are not required in security
context, but we can learn more from them
24

25. Approach to SecOps (from my experience)
•Change the culture
• Full commitment for operational works is bad
• Changed from top (leader or manager)
•Both of development skills and security knowledges
are required
• Keep your motivation for information security
• Learn from modern software development techniques
25

26. Conclusion
•SOAR is a concept of automation and orchestration in
triage and response phases in security monitoring
•DeepAlert is Cookpad owned SOAR framework with
serverless architecture on AWS
• Evaluate severity with internal/external data source and take action
• Use modern software development techniques to manage policy
•SecOps concept may become more important
• SOAR is a good example that SecOps concept works well
• SecOps can helps other systematization
26

27. Thank you
27

28. SOAR is One of Systematizations (Again)
•Handle Repeated Task
• Prevent human error
•Reduce Workload by Automation
• Evaluate severity automatically
•Standardization of Workflow
• Establish common procedures even if not automated
28

29. SecOps is a Key of Systematization
•SecOps means…
• Other Ops definitions
• DevOps = Bring culture and technology of software development to operation
• MLOps = Apply DevOps principles to ML systems
• No established definition of SecOps, but in my opinion…
• SecOps = Apply DevOps principles to Security systems
•Continuous Development by own-self
• Quick bug fix and feature update
• CI/CD for high frequent deployment
• Feedback directly (a.k.a. Dog Fooding)
29

30. Another Systematization Example
•Implement Pipeline to Scan
Vulnerability of Container
Image with Trivy & AWS
• As we say DevSecOps
• Isolated and continuous container
image scan
• Not blocking deployment
• Scalable scanner
https://techlife.cookpad.com/entry/catbox
30

31. Data Models
•Simple & Practical
•
31

32. DevOps Principles Accelerate Systematization
•Why only existing product/service is not enough?
• Various requirements according to your organization and team
• Need to integrate security components
• A lot of security products/services, but they are just components
• Increase development velocity
•DevOps for Security
32
SecOps

33. Improve Security Monitoring Operation by SOAR
•Nothing to do if severity is Safe
• Automatically close the case
• Security analyst gets only notification
•Can start response with all required information if
severity is Unclassified
• Not only alert, but also related data
• No matter who gathers data, the procedure and result will be the same
•Quarantine the compromised resource at the fastest
• If enabled
33
Response time of handling security alert can be reduced

34. Why Developed Own SOAR?
•Extensibility
• For Integration with various services/products
• For Integration with internal developed system
•Flexibility
• Workflow can be written in mature programing
language
•Workflow as Code
• Reviewable Workflow (with Github Enterprise)
• Change History Management
•Cost
• Pay per alert by serverless architecture
• Less than $1/day
•Under Control
• Update, degrade, enhancement 34
•Maintenance Cost
• Bug fix, feature update
• Operational running cost
•Skill Set of team members
• Require both skills of software
engineer and security analyst
Pros Cons

35. Security Dev&Ops
35
Dev team Ops team
Weak
Isolated Dev and Ops team
Dev team Ops team
Stronger
Integrated Dev and Ops team
Dev&Ops team
Strongest
Dev&Ops done by one team

36. Another Engineering Example
•Implement Pipeline to Scan
Vulnerability of Container
Image with Trivy & AWS
• As we say DevSecOps
• Isolated from CI/CD process
• Scalable scanner
https://techlife.cookpad.com/entry/catbox
36

37. Leverage
37
Security Operation
Gap
Cloud Services /
Tools and Products
Human
Resource
Organization
Culture
Business Goal
Existing
Infrastructure

38. 38
Cloud Services /
Tools and Products
Security Operation
Software Engineering
Reinforcement
Orchestration