The document summarizes a literature review on security orchestration. The review analyzed papers from various sources to understand different aspects of security orchestration such as definitions, challenges it addresses, proposed solutions, adoption practices, and architectural considerations. Key findings include that security orchestration aims to integrate disparate security tools, automate incident response workflows, and bridge the gap between detection and response. It addresses issues like lack of interoperability, skills shortage and inefficient manual processes. Taxonomies of proposed solutions and open challenges in technology, people and processes are also discussed.

1. Multivocal Review of
Security Orchestration
Chadni Islam
CREST centre
University of Adelaide
Australia
CSIRO’s Data61, Australia
M. Ali Babar
CREST centre
University of Adelaide
Australia
Surya Nepal
CSIRO’s Data61
Australia
Chadni Islam, Muhammad Ali Babar, and Surya Nepal. 2019. A Multi-Vocal Review of Security
Orchestration. ACM Comput. Surv. 52, 2, Article 37 (May 2019), 45 pages.
DOI: https://doi.org/10.1145/3305268

2. Security Incident
CREST Centre | University of Adelaide 2
“A security incident is an unwanted or unexpected event/events that have a
significant probability of compromising the security of an organization’s assets. ”

3. Global Cost of Cyber Crime
Security Orchestration and Automation 3
41%increase in the
cost of data breach in UK in
two year.
Source: https://www.ibm.com/downloads/cas/861MNWN2

4. Root Cause of Security Incident and
Impacted Industries
4
Source: https://ridethelightning.senseient.com/2019/04/bakerhostetlers-fifth-annual-data-security-incident-
response-report-released.html
Five Root Cause 750 Incidents
CREST Centre | University of Adelaide

5. Overview of an Organization Decision
Against Security Incident
5
IDS/IPS
Security Team
IDS
IDS
Scenario of an Organization
CREST Centre | University of Adelaide
Firewall
Alert
IDS/
IPS
IDS: Intrusion Detection System
IPS: Intrusion Prevention System
SIEM: Security Information and Event Monitoring System

6. Overview of an Organization Decision
Against Security Incident
6
IDS/IPS
Security Team
SIEM
IDS
IDS
Scenario of an Organization
CREST Centre | University of Adelaide
Firewall
Alert
IDS/
IPS
IDS: Intrusion Detection System
IPS: Intrusion Prevention System
SIEM: Security Information and Event Monitoring System

7. Overview of an Organization Decision
Against Security Incident
7
IDS/IPS
Security Team
SIEM
IDS
IDS
Scenario of an Organization
CREST Centre | University of Adelaide
Firewall
Alert
IDS/
IPS
IDS: Intrusion Detection System
IPS: Intrusion Prevention System
SIEM: Security Information and Event Monitoring System
AnalyzeSystem Activities Integrate Validate Analyze
Investigate

8. Overview of an Organization Decision
Against Security Incident
8
IDS/IPS
Security Team
SIEM
IDS
IDS
Scenario of an Organization
CREST Centre | University of Adelaide
Firewall
Alert
IDS/
IPS
IDS: Intrusion Detection System
IPS: Intrusion Prevention System
SIEM: Security Information and Event Monitoring System
AnalyzeSystem Activities Integrate Validate Analyze
Investigate
Response
Update Threat
Intelligence
Block address
Implement &
Enforce Policy
Configure
Plan

9. Organizations Plan to Response to a
Security Incident
9
Example of Incident Response Plan (IRP) for Phishing Attack
# Response Task Activity
1 Is this a phishing
attack?
Determine if this is a phishing attack? In the task, select yes or no in
the outcome.
2 Scan endpoint –
malware found?
After running a scan, determine whether malware was found. In the
task, select yes or no in Outcome.
3 Remove malware –
success?
Determine whether the malware was successfully remove. In the task,
select Yes or no in outcome.
4 Wipe and reimage If you did not successfully remove the malware found, this task instruct
you to perform a wipe and reimage on the computers infected with
the malware.
5 Update email
protection software
If it was determined that this is a phishing attack, you are prompted to
update your email protection software accordingly.
6 Remove unread
phishing email in
queue – For
Perform the steps necessary to remove the phishing email still in the
queue for all of your users
Security Orchestration and Automation

10. Different Task Performed by Security Team
10
MONITOR PROTECT PREVENT DETECT
ANALYZE PLAN RESPONSE EVALUATE
Network monitoring tool
Firewall
Intrusion Prevention System
Intrusion Detection System
SIEM
Endpoint Detection & Response
… … …
Wide Variety
of Security
Solutions
A Wide Range of Multivendor Security Solutions
On Average 25 different security systems, that can be more than 100 for some organizations
CREST Centre | University of Adelaide

11. Problem with Traditional Approach
CREST Centre | University of Adelaide 11
Security Tools Security
Experts
Millions of alerts coming
everyday
Heterogeneous security
tools work independently
Manual investigation and
response
Error-prone response
Huge response time

12. Problem with Traditional Approach …
CREST Centre| University of Adelaide
Incident Response Timeline
Source: http://e.bakerlaw.com/rv/ff00498db267a11ce4182d53934889997a36f6d4/p=8213342/
28Days
---
Time to complete
forensic investigation
66Days
---
Occurrence to
Discovery
8Days
---
Discovery to
Containment
56Days
---
Discovery to
Notification
Occurrence
Containment
Notification
Forensic Investigation
Discovery
0 122 Days

13. Problem with Traditional Approach …
13
Cybersecurity skills gap worsens, security teams are understaffed
2018
Source: https://cybersecurity.isaca.org/state-of-cybersecurity
2019
CREST Centre | University of Adelaide

14. Security Orchestration
Connect and Integrate disparate security solutions
Streamlines incident response process
Bridge the gap between detection and response
Pre-requisite for security automation
Unification of people, process and technology
Instantly perform the repetitive job of a security experts
14 |
Introduction
CREST Centre | University of Adelaide

15. Security Orchestration …
15
Introduction
MARKET PRICE –
1.6 BILLION
USD BY 2021
WIDESPREAD
ADOPTION IN LAST
COUPLE OF
YEARS
SEVERAL START
UPS AND
ACQUISITION HAVE
ARRIVED
CREST Centre | University of Adelaide

16. Security Orchestration …
16 |
Problem
… … …
Lack of
Comprehensive
view
Lack of Common
Understanding
Lack of research in
Academia
CREST Centre | University of Adelaide

17. 17
• How to make the tool interoperable?
• What are the core components of security
orchestration platform?
• How the components interact with each other?
• What the organization need to build/buy a security
orchestration platform?
Security Orchestration …
Challenges…

18. A Multi-Vocal Literature Review
Chadni Islam, Muhammad Ali Babar, and Surya Nepal, “A Multi-vocal
Review of Security Orchestration”, ACM Computing Survey, 2019
18 |

19. Research Question
 What is Security Orchestration?
 What challenges security orchestration intend to solve?
 What types of solutions have been proposed?
What practices have been reported for adopting security
orchestration?
What types of tools and techniques researchers and practitioners
use, propose, design, and implement in practice?
What aspects of architecture security practitioners consider for
large-scale deployment of security orchestration?
CREST Centre | University of Adelaide

20. Multi-Vocal Literature Review - MLR
20
Systematic literature review of state-of-the-arts and state-of-the-
practices
Planning and
Designing
Conducting Reporting
01 02 03

21. Multi-Vocal Literature Review - MLR
21
Systematic literature review of state-of-the-arts and state-of-the-
practices
Planning and
Designing
Conducting Reporting
01 02 03
Legend
Main step
Activity
Sub-step Flow
Start/End
Sub step
Flow
Start
MLR planning and design
Inclusion and exclusion
criteria
Research Identification
MLR Goal RQs
Search Strategies
Selecting Data source
Design search strings

22. Multi-Vocal Literature Review - MLR
22
Systematic literature review of state-of-the-arts and state-of-the-
practices
Planning and
Designing
Conducting Reporting
01 02 03
Legend
Main step
Activity
Sub-step Flow
Start/End
Sub step
Flow
Start
Conducting MLR
Data extraction
Data extraction based on RQ
Study Selection
Data Synthesis and Data Analysis
Generalization and
categorization
Identification of key elements
MLR planning and design
Inclusion and exclusion
criteria
Research Identification
MLR Goal RQs
Search Strategies
Selecting Data source
Design search strings

23. Multi-Vocal Literature Review - MLR
23
Systematic literature review of state-of-the-arts and state-of-the-
practices
Planning and
Designing
Conducting Reporting
01 02 03
Legend
Main step
Activity
Sub-step Flow
Start/End
Sub step
Flow
Start
Conducting MLR
Data extraction
Data extraction based on RQ
Study Selection
Data Synthesis and Data Analysis
Generalization and
categorization
Identification of key elements
MLR planning and design
Inclusion and exclusion
criteria
Research Identification
MLR Goal RQs
Search Strategies
Selecting Data source
Design search strings
End
Reporting MLR
Mapping and review results

24. Multi-Vocal Literature Review – MLR …
24 |
Study Selection
Selection of Grey Literature
IEEE
ACM
Step 2: Screen on basis of title
and abstract
SCOPUS
Step 1: Running search string Step 3: Removing duplicates
Step 4: Excluding paper shorter
than 6 pages
600
271
1017
IEEE
ACM DL
SCOPUS
N: 271
N: 290 N: 225 N: 37
DBLP N: 19
N: 274
Manual Search
Google
scholar
N: 6
Step 6: Additional search on
Google Scholar
Running search
string
Applying
eligibility criteria
Google Search Engine
N: 52
Crawl through
Websites
N: 43
N: 95
Studies included for
qualitative synthesis
Selection of Academic Literature
Step 5: Articles screened on
basis on full text

25. Findings of the MLR
25

26. What is Security Orchestration?
“Security Orchestration is the planning, integration, cooperation,
and coordination of the activities of security tools and experts to
produce and automate required actions in response to any security
incident across multiple technology paradigms.”
An Ontology-Driven Approach to Automate the Process of Integration Security Software Systems | ICSSP 2019 26
Definition
Integration
Orchestration
Automation

27. Overview of an Organization Decision
Against Security Incident
IDS
Integrate Analyze
System
Activities
Security Experts
Validate
Alerts
Update Threat
Intelligence
Organization
Block address
Investigation
Plan
Update
Threat Intelligence
Block
address
Configure
With Orchestration
Manual Automate
Implement &
Enforce Policy
Orchestration
Platform
Integrate ValidateAnalyze
Configure
Without Orchestration
Investigate
Plan
Response
CREST Centre| University of Adelaide

28. Key Functionalities of Security Orchestration
• Unify security tools
• Determine endpoint for human investigation
• Share contextual insight
Act as a hub
• Translate complex process into streamline workflow
• Maintain process consistency across security program
• Provide deployment model
• Determine appropriate course of action
Orchestrate security activities
• Automate repetitive and manual task
• Automate policy enforcement across disparate solutions
Enable automated response

29. Core Components of Security Orchestration
Security
Orchestration
Platform
Unification
Unit
Description
Module Collector Pre-
processor Dashboard
Orchestration
Unit
Planning
Module
Threat
Intelligence
Detection
Module
Automation
Unit
Remediation
Module
Action
Performer

30. Key Quality Attributes of Security Orchestration
30 |
UsabilityAdaptability
Flexibility
Timeliness
AccuracyScalability

31. What Challenges Security
Orchestration Intend to Solve?
31 |CREST Centre | University of Adelaide

32. Drivers of Security Orchestration
Socio-technical Issues
Technical Issues
Challenges
Lack of tools and
technologies to
automate response
Lack of Interoperability
among isolated security
tools
Limitation of existing
tools to provide
required services
Lack of
collaboration and
coordination
Lack of skills and
expertise
Lack of
frameworks
More
responsibility on
human experts
CREST Centre | University of Adelaide

33. What types of solutions have
been proposed?
33 |

34. Taxonomy of Security Orchestration
Workflow
Scripting
Prioritization
Learning
Plugin
Auto-Integration
Automation
Strategy
End point
Cloud
Data Centre
Threat Management
Execution
Environment
Automated
Semi -Automated
Manual
Task Mode
Central
Distributed
Hybrid
Deployment Type
Security Orchestration
Platform
CREST Centre | University of Adelaide

35. Open Issues
• Little involvement and
collaboration among
different level of staffs
during the orchestration
and automation
• Lack of security architect for
risk and policy management
• No holistic training for staff
to understand security
orchestration platform,
integrated tools and
incident response workflow
TechnologyPeople Process
CREST Centre | University of Adelaide

36. Open Issues
• Little involvement and
collaboration among
different level of staffs
during the orchestration
and automation
• Lack of security architect for
risk and policy management
• No holistic training for staff
to understand security
orchestration platform,
integrated tools and
incident response workflow
• Insufficient alignment of
Incident response process
with organizations existing
IT operational framework
• No clear agreement among
vendor on what need to
orchestrate and what can
be automated
• No guideline to assess
maturity of orchestration
process and incorporate
automation into the system
TechnologyPeople Process
CREST Centre | University of Adelaide

37. Open Issues
• Little involvement and
collaboration among
different level of staffs
during the orchestration
and automation
• Lack of security architect for
risk and policy management
• No holistic training for staff
to understand security
orchestration platform,
integrated tools and
incident response workflow
• Insufficient alignment of
Incident response process
with organizations existing
IT operational framework
• No clear agreement among
vendor on what need to
orchestrate and what can
be automated
• No guideline to assess
maturity of orchestration
process and incorporate
automation into the system
• Lack of modeling notation
and language to support
integration of security
information at runtime
• Increasing diversity of
integrated security
solutions due to dynamic
change of attack patterns
• Few research on AI for
scalable and flexible
security orchestration and
integration
TechnologyPeople Process
CREST Centre | University of Adelaide

38. Future Direction
38
• Design and implement an Architecture to support
large scale realization of security orchestration.
• To provide Reference Architecture for security orchestration
that can facilitate design and development of concrete security
orchestration architectures.
• Instantiate a distributed and self-adaptable security
orchestration engine.

39. Reference
https://dl.acm.org/doi/fullHtml/10.1145/3305268
Preprint https://www.researchgate.net/publication/332818244_A_Mul
ti-Vocal_Review_of_Security_Orchestration
Published Version
@article{10.1145/3305268,
author= {Islam,Chadni and Babar,MuhammadAli and Nepal,
Surya},
title = {A Multi-VocalReview of Security Orchestration},
year = {2019},
issue_date= {May 2019},
publisher= {Associationfor ComputingMachinery},
address= {New York, NY, USA},
volume = {52}, number = {2},
issn = {0360-0300},
url = {https://doi.org/10.1145/3305268},
doi = {10.1145/3305268},
journal= {ACM Comput.Surv.},
month= apr,
articleno = {37},
numpages= {45},
keywords = {intelligentsecurityassistant,security automation,
multivocalliteraturereview, Security orchestration}}
BibTex
Chadni Islam, Muhammad Ali Babar, and Surya Nepal. 2019. A
Multi-Vocal Review of Security Orchestration. ACM Comput.
Surv. 52, 2, Article 37 (May 2019), 45 pages.
DOI:https://doi.org/10.1145/3305268

40. Question???
Chadni Islam
CREST Centre (https://crest-centre.net/)
School of Computer Science, University of Adelaide
Adelaide, Australia and
CSIRO’s Data61, Australia
Email: chadni19@gmail.com, chadni.islam@adelaide.edu.au
@_Chadni_ https://twitter.com/_Chadni_