6. 66
Strategic Risk Assessment:
What is Important to Achieving Organizational Objectives and
Not Under [Complete] Control?
• Identifying threats and exposures without measurement only
generates lists -- that may or may not be applicable or important to
the organization.
• Some ERM projects create spreadsheets full of “Critical Risks” that
frustrate management and fail to provide a blueprint for action.
• Instead of identification run rampant, Strategic Risk Assessment
starts with corporate objectives and considers what is at risk,
identifies potential threats, and assesses the impact and the
effectiveness of current controls to counter those threats – and
points to controls where objectives are threatened.
7. 77
Strategic Risk Assessment Issues
To be effective, risk assessment cannot be merely checklists
or a process that is disconnected from business strategy.
• Risk Assessment must be integrated in a way that provides
timely and relevant risk information to management.
• For risk management to be a strategic process, risk assessment
must be owned by the business units and be embedded within
the business cycle, starting with strategic planning.
• And: Risk assessment begins and ends with the organization’s
specific objectives.
9. 99
Risk Assessment Basics
• It is a matter of widespread understanding that risks should be
assessed in terms of the likelihood (probability) that an
uncontrolled event will occur and the consequences (impact)
to achieving one or more organizational objectives.
– Applicable to both qualitative and quantitative methods of
assessment.
• Strategic Risk Assessment requires pursuing a systematic,
logical set of actions to identify the magnitude of hazards and
exposures, assess threats, and implement controls to mitigate,
eliminate or control high-risk conditions.
12. Data gathering &
representation
Select appropriate
techniques(s)
Risk analysis & modeling
Expert judgment
RISK
But Quantitative
Methods are
Often Required
to Identify
Corrective
Actions
13. 1313
Risk Assessment Tools & Techniques Are
Rapidly Evolving
Risk Assessment needs to move beyond Probability x Severity and
Risk Maps to evaluate emerging issues, warning & detectability,
and other key threats to strategic objectives.
• Over the past decade, developments in economic and
financial theory -- plus computing and data advancements –
are providing new methods for quantitative risk assessment, as
well as improvements to existing techniques.
• Risk Managers should understand available risk assessment
techniques and adopt a set of tools they can apply to their
organization's unique Risk Management requirements.
14. 1414
Three Basic Types of Quantitative Assessment
Tools – In Order of Complexity
1.Comparative methods;
2.Temporal methods; and,
3.Functional methods.
15. 1515
Comparative Assessment Methods
A Comparative Analysis takes an explicit standard – eg., “Best Practices”
– and compares a system, process and/or set of procedures to that
standard, resulting in a “Gap Analysis”.
• A “good standard” is prepared and maintained as “the distillation of
continually developing expert opinion and experience in the face of
a continually changing environment”.
• One of the strengths of this approach is its simplicity. Comparative
methods can be ideal for organizations as they begin to focus
attention on specific systems, processes or threats.
• A weakness is that there is no explicit list of threats as there is in other
approaches.
16. 1616
Sample
“Best
Practices”
Matrix –
Claims
Handling
O Managerial Oversight
P Primary
S Secondary
C Consultative Input
D Data Resource
DirectorofInsurance
DirectorofLegalSupport&
Claims
ExecutiveVicePresident,Aon
SeniorVicePresident,Claims
VicePresident,Claims
AssistantVP,Claims
SeniorConsultant,Claims
SeniorClientSpecialist,
Claims(Megan)
SeniorClientSpecialist,
Claims(Martha)
ClaimAssistant
CLAIM MANAGEMENT PROCEDURES
1)
Establish formal claims service standards for TPA's, carriers and
other vendors
C P C P C C
2)
Develop annual written service plan for TPA's and other
vendors and monitor performance
C P O P C C
3) Develop written Claims Procedures or Manual C O C P C C
4)
Establish internal claims reporting and management
procedures and monitor compliance
C O C O P S
5)
Develop claim reports, distribute and review with business
units as necessary
C O C O C P S S S D
6) Maintain listing of all insured claims O O O C P S S S D
7) Maintain listing of all self-insured claims O O O C P S S S D
8) Establish and monitor WC post-injury management program C O O O D P S
9) Manage claims litigation process C O O P D S D
10) Administer OCIP claims C O O O P S
11) Administer non-litigated GL claims O O O P S S D D
12) Administer auto claims O O O P S S D D
13) Administer D&O, fidelity, fiduciary, EPL C P C
14) Administer Litigated GL claims O O P D S D
15) Administer Property claims O O O P D S D
16) Pursue subrogation activities O O O P S S S D
17) Review losses and identify trends C C C O C P S S S D
18) Conduct/coordinate periodic claims audits D D
19) Monitor large loss activity C O O P D D
20)
Review and adjust safety/loss control initiatives as needed to
proactively treat risk and address trends observed in claims
management activities
O C C C D C D
Management assures activity is addressed
Principally responsible for driving the activity
Responsible to perform or drive certain aspects of the activity, but is not the leader
Can provide guidance or feedback at a high level for activity
Provides data or information that is used in the activity
18. 1818
Temporal Analysis Methods
A Temporal Assessment applies quantitative tests to a system, process or
set of procedures. These “tests” involve analyzing the results of specific
threats or attacks against actual protections and controls, subject to
some constraints.
• Since it is often impractical to test a system directly, a model of the
system is generally used instead.
– However, a model introduces the question of fidelity: an
inaccurate model may not only confuse matters; it may provide a
false sense of security that is even worse than confusion.
• A key weakness of a temporal method is that it is not possible to
model all possible threats; it is not even possible to list them all.
19. 1919
Temporal Method: Scenario Analysis
Scenario analysis considers the questions ‘what might happen
and what should/would we do?’ It can not only highlight risks and
opportunities in the short and long term; but also test the
effectiveness and efficiency of specific controls and plans.
• The central idea is to consider a variety of possible futures that
include many of the important uncertainties in the system,
rather than to focus on the accurate prediction of any
particular outcome.
• A strength of scenario analysis is that it can consider “existential
threats” that involve large swaths of the organization.
20. 2020
Four Critical Components of Scenario Analysis
1. Determining which factors the scenarios will be built around. In
general, analysts should focus on the two or three most critical
factors.
2. Determining the number of scenarios to analyze for each factor.
Depends upon how different the scenarios are, and how well the
results of each scenario can be forecast.
3. Estimating results – e.g., asset cash flows, control failures, unexpected
breakdowns, etc. -- under each scenario.
4. Assigning probabilities to each scenario. Note that this makes sense
only if the scenarios cover the full spectrum of possibilities; otherwise,
the probabilities will not add up to 100%
21. 2121
Sample Scenario
A Scenario Analysis can be used to ensure effective and reliable
insurance coverage.
• It typically involves sitting down with brokers, underwriters, lawyers,
adjusters and managers to analyze and talk through how each
insurance policy would respond to different circumstances.
• The results are compiled in systematic tables and charts that point
out problem areas and suggest solutions.
• One of the strengths of Scenario Analysis is that it tests the system
itself (or a model), clearing away misconceptions and uncovering
specific elements or issues needing attention.
22. 2222
Other Temporal Analysis Methods
The most important Temporal Assessment methods use
Predictive Analytics to not only determine What might
happen, but How Much it could impact objectives.
• Two useful tools are:
– Decision Tree Analysis; and,
– Modeling & Simulation.
23. 2323
Decision Tree Analysis
A Decision Tree is a structure in which each internal node represents a
"test" on an attribute; each “branch” represents the outcome of the test;
and each “leaf” represents a decision taken after computing all attributes.
• The paths from root to leaf represent classification rules:
– A Root node represents the start of the decision tree, where a decision
maker is faced with an uncertain outcome. The objective is to evaluate
the overall net positive or negative outcomes at this node.
– Event nodes represent outcomes based upon the probable occurrence
of various events.
– Decision branches represent choices that are made by the decision
maker.
– End nodes represent final outcomes where a payoff value is identified.
24. 2424
Sample Decision Tree: Jenny Lind
• Jenny Lind is a writer of romance novels. A movie company
and a TV network both want exclusive rights to one of her
more popular works.
• If she signs with the network, she will receive a single lump sum,
but if she signs with the movie company, the amount she will
receive depends on the market response to her movie.
• What should she do?
25. Jenny Lind Decision Tree
Small Box Office
Medium Box Office
Large Box Office
Small Box Office
Medium Box Office
Large Box Office
Sign with Movie Co.
Sign with TV Network
$200,000
$1,000,000
$3,000,000
$900,000
$900,000
$900,000
.3
.6
.1
.3
.6
.1
Estimated
Outcomes
Estimated
Likelihood
Root
Node
Event Nodes
26. Jenny Lind Decision Tree - Solved
Small Box Office
Medium Box Office
Large Box Office
Small Box Office
Medium Box Office
Large Box Office
Sign with Movie Co.
Sign with TV Network
$200,000
$1,000,000
$3,000,000
$900,000
$900,000
$900,000
.3
.6
.1
.3
.6
.1
Expected
$900,000
Expected
$960,000
Best Result
$960,000
27. 2727
Modeling & Simulation
Where Scenario Analysis and Decision Tree Analysis are techniques to
assess discrete risk events, simulation methods measure continuous risk
exposures and outcomes.
• Simulations yield a distribution of outcomes rather than a single point
estimate.
• One simulation tool is an “Exceedance Probability Curve” that
measures whether an outcome will exceed a specific estimate,
based upon predetermined probabilities.
• Simulation has few limitations in terms of events, probabilities and
outcomes – very robust models may be constructed, evaluated and
displayed graphically.
28. 2828
Simulation Example: Quantifying the Risk of
Natural Catastrophes
How do companies prepare for the financial impact of natural
catastrophes? How can they possibly have an idea of what the
potential cost can be for events that haven't yet happened?
Catastrophe Modeling provides the answers. A catastrophe model
can be roughly divided into three modules:
• The Hazard Module looks at the physical characteristics of potential
disasters and their frequency.
• The Vulnerability Module assesses the vulnerability (or
“damageability”) of buildings and their contents.
• The Damage Module determines the overall loss distribution for a
specific event by multiplying building values by potential damage.
30. 3030
Functional Assessment Methods
A Functional Analysis focuses on specific threats and protections.
• A threat model -- a list of system vulnerabilities, and the likelihood of
successful threats against those vulnerabilities -- is weighed against
organizational objectives, assets, protections, and the likelihood of
available protections successfully defending those assets against
specified threats.
• Temporal Assessment methods, such as statistical modeling; and
Comparative Assessment techniques, such as expert systems, are often
employed jointly.
• The key strength of a Functional Assessment is its ability to consider a
wide range of threats, vulnerabilities, assets and countermeasures.
31. 3131
Failure Mode & Effects Analysis (FMEA)
FMEA identifies where & how failures can occur within processes
and measures the impact of those failures.
• The FMEA Process has 4 basic steps:
1. Determine the failure modes of specific process elements;
2. Analyze the effects on other elements and the overall system;
3. Rank criticality; and,
4. Identify existing and potential controls.
• FMEA is particularly useful for evaluating critical risks in very
complex systems.
33. 3333
Sample FMEA Template
Action Results
Item / Function
Potential Failure
Mode(s)
Potential
Effect(s)
of Failure
S
e
v
Potential
Cause(s)/
Mechanism(s)
of Failure
P
r
o
b
Current Design
Controls
D
e
t
R
P
N
Recommended
Action(s)
Responsibility &
Target
Completion Date
Actions Taken
NewSev
NewOcc
NewDet
NewRPN
Coolant
containment.
Hose
connection.
Coolant fill. M
Crack/break.
Burst. Side wall
flex. Bad seal.
Poor hose rete
Leak 8 Over pressure 8 Burst, validation
pressure cycle.
1 64 Test included in
prototype and
production
validation testing.
J.P. Aguire 11/1/95
E. Eglin 8/1/96
Response Plans and Tracking
Risk Priority Number - The combined weighting
of Severity, Likelihood, and Detectability.
RPN = Sev X Occ X Det
Likelihood - Write down
the potential cause(s), and
on a scale of 1-10, rate the
Likelihood of each failure
(10= most likely). See
Severity - On a scale of 1-
10, rate the Severity of
each failure (10= most
severe). See Severity
Detectability - Examine the current
design, then, on a scale of 1-10,
rate the Detectability of each failure
(10 = least detectable). See
Detectability sheet.
Write down each failure
mode and potential
consequence(s) of that
35. 3535
FMEA Technique: Fault Tree Analysis
• A Fault Tree is a logical diagram that starts with an actual or
potential failure and works backward to identify all of the
possible causes or origins of that failure.
• Made up of branches connected by AND nodes and OR
nodes.
– ALL of the branches below an AND node must occur for the
event above the node to occur.
– Only ONE of the branches below an OR node needs to occur for
the event above the node to occur
37. 3737
FMEA Technique: Event Tree Analysis
• An Event Tree is a logical diagram that starts with an actual or
potential event and works forward to identify all of the
possible corrective actions -- and failures that could result.
• Essentially the reverse of a Fault Tree; in an analysis, one Event
Tree may lead to multiple Fault Trees and vice-versa.
• Although initially developed by engineers to determine
vulnerabilities in nuclear power generators; it is applicable,
and has been applied, to assess many complex processes.
39. 3939
Summary – Strategic Risk Assessment
Various strategic risk assessment methods view the landscape
from different heights, so to speak -- altitude is a tradeoff
between scope and detail.
• The more abstract the method, the greater the scope but the
coarser the detail; the more concrete the method, the smaller
the scope and the finer the detail.
• Different objectives, systems, threats, perils, hazards, controls,
etc. dictate the use of different assessment tools and methods.
• Identifying the appropriate technique should be the first – and
most important – step in risk assessment.
40. 4040
And, Don’t Forget – the Real Objective is to
Manage Risk
• The techniques examined in this discussion should
only be used when you need to identify exposures,
risks, perils and/or hazards that can be eliminated,
mitigated or otherwise managed.
• NO measurement is necessary when you KNOW
what to DO – and everyone AGREES!
44. 4444
Categorizing Risk Assessment Techniques
• Three basic types of assessment tools are:
1. Temporal methods;
2. Comparative methods; and,
3. Functional methods.
• Assessment techniques and tools can be classified on three axes:
1. by their level of formality on a continuum from abstract to
concrete;
2. the type of analysis performed; and
3. the threats they are attempting to find and address.
45. 4545
Types of Temporal Assessment Methods
• An Engagement consists of experts looking for any way, within given
bounds, to compromise assets.
• An Exercise links experts and owners together in order to test the
protection on assets particular to a particular system.
• Compliance Testing includes methods that the owner can execute
them himself without the aid of an expert.
46. 4646
Types of Comparative Assessment Methods
• A Principles Method type, like all of the Comparative types, is a list.
This type asks the user to apply the principles to their system.
• A Best Practices list consists of directives: Do this, Don’t do that. This
method type asks the user to compare what they do—their current
practice—with the best practice list: the list of differences represents
the “Gaps” between actual practices and ideal.
• An Audit is based on an explicit standard, such as a Best Practice list
or a Principles list. This type asks the user to evaluate the effectiveness
of the controls in place in fulfilling each item in the standard.
47. 4747
Types of Functional Assessment Methods
• Sequence Methods are the epitome of abstract methods. A simple
sequence method asks the questions:
1. What can happen? (i.e., What can go wrong?)
2. How likely is [it] that that will happen?
3. If it does happen, what are the consequences?
• An Assistant Method type keeps track of details; best instances of this type
“walk” the user through the process, prompting for the input needed to
populate and rank lists of threats, vulnerabilities and remedial actions.
• A Matrix Method asks the user to select ranges for n dimensions – assets,
threats, vulnerabilities and protections. The information in the cells of the
corresponding n-dimensional subspace is the result of analysis.
• An Expert System is one implementation that is representative of the
functional approach.
