Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Secure360 on Risk

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Nächste SlideShare
Evolving brains
Evolving brains
Wird geladen in …3
×

Hier ansehen

1 von 87 Anzeige
Anzeige

Weitere Verwandte Inhalte

Ähnlich wie Secure360 on Risk (20)

Anzeige

Aktuellste (20)

Secure360 on Risk

  1. 1. Challenging Conventional Wisdom: A New Approach to Risk Management Alex Hutton Jay Jacobs
  2. 2. What’s this We think you’re getting bad information! about? We think our industry can do better! We think this will make us “more secure!”
  3. 3. Security is now so essential a concern that we can no longer use adjectives and adverbs but must instead use numbers. – Dan Geer
  4. 4. How are you making decisions now?
  5. 5. What’s the quality of those decisions?
  6. 6. Effective Decisions need quality data, models, execution
  7. 7. Our vendors and standards aren’t helping us (-:
  8. 8. hey, why are you getting lousy information from standards and vendors?
  9. 9. The science hey, why are of information you getting security & risk management lousy is hard information 1. Pseudo Science & Proto Science from 2. Models & Data standards 3. Complexity and vendors?
  10. 10. The science hey, why are of information you getting security & risk management lousy is hard information 1. Pseudo Science & Proto Science from 2. Models & Data standards 3. Complexity and vendors?
  11. 11. State of the Industry (a) (Thomas Kuhn is way smarter than we are) proto-science somewhat random fact gathering (mainly of readily accessible data) a“morass”of interesting, trivial, irrelevant observations a variety of theories (that are spawned from what he calls philosophical speculation) that provide little guidance to data gathering
  12. 12. State of the Industry (b) At our present skill in measurement of security, we generally have an ordinal scale at best, not an interval scale and certainly not a ratio scale. In plain terms, this means we can say whether X is better than Y but how much better and compared to what is not so easy. – More from Dan Geer
  13. 13. If Science is based on inductive observations to derive meaning and understanding and measurement on quality (ratio) scales, how about InfoSec? Where do we sit in the family of sciences?
  14. 14. We’re the Crazy Uncle with tinfoil hat antennae used to talk to the space aliens of Regulus V, has 47 cats, and who too frequently (but benignly) forgets to wear pants.
  15. 15. Take, for example, CVSS
  16. 16. “the Base Equation multiplies Impact by 0.6 and Exploitability by 0.4”
  17. 17. Jet Engine X Peanut Butter = Shiny
  18. 18. decimals aren’t magic. adding one willy-nilly doesn’t suddenly transform ordinal rankings into ratio values.
  19. 19. The science hey, why are of information you getting security & risk management lousy is hard information 1. Pseudo Science & Proto Science from 2. Models & Data standards 3. Complexity and vendors?
  20. 20. Data must exist in order to feed our models... ... but creating the right models are dependent on understanding what data is useful! 20
  21. 21. Data, Models, Execution: Garbage in-Garbage Out
  22. 22. Data, Models, Execution: Treat Data Poorly
  23. 23. Data, Models, Execution: Adapting to Situations
  24. 24. The science hey, why are of information you getting security & risk management lousy is hard information 1. Pseudo Science & Proto Science from 2. Models & Data standards 3. Complexity and vendors?
  25. 25. These “risk” statements you’re making... I don’t think you’re doing it right. - (Chillin’ Friederich Hayek)
  26. 26. A Comforting Thought... “Given Newton's laws and the current position and velocity of every particle in the universe, it was possible, in principle, to predict everything for all time.” -- Simon-Pierre LaPlace, 1814
  27. 27. 8 4 4 2 2 2 2 Reductionism
  28. 28. 8 ? 4 4 ? 2 2 2 2 Functionalism
  29. 29. Asset Reductionism Functionalism Comp. Comp. Sub. Sub. Attribute Attribute Attribute Attribute
  30. 30. Awww man... ...even if it were the case that the natural laws had no longer any secret for us, we could still only know the initial situation approximately. ... small differences in the initial conditions produce very great ones in the final phenomenon. A small error in the former will produce an enormous error in the latter. Prediction becomes impossible... -- Henri Poincare, 1887
  31. 31. ty non lexi -l i p nea C om r 13 5 6 2 2 2 2 Systems Approach Holism
  32. 32. Complex systems contain changing mixtures of failures latent within them. The complexity of these systems makes it impossible for them to run without multiple flaws being present. ... individually insufficient to cause failure ...failures change constantly because of changing technology, work organization, and efforts to eradicate failures. Complex systems run in degraded mode. “How Complex Systems Fail” - Richard Cook
  33. 33. Security is a characteristic of systems and not of their components Security is an emergent property of systems; it does not reside in a person, device or department of an organization or system. ... it is not a feature that is separate from the other components of the system. ...the state of Security in any system is always dynamic “How Complex Systems Fail” - Richard Cook
  34. 34. We may want to rethink our approach.
  35. 35. Overcoming the problem • Medicine uses an “Evidence- Based” approach to solving problems in the complex system that is the body. • Dr. Peter Tippett (MD, PhD) applies Evidence-Based principles to Information Security. 36
  36. 36. What to study: Sources of Knowledge Suggested  context: Capability  to  manage (skills,  resources,   asset decision  quality…) landscape impact landscape risk threat landscape controls landscape
  37. 37. How: Data Quality in Evidence-Based Practice Evidence  level  D Evidence  level  C Evidence  level  B Evidence  level  A Evidence  level  A Case-­‐series   Consistent   Consistent   “Expert  opinion   study  or   Retrospec8ve   Randomized   without  explicit   extrapola8ons   Cohort,  Exploratory   Controlled  Clinical   cri8cal  appraisal,   from  level  B   Cohort,  Ecological   Trial,  cohort  study,   or  based  on   studies. Study,  Outcomes   all  or  none,  clinical   physiology,  bench   Research,  case-­‐ decision  rule   research  or  first   control  study;  or   validated  in   principles.” extrapola8ons  from   different   level  A  studies. popula8ons. beNer
  38. 38. Evidence-Based Risk Management State of Nature State of Knowledge State of Wisdom Evidence level D Lists Feeling like we’ve done something Evidence level C Simple derived values Outcomes with ad-hoc with ad-hoc modeling deductive selections Evidence level B Formal Modeling Decision making constructs Evidence level A
  39. 39. Evidence-Based Risk Management State of Nature State of Knowledge State of Wisdom Evidence level D Lists Feeling like we’ve done something Evidence level C Simple derived values Outcomes with ad-hoc with ad-hoc modeling deductive selections Evidence level B Formal Modeling Decision making constructs Evidence level A
  40. 40. Evidence-Based Risk Management State of Nature State of Knowledge State of Wisdom Evidence level D Lists Feeling like we’ve done something Evidence level C Simple derived values Outcomes with ad-hoc with ad-hoc modeling deductive selections You  are  here Evidence level B Formal Modeling Decision making constructs Evidence level A
  41. 41. So  How  Do  We  Change? Data Models… Standards START  WITH   THE   OUTCOMES!
  42. 42. Two True Security Outcomes: Success and Failure
  43. 43. Knowing Success in InfoSec is hard - Known Success (anti-Threat ops) - Unknown success (controls work without us knowing) - Dumb luck (We’re not targeted, but our neighbor is)
  44. 44. Getting the outcomes: Success
  45. 45. Getting the outcomes: Success stronger processes result in fewer availability incidents
  46. 46. Getting the outcomes - Successes: - Existences of processes - Operational (performance) metrics - Maturity ratings WHAT WE WANT ARE PATTERNS!
  47. 47. Knowing Failure is (somewhat) easier
  48. 48. Getting The Outcomes: Failures VERIS | Verizon Enterprise Risk and Information Sharing VERIS takes the incident narrative and creates metrics (risk determinants)
  49. 49. VERIS | Verizon Enterprise Risk and Information Sharing A  free  (as  in  beer*)   framework  created  for   metrics,  modeling,  and   compara8ve  analy8cs. A  security  incident  (or  threat  scenario)  is  modeled  as  a   series  of  events.  Every  event   is  comprised  of  the  following  4  A’s: Agent:  Whose  acLons  affected  the  asset AcLon:  What  acLons  affected  the  asset Asset:  Which  assets  were  affected   AOribute:  How  the  asset  was  affected
  50. 50. VERIS takes this : INCIDENT REPORT “An attacker from a Russian IP address initiated multiple SQL injection attacks against a public-facing web application. They were able to introduce keyloggers and network sniffers onto internal systems. The keyloggers captured several domain credentials which the attackers used to further infiltrate the corporate network. The packet sniffers captured data for several months which the attacker periodically returned to collect…” and…
  51. 51. …and translates it to this… Event 1 Agent: External (Org crime) Action: Hacking (SQLi) Asset: Server (Web server, Database) Attribute: Integrity Event 2 Agent: External (Org crime) Action: Malware (Keylogger) 1 > 2 > 3 > 4 > Asset: Server (Web server) Attribute: Confidentiality Event 3 Agent: External (Org crime) Action: Hacking (Use of stolen creds) Asset: Server, Network (multiple) Attribute: Confidentiality, Integrity Event 4…
  52. 52. patterns!
  53. 53. Framework = ∑ ∩ ∫√ Models Data
  54. 54. Framework Framework Data Process = Process ∑ ∩ ∫√ Models = ∑ ∩ ∫√ Data Models Process Process Data
  55. 55. Using your metrics program - Identify & Measure your processes - Identify & Measure your failures - Get into loss factors (ABC) - Share data - Support data sharing efforts
  56. 56. Bring it Home: your metrics program
  57. 57. Bring it Home: your metrics program or
  58. 58. Bring it Home: your metrics program or The Amazing Technicolor Scorecard
  59. 59. Priority #1: no more surrogate data
  60. 60. Priority #1: (meaning) no more risk analysts*
  61. 61. Priority #1: (really) create data analysts
  62. 62. Data analysts need to focus on quality data, models, execution
  63. 63. Evidence-Based Risk Management State of Nature State of Knowledge State of Wisdom Evidence level D Lists Feeling like we’ve done something Evidence level C Simple derived values Outcomes with ad-hoc with ad-hoc modeling deductive selections Evidence level B Formal Modeling Decision making constructs Evidence level A
  64. 64. asset landscape A balanced scorecard of sorts threat impact landscape landscape risk controls landscape
  65. 65. Where to look? The Two True Security Outcomes: Success and Failure
  66. 66. Failures: threat landscape incidents, red/blue team asset vulnerabilities, misconfigurations, landscape unknowns... gaps in coverage, known lack of controls landscape effectiveness, known underskilled/ utilized... impact Cost-Based Accounting around landscape incidents, cost of operations, etc...
  67. 67. Successes: threat landscape intel, red/blue teams, SIEM asset vulnerabilities, misconfigurations, landscape unknowns, skills, training controls positive threat outcomes (tOps), skills, landscape training impact landscape ROI? ROSI? (ducks to avoid tomatoes)
  68. 68. What to look? Two types of data to find: Focus initially on Visibility, then look to find Variability.
  69. 69. How to look? The GQM Approach: For each “where” for each “what” use the following “how”
  70. 70. How to look? The GQM Approach: For each “where” for each “what”, start by using GQM as “how.”
  71. 71. Goal, Question, Metric Conceptual level (goal) goals defined for an object for a variety of reasons, with respect to various models, from various points of view. Operational level (question) questions are used to define models of the object of study and then focuses on that object to characterize the assessment or achievement of a specific goal. Quantitative level (metric) Victor Basili metrics, based on the models, is associated with every question in order to answer it in a measurable way.
  72. 72. The Book You Should Buy (Jay & Alex aren’t getting a kickback, in case you’re wondering)
  73. 73. GQM for Fun & Profit Goals establish what we want to Goal 1 Goal 2 accomplish. Questions help us understand how to meet the goal. They Q1 Q2 Q3 Q4 Q5 address context. Metrics identify the measurements that are needed to answer M1 M2 M3 M4 M5 M6 M7 the questions.
  74. 74. GQM for Fun & Profit Execution Goal 1 Goal 2 Models Q1 Q2 Q3 Q4 Q5 Data M1 M2 M3 M4 M5 M6 M7
  75. 75. data about defined success and failures models of assets, controls, threats contributing to impact execution by data analysts ...Feeding standards, audits and governance
  76. 76. Using your metrics program - Identify & Measure your processes - Identify & Measure your failures - Get into loss factors (ABC) - Share data - Support data sharing efforts
  77. 77. Using your metrics program - Identify & Measure your processes - Identify & Measure your failures - Get into loss factors (ABC) - Share data - Support data sharing efforts
  78. 78. Security is now so essential a concern that we can no longer use adjectives and adverbs but must instead use numbers. – Dan Geer
  79. 79. Questions? Jay Jacobs Alex Hutton @jayjacobs @alexhutton jay@beechplane.com alex@alexhutton.com
  80. 80. Approaching the system as a system asset landscape impact Prioritize landscape risk threat landscape controls landscape De-prioritize
  81. 81. Suggested context: Capability to manage (skills, resources, decision quality…) asset landscape impact landscape risk threat landscape controls landscape
  82. 82. Data Sharing: - Sources: - Qualify this Intel according to framework - Treat with appropriate data quality listings (let models shape the certainty)
  83. 83. Get Into Accounting - Use existing models that take advantage of accounting concepts (ABC) to Talk to the LOBs
  84. 84. Using your metrics program - Identify & Measure your processes - Identify & Measure your failures - Share data - Support data sharing efforts - Get into loss factors (ABC)
  85. 85. Challenging Conventional Wisdom Conventional Wisdom may not be wrong - Question current practices - Seek Evidence and Feedback

×