1. How SOX-404 Exposed The
Dysfunctional Marriage Between
Business And IT...
And How Lawyers Can Help
Gene Kim
@realgenekim
genek@realgenekim.me

2. Where Did The High Performers Come
From?

3. Since 1999, We’ve Benchmarked
1500+
IT Organizations
Source: EMA (2009)
Source: IT Process Institute (2008)

4. Visible Ops: Playbook of High
Performers
• The IT Process Institute has
been studying high-performing
organizations since 1999
– What is common to all the high
performers?
– What is different between them
and average and low
performers?
– How did they become great?
• Answers have been codified in
the Visible Ops Methodology
www.ITPI.org

5. Story of GAIT and SOX-404
• Tell you a story involving IT
organizations, businesses, their auditors, the
auditors’ regulators
– A large and complex problem
– How defining two words solved it and made a
difference
• My top lessons learned
• What I’m doing about it now

6. Problem Statement
• 2001: Enron fails ($63B
market cap), Arthur
Andersen dissolution
• 2002: WorldCom (peak
$117B market cap)
• Leads to Sarbanes-Oxley
Act of 2002

8. “OMG. 952 IT Deficiencies?!?”

9. Holy cow!!! Enron wasn’t
caused by a DBA. So, why are
What were/are people worried about? the auditors digging here?? --gk
IT controls dominate the deficiencies, significant
deficiencies, and material weaknesses identified through the S-
O 404 assessment.
The estimated percentage of deficiencies identified show IT controls
accounting for the most (34 percent), followed distantly by revenue
(13 percent), procure to pay (10 percent), and fixed assets (10
percent).
The estimated percentage of significant deficiencies identified again
shows IT controls leading the way (23 percent), followed by financial
reporting and close (14 percent), procure to pay (13 percent), and
revenue (12 percent).
The estimated percentages of material weaknesses identified include
IT controls (27 percent), revenue (18 percent), taxes (11 percent), and
financial reporting and close (10 percent).
It is important to note that the results presented here are based on self-reporting by the companies that
participated in the survey. Conclusions may be affected by the differing methods companies use to report on
various elements of Sarbanes-Oxley compliance.
© 2004 KPMG LLP, the U.S. member firm of KPMG International, a Swiss cooperative. All rights reserved. Printed in the U.S.A.
9

10. Again, holy cow!!! If the risk isn’t in
PROBLEMS & CHALLENGES IT, then auditors are not only
generating efforts, but finding
deficiencies that don’t matters…
IT V NON - IT COMPARISON
--gk
100%
 Disproportionate Share:
 Compliance effort.
% 50%
IT  Deficiencies.
NON - IT
 Non Finance Apps.
0%
EFFORT DEFICIENCIES
 Financial Statement Impact:
 Indirect linkage
Applications in Scope  Least likely impact
100%
 Business & IT integration.
% 50%
0%
Fin Apps Non Fin Apps
10 February 2006 Corporate Finance

11. Vision: Create Equivalence to
Nine Firm Document on IT
Control Exceptions
GAIT takes the
approach used in the
nine firm document.
GAIT represents the
upfront scoping
exercise to
appropriately
identify the IT
controls work
relevant to overall
internal controls
objectives
www.theiia.org
Chart 3: Evaluating Information Technology General Control (ITGC) Deficiencies
, “A Framework for Evaluating Control Exceptions and Deficiencies” (December 20, 2004)

12. There Had To Be A Better Way

13. SOX-404 Value Network: Primary Constituencies

14. The Problem
• The IT portions of SOX-404 compliance has
frustrated auditors and management
– Significant key controls reside inside IT and IT
processes as well as in the business processes
– No well-established guidance for scoping IT work
results in inconsistency and the process being overly
subjective
– Sometimes result in overly broad scope and
excessive testing costs
– Significant risks to financial assertions may be left
unaddressed
– Suboptimal use of scarce resources
www.theiia.org

15. Why Is There A Problem?
• No clear guidance exists to define how IT
processes and activities can invalidate
financial application processing or financial
assertions
– COSO provides an accepted construct for defining
overall internal control objectives, assertions, risks
and controls, but its application to the IT environmet
is ambiguous
– COBIT doesn’t provide a clear mechanism to scope
IT processes and controls to the achievement of
specific internal control objectives (e.g., COSO
objective for internal control over financial reporting)
• Something else is needed…
www.theiia.org

16. COSO ERM Cube v2

17. COBIT

18. Why Is There A Problem?
• No clear guidance exists to define how IT processes and
activities can invalidate financial application processing or
financial assertions
– COSO provides an accepted construct for defining overall internal
control objectives, assertions, risks and controls, but its application to
the IT environment is ambiguous
– COBIT doesn’t provide a clear mechanism to scope IT processes and
controls to the achievement of specific internal control objectives
(e.g., COSO objective for internal control over financial reporting)
• Something else is needed…

19. Thought Experiment
• Auditors vs. Management
• We can agree that there are two extremes in
spectrum of financial reporting risk
– eBay auction settlement business process
– Grain elevators
• Extremes are easy… Middle is hard…

20. Language Is Often An
Obstacle
• In Newton’s time, there were not
concrete terms for several critical
concepts:
– Force, acceleration, mass, inertia
• In the following slide, note how
difficult it was for Newton to frame
the “three laws of motion” without
these concepts…
www.theiia.org

21. Early Drafts Of Three Laws
Of Motion
• 1. If a quantity once move it will never rest unless
hindered by some externall cause.
• 2. A quantity will always move on in the same straight
line (not changing the determination nor celerity of its
motion) unless some externall cause divert it.
• 3. There is exactly so much required and no more force
to reduce a body to rest as there was to put it upon
motion.
• Axiom 100: A body once moved will always keep the
same celerity, quantity and determination of its motion
• Axiom 103: ...as the body (a) is to the body (b0), so
must the power of efficacy vigor strength or virtue of
the cause which begets the same quantity of velocity
www.theiia.org
Source: Isaac Newton, James Gleick.

22. Benchmarks
• Pythagorean theorem: 24 words
• Archimedes' Principle: 67 words
• Newton’s Three Laws Of Motion: 91 words
• The 10 Commandments: 179 words
• GAIT Proposed Principles v3.0: 168 words
• The Gettysburg Address: 286 words
• The Declaration of Independence: 1,300 words
• GAIT Principles v1.3: 6,856 words
• GAIT Methodology v2.2: 11,348 words
• The US Government regulations on the sale of cabbage:
26,911 words
www.theiia.org

23. Solution: GAIT…
• Released in Feb 2007, Establishes four principles that
– Defines the relevance of IT infrastructure elements to financial reporting
integrity
– Define the three types of IT processes that can affect them: change
management and systems development, operations and security
– Defines an end-to-end process view of these three processes
– Defines an approach to defining objectives and key controls within those
three processes
• Provides a methodology and thinking process that
continues the top down, risk based approach started in
AS2 to scope IT general controls
• Provides a common context for management and
auditors to support and test management’s assessment
that the necessary IT controls exist and are effective
– Initial target is internal control objectives for financial reporting, but
should extend to operating effectiveness and complying with laws and
regulations (as defined by COSO)

24. GAIT Principle #1
• The only IT infrastructure elements
(e.g., databases, operating
systems, networks) relevant to ITGC
assessment are those that support
financially-significant applications and
data.
(“What are the relevant IT infrastructure
elements?”)

25. GAIT Principle #2
• The IT processes primarily relevant to ITGC
assessment are those that directly impact the
integrity of financially-significant applications
and data:
– Change management and systems development: the processes
around developing, implementing, and maintaining financially
significant applications and supporting IT infrastructure
– Operations management: the processes around managing the
integrity of production data and program execution
– Security management: the processes around limiting access to
information assets
(“What are the relevant end-to-end IT
processes?”)

26. GAIT Principle #3
• Implications to the reliability of financially-
significant applications and data, including
controls, are based upon the achievement or
failure of IT process objectives, not the design
and operating effectiveness of the individual
controls within those processes.
(“What are the relevant objectives of those IT
processes? In other words, we shouldn’t get
carried away when reaching a conclusion when
testing a control.”)

27. GAIT Principle #4
• The basis for identifying key controls in
the three IT processes is based on:
– Inherent risk of not achieving the IT process objectives
– IT process risk indicators
(“How do we select key controls within
those IT processes?”)

28. GAIT Scoping: Step By Step
AS2 begins here
Identify key financial statement captions
Identify the general ledger accounts related to the key
financial statement accounts (significant account)
Identify key transaction processes that affect the
general ledger accounts
Identify and understand related business processes
Identify and understand applications and modules that
support financially relevant business processes
Identify and understand infrastructure that supports the
business processes
Analyze the risks within the integrated business process
(Identify risks)
Identify manual & automated controls & key functionality within Evaluate overall entity level controls
the process that mitigate the risks (Identify key controls)
Identify IT infrastructure elements which support the Identify IT entity level elements and the
demonstrated maturity of the process
application (the rest of the stack)
Validate IT entity level controls
Evaluate the risks related to (and within) the IT
GAIT Starts Here processes which manage the infrastructure & apps

29. GAIT Tools
• Principles Document
• Scenarios and Tutorials
– Online auction settlement process (high IT)
– Rebate approval process (med IT)
– Option expensing process (low IT)
• Ask Dr. GAIT

30. Conclusions and Lessons Learned, Continued
► Improved audit comment wording helps to connect to
things management cares about:
• “We noted poor change control procedures and were unable
to obtain comfort that all changes were authorized and tested
as required”
-- vs. --
• “Poor change control practices introduced the risk of
unauthorized or untested changes to key data such as
annual threshold amounts for toxic chemical releases. Given
the level of precision applied to reviewing the final report
downstream, it is unlikely management would detect such
errors. Our testing disclosed numerous “break/fix” changes
had been made to code or data without supervisory review
and approval or notifying the users.”

31. GAIT Evolution
• Elements of GAIT was incorporated into
PCAOB AS-5
• GAIT-R for Business Risk
– To me, it's the first really well thought out way of
linking IT to any COSO internal control objective
– Unlike ITIL, COBIT: it helps focus on what matters
• The Integrated Auditing Project (“Magic
Glasses”)

32. PCI Problem Definition
• Success of any PCI DSS compliance initiative is
very dependent on accurate definition and
scoping of the Cardholder Data Environment.
• There is a wide variance in practice, experience
and guidance in merchant and QSA community.
• These contribute to scoping errors that result in:
– Overly narrow scope that jeopardizes cardholder data
– Overly broad scope that adds unnecessary cost and
effort for compliance
– Decreased confidence in and frustration with the PCI
DSS standard

33. 33

34. 34

35. 35

36. Source: Gartner RVM Model
(Proctor, Smith)
36

37. 37

38. 38

39. Top A-Ha Moments
• I love auditors: they have a comprehensive
vocabulary that we need – otherwise, we’re
stuck in Flatland
• Principles based guidance is great, as long
as the words are precisely defined
• Auditors have seen the dead people longer
than anyone
• It is possible to make a difference, even in
complex social scenarios
• COSO Cube is simple but great

40. You are only as smart as the
average
of the top 5 people you hang out
with
40

41. The Prescriptive DevOps Cookbook
• “DevOps Cookbook” Authors
– Patrick DeBois, Mike Orzen, John
Willis
• Goals
– Codify how to start and finish
DevOps transformations
– How does Development, IT
Operations and Infosec become
dependable partners
– Describe in detail how to replicate
the transformations describe in
“When IT Fails: The Novel”

42. “The Goal” by Dr. Eliyahu Goldratt

43. 43

44. 44

45. Fred Pond, CIO, Columbia Sportswear
• “When you finish that book, everyone on my
team will need to read it, as well as my
auditors, my boss, and my boss’ boss…”

46. When IT Fails: The Novel and The
DevOps Cookbook
• Coming in July 2012
• “In the tradition of the best MBA case studies, this
book should be mandatory reading for business and
IT graduates alike.”
Paul Muller, VP Software Marketing, Hewlett-
Packard
Gene Kim, Tripwire founder, • “The greatest IT management book of our
Visible Ops co-author generation.”
Branden Williams, CTO Marketing, RSA

47. When IT Fails: The Novel and The
DevOps Cookbook
• Our mission is to positively affect the
lives of 1 million IT workers by 2017
• If you would like the “Top 10 Things Infosec
Needs To Know About DevOps,” sample
chapters and updates on the book:
Gene Kim, Tripwire founder, – Sign up at http://itrevolution.com
Visible Ops co-author
– Email genek@realgenekim.me
– Hand me a business card

48. If you’d like the slides from today’s
presentation…
• Text your name, email, website and the
number 59871 to +1 (858) 598-3980
• Visit:
http://www.instantcustomer.com/go/59871
• Or, scan this QR code:
48