The Role of AI and Automation in helping Internal Audit identify and assess emerging risks

1. The Role of AI and
Automation in Helping
Internal Audit Identify and
Assess Emerging Risks
Steve Biskie, Director, RSM
Manuel Coello, Director, CVS Health

2. AGENDA
• Audit of the future vs. the
audit of today
• Supporting technologies
and processes
• Making it work in real life
• Getting your own program
off the ground

3. Introductions
Steve Biskie Manuel Coello
50 years
combined
analytics &
automation
experience
RSM
steve.biskie@rsmus.com
CVS Health
CoelloM@aetna.com

4. Polling Question 1
Please open the conference app to participate

5. Polling Question
Where are you currently in your journey to a
mature automation program within IA
a. Just getting started
b.
c. Good progress, but much more to accomplish
d.
e. Mature, stable process covering the majority of our risks
Please show in a bar graph

6. Audit of the future vs. the audit of today

7. What does the audit of the future look like?
As an auditor, my day starts with:
1. Notifications of any significant risk
changes that occurredovernight
• The risks themselves
• The tools management uses tomonitor risks
2. Options for how those risk changes
might influence my day/week
3. Actioning the “next steps” of any testing
that could not be fully automated
4. Discussions with management and
other experts about emerging risks and
indicators that could be used to
enhance risk monitoring
…and my day ends with:
1. Planning the next phase of audit
optimization
2. Adjusting/training my army of audit
bots as new information is learned
3. Pressing the “do audit” button as I
head home
©2019 RSM USLLP.All RightsReserved.
One Audit = One Day
Data-DrivenStories build Insight
Speed/Value/Quality

8. IIA Audit Executive Center
2017 North American Pulse of Internal Audit Survey

9. IIA Audit Executive Center
2017 North American Pulse of Internal Audit Survey
“CAEs are often eager to use data analytics becauseit enables them to look at
large volumesof data and quickly identify nonconformingactivities or outliers.
Leveragingthe vast amount of data available in most organizationscan
enhancethe capacity andimpact of internal audit, instilling confidencein
internal audit among our key stakeholders.
These potential benefits may compel CAEs to implement data analytics,
even when the needed structuresand processes are not fully in place.
Pulse results suggest that if CAEs were to audit their own data analytics
practices, many would not have positive results.”

10. “Emerging Risk” both strategic and granular
Strategic ----------------Risks can be identified anywhere in the audit process. ---------------→ Granular

11. Polling Question 2
Please open the conference app to participate

12. Polling Question
Where do you see the largest untapped
value for applying automation
a. Risk Assessment
b. Audit Planning
c. Fieldwork
d. Reporting
e. Post-Audit Follow-up
Please show in a pie chart graph

13. The Need for InnovativeAuditing
Risk
Analytics
Answer questions about past, present, and
future
• IFTTT, SoD, and business rules
• Data visualization
• Process mining
• Risk scoring, modeling, and statistics
• Text mining, machine learning, and AI
RPA
Automate and routinize key audit
tasks
• Scheduled jobs
• Low cognitive task automation
• Cross-application “macros”
• Manual, repetitive or high volume tasks
• Higher-order task automation (w ith AI)
Agile
Organize, prioritize and deliver on audits
• Risk backlog vs defined plan
• Quick sprints, adaptable to changes
• Incremental w orkvs all at once
• Increased information and communication flow
• Client collaboration

14. Supporting technologies and processes

15. Reality Check: We’ve had the tools for awhile…
• Internal Audit Automation has actually been
around for decades
• Traditional audit technologies helped to
automate data analysis procedures
• PC-integrated technologies helped to automate
tasks
• Newer Robotic Process Automation (RPA)
technologies automate where back-end system
access is unavailable

16. RPA Overview
Robotic Process Automation (“RPA”)
RPA refers to a set of modular software programs (or
“bots”) to complete structured, repeatable, and logic-
based tasks by mimicking the actions taken by existing
human staff.
• Developed bots are capable of interacting with and integrating
disparate enterprise applications, databases, and files to limit
the business need to develop custom, application specific
integrations.
• A set of scheduled bots are capable of running on multiple
servers within a company’s environment simultaneously with
minimal impact to resource and network capacity.
RPA Value Proposition
Across industries, RPA enables
organizations of all sizes to efficiently scale
operations with minimal impact to existing
business processes.

17. IA/ComplianceAutomation: The complete toolbox

18. Making it work in real life

19. Re-thinking the Audit Analytics Model
DATA RPA AI+ + +
DO
ANALYTICS = ASSURANCE
THINK ANALYZEGET VALUE
OrganizationOperationTools
DigitalWorkforce

20. Mindset
Startwith capabilitiesthendeploy to auditprojects
Intentional
Daily scrumswith 2-week sprintsarranged to
deliverto thecustomer
Experimental
Data ScienceLab approach- don’tmix
undeveloped capabilitieswith activeaudits
Commitment
Resources& fundsdedicated to
Analytics,Automation,RPAandAI
Authority to decideourown projects
Autonomy
LeverageCorporateResources
Resourceful Capability based
1 32
4 65
➢ Risk & predictive modeling
➢ Natural Language Processing
➢ Geospatial analytics
➢ In-database analytics
➢ Unsupervised models
➢ OCR & encryption
➢ Unstructured data
➢ Robotics
➢ Dashboarding
➢ Self-Service

21. Robotic Process Automation
VALUE
Maturity Level
AUDIT PROCESS
MATURITY
DATA COLLECTION
EFFECTIVENESS META-BOTS
Basic Enhanced Intelligent
ELI-1 – Preparesa dashboard with30+ descriptive analyticsin5
minutes
Penny - Logs intothe systemsand traces transaction IDs and extracts
10+ support docs and puts it in a single PDF
ELI-2 – Intelligentauditornotifications(auditplanprogress,IT charges,
earlyanalytics exceptions,etc.)
Ron – Verifiesanddocumentsthat automatedprocesseswere
executedasintended
Lucy - Manages data requests (In development)
Angela – Navigatesthrougha systemand extracts customer
correspondence
Luca – Logs intobank websitesandextractscustomer payment
information
Webster – Web scrapper that crawls through relevantinformation
containedin a website
AUDIT PROCESS
DATA COLLECTION
EFFECTVENESS
ELI1
Penny
Angela
Luca Webster
ELI2
Ron
Lucy
PDfer
Filer

22. AI / Machine Learning
VALUE
Maturity Level
Supervised
MATURITY
Semi-Supervised
Unsupervised Other
Basic Enhanced Intelligent
Risk Scores – Assessesa transaction riskfrom 1 to 100
Fraud Scores – Assessesa fraudrisk from 1 to 100
Correlations – Relationshipbetweenmultiple variables
Prediction Scores – Calculatesthe audit exceptionprobabilityscore
from 1 to 100
Clustering – Groups unlabeleddatainto similarclusters
Auto-Encoding – Re-construct data usingartificial neural network
Kamila Cluster Model– Clustergroups based on a riskpattern
Text Mining – Investigative fraudmodel usingemails
SUPERVISED
SEMI-SUPERVISED
UNSUPERVISED
Risk Score
Models
Prediction
Score Models
Correlations
NLP
Spatial
Analytics
Auto-Encoding
Fraud Models
Clustering
NLP – Structuresdata from free formtext or phone recordings
Spatial – Re-construct data usingartificial neural network
Other
Kamila Clustering
Text Mining

23. Putting everything together…
Risky T&E
Transaction
ANALYTICS
RPA
AI / Machine Learning
Transaction Source System Audit DW DataEnrichment
- MCCs
- Org Details
- Demographics
DataValidation
Testing rules
to validate the
data
Merchant Info
Webster obtained
additional detailsfrom a
website about the
merchant
Descriptive Analytics

24. Putting everything together…
ANALYTICS
RPA
AI / Machine Learning
PrescriptiveAnalytics Risk Scoring Prediction
20+ Audit
Tests
EarlyWarning UnsupervisedValidation
Ron validatessuccess
of the automated feed
and documents
completenessand
accuracy
Prediction model uses
historical audit findings
to assess the likelihood
of the transaction of
being an exception
Risk model uses
quantitative and
qualitativecalculationsto
assess transaction risk
ELI2 Identifiesa risky
transaction and sendsan
email with theanalysis
Unsupervised models
create clusters of
entertainment and
miscellaneousexpenses

25. Putting everything together…
ANALYTICS
RPA
AI / Machine Learning
TracingDocumentsTransaction Selection
Transaction selection
leveragesrisk and
prediction scores
Penny grabs expense IDs,
accesses Concur, takes
key screenshots/receipts
and consolidatesinto a
PDF
NLP
NLP uses expense
commentsentered by
the employee, structures
data and calculatesrisk
Fraud Analytics
Fraud modelsidentify
risky employees
SpatialAnalytics
Recalculates the
mileagesfrom two
different points

26. Getting your own program off the
ground

27. Typical Progression to Full Automation
Opportunity
identification
&
prioritization
Micro-Task
Automation
Integrated
Task
Automation &
Workflow
RPA Pilot
RPA Task
Bots
RPA
Predictive
Bots
RPA
Cognitive
Bots
“Do Audit”
button
Considerations
• Access to underlying
data
• Process stability
• External auditor
expectations
• Enterprise initiatives
• Resource constraints
• Quality of past
process outcomes

28. 5 Immediate Steps you Can Take
1. Pick a starting point
• Have data
• Have knowledge (and can thus benchmark)
• Likely to get management attention
2. Define KRIs (Key Risk Indicators) that you can measure
• Using data you already have access to
• Using data you can get access to quickly
3. Determine what can be automated immediately, and what should be
automated longer-term
4. Establish a baseline and achievable success measures
5. Start a pilot
• Fail quickly and learn fast

29. Polling Question 3
Please open the conference app to participate

30. Polling Question
In a single word, what do you see as your
biggest barrier to implementing automation
and AI in your department?
Please show in a Word Cloud

31. Summary
• There should be no significant barriers to beginning your
automation initiative TODAY
• Consider quick-hit process improvement opportunities
prior to automation
• Recognize the tools in your toolbox that are right for the
job
• Prioritize low-risk, low-effort areas
• Get started!

32. Q&A

33. TELL US WHAT YOU THINK!
Evaluate this session right in
the IIA Conference App!
Not using the conference app?
Visit: gam.cnf.io to
complete your session
evaluations.