1. Digital Forensics for Artificial Intelligence (AI) Systems
Module 10
1
CIS8708-Digital Forensics
(Guide to Computer Forensics and
Investigations)
Dr. Mahdi Fahmideh
School of Business
University of Southern Queensland (UniSQ)
Semester 1 - 2023

2. AI systems make decisions impacting our daily life. Their actions might cause accidents, harm or, more generally, violate
regulations – either intentionally or not – and consequently might be considered suspects for various events.
Malicious by design (Malicious developers) !
◦ Performing a malicious act and the AI system is simply a means to this end. That is, an AI might be used as any other tool, e.g., a
crowbar for burglary, to conduct a criminal act.
◦ Data injection using sophisticated adversarial examples
Examples:
◦ Did the drone drop the object on purpose?
◦ Did the chat bot contact a person, attempting to lure her into a scam?
◦ Did the autonomous car cause the accident due to risky driving?
2

3. Options to tamper with an AI depend on the system’s capabilities and design
◦ Train a model to conduct a malicious act
◦ Use AI system as is, but specify malicious objectives
◦ Altering system internals
◦ Leave model as is but manipulate through adversarial inputs
◦ What else???
Generating cyber crims
3

4. Today, AI systems are ultimately controlled by humans, but they are already capable of conducting tasks autonomously with
learning capabilities which makes them superior to traditional computer programs.
◦ Thus, AI systems provide novel opportunities for attackers!
AI systems might be modified to be malicious where the only security boundary is the acquisition of the AI, which is in
contrast to classical attacks targeting typically protected systems during their operation.
What are other examples of attacker actions and tampering drone?
Malicious by design
4

5. Digital evidence in AI forensic
What are other important digital evidence in AI forensic to be used during crim investigation?
5

6. In the near future, we can imagine foods and groceries
being drone-dropped not just to the time-crunched dual-
income couple with children, but also to the doorstep of an
older adult who is no longer comfortable or capable of
driving to the local market.
◦ List examples of cyber forensic crims that may threat elderly
using this technology
◦ In example that you listed, what are key digital evidence for
the forensic investigation?
◦ What is your countermeasure to tackle these threats in
advance?
Real world example 1: Delivering Meals to the Elderly With Drones
6

7. Cargo drones can soon move goods across the globe.
◦ List examples of cyber forensic crims that may threat sender and receivers using cargo drones technology.
◦ In example that you listed, what are key digital evidence for the forensic investigation?
◦ Discuss (Recall Module 4)
◦ The data acquisition methods, e.g., Bit-stream disk-to-image file, Bit-stream disk-to-disk, etc.
◦ Types of acquisitions static acquisitions and live acquisitions
Real world example 2: Cargo drones
7

8. Strategies for Investigation
Strategies can focus on each of AI components that determine behaviour:
model, model objective and data
◦ Data focused: Since training data determines model behaviour and
operational data reflects model behaviour, data on its own might be
sufficient to determine malicious intent
◦ Model focused: Model analysis might use abstract reasoning based on
model definitions. Models might also be analysed through empirical
investigation, i.e. input-responses
◦ Investigate the input-output relationship of a model: The model can be
treated as a black-box. The analysis relies on investigating model
behaviour based on its decisions
◦ Investigate the reaction of model internals to inputs: This strategy
requires more access to allow for white or grey-box testing. It includes
analysing the AI system components. For instance, a deep learning
network consists of layers and each layer has neurons that perform simple
computations
8

9. Strategies for Investigation (continue)
Example: Data mining of bank statement may help to provide
some insights into criminal activities
A strange change in expenditure behaviour (cost) in certain
days or dates, etc
Superficial bank statement (with kind permission from Commonwealth Bank)
9

10. Reviewing a Case (Recall from Module 3)
General tasks you perform in any computer forensics case:
◦ Identify the case requirements
◦ Plan your investigation
◦ Conduct the investigation
◦ Complete the case report
◦ Critique the case
Strategies for Investigation (continue)
10

11. Strategies for Investigation (continue)
◦ Investigative Questions
◦ what is the likelihood of a given suspected AI performing these decisions compared to those of other models?
◦ is the suspected AI reacting to objects related to the incident more strongly than other models?
◦ is the suspected AI behaving normally?
◦ Access to System Internals
◦ black-box model
◦ white-box model
◦ grey-box model
◦ Access to training and test data
◦ Attacker Model
◦ The attacker wants to trigger an action A, e.g. dropping of the parcel given a person’s face
◦ Challenges of Data-driven Investigation
◦ Analyzing input output behavior which comes with specific difficulties
◦ Check if all input samples produce the correct output. If not, the system is suspicious. In practice, the situation
is more difficult. There might be too many inputs to test and not all of them are typically available!
◦ Technical Setup
◦ Training data
11

12. An example of Deep Learning Cyber Framework (DLCF)
12

13. ◦ Initialization Process
◦ the initialization process deals with the procedures of initiating an investigation whenever an incident is detected. This is mostly a post-event response
mechanism and includes first response after incident detection, planning and preparing a digital investigation process.
◦ PDE (Potential Digital Evidence) Data Sources Identification
◦ different types of PDE that can be captured
◦ identify reliable sources and/or the origin of each of the different types of PDE at hand before the analysis process begins
◦ Deep learning enabled cyber forensic investigation engine
◦ this layer is meant to handle the investigative process. The phases integrated in this layer include: evidence acquisition, evidence preservation, evidence
analysis and finally evidence interpretation
◦ Forensic Reporting and Presentation
◦ once the investigative process is complete, a forensic report needs to be provided
◦ this report is what is then presented to the different stakeholders
◦ draw a conclusion from observed values and determine to what category new observations belong
◦ Report
◦ a detailed analysis of all the PDE captured
◦ proof and justification of all sources of each captured item of the evidence
◦ a detailed descriptions of each captured item of evidence and how it was preserved
◦ links and relationships that exist between sources and evidence captured
◦ detailed descriptions of the intentions of the attacker to the targeted victims
◦ explanations on the effects of the attack to the targeted victims
◦ and any other relevant information to the investigation at hand
An example of Deep Learning Cyber Framework (Continue)
13

14. ◦ Decision making and case closure
◦ the last layer handles decision making and case closure
◦ the jury and the law enforcement agencies in most cases are human beings hence the inability to fully automate this phase
An example of Deep Learning Cyber Framework (Continue)
14

15. Responsible AI. When it comes to AI and Ethics/Law, there are two interrelated aspects of the topic. One is on how
to design, develop, and validate AI technologies and systems responsibly (i.e., Responsible AI) so that we can
adequately assure ethical and legal concerns, especially pertaining to human values.
The use of AI itself as a means to achieve the Responsible AI ends!
The inherent and technical trustworthiness of an AI system can be directly reflected in technologies/products via
code, algorithms, data or system design or indirectly reflected via the software development processes).
Ethical principles in designing Responsible AI
15

16. Australia’s ethical AI principles contain eight key principles (CSIRO Data61 team proposal)
P1: Human, social and environmental wellbeing: Throughout their lifecycle, AI systems should benefit individuals,
society and the environment.
P2: Human-centred values: Throughout their lifecycle, AI systems should respect human rights, diversity, and the
autonomy of individuals.
P3: Fairness: Throughout their lifecycle, AI systems should be inclusive and accessible, and should not involve or result in
unfair discrimination against individuals, communities or groups.
P4: Privacy protection and security: Throughout their lifecycle, AI systems should respect and uphold privacy rights and
data protection, and ensure the security of data.
P5: Reliability and safety: Throughout their lifecycle, AI systems should reliably operate in accordance with their intended
purpose.
P6: Transparency and explainability: There should be transparency and responsible disclosure to ensure people know
when they are being significantly impacted by an AI system, and can find out when an AI system is engaging with them.
P7: Contestability: When an AI system significantly impacts a person, community, group or environment, there should be a
timely process to allow people to challenge the use or output of the AI system.
16

17. Twitter allows businesses to engage personally with consumers. However, there’s so much data on Twitter that it can be hard for brands to prioritize
which tweets or mentions to respond to first. That's why sentiment analysis has become a key instrument in social media marketing strategies. Sentiment
analysis is a tool that automatically monitors emotions in conversations on social media platforms and can aid twitter users or policy makers in decision
making, e.g., newspaper, disaster management, gossip, fake news
A poor sentiment analysis application design may cause some forensics threats for example:
(i) false information or misleading its users
(ii) polarization, e.g., dividing communities and hindering constructive dialogue
(iii) malicious users exploit the outputs of a sentiment analysis to identify and target specific individuals for online harassment or bullying
(iv) And other examples..
What are the examples of AI principles that should be supported by a Twitter Sentiment Analysis application to avoid forensic threats?
AI principle Example
P1: Human, social, and
environmental wellbeing
Provide a safe and respectful environment for users and take action
against accounts or content that violate community guidelines or terms of
service such as online harassment and hate speech
P3: Fairness Ensure that the sentiment analysis application does not favour or
discriminate against specific individuals, groups, or opinions
P4: Privacy protection
and security
Anonymize or aggregate data to prevent the identification of individuals
P5: Reliability and safety Collect comments that are from valid users, e.g., not fake, comment/post
pre-processing
P6: Transparency and
explainability
Help users understand how their comments/posts is being analysed and
reduces the potential for misunderstandings or mistrust
Case study: Application of AI principles in developing Twitter Sentiment Analysis applications
17

18. For more information:
- Schneider, Johannes, and Frank Breitinger, “AI Forensics: Did the artificial intelligence system do it? why?” (2020).
- Karie, Nickson M., Victor R. Kebande, and H. S. Venter, “Diverging deep learning cognitive computing techniques into cyber
forensics”, Forensic Science International: Synergy 1 (2019): 61-67.
- Zhu, Liming, et al., “AI and Ethics—Operationalizing Responsible AI” Humanity Driven AI. Springer, Cham, 2022. 15-33.
- Sarker, Iqbal H. "Machine learning: Algorithms, real-world applications and research directions." SN Computer Science 2.3 (2021):
1-21.
18