data pipeline, governance, and for growth and updating models regularly needs to be part of the AI strategy from the outset. This session will cover: Defining AI governance: What this means and how definitions of subjects like ethics and effectiveness can differ between organizations. Data governance: Companies must rely on an AI governance program to ensure only high-quality, unbiased and consistent data are used in training. AI is a growing necessity for enterprises / businesses; it provides an avenue for scaling quickly and efficiently. Best practices / implementation: how to implement AI that meets the requirements of the organization’s defined sets of governances. Planning the data pipeline and growing/updating the models: AI is not static in the real world; models must be frequently updated to maintain relevance and accuracy. 3 key takeaways or attendee benefits of the session: Understand how to assess your organization’s need for AI; how to identify the opportune areas for transforming processes, interactions, scaling, cost. How to start the implementation process. Defining data and AI governance and how to build the training data pipeline within that framework. Best practices for maintaining AI; how to use data to evaluate models and continuously iterate on them to reflect the real world.

1. AI Data Acquisition and Governance:
Considerations for Success
Presented by:
Kirsten Gokay
Senior Product Manager

2. Defining AI Governance
2

3. Defining AI Governance
AI Governance is the framework that guides an
organization's AI usage and implementation.
There is no one-size-fits-all framework; each org
must determine what best models its values and
suits its needs.

4. Key Areas of AI Governance
• Performance
• Transparency
• Ethics

5. Performance
• Accuracy
• How well does AI on real-world data?
• Bias/fairness
• Are you coding human bias into your
AI?

6. Performance
Accuracy
• Precision/recall
• Which is more important and
when
• Completeness of context
• Ensuring AI has all the context
when making predictions
Bias/Fairness
• Sampling bias
• How you collect data, from where,
by whom
• Training data bias
• Who is annotating the data

7. Transparency
• Explainability
• Provide clear insight into why AI
performs a certain way
• Objective
• What is the objective or goal of this AI
implementation

8. Ethics
• Intent
• What is the intent of the AI
implementation – why is it being built?
• Responsibility
• What does responsible AI look like

9. Case Study: The Consequences of Bias
• COMPAS algorithm
• Facial recognition
• Amazon hiring
• US healthcare

10. Data Governance
10

11. Data Governance for AI
Data governance is crucial for working
within the organization's AI governance
framework.
It includes:
• Availability
• Usability
• Integrity
• Security

12. Data Governance
Availability
• The data is accessible and
consumable by those who need
it
Usability
• The data is structured, labeled,
and easy to use

13. Data Governance
Integrity
• Data maintains its structure,
qualities, completeness across
its lifecycle
• Data consistency
Security
• Data is protected from
corruption, unauthorized use, or
modification across its lifecycle

14. Case Study: When Security Fails
• Equifax Data Breach: 143 million
• Facebook: 419 million
• Aadhaar: 1.1billion

15. The Growing Necessity of AI
15

16. The Growing Necessity of AI
AI allows organizations to scale quickly and
efficiently.
It can provide data quickly, improve user
experiences, and even save lives.

17. Opportune Areas for AI
User Experience
• Chatbots
• Product
recommendations
• Routine/regimen
recommendations
Process Scaling
• Legal/financial
document extraction
• Security screening
• Updating information
across systems
Robotics
• Vehicles
• Surgery
• Manufacturing
• Agriculture
17

18. Training Data Pipeline &
Maintenance
18

19. Training Data Pipeline & Maintenance
• Data acquisition
• Data annotation
• Auditing the data
• Updating the model(s)

20. Training Data Pipeline
Data Acquisition
• Readily available data
• Open-source datasets
• Third party vendors
Data Annotation
• E.g., image classification, audio
transcription, video event
annotation

21. Training Data Pipeline & Maintenance
Auditing the Data
• Ensure the annotated data is
accurate
• Datasets should be well-
rounded
• Account for edge cases
Updating the Model(s)
• Models don't need to be static!
• Update models frequently
to reflect the real world and
changing data

22. Case Study: Maintaining AI
• Wellio: using human-in-the-loop data
annotation in the data pipeline
• Microsoft Tay: a public chatbot gone
wrong

23. Thank You!
23