AICHROTH Systemaic evaluation and decentralisation for a (bit more) trusted AI

© Fraunhofer IDMT
SYSTEMATIC EVALUATION AND DECENTRALIZATION
FOR (A BIT MORE) TRUSTED AI
Patrick Aichroth
Fraunhofer Institute for Digital Media Technology (IDMT)
FIAT/IFTA World Conference 2019
Dubrovnik, 2019-10-23

© Fraunhofer IDMT
Systematic evaluation and decentralization for Trusted AI
Intro – AI vs. Machine Learning (vs. DeepLearning)
“Difference between machine learning and AI:
If it is written in Python, it's probably machine learning.
If it is written in PowerPoint, it's probably AI” – Mat Velloso
(https://twitter.com/matvelloso/status/1065778379612282885)

© Fraunhofer IDMT
Intro – AI vs. Machine Learning (vs. DeepLearning)
 AI: “Components or systems which behave intelligently“, typically (but not always) including:
 Sensing: Translation (of sensor data) into a conceptual representation
 Reasoning: Manipulation of the conceptual representation
 Acting: Translation into actions
 Types:
 Strong / broad / general AI: can perform all tasks as well as or better than humans
 Weak / narrow AI: specialized for specific tasks
 Technology:
 Logic / rule-based: machine reasoning (e.g. symbolic logic, rule engines, expert systems, etc.)
 Learning-based: machine learning (“mathematical model based on sample data, to make
predictions or decisions without being explicitly programmed”, e.g. Deep Learning)

© Fraunhofer IDMT
Intro – Example #1: Automatic Metadata Extraction
 3 elements:
 Reasoning: Manipulation of the
conceptual representation
 Sensing: Translation (of sensor data)
into a conceptual representatioN
 Borders can be blurry!
TECHNOLOGIESCAPABILITIES
TEXT
Semantic analysis
Content classification
Natural language search
Machine translation
Emotion detection
Language detection
…
Virtual
Agents
Text
Analytics
VISION
Object recognition
Face recognition
Object tracking
Optical character recognition
Handwriting recognition
Emotion detection
Gender/age detection
Scene recognition
…
Video
Analytics
Image
Analytics
SOUND
Speech To Text/Diarization
Speech Recognition
Music and Speech detection
Audio forensics
Emotion detection
Language detection
Sound recognition/Audio matching
Music Annotation
…
Music
Analytics
Speech
Analytics

© Fraunhofer IDMT
Intro – Example #2: Hybrid Recommendation
 3 elements:
 Reasoning: Manipulation of the
conceptual representation
 Sensing: Translation (of sensor data)
into a conceptual representation
 Borders can be blurry!
Item 1 … Item N
User 1 X ?
…
User N X X

© Fraunhofer IDMT
AI - Opportunities & Risks (as always)
“Machines were, it may be said, the weapon employed by the capitalists
to quell the revolt of specialized labor” - Karl Marx
Vs.
“Our technology, our machines, is part of our humanity. We created them
to extend ourselves, and that is what is unique about human beings” - Ray Kurzweil

© Fraunhofer IDMT
AI – Opportunities for the media domain
 journalists and content-creators:
 better content discovery
 increased efficiency
 more comprehensive understanding of reality
 providers:
 better engage with audience, e.g. via recommendation
 valorize archives, improved advertising placement
 better insights into consumption habits
 readers / audience:
 personalized news
 improved discovery

© Fraunhofer IDMT
AI - Risks – Example #1 (Security – “DeepFakes”)
How to distinguish natural vs. synthetic material (but there are approaches that could help)
Example A B
#1
#2
#3
#4
“This person does not exist” based on StyleGAN
(https://thispersondoesnotexist.com/)
Quiz: “Tacotron 2 or Human?”
(https://google.github.io/tacotron/publications/tacotron2/index.html)
“Fictitious speakers”: Google Tacotron speech synthesis
(https://google.github.io/tacotron/publications/speaker_adaptation/index.html)

© Fraunhofer IDMT
AI - Risks – Example #2 (Privacy – “Unwanted Data Loss”)
e.g. AshleyMadison 2015: several suicides and huge costs
35+ Mio names, marital status, credit card details, locations published
Some AI systems deal with very sensitive data (but there are ways to address the related risks)
Nothing to hide?
(https://www.ashleymadison.com/en-us/) (https://upload.wikimedia.org/wikipedia/commons/0/02/CCTV_in_toilet.jpg)
 WI

© Fraunhofer IDMT
AI - Risks – Example #3 (Diversity – “Filter Bubbles”)
"Personalization is sort of privacy turned inside out: it’s not the problem of controlling what
the world knows about you, it’s the problem of what you get to see of the world."
http://www.brainpickings.org/index.php/2011/05/12/the-filter-bubble/
 some degree of ideological bias is unavoidable
 to be able to complete avoid contradicting opinions
(or facts) is another story
 we tend to engage most with who and what fits to
our beliefs: confirmation bias
 personalization / recommendation can (and currently
tends to) reinforce that bias
 this is due to a focus on one success criteria: utility
we choose to become isolated and polarized (but we can decide otherwise, the tools are there)
https://miro.medium.com/max/875/1*1xBV-hwspHlNdy7sW3Oouw.png

© Fraunhofer IDMT
Trusted AI - HLEG guidelines
 “Ethics guidelines for trustworthy AI” from the EC’s High-Level Expert Group on AI (HLEG),
(https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai)
 Criticism (business interests, vagueness, etc.), but first “official” basis for a discussion about TAI
 TAI Principles (lawful, ethical, robust) and requirements:
 Human agency and oversight: empower humans, informed decisions, oversight mechanisms
 Technical Robustness and safety: resilience, security, safety, accuracy, reliability, reproducibility
 Privacy and data governance: privacy & data protection, data governance
 Diversity, non-discrimination and fairness: “unfair bias must be avoided”, stakeholder inv.
 Transparency: transparent data, system, process, business models, explainability
 Accountability: responsibility and accountability, auditability of algorithms, data and design
 Societal and environmental well-being: “should benefit all humans” incl. future generations
 There are many tools and technologies that can help, e.g. PET, but this is for another time ;-)

© Fraunhofer IDMT
Requirements & Evaluation
“All models are wrong, but some are useful” - George E. P. Box
“You'll fail at a 100% of the goals you don't set” - Mark Victor Hansen
“The most important thing about goals is having one” – Geoffrey F. Abert

© Fraunhofer IDMT
Human staff recruiting AI component “recruiting” / dev
TECHNOLOGIESCAPABILITIES
TEXT
Semantic analysis
Content classification
Natural language search
Machine translation
Emotion detection
Language detection
…
Virtual
Agents
Text
Analytics
VISION
Object recognition
Face recognition
Object tracking
Optical character recognition
Handwriting recognition
Emotion detection
Gender/age detection
Scene recognition
…
Video
Analytics
Image
Analytics
SOUND
Speech To Text/Diarization
Speech Recognition
Music and Speech detection
Audio forensics
Emotion detection
Language detection
Sound recognition/Audio matching
Music Annotation
…
Music
Analytics
Speech
Analytics

© Fraunhofer IDMT
Human staff recruiting
1. job description:
define needs, define role / position
2. job interview: short list of applicants,
demonstration of capabilities, selection
3. KPI definition:
set goals to be met, define metrics
4. performance reviews: monitoring,
and in case, replacement or retraining
…
“How managing AIs compares and contrasts to managing human staff”
(https://blog.datarobot.com/nine-ways-that-managing-an-ai-is-like-managing-a-human-and-two-ways-its-different)
AI component “recruiting” / dev
Define needs & requirements
Select potential candidates
Define success criteria / evaluation metrics
Create training and test data + train
Perform Evaluation + tune + retrain

© Fraunhofer IDMT
Requirements & Evaluation: The Key for TAI!
 Ensure that AI provides value and is
aligned with requirements, e.g. TAI
Functional, e.g. user story
Non-functional, e.g. usability, security, privacy,…
 Ensure that success criteria / eval metrics
optimize AI for the right thing, see above
 Ensure that data fits to needs (wrt features,
variability, breadth, bias, security, privacy)
 Perform evaluation efficiently and
continuously, using user feedback

© Fraunhofer IDMT
e.g. for an AI-based recommendation system:
 As a user, I want to be notified about
a diverse set of news articles related to a topic I
have expressed interested for, so that I learn more
about that topic
 As a user, I want to be able to be able to see and
edit the topics the system currently associates me
with (based on my viewing habits), so that I can
understand and control what I am able to see
 As a publisher, I want to be able to provide
privacy-enhanced recommendations for users, so
that I reach a larger audience
 The are many potential conflicts / trade-offs
between requirements to be managed!

© Fraunhofer IDMT
Based on the requirements
 select appropriate criteria / metrics, e.g. for an AI-
based recommendation system:
 Use not only utility („how relevant are recos for
users“), measured e.g. via precision and recall
 Also use e.g. diversity & possibly unexpectedness
and novelty metrics against filter bubble effect
Regression
• Root mean square
error
• Correlation coeff
• R Square (R²)
• MSPE (Mean square
percentage error)
Classification
• Precision, recall,
F-measures
• ROC, AUC
• Accuracy
• Balanced Accuracy
• Confusion Matrix
Unsupervised
Models
• Rand index
• Mutual information
• Homogeneity score
Retrieval
• Precision, recall,
F-measures
• Mean average
precision (MAP)
• Precision at K
• Discounted cumulative
gain

© Fraunhofer IDMT
Ensure that data fits to user stories /
requirements wrt features, variability, breadth
 address problematic bias, considering the
user stories / requirements
 address security and privacy issues
https://towardsdatascience.com/using-what-if-tool-to-investigate-machine-learning-models-913c7d4118f

© Fraunhofer IDMT
 Perform evaluation, and in case of issues,
use e.g. the What-If Tool (OSS, Google) to
examine, evaluate, and compare models
 Can also be applied to investigate bias &
fairness issues
https://towardsdatascience.com/using-what-if-tool-to-investigate-machine-learning-models-913c7d4118f

© Fraunhofer IDMT
Requirements & Evaluation – AI vs. “normal” dev
ID/Name ID and concise, results-oriented name for the use case.
Actors Actor (user, component) that will be initiating this use case and any other actors who will participate in completing the
use case.
Description Brief description of the reason for and outcome of this use case, and a high-level description of the sequence of actions
and outcome of executing the use case.
Pre-Conditions Any activities that must take place or any conditions that must be true, before the use case can be started.
Post-Conditions State of the system at the conclusion of the use case execution.
Normal Flow Detailed description of the user actions and system responses that will take place during execution of the
use case under normal, expected conditions.
Alternate Flows Other user actions that can take place within this use case.
See Software Development Lifecycle (SDLS)
 Modularization for Testing and improved
transparency / auditability? Sensing &
Reasoning vs. Acting?
 AI vs. “normal” dev: completely different?
 AI as part of bigger systems
 Need for comparative testing
 Similar, with some differences
 unit testing applicable
 Importance of public test datasets https://melsatar.blog/2012/03/15/software-
development-life-cycle-models-and-methodologies/

© Fraunhofer IDMT
Requirements & Evaluation: Automatic Testing!
Automatic testing can be much more efficient, see e.g. EU project REWIND
(https://sites.google.com/site/rewindpolimi/home):
 Idea: Apply the concept of Unit Testing to Classifier Testing
 Specify component input and output (once per component)
 Specify use case and evaluation metrics (once per use case)
 Create and run tests with random test material (many times)
 Benefits
 Continous testing and Comparative testing
 Collaboration: Reuse (or create) test content
 was implemented as decentralized / on-prem solution
(to enable testing without the need to integrate components)

© Fraunhofer IDMT
Requirements & Evaluation: Automatic Testing!

© Fraunhofer IDMT
Trade-Offs, Bias and Fairness
“A good intention, with a bad approach, often leads to a poor result” - Thomas A. Edison
“Machines have altered our way of life, but not our instincts.
Consequently, there is maladjustment” - Bertrand Russell

© Fraunhofer IDMT
Trade-Offs - Example
 There is often a trade-off between
precision and recall for classification
 e.g. biomarker testing:
 TP: Good - patient can move forward with treatment!
 FP: Bad - patent doesn't have the disease but e.g.
unnecessary fear, follow-up treatments etc.
 TN: Good - all OK
 FN: Bad - patient is sick but it goes undetected
 Threshold: either less sick people are correctly identified,
or less healthy people  trade-off
 Trade-off needs to be made based on the context /
requirements and priorities
http://i1.wp.com/thingsitellmymom.com/wp-
content/uploads/2015/05/sensitivity_specificity.jpg
TP
TNFN
FP
http://www.medcalc.org/manual/_help/images/roc_intro1.png

© Fraunhofer IDMT
Biases - examples (image labeling)
 “Google ‘fixed’ its racist algorithm by removing
gorillas from its image-labeling tech”
(https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-
algorithm-ai )
Nearly three years after the company was called out, it
hasn’t gone beyond a quick workaround [...] A
spokesperson for Google confirmed to Wired that the
image categories “gorilla,” “chimp,” “chimpanzee,” and
“monkey” remained blocked on Google Photos after
Alciné’s tweet in 2015. “Image labeling technology is
still early and unfortunately it’s nowhere near
perfect,” said the rep.“
 Problem with the training data, but not so easy to fix
 See trade-off https://media2.wnyc.org/i/402/317/l/80/1/gorillas_google.png

© Fraunhofer IDMT
Biases – issues and challenges
 “bias: mismatch between the training data distribution and a desired fair distribution” (IBM)
 i.e. the assessment of bias depends (also) on the goals / requirements and context, and on
what “we” want ( fairness)
 types of biases (sometimes conflicting!): (https://towardsdatascience.com/5-types-of-bias-how-to-eliminate-them-in-your-machine-learning-project-75959af9d3a0)
 sample bias: training data not adequate for the context
 exclusion bias: due to the exclusion of feature(s) from the dataset
 observer bias: the tendency to see what we expect / want to see (less relevant?)
 prejudice bias: cultural influences or stereotypes (e.g. people at work reflecting gender roles)
 measurement bias: issue with device / tool used for observation (e.g. broken camera)
 direct bias: use a protected attribute to make a decision - easy to avoid
 indirect bias: value of other attributes correlated with value of a protected attribute (e.g. height
related to gender) – can be very difficult to avoid

© Fraunhofer IDMT
Biases - examples (facial recognition)
 In a big city with 1 Million inhabitants, there are 10 terrorists. In an attempt to catch the
terrorists, the city installs an alarm system with a surveillance camera and automatic facial
recognition software, which has two failure rates:
(a) 1% false negative rate: If the camera scans a terrorist, a bell will ring 99% of the time, and it will
fail to ring 1% of the time
(b) 1% false positive rate: If the camera scans a non-terrorist, a bell will not ring 99% of the time, but
it will ring 1% of the time.
 Suppose now that an inhabitant triggers the alarm. What is the probability that the person is a
terrorist?
(a) 99% (b) 98% (c) 50% (d) 10% (e) 1% (f) 0.1%

© Fraunhofer IDMT
Biases - examples (facial recognition)
 0,1%
 Intuition often fails us, we have many cogitive biases, which can lead to serious mistakes
 AI and other tools can sometimes help to address them and/or to understand them better
fp (error #1) 1,00% signals a match, where it shouldn't (false-positive)
fn (error #2) 1,00% does not signal a match, where it should (false-negative)
frquency of occurence 10 of 1000000 (per time unit)
real hint wrong hint
signal 10 10000
no signal 0 990000
10 1000000
1 : 1010 0,1% likelihood that a match is a true alarm
1010 : 1 99,9% likelihood that match is a false alarm

© Fraunhofer IDMT
Fairness: Do we agree on what "fair" means?
 equal outcomes or equal opportunities?
 group fairness: ensure that on average, you
don’t discriminate against a protected group
 individual fairness: apply the protection to
each and every individual
 if you choose to target equal outcomes, you
cannot ensure individual fairness
Example: Loan strategy for two populations with
different chances for paying back a loan
 There is no obvious solution as to which fairness
principle to apply
 All solutions are “unfair” from some perspective
 Interesting: effects of assigning labels to groups
https://research.google.com/bigpicture/attacking-discrimination-in-ml/

© Fraunhofer IDMT
Biases – proposal
 Can bias be “eliminated”? No: "statistical bias is intrinsic to AI in contexts where relevant features
are not distributed homogeneously in different (sub-)groups of the population“
(https://algorithmwatch.org/en/trustworthy-ai-is-not-an-appropriate-framework/)
 But we can make choices re which biases to accept and which to neutralize, based on "our"
definition of fairness for a given context, and using guidelines and tools
 There can be bias in data that should not be there (because it does not reflect reality)
 There can be bias that probably should be there (because it does reflect reality)
 we need to know what the world looks like (to see problems and fix them)
 we have to agree on what “we” want beforehand, i.e. what the world should look like –
which is neither obvious nor static, and for a society / politics, not engineers to decide
 If possible, it can help to differentiate sensing & reasoning vs. acting (which is more critical)
 example: radio should e.g. play 25% from regional artists, implemented “on top”  transparency!

© Fraunhofer IDMT
Decentralization (and Cooperation)
 In some cases, there can be good reasons for on-prem processing of data
 Can promote e.g. privacy and security, and therefore TAI!
 Rights or legal issues, traffic cost
 Annotated content is valuable (key driver for AI)!
 At the same time, collaboration can bring huge benefits or is simply necessary
 Depending on the case, decentralization can be a way to combine both:
 Federated recommendation: One “face” to the customer / user
 Collaborative curation & (rights) metadata exchange: Share resources and metadata
 Collaborative evaluation & dev of AI / AME: Share resources and know-how

© Fraunhofer IDMT
 Requirements for doing such things are
 Interoperability
 Common standards (e.g. metadata)
 Common IDs, hashing, perceptual hashing
 Depending on the case, “new” tools can help:
 Federated learning
 Secure Multiparty Computation
 Differential Privacy
 Full Homomorphic Encryption
But this is for another time ;-)

© Fraunhofer IDMT
“We may have all come on different ships,
but we're in the same boat now” - Martin Luther King Jr.
“If we do not hang together, we will all hang separately” - Benjamin Franklin
“In the long history of humankind (and animal kind, too) those who learned to
collaborate and improvise most effectively have prevailed” - Charles Darwin

© Fraunhofer IDMT
Patrick Aichroth
Head of Media Distribution and Security
E-Mail: patrick.aichroth@idmt.fraunhofer.de
Thank you!

AICHROTH Systemaic evaluation and decentralisation for a (bit more) trusted AI

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie AICHROTH Systemaic evaluation and decentralisation for a (bit more) trusted AI

Ähnlich wie AICHROTH Systemaic evaluation and decentralisation for a (bit more) trusted AI (20)

Mehr von FIAT/IFTA

Mehr von FIAT/IFTA (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

AICHROTH Systemaic evaluation and decentralisation for a (bit more) trusted AI

Hinweis der Redaktion