Machine learning model fairness and interpretability are critical for data scientists, researchers and developers to explain their models and understand the value and accuracy of their findings. Interpretability is also important to debug machine learning models and make informed decisions about how to improve them.
In this session, Francesca will go over a few methods and tools that enable you to "unpack” machine learning models, gain insights into how and why they produce specific results, assess your AI systems fairness and mitigate any observed fairness issues.
Using open-source fairness and interpretability packages, attendees will learn how to:
- Explain model prediction by generating feature importance values for the entire model and/or individual data points.
- Achieve model interpretability on real-world datasets at scale, during training and inference.
- Use an interactive visualization dashboard to discover patterns in data and explanations at training time.
- Leverage additional interactive visualizations to assess which groups of users might be negatively impacted by a model and compare multiple models in terms of their fairness and performance.
3. Mehrnoosh Sameki
Senior Product Manager
Microsoft Azure ML
Minsoo Thigpen
Product Manager
Microsoft Azure ML
Ehi Nosakhare
Machine Learning Scientist
Microsoft AI Acceleration and
Development Program
4. Machine Fairness
How to assess AI system’s fairness
and mitigate any observed
unfairness issues
www.aka.ms/Fairlearn-build2020
Article
20. Interpretability Approaches
• Given an existing model/system, extract
explanations
• Typically provide approximate explanations
• Examples: LIME, SHAP, Permutation Feature
Importance, Sensitivity Analysis, Influence
Functions, etc.
Tools to explain the system
www.aka.ms/InterpretML-github
21. SHAP
(SHapley Additive
exPlanations)
• Not a new concept
• Concept based on game theory
• Mathematically sound
• Application in ML relatively new
www.aka.ms/InterpretML-github
23. How much has each feature contributed
to the prediction compared to the average prediction?
House price
prediction:
€ 300,000
Average house
price prediction
for all apartments
is € 310,000
Delta here is
- €10,000
www.aka.ms/InterpretML-github
25. SHAP
How do we calculate Shapley values?
• Take your feature of interest (e.g.,
cats-banned) and remove it from the
feature set
• Take the remaining features and
generate all possible coalitions
• Add and remove your feature of
interested to each of the coalitions
and calculate the difference it makes
www.aka.ms/InterpretML-github
26. SHAP Pros SHAP Cons
Based on a solid theory and distributes the
effects fairly
Computation time: 2k possible coalitions of the
feature values for k features
Contrastive explanations: Instead of comparing a
prediction to the average prediction of the entire
dataset, you could compare it to a subset or even
to a single data point.
Can be misinterpreted
Inclusion of unrealistic data instances when
features are correlated.
www.aka.ms/InterpretML-github
27. Interpretability Approaches
• Models inherently interpretable
• Typically provide exact explanations
• Examples: Linear Models, Decision Rule Lists,
Explainable Boosting Machines, Shallow
Decision Trees, etc.
Tools to explain the system
www.aka.ms/InterpretML-github
28. Linear Models
• (Generalized) Linear Models
• Current standard for interpretable
models
• Learns an additive relationship
between data and response:
• y = β0 + β1x1 + β2x2 + ... + βn xn
• βi terms are scalar – single number
that multiplies to each feature value
• Each feature contributes a “score”
that adds up to final output
Example Model :
House Price = 50,000 + 0.2*sq_ft +
10000*bedrooms +
7000*bathrooms
Using this model on a new house:
- 5 Bedrooms
- 3 Bathrooms
- 3000 Sq Feet
- House Price = 50000 + 0.2*3000
+ 10000*5 + 7000 * 3
- House Price = $121,600
www.aka.ms/InterpretML-github
29. Explainable Boosting Machine
• Generalized Additive Models (with Pairwise
Interactions)
• GAMs have existed since 1980s
• MSR invented new methods of training GAMs for
higher accuracy
• Think of it as a “more expressive” linear model
• Still additive!
• Linear Model: y = β0 + β1x1 + β2x2 + ... + βn xn
• Additive Model: y = f1(x1) + f2(x2) + ... + fn (xn)
• Additive Model with Pairwise Interactions
(EBM):
y = Ʃi fi (xi) + Ʃij fij (xi , xj )
• Full Complexity Model: y = f (x1, ..., xn)
Linear Model can only learn a
straight line!
www.aka.ms/InterpretML-github
30. Demo of
Interpretability for
Tabular Data
Hardware Performance Dataset - The
Regression goal is to predict the
performance of certain combinations
of hardware parts.
www.aka.ms/InterpretML-github
31. Demo dataset
https://archive.ics.uci.edu/ml/datasets/Computer+Hardware
Hardware Performance Dataset - The Regression goal is to predict the
performance of certain combinations of hardware parts.
• vendor name: 30
(adviser, amdahl,apollo, basf, bti, burroughs, c.r.d, cambex, cdc, dec, dg, formation, four-phase,
gould, honeywell, hp, ibm, ipl, magnuson, microdata, nas, ncr, nixdorf, perkin-elmer, prime,
siemens, sperry, sratus, wang)
• Model Name: many unique symbols
• MYCT: machine cycle time in nanoseconds (integer)
• MMIN: minimum main memory in kilobytes (integer)
• MMAX: maximum main memory in kilobytes (integer)
• CACH: cache memory in kilobytes (integer)
• CHMIN: minimum channels in units (integer)
• CHMAX: maximum channels in units (integer)
• PRP: published relative performance (integer)
• ERP: estimated relative performance from the original article (integer)
34. • Allocation: extends or withholds opportunities, resources, or information.
• Quality of service: whether a system works as well for one person as it does
for another
• Stereotyping: reinforce existing societal stereotypes
• Denigration: actively derogatory or offensive
• Over or under representation: over-represent, under-represent, or even
erase particular groups of people
Crawford et al. 2017
Types of harm
35. What is
Fairlearn?
A new approach to measuring and mitigating
unfairness in systems that make predictions, serve
users, or make decisions about allocating resources,
opportunities, or information.
www.aka.ms/FairlearnAI
36. There are many ways that an AI system can behave unfairly.
Fairness in AI
Avoiding negative outcomes of AI systems for different groups of people
A model for screening loan or job application
might be much better at picking good candidates
among white men than among other groups.
A voice recognition system might
fail to work as well for women as it
does for men.
www.aka.ms/FairlearnAI
37. A toolkit that empowers developers of artificial intelligence systems to
assess their systems' fairness and mitigate any observed fairness issues.
Helps users identify and mitigate unfairness in their machine learning
models with a focus on group fairness.
Automatically
analyze a model’s
predictions
Provide the user with
insights into (un)fairness of
their model’s predictions
Support (algorithmic)
methods to mitigate
unfairness
www.aka.ms/FairlearnAI
38. • Allocation: extends or withholds opportunities, resources, or information.
• Quality of service: whether a system works as well for one person as it does
for another
• Stereotyping: reinforce existing societal stereotypes
• Denigration: actively derogatory or offensive
• Over or under representation: over-represent, under-represent, or even
erase particular groups of people
Crawford et al. 2017
Types of harm addressed by Fairlearn
www.aka.ms/FairlearnAI
39. Harm of “allocation”
Example Scenarios: Lending
The data set describes whether each individual repaid the loan or not.
• [Classification] Recommends whether a given individual should get a loan. The
trained model outputs: Yes/Maybe/No decision.
Example Scenarios: School Admissions
The data set describes what was the first-year GPA of each student.
• [Regression] For a given applicant, predicts their GPA at the end of the first year.
The trained model outputs a real-valued prediction that is used as a score to
screen applicants.
www.aka.ms/FairlearnAI
40. Harm of “quality of service”
Example Scenarios: News Recommendation
The training data indicates what article was presented to which user, whether the user
clicked, and how much time the user spent on the article.
• [Classification] Predict which article to show to each user to optimize click-
through rate. The trained model outputs: Yes/No decision.
Two kinds of (group) fairness: across users (quality of service), or across
publishers/topics of articles (quality of service, but also allocation).
www.aka.ms/FairlearnAI
42. Fairness assessment
through disparity metrics
Disparity in Performance
• How the model accuracy differs across different buckets of a sensitive feature
(e.g., how accuracy of the model differs for "female" vs. "male" vs. “unspecified"
data points)
Disparity in Selection Rate
• How the model predictions differ across different buckets of a sensitive feature
(e.g., how many "female" individuals have received prediction `approved` on their
loan application in contrast to "male" and “unspecified" data points?).
www.aka.ms/FairlearnAI
43. Fairness assessment
via Fairlearn visualization dashboard
Let’s see a demo!
www.aka.ms/FairlearnAI
Mitigating Disparities in Ranking from Binary Data
- An example based on the Law School Admissions Council's National Longitudinal Bar
Passage Study
45. Demo dataset
Law School Admissions Council's (LSAC) National Longitudinal Bar Passage Study
• The data set contains information about law students collected by LSAC between
1991 and 1997.
• Some of the information is available at the admission time (such as the undergraduate
GPA and LSAT score), and some describes the performance of the students once
admitted.
• We also have access to their self-identified race.
• To simplify this example, we will limit the attention to those self-identified
as black and white (two largest groups) and restrict our attention to two features
(undergraduate GPA and LSAT score).
47. Demographic parity:
Applicants of each race (gender, ...) have the
same odds of getting approval on their loan applications
Loan approval decision is independent of protected attribute
Equalized odds:
Qualified applicants have the same odds of getting approval on their
loan applications regardless of race (gender, …)
Unqualified applicants have the same odds of getting approval on their
loan applications regardless of race (gender, …)
Fairness Criteria
www.aka.ms/FairlearnAI
48. Reductions
approach:
Wrapper
around
standard ML
algorithms
Input:
• any standard ML training algorithm (as a
black box)
• data set including sensitive feature
Output:
• a trained model that minimizes error
subject to fairness constraints
Advantages:
• doesn’t need to access the sensitive
feature at test time
• works for a wide range of disparity metrics
• allows extracting the full disparity-accuracy
frontier
Disadvantages:
• requires re-training: the black box is called
10-20 times
www.aka.ms/FairlearnAI
49. Post
processing:
Picking a fair
threshold rule
Input:
• an existing (already trained) scoring model
• data set including sensitive feature
Output:
• the most accurate among all fair threshold rules (a
separate threshold for each subpopulation)
Advantages:
• simplicity
• no need to re-train the model
Disadvantages:
• requires sensitive feature at test-time
• doesn’t allow trade-offs between disparity and
accuracy
www.aka.ms/FairlearnAI
50. Unfairness
mitigation
Let’s see a demo!
www.aka.ms/FairlearnAI
Mitigating Disparities in
Ranking from Binary Data
- An example based on the Law
School Admissions Council's
National Longitudinal Bar
Passage Study
51. Article:
Machine Fairness
How to assess AI system’s fairness
and mitigate any observed
unfairness issues
www.aka.ms/Fairlearn-build2020
52. Mehrnoosh Sameki
Senior Product Manager
Microsoft Azure ML
Minsoo Thigpen
Product Manager
Microsoft Azure ML
Ehi Nosakhare
Machine Learning Scientist
Microsoft AI Acceleration and
Development Program
