This session was recorded in San Francisco on February 5th, 2019 and can be viewed here: https://youtu.be/LwcQo2gxxog Bio: Bharath Sudharsan is the Director of Data Science and Innovation at ArmadaHealth. He leads a team of data analysts who develop and implement AI tools that are at the heart of objective and data-driven specialty care referral process synonymous with ArmadaHealth. Bharath has also held positions at Fractal Analytics and Quanttus, Inc. and WellDoc, Inc. He is also the founder of Geetha, LLC, a provider of best in class healthcare analytics consultation including implementation of NLP and AI.

1. Emotional AI
Name Bharath Sudharsan
Title Emotional AI
Company ArmadaHealth
#EmotionalAI
#H2OWORLD

2. PreviousNext
2
Hello from ArmadaHealth and H20
Bharath Sudharsan
Director of Data Science
ArmadaHealth
Sudalai Raj Kumar
NLP Lead
H20
Ryan Kosiba
Data Scientist
ArmadaHealth
Shwetank Sonal
Data Scientist
ArmadaHealth
Shruti Padmanabhan
Data Scientist
ArmadaHealth
Andrew Corson
Software Engineer
ArmadaHealth

3. PreviousNext
3
What if AI could
understand
human
emotions?
And why?
1 It is no longer a science fiction
2 It is often associated with facial and
speech-based approaches
3 Text-based approach is more
mainstream, given we spend
most time typing than talking
Emotion AI
AI Understanding Human Emotions could have real and
measurable impact in Healthcare.

4. PreviousNext
4
Every Technology needs a compelling and meaningf ul
vehicle t o reach t ipping point

5. PreviousNext
5
What’s the
problem?
Consumers are not
equipped to navigate
the complex and
confusing healthcare
system.
Finding a good
physician solves
major problems..
CAN SAVE LIVES
Patient is
here
The right
doctor for
patient is here.
Specialty care access problem
30,000+ diseases
900,000 specialists
100s of subspecial es
Varying quality of training & outcomes
No transparency to the pa ent or referring
physician
Huge health systems: Who’s in-network?
3rdleading cause of death:
medical errors5
50%of consumers will receive
incorrect treatment in their
lifetime4
Adult patients receive an
incorrect diagnosis
each year2
12m
34%of consumers self-refer
to specialists3
$29b
Lost each year in
preventable waste1

6. PreviousNext
6
People used to ask only
other people for help
People started asking
technology for help
Technology can help
understand other’s
experiences and bring in the
right help
 Not objective enough
 Not convenient enough
 Not trustworthy
 Not descriptive/experiential enough
 Objective, trustworthy and
descriptive
When we need help… Patient Reported
Experience
Measures (PREMs)

7. PreviousNext
7
What’s the
Solution?
Objective approach
to leverage wisdom
of experts, AI and
people
Wisdom-based Approach

8. PreviousNext
8
What’s the
problem with
current
approaches?
100s of Reviews to sift
through
Keyword-based match is
highly inaccurate
Ratings are highly
subjective
Doesn’t answer
consumer’s key
questions

9. PreviousNext
9
Key Questions consumers have while reading reviews…..
Do other patients get better?#01
Outcomes, including ease of getting back to
normal quality of life, after a major surgery, is a
key focus.
Positive Negative
Treatment
Outcome
Is the physician and office staff friendly?#02
Patients and their family undergo a lot of pain,
anxiety and stress during an episode. It is
important for their care team to be friendly and
understanding.
Positive Negative
Attitude
Do they communicate treatment options clearly and
involve the patient in decision making?
#03
Shared decision making is key to better outcomes, and patient empowerment.
Positive Negative
Communication
Patient Reported Experience
Measures (PREMs)

10. PreviousNext
10
It’s a NLP approach
that gives a general
idea about the
positive, neutral, and
negative sentiment of
texts
01. It is important to understand the
context of each sentence
02. There are multiple layers of
meaning:
Rhetorical devices like sarcasm, irony,
and implied meaning can mislead
03. It is a hard challenge for
language technologies, and
achieving good results is much more
difficult
Sentiment Analysis

11. PreviousNext
11Example
Don't go to this doctor at all!! Uncaring, does not
explain to the patient. God knows if he really did see
what needed to be done on my knee. Did not say
anything before and after the operation. I had to go
through multiple surgeries to get back to being
normal.
Treatment Outcome Communication
Positive Negative
Vs.

12. PreviousNext
12
Key
Takeaways
01. Context matters a lot
02. Need to look beyond word-based
approaches
03. Aspects are important, to move
beyond overall sentiment scores

13. PreviousNext
13
Our Initial Model – Treatment Positive/Negative
1500 labeled rows
14,000 unlabeled reviews
Used Driverless AI NLP features and word2Vec
approach

14. PreviousNext
14
Our Other Models
Was unstable due to class imbalance
Used Driverless AI NLP features and word2Vec
approach
Physician communication Model Office Staff Attitude Model

15. PreviousNext
15
Language Models
What is a language model?
• Language model is a model to calculate probability distribution over sequences of
tokens in a (natural) language
• Make use of distributed representations, that is, low-dimension vector
representations of tokens that can mitigate the curse of dimension
How can it help?
• Because they can empower a very efficient way of learning, called transfer learning.
Transfer learning helps you to train, often unsupervised on a lot of data, then to
apply the pre-trained model to efficiently learn downstream tasks.
What are some examples?
• Word2vec is one example of transfer learning, a universal one.
• Researchers since then have been looking for even better way of transfer learning, the current
state-=of-the-art ones are ELMo, ULMFit, OpenAI Transformer, and BERT.

16. PreviousNext
16
Language Model – Architectures and
Approaches
Left to right
Right to left
Shallow
Bidirectional
And
deep
Architectures – LSTM Vs. Transformer Attention Approach

17. PreviousNext
17
Comparison

18. PreviousNext
18
BERT-based Model - Supervised
4000 labeled rows
14,000 unlabeled reviews
Models Results
Treatment Outcome 77% (vs DAI – 73%)
PhysicianAttitude 89.4%
Physician Communication 89%
Office Staff Attitude 83%
Office Staff Communication 84%
4000 labeled rows
1M+ Unlabeled domain-specific rows
Subjectivity Detection
Fake Reviews using Linguistic patterns and
models

19. PreviousNext
19
Next Frontier – Personality and Emotion-centric
analysis
Pleasantness Attention Sensitivity Aptitude
Ecstasy Vigilance Rage Admiration
Joy Anticipation Anger Trust
Serenity Interest Annoyance Acceptance
Pensiveness Distraction Apprehension Boredom
Sadness Surprise Fear Disgust
Grief Amazement Terror Loathing
A truly consumer-centric
empathy-based AI
Understanding the person behind the
patient

20. PreviousNext
20
A truly consumer-centric
empathy-based Healthcare experience using AI
Understanding the person behind the patient
Next Frontier in Healthcare

21. PreviousNext
21
Summary Summary
• Physician reviews is a compelling and impactful use case for sentiment
analysis
• Reviews need a both objective and descriptive approach – Expert-based and
AI-based
• Context is key to sentiment analysis
• Language models are better at understanding and preserving context
• BERT model, given its bidirectional approach, outperforms other models and
looks very promising
• Understands context better
• Aspect-based approaches add more meaning
• Emotion detection via text is the next major frontier for Patient Reported
Experience Measures (PREMs) and is key to a truly value-based and wisdom-
based care

22. PreviousNext
22