Dr. Nabarun Dasgupta presents to the National Board of Medical Examiners in Philadelphia

1. Can Unstructured Data Be Good For Your
Health?
Nabarun Dasgupta
Chief Science Officer
Epidemico, Inc.
NBME Philadelphia
July 13, 2015
Slides available @epidemico

2. Traditional
Disease Reporting
Public1
Healthcare Workers, Electronic Health Records2
Laboratories
3
Ministries of Health4
World Bodies (WHO, EMA)5

3. Public1
Healthcare Workers, Electronic Health Records2
Laboratories
3
Ministries of Health4
World Bodies (WHO, EMA)5

4. Digital
Disease
Detection
Public
Healthcare Workers
Electronic Health
Records
Laboratories
Ministries of Health World Bodies
(WHO, EMA)

6. Digital disease detection picks up early signals
Traditional public health monitoring

7. Distribution of Disease

11. 200,000+
web sites
171public & private
sources
15languages
every hour, 24/7

12. Typhoid cases in Mufulira have
reached 2, 227 with health authorities
calling for increased efforts to prevent
new infections in Mupambe Township.
2, 227
#
Typhoid Mufulira
Mupambe Township

13. Natural language processing
EACH POST IS ANALYZED FOR KEY INFORMATION
Case & death
counts#
255
species
300
Diseases
40,000
locations

14. Alerts are categorized using
more than 19 million phrases
96%
accuracy

16. Focus on Liberia

17. Satellite Mapping Of Interventions
Monrovia, Liberia

19. Brazil World Cup 2014
Alerts from participating countries and hosting cities
19

21. In northern Nigeria, religious and political leaders, led by the chairman of the
Supreme Council for Sharia in Nigeria, Datti Ahmed, boycotted the polio vaccine
in 2003, claiming that the oral polio vaccine was contaminated with HIV and
could also cause sterilisation in those vaccinated, fuelling widespread public
distrust.

22. Monitoring Vaccine Sentiment

23. Black Markets

24. Sonrick Hansen et al. PLoS One. 2012

32. 32

33. SOCIAL
The WEB-RADR project is supported by the Innovative Medicines Initiative Joint Undertaking (IMI JU) under grant agreement n° 115632,
resources of which are composed of financial contribution from the European Union's Seventh Framework Programme (FP7/2007-2013)
and EFPIA companies’ in kind contribution.

34. Medicine Use
Negative Reaction
Suspected Problem
Require medical attention
Life threatening
Reported
Verified
Patients don't report bad reactions
because they don't know they should.
We can amplify their voices by meeting patients
where they are at and filtering out noise.

35. Patients are speaking
Shouldn’t we be listening?
Beyond informing the public about the
importance of adverse event reporting, we
can take steps to encourage patients to
report.
01 02 03
Meet patients
where they’re at
Protect their
privacy
Give them
easy-to-use
tools

36. Data flow for
detecting adverse
drug reactions in
social media data.
36
DATA PROCESSING
Acquire
Filter
Curate
Causation
Describ
e
Collect social
media data
Detect
events,
translate
vernacular
Reduce
false
positives
Descriptive
statistics
Synthesis
of
information

37. SOCIAL
“posts with resemblance to adverse events”
Proto-AE

38. 50 million posts that
mention a medical
product have been
collected. The classifier
has been trained with
360,000 manually coded
posts.
38
AUTOMATED FILTERING
68%
Research-ready data available
daily.
PRECISION
[+ predictive value]
88%
RECALL
[sensitivity]
Bayesian classifier
identifies posts with
resemblance to AEs
Dictionary translates
vernacular to
MedDRA
Multiple copies of posts
are consolidated and
PII removed
01
02
03

39. Vernacular-to-Regulatory
Translation
39

40. Spelling VariationsCombined or invented wordsImplied phrasesInternet Vernacular Translation
lost their eyesight
seeing weird color
seeing weird colour
doublevision
uldn’t see
double vision
blind
googley eyed
blurry vision
changes in viscross eyed
seeing weird
vision change
blindness
cross visionvisual snow
googly eyed
seeing doub
making me eat like a mouse
anorexic
st appetite
#notevenhungry
appetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
killed my apetite
miss feeling hung
killed my appetite
can’t eat
lost apetite
lost my appetite
no appetitey
lack of apetite
stomach small
lost teh appetite
never hnever want to eat
cant eat
coucrosseyed
blurry
apetite surpressed
killed my apetite
lost apetite
lack of apetite
lost teh appetite
Typos
seeing weird color
seeing weird colour
googley eyed
googly eyed
#notevenhungry
no appetitey
making me eat like a mouse
never want to eat

41. Visual impairment
MedDRA 10047571
Decreased appetite
MedDRA 10061428
making me eat like a mouse
anorexic
st appetite
#notevenhungry
appetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
miss feeling hung
killed my appetite
can’t eat
lost apetite
lost my appetite
no appetitey
lack of apetite
stomach small
lost teh appetite
never hnever want to eat
cant eat
lost their eyesight
seeing weird color
seeing weird colour
doublevision
uldn’t see
double vision
blind
googley eyed
blurry vision
changes in viscross eyed
seeing weird
vision change
blindness
cross visionvisual snow
googly eyed
seeing doub
coucrosseyed
blurry
killed my apetite

42. • Victoza® - liraglutide [rDNA origin] injection
• Diabetes indication
• UK, US, IN, CN, EU, JP, DE, DK, NL, IE, SE, FR, MY, SG
• Public Social Media (FB & Twitter)
• Jan 1, 2014 to March 9, 2015
• English language only
• Compare to US, UK and Canada health authorities data
42
• DESCRIPTIVE METHODS

43. 43
OVERALL RESULTS
10,433
Total number
of public
mentions in
Facebook or
Twitter.
ACQUIRE
368
Total number
of Proto-
AEs. 3.5% of
mentions are
Proto-AEs.
FILTER
158
False
positives
removed by
manual
curation
30%
CURATE
699
Drug-event
pairs at the
MedDRA
preferred
term level
DESCRIBE

44. Victoza only
44
Descriptive Analysis

45. 45
MENTIONS & PROTO-AEs
6360 4098
29 411
TWITTER
All mentions
FACEBOOK
All mentions
TWITTER
Proto-AEs
FACEBOOK
Proto-AEs
01
02
03
04

46. 46
COMPARISON SOCIAL & SPONTANEOUS

47. 47
SOCIAL MEDIA PREFERRED TERMS (>2)

48. 48
QUANTIFYING EVENTS

49. 49
IN THEIR OWN WORDS
Typical
Proto-AEs
& Patient
Benefits
@user I just restarted
taking victoza which made
me so sick to my stomach
till my body got used to it.
Aside from the nausea
(worse today) and
occasional bouts of
hypoglycemia (better
today), four days on
Victoza and down 5.3 lbs!
Amazing!
Twitter, May 3, 2014
Facebook, December 9, 2014

50. 50
IN THEIR OWN WORDS
Product
Quality
Issues
Anyone else having issues
with plugger jamming on
their flex pen of victoza?
The only difference I see is
the new needles . I had to
stick myself 4 times today
to get almost a full dose.
Frustration. Doesn't seem
related to where I am into
pen, first dose, midway ,
last. Third pen this has
happened on in a row
Facebook, December 9, 2014

51. 51
IN THEIR OWN WORDS
Diversion &
Re-Sale
I have 2 boxes of victoza
insulin pens if interested
inbox for more information
Facebook, February 3, 2015
Please, if anybody is on
Victoza for diabetes or insulin
resistance < i ordered too
much stock and it is non
returnable being a fridge item
> please pop in at [redacted]
pharmacy .....will organise a
good price to compensate for
my blunder.
Facebook, February 27, 2015

52. 52
IN THEIR OWN WORDS
Product
Pricing
Just bought 4 Victoza pens
from my pharmacy (CVS).
$800 out of pocket. Uggg. :(
Facebook, February 6, 2015

53. 53
IN THEIR OWN WORDS
Off-label
Use
Had my first injection of
Victoza. So if it doesn't kill
me from one of side effects
my sugar should stabilize.
One side effect is puzzling.
"For type 2 Diabetics only.
Not intended for type 1
diabetics". I'm the poster
child for type 1 Diabetics.
Facebook, February 6, 2015

54. 54
7.7%
Posts with HSI
“I've been on Victoza for a total of 17
days now for pre-diabetes. After the 5th
day the nausea hit and it's horrible. I'm
afraid to eat anything and the smell of
fried foods kill me. I started on 0.6 for 7
days and now I'm up to 1.2.....when
does it get better? My Dr
recommended that I go back to 0.6 for
a couple weeks to see if it helps. I feel
nauseous, bloated, stomach hurts,
dizzy at times......I have lost weight
(which for me is a good thing ) but it's
not good enough to keep feeling so
horrible. Anyone have any suggestions
for the nausea and bloatng?”
Facebook, January 23, 2015
HEALTH SYSTEM INTERACTION

55. 2 4 2 10
1
2
3
4
5
Pancreatitis Hypersensitivity Angioedema Haemorrhage
The EMA’s list of
Important Medical
Events (IME) can be
used to identify posts of
greatest medical
consequence
55
IMPORTANT MEDICAL EVENTS
2.0%
Posts with IME

56. 2
23
21
158
Twitter
United Kingdom Spontaneous Reports
United States Spontaneous Reports
(w/o multiplicate reduction)
Facebook
56
REPORTING BURDEN PER MONTH
10
Canada Spontaneous Reports
Twitter & Facebook: Jan ‘14 – Mar ’15
MHRA Drug Analysis Print: Jun ‘09 – Mar ‘15
FDA FAERS: Oct ‘12 – Mar ‘14
Health Canada: Aug ‘10 – Dec ‘14

57. SOCIAL
demo.medwatcher.org

58. 58
CLINICAL TRIAL DISCUSSION
I saw some discussion about the clinical trial for ####### but
don't think I've seen my question asked before. I started the
trial a little over three months ago and have seen zero
improvement...
…I am due for another appointment tomorrow and am still
feeling intense frustration over the lack of improvement…I've
been sorely tempted to give up and try something else…

59. 59
Illustration: Curt Merlo, The Village Voice

60. 60

61. 61

62. 62

63. StreetRx.com
StreetRx (streetrx.com) is a collection of databases, websites, and citizen reports enabling real-time collection,
organization, and display of street price data on diverted pharmaceutical controlled substances. Based on
principles of crowdsourcing for public health surveillance, the site allows users to anonymously submit prices
they paid or heard were paid for diverted prescription drugs.

65. External Referrals
Find Addiction Treatment
SAMSA, ADIN, DroguesInfoService.fr,
L’ANAPAA, DBDD
Overdose Prevention
Medication safety sites, HRC, naloxone
Safe Storage, Disposal & Takeback
FDA, Health Products Stewardship
Association
Safe Pain Management & Advocacy
Canadian Pain Coalition,
PainAustralia.org, RBK Stuttgart
Harm Reduction
HRC, EuroHRN
Public health, pain patient advocacy, proper disposal

66. Cross-Validation of Black Market Prices
66
Dasgupta et al. Journal of Medical Internet Research. 2013

67. Dasgupta et al. Journal of Medical Internet Research. 2013

68. 68

69. Can the Black Market Discriminate betweenADFs?Do different deterrence strategies and platforms result in discernible street price differences?
STEP1
STEP3
STEP2
STEP4
USA Data in USD
Nov 1, 2010 through Mar 31, 2015
Solid oral formulations
Compare 14 ADF to
unprotected street price per
milligram, within active
ingredient
Geometric means and 95% CI
N = 49,995
Non-parametric Wilcoxan
rank-sum (Mann-Whitney)
tests
(Partial-) Antagonists: Subutex vs. Suboxone
Prodrugs: morphine vs. codeine; Vyvanse vs. amphetamines
Phyical barriers: Zohydro vs. Hysingla ER vs. IR hydrocodone; OxyContin OP vs. OC;
Opana ER reformulation vs original; Xartemis XR vs. IR oxycodone;
Exalgo vs. IR hydromorphone; Concerta vs. Ritalin LA vs. methylphenidate, Nucynta ER vs. IR tapentadol
Not enough data to evaluate yet:
Embeda,Targin(iq), Oxytrex, Oxecta, Quillivant XR, Talwin NX, Amrix

70. Percent Discount for ADF Versions
Putative abuse-deterrent formulation versus unprotected formulations with same active ingredient, using statistical comparison that is less influenced by outliers.
Physical Barrier Prodrug(Partial-)
Antagonist
Concerta
Exalgo
OpanaER
OxyContinOP
Zohydro
Vyvanse
codeine
Suboxone
tablet
XartemisXR
p<.01
z -38
p<.01
z -38
p<.01
z -3.0
p<.01
z -3.7
p=.78
z 0.3
p<.01
z -8.6
p<.01
z -13
p=.34
z -0.9
p=.24
z -1.2
TBD
?
OROSOROSINTAC INTAC PolyOx
RitalinLA
p<.01
z -3.9
SODAS
NucyntaER
p=.10
z -1.6
INTAC
HysinglaER
p=0.46
z -0.7
INTAC INTAC

71. Physician/Patient forum
 Safety concerns over drug products are
frequently raised in some physician or
patient forums
Doctor ordered an IV with 40mg Omeprazole mixed with 5% Glucose
solution. The solution turned black after mixing the two…
- AI AI YI: http://www.iiyi.com/med/

72. Weibo
Worm found in Amoxicillin
capsule packaged in blister
package

73. Weibo
Consumer purchased
extremely cheap ointment sold
under brand name Bactroban
but were not manufactured by
GSK.
GSK’s Bactroban

74. 74
Food safety signals from Twitter
related to Target
Booz Allen Hamilton and Client proprietary and business confidential
Pet food

75. Supply chain analysis
In compliance Pending actions
Imports and exports
Critical vulnerability
FROM: Imagined Food Co.Ltd
ADDRESS: 188 Imagined Lane, Boston, MA, 02116
CONTACT: Alpha M. Bravo, (617) 893-xxxx
MAIN SERVICE: Processed meat, bacon, etc.
TO: Madeup Logistics Co.Ltd
ADDRESS: 61 Foodstuff Rd, Newton, MA, 02459
CONTACT: Charlie N. Delta, (617) 112-xxxx
MAIN SERVICE: Cold chain logistics
TRANSPORTATION METHOD: Truck (cold chain equipped)
TRANSPORATION DISTANCE: 5.1 miles
FOOD CATEGORY: Processed meat
AVERAGE VOLUME OF TRANSACTION: 10 ton/day
RISK SCORE FOR CONTAMINATION: 1.0 (1-10 Scale)

76. 76
Source: https://blog.fitbit.com/heart-racing-moments-from-the-big-game/

77. 77
QUESTIONS?
epidemico.com
@epidemico
demo.medwatcher.org
@med_watcher
HealthMap.org
nabarun@epidemico.com