Fernandez-Luque L, Karlsen R, Melton GB
HealthTrust: A Social Network Approach for Retrieving Online Health Videos
J Med Internet Res 2012;14(1):e22
http://www.jmir.org/2012/1/e22/
Kolkata Call Girls Shobhabazar 💯Call Us 🔝 8005736733 🔝 💃 Top Class Call Gir...
HealthTrust: A PhD Dissertation on the Retrieval of Trustworthy Health Social Media
1. HealthTrust: A PhD Dissertation on the
Retrieval of Trustworthy Health Social Media
PhD Defence, 24th October 2014
“How can computing techniques
support the retrieval of trustworthy
health social media?“
Luis Fernandez Luque (@luisluque), eHealth
Researcher, Norut Tromsø (Norway)
2. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
2
3. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
3
4. “Do not search: Twin-to-Twin
transfusion syndrome”
Part 1- A personal example 4
7. Results
• Hospitals: out-dated focused on worse case
scenarios
• Research literature: focused on complicated
cases
• Social Media of Patients: obituaries
• Social Media of Hospitals: to the point accurate
information
Part 1- A personal example 7
8. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
8
11. Main open questions
• How to find the “good” content?
• What is “good” content?
• Why sometimes “Google” is failing?
• Is it just content? Is it content-based
communities?
• How is bad content disseminated or filtered? And
good content?
Part 2 - Introduction 11
12. Research Gaps
• Lack of knowledge about health social media:
motivations, dynamics, harmful content.
• Lack of information about technical solutions
for finding health social media: new
techniques were emerging for retrieving social
media, but none specialized in the health context
• Lack of trust-based approaches for retrieving
health social media: previous online information
retrieval tools focused on metadata and not
leverage in trust from online health communities.
Part 2 - Introduction 12
13. Research Questions
How can computing techniques support the retrieval of trustworthy
health social media?
•RQ1) What are the characteristics of health social videos?
•RQ2) Are there technical solutions for modelling health social media?
•RQ3) How can Social Network Analysis be used to extract information
about the characteristics of health social media?
•RQ4) Can trust-based metrics improve the retrieval of social videos
about diabetes?
Part 2 - Introduction 13
16. Papers I
RQ1.Paper 1: Gómez-Zúñiga B, Fernandez-Luque L, Pousada M, Hernández-Encuentra
E, Armayones M. ePatients on YouTube: Analysis of Four Experiences From the Patients'
Perspective. Med 2.0 2012;1(1):e1
RQ1.Paper 2: Fernandez-Luque L, Elahi N, Grajales FJ 3rd. An analysis of personal
medical information disclosed in YouTube videos created by patients with multiple
sclerosis. Stud Health Technol Inform. 2009;150:292-6.
RQ1.Paper 3: S Konstantinidis, L Fernandez-Luque, P Bamidis, R Karlsen. The Role of
Taxonomies in Social Media and the Semantic Web for Health Education. Methods Inf
Med 2013; 52
RQ1.Paper 4: E Gabarron, L Fernandez-Luque, M Armayones, A YS
Lau. Identifying measures used for assessing quality of YouTube videos with patient
health information: A Review of Current Literature. Interact J Med Res 2013;2(1):
RQ1.Paper 5: Syed-Abdul S, Fernandez-Luque L, Jian WS, Li YC, Crain S, Hsu MH,
Wang YC, Khandregzen D, Chuluunbaatar E, Nguyen PA, Liou DM. Misleading health-related
information promoted through video-based social media: anorexia on YouTube. J
Med Internet Res. 2013 Feb 13;15(2):e30.
Part 2 - Introduction 16
17. Papers II
RQ2.Paper 1: Fernandez-Luque L, Karlsen R, Bonander J. Review of extracting
information from the Social Web for health personalization. J Med Internet Res. 2011 Jan
28;13(1):e15. doi: 10.2196/jmir.1432.
RQ3.Paper 1: Yom-Tov E, Fernandez-Luque L, Weber I, Crain SP. Pro-anorexia and pro-recovery
photo sharing: a tale of two warring tribes. J Med Internet Res. 2012 Nov
7;14(6):e151. doi: 10.2196/jmir.2239.
RQ3.Paper 2: Chomutare T, Arsand E, Fernandez-Luque L, Lauritzen J, Hartvigsen G.
Inferring community structure in healthcare forums. An empirical study. Methods Inf Med.
2013;52(2):160-7. Epub 2013 Feb 8.
RQ4.Paper 1: Fernandez-Luque L, Karlsen R, Melton GB. HealthTrust: Trust-based
Retrieval of YouTube's Diabetes Channels, 2011, 20th ACM international conference on
Information and knowledge management.
RQ4.Paper 2: Fernandez-Luque L, Karlsen R, Melton GB. HealthTrust: A Social Network
Approach for Retrieving Online Health Videos. J Med Internet Res. 2012 Jan
31;14(1):e22.
Part 2 - Introduction 17
18. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
18
19. RQ1: What are the characteristics of health
social videos?
• RQ1.1: Does the online community influence the
motivation of people with chronic conditions to publish
videos about their health?
• RQ1.2: Do health videos contain relevant medical
vocabulary in their textual metadata?
• RQ1.3: What are the quality features of online health
videos?
• RQ1.4: Do misleading and informative online videos on
the topic of anorexia have different characteristics?
Part 3 - RQ1 Health Videos 19
20. RQ1.Study 1: Characteristics of metadata in
health social videos
L. Fernandez-Luque et al. / An Analysis 294 of Personal Medical Information
REALLY have MS). Overall, 70 comments (22%) contained personal health
information concerning their creators or a third party (e.g., relatives).
The comments with personal health information (PHI) were further stratified. As
Figure 2 denotes, almost half of the comments contained information about
medications (73%, n=51). Comments about symptoms (50%, n=35) and diagnoses
(39%, n=27) were also prevalent. In one case, the information disclosed was the PHI of
a third party:
S Konstantinidis, L Fernandez-Luque, P Bamidis, R Karlsen.
The Role of Taxonomies in Social Media and the Semantic Web
for Health Education. Methods Inf Med 2013; 52
Figure 1. Total number of comments classified into the main categories
Fernandez-Luque L, Elahi N, Grajales FJ 3rd. An analysis
of personal medical information disclosed in YouTube videos
created by patients with multiple sclerosis. Stud Health
Technol Inform. 2009;150:292-6.
I have been watching your videos since my daughter was diagnosed with
MS on 28-12-07. She was diagnosed with an aggressive form of MS.
Betaferon caused liver problems in a very short time. She has now had 4
infusions of tysabri and now feels she is well enough to try and go back
to work. Tysabri only became available in Aust on July 1/08. This drug
has given her hope that she still has a future to look forward to as she is
26 yrs old. Your improvement since your first dose gave us all hope.
Part 3 - RQ1 Health Videos 20
21. RQ1.Study 2: What is quality of health social
videos?
E Gabarron, L Fernandez-Luque, M Armayones, A YS Lau. Identifying measures used for assessing quality of
YouTube videos with patient health information: A Review of Current Literature. Interact J Med Res 2013;2(1):e6
22. RQ1.Study 3: Motivations of patients sharing
videos
...And part of why I started my blog in the first
place was because, even though I’ve lived
with diabetes for such a long time and I didn’t
known (sic) anyone else who had it, and I
literally felt like the only diabetic on the planet.
[KS]
I met so many people from all over the world that I
would never have been able to talk to, before the
Internet of course, and then now, with the MS
community on YouTube it’s incredible. [VB]
Gómez-Zúñiga B, Fernandez-Luque L, Pousada M, Hernández-Encuentra E, Armayones M. ePatients
on YouTube: Analysis of Four Experiences From the Patients' Perspective. Med 2.0 2012;1(1):e1
23. RQ1.Study 4: Study about pro- and
anti- anorexia videos
Syed-Abdul S, Fernandez-Luque L, Jian WS, Li YC, et al. Misleading health-related information promoted
through video-based social media: anorexia on YouTube. J Med Internet Res. 2013 Feb 13;15(2):e30.
24. RQ1.Study 4: Study about pro- and
anti- anorexia videos
Syed-Abdul S, Fernandez-Luque L, Jian WS, Li YC, et al. Misleading health-related information promoted
through video-based social media: anorexia on YouTube. J Med Internet Res. 2013 Feb 13;15(2):e30.
25. RQ1: Characterizing Health Social Media
Key Findings
• Social interaction is one of the main driving forces behind
those publishing videos about their health.
• Textual metadata can be of very heterogeneous quality,
but still contains a lot of relevant health information for
modeling.
• The quality of health videos is a multidimensional
concept. Reliability of the content and its provider are
very important quality criteria according the literature.
Part 3 - RQ1 Health Videos 25
26. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
26
27. Modeling Health Social Media
RQ2: Are there technical solutions
for modeling health social media?
Part 4- RQ2 Modeling Health Social Media 27
28. RQ2.Study 1: Review on techniques for
modeling health social media
Fernandez-Luque L, Karlsen R, Bonander J. Review of extracting information from the Social Web for
health personalization. J Med Internet Res. 2011 Jan 28;13(1):e15. doi: 10.2196/jmir.1432.
29. RQ2: Extracting Information from
Health Social Media
Key Findings
• Most technical solutions for modeling social
media will have shortcomings in the health
domain due to text analysis complexities.
• Questions about privacy issues.
• Link and Social Network Analysis is promising but
has not been studied in detail in the health
domain.
Fernandez-Luque L, Karlsen R, Bonander J. Review of extracting information from the Social Web for
health personalization. J Med Internet Res. 2011 Jan 28;13(1):e15. doi: 10.2196/jmir.1432.
30. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
30
31. RQ3.Study 1: Structure of Pro-anorexia
& pro-recovery groups in Flickr
Yom-Tov E, Fernandez-Luque L, Weber I, Crain SP Pro-Anorexia and Pro-Recovery Photo
Sharing: A Tale of Two Warring Tribes J Med Internet Res 2012;14(6):e151
32. RQ3.Study 1: Structure of Pro-anorexia
& pro-recovery groups in Flickr
Figure 4. Network graphs according to four connection
types (from top left, clockwise): Contacts, Favorites,
Tags, Comments.
Yom-Tov E, Fernandez-Luque L, Weber I, Crain SP Pro-Anorexia and Pro-Recovery Photo
Sharing: A Tale of Two Warring Tribes J Med Internet Res 2012;14(6):e151
33. RQ3.Study 2: Structure of diabetes
communities
Chomutare T, Arsand E, Fernandez-Luque L, Lauritzen J, Hartvigsen G. Inferring community structure in
healthcare forums. An empirical study. Methods Inf Med. 2013;52(2):160-7. Epub 2013 Feb 8.
34. RQ3: Social Network Analysis for
characterizing Health Social Media.
Key Findings
• On a photo-sharing site, the best predictors of users
belonging to the sub-community promoting anorexia are
social network metrics. Tag-based classification was less
accurate.
• Most centric members on online diabetes communities
had longer experience living with the disease.
Part 5 - SNA Health 34
Communities
35. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
35
36. Online search of health diabetes videos
HealthTrust - a trust-based
metric for retrieving diabetes
videos
36
37. Online Search: PageRank & TKC
effect (Tightly Knit Community)
Sergey Brin and Lawrence Page. 1998. The
anatomy of a large-scale hypertextual Web
search engine. Comput. Netw. ISDN Syst.
30, 1-7 (April 1998), 107-117.
DOI=10.1016/S0169-7552(98)00110-X
http://dx.doi.org/10.1016/S0169-
7552(98)00110-X
37
http://en.wikipedia.org/wiki/PageRank
R. Lempel and S. Moran. 2000. The stochastic
approach for link-structure analysis (SALSA)
and the TKC effect. Comput. Netw. 33, 1-6
(June 2000), 387-401. DOI=10.1016/S1389-
1286(00)00034-7
http://dx.doi.org/10.1016/S1389-
1286(00)00034-7
38. RQ4: HealthTrust - a trust-based metric for retrieving
diabetes videos
Fernandez-Luque L, Karlsen R, Melton GB. HealthTrust: A Social Network Approach for
Retrieving Online Health Videos. J Med Internet Res. 2012 Jan 31;14(1):e22.
39. RQ4: HealthTrust - a trust-based metric for retrieving
diabetes videos
YouTube’s API
Videos, Tags,
Users
Fernandez-Luque L, Karlsen R, Melton GB. HealthTrust: A Social Network Approach for
Retrieving Online Health Videos. J Med Internet Res. 2012 Jan 31;14(1):e22.
40. RQ4: HealthTrust - a trust-based metric for retrieving
diabetes videos
Fernandez-Luque L, Karlsen R, Melton GB. HealthTrust: A Social Network Approach for
Retrieving Online Health Videos. J Med Internet Res. 2012 Jan 31;14(1):e22.
41. RQ4: HealthTrust - a trust-based metric
for retrieving diabetes videos
Fernandez-Luque L, Karlsen R, Melton GB. HealthTrust: A Social Network Approach for
Retrieving Online Health Videos. J Med Internet Res. 2012 Jan 31;14(1):e22.
42. RQ4: HealthTrust - a trust-based metric
for retrieving diabetes videos
Key Findings
• In diabetes online communities the most
reputable members are those with more
experience with diabetes.
• The HealthTrust metric based on Social Network
Analysis to infer quality of health videos performs
well for filtering misleading content compared to
YouTube searches.
Fernandez-Luque L, Karlsen R, Melton GB. HealthTrust: A Social Network Approach for
Retrieving Online Health Videos. J Med Internet Res. 2012 Jan 31;14(1):e22.
43. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
43
44. Claimed contributions
• C1: increase in the knowledge about health
social videos.
– Published results have been cited more 400 times since 2009.
– Startups and journalists have requested interviews to share my
knowledge. Also keynotes in Taiwan and Norway.
• C2: increased knowledge on the challenges
related to model health social media
– The RQ2.P1 is the first paper that systematically reviews the
challenges of modeling health social media. It has been cited 25
times since 2011.
Part 7 - Discussion 44
45. Claimed contributions
• C3: social network analysis of online health
communities
– Research in this PhD has increased the understanding
of the social dynamics in health related communities
(e.g. anorexia, diabetes).
• C4: social network analysis of health social
media to infer quality
– The algorithm HealthTrust is the first one focused on the use of
social network analysis to retrieve trustworthy health videos for
patients.
– The algorithm has been designed, tested and evaluated.
Part 7 - Discussion 45
46. Discussion & Limitations
• Social network features of health communities can provide
clues regarding quality and trustworthiness of content.
• Each platform and disease is different. Evaluation was
online done in Diabetes in a offline experiment. Can we
generalize HealthTrust?
• Social media is becoming more heterogeneous (Twitter,
YouTube, etc.), but HealthTrust has been tested only with
one type of content (i.e. videos).
Part 7 - Discussion 46
47. Agenda
1 2 Modelling Health
3 4 Why? A personal
example
Social Media
Health Social
Media & Online
Introduction &
Overview
Videos
7 8 Social Network
Analysis of Health
Communities
6 Discussion Future work
5 HealthTrust and
Information
Retrieval
47
48. Future Work
• Creation of a portal (spin-off) to access many
users for better experimentation and evaluation.
• To expand our knowledge about why misleading
and harmful content is highly visible and ranked.
– Better strategies for disseminating good health social
media.
– Better information retrieval tools to help finding
content.
• Study case: the visibility of the online anti-vaccination
movement might be already killing
children.
Part 8 – Future work 48
Searching was not a easy task: problems with research literature, webs from hospitals, and also social media (specially patients).
After 1 whole (and very tense) afternoon I found one video explaining everything (I wanted to know) in 3 minutes
By the time this PhD started Health Social Media was ”new” in the health domain. Today it is an overwhelimgn reality
Not everything is good, but most important to find the good is very hard
1) There is a huge amount of health content in social media channels. Thousands of videos from hospitals, health authorities, patients, etc. Sadly, that content is not always on top when searching-
During the first week of December 2008 we searched for YouTube users who had published at least three videos in English about living with multiple sclerosis. In total we found 27 of such users. Using a crawler I developed we extracted a total of 769 videos, 7,047 comments generated by 2,426 users. Using a random method we selected 25 of those videos with their 557 comments. Comments that were not in English or posted in non health-related videos were excluded. A final set of 320 comments were analyzed and classified as follows:
Personal health information: comments containing personal health experiences such as diagnosis, symptoms, etc.
Video discussion: comments with discussions about the videos (e.g. adding information about the videos).
Appreciation: appreciations from the commenters towards the video author.
Criticism: complains about the quality of the video or any other aspect.
Unrelated: comments that are not covered in any of the other categories (e.g. comments about the haircut style)
YOUTUBE MS: 27 of MS ePatients with 769 videos7,047 comments generated by 2,426 users. Random selection of 25 videos we analyzed 320 comments.
YOUTUBE TAG: 4,307 videos (64,367 tags) from 500 US Hospital channels..BioPortal REST services were used within our portal to match SNOMED CT terms. The average percentage of YouTube tags that were expressed using SNOMED CT terms was about 22.5%
RESULTS: 456 abstracts were extracted and 13 papers. (1) quality of content in 10/13 (77%), (2) view count in 9/13 (69%), (3) health professional opinion in 8/13 (62%), (4) adequate length or duration in 6/13 (46%), (5) public ratings in 5/13 (39%), (6) adequate title, tags, and description in 5/13 (39%), (7) good description or a comprehensive narrative in 4/13 (31%), (8) evidence-based practices included in video in 4/13 (31%), (9) suitability as a teaching tool in 4/13 (31%), (10) technical quality in 4/13 (31%), (11) credentials provided in video in 4/13 (31%), (12) enough amount of content to identify its objective in 3/13 (23%), and (13) viewership share in 2/13 (15%).
In this qualitative study, we performed an analysis of the videos created by 4 patients about their self-reported motivations and challenges they face as YouTube users. First, two judges compared the transcriptions and decided the exact wording when confusing content was found. Second, two judges categorized the content of the videos to identify the major themes.
RESULTS: four main categories emerged: (1) the origin or cause for making the first video, (2) the objectives that they achieve by continuing to make videos, (3) the perception of community, and (4) the negative consequences of the experience.
Three doctors reviewed 140 videos with approximately 11 hours of video content, classifying them as informative, pro-anorexia, or others. Statistical analisys of the top 40 most viewed in each category
The interrater agreement of classification was moderate (Fleiss' kappa=0.5), with 29.3% (n=41) being rated as pro-anorexia, 55.7% (n=78) as informative, and 15.0% (n=21) as others. Pro-anorexia videos were favored 3 times more than informative videos (odds ratio [OR] 3.3, 95% CI 3.3-3.4, P<.001).
Three doctors reviewed 140 videos with approximately 11 hours of video content, classifying them as informative, pro-anorexia, or others. Statistical analisys of the top 40 most viewed in each category
The interrater agreement of classification was moderate (Fleiss' kappa=0.5), with 29.3% (n=41) being rated as pro-anorexia, 55.7% (n=78) as informative, and 15.0% (n=21) as others. Pro-anorexia videos were favored 3 times more than informative videos (odds ratio [OR] 3.3, 95% CI 3.3-3.4, P<.001).
1) There is a huge amount of health content in social media channels. Thousands of videos from hospitals, health authorities, patients, etc. Sadly, that content is not always on top when searching-
identify relevant research literature that addressed the following aspects of health personalization in the Social Web: (1) studies about the disclosure of health information in the Social Web, (2) techniques to extract that information, and (3) examples of applications. Major scientific databases in computer science (eg, ACM Digital Library) and biomedicine (eg, PubMed) were searched. In addition, we searched through the references of the selected papers, contributions to conferences, and nonresearch literature (eg, websites, books, technical reports). The background section provides an overview of the different research areas where the search was performed.
The multidisciplinary team of authors performed the selection and analysis of the relevant articles. Their backgrounds cover the different domains of the review (eg, information retrieval, computer science, health informatics, and public health). The different studies were analyzed to understand the implications for health personalization, including technical and socioethical aspects.
Methods: The extraction of pro-anorexia and pro-recovery photographs Flickr pertaining to 242,710 photos from 491 users
The ROC using the comments or contacts was 0.74, whereas the area using favorites was 0.53 and 0.52 using the tags network
The forums have a total in excess of 140,000 registered users and over 1.6 million posts. Years of diagnosis was only available in 3 forums. We used datasets from five online diabetes forum. We use two well-known community detection algorithms that deal very well with large datasets; Greedy Optimization(GO) [27] and the Affinity Propagation(AP) [28] algorithms. GO is based on the Girvan-Newman algorithm; hierarchical agglomeration. We developed a web data extraction program in Python to crawl five diabetes forums on the Internet, comprising Spanish and English forums, and two of them were dedicated to juvenile diabetes.
Results: Results show that we can infer meaningful communities by observing forum interactions. Closely similar users tended to co-appear in the top communities, suggesting the discovered communities are intuitive. The number of years since diagnosis was a significant factor for cohesiveness in some diabetes communities. The figure illustrates the dominant star topology, and shows how the central nodes in the network have at least two years’ experience with diabetes. (78%) of the users who provided the data have been diagnosed less than two years ago (green). (78%) of the users who provided the data have been diagnosed less than two years ago (green)
Objectives: To explore approaches for extracting metrics about authoritativeness in online health communities and how these
metrics positively correlate with the quality of the content.
Methods: We designed a metric, called HealthTrust, that estimates the trustworthiness of social media content (eg, blog posts
or videos) in a health community. The HealthTrust metric calculates reputation in an online health community based on link
analysis. We used the metric to retrieve YouTube videos and channels about diabetes. In two different experiments, health
consumers provided 427 ratings of 17 videos and professionals gave 162 ratings of 23 videos. In addition, two professionals
reviewed 30 diabetes channels.
In two different experiments, health consumers provided 427 ratings of 17 videos and professionals gave 162 ratings of 23 videos. In addition, two professionals reviewed 30 diabetes channels.
Of 20 potential channels, HealthTrust’s filtering allowed only 3 bad channels (15%) versus 8 (40%) on the
YouTube list.
When comparing video ratings from our reviewers, we found that HealthTrust achieved a positive and statistically significant correlation with professionals (Pearson r10= .65, P = .02) and a trend toward significance with health consumers (r7 = .65, P = .06) with videos on hemoglobinA1c, but it did not perform as well with diabetic foot videos.
Conclusions: The trust-based metric HealthTrust showed promising results when used to retrieve diabetes content from YouTube.
Our research indicates that social network analysis may be used to identify trustworthy social media in health communities.