How do platforms enable to spread of misinformation about vaccines and other health concerns? And what could they be doing to stop it? In this IGNITE talk, Daniel Jones presents the examples of pro-eating disorder, pro-self harm, and anti-vaccine communities on social media. This talk presents the case for a public health emergency response approach to reducing the spread of forms of content on social media that can have widespread and harmful offline impacts. This talk delivered as part of the IGNITE Talks at BKC Luncheon Series event, May 7, 2019, at the Berkman Klein Center for Internet & Society at Harvard University. https://cyber.harvard.edu/events/2019-05-07/ignite-talks-bkc

1. Public Health & Content Moderation
1
Public Health
&
Content Moderation

2. Public Health & Content Moderation
2
Social Media for Health
A Tumblr experiment

3. Public Health & Content Moderation
3
4 Burritos & 1 Mile in 35 Minutes
A bad idea

4. Public Health & Content Moderation
4
creepallaandjenny.tumblr.com
Social Media for Health
Struggling online

5. Public Health & Content Moderation
5
Tumblr Health Communities
Pro-health and pro-self-harm overlap
Celebrating health
& emphasizing
recovery
Embracing disorder
& glorifying risky
behavior

6. Public Health & Content Moderation
6
Tumblr Responds
Reacting to Self-Harm

7. Public Health & Content Moderation
7
Pro-Self Harm on Social Media
https://www.nytimes.com/2002/06/23/
health/a-disturbing-growth-industry-web-
sites-that-espouse-anorexia.html
2002 2019

8. Public Health & Content Moderation
8
Public health
Areas for misinformation
Zika
Water fluoridation
Vaccines
EbolaCancer
Reproductive health
Abortion
Self-Harm
M
ental health
Weight loss
HIV

9. Public Health & Content Moderation
9
Yes No
States that have reported measles cases in 2019
As of April 27, 2019
https://www.cdc.gov/measles/cases-outbreaks.html

10. Public Health & Content Moderation
10
Misinfodemics
A spread of a particular health outcome or disease facilitated by viral misinformation

11. Public Health & Content Moderation
11
Counterspeech
Countering Misinformation Through Humor

12. Public Health & Content Moderation
12
Blocking Content
De-surfacing or Banning Certain Forms of Conversation
Seth Wenig, AP

13. Public Health & Content Moderation
13
Down the Rabbit Hole
Succumbing to Misinformation on Social Media
Normalization
Misinformation
Extremism
TrustinExistingInstitutions
Engagement with Misinformation

14. Public Health & Content Moderation
14
Normalization
Down the Rabbit Hole
Succumbing to Misinformation on Social Media
Vaccine Hesitant
“Is this safe?”
Vaccine Resistant
“Unvaccinated and Proud”
Vaccine Extremist
“No one should vaccinate”
Normalization
Misinformation
Extremism
TrustinExistingInstitutions
Engagement with Misinformation

15. Public Health & Content Moderation
15
Vaccine Sentiment
Many mindsets, all wrong
Trust
Alternatives
Safety
Conspiracy
Hoffman, et al
https://www.sciencedirect.com/science/article/pii/S0264410X19303032?via%3Dihub#f0005

16. Public Health & Content Moderation
16
Hoffman, et al
https://www.sciencedirect.com/science/article/pii/S0264410X19303032?via%3Dihub#f0005
Vaccine Sentiment
Many mindsets, all wrong

17. Public Health & Content Moderation
17
Beyond Blocking
De-surfacing Harmful Content
1. Engage the public health community
Support efforts at counter-speech

18. Public Health & Content Moderation
18
Beyond Blocking
De-surfacing Harmful Content
1. Engage the public health community
Support efforts at counter-speech
2. Avoid all-or-nothing thinking
Banning is good, but so is intervening

19. Public Health & Content Moderation
19
Beyond Blocking
De-surfacing Harmful Content
1. Engage the public health community
Support efforts at counter-speech
2. Avoid all-or-nothing thinking
Banning is good, but so is intervening
3. Think of every feature on your platform as its own platform
From recommendation algorithms to reaction buttons, think about how it can be used for harm

20. Public Health & Content Moderation
20
Beyond Blocking
De-surfacing Harmful Content
1. Engage the public health community
Support efforts at counter-speech, prevention, intervention, and standard-setting
2. Avoid all-or-nothing thinking
Banning is good, but so is moderating
3. Think of every feature on your platform as its own platform
From recommendation algorithms to reaction buttons, think about how it can be used for harm
4. Consider a public health approach for other behavior with harmful
offline impacts
Radicalization, hate speech, dehumanization