This document provides an overview of Respondent-Driven Sampling (RDS), a method for sampling hidden populations using their social networks. RDS begins by selecting initial participants, called "seeds", who each recruit a small number of new participants into the study. Those new participants can then recruit others, creating a chain-referral sample. The document discusses how RDS works, its applications across many hidden populations, and some of its promises and pitfalls, including high sampling variance requiring large sample sizes compared to simple random sampling. It also reviews recent progress on estimating sampling variance from RDS studies.

1. Respondent-Driven Sampling:
An Overview
Ashton M. Verdery
Duke Network Analysis Center
May, 2019

2. Outline
• Intuition about network sampling
• Leveraging social networks for sampling
– Why?
– How?
• What is RDS?
– Hidden populations
– RDS origins and concepts
– RDS applications
– Pitfalls and promises of RDS
– New directions
Ashton M. Verdery 2

3. Samples from social networks
Ashton M. Verdery 3
1
6
2
8
3
11
4
5
9
7
10

4. Samples from social networks
Ashton M. Verdery 4
1
6
2
8
3
11
4
5
9
7
10

5. Samples from social networks
Ashton M. Verdery 5
1
6
2
8
3
11
4
5
9
7
10

6. Samples from social networks
Ashton M. Verdery 6
1
6
2
8
3
11
4
5
9
7
10

7. Samples from social networks
Ashton M. Verdery 7
1
6
2
8
3
11
4
5
9
7
10

8. Samples from social networks
Ashton M. Verdery 8
1
6
2
8
3
11
4
5
9
7
10

9. Samples from social networks
Ashton M. Verdery 9
1
6
2
8
3
11
4
5
9
7
10

10. Samples from social networks
Ashton M. Verdery 10
1
6
2
8
3
11
4
5
9
7
10

11. Why do it?
• Future of social science research
– New populations of interest are hard to survey
• e.g., undocumented migrants, people who use drugs
– New theories & tools require new types of data
• e.g., social network analysis
– Existential threat of declining survey participation
• i.e, all groups are becoming hidden populations
Ashton M. Verdery 11

12. Silliness
12
http://www.pewresearch.org/
2017/05/15/what-low-
response-rates-mean-for-
telephone-surveys/

13. Hidden populations
• Collecting data from
hidden populations is
difficult because the
absence of a sampling
frame
– Stigma
– Non response
– Lack of trust
– Rarity
Ashton M. Verdery 13
Household based sampling in Lilongwe, Malawi
Escamilla et al. 2014

14. How to sample hidden populations?
• Traditional approaches
– Convenience samples
– Clinical samples
– Location samples
• Problems
– Are we learning about people other than those sampled?
• Limited ability to infer representation
• Poor coverage for sampling frame
• Often time intensive, costly, very small samples
Ashton M. Verdery 14

15. Respondent-Driven Sampling (RDS)
• A sociological method with wide applications
– Heckathorn 1997
• Most popular solution to problems of hidden
populations in recent decades (as of May 2019)
– 544+ studies
– 1.2k+ papers, 24k+ cites
– H-index of 59
– Over $213 mill. from NIH
• Compare to “ego centric”
– 254 studies funded
– $59 million since 1990

16. RDS applications
• Hidden populations of many stripes
– Men who have sex with men
– People who inject drugs
– Commercial sex workers
– High risk heterosexuals
– Other drug users (opioids, methamphetamines)
– Domestic violence victims
– Victims of sexual violence (child prostitution, sex trafficking, war-time rape)
– Jazz musicians
– Vegetarians and vegans in Argentina
– Wheelchair users
– Non-institutionalized older adults (85+)
• Most common questions
– Can we sample this population?
– What are the characteristics of this population?
– What is the size of this population? 16

17. Top 10 fields
Ashton M. Verdery 17
Web of Science. May 2019.

18. RDS overview
Two parts
1) Chain referral / peer recruitment
– “Seed” participants receive 2 coupons
• Recruit 2 new participants each
• Dual incentives for participation & recruitment
• Each new respondent given 2 coupons to recruit others
• Process continues until desired sample size is obtained
• (No one participates more than once)
• *Researchers lack control of sampling process
2) Post-recruitment weighting of cases
– Correct for theoretical sampling process
– Make inferences about population & quantify uncertainty

19. Seeds & coupons
19
Wirtz et al. 2017
• Seeds
– 7-10 population members
– Convenience selection
• Willing to participate
• Large personal networks
• Diverse on relevant attributes
• Coupons
– Give 2 to 3 per respondent
• Non-seeds can only
participate with a coupon
– Uniquely coded for tracking
• Codes given out & redeemed
– Non-physical coupons
• Possible, but challenging

20. Coupons
Ashton M. Verdery 20
Contact number
Consent and study
description (on back)
Valid dates
Interview site location
Tracking codes

21. Example
Ashton M. Verdery 21(Fisher and Merli, Net. Sci.
2014)

22. Example
22(Verdery, et al Soc. Meth. 2017)
Sampled = black

23. Core resources
• Useful website from Handcock, Gile, & collaborators
– http://hpmrg.org
• Manuals for RDS survey design
– Johnson tutorial, with questionnaires, consent forms, etc.
• http://applications.emro.who.int/dsaf/EMRPUB_2013_EN_1539.pdf
– CDC, UNAIDS, & others also have useful manuals
• https://www.cdc.gov/hiv/pdf/statistics/systems/nhbs/nhbs-idu3_nhbs-het3-protocol.pdf
• https://globalhealthsciences.ucsf.edu/sites/globalhealthsciences.ucsf.edu/files/ibbs-rds-protocol.pdf
• Software for RDS analysis
– Stand alone software for RDS coupon management & analysis
• http://www.respondentdrivensampling.org/main.htm
– R package “RDS” for analysis & diagnostics
• https://cran.r-project.org/web/packages/RDS/index.html
– Stata packages for analysis
• http://www.stata-journal.com/article.html?article=st0247
• I have unreleased Stata packages for many RDS estimators and RDS multivariate regression
• Diagnostics for RDS preplanning and post-survey analysis
– http://www.princeton.edu/~mjs3/gile_diagnostics_2014.pdf

24. Network structure assumptions
There is a social network
Population size large (N>>n)
Homophily weak
Community structure weak
Connected graph w/1 component (giant
component)
All ties reciprocated (undirected)
Known population size N
Sampling assumptions
Sampling with replacement
Single, non-branching chain (1 seed; 1
coupon)
Sufficiently many sample waves
Initial sample of seeds unbiased
Degree accurately measured
Conditionally random referrals (random
Key concepts & assumptions
• Baseline assumptions
– Population members are linked
in a social network & will refer
other members into the study
• Key concepts
– Primary & secondary interviews
– Respondent degree
– Random recruitment
– Bottlenecks
– Bias, sampling variance, & RMSE
• Different estimators make
different assumptions about
recruitment process and
underlying network
Ashton M. Verdery 24
(see Gile 2011:144)

25. Primary & secondary interviews
Ashton M. Verdery 25

26. Respondent degree
• Degree
– Popularity
– How many incoming ties
• network assumed undirected
• Typical solicitation
– “how many people do you
know (you know their name
and they know yours) who have
exchanged sex for money in the
past six months?”
– Often, successive restrictions
• Last 30 days, live in area, etc.
• Key element of most mean
estimators
𝑤𝑖 = 𝑑𝑖
−1
𝑖
𝑑𝑖
−1
Merli, et al Soc. Sci. Med.

27. Assumption: “random recruitment”
Ashton M. Verdery 27
A
B
C
D
3/9
3/9
3/9

28. In practice: “preferential recruitment”
Ashton M. Verdery 28
A
B
C
D
4/9
4/9
1/9

29. Reasons for preferential recruitment
• NOT A REASON
– Has more connections to similar people
• In principle, the weighting approaches should deal with this
• Reasons (not exhaustive)
– Better relationships with similar people
– Wants to help friend who needs money
– Wants friend to get HIV test
– Only friends who do riskier things want to get tested
– Unemployed friends more likely to be encountered
– Etc.
Ashton M. Verdery 29

30. “Bottlenecks”
• Few ties between clusters
– Assumed to matter
substantially
– Somewhat overstated
• General advice:
– Split sample
– Tough to achieve a priori
With n=500, rds on this
network exhibits 150X
the sampling variance of
SRS and the estimated
sampling variance bears
no relation to this, we
see this in network after
network after network
Mouw & Verdery Soc. Meth. 2012
Salgnik & Goel Stat. Med. 2009

31. Key concepts
• Bias
– “Accuracy”
– How far from the population parameter
is the average sample?
• Sampling variance
– “Precision”
– How variable are the results, sample to
sample, on average?
– Often expressed as Design Effects
• Ratio of RDS to SRS sampling variance
• Interpretable as sample size multiplier
• Root Mean Square Error (RMSE)
– Balancing accuracy and precision
• There are many other error metrics
Verdery, Merli, et al. Epid. 2015.
Just right?

32. Contrast with SRS
Network: Project 90 (N=4413)
Variable: Percent White
RDS
– Unbiased, 10 seeds, 3 coupons
– Without replacement
– N=150
SRS
– Without replacement
– n=150
Ashton M. Verdery 32
Project 90 network, red nodes=non-white
Verdery et al. 2017

33. Contrast with SRS
Ashton M. Verdery 33
0
50
100
150
200
250
40 60 80 100
Estimated percentage
RDS
Sample
0
50
100
150
200
250
Numberofsamples
40 60 80 100
Estimated percentage
SRS
Sample

34. Contrast with SRS
Ashton M. Verdery 34
Backup: https://youtu.be/BZL3XBeG7W8

35. Contrast with SRS
Ashton M. Verdery 35

36. Contrast with SRS
(n=400)
Ashton M. Verdery 36
0
10
20
30
Frequency
.5 .6 .7 .8 .9
Estimate
RDS VH weights
SRS w/o replacement

37. • Early RDS work focused on bias, but
sampling variance is also critical
• A related concern:
– Quantifying uncertainty
– After data collection, can you say:
• How biased your sample is?
• How results would vary sample to sample?
– Key feature of inferential statistics
• E.g., if sampling conformed to
assumptions, we can provide a confidence
interval for an estimate and be reasonably
sure the confidence interval is accurate
• Is this true in RDS?
Bias, sampling variance, & uncertainty
37

38. Quantifying uncertainty
• Traditional estimators of RDS sampling variance are bad
• Example
– Sampling variance (SV)
• RDS mean estimators have high SV
– Estimated sampling variance
• RDS SV estimators have high bias
Verdery et al.,
Plos1 2015

39. Recent progress on estimating
RDS sampling variance
Ashton M. Verdery 39
Baraff, et al., PNAS. 2015

40. Estimators
• Of the population mean
– At least 11 in current use
• Table on right
• McCreesh et al. 2013
• Crawford 2016
• Gile & Handcock 2015
• Berchenko 2017
• Of the sampling variance
– 5 primary methods in use
• Bootstrap (Salganik 2006)
• Analytical (Volz & Heckathorn 2008)
• Successive Sampling (Gile 2011)
• Model assisted (Gile & Handcock 2015)
• Tree Bootstrap (Baraff et al. 2016)
eTable 1. The seven respondent-driven sampling estimators evaluated in this paper.
Estimator Source
1. Naïve None
2. RDS1-SH Salganik MJ, Heckathorn DD. Sampling and Estimation in Hidden Populations Using
Respondent-Driven Sampling. Sociol Methodol. 2004;34(1):193–240.
doi:10.1111/j.0081-1750.2004.00152.x.
3. RDS1-DS Heckathorn DD. Respondent-Driven Sampling II: Deriving Valid Population Estimates
from Chain-Referral Samples of Hidden Populations. Soc Probl. 2002;49(1):11-34.
doi:10.1525/sp.2002.49.1.11.
4. RDS1-DG Heckathorn DD. Extensions of Respondent-Driven Sampling: Analyzing Continuous
Variables and Controlling for Differential Recruitment. Sociol Methodol.
2007;37(1):151–207. doi:10.1111/j.1467-9531.2007.00188.x.
5. RDS1-LEN Lu X. Linked Ego Networks: Improving estimate reliability and validity with
respondent-driven sampling. Soc Netw. 2013;35(4):669-685.
doi:10.1016/j.socnet.2013.10.001.
6. RDS2-VH Volz E, Heckathorn DD. Probability based estimation theory for respondent driven
sampling. J Off Stat. 2008;24(1):79.
7. RDS2-SS Gile KJ. Improved Inference for Respondent-Driven Sampling Data With Application
to HIV Prevalence Estimation. J Am Stat Assoc. 2011;106(493):135-146.
doi:10.1198/jasa.2011.ap09475.
40
Verdery, et al., Epid. 2015

41. General comments on estimators
• For the population mean
– “linked ego networks” is best
• Requires respondents know
peer attributes reasonably well
• Can’t calculate for many
variables of interest
– Naïve estimator often works
– Most common
• Volz-Heckathorn
• Successive Sampling
– (In general, SS is better)
• For the sampling variance
– Only the tree bootstrap
method seems to have
anything resembling
reasonable properties
41
Verdery, et al., Epid. 2015

42. Diagnostics
• Embed questions in the survey to
allow you to estimate whether
assumptions were met
– E.g., ask why people recruited those
they did, how many people they
tried to recruit who had already
participated, etc.
• Assess potential bottlenecks and
seed bias with convergence plots
Ashton M. Verdery 42
Johnston, et al., Epid. 2015

43. A few notes on web-based RDS
• Developing area with challenges but lots of potential
• Recommendations
– Differences from traditional
• Be prepared to expand to 30-60 seeds; 20+ waves
– Verification
• Respondent Uniqueness
– IP address verification; web-cam interview?
• Respondent is in target population
– In geographic area of interest? Fits other criteria?
• Coupon management
– Careful with secondary incentives
– Remember limitations
• Internet access, etc.
43

44. If problems…
• Expand recruitment
– Expand number of seeds
– Expand allowable recruits
– Raise incentives
– Reduce burdens
• Greater emphasis on anonymity
• Shorten survey
• Drop secondary interview
• If all else fails…
– Convenience sample
– Lean on other features
• It won’t always look like it does on paper
Ashton M. Verdery 44

45. My recommendations
• 1) Embed additional data collection in RDS
– Qualitative interviews
– Ego network rosters
– Minimally identifiable information about alters
• 2) Examine more than just prevalence
– Population size
– Network structure
– Multivariate relationships
Ashton M. Verdery 45

46. Promises & pitfalls
Weighting/estimation can yield asymptotically
unbiased estimates of population mean
– Unrealistic, hard to verify assumptions required
Design effects remain high
– Orders of magnitude larger N needed
But…
– New data on understudied populations
– Effective, fast method (50 cases/week)
– Possible to learn a lot about networks (underutilized)
Ashton M. Verdery 46

47. Thank you!
Portions of this work were supported by a grant from the National Institutes of
Health (1 R03 SH000056-01; Verdery PI): “Multivariate Regression with Respondent-
Driven Sampling Data.”
I also appreciate assistance from the Justice Center for Research, the Institute for
CyberScience, the Social Science Research Institute, the College of the Liberal Arts,
and the Population Research Institute at Penn State University, the last of which is
supported by an infrastructure grant from the Eunice Kennedy Shriver National
Institute of Child Health and Human Development (P2CHD041025 & R24 HD041025).
Other portions of this work benefitted from support from the Duke Network Analysis
Center, the Duke Population Research Institute, and the Carolina Population Center.
Ashton M. Verdery: amv5430@psu.edu
I thank many coauthors: M. Giovanna Merli, James Moody, Ted Mouw,
Peter J. Mucha, Jacob C. Fisher, Shawn Bauldry, Nalyn Siripong, Jeff
Smith, Kahina Abdessalem, Sergio Chavez, Heather Edelblute, Jing Li,
Jose Luis Molina, Miranda Lubbers, Sara Francisco, Claire Kelling, Anne
DeLessio-Parson, & David Hunter.

48. alternate link-tracing designs
• Network Sampling with Memory
– collect network data from respondents
– minimally identifying information to link
nominated but not sampled individuals
– “search” algorithm to explore the network more
efficiently based on currently uncovered data
– recovers sampling frame
– “list” algorithm samples frame as if at random
48
Mouw & Verdery. 2012. Sociological
Methodology.

49. network sampling with memory
• Two sampling modes:
– Search
• Push sample to explore network by seeking bridge ties
– List
• Keep a list L of unique members, both nominated & sampled
• Sample with replacement from L
• “Even” sampling of nodes ensured by probabilistic selection
• When whole network nominated, converges to SRS
• Simulated sampling showed hybrid (S -> L) best
Ashton M. Verdery 49

50. network sampling with memory

51. test network
• Add Health high school
– 1,281 students
– 67.3% white
– 10,414 edges in data
– 587 cross race ties (w->nw)
– 8% of whites’ friends n.w.
• Conclusions:
– Homophily in the data
– But no “choke points”
– Lots of cross group ties
• Method
– Test simulated FNSM
• 500 samples, 500 cases each
– RDS, NSM, FNSM
• Calculate Cis and DEs 51

52. results in test network
Ashton M. Verdery 52

53. empirical results

54. Key concepts
• Where 𝑎 is number of samples, 𝑐𝑖 is the
estimated statistic from sample 𝑖, and 𝐶 is
the population parameter:
– Bias
• 𝑏𝑖𝑎𝑠 = 𝑎−1
𝑖=1
𝑖=𝑎
𝑐𝑖 − 𝐶
• “Accuracy”
– Sampling variance
• 𝑆𝑉 = 𝑎−1
𝑖=1
𝑖=𝑎
𝑐𝑖 − 𝑎−1
𝑗=1
𝑗=𝑎
𝑐𝑗
2
• “Precision”
– Root Mean Square Error (RMSE)
• 𝑅𝑀𝑆𝐸 = (𝑏𝑖𝑎𝑠2 + 𝑆𝑉)
• Balancing accuracy and precision
– There are many other error metrics; I like this
– Design effects
• 𝐷𝐸 = 𝑆𝑉𝑅𝐷𝑆 𝑆𝑉𝑆𝑅𝑆
• Precision ratio compared to simple random samples
• Sample size ratios for equivalent efficiency
Verdery, Merli, et al. Epid. 2015.
Just right?