1. AAPOR 2016 Reshaping the Research Landscape: Public Opinion and Data Science
Respondent Reactions to a
Sexual Victimization Survey in a
Multi-mode Pilot Study
Suzanne Kaasa, Ph.D.
Darby Steiger, M.A.
Leanne Heaton, Ph.D.
David Cantor, Ph.D.
Rose McAloon, M.A.
3. AAPOR 2016
Pilot Study to Test Alternative Survey Methods
Phone (CATI) vs. laptop (ACASI)
Over 10,000 females from 5
locations across the country
4. AAPOR 2016
Challenges to Sensitive Survey Data Collection
(e.g., Jaffe et al., 2015; Baldor, 2013)
5. AAPOR 2016
Methodological Precautions for Sensitive Surveys
Train interviewers to be sensitive and identify
respondent distress
Assess respondent reactions through self-report
Provide sexual victimization resources to respondents
6. AAPOR 2016
Prior Research on Respondent Reactions
Sensitive surveys produce more self-reported
positive reactions than negative reactions.
Most respondents who report some feelings of
harm do not regret taking the survey.
(e.g., McClinton, 2015; Newman et al., 2006)
8. AAPOR 2016
Research Questions
1. How likely were respondents to experience positive and negative
reactions, and regret participation in the study?
2. Does survey methodology affect respondent reactions?
3. Are interviewers able to identify respondent distress?
4. Do respondents find sexual victimization resources helpful?
(e.g., McClinton, 2015; Newman et al., 2006)
10. AAPOR 2016
• You experienced intense emotions while
completing the survey.
• The research made you think about things you
didn’t want to think about.
• The research raised unpleasant emotional
issues for you that you had NOT expected.
Measuring Reactions to the Survey
• You believe you have been able to help others
by participating in this study.
• You were glad to have had the opportunity to
participate.
• You feel you gained something positive from
participating.
Negative Reactions Scale Positive Reactions Scale
(No) Regret
Now that you know what the survey is about, you would have made the same choice to participate.
Chronbach Alpha = .80 Chronbach Alpha = .77
11. AAPOR 2016
Strongly Disagree Disagree Neutral Agree Strongly Agree
1 2 3 4 5
Higher Mean Scores
More negative reactions
More positive reactions
Less regret
Response Scale
12. AAPOR 2016
Did you use any of the resources that
were shared with you?
[If yes] Did you find these resources
helpful?
Resources
15. AAPOR 2016
Emotional Reactions to the Survey
22.7
48.3
25.3
32.7
83.2
69.1
79.3
77.3
51.7
74.7
67.3
16.8
30.9
20.7
(No regret) Would have made the same choice to participate
Gained something positive from participating
Glad to have had the opportunity to participate
Believe you have been able to help others
Raised unpleasant emotional issues
Thought about things you didn’t want to think about
Experienced intense emotions
Percent of Respondents Reporting Emotional Reactions (ACASI)
No Yes
NegativePositive
*
*
*
16. AAPOR 2016
Emotional Reactions to the Survey
*
*
*13.1
42.3
21.6
20.8
79.4
67.5
79.1
86.9
57.7
78.4
79.2
20.6
32.5
20.9
(No regret) Would have made the same choice to participate
Gained something positive from participating
Glad to have had the opportunity to participate
Believe you have been able to help others
Raised unpleasant emotional issues
Thought about things you didn’t want to think about
Experienced intense emotions
Percent of Respondents Reporting Emotional Reactions (CATI)
No Yes
NegativePositive
17. AAPOR 2016
Self-reported Negative Reactions by Interviewer
Observed Distress
3.2
3.7
2.6 2.6
ACASI CATI
Signs of Distress No Signs of Distress
Interviewer Observations of Respondent Distress
RespondentSelf-reported
NegativeReactions
*
*
18. AAPOR 2016
Reactions to Sexual Victimization Resources
86.5%
12.9% 0.6%
Reactions to Resources (ACASI)
Did not use
Did use and found
helpful
Did use and didn't find
helpful
94.5%
5.1% 0.4%
Reactions to Resources (CATI)
Did not use
Did use and found
helpful
Did use and didn't find
helpful
19. AAPOR 2016
Discussion
1. Respondents reported more positive reactions than negative reactions
or regret.
2. Interviewers were able to accurately assess and respond to distress.
3. Although resources were rarely used, they were found to be helpful.
4. Survey mode did not affect respondent reactions or interviewer ability
to detect signs of distress. Mode did affect use of sexual victimization
resources.
