An experimental study on show cards in social surveys

1. 1
1
An experimental study on show cards
in social surveys
Nordic Statistical Meeting in Bergen
14th-16th of August 2013
Karianne Lund, Frode Berglund og Kjersti Stabell Wiggen
Division for Data Collection Methods, Statistics Norway
Division for analyses, Directorate of Labour and Welfare
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare

2. Background
• Investigate the use of show card in computer assisted telephone
interviews (CATI).
• The show card is distributed by mail prior to the interview, together
with an information letter about the survey.
• The purpose of the show card is to reduce the response burden and to
improve data quality.
2
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare

3. 3
Research questions
o How do show cards affect respondents’ preconception of the
survey? To what extent can show cards be interpreted as effecting
response rate?
o Do the respondents use the show card?
o How do the use of show cards affect the response distribution and -
more importantly- the data quality?
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare

4. 4
Test design
Experiment I Experiment II Experiment III
Show card on alcohol
unit (instructive)
Show card on alcohol
unit (instructive)
Show card on chronic
diseases (retrieval)
N/A Survey on Alcohol,
Tobacco and Drugs
(SATD)
Survey on Living
conditions and health
(LKH)
N/A CATI CATI
Controlled experiment Embedded experiment Embedded experiment
N=260 N=1947 N=9601
Response rate,
preconceived response
burden
Mapping of use,
response
distribution/prevalence
Mapping of use,
response rate,
response
distribution/prevalence
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare

5. 5
Show cards effect on respondents willingness to
participate in a survey (Data source: LKH)
Response rates by presence of show card and education
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare
Education No
show card
With
show card
Difference
Low 53 47 6
Middle 66 60 6
High 71 72 -1
Total 64 60 4

6. 6
Respondents use of show card – if and who (Data
source: SATD )
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare
Low Middle High
Say they did not
get show card 31,6 25,7 24,8
Did not use 29,3 29,4 30,3
Used 39,1 44,8 44,9
Total (= 100 %) 1401 2358 2193
The distribution of the use of show card in the survey.

7. 7
Show cards effect on response distribution and
(more importantly) the data quality? (Data source:
SATD)
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare
Show card Mean N Std. dev.
Not received 20,4 329 19,9
Active user 19,4 245 20,9
Knows content 20,3 495 19,7
Refuse to use 18,9 234 17,8
Total 19,9 1303 19,6
Alcohol consumption of alcohol units related to use of show card

8. 8
Show cards effect on response distribution and
(more importantly) the data quality? (Data source: LKH)
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare
Show card Mean N Std. dev.
Did not receive 1,26 834 1,74
Say they did not
get
0,99 1302 1,52
Did not use 1,05 1437 1,51
Used 1,45 2088 1,78
Total 1,21 5661 1,66
Average number of reported diseases by use of show card.

9. 9
Conclusions from the study
• Show cards had a negative effect on the response rate, especially
among people with low education, and thus increased bias for this
group.
• The presence of show card did not have an effect on response
distribution, and we can assume it did not improve the data quality.
• Not a large share of the respondents used the show card
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare

10. 10
Thanks for your attention!
An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour
and Welfare