Presented by Edward Paul Johnson, Director of Analytics ,SSI Christi Walters, Principal, Gongos at Market Research in the Mobile World North America 17 - 18 July 2013, Minneapolis, USA This event is proudly organised by Merlien Institute Check out our upcoming events by visiting http://www.mrmw.net

1. JULY 16 - 18, 2013, Minneapolis, USA
WWW.MRMW.NET
The original, premier event
for the Mobile Marketing
Research Industry

2. WWW.MRMW.NET
TITLE SPONSOR DIAMOND SPONSOR PLATINUM SPONSOR
GOLD SPONSORS
WORKSHOP HOST
SILVER SPONSORS
PREMIERE SPONSOR
NETWORKING EVENING SPONSOR BAG SPONSORPREMIERE SPONSOR

3. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Assembling
Modular
Mobile
Surveys
to
Create
Complete
Datasets
Market
Research
in
the
Mobile
World
July
17,
2013

4. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Edward
Paul
Johnson
Director
of
Analy6cs
SSI
Chris1
Walters
Principal
Gongos
Research
|
2
|

5. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Current
Environment
Researcher
to
Respondent
Rela1onship
VS.
respondents
researchers
I
want
to
know
what
you
think
about
soH
drinks
Great!
Here’s
a
30-­‐
minute
survey…
Sure
I
have
some
6me
to
help
you
out
Wait,
what
does
my
hair
color
have
to
do
with
beverages?
|
3
|

6. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Problem
New
Mobile
World
Consumers
want
control
Where.
When.
How
long?
How
can
we
give
them
what
they
want?
|
4
|

7. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Ques6ons
Can
we
modularize?
→
Within-­‐respondent
→
Between-­‐respondent
Can
we
fuse
the
data?
|
5
|

8. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Key
Hypotheses
Willingness
to
par1cipate
in
“long”
surveys
on
mobile
devices
will
be
increased
by
offering
incremental
incen1ves
2
There
are
minimal
data
effects
on
modularizing
surveys
on
smartphones
3
Data
fusion
techniques
allow
advanced
analysis
despite
missing
data
4
Bivariate
analysis
is
possible
with
data
fusion
techniques
5
AStude
and
behavioral
hooks
will
prove
superior
to
demographic
hooks
6
Modularizing
surveys
will
create
a
beVer,
more
enjoyable
experience
for
the
respondent
1
|
6
|

9. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Methods
Ques1onnaire
Design
Full
Ques1onnaire
Sec1on
A
Sec1on
B
Sec1on
C
We
started
with
a
full
ques1onnaire
with
three
sec1ons
|
7
|

10. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Methods
Ques1onnaire
Design
Full
Ques1onnaire
Hook
Ques6ons
Sec1on
C
Sec1on
B
Sec1on
A
Pieces
from
each
sec1on
were
removed
as
hook
ques1ons
|
8
|

11. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Methods
Ques1onnaire
Design
Full
Ques1onnaire
Hook
Ques6ons
The
rest
of
the
ques1onnaire
was
split
into
three
modules
Module
Module
Module
Sec1on
C
Sec1on
B
Sec1on
A
|
9
|

12. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Methods
Ques1onnaire
Design
Respondents
were
randomly
assigned
to
a
star1ng
module.
Respondents
always
saw
a
specific
set
of
‘hook’
ques1ons
within
their
first
module.
Respondent
Survey
Hook
Ques1ons
1st
Module
2nd
Module
3rd
Module
Choice
to
con1nue
Choice
to
con1nue
|
10
|

13. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Need
to
Assemble
the
Pieces
|
11
|

14. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Assembly
Method
Hot
Decking
Imputa1on
• Finds
similar
respondents
• Based
on
similari6es
to
link
respondents
together
• Drops
unlinked
data
— Only
actual
responses
Respondent
Matching
• Es6mates
missing
data
• Based
on
the
similari6es
to
the
remaining
sample
• Includes
all
respondents
— actual
responses
and
— es6mated
responses

15. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H1
–
enjoyable
experience
Average
Difference
in
Sa6sfac6on
Ra6ngs
Sa6sfac6on
-­‐0.06
-­‐0.03
-­‐0.02
-­‐0.14
Bad
Data
Checks
No
Straightlining
84.0%
81.7%
82.3%
87.0%
Failed
Grid
1
Instruc6ons
17.6%
13.9%
12.8%
16.9%
Failed
Grid
2
Instruc6ons
17.3%
18.1%
9.7%
12.7%
Failed
Zip
Code
Match
2.2%
1.6%
3.4%
1.4%
Control
[Online
Complete]
Control
[Mobile
Complete]
Test
[Mobile
Modular]
Test
[Online
Modular]
|
13
|

16. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
11%
9%
25%
8%
9%
8%
4%
8%
80%
83%
71%
84%
Abandoned
Removed
Completes
Results
H2
–
willingness
to
par1cipate
Control
[Online
Complete]
Control
[Mobile
Complete]
Test
[Mobile
Modular]
Test
[Online
Modular]
|
14
|

17. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H2
–
willingness
to
add
modules
9%
19%
72%
1
Module
2
Modules
3
Modules
1
Module
2
Modules
3
Modules
|
15
|

18. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H3
-­‐
minimal
sta1s1cal
effects
Compared
to
Online
Control
Online
Modular
Sta1s1cal:
No
discernible
differences
Mobile
Modular
Sta1s1cal:
25%
of
the
ques6ons
showed
sta6s6cal
differences
|
16
|

19. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H3
-­‐
minimal
prac1cal
effects
Compared
to
Online
Control
Online
Modular
Prac1cal:
Findings
and
resul6ng
insights
are
the
same
Mobile
Modular
Prac1cal:
Resul6ng
insights
were
similar,
however,
some
differences
do
exist
|
17
|

20. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H4
–
segmen1ng
the
data
Hot
Deck
Data
Imputa1on
Data
Matching
Based
on
AStudes
&
Behaviors
Online
Control
Segmenta1on
analysis
resulted
in
a
three-­‐segment
solu1on.
|
18
|

21. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H4
-­‐
checking
segment
algorithms
When
applying
the
online
control
algorithm
to
each
data
set…
When
applying
algorithms
from
each
data
set
to
the
online
control
data…
|
19
|

22. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H4
-­‐
segment
solu1on
results
20%
26%
26%
53%
51%
50%
28%
23%
24%
Segment
Sizing
Across
Techniques
Online
Control
Mobile
Hot
Deck
Data
Imputa1on
Segment
1
Segment
2
Segment
3
Data
Matching
Based
on
AStudes
&
Behaviors
|
20
|

23. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H4
–
accuracy
of
segment
assignment
83%
86%
86%
86%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Imputed
Matching
Missing
Full
Data
Matching
Based
on
AStudes
&
Behaviors
Mobile
Hot
Deck
Data
Imputa1on
|
21
|

24. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H5
–
bivariate
rela1onships
Hot
Deck
Imputa1on
Respondent
Matching
501
424
17%
20%
0.29
0.22
.24
0.29
.19
0.19
Control
Modular
No
Imputa1on
Sample
Size
333
416*
%
Significant
Differences
0.0%
15%
Average
Reasons
r
0.39
0.31
Average
AVribute
r
0.30
0.24
Average
Reason
x
AVribute
r
0.22
0.20
Distance
Correla1on
|
22
|

25. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H6
-­‐
aStudinal
vs.
demographic
s1tching
Online
Control
Segment
1
Segment
2
Segment
3
Data
Matching
Based
on
AStudes
&
Behaviors
20%
19%
26%
53%
55%
50%
28%
25%
24%
Data
Matching
Based
on
Demographics
|
23
|

26. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Results
H6:
aStudinal
vs.
demographic
s1tching
92%
86%
87%
86%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Demos
Aetudes
Missing
Full
Data
Matching
Based
on
AStudes
&
Behaviors
Data
Matching
Based
on
Demographics
|
24
|

27. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
Conclusions
Allow
respondents
to
choose
mode
(online
vs.
mobile
vs.
mul1modal)
1
Within-­‐respondent
modulariza1on
key
to
reducing
holes
in
data
2
Advanced
analy1cs
feasible
(i.e.
segmenta1on)
3
Both
fusion
techniques
work
with
unique
advantages
4
|
25
|

28. SSI
Confiden*al
©
2013
Survey
Sampling
Interna6onal
cwalters@gongos.com
Edward.Johnson@surveysampling.com
|
26
|

29. WWW.MRMW.NET
TITLE SPONSOR DIAMOND SPONSOR PLATINUM SPONSOR
GOLD SPONSORS
WORKSHOP HOST
SILVER SPONSORS
PREMIERE SPONSOR
NETWORKING EVENING SPONSOR BAG SPONSORPREMIERE SPONSOR

30. JULY 16 - 18, 2013, Minneapolis, USA
WWW.MRMW.NET
The original, premier event
for the Mobile Marketing
Research Industry