Doing More with Less: Getting Greater Value from Mobile Quant

1. A
Presenta*on
from
The
NewMR
“Advances
in
Quan*ta*ve
Research”
Event
19
September,
2012
Event
sponsored
by
Affinnova
All
copyright
owned
by
The
Future
Place
and
the
presenters
of
the
material
For
more
informa=on
about
Affinnova
visit
www.affinnova.com
For
more
informa=on
about
NewMR
events
visit
newmr.org
Doing
More
with
Less:
GeFng
Greater
Value
from
Mobile
Quant
Michael
Sosnowski,
TRC
Market
Research

2. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Doing
more
with
less:
GeFng
greater
value
from
mobile
quant
Michael
Sosnowski
Execu=ve
Vice
President
TRC
Market
Research
hNp://newmr.org/videos-­‐slides/advances-­‐in-­‐
quant-­‐sept-­‐2012/steve-­‐nutall-­‐and-­‐adrian-­‐
morgan/
hNp://newmr.org/videos-­‐slides/advances-­‐in-­‐
quant-­‐sept-­‐2012/steve-­‐nutall-­‐and-­‐adrian-­‐
morgan/

3. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Primary
researchers
in
the
pressure
cooker.

4. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Is
mobile
a
way
to
do
more
with
less?
• The
most
interes=ng
and
value-­‐
genera=ng
work
we
do
requires
some
degree
of
complexity.
q Conjoint
q Priori=za=on
techniques
q “What-­‐if”
simula=ons
• A
powerful
way
to
get
at
otherwise
hard-­‐to-­‐reach
popula=ons.
• An
invita=on
to
gather
data
at
=mes
and
places
not
previously
possible.
• Less
informa=on;
higher
frequency.
• Increasingly
consistent
with
how
consumers
want
to
give
feedback.
…
just
not
too
much
less.
Yes...

5. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Is
mobile
a
way
to
do
more
with
less?
Embrace
the
mobile
impera=ve
but…
…don’t
marginalize
yourself
with
low
involvement
or
low
value
contribu=ons.

6. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Short
and
sweet
won’t
be
enough.
Much
of
our
current
focus:
• 5
to
10
ques=ons.
• Simple
response
dynamics.
• Assume
short
aNen=on
span.
• Maximize
experience
at
expense
of
informa=on
value.
• All
good
things
of
course,
but
we
need
to
draw
upon
our
other
strengths
–
design
and
analysis.
Consumer-­‐
and
mobile-­‐friendly
surveys

7. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Short
and
sweet
won’t
be
enough.
More
with
less
in
a
mobile
environment
Consumer-­‐
and
mobile-­‐friendly
surveys
Don’t
maximize.
Op=mize!

8. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize
&
challenge
conven*onal
wisdom.
“You
can’t
do
advanced
choice-­‐based
research
on
a
handheld
device.”

9. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize
&
challenge
conven*onal
wisdom.
Is
that
true?
Let’s
ask
ourselves
some
fundamental
ques=ons:
Why
do
we
do
advanced
choice-­‐based
studies?
• To
understand
preferences
and
build
beNer
products.
• Because
we
believe
trade-­‐offs
are
superior
to
ra=ngs.
Why
don’t
we
do
such
studies
on
mobile
devices?
• They
require
too
many
choices.
• It’s
difficult
to
render
mul=-­‐op=on
choices
on
the
small
screen.

10. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize
&
challenge
conven*onal
wisdom.
An
es*ma*on
challenge:
What’s
the
right
number
of
choices?
Case
study:
Understanding
preferences
for
HDTVs
• 7
unique
aNributes
–
brand,
price,
specifica=ons.
• 331
par=cipants.
• 3
HDTV
profiles
(plus
“none”)
per
choice
task
• 13
tasks
(cards).

11. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize
&
challenge
conven*onal
wisdom.
53
61
65
68
7.1
4.7
3.3
2.9
0
10
20
30
40
50
60
70
80
3
6
9
13
Hit
Rate
Error
Perhaps
we
don’t
need
so
many
choice
tasks?
Maximum
hit
rate
Minimum
error
Case
study:
Understanding
preferences
for
HDTVs
#
of
choice
tasks

12. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize
&
challenge
conven*onal
wisdom.
A
design
challenge:
Can
choices
be
simplified?
Case
study:
Stemming
aKriLon
among
mobile
subscribers
• 9
unique
aNributes.
• 1,208
par=cipants.
• Take
it
or
leave
it
-­‐
1
mobile
(plus
“none”)
per
choice.

13. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize
&
challenge
conven*onal
wisdom.
89%
3.7
0
50
100
Hit
rate
Error
Stark
“yes
or
no”
choices
can
yield
robust
models
of
choice.
Case
study:
Stemming
a:ri;on
among
mobile
subscribers

14. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize
&
challenge
conven*onal
wisdom.
“Fewer
tasks?
Simplified
choices?
That
might
just
work!”

15. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize
&
do
more
powerful
quan*ta*ve
studies
via
the
mobile,
today.
Tournament-­‐style
prioriLzaLon…
good
for
online;
GREAT
for
the
mobile
plaWorm
• What
makes
it
good
for
lengthy
feature
sets
makes
it
a
great
poten=al
fit
for
mobile
MR.
q Fewer
ques=ons
q More
relevant
ques=ons
q Handheld-­‐friendly
formaFng
requirements
q Simple
response
mechanics
Consumer-­‐
and
mobile-­‐
friendly
surveys
More
with
less
in
a
mobile
environment
A
potent
applicaLon
that
combines
innovaLve
design
and
analysis
with
a
simple
and
engaging
task

16. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Mobile
“more
with
less”
in
ac*on.
Want
to
catch
a
movie
in
a
theater?
How
do
you
decide
what
to
see?
An
everyday
priori=za=on
problem.

17. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Mobile
“more
with
less”
in
ac*on.
• So
many
factors
to
consider.
• We
stopped
at
18.
• Max-­‐Diff
=
14
choice
sets.
• Tournament-­‐style
approach:
9.
• “That’s
more
mobile-­‐
appropriate,”
we
thought.
“Let’s
test
it
out.”
A
tournament
style
approach

18. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Mobile
“more
with
less”
in
ac*on.
• 414
U.S.
adults.
• 20-­‐ques=on
survey
(6-­‐7
minutes)
• All
accessed
and
completed
the
survey
using
a
mobile
device.
• Main
tasks
–
review
and
priori=ze
across
our
18
“go
to
the
movies”
choice
factors.
• Simple,
appropriately-­‐formaNed
tasks
presented
on-­‐device.

19. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Results
confirm
our
ability
to
tackle
difficult
quant
work
via
mobile.
a.
You
like
the
plot
or
story
b.
It
is
your
favorite
movie
genre
c.
It
is
adapted
from
something
you
are
familiar
with
d.
It
received
good
reviews
from
movie
cri=cs
e.
Your
friends
and
family
liked
it
f.
The
movie
has
your
favorite
stars
g.
The
theater
is
conveniently
located
h.
It
is
playing
at
a
convenient
=me
at
the
theater
i.
It's
a
sequel
to
a
movie
you've
already
seen
j.
It
is
shown
in
3D
k.
The
theater
has
good
quality
and
variety
of
food
l.
You
have
a
loyalty
card
or
get
points
or
rewards
m.
It's
targeted
towards
people
like
you
n.
You
think
it
will
be
a
popular
movie
o.
It
has
your
favorite
director
p.
It
is
a
long
movie
q.
It
has
an
R
ra=ng
r.
It
is
animated
A
rank-­‐ordering
of
importance
Most
important
decision
factor
Least
important
decision
factor
Case
study:
What’s
important
to
you
when
choosing
a
movie?

20. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Results
confirm
our
ability
to
tackle
difficult
quant
work
via
mobile.
Case
study:
What’s
important
to
you
when
choosing
a
movie?
0
5
10
15
20
25
30
35
40
45
50
a b c d e f g h i j k l m n o p q r
Mobile
Tradi=onal
Bayesian
esLmaLon
of
uLliLes
helps
idenLfy
Lers
of
importance.
Results
match
nicely
to
those
obtained
using
tradiLonal
online
data
collecLon.

21. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
“More
with
less”
mobile
research:
Beyond
short
&
sweet.
More
than
fast
&
cheap.
• Don’t
marginalize
the
role
of
custom,
quan*ta*ve
research
in
an
increasingly
mobile
environment.
• Remember
we
have
more
tools
in
our
belt
than
programming
and
crea*ve
formang.
• This
is
relevant
right
now.
• “More
with
less”
mobile
can
help
improve
all
methodologies.
• Be
an
op*mizer,
not
a
maximizer.

22. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Thank
you.
Michael
Sosnowski
Execu=ve
Vice
President
TRC
Market
Research

23. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Q
&
A
Sue York
NewMR
Michael Sosnowski
TRC Market Research

24. Speaker: Michael Sosnowski, TRC Market Research
NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Michael
Sosnowski
TRC
Market
Research
Visit
TRC’s
site
at
hNp://trchome.com
Read
Michael
and
his
colleagues’
blog
posts
@
hNp://trchome.com/trc-­‐blog.html
Follow
Michael
@
hNp://twiNer.com/MRSosnowski
(@MRSosnowski)
Connect
with
Michael
on
LinkedIn
@
hNp://linkedin.com/in/michaelrsosnowski
Contact
Michael
at
215.641.2258
or
at
msosnowski@trchome.com

25. A
Presenta*on
from
The
NewMR
“Advances
in
Quan*ta*ve
Research”
Event
19
September,
2012
Event
sponsored
by
Affinnova
All
copyright
owned
by
The
Future
Place
and
the
presenters
of
the
material
For
more
informa=on
about
Affinnova
visit
www.affinnova.com
For
more
informa=on
about
NewMR
events
visit
newmr.org
Doing
More
with
Less:
GeFng
Greater
Value
from
Mobile
Quant
Michael
Sosnowski,
TRC
Market
Research