Webinar Mobile Conjoint Analysis: Are the results valid?

Share your thoughts online:

#SKIMwebinar

Mobile Choice-based Conjoint
A valid alternative

Gerard Loosschilder
CMethO

John Ashraf
Conjoint expert

Abigail Joffre
Today’s webinar host

Mobile surveys: it’s time to act
Increasingly people take surveys on a
mobile platform. Filtering them out
may skew the sample.

In

Standard
survey design

7%
4%

% of people taking
the test survey by
platform
Smartphone

89%

Tablet
PC

Mobile survey
design

Screen

Source: Burke, July 2013

Out
Biased sample

2

2013

SKIM Webinar “Mobile Choice-based Conjoint: A valid alternative”

But we should do a better job tailoring to their needs.
The respondent may have something better to do
Survey is
enjoyable
PC

24%

Tablet

Mobile ST

Mobile+

9%

PC

This is
bad news

Mobile ST

Mobile+

PC

79%

Tablet

14%

10%

Would take another
survey on the same
device

Responding
was very easy

53%

38%

31%

This is
bad news

Tablet

66%

48%

Mobile ST

35%

Mobile+

This is
bad news

32%

Source: Burke, July 2013. Mobile ST is a standard survey on a smartphone; Mobile+ is a dedicated mobile survey design

3


Today’s presentation is about the
validity of mobile choice-based conjoint

Introducing 3*3 mCBC

4


Adapting to the shorter attention span of the mobile user
By reducing the number of choice tasks and number of choices per task
Three choice tasks

5

Three options each


Completed in

At the same valid results as a traditional CBC exercise
The good news

The bad news

The MAE of share predictions is
<4% at the correct sample size

So far, it is impossible to reduce
the number of choice tasks and
options per task to fewer than the
3*3 platform

Overall results, conclusions and
recommendations are the same

6


3*3 mCBC requires a larger sample size
As a rule of thumb, sample size is
tied to number of parameters to be
estimated, determined by the
number of attributes and levels
Traditionally, we recommend a minimal
sample size of n= 200 for any conjoint
study. Now, the sample size goes up
with the number of parameters to be
estimated.

7

If # of
parameters is

Sample size
n=

12 (3 atts*4 levels)

400


800


1,200


1,600


3*3 mCBC requires a well-balanced research design
In a 3*3 mCBC research design,
the number of data points
collected is much smaller than the
number of parameters to be
estimated.
So, the key success factor is a
high-quality research design

8

Frequency balanced
Every combination of two
attributes and their levels is
represented well by all research
designs
Market knowledge included
We take current market shares
and knowledge about consumer
preferences into account in design
generation


Empirical evidence

How do we know if mCBC
using a 3*3 design is valid?

9


Because we have validated it …

We determined the relation between
sample size and convergent validity
It shows that at increasing sample sizes,
mobile CBC is a good approximation of
common CBC on traditional platforms

10


… in a study into pain killers and batteries
Attributes:
 Brand
 Admin method
 Dosage
 Package
 Price
3-5 levels each;
16 parameters

Attributes:
 Brand
 Price
 Size
3-5 levels each;
12 parameters

What is the validity of mobile CBC at various sample sizes
if compared with the same conjoint study in a traditional way at n= 200?
11


Attribute importance values show great similarities
at all sample sizes for mobile CBC
The values for
batteries are
equally similar

Attribute importance

Pain killers
40%
30%
Brand

20%

Admin method
10%

Dosage

0%

Package
Price

Variations in
attribute
importance values
are insignificant

Platform and sample size

12


However, at larger sample sizes, preference shares get
closer to benchmark leg, demonstrating greater validity
10.0%

Mean Absolute Error values (MAE, in %)

MAE value in %

7.5%

The break point is at n= 400; twice the
sample size of the benchmark
5.0%

Batteries
Pain killers

2.5%

Especially at pain killers at 16
parameters (instead of 12 for batteries)
0.0%
200

13

300 400 500 600 700 800
Sample size in the mobile leg

Mean Absolute Error
(MAE) is a metric
expressing how far
apart the preferences
shares are as
measured by mCBC at
increasing sample
sizes compared with
the benchmark CBC
at n= 200

900


Conditions to make 3*3 mCBC work – research design
Every display of product profiles in a choice task is valuable. It
should not be wasted on prohibitions or alternative specific designs.
• We can not tolerate any reduction in D-efficiency values

So for now, we can apply mCBC in some standard situations.
We do not apply 3*3 mCBC in situations in which there are
prohibitions, utility balanced designs and alternative-specific
designs
14


Conditions to make 3*3 mCBC work – real estate
There is no space for large
amounts of information
Max. 5 attributes

Text is succinct, keywords
only
No complex descriptions

Pictures are
icons or schematic
Be careful with package or
ad testing

15


Powered by

Interaction design
http://tinyurl.com/skimmcbc

16


3*3 mCBC in Price Sensitivity Analyses – batteries

17


3*3 mCBC in Price Sensitivity Analyses – pain killers

18


3*3 mCBC in Telecom – design of a postpaid portfolio

19


Mobile in MaxDiff applied to Semantics
and other tests of claims and benefit statements

20


To conclude

3*3 mCBC is a valid addition

21


To conclude – mCBC is a valid addition
Do

Don’t

• Take market knowledge into
account

• Use it for large CBCs

• Use a larger sample

• Include research design
restrictions

• Use succinct text, iconized
visuals

• Use it for package, ad or
concept work

• Have five attributes at five
levels max

22


Next steps
• SKIM researchers will continue to validate mCBC …
• For allocation studies
• For large research designs using consideration sets
• For more complex visuals

• Keep improving UI for Mobile CBCs
• Across dominant platforms

• Translate learnings back to any CBC
23


Share your thoughts online:

#SKIMwebinar

Go to www.skimgroup.com/webinars
for today’s presentation slides and more!
Gerard Loosschilder

John Ashraf

Abigail Joffre

Chief Methodology Officer
g.loosschilder@skimgroup.com
@gloosschilder

Conjoint expert
j.ashraf@skimgroup.com

a.joffre@skimgroup.com
+1 201 963 8430
@SKIMgroup

Webinar Mobile Conjoint Analysis: Are the results valid?

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Mehr von SKIM

Mehr von SKIM (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Webinar Mobile Conjoint Analysis: Are the results valid?