Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Big simulators with
dozens of products:
Which conjoint method is most suitable?
Jeroen Hardon | Marco Hoogerbrugge
2
Required prior knowledge: CBC vs ACBC
Choice-Based Conjoint: (nearly)
balanced level frequencies for every
respondent
Ad...
3
Introduction
4
What we test What we simulate
What is our problem?
5
Holdout tasks with
50-100 products are
impossible(?)
Hit rate in CBC 20.5%
> Much better than random (5%)
> But very poo...
6
Performance of standard ACBC* is very similar
Hit rate 19.8%
> Although slightly lower, the hit-rates are very comparabl...
7
Preference-Based Conjoint (PBC)
o level 1 (fixed)
o level 2 (fixed)
o level 3 (fixed)
o level 4 (fixed)
o level 5 (fixed...
8
Preference-Based Conjoint (PBC)
o level 1 (fixed)
o level 2 (fixed)
o level 3 (fixed)
o level 4 (fixed)
o level 5 (fixed...
9
Preference-Based Conjoint (PBC) as a midway
between CBC and ACBC
0
2
4
6
8
10
12
14
16
Frequency
0
2
4
6
8
10
12
14
16
F...
10
Test study
Different “legs”, each 250 respondents
11
1. CBC
2. ACBC*
3. PBC*
4. PBC2*
5. PBC2, frequency of all attributes dependent ...
12
Note on research leg with 10 concepts/screen
Task 1-6
Task 7-9
Task 10-12
Holdout
task
13
Results
0.01
0.015
0.02
0.15 0.2 0.25 0.3
14
Scatterplot of hit rate and MAE
Good direction
Wrong direction
Hit rate (FC-based)
MA...
15
PBC clearly performs better than currently
available methods
0.01
0.015
0.02
0.15 0.2 0.25 0.3
Hit rate (FC-based)
MAE ...
16
PBC2 worse than PBC
0.01
0.015
0.02
0.15 0.2 0.25 0.3
Hit rate (FC-based)
MAE (SoP-based)
PBC
ACBC CBC
PBC2
17
PBC2 worse than PBC
0.01
0.015
0.02
0.15 0.2 0.25 0.3
Hit rate (FC-based)
MAE (SoP-based)
PBC
ACBC CBC
PBC2
PBC (all at...
18
PBC2 worse than PBC
0.01
0.015
0.02
0.15 0.2 0.25 0.3
Hit rate (FC-based)
MAE (SoP-based)
PBC
ACBC CBC
PBC2
PBC2 (all a...
Adding BYO data improves hit rates even further
0.8%
1.2%
1.6%
2.0%
15.0% 20.0% 25.0% 30.0% 35.0% 40.0%
Hit rate (FC-based...
Adding brand covariate improves hit rates further
0.8%
1.2%
1.6%
2.0%
15.0% 20.0% 25.0% 30.0% 35.0% 40.0%
Hit rate (FC-bas...
21
By combining multiple approaches
we successfully improved the
hit-rate from ~20% to ~40%
> New method – PBC
> Adding ta...
22
Further research needed
> Applying PBC to all attributes seems promising
> Although we doubled the hit-rate from 20% to...
SKIMgroup.com
Thank you
Jeroen Hardon
Director Methodology & Innovation
j.hardon@skimgroup.com
Marco Hoogerbrugge
Research...
Nächste SlideShare
Wird geladen in …5
×

"Big simulators with dozens of products: Which conjoint method is most suitable?" at ART Forum 2017

252 Aufrufe

Veröffentlicht am

Download this presentation https://goo.gl/c77VU7

The key question in this presentation was to check what extent the discrepancy between the number of concepts in a task (3, or 4) versus the number of products in the conjoint simulation model (20, or 50) may influence the results of the study, by comparing CBC, ACBC and PBC (preference based conjoint).

Veröffentlicht in: Business
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

"Big simulators with dozens of products: Which conjoint method is most suitable?" at ART Forum 2017

  1. 1. Big simulators with dozens of products: Which conjoint method is most suitable? Jeroen Hardon | Marco Hoogerbrugge
  2. 2. 2 Required prior knowledge: CBC vs ACBC Choice-Based Conjoint: (nearly) balanced level frequencies for every respondent Adaptive Choice-Based Conjoint: level frequencies are skewed and different for every respondent depending on his/her earlier answers in BYO task 0 2 4 6 8 10 12 14 16 Frequency 30 GB was the respondent’s preferred level in the earlier BYO task 0 2 4 6 8 10 12 14 16 Frequency
  3. 3. 3 Introduction
  4. 4. 4 What we test What we simulate What is our problem?
  5. 5. 5 Holdout tasks with 50-100 products are impossible(?) Hit rate in CBC 20.5% > Much better than random (5%) > But very poor in absolute sense > And it will get worse in a real simulator MAE 1.81% > Nearly random (1.85%) We have a wrong prediction for 80% of our respondents In this test study we used a holdout task with 20 concepts.
  6. 6. 6 Performance of standard ACBC* is very similar Hit rate 19.8% > Although slightly lower, the hit-rates are very comparable to CBC > And again, it will get worse in a real simulator MAE 1.83% > Nearly random (1.85%) * With 3 concepts per screen, no screening section, BYO tasks included in estimation
  7. 7. 7 Preference-Based Conjoint (PBC) o level 1 (fixed) o level 2 (fixed) o level 3 (fixed) o level 4 (fixed) o level 5 (fixed) o level 6 (fixed) o respondent's choice in BYO task (flexible) o respondent's choice in BYO task (flexible)
  8. 8. 8 Preference-Based Conjoint (PBC) o level 1 (fixed) o level 2 (fixed) o level 3 (fixed) o level 4 (fixed) o level 5 (fixed) o level 6 (fixed) o respondent's choice in BYO task (flexible) o respondent's choice in BYO task (flexible) o respondent's choice in BYO + 1 level (flexible) o respondent's choice in BYO - 1 level (flexible)
  9. 9. 9 Preference-Based Conjoint (PBC) as a midway between CBC and ACBC 0 2 4 6 8 10 12 14 16 Frequency 0 2 4 6 8 10 12 14 16 Frequency 0 2 4 6 8 10 12 14 16 Frequency ACBC PBC (for example) CBC
  10. 10. 10 Test study
  11. 11. Different “legs”, each 250 respondents 11 1. CBC 2. ACBC* 3. PBC* 4. PBC2* 5. PBC2, frequency of all attributes dependent on initial preference 6. PBC*, half with 4 concepts/task, half with partial profile 10 concepts/task * Frequencies of three attributes dependent on initial preference (e.g. minutes and data, but not brand)
  12. 12. 12 Note on research leg with 10 concepts/screen Task 1-6 Task 7-9 Task 10-12 Holdout task
  13. 13. 13 Results
  14. 14. 0.01 0.015 0.02 0.15 0.2 0.25 0.3 14 Scatterplot of hit rate and MAE Good direction Wrong direction Hit rate (FC-based) MAE (SoP-based)
  15. 15. 15 PBC clearly performs better than currently available methods 0.01 0.015 0.02 0.15 0.2 0.25 0.3 Hit rate (FC-based) MAE (SoP-based) PBC ACBC CBC
  16. 16. 16 PBC2 worse than PBC 0.01 0.015 0.02 0.15 0.2 0.25 0.3 Hit rate (FC-based) MAE (SoP-based) PBC ACBC CBC PBC2
  17. 17. 17 PBC2 worse than PBC 0.01 0.015 0.02 0.15 0.2 0.25 0.3 Hit rate (FC-based) MAE (SoP-based) PBC ACBC CBC PBC2 PBC (all atts) PBC2 (all atts)
  18. 18. 18 PBC2 worse than PBC 0.01 0.015 0.02 0.15 0.2 0.25 0.3 Hit rate (FC-based) MAE (SoP-based) PBC ACBC CBC PBC2 PBC2 (all atts) PBC (partial concepts)
  19. 19. Adding BYO data improves hit rates even further 0.8% 1.2% 1.6% 2.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% Hit rate (FC-based) PBC PBC2 (all atts) ACBC CBC PBC2 PBC (partial concepts) Note: for ACBC this is the default setting 19 MAE (SoP-based)
  20. 20. Adding brand covariate improves hit rates further 0.8% 1.2% 1.6% 2.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% Hit rate (FC-based) MAE (SoP-based) PBC PBC2 (all atts) ACBC CBC PBC2 PBC (partial concepts) 20
  21. 21. 21 By combining multiple approaches we successfully improved the hit-rate from ~20% to ~40% > New method – PBC > Adding tasks with 10 partial profile concepts, sorted by brand > Adding BYO data to estimation > Adding current brand as a “smart” covariate
  22. 22. 22 Further research needed > Applying PBC to all attributes seems promising > Although we doubled the hit-rate from 20% to 40%, there are opportunities to improve further > Can we have a holdout tasks with 50 products? > More test studies needed to validate findings
  23. 23. SKIMgroup.com Thank you Jeroen Hardon Director Methodology & Innovation j.hardon@skimgroup.com Marco Hoogerbrugge Research Director m.hoogerbrugge@skimgroup.com

×