Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Wird geladen in …3
×
1 von 28

Whats really going on inside panels and does it matter

2

Teilen

Herunterladen, um offline zu lesen

Ähnliche Bücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Ähnliche Hörbücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Whats really going on inside panels and does it matter

  1. 1. What’s really going on inside panels? And does it matter? ________ NORSTAT SUMMER LOUNGE June 2014 Dan Kvistbo
  2. 2. SORT OF AN AGENDA  HOW FAR WE ARE WHERE WE ARE GOING 
  3. 3. THE BEGINNING
  4. 4. BARRIERS ALL BUT GONE 1999-2001 _____ • Toluna • GMI • Ciao! • Lightspeed • Zapera • Research Now • etc. _____
  5. 5. LOW COST | HIGH SPEED LightSPEED | FLYresearch | ResearchNOW | SPEEDfacts etc.
  6. 6. FOCUS ON BENEFITS | DATA ASSUMED REP2 RESEARCH BUYERS IN LINE TO MOVE STUDIES ONLINE
  7. 7. CASE IN POINT | P&G P&G | 1999: 15%  2000: 40%  2001: 50% ”To save money and expedite market research surveys”... _____ ”[...] because surveys are completed 75% faster over the Web-and at half the cost of conventional methods [...]”
  8. 8. TROUBLED WATERS | 2003‘ish - 2006 ()
  9. 9. CULMINATION | SEP 2006 RESEARCH INDUSTRY SUMMIT “Two surveys a week apart yielded different recommendations… _____ I never thought I was trading data quality for cost savings” A WOMAN WITH A BUDGET Kim Dedeker, Procter & Gamble's VP-Global Consumer & Market Knowledge, 2006 (Now Chairman, the Americas at Kantar)
  10. 10. WHAT HAD GONE WRONG?
  11. 11. PART (I) OF THE TALE “...clients, have put many pressures on the research suppliers […]” _____ ”We’ve focused on cost and speed and an unintended consequence was erosion in quality. Now, we’re starting to feel the effects”… Kim Dedeker
  12. 12. PART (II) OF THE TALE • Heavy survey taking • Panel conditioning • Declining cooperation • Increase in demand • Persistent mode differences • Long and complex questionnaires • Multi panel membership • The 0.25% doing 30+% • ‘Yes we can’ • Etc. _____ (Source: Reg Baker, Market Strategies International)
  13. 13. MR INDUSTRY RESPONSE? ”It’s like the whole in the ozone layer. Everyone knows it’s a growing problem.” _____ ”But they just ignore it and go on to the next project.”… (S. Morwood, VP Worldwide Market Research for IBM. AdAge oct. 2006)
  14. 14. TRUTH BE TOLD ESOMAR 25 2005 – 2006 – 2007 – 2008 – 2009 – 2010 – 2011 – 2012 – 2013 – 2014 – The Grand Mean Project ARF ORQC ”Million dollar” FoQ project ISO 26362 AAPOR Report on Online Panels FoQ 2 (Reduce the Risk) Report on the AAPOR task force on non-probability sampling ESOMAR/GRBN (draft) Guideline For Online Sample Quality *(Consultation draft) < 26 28  HALFWAY THROUGH! 
  15. 15. BUT IN ESSENCE “75 years of basing quantitative market research on a paradigm of projecting the results of samples onto populations via assumptions of random probability sampling, has been replaced by a willingness to use online access panels, which do not approximate to this model.” _____ - Ray Poynter (Handbook of Online and Social Media Research, 2010)
  16. 16. AND DESPITE THE EFFORTS Lemon market | Conditions: _____ • Asymmetry of information (buyer vs. seller) • Incentive for the seller to pass off a low-quality product as a higher-quality one • Sellers with a bettter product have no way of demonstrating this credibly to buyers • Deficiency of effective quality assurances/guarantees _____ http://en.wikipedia.org/wiki/The_Market_for_Lemons (Paraphrased) Ref: The Market for "Lemons": Quality Uncertainty and the Market Mechanism - George A. Akerlof The Quarterly Journal of Economics, Vol. 84, No. 3. (Aug., 1970), pp. 488-500
  17. 17. BLACK BOX White-box testing _____ a method of testing [software] that tests internal structures or workings [of an application], as opposed to [its] functionality. (http://en.wikipedia.org/wiki/White-box_testing)
  18. 18. SO... WHAT IS GOING ON INSIDE?  CLOSER...
  19. 19. DIRTY LITTLE SECRETS OF ONLINE PANELS
  20. 20. ISSUE #1 CONDUCT _____ N = 1000, whatever it takes
  21. 21. ISSUE #2 RECRUITMENT
  22. 22. ISSUE #3 RESPONSE RATES Human Interaction? Trust that the survey is legitimate? Time involvement with each respondent? Attention given to each respondent? Respondent control over access? Respondent control over whether to respond?      
  23. 23. CONSEQUENCE The concept of TSE gets a whole new meaning… • “Total” sampling error • “Total” coverage error • “Total” non-response error • Etc.
  24. 24. IS IT EITHER OR? A Data Q sample A convenience sample of a convenience sample Random probability sample Convenience sample A quota sample of a random sample Yes, you are ”...trading data quality for cost savings...”
  25. 25. TO CONCLUDE “If you pay peanuts for your sample, chances are you're getting monkeys” (Gordon Morris Global Insights Manager at Sony Ericsson) ”I don’t want it done scientifically, I just want it done right!” (Client, Ron Sellers) “Sometimes close counts in more than horseshoes” (Reg Baker)
  26. 26. LOOKING FORWARD A few words on the future of online research 1. It’s (increasingly) mobile 2. It moves beyond self-reported attitudes, emotions and behaviours to merge survey data and behavioral data 3. Success depends as much on buyers acknowledging their responsibility (ask better questions and adapt to the realm of mobile!) as on sample providers to deliver the appropriate sample & platform
  27. 27. THANK YOU Or google ...there’s only one of me _____ DAN KVISTBO dan.kvistbo@norstat.dk +45 27772016 @Kvistbo
  28. 28. Questions?

×