Online Research Coming of Age

Holger Lütters -Online Research Coming of Age -University of Pretoria September 2014
Prof. Dr. Holger Lütters
Hochschule für Technik und Wirtschaft Berlin, Germany
Online Research Coming of AgeBrown-Bag presentation 18.09.2014

Holger Lütters –a little older than the internet
My unplanned way into online research
•First E-Mail sent 1994
•First Webpage 1996
•1999 Own online survey created with HTML
•2004 PhD on web based Analytic Hierarchy Process
•2006 Interactive SniperScale
•2007 Panel study
•2008 International Multi-language studies
•2010 Facebook for recruiting
•2011 Mobile only study
•2012 Avatar study
•2013 Gyroscale use
•2014 Mobile only Panel

Problem with online market research: Lack of representativeness
Quelle: FesselGfKAustria

Online Market Africa: SA at 50% of Internet penetration leading on the continent, but still away from a representative popuation
http://www.internetworldstats.com/stats1.htm

Methods of online research are not a substitute but a complement to the existing methods of market research
•Online market research has established itself as a serious approach (already 1st place in many countries)
•There are outstanding online solutions for many questions available
•For some topics online still is an inappropriate method due to the lack of representativeness of the population
•The time continues to work for the online market research idea:
•Expiration of classical structures of accessibility of people on the phone (5 phones in the home, multiple phone numbers, only mobile users, VoIP, etc.)
•Rising costs for traditional types of surveys
•Many online groups are already covered at 100% (students, teenagers, business surveys)
•Everything presented today will be in South Africa sooner or later. With our university cooperation it may be sooner and give you a competitive advantage over other institutions in your country and will help you compete with the international research community

Fundamentalsofonline research
1

„Internet as an object“
Analysis of online users, online sites and online research methods
„Internet as a method“
Collection of market and customer data with the help of the Internet
Online Research: Internet as an object or a method?
Source: Zerr
Online-Research
Non-reactive data
•User-Tracking
•Visitor Analysis
•Conversion Analysis
•…
Reactive Data
•Interviews
•Focus Groups
•Web- Experiments
•Delphi
•…

Online Market Research: Primary and secondary research with the help of Internet technologies
Example
-Reports
-Statistics
-Databases
in
-WWW
-Intranet
-Extranet
-Social Media
Reactive
Data
Non-reactive
Data
Primary researchInternet as a tool
Secondary research (Desk)
Internet as a source
Online-Market Research
SocialMedia Research

Online-interview = location-independent, impersonal, self-administered computer-interview
Face-to-Face
Telephone
impersonal
personal
Computer-based surveys
Computer Assisted
Personal Interview
CAPI
Computer Assisted
Telephone Interview
CATI
Computer Assisted
Web-Interview
CAWI
Many methodologically supported interviews are computer-assisted interviews ... ... with the respective advantages and disadvantages of the specific interaction method

5 standard survey modes in 2014
Advantages of the different survey methods? Every method has its own purpose and target group
Effect of the chosen interview mode on results?
Effect on cost, time, quality?
Personal Interview
Paper PencilSurvey
Telephone
Interview
Online
Interview
Mobile Interview

Threestepstoruna surveyonline
2

Three key challenges arise after the decision for an online market research project
(Scientific) Modelingof
questionnaires
Development of a survey design
Programming
Distribution & Response Management
Reach the defined participants
Controlling
Screen-out
Quota Definition
Analysis & Reporting
Statistical analysis
Communication of results

Evaluation of an online software for your own research project
Make sure that the software…
•Does what you need
•Has certain methodical features (Conjoint; PSM; AHP)
•Can be connected to a panel / CRM-System / Facebook
•Gives you total control over the entire study process (example: no advertising from third parties; no access from outside; no logos etc.)
•Creates appealingquestionnaires
•Is accessible during and a certain time after the actual study (at least for more than 4 weeks / for free)
•Is exporting the data to SPSS/Excel
•Is easy to use (for surveymakerANDsurveytaker)

Online survey software should be online (SaaS) Some providers offer free access for non-commercial scientific research
http://www.gesis.org/dienstleistungen/methoden/beratungen/datenerhebung/online-umfragen/software-fuer-online-befragungen/#listings
Freeware:

Creatingmotivatingquestionnaires
3

Don’t worry about the duration. The first minute is the most important!

How do users perceive the interview experience? Leaving HTML-Standards towards bigger elements grows usability
HTML
AJAX

Why taking over paper pencil standards onto the color screen? New forms are available and show very positive effects
Simple Radio Button
Simple Slider Design
Slider Design with Graphical Support
Flash Scale
Lütters, H. / Westphal, D. / Heublein, F. (2007): SniperScale: Graphical scaling in data collection and its effect on the response behavior of participants in online studies; General Online Research Conference (GOR) 2007, Leipzig.

WithgrowingbandwithmoreappealingquestionnairescanbecreatedEffect: Higher willingnesstoparticipate

Lütters/Westphal/Heublein (2007): “The future of graphical scaling is interactive”
Flash interaction with movement and sound
Non-interactive scales with AJAX technology
Interactive tests of the team based on Flash technology in 2007
Examples based on Ajax technology

Audienceadopteddesign createshigherresponserates

Adopttotheaudienceandthedeviceused: ExampleoffirstiPad-onlystudy

The progress bar: Never ending story of misunderstandings
Summary:
•Has a strong effect (any kind of undesired effect is negative)
•Makes people thing about the duration more than they should
•Is only motivating when it is faked (53% after 2 questions)
•Faking vs. AAPOR/ESOMAR code of ethics
•People believe that it is a necessary part of a questionnaire. It’s not!
•Avoid the progress bar! Try the count-down…
http://conftool.gor.de/conftool11/index.php?page=downloadPaper&filename=Callegaro-hould_we_use_the_progress_bar_in_online_surveys_A_meta- analysis-151.pdf&form_id=151&form_index=&form_version=final&CTSID_GOR11=AgPbzZ3ygg6WVMGwtHF8Y0

Mobile research about to break through. Device Agnostic Research is the way to go on the long term

The questionnaire designed often looks different to interviewee than expected by the designer
Source:pangealabs;Dillman2000:360
Online
Questionnaire
Designer
Designer‘s
Hardware &
Software
Questionnaire
as seen by
Designer
Questionnaire
as seen by
Designer
Respondent‘s
Hardware &
Software
Online
Respondent
Questionnaire
asseenby
Respondent
Questionnaire
asseenby
Respondent

Invitingpeopletoonline questionnaires
4

Where will the participants be coming from? Why should they participate in such a survey?
•Employees
•Customers
•Company Website
•Other Website
•Convenient Sample (Ponzi scheme )
•Riversampling
•Panelrecruiting
•Offline
•…
1.Topic
2.Incentives(monetary / non-monetary)
3.Offering Results
4.Design
5.Anonymity
6.…
•Open vs..closed
•Quota features
•Plausibility check
•Consistency Check
•…
Controlling
of sources and quality of answers
Motivation
(to fully complete)
Sources of respondents

Motivation to complete starts even with the invitation approachInfluence on opening rate and click rate
Sarah?
Isabel?

SPAM as a central problem of email invitationsHTML vs. Plain Text:

Social Media Recruiting isan opportunitywhichneedssomeextra care
0
20
40
60
80
100
120
140
160
Anzahl der Starter/ Completeresponses
Completes Social Media
Starter Social Media
14.2.
23.8.
5.10.
Daily averagecompletionrate: 80%
First reactiontoblogposton web
First Invitation
asposton
adidas Running
Reminder
(Second Invitation)

Combining offline and online with incentivesThe GAP invites to the customer survey on the receipt.
http://outlet-feedback.gap.com/

Controlling of online interactions
5

Controlling isimportanttoseesuccessandunderstandeffects
date
total noofcontacts
quota full(interview terminated by questfox)
screened out(interview terminated by questfox)
Noofinterviewspossible
Completedn=
Completed%
26.08.2013
41
0
1
40
40
100 %
27.08.2013
175
0
3
172
159
92,4 %
28.08.2013
235
103
1
131
103
78,6 %
29.08.2013
1041
639
14
388
166
42,8 %
30.08.2013
1280
976
26
278
194
69,8 %

Quality isinsuredbymonitoring

Controlling eachquestiontellsaboutproblemsofthequestionnaire
questionpage
responsestoquestionpage
surveycompleted
Q1: ### Steuerung der Befragung
2772(99,9% oftotal)
662(23,9%)
Q2: Screener (Geschlecht)
2726(98,3% oftotal)
662(24,3%)
Q3: Screener (Alter)
2613(94,2% oftotal)
662(25,3%)
Q4: Screener (Bildung)
2597(93,6% of total)
662(25,5%)
Q5: Screener (Bundesland graphisch)
2576(92,9% oftotal)
662(25,7%)
Q6: Screener (Stadt oder Land)
2557(92,2% oftotal)
662(25,9%)
Q7: Screener (Location)
2547(91,8% oftotal)
662(26%)
Q8: Erfahrung mit Befragungen der Marktforschung
2536(91,4% of total)
662(26,1%)
Q9: Screener (Teilnahmebereitschaft)
2528(91,1% of total)
662(26,2%)
Q10: Screener (Device)
2773(100% of total)
662(23,9%)
T11: Politik, Liebe, Drogen
719(25,9% of total)
639(88,9%)
Q12: ### P01 -Regierung weiterempfehlen
707(25,5% of total)
641(90,7%)
Q13: P02 -Wo stehen Sie politisch?
700(25,2% of total)
644(92%)
Q14: P02_Sonntagsfrage
695(25,1% of total)
642(92,4%)
Q15: ### S01 -Beziehungsstatus
689(24,8% of total)
641(93%)

Quota control attempts to portray the planned number of representative cases and close the study in case of reached limits
Example quota monitoring during an ongoing study
Example quota achievement after completing a study

IP to location creates detailed reports for regions (without asking)
Result of a UK study recruited over a panel

Buyingtherespondents‘ willingnesstoparticipate
6

Example of a worst case scenario: Recruiting from company database2780 email invitations resulted in 27 completed interviews
PERSONAL INVITATION
INFO @ INVITATION
Source: B2B research project in EU research
Response to an international survey (6 languages) after one week
Σ=27

Recruitment: Make or Buy? Recruiting for the same study with a commercial panel
Σ=638
Connection to a commercial panel in the same period
638 full interviews from panel vs. 27 full interviews with company e-mail-addresses
Source: B2B research project in EU research

Many service providers are established in the new market of recruitment of participants (so-called online panel or MROC)

The price of ONE participant in an online study is a calculation based on duration and incidence of the targeted criteria
Example from SmartResearchin Euro: 1 €= 14 SAR
Number of questions
Panel
Number of Interviews
Incidence
9500 SAR
Drivers of the price
•Duration (calculate €0,10 cent per Minute) thus less than 15 rand for a complete 10 minutes interview
•Only complete interviews are paid
•2 Minutes sampling time allowed to define screen-outs and quota rules
Incidence:
•50% Male / Female
•Car Owner: 70%
•Owner of a Volkswagen: 10% (equal to market share)
•Owner of a VW Polo 1%
In words: you need to contact 100 people to get one polo driver into your study. And what if he/she drops out?

The willingness to participate is guaranteed through incentivesThis approach creates less topic involvement

Future of sampling: AYTM All-in-One-Solutions per Click

Google consumer surveys is a combination of survey making and sampling at the same time

panelfinder.com as a meta platform for all kinds of panels

Set up minimum standards and monitor your panel. Good Panels have full control over the relation to the participants

Summary: Panels are very functional and cause less organizational pain… but need to be paid
Hints:
•Always ask for 3 quotes from different companies and ask for their calculation of the INCIDENCE
•Only negotiate on full interview
•Start small (n=300) -> you can still buy more respondents later
•A panel telling you “no problem” might not be your first choice
Suggestion:
•Why not doing general research in the USA (cheapest) or Europe (widely available)?
•Why not building up the first university panel in South Africa?

A few tips for your online project: Serious fun®
•Select the online method only when it is appropriate (not for financial reasons)
•Try on-line studies: you will learn a lot about yourself and your relationship with employees and customers or other groups much quicker
•Take your time: An online project requires a much time as a classic data collection project. Perhaps a little more when doing it for the first time
•Emphasize more on methodology than on technology
•Take care of your population.They are interviewed more often than needed.Reconsider the number of required interviews in every study
•Take care of every single respondent. Rethink difficulty, design, duration for each project. Find an average between fun and torture.
•Think from the perspective of your participants: Try to take into account an external point of view throughout your project. Is it really as easy as you think?
•Remain critical: Do not just mindlessly accept approaches from the past or from offline into this new world.Go into a critical review process after a project
•Online is different.Online needs to be different!Take advantage of the benefits of a new world of data collection opportunities
•Be open to new ideas. The internet is faster than you are. Try to incorporate some of the brilliant ideas into your research

Newerformsofonline research
usedbyHTW Berlin Marketing Research Team

Eyetracking: Registeringpeopleslooks
Eyetrackingofa singleperson
HeatMapasaggregationofviews

Face Detection technology from the FraunhoferInstituteGfK EMO Scan isthecommericalisedversion
Idea: FacialScan
Field ofapplication
•Gender recognition
•Estimate of age
•Mood measurement
•Recognition of people in groups
•Recording the changes in mood on exposition to changing stimulus (ex. Video ad)
•Possible with simple technologyhttp://www.youtube.com/watch?v=Fs-ngMK09Vk

Listening isthenewasking(Poynter) Insightsfromsocialmediawithoutaskinganyoneanymore

Interaction through device movement: The Gyroscaleliterally moves answers
The Gyroscale® is available in questfox®
Source: www.pangealabs.comGyroscale® and questfox® are trademarks of pangea labs GmbH Switzerland

The avatars are empowered by text-to-speech technology which allows them to read out loud written texts
Different people have very different relations to ....
Text for reading:
Text is read by Avatar:
R1
R2
R3

Outlook intothefutureofonline research

Some trends from the US are online on ist way to Europe
http://issuu.com/researchshare/docs/grit-winter-2014

#newMRis discussing the future of online
http://www.mrweb.com/mrt/

Greenbookas a good online source to start with
http://www.greenbook.org/

What happens to market research, if the boundaries are broadened?
“In the future many professions and sectors will go the way of steam engine makers and the man who used to walk in front of cars carrying a red flag.
This might include the definition of market research as a business sector.”
Discussion under http://krz.ch/vkib

Thank you for your interest Feel free to contact me
HolgerLütters–HTW Berlin
Contact information
Prof.Dr.HolgerLütters
HochschulefürTechnikund WirtschaftBerlin
Holger.Luetters@htw-berlin.de
www.luetters.com Twitter @luetters
xing.com/profile/Holger_Luetters
de.linkedin.com/in/luetters/

Online Research Coming of Age

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (11)

Ähnlich wie Online Research Coming of Age

Ähnlich wie Online Research Coming of Age (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Online Research Coming of Age