Presentation held at General Online Research conference 2009 (GOR09)

1. Agenda
1 Welcome and round of introduction
Online surveys at a glance
2 - Trends and facts at your fingertips -
3 Practical Introduction
Advanced features of online surveys
4 - Which tools to use when and how -
Discussion
5 - Final Discussion and Exchange -

2. Trends towards in-house market research
2

3. Development of in-house market research
Online surveys in companies up to 2003: Slow start
Relatively limited range of applications up to 2003 (website checks, ad-hoc projects)
The same method-focused approach as applied by institutes
Used almost exclusively by market research departments
Only a few major innovative projects are introduced by companies
3

4. Development of in-house market research
Online surveys in companies from 2003: Fast, decentralized
development
Above-average growth in companies
Internet diffusion
Data basis
Key to sales/communication
Wide-ranging spectrum (marketing, HR, QM, development, services, etc.)
High efficiency and effectiveness in individual projects
Often insufficient integration into the company strategy; little operational
transparency
4

5. Factors in the success of in-house
online market research
Broad level of Internet use in the consumption-relevant target groups
(Source: ARD/ZDF study)
14 – 19 years: 96 %
20 – 29 years: 86 %
30 – 39 years: 77 %
40 – 49 years: 70 %
50 – 59 years: 58 %
Decline in acceptance of telephone surveys (CATI)
Decline in number of landlines
Online surveys are more compatible with the new culture
Increased demand for customer and market proximity (feedback culture)
Continued improvements in corporate databases
Web shop
Customer clubs
CRM and customer databases
Boom in sampling panels (ciao, respondi, webfrager, eCircle, etc.)
5

6. Current trends in in-house
online market research
Above-average growth in in-house online market research
Online customers/market research panels (ad hoc and longitudinal)
Personalized and anonymous online surveys
Automation of survey and analysis processes
Broad area of application: Website checks, advertising impact, product tests,
price research, customer satisfaction, QM, feedback
Forrester Research
quot;At least one-quarter of Fortune 100 companies will launch online customer
communities this year (2008) in order to create higher levels of engagement
with their customers and prospectsquot;
quot;More and more large companies are moving market research in-house using
do-it-yourself technologiesquot;
6

7. Online-Surveys
7

8. What happens when filling out the
questionnaire?
Fill out
1. page Saving answers
Loading 2. page 2. page Fill out
The given answers are saved by the server after every survey
page and the next page is displayed
8

9. Access to online surveys
Pop-ups/layers
Survey
Link on website
E-mail invitation
9

10. Anonymous and personalized surveys
Anonymous survey
Online survey while the user is visiting a website
• Inivitation to participate in a survey by pop-up window on entering or
leaving the site
Cookies prevent users from participating more than once
Trend: Semi-automatic survey with trigger
n-th visitor procedure
Survey opens in a Survey is managed via
pop-up quot;EFS Surveyquot;
Anonymous user
accesses the
website
10

11. Access to anonymous surveys
- EFS Survey project type: anonymous
Popup Banner
Layer
Link on website
11

12. Anonymous and personalized surveys
Personalized survey
Importing address data and additional information in EFS survey
Automatic generation of individual access codes
Area of application: Customer surveys
Higher return rate than for anonymous surveys
Personal
invitations by
e-mail
Survey is managed via
quot;EFS Surveyquot;
User data is imported
into EFS Survey
Personalized user
performs survey
12

13. Access to personalized surveys
- EFS Survey project type: personalized
Link and password
Personalized link
13

14. Type of projects
anonymous personalized
Implement popup window on Inviting participants by a
website personalized e-mail
Publish the survey link on a website Password is hidden in the link
Publish the survey link in the User will be asked to enter the
newsletter password
14

15. Online Surveys
Facts at your fingertips
15

16. The basic outset
Instrument
Self-adminstered questionnaire
No assistng interviewer
Filling in and finishing the questionnaire based on the impression of the
questionnaire
Content
Needs a clear structure
Layout
Questions and additional text
Challenges
Use of multi media elements (pictures, sounds, movies)
Accessability (e.g. blind participants)
Tech (bandwidth, screen resolution, Flash)
16

17. Advantages of online questionnaires
Preparation
No need to buy letters and stamps or hire interviewers
No interviewer training necessary
Can be created in short time
In the field
Quick feedback
Mandatory questions
Check answers for plausibility
Automated sampling possible (quotas)
Pictures, sounds or Movies can be displayed
Realtime field report and statistics
Data preparation
Results data is readily available in the format desired
Open answers are readily readable
Tables and presentations can be produced directly 17

18. Possible errors in online survey
(and surveys in general)
Coverage Error
Difference between the sample group and the sample population
Nonresponse Error
Certain subgroups of the sample didn‘t take part.
Unit-Nonresponse / Item-Nonresponse
Measurement Error
Error caused by the instrument of measure (e.g. faulty questions)
The measurement is incorrect (reliability)
The wrong item has been measured (validity)
Groves, Robert M. (1987): „Research on survey data quality“. In: Public Opinion Quarterly 51, 156-172
18

19. Answer behavior based on the questionnaire as a
whole
Basic approach of Web Usability Engineering
Users don’t want to scroll
Participants want to scroll. They want to see the
whole page at once.
Users don’t want to read
Participants most likely only take notice of
highlighted text elements.
Of the non-highlighted text the upper left has the
highest probability of being read.
Most other text will be ignored
Text from paper surveys should be cut by at least
50%.
19

20. Considerations
When creating an online survey one has to consider
The 10 Golden Rules
Not so apt internet users
Java-Script avaiable
Flash available
Screen resolution
Slow connection issues (e.g. in Russia)
20

21. The Ten Golden Rules (1-5)
Keep it simple
(avoid foreign words, technical terms, initials or slang)
Keep it short
(avoid long sentences or sentence compounds)
Keep it tangible
(avoid abstract terms)
Keep it honest
(avoid leading questions)
Keep it neutral
(avoid controversial or loaded terms)
21

22. The Ten Golden Rules (6-10)
Keep it real
(do not you hypothetical questions)
Keep it focussed
(adress one item at the time, avoid „and“, „or“,“because“, etc.)
Keep it clear
(avoid double negations)
Keep it manageable
(do not demand too much from the participants)
Keep it balanced
(try to use an equal amount of positive and negative answers)
22

23. Question types
Open ended questions excluded
One can differentiate between 5 basic question types:
Single response questions
Multi response questions
Matrix questions
Drop-down questions
Others (mostly combinations or variations of the
above)
23

24. Question types: Single response question
Single response question:
• Straight forward question type
• Uses radio buttons
• Mind that it can’t be deselected!
• Easy to realize and answer
• Too many identical questions will become boring (Loop)
24

25. Question types: Multi response questions
Multi reponse question:
• Similar to singlepunch question
• Allows for multiple answers
• Can be deselected
• Issue of the residual category (“None of the above”)
25

26. Question types: Matrix question
Matrix question:
• More efficient then several singlepunch questions
• A lot of input on one screen
• Randomization is advised
• Limited number of statements is advised
• Might become frustrating if mandatory
• Might become boring if asked a lot in a row
26

27. Question types: Drop-down question
Drop-down question:
• Basically a variation of single
response question
• Unique to online questionnaires
• Needs a “Please select” category
• Essential: Not all answer options
are displayed at once
• Ideal for long answerlists
• Best attention grabber when used
as mandatory question
(also see Dr. Mirta Galesic et al.’s work Eye-Tracking)
27

28. Question types: Variations
E.g. Image questions:
• Variation of the single/multi
response question
• Instead of buttons one clicks on
pictures
28

29. Question types: Combination
E.g. Drop-down Matrix:
• Combination of Drop-down and
Matrix question
• Even more input then Matrix
question (3 dimensions)
29

30. Practical introduction
The first survey project
- Sequence of operations -
30

31. Process sequence:
Online survey in 7 steps
1. Create project
2. Enter questions
3. Activate project
4. Test project
5. Invite participants
6. Control progress
7. Export data
31

32. Question editor: Working on 3 levels
Level 1: Level 2: Level 3:
Structure view Page view* Question view
(page list)
Question 3: Family status
Page 1: Personal data
Project: Soziodemographic
question
survey
Question 1: Question text:
Page 1:
Gender Please enter your
Personal data questions
family status.
Question 2:
Page 2:
Age
Employment status Answer 1
not married
Question 3:
Page 3:
Answer 2
Family status
Income status
married
Question 4:
Page 4: Answer 3
Nationality
Household structure divorced/widowed
*1 page = 1 survey screen
32

33. Access to administration area
Administrator creates projects in the password-protected login area:
Username and password
must be entered to login.
http://schulung.efs-survey.com/www/
33

34. Screen structure
1 3
Navigation bar with modules Menu of the current module
Breadcrumb / path indication Content
2 4
1
2
3
4
34

35. Online Surveys
Advanced Features
35

36. Advanced features of
an online survey
Online survey offer many additional features to the fray
which cannot normally be used in all other survey
variants:
URL/GET-Parameters and/or participants
administration
Mandatory questions
Plausibility checks
Randomization
Dynamics in the questionnaire through routing,
placeholders, dynamic lists
36

37. GET-/URL-parameters
GET-/URL-parameter
GET/URL-Parameter
• Are one way to relay data about the participant to the results data prior to
the start of the questionnaire
• Commonly used for anonymous projects
• They are essential when dealing with panel providers
• E.g. If you are recruiting from several different sites and you want to
differentiate from which site each participant came from
37

38. Participants administration
Participants administration
• Commonly used in personalized projects
• A participants administration should ideally be able to contain all the
additional variables which are needed for routing and data presentation for
each participant. Thus making redundant questions unnecessary.
• As well an panelist administration should enable you to invite and remind
38
all or selected participants.

39. Mandatory questions
Mandatory question
• Enable you to ensure that certain or all questions are filled in.
• Can but should not be used with open ended questions
• Might significantly lower the response rate or the response quality
• Involvement of the participants can counter this effect (e.g. employee
survey)
• Incentives will make the response rate to stay but will lower the quality 39

40. Plausibility Checks
Plausibility Checks
• Enable you to ensure that certain or all questions are filled in correctly.
• Here one should be especially cautious to test because an error here
might ruin the whole study
• Avoid checking to complex issues with these checks. Rather simplify
them.
40

41. Exclusive Checkbox
Exclusive
Checkbox
• Special solution to
solve the issue of
residual categories in
multi response
questions
• Requires Java-
Script
• If available it is
preferable to checking
with a plausibility
By clicking =>
deselect the
check.
selection
Activate in
answer options
41

42. Randomization of Items
Randomization
• Randomization of Items
significantly lowers order
effects
• Additional functionalities
like Grouping can allow for
the participant to have an
overview while still being
shown a randomized list of
items
• Another approach is the
“flip” which simply mirrors
scales.
42

43. Filter/ Routing
• Filters and Routing are one of the major advantages of online
questionnaires compared to paper&pencil.
• The participants will simply not see the pages they are not supposed to
see.
• Again, testing is crucial at this point. As with a faulty filter data will be
inevitably lost.
43

44. Placeholder
Placeholders are a simple and straight forward way to have a dynamic
questiontext. Thus sparing the need to create multiple questions (and
variables) just to have adifferent wording
44

45. Dynamic questionnaire structure (lists)
Dynamics
• As a more sophisticated version of filters further dynamics can be applied.
• E.g. the answers to one questions determine the answer-options in the
following question
• One should be aware of avoiding participants seeing empty pages
45

46. Dynamic questionnaire structure (loops)
Loops
• A Loop is a function that allows to automatically generate multiple
repeating questions from the answers to a source question.
• The particular risk here is that with too many questions asked in a row
either the response quality will decrease or the drop-out rate will increase or
both.
46

47. Further possibilities
• Recoding of data in the questionniare.
• Quotas and Screen-outs
• Dia-show
• Prefilling Form Elements
• Sending out mails from the questionnaire (e.g. “Tell a Friend“)
• Questionnaire receipt
47

48. Working with the platform
Setting up Working with
Field phase Analysis Follow-up
the platform the platform
Enter or import
participants at
department level
Create personalized
invitation for
participants
Define printing and
shipping points for
paper-based
questionnaires
48

49. Automation / CRM /
transaction-based feedback
49

50. Challenge:
Transaction-based feedback
Transaction
- Sales
Company Customer
- Consulting
Feedback?
Feedback is a prerequisite for optimizing products, services, advertising campaigns,
support services, etc.
If feedback is not requested, customers are likely to tell friends what they think and/or post
information in the Internet (forums, blogs, ratings).
People are ten times more likely to talk about negative experiences than
•
positive ones!
50

51. Standard process for automated
online surveys
Trigger
1 e.g. transaction
CRM / ERP 2
integration Data management: Survey
3
and participants
Report distribution
Invitation
4
8
Report
Field phase
5
Reporting 7
6
Real-time check
51

52. Interface to customer database
CRM / ERP
CRM / ERP EFS
interface Web Services
CRM / CC database manages customer transactions
Simple connection via EFS Web Services
Conculding transaction = ticket/event closed
Relevant customer data is transferred automatically
52

53. Participant management & e-mail invitations
Participant management
• Export random sample
• Import to participant management in EFS
• Generate customised survey codes & URLs
• Generate personal e-mails with title, first
name, last name and URL
• Take into account panelists who have been
filtered out or who are no longer registered
Integrated e-mail tool
E-mail is either sent to the customer
immediately or at a time specified
53

54. Field phase
High-end solution: EFS Survey
• CI-compliant layout
• Password/code protection
• SSL security
• Plausibility checks during data
entry
• Personalization of the questionnaire
• Complex questionnaire with a high
level of quality
54

55. Automatic crisis management
Process chain for negative assessments
An e-mail is sent to a
E-mail
sales employee or the
Trigger person responsible for the
process
Customer complaint
• Customer number
• Process
• Trigger
• etc.
An action is triggered if values in the questionnaire exceed or fall short
of specified limits (e.g. an infomail is sent to the service employee
responsible)‫‏‬
55

56. Reporting
Field report
Online reporting center
Return checks
Reporting platform with separate
Time-dependent
log-in for employees and customers
access
All the functions of a field report
Termination rate/page-dependent
Report downloads
termination
Survey frequencies & times
Participants who have been filtered out
Automatic reports Data export
Results reports for decision-makers Export in all common data formats
Index calculation Interface to evaluation software
Download in reporting center
56

57. Example: After-sales feedback
Automated customer survey
1 B2B platform
Challenge:
Increase customer loyalty
Implementation:
3 2 1. Customer buys a product
2. Transaction is reported automatically to EFS Survey
3. EFS Survey automatically sends questionnaires
tailored to the transaction (product, customer) to
customers. Customer provides online feedback.
Customer care
management 4. Reporting to customer care management, PM or
4 similar
5. Specific data is transferred automatically to the
CRM system
5 Benefit:
Increases customer loyalty
quot;Early-warning systemquot; for customer satisfaction
Information for quality and innovation management
CRM
system
57

58. Example: Call center feedback
1 Automated customer survey
Customer care
Challenge:
Increase customer loyalty
Implementation:
1. Customer uses online or telephone support
4 2 service. Customer center solves the problem.
2. The CRM system records the process
6 3. The CRM system reports the process
CRM automatically to EFS Survey
system
3 4. EFS Survey automatically invites customers to
take part in an online survey; consumer provides
feedback
5 5. Reporting sent to the customer care management
6. Results data is transferred automatically
Customer care Benefit:
management
Increase in customer loyalty secures sales
through:
Optimization of the service quality
quot;Early-warning systemquot; for customer
58
satisfaction

59. Feedback triggers continuous
improvement
Collect / evaluate
data
Continuous
optimization process
Check / analyze
Introduce
measures
59

60. The Unipark program
Support program for university research
institutes and students
Feedback channel for academic users
Over 300 cooperation projects with university
chairs
Europe
U.S.
Canada
Asia 60