Mobile surveys - The 5th methodology: a new paradigm or a strategic enhancement? Michael Francesco Alioto - Vice President, Marketing Sciences - Gongos Research Comparing smartphone-based and online survey data: assessing research-on-research done with Best Buy. Developing effective best practices for designing mobile-based quantitative research. Understanding when to use a mobile/online hybrid approach to surveys.

1. Mobile Surveys | The 5 th Methodology:
New Paradigm or Strategic Enhancement ?
Market Research in the Mobile World
19-20 July 2011: Atlanta
Michael Francesco Alioto
Vice President, Marketing Sciences

2. Surveying consumers in
their natural environment
What Role Will Mobile Play in the Future ?
Critical Study Objectives:
• Understand/confirm current mobile survey usability—
integrative hybrid or next survey platform?
• Confirm consumers’ usage of mobile devices in their
daily lives—communication and environment
• Confirm if surveys can be completed within an integrated
platform with similar results
• Compare and contrast smartphone and online surveys—
scale usage, results, incidence rates, survey length, etc.
• Assess potential to analyze data utilizing more advanced
analytical techniques (e.g., MaxDiff) on mobile devices

3. Path to understanding
research design
A Four-Phase Process
Approaches Toward Analysis:
PHASE ONE: PHASE TWO: PHASE THREE: PHASE FOUR:
Desk Usage & Survey Survey Advanced
Research Lifestyle Design Results Statistical
Comparison Comparison Analysis
Review literature, Confirm
Flash Polls & Smartphone survey
conventional Confirm data comparability of
vs. online version
thoughts, and Dialogues comparability and more complex
secondary data; n=1,006
n=496 scope across quantitative
establish baseline (495 smartphone/
platforms techniques
511 online)
In partnership with

4. PHASE ONE:
Current understanding
and benchmarking
Desk Research: Key Findings
Macro Trends Micro Trends
• Two schools of thought: strategic enhancement— • Data comparability between online and mobile
online supplement vs. platform evolution— survey platforms debatable— supplement vs.
5th Methodology independent platform
• Tendency toward hybrid platforms and data • Critical limitations concerning mobile research
integration, but results and strategy not conclusive identified: e.g. survey length, exhibits, types of
questions, scales, incentives
• Suggests major limitations for mobile survey
component of hybrid platform—device/technology, • Mobile segments illustrate personal differences:
research design elements Millennials, Multicultural
• Academic community mostly silent on the issue • Online sample providers just beginning to
understand/get comfortable with mobile
• 5th Methodology as mainstream future platform— surveys (incidences, costing, viability)
will be driven by behavior and mobile device
technological improvements

5. PHASE TWO:
The mobile device
lifestyle integration
Usage & Lifestyle: Key Flashpoll Findings
Smartphone Lifestyle Usage
Smartphone Usage/Integration:
• Placing phone calls, texting, sending e-mails, and At home 94%
Internet searches are in the top five usage
categories—ideal for communication
At work 70%
• Supports desk research concerning smartphones
as “connectivity to the world” In the car 62%
• Smartphone usage is well integrated into daily
lifestyle (home, work, vehicle) While shopping 61%
• Smartphones used extensively while shopping Internet café/
35%
Coffee house
• Smartphones used to conduct a wide variety of
tasks, many times simultaneously At school 24%
• Smartphones used throughout the day/within
periods of “constant interruption” Other 19%

6. PHASE THREE:
Setting the stage
quantitative survey
Questionnaire and Design: Key Elements
Design elements: Scales types:
 Response Rates  Multiple Mention
 Incentive Structure  Single Response
 Survey Timing  Three-point, fully-anchored
 Open-Ended Questions  Five-point, fully-anchored
 Respondent Assessment (Worldview)  Five-point, end-anchored
 Technology Limitations  Five-point, semantic differential
 Six-point, fully-anchored
Survey subjects:
 Internet Sites Visited, Grids
 Store Display, Image Testing Disagree Agree
 Mobile Preferences, MaxDiff Completely Completely
 Concept Evaluation 1 5
 Demographics
 Online Survey Assessment
 Mobile Survey Assessment
 Best Buy Segmentation

7. PHASE THREE:
Key Comparisons
quantitative survey
Design Elements: Key Findings
Traditional Research
Thoughts Results
Significantly lower for No difference between
RESPONSE RATES
smartphones platforms
Higher incentives required INCENTIVES Higher incentives not required
for smartphones for smartphones
Restricted for smartphones TIMING Not as restricted as thought
Difficult for smartphones VERBATIMS Minimal verbatims possible
Much different for Different, but thought to be
WORLDVIEW
smartphone users converging
Limited significantly TECHNOLOGY Limited by device

8. PHASE THREE:
Comparison of Results
quantitative survey
Questionnaire Design: Key Findings
Image Affinity: 3-Point Scale Respondents were asked how much they liked a
store display using a 3-point scale, where:
1 = Dislike it
2 = Feel neutral about it
3 = Like it
25%
40% 41% Like vs. Dislike
A A
Like it
Feel neutral about it
66% Dislike it
49% 49%
B,C
9% 11% 10%
Online: No Online: Have Smartphone
Smartphone Smartphone (C )
(A) (B)
n=330 n=174 n=487
Significance testing at the 95% confidence level
“Using the scale below, overall, how much do you like this store display?”

9. PHASE THREE:
Comparison of Results
quantitative survey
Questionnaire Design: Key Findings
Concept Test: 5-Point Fully Anchored Respondents were asked to rate the
concept on a 5-point anchored scale
A
B,C A
11% A
T2B = 39% 24% 24%
A A
28% T2B = 64%
A T2B = 67%
C
C 5 Strongly like it
40% 43%
A 4 Somewhat like it
A
B,C 3 Neither like it nor dislike it
47%
B,C 2 Somewhat dislike it
31% 24% 1 Strongly dislike it
C
7% B B,C
8%B,C 2% A 7% B A A,B
3% 2%
Online: No Online: Have Smartphone (C )
Smartphone (A) Smartphone (B) Significance testing at the 95% confidence level
n=335 n=176 n=495
“Which of the following statements best describes how much you like or dislike this product/idea?”

10. PHASE THREE:
Comparison of Results
quantitative survey
Questionnaire Design: Key Findings
Display Appearance: 5-Point End Anchored Respondents were asked to rate the
display table on a 5-point scale, with:
1 = Disagree Completely
5 = Agree Completely
15% 20% 23%
27%
33% 33% Agree Completely
4
31%
C 24% 21% 3
2
18% 14% 16%
Disagree Completely
8% 8% 7%
Online: No Online: Have Smartphone
Smartphone Smartphone (C)
(A) (B)
n=300 n=174 n=487 Significance testing at the 95% confidence level
“The table is inviting and I would want to look at the products displayed.”

11. PHASE THREE:
Comparison of Results
quantitative survey
Questionnaire Design: Key Findings
Clearly View Image: Respondents were asked to rate the image on
Semantic Differential Mean Summary four 5-point differential semantic scales.
Online: Have
Online: No Smartphone Smartphone
Smartphone
Sample Size n=312 n=174 n=487
(1) Organized vs. (5) Messy 2.26 2.18 2.30
(1) Appealing vs.(5) Unappealing 2.65* 2.40 2.53
(1) Stands out vs. (5) Blends in 2.72 2.57 2.68
(1) Clear vs. (5) Confusing 2.31 2.28 2.33
* Significantly Higher than 1 other group
Significance testing at the 95% confidence level
“Below is a list of attributes with differing meanings. For each pair, please
select the one attribute that best describes the store display.”

12. PHASE FOUR:
Quantitative comparisons
statistical analysis
MaxDiff Analysis: phone Advanced Analytics: Key Findings
feature rankings Respondents completed a MaxDiff exercise where they ranked various mobile phone features,
including: brand, price, battery life, size, features (e.g. camera, video) and memory.
B,C
4% 6% A 8% A,B
25%
30%
A 36%
11% C A,B Memory
10% C Features
19% 8%
B,C Size
17%
15%
Battery
33% 26% 21% Price
B,C C
A
Brand
8% 12% A 13%A
Online, No Online, Have Smartphone (C)
Smartphone (A) Smartphone (B)
n=495
n=335 n=176
Significance testing at the 95% confidence level
“Of the attributes listed below, which one is most important when choosing
a mobile phone?”

13. PHASE THREE:
Survey comparisons
respondent participation
Survey Satisfaction: Key Findings
Mobile Online
99%
90%
76%
73%
68% 66% 69%
66%
Survey Enjoyment Willingness to Participate Willingness to go to Fair Incentive
in Future Specified Location/Store to
Smartphone/Online Survey Participate
Smartphone: n=495 On-Line: n=511
NOTE: Top two boxes on a five-point scale.

14. PHASE THREE:
Mobile Survey
respondent participation
Survey Satisfaction: Key Verbatims
LIKES DISLIKES
Anywhere at anytime Difficulty typing/Small keyboard
• “I don't have to be at home or in front of the computer. I • “Don't like typing on my phone cause the keyboard is too
can do this anywhere.” small.”
• “Ease of being able to complete the survey from anywhere. • “Excessive typing can be a problem. I'd prefer longer, essay
Not needing to be tethered to my desktop.” response answers for the computer.”
Convenience Pages load slowly/Too long/Drained battery
• “I always have my phone on and with me. Not so for the • “Pages load slowly, takes longer than computer.”
computer. So, convenience.” • “Too much time, wastes battery life.”
• “It's convenient, and I can get them done quickly because I • “Doing anything on the web on my phone takes longer.”
always have my phone with me.”
Screen too small, survey content too much for screen
Easy to do, easy to access
• “The screen on my phone is too small to see the questions
• “Easy to complete and does not require more effort than a and the answers at the same time.”
survey taken on a computer.”
• “My blackberry is too small to comfortably take the
• “First time taking survey on mobile – it was easy to follow survey.”
and easy to get thru, not confusing in the least.”
• “The survey was made bigger than my screen and it was
Fun, different, new hard to select my choices at times.”
• “It was fun and easy to answer the questions. Normally I Too much scrolling
take surveys on my laptop, so using a different method
• “It takes so long, and I have to scroll left and right, up and
such as a mobile phone was interesting.”
down to read the question.”
• “It was new, interesting, and fun!”
Prefer participating from computer
• “It’s easier to do most things on my computer.”

15. Critical Findings
quantitative survey
Design Elements – Scale Types
Response rates are similar across platforms.
Males trend directionally higher.
Smartphone respondents will not require higher
or significantly different incentives.
There are both technology limitations and different “worldviews” for
smartphone users, but this should dissipate in the near future.
Smartphone platforms can viably support 5-point fully-anchored scales, but
3-point anchored and 5-point end-anchored scales are recommended.
Scale usage does not differ significantly, supporting the use of a hybrid
methodology and the integration of mobile and online platforms.
Statistical techniques have similar results across survey platforms, supporting
the smartphone platform for in-store, real-time data collection.

16. Smartphone platform
critical implications
Smartphone surveys are a viable option to supplement studies
with hard-to-reach segments of the population
• Difficulty typing/Small keyboard
• “Don't like typing on my phone cause the keyboard is
Data and scale usage is comparable across platforms,
too small.”
supporting the use of hybrid designs
• “Excessive typing can be a problem. I'd prefer longer,
essay response answers for the computer.”
• Pages load slowly/Too long/Drained battery
• “Pages load slowly, takes longer than computer.”
Smartphone users today tend to be more tech-savvy with battery life.”
• “Too much time, wastes unique
shopping, usage and behavior preferences regardless the platform phone takes longer.”
• “Doing anything on of web on my
• Screen too small, smaller text, survey content too much
for screen
• “The screen on my phone is too small to see the
questions and the answers at the same time.”
Smartphone platform may more readily support specific subgroup
segments, i.e. Hispanic, Millennials, “Mywell as global small to comfortably take the
•
as blackberry is too markets
survey.”
• “The survey was made bigger than my screen and it
was hard to select my choices at times.”
• Too much scrolling
Mobile-enabled surveys are ideal for in-store shopping and exercises,left and right, up
• “It takes so long and I have to scroll
real-time assessment, and interruptedto read the question.”
and down surveying
• Prefer participating from computer
• “It’s easier and faster to do most things on my
computer.”

17. Mobile Research
considerations for the future
The “Strategic Enhancement” of today could well be
the next paradigm—or the 5th methodology—of
tomorrow.
• How will the marketing research industry prepare for this
evolution?
• Does it reduce barriers to reaching specialized segments
(Millennials, Hispanics, global markets?)
• Will this offer greater authenticity to marketing
research (rational/emotional surveying). Are
smartphone devices more conducive for lifestyle-based
interviewing?
• Does it support the evolution of the marketing research
discipline from the current scientific/data-based structure
to a humanistic/decision-making approach?

18. Mobile Surveys | The 5 th Methodology:
A New Paradigm or a Strategic Enhancement ?
q&a | discussion
Michael Francesco Alioto, Ph.D.
Vice President
Marketing Sciences
Gongos Research
Greg Heist
Vice President
Research Innovation
Gongos Research

19. Presented at:
Market Research in the
Mobile World
2nd International Conference | July 19 & 20, 2011 Atlanta
Organized by: Thank you to sponsors:
LinkedIn Group: Mobile MR
Upcoming Merlien Events: Merlien.org GreenBook Directory: GreenBook.org
Market Research Blog: GreenBookBlog.org New Qual Blog & Directory: NewQualitative.org
Get Inspired. Stay Informed. Cut Through the Clutter. Sign up for our free email newsletter