Weitere ähnliche Inhalte Kürzlich hochgeladen (20) Remote research on mobile devices qrca online sig 11.19.20151. 1BellaVia Research ©2015
Julie Francis, UX Researcher & Founder / BellaVia Research
November 2015
Moderated Remote Research
on Mobile Devices
(aka, Stay off planes & save client money!)
QRCA Online SIG
2. 2BellaVia Research ©2015
Who I am & what I’ll share with you today
Julie Francis / Founder, BellaVia Research
20+ years in research
Focus on User Experience Research since 2000
Launched BellaVia in 2001, post Creative Good
Member of QRCA & UXPA
Why I needed a solution for remote moderated
research on mobile devices (and what that means)
Case study
Options + Pros & Cons of each
Tips & Tricks
5. 5BellaVia Research ©2015
So many research platforms!
Today’s focus:
Moderated
•Not unmoderated, and
Remote
•Not in-person, and
Real-time observation
•Not after-the-fact recordings,
and
Visibility to mobile
devices
•Not just computers
6. 6BellaVia Research ©2015
Case Study: multi-channel for engagement rings
Ritani needed a holistic
understanding of the UX.
Engagement ring shopping
is a multi-channel
experience.
• Guys look online.
• They use their smartphones
& tablets & computers.
• They shop in stores.
……But how? And why?
Decision
&
purchase
7. 7BellaVia Research ©2015
Case Study: Ritani.com
Buying an engagement ring
is a very unique purchase.
• They are expensive.
• There are a lot of options in
stores and online. There’s
a lot to learn & explore &
consider
• The decision takes place
over a long period of time.
This is not like
shopping for a t-shirt!
8. 8BellaVia Research ©2015
Why not in-person research?
Money.
Longitudinal nature of ring-
buying required longitudinal
study.
Geographic mix. Market
variations, retail store
partners vary by market.
Hotels and airplanes are
expensive.
Start-up budget
9. 9BellaVia Research ©2015
Why not in-person research?
Observation.
Only a few people can observe
in-person research, so it’s
“hidden” from many on the team.
Ritani wanted a solution that
enabled widespread, active
participation by their entire team
(without breaking the bank).
Remote research is easy to
observe. Lots of participation
drives empathy & innovation.
10. 10BellaVia Research ©2015
Why not unmoderated remote research?
It’s emotional. Not all people
readily share their feelings.
• Unmoderated, self-reported would
not be a good fit.
We needed the ability to easily
go off script and go deep.
Probing was essential. It was all
about the observation, and all
about the probes.
How do I pick a
diamond? Where
do I start? Who
can I trust? What
will she like?
Depth of insights.
11. 11BellaVia Research ©2015
Ritani needed:
Geographic mix
• No problem. We’ll use
screen sharing & a
remote methodology, so
we can include ring
shoppers nationwide
Moderated,
in-depth
exploration
• No problem. We
won’t use an
unmoderated tool or
a diary.
Real-time
visibility to
mobile devices
• This is where it gets
tricky…tablet +
smartphone?
?
But
how?
12. 12BellaVia Research ©2015
What we did
3 rounds of research
(longitudinal purchase),
•>4 hours of interviewing each
14 ring shoppers,
early in purchase funnel (just
starting their shopping)
Nationwide …And we found an approach
for remote moderated
research with visibility to
mobile devices! wooohooo!
13. 13BellaVia Research ©2015
Landscape: Remote Research
Computer
online
Mobile
devices
moderated
Un-
moderated
Same players as un-
moderated for mobile…
If they can do it on
mobile, they can do it on
desktop, too
14. 14BellaVia Research ©2015
Considerations when choosing a solution
Interactions?
• How important is it to
see fingers on screen?
Non-verbals?
• Do you need to see the
user’s face?
Stimuli?
• Mobile web?
• Apps?
• Native phone
functionality?
Platform?
• Android?
• iOS?
Who are your
users?
• Tech savvy? Not?
15. 15BellaVia Research ©2015
Option 1: IT Support Tool: e.g., Bomgar
How it works: Participant installs
Bomgar app. Enters session number
provided by researcher. Uses Bomgar
browser to access site or Android app.
Very clear view of device screen
Affordable if regularly used: single-
user license: >$1100 per year
Cannot test iOS apps: only Android
apps (and mobile web)
Slow screen refresh makes it hard to
see interactions
Not designed for research
(odd verbiage for participant)
Similar option: TeamViewer
Researcher view
Thanks to Nyle Feist of UX Guys for the demo!
16. 16BellaVia Research ©2015
Option 1: IT Support Tool: e.g., Bomgar
Cannot see finger
interaction
Cannot see user’s face
The built-in browser in the
Bomgar app eats screen
real estate (address bar at
top, action buttons at
bottom)
Ritani.com
via Bomgar app
Participant view Ritani.com
17. 17BellaVia Research ©2015
Option 2: document camera, e.g., Ipevo
How it works: Mail a document camera to your
particiapnt. Participant plugs in (USB). Start screen-
sharing tool.
Can see the interaction (fingers touching screen)
Easy to switch between devices
You can see their smartwatch!
BUT…you can’t see the user’s face. Can’t use
a document camera and webcam simultaneously
Note: According to UXPA 2015 session attendees, you can see the
user’s face if the document camera is used in conjunction with a
webcam and Morae Observer. A Morae license is $2,000 per year.
Logistics / fulfillment hassle: Need to send
each participant a document camera (budget:
$100 per device with shipping).
Vendors like UserTesting.com can help & have a panel.
Civicom will also ship these cameras.
Photo from ipevo.com
Thanks to session attendees for the info about document
cameras used in conjunction with Morae Observer!!
19. 19BellaVia Research ©2015
Option 3: laptop love!
How it works: Participant turns their laptop
around & “hugs” it. Use webcam & screen
sharing tool to show their mobile device.
Free! (if user has a webcam)
Can see the interaction (fingers touching
screen)
Very easy to switch between devices
BUT…you can’t see the person’s face when
they are snuggling their laptop
Challenging for participant to keep device in
webcam view (participant can’t see what the
webcam is capturing)
Image quality can be an issue (focus issues,
low quality webcams)
20. 20BellaVia Research ©2015
Option 4: Lookback’s Unicorns.io
How it works: Participant
installs Unicorns.io on Mac.
Signs in via Twitter
credentials. Connects device
via USB cord. Clicks
“stream.” Shares link with
researcher.
Free download!
(currently)
Very clear image
Streams the video &
audio
If used with screen sharing
tool, can also see the
participant’s face
21. 21BellaVia Research ©2015
Option 4: Lookback’s Unicorns.io
Only works for participants who have
Apple devices and a Mac running recent
software
•only iOS 8+ (80%)
•only Mac OSX 10.10+ (Yosemite) (60%)
Participant must have Twitter account
Cannot see finger interaction
- Note: A similar option was volunteered by a
UXPA 2015 session attendee. X-Mirage is
another iOs-only option that he feels works well.
He recommended Zoom for screen sharing.*
Thanks to the session attendee who recommended x-mirage and zoom!
22. 22BellaVia Research ©2015
Option 5: AirServer + AirPlay
How it works: Participant installs &
launches AirServer on their computer.
Swipe up on iOS device to start AirPlay.
Device is wirelessly mirrored to computer
screen. Start screensharing & webcam, &
shares screen.
Affordable: $15 per download per
participant
Very clear image
Can see the participant’s face via
webcam (screen sharing tool)
User can easily toggle between
computer, iPhone, iPad
23. 23BellaVia Research ©2015
Option 5: AirServer + AirPlay
Only works for
iOS
Only works on
iPhone 4S+ and
iPad 2+
Users may need
a second phone
(some iPhones
cannot mirror & be
used for voice
conversations
simultaneously)
24. 24BellaVia Research ©2015
Option 6: Reflector
How it works: Participant installs &
launches the Reflector app on their
computer.
Ensures computer and device are on
same WiFi network. Shares screen
via screen sharing. Starts webcam.
iOS: Swipes up to launch
AirPlay
Android: launch Screencast
app. (older Android devices
require ChromeCast)
Wirelessly mirrors device to computer
screen.
25. 25BellaVia Research ©2015
Option 6: Reflector
Very clear view of device
Free! 7 days free per user
Visibility to everything (apps, mobile
websites, native phone function)
Ability to see user’s face via webcam &
screen sharing
Cannot see finger interaction on screen
Users may need a second phone (some
phones cannot mirror & talk simultaneously)
26. 26BellaVia Research ©2015
Reflector: In the field
MindBody.com has been in the
field using this technique this
week. It’s going great!
They aren’t crazy about the
Reflector “watermark” – probably
because they are using the free
version?
Thanks to Marianne Connor and Amaya Becvar Weddle from Mind Body Online for this participant quote!
It was fine, I’m really, really
nervous with tech stuff, so
the fact that this worked is
impressive. That never
happens for me. Ever.
Participants say it’s easy!
27. 27BellaVia Research ©2015
Pros & Cons: Things are changing rapidly!
IT Support
tool
Document
camera
Laptop hug Unicorns.io AirServer Reflector
See face?
If used with
Morae observer
See interaction?
Apps?
Not on iOS
Android & iOS?
iOS only
iOS only
iOS + Android
Clarity Clear, but
slow refresh
Focus is an
issue
Focus can be
an issue
Excellent Excellent Excellent
Cost $1100 single
license / year
<$100 per
camera
Free Free (for now) $15 per license
Free for user
for 7 days
What else? Tech support
tool
Can’t see full
screen
Logistics
(sending
cameras)
Morae is
expensive!
$2,000 license
Awkward! Only works with
recent iOS &
Mac updates
Requires twitter
account
iPhone users
may need
second phone
for voice
iPhone users
may need
second phone
for voice
28. 28BellaVia Research ©2015
Lessons Learned: Tech check!
Schedule a “tech check” before the interview
• You have to help the participant get the technology set up. Do it before the
interview, so you don’t waste the time of client observers.
• Consider bringing in tech help. (I use Civicom)
Make time for tech checks
• Our projects move at Mach speed, but add 2-3 days before interviewing to
make sure all your participants are ready on the technology front.
• If all goes well, each tech check will take 10-20 minutes.
Incentivize for tech check
• What would you pay for a 30-minute interview? Add that to the incentive.
Plan for attrition
• You may lose more than normal due to tech issues / frustrations.
• Over-recruit! e.g., For 10 interviews, recruit 12-14
29. 29BellaVia Research ©2015
Tips for Success: tech check tips & recruiting
Recruit people whose mobile devices meets specs:
• IOS 7 or higher
• Android, 4.4.2 or higher
Recruit people whose mobile devices meets specs:
• Apple: OS X 10.7.5 or Later
• PC: Windows Vista, Windows 7 or Windows 8 (Windows 8 RT is not supported)
Tech setup: hold their hand!
• Webcam on, start screen sharing, download Reflector, enable Reflector, connect mobile device
• Variations in mobile device drives ability to connect to Reflector
- iOS users use Airplay to connect to Reflector
- Android 5.0 Lollipop device users cast screens to Reflector
- Devices running Android 4.4.2 use the Chromecast App to connect to Reflector
Confirm Network Connection:
• Wireless 802.11n or Ethernet highly recommended (No use of free WiFi systems)
• Both computer and mobile device must be on the same connection
• Beware of office firewalls & disconnect from VPNs (eats up bandwidth)
30. 30BellaVia Research ©2015
Lessons Learned & Tips for Success
Ask users to have a second phone available
• You’ll know from the tech check if the user can talk & mirror simultaneously.
Give them time to borrow someone’s cellphone, or do the interview at a
friend’s house where there is a landline.
Not a suitable methodology for all participants
• Works great for tech savvy participants
• Didn’t work as well for a general population study about smartphone app
abandonment. Less tech-savvy participants were frustrated and we lost
several during the tech checks. Solutions are getting easier, but...still may
not be ideal for less tech savvy participants.
Use it as part of a hybrid, mixed-method approach
• Be careful that this methodology doesn’t skew your sample towards tech
savvy participants.
• Consider using as part of a hybrid methodology: some remote interviews
for geographic mix + in-person interviews (to include less tech-savvy
participants).
31. 31BellaVia Research ©2015
How did it work for Ritani.com?
• Deep learnings without breaking the bank with travel
Affordable
• Savvy population of affluent professional guys who are comfortable with screensharing &
their smartphones. Not afraid of technology. Curious and invested in making it work.
• Ring shopping is (often) secret! Easier to hide phone interview than in-home visit +
shopalong.
Good fit for affluent ring-buyers
• Wide audience at Ritani.com could observe sessions real-time. Built empathy for customer
among a diverse set of employees.
• Empowered the team with actionable information, created appetite for findings and
recommendations.
• Seeing is believing. Direct exposure to *how* customers shop for rings across devices
surfaced lots of insights for the team.
Research observation helped with buy-in
Hinweis der Redaktion Moderated session with visibility to computer: Adobe Connect, WebEx, Google Hangouts, Join Me, etc.
Unmoderated session with visibility to mobile devices: UserTesting.com, userzoom.com, lookback.io, YouEye, Userlytics
Unmoderated session with visibility to computer
Moderated session with visibility to mobile devices: iPevo camera, IT solutions (Bomgar, TeamViewer), Reflector or AirServer (both of which use AirPlay on iOs), Hug laptop, Unicorns.io (only iOs streaming with Mac – download unicorns.io to Mac, connect iOs to your computer, private streaming or share link for public)
There’s also a lot of useful tools for in-person research on mobile devices
Record with Camtasia or Quicktime Record with Camtasia or Quicktime Viewing the stream via the URL provided by the participant, which can be public or private.
Participant:
Installs app on Mac
Connects device via USB
Starts app on Mac, click “stream.”
Share unique URL for others to see stream.
Recorded….tbd access to recordings, and if this stays free
80% have iOs 8 as of June 2015 (Intgo)(
>60% have osx 10.10 Yosemite as of June 2015 (GoSquared)
20% have a twitter account (Pew Internet Study, Jan 2015)
TBD how it compares to Meerkat and periscope
Google hangouts screenshare also lets you stream phone screen
Screen streaming mirror for Android lets you stream phone screen
Quick time capture?
Magitest: mweb only, no apps
Viewing the stream via the URL provided by the participant, which can be public or private.
Participant:
Installs app on Mac
Connects device via USB
Starts app on Mac, click “stream.”
Share unique URL for others to see stream.
Recorded….tbd access to recordings, and if this stays free
80% have iOs 8 as of June 2015 (Intgo)(
>60% have osx 10.10 Yosemite as of June 2015 (GoSquared)
20% have a twitter account (Pew Internet Study, Jan 2015)
Airplay is a wireless streaming protocol
iPhone 4S+
iPad 2+
Airplay is a wireless streaming protocol
iPhone 4S+
iPad 2+