doing lots of remote webcam interviews on the desktop user experience, but trying to figure out how & if to use screensharing for remote research on MOBILE devices? then this is for you! Moderated (not un-moderated). Remote (not in person). Real-time. Mobile devices (not just computer).

1. 1BellaVia Research ©2015
Julie Francis, UX Researcher & Founder / BellaVia Research
November 2015
Moderated Remote Research
on Mobile Devices
(aka, Stay off planes & save client money!)
QRCA Online SIG

2. 2BellaVia Research ©2015
Who I am & what I’ll share with you today
Julie Francis / Founder, BellaVia Research
20+ years in research
Focus on User Experience Research since 2000
Launched BellaVia in 2001, post Creative Good
Member of QRCA & UXPA
Why I needed a solution for remote moderated
research on mobile devices (and what that means)
Case study
Options + Pros & Cons of each
Tips & Tricks

3. 3BellaVia Research ©2015
The problem
* Moderated and real-time

4. 4BellaVia Research ©2015
The problem
We need to observe it remotely & real-time
and talk about it

5. 5BellaVia Research ©2015
So many research platforms!
Today’s focus:
Moderated
•Not unmoderated, and
Remote
•Not in-person, and
Real-time observation
•Not after-the-fact recordings,
and
 Visibility to mobile
devices
•Not just computers

6. 6BellaVia Research ©2015
Case Study: multi-channel for engagement rings
Ritani needed a holistic
understanding of the UX.
Engagement ring shopping
is a multi-channel
experience.
• Guys look online.
• They use their smartphones
& tablets & computers.
• They shop in stores.
……But how? And why?
Decision
&
purchase

7. 7BellaVia Research ©2015
Case Study: Ritani.com
Buying an engagement ring
is a very unique purchase.
• They are expensive.
• There are a lot of options in
stores and online. There’s
a lot to learn & explore &
consider
• The decision takes place
over a long period of time.
This is not like
shopping for a t-shirt!

8. 8BellaVia Research ©2015
Why not in-person research?
Money.
 Longitudinal nature of ring-
buying required longitudinal
study.
 Geographic mix. Market
variations, retail store
partners vary by market.
 Hotels and airplanes are
expensive.
 Start-up budget

9. 9BellaVia Research ©2015
Why not in-person research?
Observation.
 Only a few people can observe
in-person research, so it’s
“hidden” from many on the team.
 Ritani wanted a solution that
enabled widespread, active
participation by their entire team
(without breaking the bank).
 Remote research is easy to
observe. Lots of participation
drives empathy & innovation.

10. 10BellaVia Research ©2015
Why not unmoderated remote research?
 It’s emotional. Not all people
readily share their feelings.
• Unmoderated, self-reported would
not be a good fit.
 We needed the ability to easily
go off script and go deep.
Probing was essential. It was all
about the observation, and all
about the probes.
How do I pick a
diamond? Where
do I start? Who
can I trust? What
will she like?
Depth of insights.

11. 11BellaVia Research ©2015
Ritani needed:
Geographic mix
• No problem. We’ll use
screen sharing & a
remote methodology, so
we can include ring
shoppers nationwide
Moderated,
in-depth
exploration
• No problem. We
won’t use an
unmoderated tool or
a diary.
Real-time
visibility to
mobile devices
• This is where it gets
tricky…tablet +
smartphone?
?
But
how?

12. 12BellaVia Research ©2015
What we did
3 rounds of research
(longitudinal purchase),
•>4 hours of interviewing each
14 ring shoppers,
early in purchase funnel (just
starting their shopping)
Nationwide …And we found an approach
for remote moderated
research with visibility to
mobile devices! wooohooo!

13. 13BellaVia Research ©2015
Landscape: Remote Research
Computer
online
Mobile
devices
moderated
Un-
moderated
Same players as un-
moderated for mobile…
If they can do it on
mobile, they can do it on
desktop, too

14. 14BellaVia Research ©2015
Considerations when choosing a solution
Interactions?
• How important is it to
see fingers on screen?
Non-verbals?
• Do you need to see the
user’s face?
Stimuli?
• Mobile web?
• Apps?
• Native phone
functionality?
Platform?
• Android?
• iOS?
Who are your
users?
• Tech savvy? Not?

15. 15BellaVia Research ©2015
Option 1: IT Support Tool: e.g., Bomgar
How it works: Participant installs
Bomgar app. Enters session number
provided by researcher. Uses Bomgar
browser to access site or Android app.
Very clear view of device screen
Affordable if regularly used: single-
user license: >$1100 per year
Cannot test iOS apps: only Android
apps (and mobile web)
Slow screen refresh makes it hard to
see interactions
Not designed for research
(odd verbiage for participant)
Similar option: TeamViewer
Researcher view
Thanks to Nyle Feist of UX Guys for the demo!

16. 16BellaVia Research ©2015
Option 1: IT Support Tool: e.g., Bomgar
Cannot see finger
interaction
Cannot see user’s face
The built-in browser in the
Bomgar app eats screen
real estate (address bar at
top, action buttons at
bottom)
Ritani.com
via Bomgar app
Participant view Ritani.com

17. 17BellaVia Research ©2015
Option 2: document camera, e.g., Ipevo
How it works: Mail a document camera to your
particiapnt. Participant plugs in (USB). Start screen-
sharing tool.
Can see the interaction (fingers touching screen)
Easy to switch between devices
You can see their smartwatch!
BUT…you can’t see the user’s face. Can’t use
a document camera and webcam simultaneously
 Note: According to UXPA 2015 session attendees, you can see the
user’s face if the document camera is used in conjunction with a
webcam and Morae Observer. A Morae license is $2,000 per year.
Logistics / fulfillment hassle: Need to send
each participant a document camera (budget:
$100 per device with shipping).
 Vendors like UserTesting.com can help & have a panel.
Civicom will also ship these cameras.
Photo from ipevo.com
Thanks to session attendees for the info about document
cameras used in conjunction with Morae Observer!!

18. 18BellaVia Research ©2015
Option 2: document camera, e.g., Ipevo
Focus issues
 especially problematic in low light

19. 19BellaVia Research ©2015
Option 3: laptop love!
How it works: Participant turns their laptop
around & “hugs” it. Use webcam & screen
sharing tool to show their mobile device.
Free! (if user has a webcam)
Can see the interaction (fingers touching
screen)
Very easy to switch between devices
BUT…you can’t see the person’s face when
they are snuggling their laptop
Challenging for participant to keep device in
webcam view (participant can’t see what the
webcam is capturing)
Image quality can be an issue (focus issues,
low quality webcams)

20. 20BellaVia Research ©2015
Option 4: Lookback’s Unicorns.io
How it works: Participant
installs Unicorns.io on Mac.
Signs in via Twitter
credentials. Connects device
via USB cord. Clicks
“stream.” Shares link with
researcher.
Free download!
(currently)
Very clear image
Streams the video &
audio
If used with screen sharing
tool, can also see the
participant’s face

21. 21BellaVia Research ©2015
Option 4: Lookback’s Unicorns.io
Only works for participants who have
Apple devices and a Mac running recent
software
•only iOS 8+ (80%)
•only Mac OSX 10.10+ (Yosemite) (60%)
Participant must have Twitter account
Cannot see finger interaction
- Note: A similar option was volunteered by a
UXPA 2015 session attendee. X-Mirage is
another iOs-only option that he feels works well.
He recommended Zoom for screen sharing.*
Thanks to the session attendee who recommended x-mirage and zoom!

22. 22BellaVia Research ©2015
Option 5: AirServer + AirPlay
How it works: Participant installs &
launches AirServer on their computer.
Swipe up on iOS device to start AirPlay.
Device is wirelessly mirrored to computer
screen. Start screensharing & webcam, &
shares screen.
Affordable: $15 per download per
participant
Very clear image
Can see the participant’s face via
webcam (screen sharing tool)
User can easily toggle between
computer, iPhone, iPad

23. 23BellaVia Research ©2015
Option 5: AirServer + AirPlay
Only works for
iOS
Only works on
iPhone 4S+ and
iPad 2+
Users may need
a second phone
(some iPhones
cannot mirror & be
used for voice
conversations
simultaneously)

24. 24BellaVia Research ©2015
Option 6: Reflector
How it works: Participant installs &
launches the Reflector app on their
computer.
Ensures computer and device are on
same WiFi network. Shares screen
via screen sharing. Starts webcam.
iOS: Swipes up to launch
AirPlay
Android: launch Screencast
app. (older Android devices
require ChromeCast)
Wirelessly mirrors device to computer
screen.

25. 25BellaVia Research ©2015
Option 6: Reflector
Very clear view of device
Free! 7 days free per user
Visibility to everything (apps, mobile
websites, native phone function)
Ability to see user’s face via webcam &
screen sharing
Cannot see finger interaction on screen
Users may need a second phone (some
phones cannot mirror & talk simultaneously)

26. 26BellaVia Research ©2015
Reflector: In the field
MindBody.com has been in the
field using this technique this
week. It’s going great!
They aren’t crazy about the
Reflector “watermark” – probably
because they are using the free
version?
Thanks to Marianne Connor and Amaya Becvar Weddle from Mind Body Online for this participant quote!
It was fine, I’m really, really
nervous with tech stuff, so
the fact that this worked is
impressive. That never
happens for me. Ever.
Participants say it’s easy!

27. 27BellaVia Research ©2015
Pros & Cons: Things are changing rapidly!
IT Support
tool
Document
camera
Laptop hug Unicorns.io AirServer Reflector
See face?


If used with
Morae observer
   
See interaction?      
Apps? 
Not on iOS
    
Android & iOS?
   
iOS only

iOS only

iOS + Android
Clarity Clear, but
slow refresh
Focus is an
issue
Focus can be
an issue
Excellent Excellent Excellent
Cost $1100 single
license / year
<$100 per
camera
Free Free (for now) $15 per license
Free for user
for 7 days
What else? Tech support
tool
Can’t see full
screen
Logistics
(sending
cameras)
Morae is
expensive!
$2,000 license
Awkward! Only works with
recent iOS &
Mac updates
Requires twitter
account
iPhone users
may need
second phone
for voice
iPhone users
may need
second phone
for voice

28. 28BellaVia Research ©2015
Lessons Learned: Tech check!
Schedule a “tech check” before the interview
• You have to help the participant get the technology set up. Do it before the
interview, so you don’t waste the time of client observers.
• Consider bringing in tech help. (I use Civicom)
Make time for tech checks
• Our projects move at Mach speed, but add 2-3 days before interviewing to
make sure all your participants are ready on the technology front.
• If all goes well, each tech check will take 10-20 minutes.
Incentivize for tech check
• What would you pay for a 30-minute interview? Add that to the incentive.
Plan for attrition
• You may lose more than normal due to tech issues / frustrations.
• Over-recruit! e.g., For 10 interviews, recruit 12-14

29. 29BellaVia Research ©2015
Tips for Success: tech check tips & recruiting
Recruit people whose mobile devices meets specs:
• IOS 7 or higher
• Android, 4.4.2 or higher
Recruit people whose mobile devices meets specs:
• Apple: OS X 10.7.5 or Later
• PC: Windows Vista, Windows 7 or Windows 8 (Windows 8 RT is not supported)
Tech setup: hold their hand!
• Webcam on, start screen sharing, download Reflector, enable Reflector, connect mobile device
• Variations in mobile device drives ability to connect to Reflector
- iOS users use Airplay to connect to Reflector
- Android 5.0 Lollipop device users cast screens to Reflector
- Devices running Android 4.4.2 use the Chromecast App to connect to Reflector
Confirm Network Connection:
• Wireless 802.11n or Ethernet highly recommended (No use of free WiFi systems)
• Both computer and mobile device must be on the same connection
• Beware of office firewalls & disconnect from VPNs (eats up bandwidth)

30. 30BellaVia Research ©2015
Lessons Learned & Tips for Success
Ask users to have a second phone available
• You’ll know from the tech check if the user can talk & mirror simultaneously.
Give them time to borrow someone’s cellphone, or do the interview at a
friend’s house where there is a landline.
Not a suitable methodology for all participants
• Works great for tech savvy participants
• Didn’t work as well for a general population study about smartphone app
abandonment. Less tech-savvy participants were frustrated and we lost
several during the tech checks. Solutions are getting easier, but...still may
not be ideal for less tech savvy participants.
Use it as part of a hybrid, mixed-method approach
• Be careful that this methodology doesn’t skew your sample towards tech
savvy participants.
• Consider using as part of a hybrid methodology: some remote interviews
for geographic mix + in-person interviews (to include less tech-savvy
participants).

31. 31BellaVia Research ©2015
How did it work for Ritani.com?
• Deep learnings without breaking the bank with travel
Affordable
• Savvy population of affluent professional guys who are comfortable with screensharing &
their smartphones. Not afraid of technology. Curious and invested in making it work.
• Ring shopping is (often) secret! Easier to hide phone interview than in-home visit +
shopalong.
Good fit for affluent ring-buyers
• Wide audience at Ritani.com could observe sessions real-time. Built empathy for customer
among a diverse set of employees.
• Empowered the team with actionable information, created appetite for findings and
recommendations.
• Seeing is believing. Direct exposure to *how* customers shop for rings across devices
surfaced lots of insights for the team.
Research observation helped with buy-in

32. 32BellaVia Research ©2015
Questions?
Julie Francis / BellaVia Research
831.454.8217 o / 831.454.6456 m
julie@BellaViaResearch.com
@BellaViaResearch