This document describes an experimental evaluation of different user interfaces for visual indoor navigation. The study compared augmented reality (AR) to virtual reality (VR) and found that VR was faster and seemed more accurate to users. It also tested a feature indicator and found it increased the number of identifiable features in images. Finally, it evaluated object highlighting and found a soft border version was less distracting than a framed version. The novel user interfaces improved localization accuracy and were more effective and popular than traditional AR interfaces.

1. EXPERIMENTAL EVALUATION OF USER INTERFACES
FOR VISUAL INDOOR NAVIGATION
Andreas Möller ✽
, Matthias Kranz ❖
,
Stefan Diewald ✽
, Luis Roalter ✽
, Robert Huitl ✽
,
Tobias Stockinger ❖
, Marion Koelle ❖
, Patrick Lindemann ❖
!
✽
Technische Universität München, Germany
❖
Universität Passau, Germany

2. VISION-BASED NAVIGATION
Send query image
to server
Database of images
with known position
Return position
and orientation of most similar
reference image

3. ■ Advantages
□ No infrastructure
□ Centimeter-level accuracy (Schroth et al. 2011)
■ But: query images impact localization quality
□ Image distinctiveness
□ Motion blur
□ Pose
MOTIVATION
✘✔✘✘

4. ■ Advantages
□ No infrastructure
□ Centimeter-level accuracy (Schroth et al. 2011)
■ But: query images impact localization quality
□ Image distinctiveness
□ Motion blur
□ Pose
MOTIVATION
✘✔✘✘□ Traditional user interfaces usually require a high
degree of accuracy, e.g. maps (Kray et al. 2003)
or Augmented Reality (Liu et al. 2008)

5. ■ User interface concept for visual localization that
copes with inaccuracy,
and UI elements
to improve query images
■ First experimental evaluation
MAIN CONTRIBUTION
Augmented Reality
(AR)
Virtual Reality
(VR)

6. USER STUDY
3 Experiments
Navigation Time
Distraction
AR/VR
METHOD
!
12 Participants
!
Wizard of Oz
Accuracy
Perception
Preferences
Effectiveness
UI
ELEMENTS
RESEARCH
QUESTIONS

7. EXPERIMENT 1: VR/AR COMPARISON
■ Task: Navigate in building with AR and VR mode
■ Simulation of varying localization accuracy
■ Hypotheses: VR is faster, seems more accurate
and is more popular
AR VR
Live video Panorama

8. EXPERIMENT 1: VR/AR COMPARISON
■ Task: Navigate in building with AR and VR mode
■ Simulation of varying localization accuracy
■ Hypotheses: VR is faster, seems more accurate
and is more popular
AR VR
Live video Panorama

9. ■ AR: users were slower in error conditions
■ VR: no differences between conditions
m:ss
until destination
(average)
EXPERIMENT 1: VR/AR COMPARISON
2:39
3:04 AR
VR
Navigation time

10. EXPERIMENT 1: VR/AR COMPARISON
Guidance quality
3
VR
1
AR -3 = worst
3 = best
position error
2
VR
1
AR
orientation error
VR
2.5
AR
2
no errors

11. EXPERIMENT 1: VR/AR COMPARISON
User preferences
VR
50%
AR
33%
Undecided
17%
„Carrying the phone
was convenient“
2
VR
0
AR -3 = strongly disagree
3 = strongly agree

12. ■ Hypothesis: indicator increases average number of
features visible in the image
■ 3 random appearances of indicator during
navigation task
EXPERIMENT 2: FEATURE INDICATOR

13. EXPERIMENT 2: FEATURE INDICATOR
Features per frame
(average)
% of frames
with >150 features
42
8.1%
101
20.7%
Effectiveness
without FI
with FI

14. EXPERIMENT 3: OBJECT HIGHLIGHTING
■ Hypothesis: Soft border leads to less distraction
than Frame
■ Evaluation on Likert Scale

15. EXPERIMENT 3: OBJECT HIGHLIGHTING
Soft
Border
1
„Aroused
my attention“
„Distracted during
navigation task“
!
Frame
3
1
Frame
-1
Soft
Border
-3 = strongly disagree
3 = strongly agree

16. AR
FI
DISCUSSION
■ VR as primary visualization
■ AR and indicators improve localization
■ Automatic switching between VR and AR
■ Future Work: live system, env. transformations
AR VR
+
accurate inaccurate
after
(re-)localization
navigation
location estimate
too unreliable
Location Estimate

17. SUMMARY
■ Novel UI for visual localization
■ Faster & more popular than AR
■ Increases perceived and
system localization accuracy

18. Contact:
andreas.moeller@tum.de
www.eislab.net

19. Contact:
andreas.moeller@tum.de
www.eislab.net

20. REFERENCES
■ Slide 2: Measurement image: MS Office Clipart
■ Slide 4: Paper References:
Schroth, Georg, et al. "Mobile visual location recognition." Signal
Processing Magazine, IEEE 28.4 (2011): 77-89.
Kray, Chris, et al. "Presenting route instructions on mobile devices."
Proc. of the 8th Intl. Conf. on Intelligent User Interfaces (IUI), ACM
(2003), 117–124.
Liu, A., et al. "Indoor wayfinding: Developing a functional interface for
individuals with cognitive impairments." Disability & Rehabilitation:
Assistive Technology 3, 1-2 (2008): 69–81.
!
!
■ All other photos and graphics: own material by Andreas Möller
or TU München or Universität Passau

21. ■ Please cite this work as follows:
Andreas Möller, Matthias Kranz, Stefan Diewald, Luis Roalter, Robert Huitl, Tobias
Stockinger, Marion Koelle, and Patrick A. Lindemann. 2014. Experimental evaluation of
user interfaces for visual indoor navigation. In Proceedings of the 32nd annual ACM
conference on Human factors in computing systems (CHI '14). ACM, New York, NY,
USA, 3607-3616.
!
■ If you use BibTex:
@inproceedings{Moller:2014:EEU:2611222.2557003,!
author = {M"{o}ller, Andreas and Kranz, Matthias
and Diewald, Stefan and Roalter, Luis and Huitl, Robert and
Stockinger, Tobias and Koelle, Marion and Lindemann, Patrick A.},!
title = {Experimental Evaluation of User Interfaces for
Visual Indoor Navigation},!
booktitle = {Proceedings of the 32Nd Annual ACM Conference on
Human Factors in Computing Systems},!
series = {CHI '14},!
year = {2014},!
isbn = {978-1-4503-2473-1},!
location = {Toronto, Ontario, Canada},!
pages = {3607--3616},!
numpages = {10},!
publisher = {ACM},!
address = {New York, NY, USA},!
}