11. Augmented Reality
• Augmented Reality: Enhances Reality
• Key Characteristics
• Combines real and virtual images
• Interactive in Real-Time
• Registered in 3D
17. AR Medical Trials
Sauer et al. 2000 at Siemens
Corporate Research, NJ
Stereo video see through
F. Sauer, Ali Khamene, S. Vogt: An Augmented Reality Navigation System with a
Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial,
MICCAI 2002
18. AR History Summary
1960’s – 80’s: Early
Experimentation
1980’s – 90’s: Basic Research
Tracking, displays
1995 – 2005: Tools/Applications
Interaction, usability, theory
2005 - : Commercial
Applications
Games, Medical, Industry
20. Visual Analytics Prototype using Tangible AR
Prototype developed by Nick Hedley, at SFU.
Design based on empirical findings of several
tangible AR studies compared with other
conventional 3D visualizations
Image courtesy of N. Hedley / SIRL (2008)
22. Interface Design Path
1/ Prototype Demonstration
2/ Adoption of Interaction Techniques from other
interface metaphors Augmented Reality
3/ Development of new interface metaphors
appropriate to the medium
Virtual Reality
4/ Development of formal theoretical models for
predicting and modeling user actions
Desktop WIMP
23. AR Design Principles
Interface Components
Physical components
Display elements
- Visual/audio
Interaction metaphors
Physical Display
Elements Interaction Elements
Metaphor
Input Output
24. AR User Interface Design Space
Reality Virtual Reality
Augmented Reality
Physical Design Virtual Design
NO STANDARD PRINCIPLES !!
STANDARD PRINCIPLES STANDARD PRINCIPLES
SOME CLUES
25. Tangible Interfaces (Ishii 97)
Create digital shadows for
physical objects
Foreground
graspable UI
Background
ambient interfaces
26. Tangible Interfaces
Dangling String
Jeremijenko 1995
Ambient ethernet monitor
Relies on peripheral cues
Ambient Fixtures
Dahley, Wisneski, Ishii 1998
Use natural material qualities
for information display
27. TerraVision (ART+COM 1994)
Goal
use virtual earth to
visualize information
Features
seamless zooming
tangible interface
separation of physical + virtual
28. Lessons from Tangible Interfaces
Physical objects make us smart
Norman’s “Things that Make Us Smart”
encode affordances, constraints
Objects aid collaboration
establish shared meaning
Objects increase understanding
serve as cognitive artifacts
29. Limitations
Difficult to change object properties
can’t tell state of digital data
Limited display capabilities
pinwheels = 1D, projection screen = 2D
dependent on physical display surface
Separation between object and display
TerraVision
30. Back to the Real World
AR overcomes limitation of TUIs
enhance display possibilities
merge task/display space
provide public and private views
TUI + AR = Tangible AR
Apply TUI methods to AR interface design
31. Tangible AR Design Principles
Tangible AR Interfaces use TUI principles
Physical controllers for moving virtual content
Support for spatial 3D interaction techniques
Time and space multiplexed interaction
Support for multi-handed interaction
Match object affordances to task requirements
Support parallel activity with multiple objects
Allow collaboration between multiple users
32. Design of Objects
Objects
Purposely built – affordances
“Found” – repurposed
Existing – already at use in marketplace
Make affordances obvious (Norman)
Object affordances visible
Give feedback
Provide constraints
Use natural mapping
Use good cognitive model
33. Tangible AR: VOMAR Interface (Kato 2000)
Use of natural physical object manipulations to
control virtual objects
VOMAR Demo
Catalog book:
- Turn over the page
Paddle operation:
- Push, shake, incline, hit, scoop
35. Case Study: 3D AR Lens
Goal: Develop a lens based AR interface
MagicLenses
Developed at Xerox PARC in 1993
View a region of the workspace differently to the rest
Overlap MagicLenses to create composite effects
37. AR Lens Design Principles
Physical Components
Lens handle
- Virtual lens attached to real object
Display Elements
Lens view
- Reveal layers in dataset
Interaction Metaphor
Physically holding lens
38. 3D AR Lenses: Model Viewer
Displays models made up of multiple parts
Each part can be shown or hidden through the lens
Allows the user to peer inside the model
Maintains focus + context
46. Communication Cues
A wide variety of communication cues used.
Visual
Audio Gaze
Speech Gesture
Paralinguistic Face Expression
Paraverbals Body Position
Prosodics
Intonation
Object Manipulation
Environmental Writing/Drawing
Spatial Relationship
Object Presence
49. Face-to-face collaboration
People surround a table
It is easy to see each other
Communication Space Task Space
Computer supported
collaboration
People sit side by side
It is hard to see each other
Communication Space Task Space
51. Hybrid User Interfaces
1 2 3 4
PERSONAL TABLETOP WHITEBOARD MULTIGROUP
Private Display Private Display Private Display Private Display
Group Display Public Display Group Display
Public Display
64. Survey of AR Papers
Edward Swan (2005)
Surveyed major conference/journals (1992-2004)
- Presence, ISMAR, ISWC, IEEE VR
Summary
1104 total papers
266 AR papers
38 AR HCI papers (Interaction)
21 AR user studies
Only 21 from 266 AR papers have formal user study
(<8% of all AR papers)
66. Types of Experiments
Perception
How is virtual content perceived ?
What perceptual cues are most important ?
Interaction
How can users interact with virtual content ?
Which interaction techniques are most efficient ?
Collaboration
How is collaboration in AR interface different ?
Which collaborative cues can be conveyed best ?
68. The Future is with us
It takes at least 20 years for new technologies
to go from the lab to the lounge..
“The technologies that will significantly affect our
lives over the next 10 years have been around
for a decade.
The future is with us. The trick is learning how to
spot it. The commercialization of research, in Oct 11th 2004
other words, is far more about prospecting
than alchemy.”
Bill Buxton
83. Physical + Social Networking
Location based social networking
Where are my friends now?
Where are people with common interests?
Trend analysis
How do these people behave?
Where is my friend likely to be?
85. Conclusions
AR allows us to return to the real world
Invisible Interfaces
Enhanced Interaction
Enhanced Collaboration
Future Research Directions
Evaluation
Massive AR
86. More Information
• Mark Billinghurst
– mark.billinghurst@hitlabnz.org
• Websites
– www.hitlabnz.org
