SlideShare ist ein Scribd-Unternehmen logo
1 von 42
Downloaden Sie, um offline zu lesen
Touchless Circular Menus:
Toward an Intuitive UI for Touchless
Interactions with Large Displays
Debaleena Chattopadhyay & Davide Bolchini
AVI 2014 International Working Conference on Advanced Visual Interfaces
Como (Italy) May 27-30, 2014
Public
Spaces
Sterile
Operating
Rooms
Interactive
TVs
Large
Displays
Touchless
Interaction
Usage
contexts
Problem Space
Touchless interaction with large
displays still lacks a
standardized user interface
language for frequent user-
operations, such as..
Command
Selection
Menu
Marking Menu, 1994
FlowMenu, 2000
Toolglass, 1993
rapMenu, 2008
Grab, 2013
Finger-Count Menu, 2011
Purpose of the Work
To propose and validate a novel
form of command selection
language appropriate for
touchless interaction with large
displays.
Design Space
Current Debate on UI “naturalness”
Norman, D. A. (2010). Natural user interfaces are not natural. interactions, 17(3), 6-10.
Wigdor, D., & Wixon, D. (2011). Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier.
O'hara, K., Harper, R., Mentis, H., Sellen, A., & Taylor, A. (2013). On the naturalness of touchless: putting the “interaction” back
into NUI. ACM Transactions on Computer-Human Interaction (TOCHI), 20(1), 5.
“You are the controller”
–Microsoft®
“Most gestures are neither natural
nor easy to learn or remember.”
–Norman
Natural is ‘a design philosophy and a
source of metrics enabling an
iterative process to create a product.’
–Wigdor & Wixon
“Naturalness is not something
to be represented but is rather
an ‘occasioned property of
action’ [..].”
–O’Hara et al.
Intuitiveness of
an interface
Prior
knowledge
Expertise
Culture
Sensorimotor
Innate
Specialized knowledge
acquired with practice.
Knowledge of people
specific to culture.
Knowledge acquired since
childhood through interaction
with the world.
Reflexes and intrinsic
behavior.
unconscious
application of prior
knowledge
∝
Hurtienne, J., & Israel, J. H., 2007. Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction.
Proc. TEI,127-134, ACM.
Intuitive Interaction Framework
How can we use sensorimotor
knowledge to inform the design
of intuitive touchless selection
mechanisms?
Device – less
interaction
Device – based
interaction
Menu Invocation
Mouse click, pen down, or touching
surface
?
Menu-Selection Delimiter
Breaking contact with the interface
Menu Shape
?
Linear, radial
?
Hand posture
Hand posture
Linear, radial
What successful features of
device-based menus can be
"ported" to the touchless
environment?
Prior
sensorimotor
knowledge
Some successful
features of device-
based menus
Touchless Circular Menus (TCM)
+
Directional strokes
in mid-air
Menu-invocation:
Reaching a region-of-
interest.
Menu-selection: Crossing
Shape: Radial
Touchless circular
menus relieve users
from both recalling a
precise vocabulary of
hand-postures and
strictly complying with
them.
Demo
To select a menu option, the user makes a directional stroke.
To give feedback, when successfully selected, the menu-
option changes color.
To mitigate accidental invocation, TCM appear opposite to the
user’s direction of approach.
To cancel the invoked menu, the user may continue in her
direction of movement, or move away from the menu.
To invoke the menu, the user must reach the region-of-
interest (ROI) of the target.
Hierarchical TCM: To select a command, the user first selects
a menu option, and then accesses the submenu by continuing
her trajectory.
The Design
To trigger the contextual menu, a user must cross the region-of-interest (ROI) of a
display object.
The ROI can be of any symmetrical shape around the center of the target, with its size
directly proportional to the technique’s sensitivity.
Menu Invocation
To select a command after triggering the menu, users cross it using a stroke in the
command’s direction.
Until the crossing happens, users can cancel TCM by moving in any direction away from
the triggered menu.
To allow easy escape routes, we designed the structure of TCM as a semicircular array
of options appearing at the top-left or the bottom-right corner of the target.
Menu Selection
As users approach the menu, to give them orientation, a trace is drawn connecting the
target and the users’ hand position.
To improve users’ pointing performance, the menu options would increase in amplitude
as users approached them.
To provide further feedback, menu options changed color when selected.
Menu Selection
Currently, the menu design scales up to two levels (5 x 5), with users performing
continuous strokes.
To operate the submenu, users change their track and cross another command.
Due to the lack of precision and control of freehand movements, TCM require users to
make inflections in their continuing trajectories, and thereby avoid accidental command-
selections.
Hierarchical TCM
Evaluation
Experiments
How effectiveness and
efficiency of TCM is
affected by their triggering
locations on the visual
interface?
1
Vs.
Touchless circular menus
Linear menus using a grab
gesture
2
Participants (N = 15)
were sitting away from a
large display.
Experimental Design
Sample Demographics
N = 15
All right-handed
4/15 females
8/15 had prior familiarity with touchless gestures
11/15 were below 30 years of age
Time/participant: 1 hour
1Experiment
9 trials
X 7 blocks
X 15 participants
= 945 trials
Experiment 2
6 trials
X 7 blocks
X 15 participants
= 630 trials
Apparatus
Display—
Built by Fakespace, eight 50”
projection cubes, each with a
resolution of 1600 X 1200
pixels.
160” wide and 60” high display
Total resolution: 15.3 M pixels.
Motion Tracking
Sensor—
Kinect for Windows™
UITS Research Technologies
Advanced Visualization Lab
Outcome Measures
Efficiency:
Time on task | Successful Trigger Rate
Effectiveness:
Error Rate
User Satisfaction:
SUS | NASA TLX
1Experiment
Hypotheses
H1: Triggering location will affect the efficiency
of TCM.
H2: Triggering location will affect the
effectiveness of TCM
Triggering location of TCM significantly
affected selection time and successful trigger rate.
“I felt I had to rush to
select the menu
option”
“I was surprised that I
could do so well.”
1Experiment
Results
Arm posture affects users’ control
on their hand movements and the
required effort.
1Experiment
2D plane locations relative to the user’s
body and arm configuration affects the
consumed endurance in touchless
interactions
Hincapié-Ramos, J. D., Guo, X., Moghadasian, P., & Irani, P. (2014). Consumed Endurance: A metric to quantify arm fatigue of
mid-air interactions. Proc. CHI, 1063-1072, ACM.
Menu triggering locations will
significantly affect the user
experience of contextual touchless
menus for large displays.
Arm posture affects users’
control on their hand
movements and the
required effort.
Contextual touchless
menus for large displays
will have different menu-
triggering locations.
1Experiment
Hypotheses
H3: We predicted TCM would be more efficient
than linear menus.
H4: We hypothesized that TCM would be
easier to use than linear menus.
Experiment 2
Experiment 2
Compared with linear menus, users were more
efficient with TCM, and perceived lower overall
workload.
But TCM were less effective than linear menus.
“This is how I envision
using touchless
gestures.”
Results
“It was a lot of effort.”
Experiment 2
Compared with linear menus
using grab gestures, participants
using TCM were twice faster in
selecting commands and perceived
lower workload.
However, TCM caused 3% more
errors than linear menus.
Experiment 2Why TCM were less effective?
1st gesture registration
to invoke the menu
2nd gesture
registration to
select the menu
Gesture relaxation
Experiment 2Why TCM were less effective?
1st gesture registration
to invoke the menu
2nd gesture
registration to
select the menu
No gesture relaxation
Future Work
Minimize errors by constraining users’
freehand movements after triggering
the TCM.
Investigate bimanual gestures to operate
menu levels in hierarchical TCM.
Takeaways
Menu triggering locations significantly affect the user
experience of contextual touchless menus for large displays.
Simple directional movements for selecting commands is
more efficient and causes lower workload than (hand)
posture-based techniques.
TCM relieve users from learning posture-based commands,
and shift the interaction complexity from users’ input to the
visual interface.
Thank you.
We thank the study participants and UITS AVL. This research is partially supported by an IUPUI Research Support Funds Grant.
1
2
3
Efficient gesture relaxation techniques may have the
potential to increase effectiveness of touchless interactions.
4

Weitere ähnliche Inhalte

Ähnlich wie Touchless Circular Menus

User Interfaces and User Centered Design Techniques for Augmented Reality and...
User Interfaces and User Centered Design Techniques for Augmented Reality and...User Interfaces and User Centered Design Techniques for Augmented Reality and...
User Interfaces and User Centered Design Techniques for Augmented Reality and...Stuart Murphy
 
Interaksi manusia dan komputer
Interaksi manusia dan komputerInteraksi manusia dan komputer
Interaksi manusia dan komputergusfaleo firnando
 
Slide interaksi manusia dan komputer
Slide interaksi manusia dan komputerSlide interaksi manusia dan komputer
Slide interaksi manusia dan komputertheogrouptech
 
Comparative Evaluation of Two Interface Tools in Performing Visual Analytics ...
Comparative Evaluation of Two Interface Tools in Performing Visual Analytics ...Comparative Evaluation of Two Interface Tools in Performing Visual Analytics ...
Comparative Evaluation of Two Interface Tools in Performing Visual Analytics ...BELIV Workshop
 
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...Myroslav Bachynskyi
 
Interactively Evolving User Interfaces
Interactively Evolving User InterfacesInteractively Evolving User Interfaces
Interactively Evolving User InterfacesJuan Quiroz
 
Hci user interface-design principals lec 7
Hci user interface-design principals lec 7Hci user interface-design principals lec 7
Hci user interface-design principals lec 7Anwal Mirza
 
Hci user interface-design principals
Hci user interface-design principalsHci user interface-design principals
Hci user interface-design principalsAnwal Mirza
 
User Interface Design- Module 3 Menus
User Interface Design- Module 3 MenusUser Interface Design- Module 3 Menus
User Interface Design- Module 3 MenusbrindaN
 
System Menu And Navigation
System Menu And NavigationSystem Menu And Navigation
System Menu And NavigationOvidiu Von M
 
Fat Finger - Use of contact Size as a form of simulated pressure.
Fat Finger - Use of contact Size as a form of simulated pressure.Fat Finger - Use of contact Size as a form of simulated pressure.
Fat Finger - Use of contact Size as a form of simulated pressure.Evangelos Tzemis
 
Usability Study of the Finger Counter
Usability Study of the Finger CounterUsability Study of the Finger Counter
Usability Study of the Finger CounterChristen Ng
 
Multi(Touch) Gesture Research
Multi(Touch) Gesture ResearchMulti(Touch) Gesture Research
Multi(Touch) Gesture ResearchFlorian Weil
 
ManganoHoeeg - Virtual Supermarket Designer POSTER
ManganoHoeeg - Virtual Supermarket Designer POSTERManganoHoeeg - Virtual Supermarket Designer POSTER
ManganoHoeeg - Virtual Supermarket Designer POSTEREmil Rosenlund Høeg
 

Ähnlich wie Touchless Circular Menus (20)

Touch Screens
Touch ScreensTouch Screens
Touch Screens
 
User Interfaces and User Centered Design Techniques for Augmented Reality and...
User Interfaces and User Centered Design Techniques for Augmented Reality and...User Interfaces and User Centered Design Techniques for Augmented Reality and...
User Interfaces and User Centered Design Techniques for Augmented Reality and...
 
Interaksi manusia dan komputer
Interaksi manusia dan komputerInteraksi manusia dan komputer
Interaksi manusia dan komputer
 
Slide interaksi manusia dan komputer
Slide interaksi manusia dan komputerSlide interaksi manusia dan komputer
Slide interaksi manusia dan komputer
 
Comparative Evaluation of Two Interface Tools in Performing Visual Analytics ...
Comparative Evaluation of Two Interface Tools in Performing Visual Analytics ...Comparative Evaluation of Two Interface Tools in Performing Visual Analytics ...
Comparative Evaluation of Two Interface Tools in Performing Visual Analytics ...
 
Map Navigation using hand gesture recognition
Map Navigation using hand gesture recognitionMap Navigation using hand gesture recognition
Map Navigation using hand gesture recognition
 
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
Performance and Ergonomics of Touch Surfaces: A Comparative Study using Bio...
 
Aug menu
Aug menuAug menu
Aug menu
 
User Interface
User InterfaceUser Interface
User Interface
 
Interactively Evolving User Interfaces
Interactively Evolving User InterfacesInteractively Evolving User Interfaces
Interactively Evolving User Interfaces
 
Hci user interface-design principals lec 7
Hci user interface-design principals lec 7Hci user interface-design principals lec 7
Hci user interface-design principals lec 7
 
Hci user interface-design principals
Hci user interface-design principalsHci user interface-design principals
Hci user interface-design principals
 
User Interface Design- Module 3 Menus
User Interface Design- Module 3 MenusUser Interface Design- Module 3 Menus
User Interface Design- Module 3 Menus
 
System Menu And Navigation
System Menu And NavigationSystem Menu And Navigation
System Menu And Navigation
 
Fat Finger - Use of contact Size as a form of simulated pressure.
Fat Finger - Use of contact Size as a form of simulated pressure.Fat Finger - Use of contact Size as a form of simulated pressure.
Fat Finger - Use of contact Size as a form of simulated pressure.
 
Usability Study of the Finger Counter
Usability Study of the Finger CounterUsability Study of the Finger Counter
Usability Study of the Finger Counter
 
Characteristics of hand gesture: MYO Case Study
Characteristics of hand gesture: MYO Case StudyCharacteristics of hand gesture: MYO Case Study
Characteristics of hand gesture: MYO Case Study
 
Multi(Touch) Gesture Research
Multi(Touch) Gesture ResearchMulti(Touch) Gesture Research
Multi(Touch) Gesture Research
 
ManganoHoeeg - Virtual Supermarket Designer POSTER
ManganoHoeeg - Virtual Supermarket Designer POSTERManganoHoeeg - Virtual Supermarket Designer POSTER
ManganoHoeeg - Virtual Supermarket Designer POSTER
 
2 4-10
2 4-102 4-10
2 4-10
 

Mehr von Debaleena Chattopadhyay

Touchless Interaction from an Embodied Perspective
Touchless Interaction from an Embodied PerspectiveTouchless Interaction from an Embodied Perspective
Touchless Interaction from an Embodied PerspectiveDebaleena Chattopadhyay
 
Experimental evaluation of five methods for collecting emotions in field sett...
Experimental evaluation of five methods for collecting emotions in field sett...Experimental evaluation of five methods for collecting emotions in field sett...
Experimental evaluation of five methods for collecting emotions in field sett...Debaleena Chattopadhyay
 
Keeping things in context a comparative evaluation of focus plus context scre...
Keeping things in context a comparative evaluation of focus plus context scre...Keeping things in context a comparative evaluation of focus plus context scre...
Keeping things in context a comparative evaluation of focus plus context scre...Debaleena Chattopadhyay
 
Supporting mobility for the blind a broad lit review
Supporting mobility for the blind   a broad lit reviewSupporting mobility for the blind   a broad lit review
Supporting mobility for the blind a broad lit reviewDebaleena Chattopadhyay
 
Estimating natural illumination from a single outdoor scene final
Estimating natural illumination from a single outdoor scene   finalEstimating natural illumination from a single outdoor scene   final
Estimating natural illumination from a single outdoor scene finalDebaleena Chattopadhyay
 
Exploiting Hierarchical Context on a Large Database of Object Categories
Exploiting Hierarchical Context on a Large Database of Object Categories Exploiting Hierarchical Context on a Large Database of Object Categories
Exploiting Hierarchical Context on a Large Database of Object Categories Debaleena Chattopadhyay
 

Mehr von Debaleena Chattopadhyay (10)

Touchless Interaction from an Embodied Perspective
Touchless Interaction from an Embodied PerspectiveTouchless Interaction from an Embodied Perspective
Touchless Interaction from an Embodied Perspective
 
Think aloud protocol a reflection
Think aloud protocol  a reflectionThink aloud protocol  a reflection
Think aloud protocol a reflection
 
Experimental evaluation of five methods for collecting emotions in field sett...
Experimental evaluation of five methods for collecting emotions in field sett...Experimental evaluation of five methods for collecting emotions in field sett...
Experimental evaluation of five methods for collecting emotions in field sett...
 
Keeping things in context a comparative evaluation of focus plus context scre...
Keeping things in context a comparative evaluation of focus plus context scre...Keeping things in context a comparative evaluation of focus plus context scre...
Keeping things in context a comparative evaluation of focus plus context scre...
 
Supporting mobility for the blind a broad lit review
Supporting mobility for the blind   a broad lit reviewSupporting mobility for the blind   a broad lit review
Supporting mobility for the blind a broad lit review
 
Defocus magnification
Defocus magnificationDefocus magnification
Defocus magnification
 
Estimating natural illumination from a single outdoor scene final
Estimating natural illumination from a single outdoor scene   finalEstimating natural illumination from a single outdoor scene   final
Estimating natural illumination from a single outdoor scene final
 
Exploiting Hierarchical Context on a Large Database of Object Categories
Exploiting Hierarchical Context on a Large Database of Object Categories Exploiting Hierarchical Context on a Large Database of Object Categories
Exploiting Hierarchical Context on a Large Database of Object Categories
 
Sentence generation
Sentence generationSentence generation
Sentence generation
 
Beyond nouns eccv_2008
Beyond nouns eccv_2008Beyond nouns eccv_2008
Beyond nouns eccv_2008
 

Kürzlich hochgeladen

Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 

Kürzlich hochgeladen (20)

Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 

Touchless Circular Menus

  • 1. Touchless Circular Menus: Toward an Intuitive UI for Touchless Interactions with Large Displays Debaleena Chattopadhyay & Davide Bolchini AVI 2014 International Working Conference on Advanced Visual Interfaces Como (Italy) May 27-30, 2014
  • 4. Touchless interaction with large displays still lacks a standardized user interface language for frequent user- operations, such as..
  • 5. Command Selection Menu Marking Menu, 1994 FlowMenu, 2000 Toolglass, 1993 rapMenu, 2008 Grab, 2013 Finger-Count Menu, 2011
  • 7. To propose and validate a novel form of command selection language appropriate for touchless interaction with large displays.
  • 9. Current Debate on UI “naturalness” Norman, D. A. (2010). Natural user interfaces are not natural. interactions, 17(3), 6-10. Wigdor, D., & Wixon, D. (2011). Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier. O'hara, K., Harper, R., Mentis, H., Sellen, A., & Taylor, A. (2013). On the naturalness of touchless: putting the “interaction” back into NUI. ACM Transactions on Computer-Human Interaction (TOCHI), 20(1), 5. “You are the controller” –Microsoft® “Most gestures are neither natural nor easy to learn or remember.” –Norman Natural is ‘a design philosophy and a source of metrics enabling an iterative process to create a product.’ –Wigdor & Wixon “Naturalness is not something to be represented but is rather an ‘occasioned property of action’ [..].” –O’Hara et al.
  • 10. Intuitiveness of an interface Prior knowledge Expertise Culture Sensorimotor Innate Specialized knowledge acquired with practice. Knowledge of people specific to culture. Knowledge acquired since childhood through interaction with the world. Reflexes and intrinsic behavior. unconscious application of prior knowledge ∝ Hurtienne, J., & Israel, J. H., 2007. Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction. Proc. TEI,127-134, ACM. Intuitive Interaction Framework
  • 11. How can we use sensorimotor knowledge to inform the design of intuitive touchless selection mechanisms?
  • 12. Device – less interaction Device – based interaction Menu Invocation Mouse click, pen down, or touching surface ? Menu-Selection Delimiter Breaking contact with the interface Menu Shape ? Linear, radial ? Hand posture Hand posture Linear, radial
  • 13. What successful features of device-based menus can be "ported" to the touchless environment?
  • 14. Prior sensorimotor knowledge Some successful features of device- based menus Touchless Circular Menus (TCM) + Directional strokes in mid-air Menu-invocation: Reaching a region-of- interest. Menu-selection: Crossing Shape: Radial
  • 15. Touchless circular menus relieve users from both recalling a precise vocabulary of hand-postures and strictly complying with them.
  • 16. Demo
  • 17. To select a menu option, the user makes a directional stroke. To give feedback, when successfully selected, the menu- option changes color.
  • 18. To mitigate accidental invocation, TCM appear opposite to the user’s direction of approach. To cancel the invoked menu, the user may continue in her direction of movement, or move away from the menu.
  • 19. To invoke the menu, the user must reach the region-of- interest (ROI) of the target.
  • 20. Hierarchical TCM: To select a command, the user first selects a menu option, and then accesses the submenu by continuing her trajectory.
  • 22. To trigger the contextual menu, a user must cross the region-of-interest (ROI) of a display object. The ROI can be of any symmetrical shape around the center of the target, with its size directly proportional to the technique’s sensitivity. Menu Invocation
  • 23. To select a command after triggering the menu, users cross it using a stroke in the command’s direction. Until the crossing happens, users can cancel TCM by moving in any direction away from the triggered menu. To allow easy escape routes, we designed the structure of TCM as a semicircular array of options appearing at the top-left or the bottom-right corner of the target. Menu Selection
  • 24. As users approach the menu, to give them orientation, a trace is drawn connecting the target and the users’ hand position. To improve users’ pointing performance, the menu options would increase in amplitude as users approached them. To provide further feedback, menu options changed color when selected. Menu Selection
  • 25. Currently, the menu design scales up to two levels (5 x 5), with users performing continuous strokes. To operate the submenu, users change their track and cross another command. Due to the lack of precision and control of freehand movements, TCM require users to make inflections in their continuing trajectories, and thereby avoid accidental command- selections. Hierarchical TCM
  • 27. Experiments How effectiveness and efficiency of TCM is affected by their triggering locations on the visual interface? 1 Vs. Touchless circular menus Linear menus using a grab gesture 2 Participants (N = 15) were sitting away from a large display.
  • 28. Experimental Design Sample Demographics N = 15 All right-handed 4/15 females 8/15 had prior familiarity with touchless gestures 11/15 were below 30 years of age Time/participant: 1 hour 1Experiment 9 trials X 7 blocks X 15 participants = 945 trials Experiment 2 6 trials X 7 blocks X 15 participants = 630 trials
  • 29. Apparatus Display— Built by Fakespace, eight 50” projection cubes, each with a resolution of 1600 X 1200 pixels. 160” wide and 60” high display Total resolution: 15.3 M pixels. Motion Tracking Sensor— Kinect for Windows™ UITS Research Technologies Advanced Visualization Lab
  • 30. Outcome Measures Efficiency: Time on task | Successful Trigger Rate Effectiveness: Error Rate User Satisfaction: SUS | NASA TLX
  • 31. 1Experiment Hypotheses H1: Triggering location will affect the efficiency of TCM. H2: Triggering location will affect the effectiveness of TCM
  • 32. Triggering location of TCM significantly affected selection time and successful trigger rate. “I felt I had to rush to select the menu option” “I was surprised that I could do so well.” 1Experiment Results
  • 33. Arm posture affects users’ control on their hand movements and the required effort. 1Experiment 2D plane locations relative to the user’s body and arm configuration affects the consumed endurance in touchless interactions Hincapié-Ramos, J. D., Guo, X., Moghadasian, P., & Irani, P. (2014). Consumed Endurance: A metric to quantify arm fatigue of mid-air interactions. Proc. CHI, 1063-1072, ACM.
  • 34. Menu triggering locations will significantly affect the user experience of contextual touchless menus for large displays. Arm posture affects users’ control on their hand movements and the required effort. Contextual touchless menus for large displays will have different menu- triggering locations. 1Experiment
  • 35. Hypotheses H3: We predicted TCM would be more efficient than linear menus. H4: We hypothesized that TCM would be easier to use than linear menus. Experiment 2
  • 36. Experiment 2 Compared with linear menus, users were more efficient with TCM, and perceived lower overall workload. But TCM were less effective than linear menus. “This is how I envision using touchless gestures.” Results “It was a lot of effort.”
  • 37. Experiment 2 Compared with linear menus using grab gestures, participants using TCM were twice faster in selecting commands and perceived lower workload. However, TCM caused 3% more errors than linear menus.
  • 38. Experiment 2Why TCM were less effective? 1st gesture registration to invoke the menu 2nd gesture registration to select the menu Gesture relaxation
  • 39. Experiment 2Why TCM were less effective? 1st gesture registration to invoke the menu 2nd gesture registration to select the menu No gesture relaxation
  • 41. Minimize errors by constraining users’ freehand movements after triggering the TCM. Investigate bimanual gestures to operate menu levels in hierarchical TCM.
  • 42. Takeaways Menu triggering locations significantly affect the user experience of contextual touchless menus for large displays. Simple directional movements for selecting commands is more efficient and causes lower workload than (hand) posture-based techniques. TCM relieve users from learning posture-based commands, and shift the interaction complexity from users’ input to the visual interface. Thank you. We thank the study participants and UITS AVL. This research is partially supported by an IUPUI Research Support Funds Grant. 1 2 3 Efficient gesture relaxation techniques may have the potential to increase effectiveness of touchless interactions. 4

Hinweis der Redaktion

  1. While interacting with large, distant displays…Touchless interaction frees users from employing any intermediate technology and almost has a universal appeal. Large-display touchless interaction is being increasingly used in a number of usage contexts. Public displays, where users interact for a brief amount of time and may not spend the time and effort to connect a hand-held device with the display. Sterile operating rooms, where surgeons may need to browse medical images and cannot touch devices. Interactive TVs, where users can use touchless interactions to browse multimedia sporadically or access their favorite commands. Given this range of different scenarios, different stakeholders, and varied user expertise, it is a challenge to design an optimal touchless user experience for interacting with large displays.
  2. However, touchless interaction is still in its infancy.
  3. Specifically touchless interactions with large displays lacks a standardized user interface language for frequent user-operations, such as..
  4. Command selection. Whereas an extensive body of works investigated optimal menu designs for mouse, pen-input, or multitouch surfaces, very few have looked into touchless command-selection techniques …..especially for large displays. Recent solutions that appeared in research venues require users to comply strictly with system-defined poses, such as pinching with fingers, or making different finger combinations. These require users to recall a vocabulary of gestures, and in-lab user-studies have reported their high mental and physical demand. On product platform, recent solutions include touchless menus using grab gestures ..closing and opening the hand…. Despite the potential of such ‘smart interactions’, experts have commented on such products’ low user satisfaction.
  5. The purpose of our work
  6. Was to propose and validate a novel form of command selection language appropriate for touchless interaction with large displays.
  7. Given the variety of contexts, stakeholders, and user expertise, while designing the touchless menu, we considered several aspects.. For example, the claimed naturalness of this interaction modality, As well as its differences form other input modalities, such as pen or touch.
  8. Ironically, when you are designing touchless interactions, the word “natural” comes both as your ally and your adversary. As markerless motion tracking became mainstream, users became the controller. Yet soon, gestures, lacking feedback and learnability, were critiqued to be neither natural, nor easy to learn or remember. Natural, was then described as a design philosophy to enable the iterative creation of a product. And naturalness was explained as not something to be represented by the mechanics of a touchless gesture, but rather experienced as a property of the action taking place.
  9. To operationalize the ‘naturalness’ construct, we used the intuitive interaction framework. This framework suggests that intuitiveness………..often the twin term of naturalness….. is directly proportional to the unconscious application of prior knowledge. Now prior knowledge can be classified as expertise, culture, sensorimotor or innate. Expertise level of knowledge is specialized knowledge acquired with practice. In the Culture level, we’d find knowledge of people specific to culture. Sensorimotor level of knowledge is acquired since childhood through interaction with the world. Finally innate knowledge is developed in a prenatal state, such as reflexes, or intrinsic behavior. One important thing to note here… is that …..The higher the degree of specialization of knowledge, the smaller would be the potential number of users applying that knowledge unconsciously. Touchless interaction with large displays appeals to a variety of domains where users would have different expertise and cultural backgrounds. Moreover, given the lack of users’ control in engaging or disengaging innate knowledge, our approach uses sensorimotor knowledge to inform the design of intuitive touchless command selection techniques.
  10. But then the question stands as…. how can we use sensorimotor knowledge to inform the design of intuitive touchless selection mechanisms?
  11. Specifically, how sensorimotor knowledge can inform the design of different features of a touchless menu? For example, how would we invoke a touchless menu. How would we end a touchless menu selection? What should be an optimal shape of the touchless menu? Until now, touchless command-selection techniques have been an extension of what has proven efficient for mouse-based, pen-based, or multitouch interfaces. However, touchless interaction has a very distinctive feature. It is device less. In existing touchless menus, user’s hand postures have strictly replaced the mechanics of other input devices. This trend may lead us to a local maxima where we’d fail to explore the potential of touchless interactions beyond the device-based paradigm.
  12. In designing our touchless menu, we wanted to port some successful features of ‘device-based’ menus, but not strictly emulate them.
  13. The outcome of our design exercise was touchless circular menus… Building upon prior sensorimotor knowledge of making directional strokes in mid-air… And some successful features of device-based menus, such as reaching a region-of-interest for menu-invocation….crossing using directional strokes for menu selection….and a radial shape
  14. A touchless circular menu relieves users from both recalling a precise vocabulary of hand-postures and strictly complying with them.
  15. Now let us see how the menu works..
  16. To select a menu option, the user makes a directional stroke. To give feedback, when successfully selected, the menu option changes color.
  17. To mitigate accidental invocation, TCM appear opposite to the user’s direction of approach. To cancel the invoked menu, the user may continue in her direction of movement, or move away from the menu.
  18. To invoke the menu, the user must reach the region-of-interest of the target.
  19. In hierarchical TCM, to select a command, the user first selects a menu option, and then she accesses the submenu by continuing her trajectory.
  20. Let’s now dig a bit deeper into the design rationale for the touchless circular menus.
  21. As we saw, to trigger the contextual menu, a user must cross the region-of-interest (ROI) of a display object. The ROI can be of any symmetrical shape around the center of the target, with its size directly proportional to the technique’s sensitivity.
  22. To select a command after triggering the menu, users cross it using a stroke in the command’s direction. Until the crossing happens, users can cancel TCM by moving in any direction away from the triggered menu. To allow easy escape routes, we designed the structure of TCM as a semicircular array of options appearing at the top-left or the bottom-right corner of the target.
  23. As users approach the menu, to give them orientation, a trace is drawn connecting the target and the users’ hand position. To improve users’ pointing performance, the menu options increase in amplitude as users approached them. To provide further feedback, menu options changed color when selected by crossing.
  24. Currently, our menu design scales up to two levels …5 x 5……with users performing continuous strokes. Users first select a root menu. Then to operate the submenu, users change their track and cross another command. In device-based hierarchical menus, submenus appear in the same direction of the root menu. Due to the lack of precision and control of freehand movements, TCM require users to make inflections in their continuing trajectories, and thereby avoid accidental command-selections.
  25. We evaluated single-level TCM in two experiments.
  26. TCM is a contextual menu for large displays. So in our first experiment, we investigated how effectiveness and efficiency of TCM is affected by their triggering locations? In the second experiment, we compared TCM with linear menus using grab gestures. For the linear menus, users invoked the menu with a grab gesture, and selected a menu –option using another grab gesture. A grab gesture was defined as an open hand…closing and opening again. For both these experiments, we had 15 participants, who were sitting away from a large display.
  27. In experiment 1, we had 9 trials and 7 blocks of repetition. In experiment 2, we had 6 trials and 7 blocks of repetition. For both the experiments, we used the same sample of participants. They were all right-handed. 4 participants were females. 8 out of 15 participants had prior familiarity with touchless gestures. And 11 out of 15 participants were below 30 years of age.
  28. As apparatus for our experiment we used a large display and a motion tracking sensor. The large display was built by Fakespace labs…It is 160 inch wide and 60 inch in height..with a total resolution of 15.3 M pixels. We would like to thank the IU Advanced visualization lab, a UITS Research Technologies division for the use of their facilities. As the tracking sensor, we used a Kinect for windows.
  29. For both of our experiments, we measured efficiency as the time on task and successful trigger rate. Successful triggers were When users triggered a menu, and continued to select an option from the triggered menu. If the menus was dismissed before any option was selected, it was considered an unsuccessful trigger. We measured effectiveness as error rate….. To measure user satisfaction, users were asked to complete the system usability scale and the NASA task load index.
  30. For experiment 1, we had two hypotheses: we predicted that triggering locations of the TCM will affect the performance time and the error rate.
  31. In our first experiment, we found that Triggering location of TCM significantly affected selection time and successful trigger rate. However, we did not find a significant effect of the triggering locations on the effectiveness of TCM. Users had mixed reaction about the menu. While one said “I felt I had to rush to select the menu option.”, another was surprised that he could do so well.
  32. Our findings suggest that arm posture affects users’ control on their hand movements and the required effort. This is completely in line with another study that was published last month in CHI. In this study, the authors found that “2D plane locations relative to the user’s body and arm configuration significantly affects the consumed endurance in touchless interactions.”
  33. Now Contextual touchless menus for large displays will always have different menu-triggering locations. Hence, Menu triggering locations will significantly affect the user experience of contextual touchless menus for large displays.
  34. For experiment 2, we predicted that TCM would be more efficient, and easier to use than linear menus.
  35. In our second experiment, we found that TCM were more efficient and cause lower workload than linear menus with grab gestures. However, TCM were less effective than linear menus. Again, users had mixed reaction about the linear menu. While a 20-something participant said “This is how I envision using touchless gestures.”…..for a 50-something participant …“It was a lot of effort.”
  36. Our findings suggest that Compared with linear menus using grab gestures, participants using TCM were twice faster in selecting commands and perceived lower workload. However, TCM caused 3% more errors than linear menus.
  37. So then the question was: “Why TCM were less effective?” To figure that out we can take a closer look at the mechanics of both linear menus using grab gestures, and TCM. For linear menus, one grab gesture was registered to invoke the menu, a second grab gesture was registered to select a menu option. The grab gesture was defined as an open hand..closing and opening again. In between the two gesture registrations, users could move their hand freely without affecting the on-going interaction. That is, user’s cursor would move, but not inadvertently select a menu option. Hence the gesture relaxation phase made the linear menus more effective.
  38. However, for TCM, after the menu was triggered, users could inadvertently move their hand and select a wrong command or..dismiss the menu. Unlike linear menus, TCM required users to strictly constrain their freehand movements after triggering the menu.
  39. As our future work, we plan to…
  40. To Minimize errors by constraining users’ freehand movements after triggering the TCM. And to Investigate bimanual gestures to operate menu levels in hierarchical TCM…for which we also plan to conduct empirical evaluations.
  41. So we proposed and evaluated TCM. Now The final takeaways. TCM relieve users from learning posture-based commands, and shift the interaction complexity from users’ input to the visual interface. We found that Menu triggering locations significantly affect the user experience of contextual touchless menus for large displays. Our results further suggest that Simple directional movements for selecting commands is more efficient and causes lower workload than (hand) posture-based techniques. Finally, our findings indicate that efficient gesture relaxation techniques may have the potential to increase effectiveness of touchless interactions.
  42. As a delimiter, dynamic gestures would be more efficient than static poses, as users would not have to halt-and execute a pose, but fluidly end the selection.