During this webinar, Managing Director Robert Schumacher discussed the recently released Electronic Health Record (EHR) Usability Protocol from the National Institute of Standards and Technology. User Centric has been heavily involved in ensuring EHRs are evaluated for usability.
This webinar provides insights and tactics to help you:
• Position usability next to other methods of evaluating electronic health records
• Show the application of some key human performance principles to EHR design
• Discuss the emerging methods of expert review and validation testing for evaluating and measuring user performance of EHRs
(With audio: http://vimeo.com/36793907)
http://www.usercentric.com/healthcare
6. Utility (Usefulness)
Useful Does Not Mean Usable
Usable Does Not Mean Useful
Usability
7. yü-zə-ˈbí lə-tē
-
from O.Fr. usable (1311), from user (see use). Not a common word before c.1840.
The extent to which a product can be used
by specified users to achieve specified
goals with effectiveness, efficiency, and
satisfaction in a specified context of use
ISO definition
8.
9.
10.
11. Psychology
Cognitive
Human Factors
Engineering::Physics
as
Human Factors::Psychology
Usability
12. Methods of measuring Principles of behavior that
behavior inform design
Reaction time Performance Memory
Errors Cognitive workload Motor capabilities
Eye tracking Attention Signal detection
… Psycholinguistics
Choice theory
…
There is a rich history of Knowledge of human
constructing studies to capabilities embodied in user
understand and explain interface design can optimize
human performance user performance
13. Methods of measuring Principles of behavior that
behavior inform design
Reaction time Performance Memory
Errors Cognitive workload Motor capabilities
Eye tracking Attention Signal detection
… Psycholinguistics
Choice theory
…
There is a rich history of Knowledge of human
constructing studies to capabilities embodied in user
understand and explain interface design can optimize
human performance user performance
14. A Negative A Positive statement,
statement, with No with Yes (affirmative)
(negative) response response
Yes or No?
The circle isis above the star
The circle not above the star
15. Negative statement, Positive statement,
with No (negative) with Yes (affirmative)
response response
The circle is not The circle is
above the star above the star
• Positive statements evoke faster responses than negative statements
• Yes responses are much faster than No responses (~ 500 msec)
16. Positive
Logically
Negative
Equivalent
Affirmative Negative
Response Response
Different phrasings are used when you want different responses
- Large variability in cognitive processing times
- Significant impact on error
17. Statement Type Use When
True/Affirmative Fast, easy, low-cost-to-user outcome,
confirmation only
False/Affirmative Need user to think about the response, high
cost to use
True/Negative Should almost never use
False/Negative Never use, unless trying to deceive user
e.g., Opt out response
18. Methods of measuring Principles of behavior that
behavior inform design
Reaction time Performance Memory
Errors Cognitive workload Motor capabilities
Signal detection
Eye tracking Attention
Psycholinguistics
… Choice theory
…
There is a rich history of Knowledge of human
constructing studies to capabilities embodied in user
understand and explain human interface design can optimize
performance user performance
24. How do we measure that
change?
How do we understand
users, tasks and
environments?
25. Usability / UX Methods
Access
Testing
Card sorting
User Experience Participatory design
(UX) Research
Methods Focus groups
Eye tracking
Ethnographic
Cognitive
research
walkthrough
Usability Validation testing
Evaluation
Methods
Formative usability
Heuristic evaluation testing
KLM-
GOMS
Surveys
Do not involve users Involve users
25
26. Eye Tracking
Cognitive Formative Usability
Modeling Testing
Validation Testing
Focus Group
Expert
Review
The trick is to know when to use each tool
;
; ;
27. Describe Improve
FieldLab Environment/Tasks
Users
DevelopmentDeployed
Users
QualitativeQuantitative
AbsoluteRelative
Tasks/Users
Users
NovicesExperts
Walk Up And UseRequire Training
Tasks/Environment Attitudinal Behavioral Users
28. Fully
Deployed
Eye tracking
Validation
testing
Expert Evaluation
Describe
Improve
Cognitive Walkthrough
Formative
usability
Heuristic Evaluation testing
Focus groups
Observational Participatory
research design Card sorting
Early
Development
30. Collaborators: Supported & Improved by:
- Svetlana Lowry, NIST - SHARPC - Jiajie Zhang’s Lab
- Emily Patterson, OSU and others…
- M. Chris Gibbons, Johns Hopkins
- Robert North, Human Centered Technologies
31. Apply existing established human
factors methods in the technical
evaluation (i.e., measurement) of EHR
Usability
Focus on critical aspects of usability
(related to patient safety)
Provide detailed systematic steps for
conducting measurement usability
validation studies
Provide a framework for reporting
usability findings
Not a policy document or framework
32.
33.
34. Who are the users?
What is the user’s environment?
What are their tasks?
How well does the application do the
tasks in the environment for the users?
What mistakes might users make?
How has the application been designed to mitigate mistakes and
optimize usability?
35. An evaluation of the critical components of the user interface
by executing various use case scenarios.
Clinical subject matter experts together with usability / human
factors experts conduct a structured evaluation of the user
interface.
36.
image
• In order to assure independence in the review process, each expert reviews the
user interface on his/her own and produces a set of ratings and findings.
• Once all experts have completed the evaluations, a lead expert consolidates all
the findings into a single report.
46. Common
Industry Format (CIF)
• Specifies format for reporting
results of a summative
usability evaluation
• CIF approved as ISO
standard ISO/IEC 25062:2006
• CIF format was agreed by the
IUSR working group:
IBM, Microsoft, HP, Boeing, U
S West, Kodak
• See NIST IR 7742
48. Presenter:
Robert Schumacher, PhD.
Managing Director
User Centric, Inc
rschumacher@usercentric.com
User Centric is a global user experience
research and design firm.
We believe experiences matter.TM
www.usercentric.com
Oakbrook Terrace Chicago Atlanta
630-320-3900
Connect with us: @UserCentricInc
Webinar Schedule (www.usercentric.com/webinars):
Behaviors First, Colors Second: The Importance of Interaction
Design
Thursday, March 8
Medical Devices: Catching up to the FDA
Tuesday, March 27
49. Presenter:
Robert Schumacher, PhD.
Managing Director
User Centric, Inc
rschumacher@usercentric.com
User Centric is a global user experience
research and design firm.
We believe experiences matter.TM
www.usercentric.com
Oakbrook Terrace Chicago Atlanta
630-320-3900
Connect with us: @UserCentricInc
Webinar Schedule (www.usercentric.com/webinars):
Behaviors First, Colors Second: The Importance of Interaction
Design
Thursday, March 8
Medical Devices: Catching up to the FDA
Tuesday, March 27