Exploring Usability Testing

TI
AM Tutorial
10/1/2013 8:30:00 AM

"Exploring Usability Testing"
Presented by:
Rob Sabourin
AmiBug.com

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Rob Sabourin
AmiBug.com
Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of
software development professionals. A well-respected member of the software engineering
community, Rob has managed, trained, mentored, and coached hundreds of top professionals
in the field. He frequently speaks at conferences and writes on software engineering, SQA,
testing, management, and internationalization.

Exploring Usability Testing
Robert Sabourin
President
AmiBug.Com, Inc.
Montreal, Canada
rsabourin@amibug.com
© 2013 Robert Sabourin EUT v1.1

1


WELCOME


© 2013 Robert Sabourin

2

1

• Robert Sabourin ,
Software Evangelist
• President
• AmiBug.Com Inc.
• Montreal, Quebec,
Canada
• rsabourin@amibug.com
• www.amibugshare.com
3


Exploring Usability
Testing

• Pain points?
– What hurts?
– How Much?



4

2


SOME PHILOSOPHY

5


Conforming to
requirements
Suiting purpose
Delivering value
to stakeholders


6

3

Edsger W. Dijkstra
• “Program testing can be used to show the
presence of bugs, but never to show their
absence”


7


NON-FUNCTIONAL
TESTING


8

4

Non-Functional Testing
Quality Factors
Attributes
Characteristics
Reliability
Other “-ilities”

9

Challenges
Goals
• Elicit
• Articulate
• Quantify

Subjective
• Needs or wants?
• Relative or absolute?
• What is good enough?

Tests
• Tricky to orchestrate
• Difficult to interpret results
• Challenging to baseline or regress


10

5

Non-Functional Agile
Challenges
Time
•
•
•
•
•

Analysis
Model
Prepare
Run
Interpret

Change
• Baselines
• Goals

Software
• Continuous integration
• Incomplete product builds
11


ISO 9126 Quality
Factors

Usability



• Attributes of the effort
needed for use, and on the
individual assessment of
such use, by a set of users
• Learnability
• Understandibility
• Operability

12

6

Quality Factors

13


Product Backlog
Constraints
Tom Gilb, Competitive Engineering, suggests defining
Scale: "What is measured"
Meter: "How to measure (method)"
Target: "Level we're aiming for. Success"
Constraint: "Level we're seeking to avoid.
Failure"
Benchmark: "Where we are today"


14

7



15


NET PROMOTER SCORE



16

8

Net Promoter Score

17


Net Promoter Score



18

9

Net Promoter Score

Response Scale
•11 values
•0 not at all likely
•10 extremely likely
19


Net Promoter Score
Promoters
• respondents giving a 9 or 10 score
Passives
• respondents giving a 7 or 8 score
Detractors
• respondents giving a 0 to 6 score


20

10

Net Promoter Score
NPS
• The difference between the percentage
of Promoters and Detractors
• Is not expressed as a percentage
• Is an absolute number lying between
-100 and +100
• uTest suggests “any positive number is
good” (ref: TK Maxx Mobile Usability
Testing Results Report)
21


Net Promoter Score



22

11


23


SYSTEM USABILITY SCALE



24

12

System Usability Scale
SUS - System Usability Scale

• Measures usability in terms of
effectiveness, efficiency and satisfaction
• SUS can be read as a percentage
• SUS can be used as a benchmarking
mechanism
• SUS can be used to compare the current
system to competitors’ systems
• SUS can be used to compare the current
system to future versions

25




26

13

SUS Questions
1.
2.
3.
4.

I think that I would like to use this system frequently
I found the system unnecessarily complex
I thought the system was easy to use
I think that I would need the support of a technical
person to be able to use this system
5. I found the various functions in this system were well
integrated
6. I thought there was too much inconsistency in this
system
7. I would imagine that most people would learn to use
this system very quickly
8. I found the system very cumbersome to use
9. I felt very confident using the system
10. I needed to learn a lot of things before I could get
going with this system

27


SUS Responses



28

14

Scoring SUS

• For odd items: subtract one from the user
response.
• For even-numbered items: subtract the
user responses from 5
• This scales all values from 0 to 4 (with
four being the most positive response).
• Add up the converted responses for each
user and multiply that total by 2.5. This
converts the range of possible values from
0 to 100 instead of from 0 to 40.
29


Scoring SUS



30

15


EXPLORATORY TESTING

31


Exploratory Testing

Learning

Execution



Design

32

16

Charter Statement

•
•
•
•
•
•

Statement of mission
Ties to purpose
Focuses work
Confirms understanding
Delineates scope
Analogy to test story
33


Charter Statement

•
•
•
•
•

Short, terse
To the point
Inclusions …
Exclusions …
Limits …



34

17

eBay Usability Charters


35




36

18



37




38

19

Pairing Testers With …

Non Functional Testing
•Usability Gurus
•SMEs
•Human Factors
Experts
39


Heuristics



40

20

Heuristics
Guide
Exploration
Rules of
thumb
Fallible but
useful


41


WHITEBOARDING



42

21

Prototype Styles
•
•
•
•
•
•
•
•
•
•
•
•

Paper and pencil
Comic books style story boards
Wire frame models
Mock ups
Prototypes
Site maps
User flow
Wire flow
Story boards
Wire frame
Page description diagram
Functional specification
43


Whiteboarding



44

22

Whiteboarding

What is Whiteboarding?

Collaboration
Coordination
Conceptualization
45


Mind Maps

Tony Buzan


46

23

Whiteboarding

47


User Mind Map

Whiteboarding



48

24

Buying a Book
Usage Scenarios

49


Story Board

Whiteboarding



50

25

Wrap-O-Matic
Usage Scenarios


51

Scenario Based Testing



52

26

Buying a Book
Usage Scenarios


53

Slide 54
EUT v1.1


27

EUT v1.1



56

28

Social Networking
Usage Scenarios


57


58


29



59




60

30



61




62

31



63




64

32



65




66

33

TOFT Testing
• Task Oriented
Functional Testing
– Can the user
accomplish useful
tasks correctly?


Slide 67


USABILITY HEURISTICS



68

34

Jakob Nielsen
• Usability heuristics
• Rules of thumb
• General principles
of user interface
design
• www.nngroup.com
69


Usability Heuristics

• Visibility of system status
– Always keep users informed
– Provide appropriate feedback
– Respond in a reasonable timeframe



70

35


• Match between system and the real
world
– Speak the users' language
– Use familiar concepts
– Follow a natural order

71



• User control and freedom
– Provide means to exit from unwanted states
– Provide undo and redo capabilities



72

36


• Consistency and standards
– Same words mean same thing in different
contexts
– Follow environment conventions

73



• Error prevention
– Prevents problems from occurring
– Eliminate error-prone conditions
– Have a confirmation option



74

37


• Recognition rather than recall
– Making objects visible
– Make instructions available

75



• Flexibility and efficiency of use
– Accelerators for the expert user
– Make common options adaptable



76

38


• Aesthetic and minimalist design

– Dialogues should contain
relevant information

77



• Help users recognize, diagnose, and
recover from errors
– Use plain language for error messages
– Constructively suggest solutions



78

39


• Help and documentation
–
–
–
–

Easy to search
Focused on the user's task
Offer concrete steps to be carried out
Short and to the point


79


USABILITY SEVERITY
LEVELS


80

40

Usability Severity


81


ISO 9421



82

41

Usability ISO 9421
Effectiveness
• the extent to which the intended goals of use are achieved

Efficiency
• the resources that have been expended to achieve the intended goals

Satisfaction
• the extent to which the user finds the use of the product acceptable

Safety
• can user be harmed

83


Usability ISO 9421
Completeness
• the extent to which the system is complete in design and meets its desired
goals
Learn-ability
• Is the capability of a software product to enable the user to learn how to
use it
Convenience
• the extent to which the product fits into normal course of daily life

Ambiguity
• Confusing or vague usability. Misleading information that might also result
in providing no usability at all


84

42


USABILITY TEST
FRAMEWORK

85

Usability Test Framework
Basic Usability Test Steps
• Select representative users
• Complete defined set of tasks
• Facilitator observes user behaviour
• Interview user after task is completed
• User is asked to think aloud (recorded)
• Review findings to identify concerns
• Generate actionable recommendations


86

43


User groups
• Select 5 or 6 users for each group
• All should attempt the same tasks
• Complex applications may need many groups
• Groups can be based
• Domain (subject matter) expertise
• Experience level (task or professional)
• Solution familiarity

87

Where to perform usability tests?
• Control lab
• Conference room with recording
equipment
• Work area with
• Local
• Distributed with local observer
• Distributed without local observer


88

44

Prepare Usability Scenario
• Identify project usability test objectives
• Select relevant user tasks
• Keep user instructions short and to the point
• Do not over complicate instructions
• Use the language of the user not the
language of the system
• Make task parameters specific
• Use a logical flow
89


Running a test
•Let participants make mistakes
•Take note of questions
•Take note of decisions points
•Take note of alternative
selections


90

45

Think Aloud
Approach

Retrospective
Think Aloud

• the user
verbalizes actions
and decisions
• user verbalizes
questions
concerns or
ambiguities

• records and eye
tracking software
record event
• review with user
and ask them to
explain their
actions
91


Reporting Findings
•
•
•
•

List of usability concerns
Summary of user questions
Description of users behaviour
Tasks of concern
• Too long
• Awkward
• Unsuccessful
• Time to complete
• Feedback
• Expected
• Unexpected


92

46

Measures and metrics
•Effectiveness
•Efficiency
•Satisfaction
•Error frequency
•Memorability
93


Measures and metrics
• Successful task completion
• Error rates
• Time on task
• Subjective comments
• Likes
• Dislikes
• Recommendations


94

47

Facilitating a test
• Make participant comfortable with environment and
technology
• Give participants a change to ask questions
• Explain process
• Ask participants about the amount of experience they
have with the technology
• Ask participants about the amount of experience they
have with the domain
• Review task description with participant
• Explain facilitators involvement

95


FORMS AND WORKFLOWS



96

48

Form Design Principles

•
•
•
•

Minimize pain
Illuminate path
Consider context
Ensure consistent communication
Luke Wroblewski
Web Form Design
97


Eye Tracking



98

49

Form Usability

•
•
•
•
•
•

Number and location of errors
Severity of errors
Completion rates
Time to complete forms
Satisfaction scores
Subjective comments

99


Awkward Form Path



100

50

Clear Form Path

101


Form Usability

• Eye tracking
– What people looked at
– Number of eye fixations
– Length of eye fixations



102

51

Form Usability Checklist

Any unnecessary questions
Succinct questions
Sincere voice
Natural language
Logical groups of information
Structure form as a conversation
Ask optional questions separately

103

Form Usability Checklist

Name the form
Name sections of the form
Clear scan line
Spacing between sections
Minimal distractions
Start page guidance for long forms
Indicate progress


104

52

Progress Indication

105


Input Validation

Suggest valid answers
Illustrate valid format
Validate syntax after user enters data
Indicate limits or ranges of inputs
Provide smart defaults
Hide unneeded controls



106

53

Error Messages

Provide clear error messages
Indicate if error blocks completion
Guide users to resolve error
Provide visual emphasis of error
Use red text and icons for errors
Indicate successful completion


107


DO IT YOURSELF
USABILITY TESTING


108

54

Do it yourself

109


Do it yourself
A morning a month
Get insights from users
Watch real users


110

55

Do it yourself
Recruit diversely
Find important problems
Improve design
111


Do it yourself
Three testers
Test on site
Observe shared screens


112

56

Do it yourself
Observers take notes
User questions recorded
Action decided at lunch

113


SOME MOBILE USABILITY
CONCERNS


114

57

Mobile Usability

Difficulties
•Small Screens
•Limited bandwidth
•Fingers
•No Flash Like GUI
115


Mobile Usability

Risks
•Bad first impression
•Operator errors
•Negative feedback / reviews
•Competitor fills gap


116

58

Mobile Usability
Guidelines
•
•
•
•
•

Important stuff on top
Easy to find stuff
Single column layout
Minimize navigation
Light data
117


Mobile Usability

Guidelines
•
•
•
•
•
•

Finger taps
Avoid fancy web gui technologies
Minimize forms
Minimize feature sets
Separate mobile web resources
Fast



118

59

Mobile Usability

Mobile Usability Test
•In the lab
•Beta
•Crowd source
119


Mobile Usability
Mobile Usability Test
• Vary and control
• Technologically
• OS
• Browser
• Device
• Carrier
• Bandwidth


120

60

Mobile Usability
Vary and control
• Geographically
• Continent
• Country
• City
• Language
• Locale
121


Mobile Usability
Vary and control
• Demographically
• Age
• Gender
• Education
• Employment
• Industry
• Computer savvy
• Domain expertise


122

61


LEAN UX

123


Lean UX



124

62

Lean UX
Lean Principles
• Cross functional teams
• Small teams
• Dedicated teams
• Collocated teams
• Progress equals Outcomes
• Problem focused teams
125


Lean UX
Lean Principles
• Remove waste
• Small batch size
• Continuous discovery
• GOOB Get out of the building
• Shared understanding
• No Rock Stars Gurus or Ninjas


126

63

Lean UX
Lean Principles
• Externalize Your Work
• Making over Analysis
• Learning over Growth
• Permission to Fail
• Getting out of the Deliverables
Business
127


Lean UX
Assumptions
Hypotheses
Outcomes
Personas
Features


128

64

Lean UX

Assumptions
•Business
•User
129


Lean UX

Hypotheses



We believe that
creating this
experience for these
personas will achieve
this outcome. We will
know this to be true
when we see this
feedback or measure.
130

65

Lean UX

Outcomes
More business
More referrals

More fun
131


Lean UX

Personas
• Sketch and Name
• Demographic
• Pain points & needs
• Potential solutions



132

66

Lean UX

Features are
•Capabilities
•Tactics
•Products
133


Lean UX

Features serve

•Business
•Customer
•User


134

67

Lean UX

Features drive
customer
behaviour
135


Lean UX

Process



136

68

Thank You
• Questions?



137

69

Exploring Usability Testing

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (13)

Ähnlich wie Exploring Usability Testing

Ähnlich wie Exploring Usability Testing (20)

Mehr von TechWell

Mehr von TechWell (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Exploring Usability Testing