Identifying your goals and measuring results is essential to creating a successful web presence. And, testing every aspect of your online presence/campaigns is the only way to determine if your efforts are truly effective and how they can be further optimized.
Saavy marketers are utilizing various testing techniques to answer vital questions regarding site and campaign performance – testing everything from wording, placement, color, processing, and flow. The objective is to understand your user’s preferences and goals, and then to identify the best way to present your information so that it enables them to accomplish their goal and compels them toward the desired action.
2. Agenda
• The W’s
• Types of User-Testing
• How User-Testing Fits in the
Web Development Process
– Persona Development
– Information Architecture Testing
(IAT)
– Visual Affordance Testing (VAT)
– User Acceptance Testing (UAT)
– Multivariate Testing (MVT)
4. What?
• What is User-Testing?
User-Testing is a focal part
of the User Centered Design
philosophy that aims to gain
direct feedback from users and
collect actionable data
5. What?
• What Can We Test?
•Wireframes
•Designs
•Prototypes
•Live Websites
•Everything!
6. Why?
• Why Test Users?
• We often think we
know what is best for
our users
• You want to add
innovative
functionality to your
site
• You have low
conversion rates
• You have high
customer support
7. When?
Test Expectations
/ Mental Model
Test User
Benchmark Expectations
Test Visual
Affordance
Test Live
Website
Test User
Performance
8. Who?
• Who Do We Test?
• Users, not Testers
• Real people who use your
website
• Rinse, Lather, REPEAT!
9. Who?
• Who Do We Listen
To? We need to place four ad
banners on the homepage
I want to read content
that is fresh and
interesting
We need to drive users
to download our latest
whitepaper
I don’t want my
decisions manipulated
We want to display
company news and our
stock ticker on the
homepage
I know what I want and
want to find it
without being
distracted
10. Where?
• Where Do We Find Users to Test?
•On your website
•Through a recruiter
•In their natural
environment
11. How?
• How can we avoid having to
figure all this out on our own?
Let the users do
all the hard work
for you!
14. Types of User-Testing:
Remote
• Users sit in
their natural
environment
• Users are asked
to carry out
specific tasks
using prototypes
• Their mouse
movements and the
phone
conversation are
recorded for
15. Types of User-Testing: In
Lab
• Video camera
records users’
facial expressions
• A screen recorder
records their
screen movements
• A facilitator is in
the room with them
• Observers are in
another room,
usually looking
through a 2 way
16. Types of User-Testing:
Automated
• Users are
intercepted when
they appear on a
site
• Users are asked to
fill out a short
survey before
navigating the site
• All the user’s
movements are
recorded while
navigating the site
• Users complete an
exit survey when
they leave the site
17. Types of User-Testing
Type Pros Cons What to Use it On
Remote Nationally diverse geographic reach Lose the ability to see users’ E-commerce web sites
expressions and gestures
Quick Large, informational web sites
Unnatural environment
Less Costly Web Applications
Intranets
In Lab Obtain data based on user’s body More costly Highly secure client/server
movement and facial expressions applications
Lack of geographic reach
Handheld Apps or other
products with a significant
hardware component
Automated Captures users that come to the site Steps in between start and goal E-commerce web sites
and records users’ activities is not elicited
Large, informational web sites
Natural environment
Web Applications
Actual users
Intranets
18. Types of Data Collected
• Quantitative • Qualitative
– Statistics, – Opinions,
Trends Details
– Large Sample Size – Small Sample
– Structured Data Size
Collection – Unstructured
Methods Data Collection
Example:
Methods
Example:
•30% of users clicked on the red ―I tried clicking on the button
button because the white text on the
•70% of users clicked on the blue blue background caught my eye
button and the text was large‖
19. Informal vs. Formal
Testing
• Informal • Formal
– Casual – Structured
– 3-4 Days – 3-4 Weeks
– In Your Office – In a Lab
– Friends & Family – Broad Range of
– Paper Prototypes Users
– Less Common – High Fidelity
– More Common
– Bias
– Incorrect – Trained Moderators
Analysis & Analysts
20. POLL
• Have you conducted any type of
User-Testing on your current
website?
22. The Web Development Process
Test Expectations
/ Mental Model
Test User
Benchmark Expectations
Test Visual
Affordance
Test Live
Website
Test User
Performance
23. The Testing Methods
Card Sorting
Wireframe
Persona Testing
Development
Visual
Affordance
Testing
Multivariate
Testing
User
Acceptance
Testing
25. PD:
Card Sorting
Wireframe
Persona Testing
Development
Visual
Affordance
Testing
Multivariate
Testing &
Optimization
User
Acceptance
Testing
26. PD:
Q: How do we develop a website that
is tailored to our users needs,
behaviors, experience, and
technology?
A Ask
them!
:
27. PD: What is a Persona?
• A persona tells the story about
the users of your website and
attributes lifelike features to
make them more relatable
• Based on real data, the persona
serves as a tool to help everyone
involved in development
understand who and what the
website is for
• The persona creates a shared
vision
28. PD: Market Segments vs.
Personas
Market Segment
• Gender: 75% Female, 25% Male
• Region: 99% in United States
• Size: 70% metro, 20% suburban, 10% rural
• Generation: 10% baby-boomers, 20% Gen-X, 70% Gen-Y
• Income: 25k – 50k
• Occupation: Student, Administrative Assistant,
Intern
• Education: High School Diploma
• Family Lifecycle: Single
30. PD: Surveys & Interviews
Surveys Interviews (contextual +
• Quantitative Method individual)
• Need about 400 • Qualitative Method
responses to account • Need about 8 participants
for margin of error per segment
31. PD: Benefits
• Focus on the users’ goals and
needs throughout the development
process
• Prioritize features, design,
efforts based on what is most
needed by the personas
• Any internal battles will can be
settled by referring to the
personas
32. PD: Tips for Success
• Always include:
– A Name
– A Photo
– Demographics
– Description based on data collected
– Goals
– Common tasks carried out on the website
– Habits
• Always refer back to the personas!
34. IAT: Card Sorting
Card Sorting
Wireframe
Persona Testing
Development
Visual
Affordance
Testing
Multivariate
Testing &
Optimization
User
Acceptance
Testing
35. IAT: Card Sorting
Q: How do we ensure that the
taxonomy and site linking
structure are intuitive for our
users to find our products and
services?
A Have the user label
and organize pages
: for you!
36. IAT: What is Card Sorting?
• Users are given ―cards‖ with
labels on them and asked to
categorize the information on
your site into meaningful
groupings.
• Method: In-Lab or Automated
• Sample Size: At least 15 per
segment
37. IAT: Types of Card Sorting
Open Card Sorting Closed Card Sorting
• Categories and • Fixed categories are
category labels can provided to the user
be created or and they may not re-
changed by users label or created new
categories.
• Used early during
the development
cycle • Used later in
development cycle
• More difficult to for further testing,
analyze data or to add new
content to the
website
• Easier to analyze
data statistically
39. IAT: Card Sorting Tools
(Online)
• Online tools allow for easier
data collection / analysis and
larger sample sizes
40. POLL
• What is your primary method for
conducting card sorting?
41. IAT: Card Sorting Benefits
• Easy to administer
• Sets the foundation for the website’s navigation
and structure
• Intuitive taxonomy in the users’ ―natural
language,‖ and matches what users are searching for
(SEO)
• Intuitive taxonomy in the users’ ―natural
language,‖ and matches what users are searching for
(SEO)
43. IAT: Card Sorting Tips for
Success
• If in-lab, ask users to talk out
loud
• Limit the study to a reasonable
amount of cards (~50)
• Note the relationships between
cards and situations that indicate
cross-linking between categories
will be helpful for users
44. IAT: Wireframe Testing
Card Sorting
Wireframe
Persona Testing
Development
Visual
Affordance
Testing
Multivariate
Testing &
Optimization
User
Acceptance
Testing
45. IAT: Wireframe Testing
Q: How do we find out if the layout
of information on the site is
intuitive, and if the placement of
images, text, links, messaging, and
calls-to-action help users find
what they need?
A Watch users as they
navigate through
: the wireframes!
46. IAT: What is Wireframe
Testing?
• Wireframe testing is a usability test of
the prototype pages of your site
excluding colors and other details. This
involves creating realistic tasks and
having users navigate the prototype to
fulfill those tasks.
47. IAT: Wireframe Testing
• Method: In-Lab or Remote
• Sample Size: At least 5 per
segment
– Why 5 Users?
48. IAT: Wireframe Testing
Example
• You are out in the
field and you need to
find information on
how to troubleshoot
the network. Where
would you start?
• Your business has just
switched over to VoIP.
Where would you go to
look for information
on how to manage your
new VoIP network?
49. IAT: Wireframe Testing:
Benefits
• Find out if the layout of the page
achieves your users’ goals and your
goals
• Low cost and quick to make changes
to a wireframe versus a real
website
50. IAT: Wireframe Testing: Tips
for Success
• Create tasks that are real
situations for the user (refer to
the personas)
• Do not bias the participant by
leading or priming
• Be patient!
52. VAT:
Card Sorting
Wireframe
Persona Testing
Development
Visual
Affordance
Testing
Multivariate
Testing &
Optimization
User
Acceptance
Testing
53. VAT:
Q: How do we make sure the design is
not only aesthetically pleasing,
but also that messaging and calls-
to-action are clear for the user?
A Ask users, and
watch as they
: navigate through
the designs!
54. VAT: What is Affordance?
• Affordance is a quality of an
object, or an environment, that
allows an individual to perform
an action.
• A visual clue to the function
of an object
55. VAT: What is Visual
Affordance Testing?
• Once images, colors, and other
design elements have been
implemented based on the
wireframes, VAT requires users to
indicate what they perceive as
clickable on the design
• Method: In-Lab or Remote
• Sample Size: At least 5 per segment
56. VAT: Benefits
• Validates if key calls-to-action
are even seen by users
• Reveals elements of a design which
may mislead users to believe are
clickable
• Reduce user frustration by
addressing problems found in the
VAT
58. VAT: Tips for Success
• Pay close attention to any
hesitation or vocal expressions
• Visual affordance does not only
apply to calls-to-action or links
60. UAT:
Card Sorting
Wireframe
Persona Testing
Development
Visual
Affordance
Testing
Multivariate
Testing &
Optimization
User
Acceptance
Testing
61. UAT:
Q: How do we find out if all the
content, interactions, and media
on the prototype we’ve built will
resonate with our visitors, and if
they negatively affected our
site’s usability?
A Ask users, and watch as
they navigate through the
: prototype!
62. UAT: What is User Acceptance
Testing?
• User Acceptance Testing is a
usability test conducted on a
fully functional prototype
prior to launching the website
• Method: In-Lab or Remote
• Sample Size: At least 5 per
segment
63. UAT: Benefits
• Find out how users will really
use the website
• Figure out if all functionality
and interactivity is useful to
the user (Flash, Videos, AJAX)
• Observe where people may have
difficulty or fail and how they
recover from those errors
64. UAT: Tips for Success
• User Acceptance Testing is not
Quality Assurance!
• Allow for free exploration with
the user thinking aloud
• Test iteratively
65. POLL
• Have you added any new content
to your website in the past 6
months (whitepapers, downloads,
videos)?
67. MVT:
Card Sorting
Wireframe
Persona Testing
Development
Visual
Affordance
Testing
Multivariate
Testing &
Optimization
User
Acceptance
Testing
68. MVT:
Q: How do we make sure that our
site is always usable and that the
user-experience of our visitors is
always optimal?
A Continue to test and
optimize your site!
:
69. MVT: What is Multivariate
Testing?
• Once the website is launched,
Multivariate Testing optimizes
performance of the website by
segmenting visitors and displaying
different images/ buttons/ links/
content to them
• Method: Automated, In-Lab, Remote
• Sample Size: At least 500 visitors
71. MVT: Benefits
• Minimize risk with any website
changes
• Data drives decisions
• Data is collected 24/7
• Tangible Results
• Test New Applications
• Test the ―Third Screen‖
72. MVT: Tips for Success
• ABT – Always Be Testing!
• Preparation and set-up is
crucial
74. ―If ease of use was the only valid
criterion, people would stick to
tricycles and never try bicycles‖
Doug Engelbart
Human-Computer Interaction Pioneer
and inventor of the computer mouse
75. Thank You!
Jegan Chen, Usability Engineer
BusinessOnLine
Jegan.Chen@BusinessOL.com
http://www.businessol.com
Additional BusinessOnLine
Resources:
Re:Cognition: BusinessOnLine’s Usability Blog
http://www.businessol.com/usability_blog
Upcoming Webinar:
SEO Webinar Series:
Session 3: Not Just Video—VideoSEO
June 17, 2009 11am PST (2pm EST)
http://www.businessol.com/news/main-webinars/