SlideShare a Scribd company logo
1 of 28
Download to read offline
How to find out about
the usability
of your website
using a
surveyUX Cambridge 2012 workshop led by Caroline Jarrett
I got asked this question recently.
What are your views?
“ Can a usability test just
use a questionnaire,
no observation? ”
2
“Can a usability test just use a questionnaire,
no observation?” << NO
• We agreed that a usability test must have observation,
otherwise it’s not a usability test
• Recommended reading
Steve Krug
“Rocket Surgery
Made Easy”
Carol M.
Barnum
“Usability
Testing
Essentials” Workshop
results
3
I rejected the idea of using a questionnaire
alone. But let’s ponder this question.
“How to find out
about the usability
of your website
using a survey”
4
UX people typically encounter surveys
in these three ways
1. Post-test / post-task
surveys e.g. SUS
2. Someone is going
to do a survey anyway
3. Triangulating between
survey data and
data from elsewhere
http://www.slideshare.net/cjforms/
better-ux-surveys-at-ucd2012-by-cjforms
Today’s topic,
this time focusing
on usability
5
Let’s unpick the standard definition.
Which bits can we assess using a survey?
The extent to which a product
can be used by specified users
to achieve specified goals
with effectiveness, efficiency
and satisfaction
in a specified context of use
(ISO 9241:11 1998)
6
We picked out many details in the definition
that we could explore in surveys
Examples from the discussion:
• “Specified users” - demographics, attitudes, history
• “Specified goals” – what do users want to achieve?
• “Satisfaction”
– What did they expect?
– What is their assessment after the experience?
Corbin, J. M. and Strauss, A.L. (2008)
“Basics of qualitative research: techniques and
procedures for developing grounded theory”
Workshop
results
7
Example: the product.
Is that the whole website, a section, a page?
The extent to which a product
can be used by specified users
to achieve specified goals
with effectiveness, efficiency
and satisfaction
in a specified context of use
(ISO 9241:11 1998)
8
Used properly, NPS® (Net Promoter Score)
takes “the product” up another level – to brand
“Would you
recommend us
to a friend?”
Image credit: Rodney Burton, geograph.org.uk
Image credit:
Richard Vince, geograph.org.uk
Image credit:
Cambridge County Council
Reichheld, F. F. (2006)
The ultimate question: driving good profits and true growth
9
But sometimes NPS® gets muddled up with
other things
10
Tip
Ask one question at a time
Image credit:
Shutterstock
11
My favourite bit to look at with surveys:
what is the user’s concept of the goal?
The extent to which a product
can be used by specified users
to achieve specified goals
with effectiveness, efficiency
and satisfaction
in a specified context of use
(ISO 9241:11 1998)
What the user
wants to do
Tip
Find out about users’ goals
13
There are other goals to consider
What the
organisation
wants to achieve
What the user
wants to do
Our aims in
doing a survey
14
A set of questions starts to become a survey
when we know our goals for the survey
Goals
• Establish
your goals
for the
survey
Questions
you need
answers to
• Final version
of questions
• Build the
questionnaire
Users Build Deploy Analyse
15
• Interview
users about
the topics in
your survey
• Run the
survey from
approach to
follow-up
• Extract
useful ideas
• Share with
others
Questions
users can
answer
Questionnaire Data Insight
Exploratory:
Don’t know what is out there. Send and hope.
Image credit:
Shutterstock
16
Comparative:
Explore trends, look at before-and-after, trackers
Image credit:
Shutterstock 17
Descriptive:
Do know what is out there. Go and count them.
Image credit:
Caroline Jarrett
18
Modelling:
Find factors that show cause and effect
Image credit:
Shutterstock
19
Example: The 1946 Cohort Study combines
comparative, descriptive, and modelling
Cumulative death rates by father's social class at birth in the 1946 birth cohort study
Adapted from Kuh, D., Hardy, R., Langenberg, C., Richards, M. and Wadsworth, M. E. J. (2002),
Mortality in adults aged 26-54 years related to socioeconomic conditions in childhood and adulthood: post war birth cohort study.
20
Exploratory, comparative, descriptive,
modelling? Let’s discuss (from a pop-up survey)
21
We found plenty to say about the details of the questions –
but this is a COMPARATIVE survey, so keeping questions
the same whenever it is used is crucial
Workshop
results
22
The ideal: everything in balance
Comparisons
Good
questions
23
To get maximum insight from your survey,
follow all the steps.
• Final version
of questions
• Build the
questionnaire
Goals Users Build Deploy Analyse
• Establish
your goals
for the
survey
• Interview
users about
the topics in
your survey
• Run the
survey from
approach to
follow-up
• Extract
useful ideas
• Share with
others
Questions
you need
answers to
Questions
users can
answer
Questionnaire Data Insight
24
Tip
Interview before build
25
Bonus
Tip
Survey = Questionnaire
+ Process
Successful
That involves
lots of testing
26
Caroline Jarrett
Twitter @cjforms
http://www.slideshare.net/cjforms
carolinej@effortmark.co.uk
27
More resources on
http://www.slideshare.net/cjforms
28

More Related Content

What's hot

10 tips for a better survey at UX Bristol
10 tips for a better survey at UX Bristol10 tips for a better survey at UX Bristol
10 tips for a better survey at UX BristolCaroline Jarrett
 
Forms that work: Understanding forms to improve their design by @cjforms
Forms that work: Understanding forms to improve their design by @cjformsForms that work: Understanding forms to improve their design by @cjforms
Forms that work: Understanding forms to improve their design by @cjformsCaroline Jarrett
 
IUE14 Presentation - Studies UX Pros should know
IUE14 Presentation - Studies UX Pros should know IUE14 Presentation - Studies UX Pros should know
IUE14 Presentation - Studies UX Pros should know Kath Straub
 
Causal data mining: Identifying causal effects at scale
Causal data mining: Identifying causal effects at scaleCausal data mining: Identifying causal effects at scale
Causal data mining: Identifying causal effects at scaleAmit Sharma
 
Self-service design - Eye-tracking Findings That Will Help You Design Forms T...
Self-service design - Eye-tracking Findings That Will Help You Design Forms T...Self-service design - Eye-tracking Findings That Will Help You Design Forms T...
Self-service design - Eye-tracking Findings That Will Help You Design Forms T...Kath Straub
 
The Digital Academic
The Digital AcademicThe Digital Academic
The Digital AcademicMark Reed
 
Altmetrics: The Movement, The Tools, and the Implications
Altmetrics: The Movement, The Tools, and the ImplicationsAltmetrics: The Movement, The Tools, and the Implications
Altmetrics: The Movement, The Tools, and the ImplicationsCMHSL
 
How to avoid research debt
How to avoid research debtHow to avoid research debt
How to avoid research debtCaroline Jarrett
 
Our hearts beat for people: UX research in agile contexts.
Our hearts beat for people: UX research in agile contexts. Our hearts beat for people: UX research in agile contexts.
Our hearts beat for people: UX research in agile contexts. johanna kollmann
 
Inwards and outwards research: choosing your research methods according to th...
Inwards and outwards research: choosing your research methods according to th...Inwards and outwards research: choosing your research methods according to th...
Inwards and outwards research: choosing your research methods according to th...Caroline Jarrett
 
Writing surveysthatwork
Writing surveysthatworkWriting surveysthatwork
Writing surveysthatworkrebeccaweiss
 
MIS5101 Week 13 Security Privacy Data Mining
MIS5101 Week 13 Security Privacy Data MiningMIS5101 Week 13 Security Privacy Data Mining
MIS5101 Week 13 Security Privacy Data MiningSteven Johnson
 
Fast & Cheap UX Research
Fast & Cheap UX ResearchFast & Cheap UX Research
Fast & Cheap UX ResearchJodi Bollaert
 
Audience Research on a Dime - Nonprofit of Influence
Audience Research on a Dime - Nonprofit of InfluenceAudience Research on a Dime - Nonprofit of Influence
Audience Research on a Dime - Nonprofit of InfluenceCourtney Clark
 
Do no harm: Tips for avoiding the unethical turn of events in user research (...
Do no harm: Tips for avoiding the unethical turn of events in user research (...Do no harm: Tips for avoiding the unethical turn of events in user research (...
Do no harm: Tips for avoiding the unethical turn of events in user research (...Jennifer (Jen) McGinn
 
Label placement on forms, STC Seattle 2010
Label placement on forms, STC Seattle 2010Label placement on forms, STC Seattle 2010
Label placement on forms, STC Seattle 2010Caroline Jarrett
 
Language Empowered Recommendations
Language Empowered RecommendationsLanguage Empowered Recommendations
Language Empowered RecommendationsHari Sanadhya
 

What's hot (17)

10 tips for a better survey at UX Bristol
10 tips for a better survey at UX Bristol10 tips for a better survey at UX Bristol
10 tips for a better survey at UX Bristol
 
Forms that work: Understanding forms to improve their design by @cjforms
Forms that work: Understanding forms to improve their design by @cjformsForms that work: Understanding forms to improve their design by @cjforms
Forms that work: Understanding forms to improve their design by @cjforms
 
IUE14 Presentation - Studies UX Pros should know
IUE14 Presentation - Studies UX Pros should know IUE14 Presentation - Studies UX Pros should know
IUE14 Presentation - Studies UX Pros should know
 
Causal data mining: Identifying causal effects at scale
Causal data mining: Identifying causal effects at scaleCausal data mining: Identifying causal effects at scale
Causal data mining: Identifying causal effects at scale
 
Self-service design - Eye-tracking Findings That Will Help You Design Forms T...
Self-service design - Eye-tracking Findings That Will Help You Design Forms T...Self-service design - Eye-tracking Findings That Will Help You Design Forms T...
Self-service design - Eye-tracking Findings That Will Help You Design Forms T...
 
The Digital Academic
The Digital AcademicThe Digital Academic
The Digital Academic
 
Altmetrics: The Movement, The Tools, and the Implications
Altmetrics: The Movement, The Tools, and the ImplicationsAltmetrics: The Movement, The Tools, and the Implications
Altmetrics: The Movement, The Tools, and the Implications
 
How to avoid research debt
How to avoid research debtHow to avoid research debt
How to avoid research debt
 
Our hearts beat for people: UX research in agile contexts.
Our hearts beat for people: UX research in agile contexts. Our hearts beat for people: UX research in agile contexts.
Our hearts beat for people: UX research in agile contexts.
 
Inwards and outwards research: choosing your research methods according to th...
Inwards and outwards research: choosing your research methods according to th...Inwards and outwards research: choosing your research methods according to th...
Inwards and outwards research: choosing your research methods according to th...
 
Writing surveysthatwork
Writing surveysthatworkWriting surveysthatwork
Writing surveysthatwork
 
MIS5101 Week 13 Security Privacy Data Mining
MIS5101 Week 13 Security Privacy Data MiningMIS5101 Week 13 Security Privacy Data Mining
MIS5101 Week 13 Security Privacy Data Mining
 
Fast & Cheap UX Research
Fast & Cheap UX ResearchFast & Cheap UX Research
Fast & Cheap UX Research
 
Audience Research on a Dime - Nonprofit of Influence
Audience Research on a Dime - Nonprofit of InfluenceAudience Research on a Dime - Nonprofit of Influence
Audience Research on a Dime - Nonprofit of Influence
 
Do no harm: Tips for avoiding the unethical turn of events in user research (...
Do no harm: Tips for avoiding the unethical turn of events in user research (...Do no harm: Tips for avoiding the unethical turn of events in user research (...
Do no harm: Tips for avoiding the unethical turn of events in user research (...
 
Label placement on forms, STC Seattle 2010
Label placement on forms, STC Seattle 2010Label placement on forms, STC Seattle 2010
Label placement on forms, STC Seattle 2010
 
Language Empowered Recommendations
Language Empowered RecommendationsLanguage Empowered Recommendations
Language Empowered Recommendations
 

Similar to How to find out about the usability of your web site using a survey by @cjforms

A deep dive into questions by @cjforms at UxLx
A deep dive into questions by @cjforms at UxLxA deep dive into questions by @cjforms at UxLx
A deep dive into questions by @cjforms at UxLxCaroline Jarrett
 
Five Steps to Better Surveys: part of the Surveys that Work training course f...
Five Steps to Better Surveys: part of the Surveys that Work training course f...Five Steps to Better Surveys: part of the Surveys that Work training course f...
Five Steps to Better Surveys: part of the Surveys that Work training course f...Caroline Jarrett
 
ICIS Rating Scales for Collective IntelligenceIcis idea rating-v1.0-final
ICIS Rating Scales for Collective IntelligenceIcis idea rating-v1.0-finalICIS Rating Scales for Collective IntelligenceIcis idea rating-v1.0-final
ICIS Rating Scales for Collective IntelligenceIcis idea rating-v1.0-finalriedlc
 
Immersive Recommendation Workshop, NYC Media Lab'17
Immersive Recommendation Workshop, NYC Media Lab'17Immersive Recommendation Workshop, NYC Media Lab'17
Immersive Recommendation Workshop, NYC Media Lab'17Longqi Yang
 
Brightfind world usability day 2016 full deck final
Brightfind world usability day 2016   full deck finalBrightfind world usability day 2016   full deck final
Brightfind world usability day 2016 full deck finalBrightfind
 
UX STRAT Online 2021 Presentation by Rina Tambo Jensen
UX STRAT Online 2021 Presentation by Rina Tambo JensenUX STRAT Online 2021 Presentation by Rina Tambo Jensen
UX STRAT Online 2021 Presentation by Rina Tambo JensenUX STRAT
 
Essay On Psychology And Square Foot Condominium
Essay On Psychology And Square Foot CondominiumEssay On Psychology And Square Foot Condominium
Essay On Psychology And Square Foot CondominiumAshley Smith
 
Interleaving - SIGIR 2016 presentation
Interleaving - SIGIR 2016 presentationInterleaving - SIGIR 2016 presentation
Interleaving - SIGIR 2016 presentationXin QIAN
 
How NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling DataHow NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling DataDataCards
 
Social machines: theory design and incentives
Social machines: theory design and incentivesSocial machines: theory design and incentives
Social machines: theory design and incentivesElena Simperl
 
Taxonomy, Social Networks and Pace Layering
Taxonomy, Social Networks and Pace LayeringTaxonomy, Social Networks and Pace Layering
Taxonomy, Social Networks and Pace LayeringRoger Hudson
 
An informatics perspective on argumentation mining - SICSA 2014-07-09
An informatics perspective on argumentation mining - SICSA 2014-07-09An informatics perspective on argumentation mining - SICSA 2014-07-09
An informatics perspective on argumentation mining - SICSA 2014-07-09jodischneider
 
Visualization for public policy (2016)
Visualization for public policy (2016)Visualization for public policy (2016)
Visualization for public policy (2016)Arthi Krishnaswami
 
Measuring effectiveness of machine learning systems
Measuring effectiveness of machine learning systemsMeasuring effectiveness of machine learning systems
Measuring effectiveness of machine learning systemsAmit Sharma
 
Scalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingScalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingKatrien Verbert
 
2 Studies UX types should know about (Straub UXPA unconference13)
2 Studies UX types should know about (Straub UXPA unconference13)2 Studies UX types should know about (Straub UXPA unconference13)
2 Studies UX types should know about (Straub UXPA unconference13)Kath Straub
 
Evaluation and User Study in HCI
Evaluation and User Study in HCIEvaluation and User Study in HCI
Evaluation and User Study in HCIByungkyu (Jay) Kang
 
Research Operations at Scale (Christian Rohrer at DesignOps Summit 2017)
Research Operations at Scale (Christian Rohrer at DesignOps Summit 2017)Research Operations at Scale (Christian Rohrer at DesignOps Summit 2017)
Research Operations at Scale (Christian Rohrer at DesignOps Summit 2017)Rosenfeld Media
 

Similar to How to find out about the usability of your web site using a survey by @cjforms (20)

A deep dive into questions by @cjforms at UxLx
A deep dive into questions by @cjforms at UxLxA deep dive into questions by @cjforms at UxLx
A deep dive into questions by @cjforms at UxLx
 
Five Steps to Better Surveys: part of the Surveys that Work training course f...
Five Steps to Better Surveys: part of the Surveys that Work training course f...Five Steps to Better Surveys: part of the Surveys that Work training course f...
Five Steps to Better Surveys: part of the Surveys that Work training course f...
 
ICIS Rating Scales for Collective IntelligenceIcis idea rating-v1.0-final
ICIS Rating Scales for Collective IntelligenceIcis idea rating-v1.0-finalICIS Rating Scales for Collective IntelligenceIcis idea rating-v1.0-final
ICIS Rating Scales for Collective IntelligenceIcis idea rating-v1.0-final
 
Immersive Recommendation Workshop, NYC Media Lab'17
Immersive Recommendation Workshop, NYC Media Lab'17Immersive Recommendation Workshop, NYC Media Lab'17
Immersive Recommendation Workshop, NYC Media Lab'17
 
Brightfind world usability day 2016 full deck final
Brightfind world usability day 2016   full deck finalBrightfind world usability day 2016   full deck final
Brightfind world usability day 2016 full deck final
 
UX STRAT Online 2021 Presentation by Rina Tambo Jensen
UX STRAT Online 2021 Presentation by Rina Tambo JensenUX STRAT Online 2021 Presentation by Rina Tambo Jensen
UX STRAT Online 2021 Presentation by Rina Tambo Jensen
 
Essay On Psychology And Square Foot Condominium
Essay On Psychology And Square Foot CondominiumEssay On Psychology And Square Foot Condominium
Essay On Psychology And Square Foot Condominium
 
Interleaving - SIGIR 2016 presentation
Interleaving - SIGIR 2016 presentationInterleaving - SIGIR 2016 presentation
Interleaving - SIGIR 2016 presentation
 
How NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling DataHow NOT to Aggregrate Polling Data
How NOT to Aggregrate Polling Data
 
Social machines: theory design and incentives
Social machines: theory design and incentivesSocial machines: theory design and incentives
Social machines: theory design and incentives
 
Taxonomy, Social Networks and Pace Layering
Taxonomy, Social Networks and Pace LayeringTaxonomy, Social Networks and Pace Layering
Taxonomy, Social Networks and Pace Layering
 
An informatics perspective on argumentation mining - SICSA 2014-07-09
An informatics perspective on argumentation mining - SICSA 2014-07-09An informatics perspective on argumentation mining - SICSA 2014-07-09
An informatics perspective on argumentation mining - SICSA 2014-07-09
 
Visualization for public policy (2016)
Visualization for public policy (2016)Visualization for public policy (2016)
Visualization for public policy (2016)
 
WISE2019 presentation
WISE2019 presentationWISE2019 presentation
WISE2019 presentation
 
Measuring effectiveness of machine learning systems
Measuring effectiveness of machine learning systemsMeasuring effectiveness of machine learning systems
Measuring effectiveness of machine learning systems
 
Intranet Usability Testing
Intranet Usability TestingIntranet Usability Testing
Intranet Usability Testing
 
Scalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision MakingScalable Exploration of Relevance Prospects to Support Decision Making
Scalable Exploration of Relevance Prospects to Support Decision Making
 
2 Studies UX types should know about (Straub UXPA unconference13)
2 Studies UX types should know about (Straub UXPA unconference13)2 Studies UX types should know about (Straub UXPA unconference13)
2 Studies UX types should know about (Straub UXPA unconference13)
 
Evaluation and User Study in HCI
Evaluation and User Study in HCIEvaluation and User Study in HCI
Evaluation and User Study in HCI
 
Research Operations at Scale (Christian Rohrer at DesignOps Summit 2017)
Research Operations at Scale (Christian Rohrer at DesignOps Summit 2017)Research Operations at Scale (Christian Rohrer at DesignOps Summit 2017)
Research Operations at Scale (Christian Rohrer at DesignOps Summit 2017)
 

More from Caroline Jarrett

Creating truly accessible forms, SDinGov 2024
Creating truly accessible forms, SDinGov 2024Creating truly accessible forms, SDinGov 2024
Creating truly accessible forms, SDinGov 2024Caroline Jarrett
 
Using surveys to uncover audience insights
Using surveys to uncover audience insightsUsing surveys to uncover audience insights
Using surveys to uncover audience insightsCaroline Jarrett
 
2023 Surveys that work Olin.pptx
2023 Surveys that work Olin.pptx2023 Surveys that work Olin.pptx
2023 Surveys that work Olin.pptxCaroline Jarrett
 
The Phylogenetic Tree in forms design - making forms work for complex academ...
The Phylogenetic Tree in forms design - making forms work for complex  academ...The Phylogenetic Tree in forms design - making forms work for complex  academ...
The Phylogenetic Tree in forms design - making forms work for complex academ...Caroline Jarrett
 
Four ways to make a better survey WebExpo2023
Four ways to make a better survey WebExpo2023Four ways to make a better survey WebExpo2023
Four ways to make a better survey WebExpo2023Caroline Jarrett
 
Content design in practice May 2023.pptx
Content design in practice May 2023.pptxContent design in practice May 2023.pptx
Content design in practice May 2023.pptxCaroline Jarrett
 
Two ways to improve your survey, webinar for Delib 2023.pptx
Two ways to improve your survey, webinar for Delib 2023.pptxTwo ways to improve your survey, webinar for Delib 2023.pptx
Two ways to improve your survey, webinar for Delib 2023.pptxCaroline Jarrett
 
How to save money by fixing forms, WebExpo2023
How to save money by fixing forms, WebExpo2023How to save money by fixing forms, WebExpo2023
How to save money by fixing forms, WebExpo2023Caroline Jarrett
 
How to look at a form Boye 22.pptx
How to look at a form Boye 22.pptxHow to look at a form Boye 22.pptx
How to look at a form Boye 22.pptxCaroline Jarrett
 
Prune tune postpone explain.pptx
Prune tune postpone explain.pptxPrune tune postpone explain.pptx
Prune tune postpone explain.pptxCaroline Jarrett
 
What is a service designer SDinGOV 22 with all stickies.pptx
What is a service designer SDinGOV 22 with all stickies.pptxWhat is a service designer SDinGOV 22 with all stickies.pptx
What is a service designer SDinGOV 22 with all stickies.pptxCaroline Jarrett
 
Helping teenage boys to become responsible adults.pptx
Helping teenage boys to become responsible adults.pptxHelping teenage boys to become responsible adults.pptx
Helping teenage boys to become responsible adults.pptxCaroline Jarrett
 
Some thoughts on surveys: Boye and Company member conference call
Some thoughts on surveys: Boye and Company member conference callSome thoughts on surveys: Boye and Company member conference call
Some thoughts on surveys: Boye and Company member conference callCaroline Jarrett
 
Better surveys - presentation to Loyal Health March 2022.pptx
Better surveys - presentation to Loyal Health March 2022.pptxBetter surveys - presentation to Loyal Health March 2022.pptx
Better surveys - presentation to Loyal Health March 2022.pptxCaroline Jarrett
 
Surveys that work:training course for Rosenfeld Media, day 1
Surveys that work:training course for Rosenfeld Media, day 1Surveys that work:training course for Rosenfeld Media, day 1
Surveys that work:training course for Rosenfeld Media, day 1Caroline Jarrett
 
Surveys that work: training course for Rosenfeld media, day 2
Surveys that work: training course for Rosenfeld media, day 2Surveys that work: training course for Rosenfeld media, day 2
Surveys that work: training course for Rosenfeld media, day 2Caroline Jarrett
 
Surveys that work: training course for Rosenfeld Media, day 3
Surveys that work: training course for Rosenfeld Media, day 3 Surveys that work: training course for Rosenfeld Media, day 3
Surveys that work: training course for Rosenfeld Media, day 3 Caroline Jarrett
 
Surveys that work: an introduction to the Survey Octopus and Total Survey Error
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorSurveys that work: an introduction to the Survey Octopus and Total Survey Error
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorCaroline Jarrett
 
Surveys that work: a webinar for FocusVision 2021
Surveys that work: a webinar for FocusVision 2021Surveys that work: a webinar for FocusVision 2021
Surveys that work: a webinar for FocusVision 2021Caroline Jarrett
 
Surveys that work: a webinar for UXPA 2021
Surveys that work: a webinar for UXPA 2021Surveys that work: a webinar for UXPA 2021
Surveys that work: a webinar for UXPA 2021Caroline Jarrett
 

More from Caroline Jarrett (20)

Creating truly accessible forms, SDinGov 2024
Creating truly accessible forms, SDinGov 2024Creating truly accessible forms, SDinGov 2024
Creating truly accessible forms, SDinGov 2024
 
Using surveys to uncover audience insights
Using surveys to uncover audience insightsUsing surveys to uncover audience insights
Using surveys to uncover audience insights
 
2023 Surveys that work Olin.pptx
2023 Surveys that work Olin.pptx2023 Surveys that work Olin.pptx
2023 Surveys that work Olin.pptx
 
The Phylogenetic Tree in forms design - making forms work for complex academ...
The Phylogenetic Tree in forms design - making forms work for complex  academ...The Phylogenetic Tree in forms design - making forms work for complex  academ...
The Phylogenetic Tree in forms design - making forms work for complex academ...
 
Four ways to make a better survey WebExpo2023
Four ways to make a better survey WebExpo2023Four ways to make a better survey WebExpo2023
Four ways to make a better survey WebExpo2023
 
Content design in practice May 2023.pptx
Content design in practice May 2023.pptxContent design in practice May 2023.pptx
Content design in practice May 2023.pptx
 
Two ways to improve your survey, webinar for Delib 2023.pptx
Two ways to improve your survey, webinar for Delib 2023.pptxTwo ways to improve your survey, webinar for Delib 2023.pptx
Two ways to improve your survey, webinar for Delib 2023.pptx
 
How to save money by fixing forms, WebExpo2023
How to save money by fixing forms, WebExpo2023How to save money by fixing forms, WebExpo2023
How to save money by fixing forms, WebExpo2023
 
How to look at a form Boye 22.pptx
How to look at a form Boye 22.pptxHow to look at a form Boye 22.pptx
How to look at a form Boye 22.pptx
 
Prune tune postpone explain.pptx
Prune tune postpone explain.pptxPrune tune postpone explain.pptx
Prune tune postpone explain.pptx
 
What is a service designer SDinGOV 22 with all stickies.pptx
What is a service designer SDinGOV 22 with all stickies.pptxWhat is a service designer SDinGOV 22 with all stickies.pptx
What is a service designer SDinGOV 22 with all stickies.pptx
 
Helping teenage boys to become responsible adults.pptx
Helping teenage boys to become responsible adults.pptxHelping teenage boys to become responsible adults.pptx
Helping teenage boys to become responsible adults.pptx
 
Some thoughts on surveys: Boye and Company member conference call
Some thoughts on surveys: Boye and Company member conference callSome thoughts on surveys: Boye and Company member conference call
Some thoughts on surveys: Boye and Company member conference call
 
Better surveys - presentation to Loyal Health March 2022.pptx
Better surveys - presentation to Loyal Health March 2022.pptxBetter surveys - presentation to Loyal Health March 2022.pptx
Better surveys - presentation to Loyal Health March 2022.pptx
 
Surveys that work:training course for Rosenfeld Media, day 1
Surveys that work:training course for Rosenfeld Media, day 1Surveys that work:training course for Rosenfeld Media, day 1
Surveys that work:training course for Rosenfeld Media, day 1
 
Surveys that work: training course for Rosenfeld media, day 2
Surveys that work: training course for Rosenfeld media, day 2Surveys that work: training course for Rosenfeld media, day 2
Surveys that work: training course for Rosenfeld media, day 2
 
Surveys that work: training course for Rosenfeld Media, day 3
Surveys that work: training course for Rosenfeld Media, day 3 Surveys that work: training course for Rosenfeld Media, day 3
Surveys that work: training course for Rosenfeld Media, day 3
 
Surveys that work: an introduction to the Survey Octopus and Total Survey Error
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorSurveys that work: an introduction to the Survey Octopus and Total Survey Error
Surveys that work: an introduction to the Survey Octopus and Total Survey Error
 
Surveys that work: a webinar for FocusVision 2021
Surveys that work: a webinar for FocusVision 2021Surveys that work: a webinar for FocusVision 2021
Surveys that work: a webinar for FocusVision 2021
 
Surveys that work: a webinar for UXPA 2021
Surveys that work: a webinar for UXPA 2021Surveys that work: a webinar for UXPA 2021
Surveys that work: a webinar for UXPA 2021
 

Recently uploaded

Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comjakyjhon00
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Ed Orozco
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024Alan Dix
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxHasan S
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillCre8iveskill
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxb2kshani34
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazineRivanEleraki
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTThink 360 Studio
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtTeeFusion
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLkenzukiri
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfsaidbilgen
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Ted Drake
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxSamKuruvilla5
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before ConstructionResDraft
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfHctorFranciscoSnchez1
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsBlock Party
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...khushisharma298853
 
UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024mikailaoh
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Amil baba
 

Recently uploaded (19)

Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.com
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkill
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptx
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazine
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPT
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy Shirt
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptx
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before Construction
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdf
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teams
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...
 
UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
 

How to find out about the usability of your web site using a survey by @cjforms

  • 1. How to find out about the usability of your website using a surveyUX Cambridge 2012 workshop led by Caroline Jarrett
  • 2. I got asked this question recently. What are your views? “ Can a usability test just use a questionnaire, no observation? ” 2
  • 3. “Can a usability test just use a questionnaire, no observation?” << NO • We agreed that a usability test must have observation, otherwise it’s not a usability test • Recommended reading Steve Krug “Rocket Surgery Made Easy” Carol M. Barnum “Usability Testing Essentials” Workshop results 3
  • 4. I rejected the idea of using a questionnaire alone. But let’s ponder this question. “How to find out about the usability of your website using a survey” 4
  • 5. UX people typically encounter surveys in these three ways 1. Post-test / post-task surveys e.g. SUS 2. Someone is going to do a survey anyway 3. Triangulating between survey data and data from elsewhere http://www.slideshare.net/cjforms/ better-ux-surveys-at-ucd2012-by-cjforms Today’s topic, this time focusing on usability 5
  • 6. Let’s unpick the standard definition. Which bits can we assess using a survey? The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241:11 1998) 6
  • 7. We picked out many details in the definition that we could explore in surveys Examples from the discussion: • “Specified users” - demographics, attitudes, history • “Specified goals” – what do users want to achieve? • “Satisfaction” – What did they expect? – What is their assessment after the experience? Corbin, J. M. and Strauss, A.L. (2008) “Basics of qualitative research: techniques and procedures for developing grounded theory” Workshop results 7
  • 8. Example: the product. Is that the whole website, a section, a page? The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241:11 1998) 8
  • 9. Used properly, NPS® (Net Promoter Score) takes “the product” up another level – to brand “Would you recommend us to a friend?” Image credit: Rodney Burton, geograph.org.uk Image credit: Richard Vince, geograph.org.uk Image credit: Cambridge County Council Reichheld, F. F. (2006) The ultimate question: driving good profits and true growth 9
  • 10. But sometimes NPS® gets muddled up with other things 10
  • 11. Tip Ask one question at a time Image credit: Shutterstock 11
  • 12. My favourite bit to look at with surveys: what is the user’s concept of the goal? The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241:11 1998) What the user wants to do
  • 13. Tip Find out about users’ goals 13
  • 14. There are other goals to consider What the organisation wants to achieve What the user wants to do Our aims in doing a survey 14
  • 15. A set of questions starts to become a survey when we know our goals for the survey Goals • Establish your goals for the survey Questions you need answers to • Final version of questions • Build the questionnaire Users Build Deploy Analyse 15 • Interview users about the topics in your survey • Run the survey from approach to follow-up • Extract useful ideas • Share with others Questions users can answer Questionnaire Data Insight
  • 16. Exploratory: Don’t know what is out there. Send and hope. Image credit: Shutterstock 16
  • 17. Comparative: Explore trends, look at before-and-after, trackers Image credit: Shutterstock 17
  • 18. Descriptive: Do know what is out there. Go and count them. Image credit: Caroline Jarrett 18
  • 19. Modelling: Find factors that show cause and effect Image credit: Shutterstock 19
  • 20. Example: The 1946 Cohort Study combines comparative, descriptive, and modelling Cumulative death rates by father's social class at birth in the 1946 birth cohort study Adapted from Kuh, D., Hardy, R., Langenberg, C., Richards, M. and Wadsworth, M. E. J. (2002), Mortality in adults aged 26-54 years related to socioeconomic conditions in childhood and adulthood: post war birth cohort study. 20
  • 21. Exploratory, comparative, descriptive, modelling? Let’s discuss (from a pop-up survey) 21
  • 22. We found plenty to say about the details of the questions – but this is a COMPARATIVE survey, so keeping questions the same whenever it is used is crucial Workshop results 22
  • 23. The ideal: everything in balance Comparisons Good questions 23
  • 24. To get maximum insight from your survey, follow all the steps. • Final version of questions • Build the questionnaire Goals Users Build Deploy Analyse • Establish your goals for the survey • Interview users about the topics in your survey • Run the survey from approach to follow-up • Extract useful ideas • Share with others Questions you need answers to Questions users can answer Questionnaire Data Insight 24
  • 26. Bonus Tip Survey = Questionnaire + Process Successful That involves lots of testing 26