Database Web Application Usability Testing

USABILITY TESTING
Database Web Application

Test Information
OVERVIEW
TechSmith Morae was used on a laptop computer to conduct usability testing of the newly designed WVU Libraries Database
web application. This round of usability testing was internal and focused on WVU Libraries primary target audience.

Test Information
TARGET AUDIENCE
73%* of WVU Libraries users are undergraduate students who are 48% female or 52% male, are primarily Caucasian,
African American, Hispanic, or Asian in ethnicity, are United States citizens or are international students, are 21 years in their
average age, and are 49% from the state of West Virginia or 51% from other United States, territories, or countries.
Target audience numbers are populated from both survey polls and server statistics.*

Test Information
DATABASE USE
96%* of the WVU Libraries Database web application usage is from non-Downtown Campus Library WVU employee
computer users ‒ 64% of which are returning users, and 61% of which are from ages 18 to 34 ‒ on and off Morgantown
campus, whereas 4% of Database web application usage is from WVU employee at the Downtown Campus Library. The
Geographic Breakdown of these 96% of users is as follows: 99% from the United States; 86% from the state of West
Virginia; 77% from Morgantown, West Virginia; 1% outside the United States.
Statistics are calculated by Google Analytics.*

Test Information
TASKS
1. Go to WVU Libraries’ Databases.
2. Find the database PubMed.
3. What is the earliest coverage in the database SportDiscus?
4. What is a database for the subject Marketing, that is also an
encyclopedia?
5. Your assignment is to listen to an aria from an opera. What
database will you use?
6. How many databases cover the subject of psychology?
7. Find a subject guide for engineering.
8. Go to the tutorial for the database Ad$pender.
9. What are the best databases for finding sources in biology?
10. What is the starting year of coverage for the database Web of
Science?
11. How can you get help with database selection or searching?
12. You need a brief video clip on chemistry lab safety. Where
would you find it?
13. Find 2 databases that contain statistics for market research.
14. What is a database for Theater that is contains Images?
15. How many primary sources can you find for West Virginia
databases?

Test Information
*(SUS) QUESTIONS
1. I think that I would like to use this system frequently
2. I found the system unnecessarily complex
3. I thought that the system was easy to use
4. I think that I would need the support of a technical person to
be able to use this system
5. I found the various functions in this system were well
integrated
6. I thought there was too much inconsistency in this system
7. I would imagine that most people would learn to use this
system very quickly
8. I found the system very cumbersome to use
9. I felt very confident using the system
10. I needed to learn a lot of things before I could get going with
this system
* System Usability Scale (SUS)

Erica Erick Kayla Rebecca Shawn
Alex Brandon Chris Christian Dave
Test Information
PARTICIPANTS

Test Results:
All Tasks
AVERAGE
TIME* Average time includes delivery of tasks.

Test Results:
All Tasks
SUCCESS
RATES* Percentages based on a 0 to 3 scoring system.

Test Results:
All Tasks
ERROR
RATE* Error rate is a standard deviation based
direct path to information as it is known to
test administrator, and the rate of the user
encountering a functionality or system error.

Test Results:
All Tasks
SUS
SURVEY* Results are based on survey results taken
after usability testing, and are concerned
with the Database web application interface
and functionality.

Test Results
TASK 1Requested Task: Go to WVU Libraries’ Databases.
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.74 min.
90%
0.00
10%
Participant Observations
• 10% of users couldn’t find the Databases page
• 40% of users eventually located the Databases page by utilizing the homepage
search or ‘Databases by Subject’ link
• 50% of users utilized a search engine for ‘WVU Libraries Databases’
Suggestions
• Better incorporate Databases on WVU Libraries’ homepage
Videos
• https://testing.lib.wvu.edu/usability4/task1a.mp4
• https://testing.lib.wvu.edu/usability4/task1b.mp4

Test Results
TASK 2Requested Task: Find the database PubMed.
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
1.17 min.
90%
0.00
10%
• 30% of users couldn’t find the ‘Database Titles’ link
• 20% of users aren’t seeing the A-Z section
• 40% of users expressed interest in / wanted to search so they either went back to
the libraries homepage, or used the browser’s default <Ctrl> + <F> to search.
• 50% of users utilized a search engine for ‘WVU Libraries Databases’
Suggestions
• Incorporate predictive search for Databases on the homepage and on the
Databases web application page
• Move the subject/title/help navigation to the left, pluralize ‘Databases’, add the word
‘by’, and make it look more like navigation options (instead of being grayed-out)
• Add ‘by Subject’ as a facet
• Add padding and emphasis to A-Z section to make it stand out more
• Eliminate “by title ‘Letter’” from results, it’s confusing to read
Videos

Test Results
TASK 3Requested Task: What is the earliest coverage in the database SportDiscus?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
1.59 min.
60%
0.20
40%
• 100% of users don’t click or see ‘More Information’ in the databases results, 80%
never go back to this, and only 20% eventually clicked on ‘More Information’ just
because the task required them to do so
• 30% are still not seeing ‘Database Titles’
Errors
• Empty Help with a bullet under ‘More Information’
• Facets appear as a gray box for search results
• New/Trial databases don’t show in breadcrumbs
Suggestions
• Possibly employ an accordion as a button instead of a blue ‘More Information’ link
• Fix information in databases ‘More Information’
• Add facets for search, new, and trial
Videos

Test Results
TASK 4Requested Task: What is a database for the subject Marketing, that is also an
encyclopedia?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
1.01 min.
10%
0.00
90%
• 40% of users aren’t seeing / understanding resource types as faceting
• 20% of users are not understanding the difference between Databases by subject
and titles
• 20% of students are still going back to the homepage to search
• As users spend more time with the system they are learning how to use the system
as a successful NUI (natural user interface)
Suggestions
• Facets should only populate under resource types dynamically as they are
applicable and remain to what’s pre and post breadcrumb faceting
• Add hinting for important functionality on web applications
Videos

Test Results
TASK 5Requested Task: Your assignment is to listen to an aria from an opera. What
database will you use?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
1.07 min.
100%
0.00
0%
• 20% of users still aren’t seeing / understanding resource types as faceting
• 10% of users are still not understanding Databases by subject versus titles
• 20% went to resource type ‘Video and Sound’ first, and 30% went to the subject of
Art while 70% went to the subject of Music
• 20% of users are still going back to the homepage to search
• 20% of users used the Music tag
Suggestions
• Create visual understanding with color for facets in the left-hand, and color code
(and possibly use different icons) for tags based on resource type versus subject
• Reduce the amount of tags as is appropriate
Videos

Test Results
TASK 6Requested Task: How many databases cover the subject of psychology?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.46 min.
90%
0.00
10%
• At this point all users were familiar with the faceting, subject, and title filtering of the
database web application
• 10% of users misread the pagination indicator, the other 90% performed the task
with ease
Suggestions
• Possibly add wording to pagination such as ‘Viewing results 1-10 out of 25 total’ or
incorporate how many pages / results on pages
Videos

Test Results
TASK 7Requested Task: Find a subject guide for engineering.
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.66 min.
80%
0.00
20%
• 40% of users don’t know what a subject guide was, or why it was needed
• 10% of users used the homepage to try and find a subject guide
Suggestions
• On the macro level the WVU Libraries website could better explain the functionality
of Research Guides living at libguides.wvu.edu (which is confusing), and that
underneath Research Guides there are Subject Guides, Course Guides, Style
Guides, Finding Guides, etc.
Videos

Test Results
TASK 8Requested Task: Go to the tutorial for the database Ad$pender.
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
1.38 min.
60%
0.00
40%
• 10% looked in ‘Help’
• 20% of users went back to the homepage to search
• 80% went straight to databases instead of ‘More Information’
• Users feel there are some databases that have too many tags
Suggestions
• Database title search states: By title ‘A’ Database Results: By title ‘A’, which is
confusing and should read instead just as ‘Database by Title: A’
Videos

Test Results
TASK 9Requested Task: What are the best databases for finding sources in biology?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.99 min.
100%
0.10
0%
• 100% of users didn’t see or utilize “Librarian Top Picks’
• 40% of user utilized the ‘Primary Sources’ facet
• 20% of users went to Subject Guide for Biology
• 20% of users utilized the ‘Statistics’ facet
Error
• When users click on the ‘Statistics’ resource type facet for the database subject of
Biology there are no results
Suggestions
• ‘Librarian Top Picks’ need to be used across subject, or defaults should stick
• ‘Librarian Top Picks’ should have a different aesthetic treatment to make them
visible when they appear
Videos

Test Results
TASK 10Requested Task: What is the starting year of coverage for the database Web of
Science?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.80 min.
100%
0.00
0%
• 30% of users said 1955 based on ‘More Information’
• 70% of users went straight to database and overlooked ‘More Information’, and
71% of those users said 1864 based on information in Web of Science
• 10% of users searched on the homepage
Videos

Test Results
TASK 11Requested Task: How can you get help with database selection or searching?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.84 min.
100%
0.10
0%
• 10% of users would go to ‘Ask A Librarian’ green tab
• 20% of users would go ask a librarian in person
• 20% of users would go to ‘Contact Us’
• 30% of users would go to ‘Help’
Videos
• https://testing.lib.wvu.edu/usability4/task11c.mp4

Test Results
TASK 12Requested Task: You need a brief video clip on chemistry lab safety. Where would
you find it?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
1.72 min.
20%
0.67
80%
• 80% of users tried to use the ‘Video and Sound’ resource type facet
• 30% of those users eventually went out to the homepage to search, 20% stated
they would search in the subject guide, and 50% said they would go to YouTube
• 20% of users used the ‘Handbook’ tag to determine a database
Error
• 80% of users clicked on the ‘Video and Sound’ resource type facet for the
database subject of Chemistry, and there were no results
• Users can compound resource type facets and get no results
Videos
• https://testing.lib.wvu.edu/usability4/task12c.mp4

Test Results
TASK 13Requested Task: Find 2 databases that contain statistics for market research.
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.95 min.
90%
0.00
10%
• 20% of users read database description to resolve task
• 20% of user went to homepage to search
Videos

Test Results
TASK 14Requested Task: What is a database for Theater that is contains Images?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.60 min.
90%
0.00
10%
Participant Observation
• 10% of users didn’t understand faceting and filtering
Suggestions
• Possibly develop facets to work in reverse order, so by resource type then by
subject.
Videos

Test Results
TASK 15Requested Task: How many primary sources can you find for West Virginia
databases?
Average Time:
Completion Rate:
Error Rate:
Failure Rate:
0.84 min.
80%
0.10
20%
• 20% of users used tags to find primary resources or West Virginia databases
• 20% of users went to the homepage to search
Error
• Homepage search results didn’t yield database subject search results
Videos

The WVU Libraries’ database web application was redesigned as a
NUI, or Natural User Interface. In computing a NUI is the common
parlance used by designers and developers of human-machine
interfaces to refer to a user interface that is effectively invisible, and
remains invisible as the user continuously learns increasingly complex
interactions.
These usability test results demonstrates that as a user interacts with
the WVU Libraries’ database web application, the user continuously
learns how to use the interface. More so, one can see the decrease in
percentages from one task to another in regards to users not seeing or
knowing how the database web application interface functions.
Indeed, a NUI is ideal since our current analytics show that 96% of the
WVU Libraries database web application usage is from non-Downtown
Campus Library librarian/staff computer users on and off Morgantown
campus. Ergo, the database web application’s return users (ages 18 to
34) are majorly using this web application without assistance of any
kind.
Furthermore, according to this usability test 11 out of 15 tasks had a
failure rate ranging from 10% to 80%, and only 10 out of 15 tasks
had an easy completion rate ranging from 10% to 50%; these rates
can be dramatically improved.
Conclusion
SUMMARY

The following changes to analytics, content, development, and user
interface should dramatically eliminate errors, reduce the percentage of
failure rates, and decrease the amount of time for task completion
Analytics
- Install filters for librarian / staff traffic at the Evansdale and HSC
libraries
User Interface
- Better incorporate Databases on WVU Libraries’ homepage
- Move the subject/title/help navigation to the left, and possibly change
color
- Add padding and emphasis to A-Z section to make it stand out more
- Possibly employ an accordion as a button instead of a blue ‘More
Information’ link, and remove link from database title
- Create visual understanding with color for facets in the left-hand, and
color code (and possibly use different icons) for tags based on resource
type versus subject
- ‘Librarian Top Picks’ should have a different aesthetic treatment to
make them visible when they appear
- Possibly add # to simulate pagination reload
Content
- Pluralize ‘Databases’, add the word ‘by’, and make it look more like
navigation options (instead of being grayed-out)
- Fix information in databases ‘More Information’
- Add hinting for important functionality on web applications
- Reduce the amount of tags as is appropriate
- Possibly add wording to pagination such as ‘Viewing results 1-10 out of
25 total’ or incorporate how many pages / results on pages
- Database title search states: By title ‘A’ Database Results: By title ‘A’,
which is confusing and should read instead just as ‘Database by Title:
A’
Development
- Incorporate predictive search for databases on the homepage and on
the Databases web application page
- Add ‘by Subject’ as a facet
- Add facets for search, new, and trial
- Facets should only populate under resource types dynamically as they
are applicable and remain to what’s pre and post breadcrumb faceting
- ‘Librarian Top Picks’ need to be used across subject, or defaults should
stick
- Possibly develop facets to work in reverse order, so by resource type
then by subject.
Conclusion
ENHANCEMENTS

“Pay attention to what users do, not what they say.”
- Jakob Nielsen
THANK YOU

Database Web Application Usability Testing

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (6)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Database Web Application Usability Testing

Ähnlich wie Database Web Application Usability Testing (20)

Mehr von Tim Broadwater

Mehr von Tim Broadwater (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Database Web Application Usability Testing