You can't manage what you can't measure, so... How do you actually measure user experience?
In this webinar we covered what, why, and how to conduct website user experience & usability benchmarking. We discussed how to effectively measure the quality of a website's user experience across various competitors, within one industry, across time, using an online quantitative research methodology commonly referred to as "unmoderated remote usability testing."
6. What is UX Benchmarking & why is it important? Ok… So how do we define UX Benchmarking ?
7.
8. Some may wonder: Is UX Benchmarking… What is UX Benchmarking & why is it important? ?
9.
10. Why is it now more important than ever before? What is UX Benchmarking & why is it important? It’s all about competitiveness, productivity, & efficiency
11. ‘ You can’t manage what you can’t measure’ Attributed to Peter Drucker, Mgt Guru What is UX Benchmarking & why is it important? The question is: How are you actually measuring UX and defining your benchmark?
30. Brand Perception: Pre/Post All sites had a positive lift in brand perception. +2% +20% +22% +14%
31. Brand Pillar Metrics: Pre vs. Post Differences United showed a 26% increase on the ‘helpful’ attribute and maintained “professional” while other sites had a decrease on these two attributes. All sites showed a sharp increase in the “rude” attribute.
32. Overall Satisfaction Panelists were most satisfied with Blue Cross Blue Shield and Aetna’s site when researching and getting rates for Medicare Supplemental Insurance. Differences of 17% or greater are significant at 90% CI
33. Net Promoter Score (NPS) When asked if they would recommend PROVIDER to a colleague or friend, all sites had relatively low NEGATIVE NPS with Aetna and United Healthcare being the lowest. Detractors Promoters NPS % of Promoters (9s and 10s) Detractors (0 through 6) = - 0 1 2 3 4 5 6 7 8 9 10 NPS Aetna 12% 8% 4% 2% 10% 12% 14% 8% 18% 8% 4% -50% BCBS 8% 4% 6% 10% 2% 8% 14% 10% 4% 10% 24% -18% KP 18% 6% 8% 4% 4% 6% 6% 16% 8% 8% 16% -28% United 10% 4% 6% 6% 4% 22% 16% 8% 10% 8% 6% -54%
34. Task 1: Learn & Explore Aetna BCBS Kaiser Permanente United The success rate for the “learn and explore” task, although the frustration rate was above 40% for all sites showing room for improvement.
35. Usability Metrics: Learn & Explore Blue Cross Blue Shield had the highest mean usability scores for task 1.
36. Blue Cross Blue Shield Liked Best: Learn & Explore Ease of navigation: “ I liked the ease of finding information directly relating to the Blue Shield’s Medicare Supplement insurance.” “ it was all user friendly....I think even people who don't use web sites a lot would be able to navigate” Comparison charts: “ clearly divided options for each plan” “ the graphic showing the differences between the supplemental plans”
37. Top Frustrations: Learn & Explore About half of the panelists on all sites had some sort of frustration with learning about Medicare Supplemental Insurance online. Opportunities
40. Example: Used cars websites industry study (Spain) Tasks Description 1. Free search of a car model Users search freely on the Internet for any car model Note: 50 users completed this task 2. Guided search for an specific car model on a selected website The task requests users to search for an specific car model, on each of the second-hand cars web sites. Note: 10 users per website evaluated
41. How do users search for a second-hand car on the Internet? Example: Used cars websites industry study (Spain) % I have used search engines 61 I have surfed just by directly inserting the Website name of the page 21 I used both methods 19
42. Example: Used cars websites industry study (Spain) WEBS Effectiveness Efficiency Time Clicks Autocity.com 90% 3:31' 21 Autoscout24.es 80% 2:08' 29 Coches.net 90% 4:00' 57 Segundamano.es 90% 2:22' 24 Supermotor.com 60% 3:55' 45
43. Example: Used cars websites industry study (Spain) Easiest Website to search for a car Autocity.com Autoscout24.es Coches.net Segundamano.es Supermotor.com Media It is easy to start the car search 5.9 6.6 6.9 6 5.5 6,2 The searcher fields are easy to fill up and understand 6 6.6 7 6.3 6.2 6,4 The searcher has the requiered options to do a search 5.3 6.6 6.8 5.5 5 5,9 Important information as price, car age, milage, model, is available 6 6.6 6.8 6.2 4.7 6,1
44. Task analisys: Main problems Example: Used cars websites industry study (Spain) Autocity.com Autoscout24.es Coches.net Segundamano.es Supermotor.com Average I have the required information to make a buying decission 5.2 5.1 6.7 5.3 5 5,5 Webpage shows car features, extras and accessories 5.3 5.7 6.5 5.2 4.8 5,5 I can clearly see the final price and included concepts 6 6.1 5.4 5.9 5.8 5,8 Photographs help to have a clear idea of the car. 6 6.2 6.6 5.9 4.2 5,8 I have all the information to contact the seller 5.5 6.1 6.4 5.5 5.6 5,8 Too much advertising quantity 4.9 5.4 6.1 5.5 4.2 5,2
In this webinar, we will show you how you can remove face to face moderation, have hundred of users participate simultaneously from different locations in a cost effective and fast way. Userzoom is an on demand research service that will automatically capture and gather both customers attitudes and behaviors. After our demonstration, we will open up to Questions, but let’s start now by diving into UserZoom software as a service.
In this webinar, we will show you how you can remove face to face moderation, have hundred of users participate simultaneously from different locations in a cost effective and fast way. Userzoom is an on demand research service that will automatically capture and gather both customers attitudes and behaviors. After our demonstration, we will open up to Questions, but let’s start now by diving into UserZoom software as a service.
In this webinar, we will show you how you can remove face to face moderation, have hundred of users participate simultaneously from different locations in a cost effective and fast way. Userzoom is an on demand research service that will automatically capture and gather both customers attitudes and behaviors. After our demonstration, we will open up to Questions, but let’s start now by diving into UserZoom software as a service.
In this webinar, we will show you how you can remove face to face moderation, have hundred of users participate simultaneously from different locations in a cost effective and fast way. Userzoom is an on demand research service that will automatically capture and gather both customers attitudes and behaviors. After our demonstration, we will open up to Questions, but let’s start now by diving into UserZoom software as a service.
In this webinar, we will show you how you can remove face to face moderation, have hundred of users participate simultaneously from different locations in a cost effective and fast way. Userzoom is an on demand research service that will automatically capture and gather both customers attitudes and behaviors. After our demonstration, we will open up to Questions, but let’s start now by diving into UserZoom software as a service.
In this webinar, we will show you how you can remove face to face moderation, have hundred of users participate simultaneously from different locations in a cost effective and fast way. Userzoom is an on demand research service that will automatically capture and gather both customers attitudes and behaviors. After our demonstration, we will open up to Questions, but let’s start now by diving into UserZoom software as a service.
50% increase for Aetna pre/post for Rude attribute 36% decrease for BCBS pre/post for Professional attribute 26% increase for United pre/post for Helpful attribute 42% increase on Rude pre/post for Rude attribute