Johanna Tibbertsma is the Customer Experience Designer at icare NSW.
Johanna is an Organisational Psychologist and customer experience (CX) and service design consultant, who enjoys applying her background in psychology and behavioural science to help clients make evidence based, user informed decisions. She would like to see a future where designers are engaged early and with enthusiasm, for our evidence based approaches and commitment to meaningful, measurable outcomes.
I was at a service design conference a year and a half ago, and noticed that all the speakers spoke incredibly passionately about work they’ve done, very few mentioned how they measured their success. I later asked one of the presenters how she convinces stakeholders to hire her agency besides blind faith. She said well often by the time they reach out to us they’ve simply tried everything.
In the industry we also talk a lot about hypotheses and experimentation and the scientific method, but turn a blind eye to one of the most critical elements: Measurement and analysis.
UX professionals will be under increasing pressure to demonstrate the value of CX investment and our value as a profession. We’re pretty good at qualitative research and telling compelling user and customer stories, but it is quant measures that will give us credibility.
So I started thinking...what are some barriers to measuring our impact?
We don’t know how…
We don’t know why…
We don’t want to…
And I’m going to explore these in a little more depth and provide some suggestions for what we could be doing about them.
Some of these recommendations will seem intuitive, but the reality is that within our design teams we're not doing it consistently enough. Sometimes we just need a bit of a refresher about what we could be doing, and stretch our thinking beyond qualitative-feedback and NPS.
I can already feel some of you recoiling into their seats…I know quant seems hard but it is unbelievably useful to get your head around some of these basic concepts.
Half the problem of measuring the success of our work is we don’t collect the baseline measures, so by the end of the project, we can’t tell if anything has changed.
And we don't think to ask for metrics because we don't know what to ask for!
So here’s some tactical stuff to get your inspiration flowing.
We can think about measurement across three broad categories.
Why do we triangulate them?
Because not all customers will respond to solicited feedback requests
We need behavioural data because only 10-20% of human behaviour, on average, can be explained by what we SAY we'll do. But behavioural data on it’s own won’t tell you why they behaved in a certain manner.
Because operational data has the ability to quickly highlight issues as they’re occurring, but won’t tell you how customers feel.
We also want to try to measure at three levels.
At an organisation level, we’re looking at overall organisational performance, customer attitudes towards an organisation as a whole. Things like how environmentally sustainable or how positive a company is are likely to crop up at this level.
At a service level, the measures focus on the end to end, overall experience of a specific journey. They reflect cross-business level performance, but aren’t always very diagnostic, that is, they don’t tell us WHY an overall experience was compromised.
At an episode level, we’re looking to measure specific interactions within a journey. Here we’re trying to understand the cause of a service level outcome. These measures are more likely to reflect business unit or team or touchpoint performance.
As we move down the pyramid, the number of things you measure is likely to increase too.
I’m going to focus on the episode and service level because that probably the space most of us are likely to be working in. So how do apply it?
How do we apply it. Let’s start with an asset a lot of you already have- the journey map. Here we have a very standard purchase journey, with the physical and digital touchpoints mapped. All of you should be able to identify one of these phases you’re currently working in.
Using the website as an example, what’s one Solicited feedback measure you could use? What about a Behavioural one? Operational one?
Now if we break one of those stages down at a task level or process flow, we can start to put a measure against each stage of that flow.
Fascinatingly, if you have a 95% chance of nailing each one of these, there’s still only a 60% chance that your customer will have a seamless end to end experience.
UX designer folks:
The interactions in a customer journey are just like the steps in a user flow. Go back to your user flows, identify the key actions and end to end goal, and identify measures for each of those pages.
Enterprise folks: I’ve used a customer purchase example but designing for employee experience follows the same principles.
But it’s not just the moments and tasks in the journey we want to measure. We also want to measure how the customers FELT.
It is the moment need + the experience need that creates value for a customer and drives them to action.
How do we measure experience? Well a really simple way is to leverage the experience principles you’ve probably got in your organisation, and create a couple of survey items that assess how well you’re achieving that.
And don’t forget employee experience.
A lot of the time, our staff need the same things emotionally, as our customers. And by some accounts less than 10% of staff experiences are designed. We push business process onto customers and staff rather than design for real customer need.
Ask yourself…
What are employees experiencing and feeling while trying to complete a task?
When does employee experience directly impact the customer?
What is the impact of the experience on staff engagement?
Where are there opportunities to make the process more efficient and pleasurable for staff?
Also masquerades as “I don't have the time/scope/access.”
I think we know in our hearts that UX is a valuable exercise, but until we can make that value tangible and communicate to the business in a language they can understand, **** experience design will continue to feel like an expensive luxury.
You’ll always get hardline stakeholders who are like you know what at the end of the day we are here to do x. Why should I care how people feel?
And here’s what you’ll tell them.
Ultimately, designing for experience is about driving a response, which in turn, has impacts to the organisation.
When we can articulate the impact that a change in customer or staff behavior, that you created, this stuff becomes extraordinarily powerful.
Here are just a few basic examples of measures you can use to work out if you’re tracking to a project objective, and the impact it has financially on the organisation.
Find out what the marketing spend is per customer gained.
Find out what one second in a call centre costs your business. It’ll blow your mind.
Find out what it costs for a complaint to handled by your internal complaints team, versus handled in the frontline.
Find out what is costs the government to chase people up when they haven’t paid their water bill.
Pick a metric that your organisation cares about, and find out why that metric matters so much in financial terms. In the Workers Compensation scheme in icare, we care about the average number of days an injured worker spends off work. Because if we can improve that by just one day, through improvements in our processes, it’s worth millions annually.
We hear that designing for customer experience is expensive, but actually if it’s done right, it should reduce costs
"If we don't measure it, we can't have failed..."
Measuring our success takes courage. It's human nature to place higher value in the things we've created you know? Because we've built this attachment to something and take immense pride in our work, and we don't want to hear negative feedback. Like a baby, no-one wants to admit their baby is ugly. Who here has an ugly baby? Ohh ok couple of terrible parents up the back. Who was an ugly baby? Yeah you were, you didn't have to put your hand up, we already knew.
But you know what? It's okay if your baby is ugly. Because this is a baby you can control.
But not measuring is crazy. It's like putting money into Bitcoin and never opening your Wallet to check your balance. Don't you want to know what's happening to your investment? The market could have crashed! You could be homeless! But you're just throwing thousands more dollars into something you don't even know is working!
"But Jo, what if we don't see an effect?"
Well changing customer and organisational behaviour can be slow. Just because you don't see an effect immediately doesn't mean you won't reap the benefits 6 or 12 months down the track. A core concepts of agile methodology is continuous improvement. If we don't measure continuously, how can we improve?
Or maybe you've chosen the wrong metric. I did some work with a large financial institution once, we'll call them BommCank. To protect their privacy. We wanted to test whether improvements to signage around the ATMs would change customer behaviour in the lobbies, and one of the outcomes we were looking for was an increase in transactions at the ATMs. So we picked our 6 branches and collected data for the 6 week trial period. Do you think we saw the increase we were looking for? No. Upon reflection I realised wait why would we expect clearer signage to increase number of transactions? You're either looking for an ATM or you're not. So we were looking at the wrong metric.
Or maybe there is no effect. And that's great too. Because it means you need to try something else, which is what experimentation is all about. We should be emphasising fast learning, not success or fail.
If all else fails…just don’t tell anyone! ;)
Guys I know it's out of our comfort zones, I know it’s overwhelming. Which is why I've broken off a few bite sized chunks for you and written down some very simple next steps which you can work on for your very next project.
1. As I mentioned before, most of the reason we aren’t able to measure our success is because we don’t know what to ask for up front and therefore we don’t have baseline measures.
2. You don’t have to do the analysis, you just need to know what to ask for. Make friends with CFO, Business Analysts and Actuaries. There are nerds in your organisation who live and breathe this.
Some of you would be homeless.
That’s nervous laughter.
I’m kidding.
Thankyou for listening!
