Songkick Product Discovery FOWA (Michelle You)

PRODUCT
DISCOVERY
MICHELLE YOU CPO & CO-FOUNDER
@WRECKINGBALL37

Ian, Pete, and I started Songkick in 2007.
Songkick makes sure you never miss concerts.
You track all the artists you want to see live by, for example, scanning your Spotify playlists, and then whenever any of those bands announce a tour date
in your city, we send you a concert alert.

• 2nd-largest live music website after Ticketmaster
• 8 million fans per month using Songkick

We’re now the second-largest live music website in the world after Ticketmaster. Every month, over 8 million fans come to Songkick to find out when their
favorite artists are coming to town.

!

We’ve also been experimenting with a new platform we’re calling Detour, which enables fans to pledge in advance to buy tickets for an artist. They come to
Detour, tell us which bands they want to see live and how much they’re willing to pay. They enter their credit card to reserve their ticket, but aren’t charged
until the concert is confirmed. Since last year, we’ve been running a series of experiments to understand the potential in this platform. We’ve confirmed 45
concerts that have been created as a result of fans pledging.

BUSINESS
GOALS

PRODUCT
MAGIC

USER
NEEDS

TECHNICAL
FEASIBILITY

My role at Songkick is to make sure we’re building the right things, and building them as quickly as possible. These things we make, whether apps,
features, or entirely new products have to A) solve a real problem for the people who use them and B) achieve the company’s goals, whether that’s
revenues, sign-ups, or whatever other metric is of vital importance to your business; and C) also be something we can feasibly build in a timely manner.
This is typically what a product manager does, but even if you don’t have someone with that title on your team, someone is taking on this responsibility,
whether they’re your CTO, CEO, designer, or project manager. Someone is that glue.

Time is
the enemy

As a startup, you’re constantly under extreme existential threat. Time is your enemy and your company’s imminent death is always around the corner. You
need to show growth, hit targets, get to that next benchmark of success as quickly as possible.

!

You don’t have time to waste on building things no one wants or things that won’t achieve your business goals. But you also aren’t God or Steve Jobs, so
you can’t assume you have perfect knowledge of what people want in the ﬁrst place.

!
So you have to learn this as fast as possible. That’s what product discovery is.
!
So far so Lean Startup.
!

But how do you go about actually validating your hypotheses? How do you actually ﬁgure out what the right thing to build is? How do you do product
discovery?

GOLDEN RULE OF
PRODUCT DISCOVERY
01
SPEED & QUANTITY
OF LEARNING
First golden rule of product discovery: Speed and quantity of learning.

Tom Chi: Google X
Assume your ideas have a
5% chance of success
the time you try 20
things, you have a 64%
chance of success.

• By

the time you try 50
things, you have a 90%
chance of success.

• By

Tom Chi of Google X gave an incredible, life-changing talk at a conference last year where he schooled the attendees, including me, on the simple
mathematical reality of product discovery.

!

No matter how brilliant a product designer you are, the harsh truth is that maybe 5% of your ideas will succeed. If you assume that each idea you have has
a 5% chance of success, by the time you try 20 things, your chance of success goes up to 64%, by the time you try 50 things, it goes up to 94%. So you
need to try as many things as fast as possible. If you only get to idea #10 two years into your company’s life with 2 months of runway left, you’re shit out
of luck.

!

So your #1 priority is to make sure you’re learning as fast as possible. It’s not just about a million A/B tests, it’s about learning what works, what doesn’t
and why your users are behaving the way they do.

!

That means you’re looking for the cheapest, fastest way to get feedback on an idea, to validate an idea, to answer a question. To learn. Once you’ve
learned and proven a hypothesis, you can be conﬁdent it’s worth investing time in building something—that what you build will actually be used, solve a
problem, and help your business goals.

!

That’s the difference between product discovery vs. product delivery. Product discovery should be measured in hours and days, while product delivery is
measured days and weeks.

Product trio
Product manager
Design lead
Tech lead

So let me talk you through how we do product discovery at Songkick. Let’s ﬁrst start with who is involved in product discovery, the dream team. At
Songkick we have what we call a product triumvirate. I learned this from Marty Cagan of SVPG, one of the best product thinkers around.

!

The product triumvirate consists of your product manager, your design lead, and your tech lead. Together, they represent the 3 ingredients to product
success: the business, the user, and the engineering to make it all possible. The three of them are the BFFs responsible for product discovery. They should
be meeting and talking every day about product discovery. At Songkick, we block out an hour every day after stand-up to talk about what we’re learning
and doing regarding product discovery. Having this regular time to catch up on what we’ve learned and what we want to learn next has been really
important to keep things moving. Remember what I said about speed being everything?

!

The three of us meet every day to ask questions and ﬁgure out the fastest way to answer them. These questions form the foundation of our hypotheses.

GOLDEN RULE OF
PRODUCT DISCOVERY
02
FRAME THE QUESTION

That leads me to the next golden rule of product discovery: frame the question.

Asking the right questions
UI
What do they see?
Interaction
How will people actually interact with it?
Feature/project
How will this help your product solve the problem?
Product proposition
How will your product solve the problem?
Customer discovery
What problem are you solving and for whom?

I like to think about these questions in terms of levels of the product. Depending on which level you’re exploring in, your questions will be different and
will require different type of feedback and a different type of discovery.

!

At the bottom, deepest level of the product—the most important level—are your customers. Who are your customers? What do they have in common? And
what is their biggest problem? How do they currently solve that problem?
For example, at Songkick, we help fans go to more concerts. We promise that they’ll never miss their favorite bands live. Our customer is the lapsed
concertgoer who has a busy life and doesn’t have time to put in the work of scouring listings and consuming music news. They maybe go to 2 concerts a
year when they can be bothered, but ends up going to twice as many concerts after they sign up to Songkick. We learned early that our target customer
isn’t the avid concertgoer who’s already going to 2 concerts a week because they’re already doing all the hard work to find out when concerts are
happening. We can’t help them as much.

!

Then, you have the product you’ve created or will create to solve the problem. Will it work? Will it help people? Is it doing a good job? How do you know?
How do we solve the problem?
We help fans track their favorite artists and then send concert alerts to them via email or push notification when new concerts are announced for those
artists. We aggregate all the ticket options for the concert so they can quickly and easily buy tickets.

!

What features help fans do this?
One example for us is an iTunes import to track artists from your iTunes library. This feature sits alongside other ways of getting your taste as quickly &
easily as possible, such as importing your Facebook likes or artists in your Spotify playlists. Again, if you don’t know how your product is solving a real
problem, you won’t know if your feature will be useful or not.

!

At the interaction level, we have the specific ways people need to interact with your product in order to perform successfully complete their actions. These
are the screens, buttons, gestures, taps, flows they need to experience to achieve their goals.

GOLDEN RULE OF
PRODUCT DISCOVERY
03
THE CHEAPEST,
FASTEST FEEDBACK
Frame the question ﬁrst, then ﬁnd the cheapest, fastest way to get answers to that question.
Hint: it’s not always code and it’s not always A/B tests and it’s not always quantitative.

!
We start with the product triumvirate deciding on what questions we have and then as a trio, we come up with the fastest way to answer that question.
!
Remember what I said before about speed & quantity being everything? There’s usually a faster, cheaper way to get feedback, so keep digging.

Toolkit: quantitative and qualitative
• Analytics

• Mock-up and click tests

• A/B tests

• Landing page tests

• Real-time behavior

• Remote usability testing

• In-app feedback

• User research: in person

• Surveys

and over Skype

Here are various tools in our Toolkit of feedback, quantitative and qualitative
- Analytics: to learn what people are doing right now with your product. We use Google Analytics and Mixpanel
- Real-time user behavior: CrayzEgg, Ghostrec
- Landing page tests: Unbounce
- In-app feedback: Qualaroo and Olark
- Survey tools: Qualaroo & Polldaddy
- Mock-up and click tests: VerifyApp, FiveSecondTest
- Remote usability testing: Usertesting.com and WhatUsersDo
- User research: in person and over Skype <— my favorite!

EXAMPLE: CUSTOMER DISCOVERY
UI
What do they see?
Interaction
Feature/project
Product proposition
Customer discovery

I’ll walk you through a few layers of product discovery and the methods of feedback we’ve used at Songkick.

!

Example 1: Customer discovery
Who has the problem? What do they have in common? What is their problem and how do they solve it right now? In Y Combinator’s words, will you be
making something people want?

!

In the lean startup world, this is customer discovery. In the user-centered design world, this is contextual inquiry. No matter what it’s called, it’s the
foundation of making great products. You’re discovering your customers and their problems, often before a single wireframe is sketched, before any code
is written. There is no substitute for face-to-face empathy when it comes to customer discovery. You can’t A/B test your way into understanding what
problems people have.

!

Let’s take a look at Detour. Detour allows fans to pledge to buy tickets for artists before a tour date is conﬁrmed. This real demand signal in the form of a
saved, pre-authorized credit card then allows artists to plan their tour schedules more accurately.

!

But before Detour the product was launched, we conducted dozens of interviews with artists, managers, promoters, and agents to understand how tours
are planned. Until that point in time, Songkick was only a product for fans. We understood the fan pain points very well and had years of research under
our belt. But we knew very little about the other side of the equation: the artists.

!

So we met with all these different roles in the industry and asked them the same set of questions to start mapping out an understanding of the process
artists go through when planning tours and the common pain points they share. Questions like, Tell me about the last tour you were involved in... What
was your biggest frustration?

!

CUSTOMER DISCOVERY TIPS
• Document

what you’re trying to learn.

an interview script to learn those things. Use it for
every interview.

• Write
• Ask

open-ended questions & listen.

• Ask

people to tell you stories about specific experiences.

• Don’t
• Pair

ask people to predict future behavior.

interview & transcribe.

Here’s what I’ve learned about conducting interviews with customers:

!

- Start out with what you’re trying to learn. Document and agree on those questions with whoever has an interest in the research.
- Design a script of questions that will help you get answers to those questions. Having the same set of questions that you ask consistently will help
patterns emerge among your customers. As you hear answers from dozens of people to the same questions, you’ll start to recognize the same answers
and spot patterns.
- Ask people to tell you speciﬁc stories about past experiences. Don’t ask people to generalize. Ask “Tell me about a time this happened to you.” Don’t ask
“Do you usually.....?” People are unreliable at self-reporting behavior.
- When possible, conduct these interviews with someone else who can also observe & take notes. It’s helpful to have someone to discuss what you thought
you heard.

TAKE THE TIME TO SYNTHESIZE INSIGHTS

After each hour-long interview, we sat down and synthesized everything we learned. We would take our raw transcript and pull out observations. We
created affinity diagrams of our observations until we we had synthesized the raw transcripts into digestible insights. I call this processing raw broccoli
into Doritos. It takes a long time and a lot of debate, but the insights that come out of the other side are then the most important observations you want to
share with your team. If you just dumped your pages of transcripts onto people to read for themselves, they’d never do it, and they wouldn’t know how it
should inform their day to day.

!

This process took us about one month from start to ﬁnish. We interviewed dozens of people. We were doing customer discovery from scratch, learning
about a brand new customer and their problems.

Through this process, one of the things we learned is that the availability of streaming music services such as YouTube and Soundcloud meant that artists
were growing an international fanbase faster than ever before. But tour planning hadn’t yet adjusted to these new realities, so while artists knew they had
some fans in Brazil, Mexico, Singapore, Indonesia, they weren’t sure how many and whether there were enough to plan a tour date. And there was a lot of
waste in the system. It felt like a product such as Detour that exposed this demand in a reliable signal in the form of a pledge--a pre-ordered, guaranteed
ticket--could help solve this problem.

SONGKICK TOUR DATA 2008

Our own data backs this up. This is all the tour routes for all artists in 2008.

SONGKICK TOUR DATA 2012

And these are all the tour routes in 2012. You can see the pink lines are much more dense overall, showing that there are more artists on the road than
ever before, as well as a ton more tours going to Asia, South America, and Australia.

EXAMPLE: PRODUCT PROPOSITION
UI
What do they see?
Interaction
Feature/project
Product proposition
Customer discovery

So with Detour, we identiﬁed customers on both sides of the market: fans and artists, and had an idea for a product that could help solve their problem: a
way for fans to pledge to buy tickets before the tour is planned.

Once we’ve done the hard work to identify the customer and the problem they’re having, we still may not be sure exactly what proposition will resonate
with them. We’ll write a few different propositions with an email collection box using a tool like Unbounce, and send traffic to each version to see which
proposition converts best. This helps give a signal as to what is resonating most strongly. It won’t tell you exactly what the product should be, but it will
help point you in the right direction.

EXAMPLE: FEATURE
UI
What do they see?
Interaction
Feature/project
Product proposition
Customer discovery

I’ll now talk you through the various ways we went through redesigning a feature we thought could be improved.

!

On Detour, the critical conversion rate for the product was the % of fans who saved their credit card. Without this saved credit card, a pledge was as trivial
as a like or a listen, it didn’t help the artist forecast actual demand. So that conversion rate was the metric we wanted to improve.

!

We saw that our conversion rate to saving a credit card was lower than we thought it should be, so we came up with some questions. Remember what I said
about framing the question?

!

-

!

What step in the funnel are people dropping off?
Why aren’t people saving their credit card?
When is the right time to ask people for their credit card?
Is this a usability issue or a desirability issue? Do people understand why they need to save their credit card or do they just not want to?

Often, a lot of your discovery work will be about this distinction between usability and desirability. It’s easy to ﬁgure out whether your product is usable:
whether people understand what they should do, what steps they need to take, and why. If the low conversion is not an issue with usability and
understanding, but rather with desirability -- that’s a much harder problem to solve.

!

For each of these questions, the product triumvirate: the PM, TL, and design lead sat down and came up with ways to get feedback on those questions.

LOVE YOUR FUNNEL

Let’s take the ﬁrst question: Where are people dropping off?

!

To do this, it’s important that your product trio share an understanding of the steps people can take before they convert. Your funnel. Even for a seemingly
simple site, your funnel will have many entry points and exit points. We diagrammed our funnel to saving credit card and it looked something like this.

Once you have that funnel diagrammed, it’s important to make sure you’re measuring every step in the funnel. You have to do the boring, anal work of
defining the events & properties that will be triggered when someone performs an action. It’s painstaking stuff if you’re not part of a big company with a
business intelligence team, but when you see how much the data can teach you, it’ll be worth it.

!

We took this flow diagram and screenshot every step and put it on a wall. Then, as a product trio, we’ll sit down and write post-it notes of what we want to
learn at each step of the funnel: How many people click that button? Are they logged-in or logged out? How many people saw an error message? Did they
get to that page by clicking on the photo or the text, since both links go to the same place? What template are they on when they click that button? Are
they connected to Twitter and Facebook? Are they on Android?

!

It’s easy to go overboard, so you should step back and ask how the information will help you make better decisions. This process can take hours when
you’re doing it for the first time and are defing your categories for the first time, but once you do it, the data keeps on giving!

!

If you take the time to add the right analytics, you can answer so many questions without writing any code, like What page are people converting on? How
many pages do they visit before they convert? How much time does it take them between steps in the funnel? What is the optimal time to remind someone
about a concert? This is all learning.

!

Once we analyzed our funnel and had dropoff rates for each step, we knew what was happening, but not why.

That’s when we started getting qualitative feedback and usability testing the product.

!

At Songkick, we’ve built our own usability lab in-house. We knocked down a wall in our meeting room and created a two-way mirror, where on the one
side, the researcher and participant sit and look at prototypes, where on the other side of the wall, we can watch what’s happening, see a screenshare of
what the user is seeing. It was cheap and meant that we didn’t have to rent a lab whenever we wanted to do usability testing.

!

Through usability testing, we learned that people didn’t understand that they needed to save their credit card. They didn’t realize that it was a step they
needed to take. They would click on a “pledge” button, sit back, and think they were done.

!

Pledging to buy a ticket for a concert that doesn’t exist yet is such a weird and novel experience, they didn’t know what to expect.

EXAMPLE: INTERACTIONS
UI
What do they see?
Interaction
Feature/project
Product proposition
Customer discovery

The next step was to quickly generate a lot of ideas around how we could solve this problem. People didn’t understand that they need to save their card,
so how could we design the interaction better to help users along?

At Songkick, we often use cross-functional sessions to generate a lot of ideas. Our design director, Gideon, likes to say that everyone at Songkick is a
designer, and I agree with him. Everyone from our customer support team to our CEO shapes our users’ understanding of who we are and what we do. So
we do a lot of collaborative design studios, using a lot of post-its and Sharpies. The medium of pen & paper is fast and cheap and the appropriate level of
ﬁdelity when you’re coming up with new ideas.

We’ve even gone as far as to prototype with cardboard to get our idea across.

DESIGN STUDIO TIPS
• Involve

people outside product team.

• Frame

the problem clearly.

• Give

people time to come up with ideas solo
before going into group critique.

• Synthesize

the best ideas and prototype
different options.

• This

can be done in an afternoon.

What we’ve learned about conducting these design studios:
- Frame the problem very clearly at the start of the session.
- Give people time to generate ideas solo, before going into a group critique. This allows for a wide varieties to emerge before you narrow down to the
best ones.

!

Remember what I said before about speed and quantity? Design studios are great for slamming out a quantity of options. Then you have to give feedback
and narrow in on the best ideas you want to prototype ﬁrst. At Songkick, the design lead will take the output of our session and synthesize those ideas
into prototypes we want to test with our users.

A/B TESTS
• Start

with your hypothesis: “If… then…
because…”

• How

many people do you need?

• Record
• Take

results.

the test down.

With this speciﬁc interaction of saving your credit card, we then A/B tested different versions of the interaction, asking people for their credit card at
different steps, in modal pop-ups vs. full pages to determine which version would convert best.

!

When we A/B test an idea with real code, we always run a kick-off where we document our hypothesis for that variant. It comes in the form of an “if then
because” statement. I learned this from the Mind the Product blog.

!

If we ask you to save your credit card in a pop-up after you click a pledge button, then you’ll be more likely to save your credit card, because it’s clear
that’s the next step in the process.

!

It’s really important for the product trio to discuss this hypothesis and agree on it, because otherwise you’ll each have a different interpretation of what an
A/B test result is telling you.

!

We have a checklist kick-off doc that includes a set of questions we run through at the start of any experiment to make sure we don’t miss any important
questions. At Songkick, we love checklists. I took a lot of inspiration from Atul Gawande’s research into how checklists helped hospitals reduce infection
rates by giving them a simple list of things to do to provide the best care.

!

We’re not doing brain surgery or dealing with life or death issues, but at Songkick, our checklists make sure we go through the necessary steps to produce
the best results.

!

Some checklist questions in our experiment kick-off:
- What’s our hypothesis? What’s our goal in running this experiment? What’s our current conversion rate? What’s our expected improvement to the metric?
How many trials do we need in order to get statistically signiﬁcant results? What analytics events do we need to add to the product?

!

EXAMPLE: REMOTE USABILITY
UI
What do they see?
Interaction
Feature/project
Product proposition
Customer discovery

We did also did lot of remote usability testing on Detour to test our explanations of the concept. We knew that Detour is quite a strange experience when
compared to buying a normal ticket. We’re asking you to pledge to buy a ticket for a band who hasn’t announced a concert yet. Unlike crowdfunding sites,
the creator has not yet been involved until we aggregate their demand and reach out to them. We knew there was a big uphill battle in educating fans
about how the system works and to earn their trust.

So we put prototypes and mock-ups in front of people on sites like Usertesting.com and Verify App and ask them questions to make sure they understood
what was going on. Do they understand when they’ll be charged? Do they know why we’re asking for their credit card? How would they explain the concept
to a friend? (That’s one of my favorite go-to questions to hear people talk about our product in their own words.)

!

A click test tool like Verify App can make sure your UI design is communicating effectively. We reﬁned our design and copy until people really understood
how the platform works. Those sites can give you really great, fast feedback on simple tasks -- the raw usability -- but won’t give you the deeper
desirability insights that face to face interviews will provide.

!

But if your question is around “do people understand how to use this?”, these tools can give you the answers very quickly. Literally, in minutes, you’ll have
videos of people struggling with your product, very confused, and unsure about what to do.

!
At the end of this discovery process, we learned enough to improve our conversion funnel by about 30%.
!

GOLDEN RULE OF
PRODUCT DISCOVERY
01
SPEED & QUANTITY
OF LEARNING
This is just a glimpse into how we do product discovery at Songkick. We use methods as varied as face-to-face interviews, surveys, remote usability
testing, and A/B tests to learn as fast as possible.

!

To recap my 3 golden rules of product discovery: your #1 goal is answering questions & learning, not shipping product. This learning then informs your
product delivery phase when you’re spending the time & resources to build product based on what you’ve learned.

GOLDEN RULE OF
PRODUCT DISCOVERY
02
FRAME THE QUESTION

You need to ask questions before you learn. It’s important to know what you’re trying to understand, what your unknowns are.

Asking the right questions
UI
What do they see?
Interaction
Feature/project
Product proposition
Customer discovery

Know which level of the product you’re in when you’re learning. It makes no sense to test two different button colors at the UI level if you don’t even know
whether your product is solving a real problem for people.

GOLDEN RULE OF
PRODUCT DISCOVERY
03
THE CHEAPEST,
FASTEST FEEDBACK
As a product team, you’re always ﬁghting to ﬁnd the cheapest, fastest way to answer your questions. You should have a variety of tools in your toolkit to
get feedback and use all of them at different points in your product discovery journey. At our top speed, we’re running 3 experiments a week, but we’re
always introspecting about how to go faster.

Toolkit: quantitative and qualitative
• Analytics

• Mock-up and click tests

• Real-time behavior

• Landing page tests

• In-app feedback

• Remote usability testing

• Surveys

• User research: in person

and over Skype

You should make use of a diverse range of tools to answer your questions. Both quantitative and qualitative feedback are necessary to building the right
product. You’ll ﬁnd that as a team, you might feel more comfortable with certain methods over others, quantitative rather than qualitative or vice versa,
but both are necessary to product discovery, so force yourselves to use all of them.

MICHELLE YOU CPO & CO-FOUNDER
@WRECKINGBALL37

Thanks to @nfelger, @ClarkeTom, and @gideonb for feedback!
Happy discovering!

Songkick Product Discovery FOWA (Michelle You)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (18)

Ähnlich wie Songkick Product Discovery FOWA (Michelle You)

Ähnlich wie Songkick Product Discovery FOWA (Michelle You) (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Songkick Product Discovery FOWA (Michelle You)