As a product manager or a startup founder, you know you need to talk to your customers; Everyone talks about it enough! There are endless articles, books, blogs and talks from well respected experts that repeat that same message: "Talk to your customers". And yet, products still fail because we're so focused on talking to our customers, we forget to listen and understand them.
40. What does this look like?
DEFINE
Idea forms
Some discussion
Write up spec
BUILD
Brief dev team
Create backlog
Create project plan
DESIGN
Brief designers
Create mockups
Brief technical lead
High-level spec
40
Feedback/analytics
Market research
Problem brief
Collaborative design
User testing
Iterative releases
Hypothesis testing
56. › Understand both
› Levels of confidence
› Tailor to your situation:
› Complexity of problem
› Complexity of solution
› Engineering resource
56
Problem vs Solution
57. New Product
More unknowns
Right problem?
Harder to find “users”
Dev constraints
New Product vs New Feature
New Feature
More data/feedback
Problem details?
Existing users to draw on
Less dev constraints
57
58. Qualitative
Customer Interviews,
usability testing, etc
Understand the why
Not statistically valid
Generally not scalable
Qualitative vs Quantitative
Quantitative
Analytics, (some) surveys,
etc
Measure the what
Statistically valid
Scalable to larger numbers
58
59. › Customer Interviews
› Surveys
› Landing Page MVP
› Concierge MVP
› Wizard of Oz MVP
› Fake door MVP
› User Testing
› Core MVP
What are the techniques
59
61. › 1-1 chat with customer
› Problem focused
› Qualitative
› Key points:
› Avoid bias or leading questions
› Talk to at least 10 people
› Ask screening questions
› Reverse engineer questions
Customer Interviews
61
62. › Use at start
› Validate problem
› Existence
› Details
› Mandatory
› Use on new product or new feature
When and how to use
62
64. › Could be quant or qual
› Could be problem or solution
› In-person or digital
› Key points
› Avoid leading/biased questions
› Less suited for quant
› Keep it short! 5 questions or less
Surveys
64
65. When and how to use
65
› Problem focused
› After customer interviews
› Use above to guide questions
› Increase confidence in learnings
› Solution focused
› Get feedback on a feature
› Can target specific users
› Compliments analytics
Caution: open-ended = manual work!
70. › Do it yourself
› Problem + Solution
› Qualitative
› Key points
› Ask questions
› Be hands-on with user
› Track priority vs effort
Concierge MVP
70
71. When and how to use
71
› After problem research
› Before build
› New Products
› Complicated problems/solutions
› Know when to move on
73. › Concierge but black box
› Solution focused
› Not scalable
› Key points
› Review user sessions
› Track priority vs effort
› Step-by-step automation
Wizard of Oz MVP
73
74. When and how to use
74
› After problem understood
› After customer interviews?
› After Concierge MVP?
› Follow up with customer
interviews/surveys
› New products or new features
› Problem backog => Step-by-step
automation
77. › Validate problem by measuring “clicks”
› Very light touch solution, i.e. entry point
› For existing products
› Feature doesn’t exist
› Key points:
› Analytics are key
› Be bold!
› Appreciative tone
› Don’t be dodgy
Fake Door MVP
77
81. › Lots of “MVPs”
› Concierge MVP
› Landing page MVP
› Wizard of Oz MVP
› MVP = Viable
› MVP = Product
› Concierge experiment
› Does the product exist?
MVP Rant
81
82. › Iterating to find MVP
› Not “first version”
› Start small, i.e. alpha/beta
› Iterate towards larger release
› Build on other techniques
› Ultimate goal: prove product-market fit
“Actual” MVP
82
96. Part-time Product Management Courses in
San Francisco, Silicon Valley, Los Angeles, New York, Austin,
Boston, Seattle, Chicago, Denver, London, Toronto
www.productschool.com
Hinweis der Redaktion
notes
Clarify purpose of the talk so everyone understands up front
Bit more info on the problem so we’re clear on what we’re solving
Some key factors to consider to frame these techniques
Run through techniques
Classic Q&A
Want to understand audience
Product manager in a team of product managers?
The only product person in entire company
Looking to make a change into Product Management
Something else?
Recognise the true need and what you’re trying to achieve
Not perpetuate the usual talking points
Process is about people - so make sure people feel happy
Show of hands - who has heard this in a talk?
Been to countless talks - “talk to your customers”
What actually happens
People are biased
Most common: user testing
Tests the usability of a design of a solution
No insights into customer behaviour
Key message: customer understanding
Don’t just listen, understand
What does understanding mean?
Key message: customer understanding
Don’t just listen, understand
What does understanding mean?
Not a beginner’s guide to customer research; emphasising where people go wrong
Not a guide to techniques; more about the key points when considering whether to use
No new Navin-branded techniques;existing ones are pretty decent for most scenarios
Prob thinking - who is this guy
What makes me qualified to talk about this
My answer is… woah, what’s with the attitude
Been a developer
C++, HTML/CSS, Javascript, Perl, Java, etc
Been a technical product manager
Hands on with really technical niche products
Was a business analyst but who was also product manager
A fully fledged pure product manager
Last couple of roles have been leading product teams
Hiring new members
Training
Setting product strategy
Got to the top
What do I do now?
Gun for hire
Go where I’m needed
Usually cus things are bad
Gun for hire
Go where I’m needed
Usually cus things are bad
This is my day job now
Work at top-level to develop product strategy
Help teams that are struggling to get better
Teach at Product School
Do training and workshops
Mentor product managers, CEO, etc
Problem definition - what are we trying to do
Design - how do we solve this
Build - create product
More complicated; iterative approaches
Release and feedback also on there
Define
Idea usually comes from senior stakeholder or business “strategy”
Might be some discussion - head of product, head of engineering, etc
High-level problem spec
Design
Make sure designers understand
First pass at design
Get technical feedback
Updated spec
Build
Do Dev understand
More detailed specs
Plan of release
Bonus
Making sure problem exists w/ feedback/research
Define as problem instead of feature
Bring tech in earlier, maybe even rest of company
Check the design
Earlier release = earlier feedback
Design tests to check assumptions
Product goes out; people wait with baited breath
Big success!!!!
But what actually happens
Yeah, not quite what we expected
No-one is using the product
No-one is buying the product
Users look at it and shrug
We don’t build products for the sake of it
So what did we do or not do that led us here?
First, remember what product success
We have a list of problems that users might have
We have a list of solutions that could fix those problems
A good product brings the right solution to the right problem
Two parts to the problem
Did we identify a genuine problem for e.g. do people want to order food at night
Do we understand the details of the problem, for e.g. do people want to explore, do they just want the same stuff, convenience vs price vs quality (or both), if so, when do either happen?
Similarly two parts to the solution
Did we pick the right solution, is it an app that lets your order food
Did we target the solution to the problem (using those understandings), is it a “top rated” list with filters, is it a browse by cuisine?
Everyone focuses on solution
Easier to understand
Easier to plan; action
Less ambiguity
That’s why people jump straight to design and user testing
People make assumptions
People misunderstand what people need
I would argue that companies are getting better at knowing there’s a problem
Discount the startups that just go head first into it
I hear pitches and the top level seems reasonable
It’s when they go into the detail my product senses tingle and I cringe
Because at that point, they’re like, yes, I know what they need and go off arrogance
One startup founder, I tried to help him, but he would rather take funding and build something for 3 months before first release, rather than try one of the techniques listed. Which would cost him nothing but time
Here are some of the techniques you can use
Will run through them in more detail
Keep these in mind when deciding what to do
Use these to understand the techniques better
Need to understand both
Have to understand problem and understand how solution will fix problem
If you don’t, chances of success = low
Not a perfect science
Never going to be like, yes, I am 100% sure
Don’t take too long trying to be perfect
Consider different factors
If problem is complex, spend more time, but if easy don’t dilly dally
If solution is problem, spend more time
If easy access to devs, consider a more experimental approach of trying different things
New product
You’re starting from scratch, no data or feedback (unless you get it)
Do you even have a problem worth solving?
How do you get users? Are they even the right users? Who are your users?
Potentially no developers, or expensive outsourcing. No existing product infrastructure to “plug” into
New Feature
Existing product with analytics, customer feedback, etc so more clarity
More focus on understanding the detail of the problem
Users well understood, easier to reach out and get details
Existing team and product infrastructure
Qual
Name suggests - quality
Examples are customer interviews, usability testing
Focus on insights, i.e. why people do this
Not statistically valid so don’t assume applies to everyone
Not scalable because labour intensive
Quant
Name suggests - all about quantity of data points
Ex like analytics, surveys (in some cases, as I’ll cover later)
More about the what, doesn’t tell you why
Stat valid as there are far more data points, or rather you have to collect meaningful quantity of data
Scalable as it’s more tech based, i.e. send standard survey with options, check analytics data, etc
Here are some of the most common techniques
Let’s run through them now
Conversation with customer
Open-ended questions
Normally 1-1
Best in person
Problem focused
Usual mistake is people bring solutions
Identify whether target market has problem
If do, delve into detail
Qualitative
You get feedback you need to process
Look for insights, not stats
Applies to both new product and features (but if features, most likely exploring the details of the problem)
Key points
Don’t push people down a path; don’t use yes/no questions. Use phrases. A good way is to ask about the last time they did something, so it’s a specific story
10 people seems to be a good number. It’s not stats but gives you idea. If nobody has the problem, did you screen properly. If everyone has problem, yay! Also think about where you’re gonna find these peopel
Screen screen screen. Make sure you’re talking to the right people
Typically you have your answer, your assumption. Think about how to phrase a question that would result in that answer. And then ask it without prompting. If you’ve asked and they don’t, then can prompt, ask how it compares to what they said, why they didn’t bring it up, etc
Always use at the start, too late otherwise
You can use to validate problem and details, though you need to find the right people
Could be quant if you ask open questions gathering feedback
Could be qual if you ask multiple choice since you can collate results
Could be problem if you ask about their lives, behaviours, etc
Could be solution if you’re asking about a product they’re using
Could do it in-person or could be digital
Key points
Same principles apply; if you’re asking a question, don’t make it biased. Neutral language, not so much yes/no. Ask them to rank lists, etc
I think it loses it’s power for quant because either you’re not there or you’re not exploring why they feel that way
Don’t put too much in or you’ll never get responses!
Problem focused
Make sure you’ve collected good insights first, unless you can’t
Use this info to guide your questions
Adds some stats weight to what you’ve found so far
Solution focused
Do after a release to check perception of a feature
Can be specific with people who did some set of behaviours
Analytics doesn’t give you the why, surveys can give you that
Also called pre-order page
Fake a product
Make people think it exists
Provide whatever you need to convince them
All about whether the problem exists
Quant as you’re getting numbers
“Something” is some signal that users want this product, could be signing up, could be clicking buy now
Touches on solution because you need to describe the product, for eg app, or service, etc
Don’t have to spend time building product
Helps your marketing team
Testing your message to understand what works
Gathering future users info
Key points
Test your message, make sure people understand it. User test it
Clear signal for success, signup, book this, buy this, whatever
OK to leave your comfort zone, it’s scary. But stay true, There’s no obligation here, but don’t take money, don’t steal details and misuse them, etc. And not your final brand
Early stage
Pre-build so that you’re not wasting time
Use the learnings from customer interviews
Make sure problem exists; doesn’t tell you details of problem
It gives you an idea if the top-level description of what you’re offering is appeasing, but don’t get too caught up in it. Long way to go!
Use on new products. Could do with an existing, but doesn’t make sense since you’re not saving dev time. And new features prob fit into an existing product with your existing features. Maybe if it’s a completely new product within an eco system?
Provide the solution
You talk directly to the user
You do whatever they want for them
For e.g. food box subscription service - be their weekly shoppers
Combination of both
You still talk to customer, understand their problem in detail
You also try out the solution manually, learn the inside and out
Still gathering info on both problem and solution
You’re learning from customer
You’re learning from doing
Not scalable to large numbers
Key points
Asking questions = knowledge. You’ll see weird things and you can and should ask otherwise you’re wasting your time
You should do this, learn as much as directly as possible
Make sure you record which things are important to users, which aren’t, and understand how complicated certain things are. This all adds up to give you an idea of what you should build, when and why
Do your customer interviews first, maybe even landing page experiments
Do it before building so that you can learn about the solution, save time
Obviously. New features are a bit harder because you already have a product, though not impossible. Will come to a better technique next
If you have particularly complex problems and/or solutions, this helps flesh out a lot of the details with your focus. Otherwise you never know what to build and it fucks with you and causes stress
To learn about the problem. This doesn’t last forever, recognise you need a level of confidence and then you move on. Handily, next step could be the next technique
Same as previous technique
But you take a step back
Replace you with thin product layer
Customer interacts with product as they would
Don’t realise it’s done manually underneath
You’re looking more at the solution now
You’re still learning about how to solve need
You’re testing how the user interacts with the product
But you can’t ask questions directly about the why
Still not scalable due to manual nature of solution
Key points
Learn about how user interacts by using tools like full story that let you playback a user session. You still have small numbers so you should be looking at that data
Just as before, use your learnings about the solution to determine what you need to build next and why
Since we’re comfortable with the problem, we can start building the solution. Good approach is to think about the most painful piece and automate it. If that lets you scale out the product, even better :)
Need to be clear on problem before you go down this road
Could be immediately after customer research because problem isn’t complicated
Could be leading on from concierge MVP
If you notice something weird, don’t be afraid to go back and ask people. This is because you’re not getting the same insights as if you were talking to people
New products, obvs, but easier for new feature as you can fake the user interaction layer. For e.g. at DICE, we had new seat maps. Had no time or clue how to build a seat allocation algorithm. Did experiment for a couple of events, just faked the front-end. It just came to a spreadsheet. Interns would manually process and try to make people fit in. We learnt from it.
Want to repeat that point again. You can build the product step by step. Doesn’t mean make it a frankenstein, but with a good dev team, this is easier. You build only what you need but in a way you can extend. It might take a bit longer but at each point, you’re solving a problem. Keep in mind final goal and figure out how you get there and update as you learn.
I’ll cover other techniques for completeness, but in less detail
These are pretty well understood and for me, not as important as the others which can really improve the way you understand the problem, which is my central point
Similar to landing page, measuring whether anyone wants this
You’re kinda testing the start of the solution, i.e. the trigger that gets them there
For existing products, i.e. ties into a place that people already come to
Pre-build, you save time by checking whether anyone cares or not
Key points
You need to have your analytics sorted, that’s what tells you if it succeeded or not
Similar to landing page, be bold, don’t be afraid
Get the tone right. Be appreciative. You can be honest that this was an experiment. Most people are forgiving
Don’t try to trick people, i.e. don’t be a dick
You’re not really understanding the problem because you’re telling users what scenario
But it is good to check usability issues with the design
Everyone knows how to do this; pretty well documented
It’s still saving time so that you can avoid costly dev mistakes
Different ways to do this. Complicated paper prototypes vs static mockups vs interactive clickable prototypes
Total buzz word, potentially confusing
Personal opinion, but
Viable - means something that can stand on its own feet
Product - something that exists
Everyone just thinks of it as first iteration
Prefer the phrase experiment
That’s all it
You learn something, but you can’t make money
Best question, does something exist after you finish or is it learning?
As we talked about, MVP is more of an iterative building process to find the right point, not just the first iteration
It’s an iterative process. I don’t think you really say here’s my MVP and you build it and that’s it. It’s saying you have confidence in problem and solution, here’s your assumption, now let’s build to that release, try it out and see if it succeeds, if not we continue
Not the first version. Everyone things about it that way. That’s kinda BS. It needs to have a coherent core, not just “something”
You’re building towards a larger release in steps
You need info from other techniques. This isn’t teaching you about problem vs solution. It’s a way to derisk how you go about building things
Product Market fit - that’s your goal. Understanding that and proving it so that you can now work out how to scale.
If only one thing, remember that you need to understand the problem in more depth
Customer interviews are mandatory, if you can’t do this, it means you’re not understanding your customers
Experiments are much cheaper ways of learning. You get to see and interact with users without taking a lot of dev time.
Don’t be beholden to all these techniques. Touched apon when to use, think about what works for you and use an approach that fits your situation
Really found that this and other topics really not covered, been asked by students so running workshops
Do other things like metrics, process, product strategy, etc. This is one of them. Will run through what we discussed in more detail
But most importantly, it’s practical. For me, that’s the key. You can read this stuff, but it doesn’t settle. Have to create these things with support
I’m sorting out venues now and therefore dates, but looking like within the next couple of months
Feel free to ask me for more info; leave your email address or take a business card and email me :)
Been a developer
C++, HTML/CSS, Javascript, Perl, Java, etc
Cards are on table
Running MVT test on designs
All about delivering value
Feedback is like gold dust
Fancy an experiment