25. • Start with user problems
• List your biggest assumptions
• Look to validate your riskiest assumptions before committing to
deliver
• Validation should be focused on the minimum thing we can do to
learn the most of something
• Define success criteria up front
• Don’t be afraid to make tough decisions
Editor's Notes
Grew up in North Devon > Studied economics and management at Oxford University > From university I joined a small fintech startup of around 50 people called Totally Money – starting out as an analyst and progressing into a product lead there > around that same time I founded a clothing company with my brother which we ran in our spare time for a while > I then moved to a slightly more mature startup called Qubit (was around 100 when I joined and now 300-400), focused on personalisation in ecommerce. I was there for a couple of years, lots of growth, saw two rounds and around $75m come in the door > I joined Gumtree about 2.5 months ago. I’m a product manager for our mercury team, who are focused on communication
We’re in the business of solving user problems
In response to those problems we’re trying to find solutions which are desirable i.e. there is demand, feasible i.e. we can build it, and viable i.e. people will pay for it
But along the way we’re making a huge number of assumptions
Even when something relatively simple is put in front of us, we don’t always interpret it in the right way, and this risk increases as the problem becomes more complex
Tidal – big investment from the likes of Jay Z – Tidal’s biggest USP in competing against Spotify was around the high fidelity of their music. Their biggest assumption was that people felt music was not of high enough quality and thus this was a problem, but results to date suggest they got this assumption wrong
Like Jay Z, we always have many problems to tackle and not enough people to tackle them at once. Dev is a hugely valuable resource that shouldn’t be wasted on things that aren’t going to make a meaningful difference to our users. We want to stay really focused on the stuff that matters
We set out to validate and de-risk assumptions as an act of efficiency in meeting our user-centric goals. We want to be as quick as possible in solving our users main painpoints. We acknowledge that we can’t solve all the problems for every user, all of the time but validation reduces the risk that we ‘back the wrong horse’
In my experience culture can be the toughest thing – it really requires an organisation to embrace ‘failure’ and this is not always ease to do. In product we talk about, simplicity – the art of maximising work not done, and that doesn’t just mean we’re work shy but it means if we run an experiment around an assumption that we have, and the experiment suggests our assumption was wrong, then this is something to be celebrated. Ultimately we might have invested too much resource in building something that would have had limited user value
So really for all initiatives within product, whether you’re a startup building a whole product from scratch, or a mature business looking to refine or amend your existing product you should really be asking yourself these questions up front
Dual track agile
Discovery is your doctors waiting room, where patients are given a basic assessment to understand the severity of the problem – the most severe cases go in to see the doctor (developers) first whilst the common colds may be sent home
Tom Chi – Google innovator
Minority report
Chopstick and fishing line experiment
Cheap, quick and should involve getting regular feedback and iterating. Avoid big bangs
Qualitative data shows you the why – it gives you context to understand the problem
There are many ways to gather qualitative data so get creative
Could be shadowing the customer service team – I spent a few mornings listening in on calls whilst I worked
User testing – we use usertesting.com – good as you can give a specific task and the research scales easily but also lose some insight from face to face
Card sorting – Jamie – one of the PMs at Gumtree led a card sorting piece of research using a tool called Optimal Sort which had around 1500 respondents
Discovery interviews – separate from user testing as it tends to be more broad and focused on problems rather than solutions
Food on the table founders went and found shoppers in local supermarket. Got people to signup and would then do their shopping for them, and charged for the privilege
Zappos – validation of assumptions of shoe shopping online
Landing page to monitor signups and demand
Dropbox seeded a video around their concept well before launching the product
Button to nowhere
Survey off the back of it
Get the team involved early and throughout the process (by team I mean team leads, devs, QA’s, UX, PMs etc.)
Don’t commit to delivery before their involvement!
In all of this experimentation, it’s really important to define what success looks like up front where possible. What metrics do you care about? And how much movement are you expecting in those metrics?
Ultimately you need to make a decision about which patients get to see the doctor so you need to have a clear idea of the basis for that
On the basis of experimentation and discovery we should be asking proceed, kill, pivot? Killing ‘bad’ projects is an important part and something that the organisation needs to become accustomed to. Part of your job as a PM is ensuring that stakeholders understand this.
The founder of Slack has now failed to create a never ending virtual world game twice – the first time he repurposed some of his company to found flickr – bought by Yahoo after a year. The second time he pivoted to make Slack (they’d made a tool to communicate as a dev team) – in under two years hit a billion dollar valuation