1. Delivering the perfect MVP
Presented by Geoff Anderson, Ph.D.
Thanks to Luc, ETS and this class for having me.
2. I’m going to quickly take you through three companies
First is Nest a Home Automation company now part of Alphabet Inc. (formerly Google)
They started off with a simple thermostat to control the temperature of your home.
3. This is Flatiron Fighting cancer with big data, matching patients with clinical trials and much more
4. This is medium A place to read / write and interact with stories.
5. What do these 3 companies
share in common?
…
What do they all have in common?
6. MVP
Minimum Viable Product
Other than the fact they are all successful :)
They have embraced the culture and process of delivering MVPs
Incidentally they have all embraced something called the Design Sprint which we are going to give a try in a little bit.
7. Engineering
Machine Learning, Data Science, Java, Java Script,
Python, etc.
UI / UX Design
Sketching, Wireframes, Hi-Fidelity Mocks, Graphic
Design.
Product Guy
Talking with customers, designing product and pricing
strategies, working with amazing engineering teams,
software developers and various stakeholders.
Business
Starting and running businesses, managing people,
accounting and other boring crap.
59%23%
9%
10%
Geoff Anderson, Ph.D.
Co-Founder
Chief Product & Technology Officer
Zombie Hunter
Retro-Gamer
55%45%
B2C B2B
My Time In Software
8. B2C
B2B
Software
Hardware
MVPs Delivered
Quick idea of some of the spaces I’ve worked and delivered in.
Case studies are at the end of the deck.
Show you some positive and negatives, pitfalls and ‘pro tips’
ONE THING IN COMMON!! USERS!!
Exception the machine to machine, automation based use case, the machine is the consumer.
9. “We advise startups to launch when they've added
a quantum of utility: when there is at least
some set of users who would be excited to hear
about it, because they can now do something
they couldn't do before.”
-Paul Graham
A word on the quantum of utility. Great advice, but also as with all “pieces” of information dangerous.
Sales / Markety - will have you in market missing a wheel on a car
10. “The reality is that the minimum feature set is 1)
a tactic to reduce wasted engineering hours
(code left on the floor) and 2) to get the
product in the hands of early visionary
customers as soon as possible.”
-Steve Blank
Too “engineery” will have you refactoring for months BUT very close
11. “…that version of a new product which allows a
team to collect the maximum amount of
validated learning about customers with the
least effort.”
-Eric Ries
Enlightenment! Learning..
This captures the true essence of the test and learn mentality.
12. So WHAT exactly is an MVP?
The MVP process follows four steps:
Find a problem worth solving.
Determine the smallest possible solution (MVP).
Build and test the MVP at small scale (show its unique value).
Engage and excite early adopters (also known as earlyvangelists).
Borrowed this from Spotify.
when you are talking to product owners etc. you’ll here the term USE CASE alot, these are not to be confused with specs.
The use case is the business motivation behind using something
It’s very important that you nail the end to end user case.. for example someone get’s a car but it has no wheels, it’s a miss and it isn’t the business model or many other things that are wrong. In this case it’s a design and product
miss. This is why it’s so important to get to know your users and the motivation behind what they are doing or the why.
In this use case the end user simply wants to get from a to b in a faster way than they can walking or running.
13. What is an MVP
Who is your Target?
Have you talked to them?
What is your purpose?
What are the problems you are
solving?
Where do you want to launch?
Literal and figurative.
When do you want to launch?
Time box, feature box, growth
hacking?
Why you chose us?
Do you have a completed use case
that sets you apart or correctly tests
assumptions?
How are you collecting data?
What questions are you finding
answers to, how are you collecting
and organizing that data?
Take a product centric approach
“Target” your customers thoroughly
-Talk to them
-Shadow them
-Ladder them!
Ask REALLY hard questions
There are a lot of questions as illustrated here but don’t miss the two questions noted here
-WHAT is the problem we are solving?
-How do we make money with this feature? / MVP
14. Figuring Out MVP
Market Positioning
Move strategically or rapidly to
differentiate.25%
Innovation
Is this new, new?
25%
Customer Feedback
Can this feature address my
80%?25%
Internal Stakeholders
UI / UX, Engineering, Execs,
Investors, Sales. Yup.25%
Try to always stay product centric
15. Are you hitting the high notes?
Description
Does your MVP ask and test all the vital hard questions you
have in front of you?
Use Cases
Have you delivered 1 complete end
to end ‘Epic’ use case?
Measurable
Are you measuring important KPIs for your
business? Up time, usability, hardware stability?
Pricing and Packaging?
Will a customer pay for this? Is it
enough to test your business model
and pricing + packaging?
Take a product centric approach
-Make sure you use case is end to end, get the walker to their destination faster and ideally in one piece
-Is this REALLY solving a problem
-Are you making money, will they pay for it?
-Are you LEARNING and measuring the outcomes?
-Can this drive smart pricing and packing decisions later?
16. 2 Types of MVPs
Solving a use case
Solving a problem
Human Touch
Hamster Wheel
Full Concierge
PACKAGE FEATURES
Get it Now
Basic
Pro
Starter
$29,99
Per Month
$9,99
Per Month
$39,99
Per Month
$19,99
Per Month
Standard
Solving a ‘use case’ is a user can login and complete a business value
Solving a problem is the user finds efficiency or a reputable and enjoyable value
Human Touch, means some pain is removed from the MVP by a human being involved, often with setup or training
Hamster Wheel, means a human is mechanically bringing value to the use case or solving a problem supply chain with the idea that they can be automated out
Full Concierge is a condition where without sparing cost or expense a human providers a super high value, highly human service with the intent that cost can be driven down at scale or automation / replacement of human services can make the process viable.
18. Break the cycle
with MVPs
Code Quality Losing Customers
Pitfalls Common
When you find yourself inside enterprise or consumer software land with problems like e
Endless train of rushing from one customer about to cancel to the next “Red Accounts”
Endless customer service issues and reports
Developing months on end without a release..
20. 20
Checked out the book, was part of Google Design Sprint Week with GV and a local company GoDynamo
Went through the whole process, found very valuable
Good for
Solving complex or “hard” problems
Getting everyone on the same page - many founders
Bad For
Product and Engineers ;)
You aren’t coming out of this with a business plan or roadmap.
HUGE time commit - Make sure you are SOLVING for a BIG problem
21. 21
Break Out into Teams
Choose your actual product or a made-up one to solve a problem for.
22. 22
THE BIG PROBLEM!
• Get optimistic
• Get pessimistic
• Define the problem
Typically you sketch out a user flow from the beginning of their interaction with your company or service / product to the final place or ‘goal’ of your company.
Then teams with sync / diverge and create lists of customers they want to address
Problems they want to solve. A lot of consensus building is happening early.
For more details checkout the book or a the website.
Today we are going to focus on the problem what is it exactly your Team is going to solve for today.
23. 23
Sketch!
• Everyone has to
do it!
• Draw out the
solution to the big
Problem!
X2
Tuesday there is a lot of drawing typically.
Today we are going to focus just on one sketch and cut right into solutioning.
Sketch out how you can envision the problem being solved..
When you are optimistic think about how you are going to change the world
When you are pessimistic think about how this project can fail
1) Draw the actual interface, WORDS matter.
2) If it’s a physical problem / solution as opposed to software you can focus on the ‘story line’ as opposed to a specific feature set or MVP focus
24. 24
Decide on the best~!
X2
Typically we would use a form of “anonymous” voting with dots and we would put all the drawings up on the wall then everyone from each group can decide on the solution.
Today if you can reach a consensus in 5 minutes then we can circle around to each others top 2 solutions and vote on those
If we need the full 10 minutes for voting internally within the teams that’s ok as well!
Luc, Sara and I will go around ask questions
25. 25
Build!
X2 (maybe 3x)
Work together to polish the prototype
If any of you have tools from powerpoint to a tool like sketch or UX Pin / Axure` etc go ahead and use those tools
If not draw it out as best you can focus on the solution as opposed to the finished UX experience
29. 29
Radian6
“MVP to strategic exit”
The first Radian6 use case is a great of example of a catalyst behind a $400 million Canadian exit and an Atlantic Canada success story.
Radian6 was a company that grew from a small base to about 300 employees and several hundred customers by the time we were acquired by salesforce.com in March 2011.
We sold Social Media listening software which effectively provided you analytics to understand things ranging from Sentiment towards your brand, to what were popular conversations in an industry to listening to people talk about
your customers. Memes were still new here so the timing was right for companies that were terrified about conversations happening across the web they were not aware of. A perfect example of this comes from an early customer
Dell. They had an issue with a model of a laptop that had the battery overheating on people laps and injuring them. The right people even with our software did not find out about this problem soon enough and could not take
action.
The investigation
I was dispatched to talk to all our key clients, a common finding amongst our 20 top customers including Bank of American, Dell, HP, Activision, Comcast and more was that they were all having the same problem. They key
employees that needed to find out about incidents like this and could take immediate action to remediate the situation were doing the swivel chair between our software and other systems such as CRMS. Our software was complex
to configure and the analytics were complicated to understand.
Phase 1
The solution phase, with internal stake holders. What is the solution for these customers.
MVP proposals
A method to “frame” radian6 inside their existing software systems.
many other TERRIBLE ideas
API approach to look at creating “cases” or “tickets” in the software they were already using.
Pressure
The pressure from the customers and internal stakeholders were the tickets needed to be “Perfect” it needed to seek out and find existing customers inside their CRM that match the incoming tweet and more. They didn’t want
their reports being messed up with bad tickets etc.
The MVP solution - simply put a ‘button’ into Radian6 which ‘pushed’ a tweet into the CRM, from there the user could make the match and fill out the details. The user then could escalate and treat like a traditional CRM ticket or
Case.
The strategic move, this was a VERY skateboard approach. However no other competitors were doing it and it was found that many of the CRM systems in use have APIs in which we can leverage to create tickets and cases with.
30. 30
Radian6
“MVP your way to disaster”
http://www.marketingcloud.com/blog/introducing-salesforce-social-hub-and-radian6-for-the-service-cloud
Later that year a hard lesson was learned. Eager on the heels of an acquisition, salesforce.com wanted a higher level of automation and a great method of integration so basically the “human” factor of pushing cases to
salesforce.com could be eliminated so effectively Service Agents would never have to leave salesforce.com.
A great MVP would have been to simply deliver a single feature to that extent, such as a single keyword, single social channel like twitter etc.
However if you think back to the example of a minimum viable “feature set” and how it can go wrong.. all of it went wrong.
We literally built a rail gun which would later be the source of accidental DDoS to salesforce.com, was the industries most advanced NLP engine, Machine Learning System, and much more. It could automatically detected
“customer service cases” and then push them into salesforce.com AND try to connect the case with an existing customer in the CRM and so much more. We went underwater working with a couple customers like Bank of America
(chose your customers wisely they were too complex in their needs) and then launched in November.
The result was a VERY VERY impressive piece of technology which in turn cost salesforce.com millions in lost customers .. Why?
Because it was too complicated for the users who had to configure it.
Because the automation was actually TOO good, when Occupy Wallstreet kicked in and everyone was yelling at Bank of America they had hundreds of millions of Tweets sent into their CRM over the course of a week.
Because we delivered a comprehensive solution as opposed to an MVP or several small pieces at a time.
To compile the issue there were endless code quality problems as well because we would deliver into production in huge chunks and customers were finding bugs and we were experiencing downtime.
31. 31
OMsignal
“A solution seeking a problem”
OMsignal was an interesting piece of technology and a great “story” however several classic mistakes were made.
Upon my arrival there they had built some impressive tech in garments and hardware but had never delivered a true “product”..
We put together a great series of screens, had many customers interviews however the Beta period of “MVP” period was skipped by exec decision.
What we ended up delivering into the market was a “cool gadget” without a purpose and without an understanding of who actually purchased the product.
After months of recovery and a true “failure” around launch we were able to gather enough data and enough insights to figure out
1) It’s old people with medical conditions that are most likely to buy the gear and have the disposable income to spend on it.
2) The “app” was so generic no one had a purpose for it. People using the app need specific guidance on “what to do” to get the most out of the product
3) The introduction of a simple “test” you could do and show off to your friends increased utilization by 200% and increased sales by over 1000% - this “feature” was literally an MVP which was designed from the group up to ‘tune’
the equipment, give the user purpose and provide a real solution to a real problem which all our users were having which was a very simple question “how fit am I?”
4) From there we easily were able to start answering and identifying other important questions as, am I getting healthier and fitter? Which kept utilization heading in the right direction.
32. 32
Resources
Spotify’s Approach to MVP
https://speckyboy.com/building-minimum-viable-products-spotify/
https://www.youtube.com/watch?v=1FoCbbbcYT8
Eric Ries The Rules of MVP
https://hbr.org/2013/09/building-a-minimum-viable-prod
Harvard on the 2 types of MVP
34. 34
Veri Design Sprint
The Team in Action
Day 1 working on the map
Who was there?
Our sprint team
We were an amalgamation of a project team from Dynamo as well as key client stakeholders.
From Dynamo:
Beckii Adel (UI Designer)
Rodrigo Dalcin (UX Designer)
Max Kaplun (Art Director)
Nancy Naluz (Front-end Engineer)
Harlie Dover (Project Manager)
Alex Nemeroff (Me, the Facilitator)
From Famespike:
Dr Geoffrey Anderson (Product/Tech lead)
Sara Krejcik (Marketing)
Jon Bucci (CEO and The Decider)
Chris Bucci (COO)
Sandro (Counsel)
35. 35
Day 1 The Map
Intentionally blurred :) The map.
What’s the big question we decided to tackle?
Our sprint question
Let’s answer key questions about whether we could ensure a “win-win” outcome when connecting fans with talent through a mobile app.
36. 36
How we made our prototype
We built a personalized user flow for each of the 6 user testers, which included their name and their favourite celebrity’s profile.
First we wrote out a list of things that needed to be included on each of the screens, then sketched out on paper some rough wireframes to get an idea of the complete flow.
We used Sketch to create designed wireframes, and then used InVision to stitch it all together and create an interactive user flow.
We made sure to use text styles and symbols in Sketch in order to work quickly and be able to make changes that were repeated throughout the document instantaneously, so as not to waste time.
37. 37
What did we learn?
What we learned from the test
We discovered that users were excited about the concept, and that they would pay for the service, although some fine tuning was necessary in terms of when to ask about payment. Probably the most essential information to come
out of the test was learning which types of content people were comfortable sharing about themselves — for example, sharing messages was ok, but they didn’t want to share videos of themselves. Our hypothesis had been the
opposite. Overall, our users saw value in the product, which was great to confirm.
38. 38
MVP?
What’s next for our project
Given the stuff we learned, we were able to create a good set of user stories; with these, we’ve started building the actual MVP. Given that there were a number of questions we simply weren’t able to answer in the first sprint,
though, we’ll do another one once we have a more complete MVP.
Here we are MVP weeks away or less and we are about to engage another design sprint in October.
