1. Customer Feedback
The missing piece of the Agile Puzzle
June 22, 2012
Agile Roots 2012
Salt Lake City, Utah

2. Who am I?
• This guy
• Maciej (“Ski”) Skierkowski
• @skierkowski

3. Why are we here?
• We are learning and we are sharing
• I learned a lot and I’ve got a lot more to learn

4. Why are we really here?
Building products is easy,
building products customers want is hard.
• I will share a few…
– basic rules
– basic mechanisms

5. Rule #1: Your opinion is wrong
• Problem: you think you know what the
customer wants, but you are probably wrong.
• Customers will use your product in a different
way than you expected
• They might not use the product at all

6. Rule #1: Your opinion is wrong
• The litmus test
– “I think customers need…” (ur doin it wrong)
– “Customer *xyz+ needs…” (ur doing it right)

7. Rule #1: Your opinion is wrong
• Solution: build a machine that validates (or
rejects) your assumptions about customer
needs

8. Rule #2: Don’t ask what people want
Problem: Customers ask for features. Your are in
the business of solving problems, not building
features; you need to identify the problems to
solve, not the features to build.

9. Rule #2: Don’t ask what people want
Example 1
Consider a feature request such as “GitHub
should let me FTP up a documentation site for
my project.” What this customer is really trying
to say is “I want a simple way to publish content
related to my project…” – Tom Preston-Werner
from “Ten lessons from GitHub’s first year”

10. Rule #2: Don’t ask what people want
Example 2… it’s not just about features.
• Friend: “How much are you willing to pay for
an iPhone”?
• Me: “No more than $250”
• (3 months later)
• Me: “I just bought an iPhone for $450”

11. Rule #2: Don’t ask what people want
Example 3…
Customers: “we need custom SSL”
• That was a no-brainer, but we had more
questions:
– How much are people willing to pay for it?
– How much demand is there?
– Do they need to update certificates?

12. Rule #2: Don’t ask what people want
Solution: when someone asks for a feature, ask
“why”, and do it at least 3 times

13. Rule #3: Fake it till you make it

14. Rule #3: Fake it till you make it
• Problem: If you build it first…
– Will people use it?
– Will they pay for it?
– How much are they willing to pay for it?
• Solution…
• First build a façade to allow users to express
interest
• Once you have traction/evidence, then build
the real thing

15. Rule #4: Make it cheap, throw it out
• Problem: you invest in building a product but
nobody wants it
• Don’t get too attached
• Take shortcuts
• Build only what you need. Cutting is king.
• Throw it out if people aren’t using it; it’s just
dead code.

16. Rule #5: Scalability is not a problem,
make it one
• Problem: Engineers want to make it scalable… but
nobody is using it
• The real problem is getting people to use your
product and to pay for it. Scalability is just
premature optimization.
• Scalability is a great problem to have.
• (e.g. PHP Fog hitting 2^15-1 apps)
• Keep an eye out for “quality” too.

17. Rule #6: Overcommit and deliver
• Keep resources fixed, keep schedule fixed, and
keep the functionality
• Leverage creativity and 3rd party services to
deliver without compromise.
– Pusher, Loggly, Airbrake, Sendgrid, Mailchimp,
Google Forms, GitHub, RedisToGo, …

18. Rule #7: Don’t bother with feature lists
• If you can’t remember it, it’s probably not
worth building
• The things you need to build are the ones your
customers won’t shut-up about.
• If you have a big backlog/icebox… delete it.
Nobody will miss it.

19. Rule #8: Not all users are customers
• Problem: Listening to the wrong customer will
lead you to solving the wrong problems.
• Learn why customers use your product
• Learn to identify users who are your target
customers

20. Recap of the rules
1. Your opinion is always wrong
2. Don’t ask what people want
3. Fake it till you make it
4. Make it cheap, throw it out
5. Scalability is not a problem, make it one
6. Overcommit and deliver
7. Don’t bother with feature lists
8. Not all users are customers

21. Rules => Mechanisms
• Rules give us structure
• Product team delivers value to customers
• Customers tell us what is valuable… but how?

22. Outstanding Support
• Everyone provides support
– Executive team (CEO, Director)
– Developers
– Support team
• A business differentiator
• Very easy to identify problems for existing
customers

23. Net Promoter Score
• The survey:
– [0 to 10] Would you recommend this service?
– Why?
• Powerful free response
• Score doesn’t tell you much, but lots of great
candid feedback about product
• Identifies with both existing and lost
customers

24. Live Chat
• “Hi, anyone there”
• Makes the company personal
• Immediate gratification and solutions for
support issues

25. Interviews
• Every week interview a customer for 15-30
minutes over the phone or over coffee
• Common questions: Name, Job, story, their
business, what they love/hate, etc.
• Special question: Ask for advice
• Share info with the company
• Helps make the customer a promoters
• Makes customers the advisors

26. Surveys
• Use sparingly
• “Data driven decision making” or “decision
driven data making” (hint: it’s the latter)
• Great for targeted data (e.g. marketing
material)
• Terrible for testing your value proposition

27. Usability study
• Don’t make this formal
• Make it cheap and easy
• Hire a good designer/usability person
• Everybody focuses on usability
• KEEP IT SIMPLE!!

28. Discussion Forums
• http://community.phpfog.com/
• Surprised at engagement
• Customers helping other customers
• Great way to engage with groups of
passionate people

29. Third party services
• Customer information is everywhere
• Services we monitor and engage
– Quora
– Blogs and comments
– Stack overflow
– Google Alerts
– Hacker News

30. Workflow analytics
• Tools
– Google Analytics
– KISS Metrics for flow measurement
• Great for optimization
• Bad for testing hypothesis

31. Retention Emails
• “I noticed you haven’t logged in over the past
2 weeks”
• You got the user over the biggest hurdle
(getting their attention) but never got them to
the finish line (converting them to a
customer).

32. List of mechanisms
• Outstanding support
• Net Promoter Score (NPS)
• Live Chat
• Interviews
• Surveys
• Usability studies
• Discussion forums
• Third party services
• Workflow analysis
• Retention emails

33. Thank You
• @Skierkowski
• http://blog.skierkowski.com/