Lean Startup at HubSpot
•Als PPTX, PDF herunterladen•
5 gefällt mir•1,248 views
Yoav Shapira of HubSpot describes how they approach the Lean Startup principles, including learning from customers and measuring the business. You can download the original PowerPoint presentation, which has more speaker notes and links. It also looks nicer ;)
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (12)
Ähnlich wie Lean Startup at HubSpot
Ähnlich wie Lean Startup at HubSpot (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Lean Startup at HubSpot
- 1. Lean @HubSpotaka “My Many Mistakes” Yoav Shapira / @yoavshapira August 25th, 2011
- 2. TABLE OF CONTENTS 1 HubSpot Context 2 Things Not Covered 3 Meaty Data Q&A 4
- 5. Actual customer giving actual feedback! SaaS $250 - $1,500+ / month.
- 6. Hey, an actual revenue ~hockey stick
- 8. Lean Startup Theory – you know it already.
- 10. Blank
- 11. Maurya
- 12. Prendergast
- 13. Cancel
- 15. 3 The Meat
- 16. You know the loop…
- 17. So HOW do we learn? Let me count the ways…
- 18. Dev sits next to our lead customer, HubSpot Marketing.
- 19. Customer Support: a chance to teach
- 20. UserVoice: near-free, fun, easy
- 21. NPS: open-ended comments FTW
- 23. CHI: Customer Happiness Index
- 24. Learn from each other: wiki (under-rated)
- 26. You know the loop…
- 27. Everyone codes + deploys…
- 28. Deploy moreoften as you grow…
- 29. Good for theory geeks like me: Flow.
- 30. … with an improving safety net.
- 31. I ♥ Nagios
- 32. I ♥ Graphite…
- 33. … so much that I have 2 examples!
- 39. You know the loop…
- 41. Scale horizontally: small teams
- 42. Each team does their own planning
- 43. “Science Fair” Commitment & demos
- 44. 1 line of code to create experiments
- 45. 1 line to deploy, thx Q Team!
- 46. All trunk, no branches. Feature gates / switches control rollout. See Flickr / Etsy/ Facebook.
- 47. Tech stack matters less than most think.
- 48. Avoid Vanity Metrics 1 Real Metrics 1 CHI Code Coverage? Usage NPS Adoption
- 49. 91%of executives hear “fail” when you say “fail fast”* 91
- 50. Talking << doing. #JFDI
- 51. Product – Market fit is never done. The product will never be ready. Image credit: netzkobold
- 52. I’m Yoav Shapira. Nice to meet you ;) Thanks for listening. @yoavshapira
Hinweis der Redaktion
- - Unlike most startups where this is projected, ours is actual, w00t.
- - Image from Flickr, free for commercial use, photo credit to irene @ http://www.flickr.com/photos/irenetong/
- We had an interesting start. I could speak a lot about our early product-market fit experiments, but no time today.Q&A on this is welcome.
- No need to schedule meetings, just turn around and ask.Obviates need for some classic PM-type activities.Eat your own dog food, and all that. www.hubspot.com runs on our CMS, hosted by us, etc.
- Encourage any interested developer to do Support.Doesn’t have to be phone, can be web chat.E.g. I often do an hour a week of Support.Treat Support as a chance to learn and teach, not just answer questions, file bugs, and get people off the phone.Measure standard metrics, like call time, wait time, % tickets resolved within timeframes, etc.But also do a Support NPS survey on every 5th call, track it over time.
- Also our customer forums, very active.Customers like this and use it.Developers can mark items as planned, in progress, and doneA little duplication with our internal issue tracker, but not much, and not badBeware false democracies
- Net Promoter methodology: http://en.wikipedia.org/wiki/Net_PromoterNote criticisms. Overall, we believe in it, and it’s been insightful.Grouping of promoters / detractors into categories is always contentious, always interesting.Slice and dice by customer type, by customer cohort (when they joined), by product type, by feature area, and more.
- This is just volume of respondents to NPS survey who talked about the given category.
- An internal algorithm we constantly iterate. Major credit to Jonah Lopin (@jlopin) and Brad Coffey (@bradfordcoffey).> 100 individual factors and data points for each customer., grouped into “Setup” “Usage” and “Results.”Different weights calculate from regression analysis, with the goal of predicting customer success.Note: customer success != churn always. We’ve had successful customers churn, and vice versa.Started in Excel, #JFDI as David Cancel says.Now automated, daily. Next step: Salesforce.com Black Tab.
- Share everything on the internal wiki.People read and comment, leading to good discussion.The wiki is “addictive”Helps newcomers learn company history and context async, on their own, even before they start.Helps transparency at all levels. Everyone’s accountable.
- Annual HubSpot User Group “big deal,” quarterly meetings in our office, groups organizing their ownmeetups around the world.Weekly HubSpot TV “Marketing Update” webcast, open to the public, customers invited. Every Friday at 4pm.Yes, that’s the back of my head, I think.Obviously, not as applicable until you have users.
- CI red light goes on when the build breaks (as well as an email to folks.)Everyone is encouraged to code and deploy. Screenshot shows a consultants, a designer, and a QA / test automation engineer.Doesn’t have to be serious back-end stuff. Can be simple UI A/B tests, e.g. moving a button or changing its color.I wish absolutely everyone coded, but maybe at the next company…
- We’re not quite at Continuous Deployment, but believe in it and hope to get there in the future.Importantly, we’ve accelerated our deployments as we added customers.A “release” may or may not be a new feature. It can be a bug fix, a new experiment rolling out, going to just a few customers, and more.
- Good, advanced book on the topic: Flow. (http://www.amazon.com/Principles-Product-Development-Flow-Generation/dp/1935401009)Hat tip to Dan Milstein (@danmil) again. Good book to explain economic cost of not shipping.
- 2 continuous integration (CI) environments, each a cluster of machines running Jenkins (formerly Hudson) on EC2.One cluster runs unit tests, another runs integration tests for particular scenarios at the web app level. (Those are written using Selenium.)Free, open-source tools.Thousands of tests, developers add more all the time.
- Can’t see clearly in picture, but that’s hundreds of hosts and thousands of services.Scales beautifully. Easy to add checks.Every team owns their own checks. You add them, you keep them up to date.Another great, free, open-source tool.
- Awesome to overlay deploys on app speed.Interesting spikes between 95th and 99th percentile sometimes.Near-real-time updates (every second).Can do simple functions, e.g. algebratic transforms, some stats, on every metric, and graph that.And again, free + open-source.
- Interesting periodicity, eh?And what’s up with those error spikes? Go look at logs.We use Splunk a little bit to help us, but can also just look at the logs on the relevant machine(s) or sometimes at the load-balancer level.
- This is an example of the “SLA” between our Sales and Marketing teams.Marketing promises sales a certain amount of points each month.Different types of leads are worth different amounts of points, based on regression analysis of likelihood to convert to a customer.E.g. a demo request is worth more than a whitepaper download.
- Another set of business metrics, this one for our service marketplace.Cite Eric Ries “white on white checkout button” example for why you monitor these and look for trends.Look for slope differences and trends on service volume, frequency, dollar volume.These are just two of literally hundreds of dashboards.
- Nice to see how Support tickets are handled.This graph by itself doesn’t tell the full picture, but it’s a piece of the puzzle.Do we get back to customers when these are resolved? Do the customers agree they are resolved? What does “resolved” mean? Etc.
- Sales funnel example. Sorry for redacting axes labels.Can drill down by lead type, segment on different sales reps, sales managers, etc.Also track things like individual productivity, team productivity, segment productivity, sales cycle, closed lost, and more.Track everything over time, collecting data on trends, and looking for anomalies.
- Sales productivity example. Sorry for redacting axes labels, it’s to protect individual privacy.Maybe you see patterns, maybe you don’t. It’s an interesting visualization putting together some important metrics.
- No HR department. It’s not needed and it’s a downer.Actually hurts recruiting engineers. For other functions, it may be valuable.Each function can do its own thing.Each function crowd-sources for scalability.Recruiters suck for software engineers, but I’m not going to dwell on that today.Gimmicks can work. Referral bonuses can work.
- 2-3 developers, maybe a PM (not required), maybe a designer (we don’t have enough), that’s about it.Practically no middle management. Team self-manages, tech lead does most BS management-y tasks.Scale the organization horizontally, just like you would the software.Each team completely owns its area of the product (suite), start to finish, from customer development on.Daniel Pink speaks about Autonomy, Mastery, and Purpose. This helps all 3. Watch http://www.ted.com/talks/dan_pink_on_motivation.html
- Used to do it all together in one big room. (CIC 5th floor Training Room!)Grew too big. Now each team does their own planning.But the commitment is joined, so it’s transparent and everyone knows.See “Science Fair” on next slide.We don’t tell teams how to do planning. We tell them to do whatever it takes them to commit to a chunk of work.
- Once a month, everyone invited. We’ve had investors and customers as well. Most important meeting of the month for most HubSpotters.Every team tells their commitment and has a demo station to show work from the past month (2 sprints).Combined “Sprint Planning” and “Sprint Retrospective” meeting from classic Scrum.Worked and scaled pretty well, but we’re always tweaking and improving.Example: had a “best in show” award at first, which was fun, but led to competitive demo station decoration.Cover what got done, what didn’t, and why. The public shaming element helps motivation and commitment.
- Exactly 1 (one) line of code for a developer to create an experiment or split tests, per Eric Ries (http://www.startuplessonslearned.com/2008/09/one-line-split-test-or-how-to-ab-all.html).Automatically collect cohort information and report on it.Automatically do the statistics, report on them in a human-friendly format.All credit to Dan Milstein (@danmil).
- Q Team consists of 3 devs that help HubSpot move faster, learn faster, scale.In retrospect, we started this team far too late. Its benefits are great, should have started sooner.Deploy script handles all our main languages, does a bunch of smart stuff like CI checks, config file checks, and more.Much credit to Jeremy Katz, @katz, leader of this team.
- No branches in source code repository. All trunk.Ship from trunk all the time, trunk is always stable.No time wasted merging, not even with git.This is often counter-intuitive to folks, but once you get used to it, it’s like magic.Read linked presentations on the slides, they are worth your time.
- In almost all non-trivial products and companies, you will have multiple languages. Recognize this instead of fighting it.You will rewrite things numerous times.And the people rewriting will often be different than the original authors. (This is what happens after 2-3+ years.)Pick the right tool for the job, but don’t let it get out of control.At HubSpot, we’ve experimented with a few, standardized on Java and Python.It matters for recruiting, but if the candidate is tied to a fixed, specific programming language, do you really want them?
- Which of these is not like the others?See Ash Maurya post on actionable metrics: http://www.ashmaurya.com/2010/07/3-rules-to-actionable-metrics/Stop wasting time on vanity metrics. They are actively dangerous when mis-read, mis-interpreted, or leading to wrong decisions.
- = that91% number is made up, a joke. Truth might be closer to 99%.Need to keep messaging carefully about failing fast, failing often.Present results in economic terms when possible. We spent $X to fail, but if we found this out later, it would have cost $Y more.Even that’s not enough. You have to have successes along the way, even minor ones, to keep team momentum.If you keep failing fast, you also just keep failing. Need a success, however minor, to buy time and increase morale.
- I’m a huge believer in Paul Graham’s “Maker’s Schedule, Manager’s Schedule” essay: http://www.paulgraham.com/makersschedule.htmlAt HubSpot, the dev team’s picked two days a week (Tue, Fri) as no-meeting days.Different people are more or less flexible on these. I completely block mine and it helps me get a ton of work done.Can go nicely with working from home occasionally.#JFDI hashtag / concept credit to David Cancel, @dcancel.This survives at scale, if you make an effort to keep it. Inertia is against you.
- It’s evolving, changing, as markets and products change.Don’t think of it as a binary event / black and white.It might be readier, but iterate often.You will always have bugs, issues, things that embarrass you, things that make you lose sleep at night. I do all the time.It doesn’t really get better, not for a while, maybe not forever.Keep on chugging, look at the good stuff as well, don’t just be negative.
- Happy to meet folks any time, just reach out.