Managing an Impure SaaS Model Using Atlassian Tools
•Als PPTX, PDF herunterladen•
1 gefällt mir•1,719 views
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Managing an Impure SaaS Model Using Atlassian Tools
Ähnlich wie Managing an Impure SaaS Model Using Atlassian Tools (20)
Mehr von Atlassian
Mehr von Atlassian (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Managing an Impure SaaS Model Using Atlassian Tools
- 2. 2 Managing an Impure SaaS Model
- 3. What we’re trying to solve 3 Engineering Sales PM Product Vision Client Input Capacity
- 4. I will be talking about all of these: JIRA: Green Hopper Custom Fields RSS Feed (and some Ruby development) Transitions and Transition screens Confluence Smart page design Run Macro JIRA Issues Macro
- 5. Agenda 5
- 8. Leverage insights from one client to the rest
- 9. Create a positive “network effect”
- 10. Always deploy trunk7
- 11. Meet our ecosystem Waterfall-ish clients $M deals Sale Cycle ~ 8 Months, contracts span multiple years Fixed time, fixed scope RFPs 8 More disciplined LT planning process The Roadmap is not just a “plan of intent”
- 15. Pure agile development: maximum flexibility = maximum value9
- 16. Agile development / SaaS model pose unique challenges in some business environments and require creative mitigation techniques #summit11
- 17. Do nothing 11 Sales Engineering PM
- 18. “It’s not us, it’s them” Shape the market – regulation & policy Sell clients on our vision of the product Be an agile evangelist All great. But not enough 12
- 19. “It’s not them, it’s us” Carefully balance strategy-driven and opportunity-driven product development Deep product empowerment of account specific experts (sales, engagement managers) 13
- 20. Friction between your business/development model and your market is your problem (not your clients’). Be proactive. Adapt. #summit11
- 21. Easily communicable requirement model Common language is a pre-requisite for everything else Handle requirements of different scale Need to address varying levels of uncertainty 15
- 23. Investment themes Not a requirement Not estimated Ensures alignment JIRA custom field 17
- 24. Initiatives Project-like Product-led decomposition Goes on Roadmap Estimate using DTIs (JIRA custom field) 18
- 25. Features Avoid “Epic” Anti patterns Functional collection of user stories Joint (PM/Dev) decomposition Goes on wave (release) plan Leads estimate using story point ranges 19
- 26. User stories Nothing new here Decomposed by leads Goes on iteration plan Teams estimate using story points 20
- 27. Easily communicable requirement model 21
- 28. Case study: rate analysis tool 22 1 2 Feature User Story Initiative 3 4
- 29. Balancing strategy and opportunity 23 2010 2011
- 30. The token system Minimizes disruption to the strategic road map but still enables us to win deals 24
- 31. Committing & tracking 25 Token System
- 33. Status
- 34. Token Budget
- 35. DTI
- 37. Tokens case study Multi $M RFP Saying “yes” to everything: 25 DTIs, ~15% of development capacity for 18 months Using the “token system” resulted in: Productive discussions and creative solutions to trim scope Tough compromises on both sides Scope of work reduced to <10 DTIs 28
- 38. Consider using a “Token System” to balance strategy and opportunity driven product development #summit11
- 40. Live Roadmap: Jira+Ruby+mongoDB Filtering Title Details Release Metadata
- 41. Not your typical release notes What got done in the last iteration? Planning vs. delivery What will get done in the next iteration? What’s in the pipeline? 32
- 42. Different people are looking for different things Changes to the product that affect sales Set expectations with existing clients Configuring the product to enable new features General stuff we’re working on 33
- 43. Key features Tech impl Demo Day vids Code Name Screenshots
- 44. One line. Just the important stuff
- 45. Release dates Intentionally vague
- 48. Pushing the envelope Agile Enterprise-level long-term capacity planning: 38
- 49. Agenda Background – balancing client input, product vision and engineering capacity in a challenging environment Approach – balancing strategy and opportunity, empowerment Process – requirement model, token system Tools – live roadmap, awesome release notes 39
Hinweis der Redaktion
- Hi, my name’s Itamar, I do product management operations at opower, at let me jump straight to the problem that we’re trying to solve at OPOWER, which is
- This one. This is not a problem that’s unique for OPOWER, but is aggravated in our case. Every Product Manager has to be smart about balancing client input, product vision and engineering capacity. And if you don’t do a smart job about it – this is exactly what happens – you product vision gets crushed. Let me show you why that is, and the creative ways we chose in order to address it
- I’m planning on talking about all of these cool JIRA and Confluence features in my presentation today but before I do that I have to lay some ground work first
- Our Agenda for today is pretty straight forward: give you some background about OPOWER and the problem we’re trying to solve, talk about the approach that we took in order to solve it and then talk about the processes and tools that we put in place in order to do so
- As I mentioned To understand our challenge you have to know a little bit about us, so let me take just a couple of minutes to tell you. We’re an energy efficiency and smart grid software company, selling a multi-channel customer engagement platform leveraging cutting edge behavioral science. We enable our utility partners to better connect with their customers, motivate reductions in energy use and increase program participation and overall customer satisfaction rates. We currently work with over 50 utilities reaching more than 10M homes. In 2010 we helped consumers save about 150gwh of energy which is equal to the entire energy consumption of Belize and in 2011 we’re shooting for helping consumers save about 400gwh of energy, the equivalent of taking Niger completely off the map ,and if you want to leave the geography quiz for later, it’s also 20% of RI’s energy consumption. And that’s exactly why we’re so excited about solving the challenges: in our case, smart product management equals more kwh saved. From a product development perspective, we do Agile software development which we deliver using a SaaS business model. And we all love SaaS and Agile, right? Well, what I want to show you is that in our case, the answer is both “yes” and “no”
- SaaS is a great model because if I run all my clients on the same system in the same place, I get a lot benefits, here are a few:»When two clients are asking for the same thing – we only need to build it once. We don’t have to do anything special to make sure that it’s available for both. No duplication is required »We benefit from having different clients using the same system b/c we can leverage learning from one client and apply them for all other clients. » We can easily collect large chunks of data from multiple sources - creating a positive network effect – the more data I have, the better the analysis I can generate, and everybody benefits. »And I can have a nimbler support structure: I don’t have to deal with multiple live versions of my product - I can just deploy trunk to everybody. All of these apply in our case and we want to take advantage of them. BUT. SaaS also poses unique challenges to us, because of the business environment that we operate in
- Our clients are relatively slow moving and often our solutions are part of larger IT projects that span several years.» we’re also talking about deals in the millions of dollars, so our clients tend to have slightlymore say than a typical SaaS client»Our sales cycles are long: about 8mo on average and can take longer than a year in extreme cases So we need to sell our product not as what it is today, but as what it would be 8-12 months down the road»And finally, we do most of our business through RFPs which are fixed time, fixed scope in nature – there’s not much that we can do about that, and it means that what we end up building is determined not only by our strategy, but also by specific demands that our clients make.»Some of the implications of all of these is that we want to be as agile as possible, but we need to have a much more disciplined LT planning process because our roadmap needs to have a much higher level of commitment around it, beyond just a “plan of intent”
- So lets take a look at some of the elements of the SaaS business model and agile development methodology that we find particularly challenging:SaaS is usually a WYSIWYP product - what you see is what you pay for: customers are planning on using the software as-is (atlassian is a prefect example), with only a vague sense of what future enhancements may be. »We can’t really do that since our sales cycles are too long»Product development process can be described simply as: collect a set of inputs, synthesize, figure out what you want to build, build it. The major constraint is only your capacity - how much you can build. » Since most of our business comes from responding to RFPs, we can’t just say something like:“we schedule features based on a variety of factors”, it would pretty much guarantee us losing the deal»Most SaaS companies have the benefit of not having to maintain an external roadmap and can have a rather high level of opacity when it comes to discussing future product plans and delivery dates. » We can’t. Our client demand better visibility into our roadmap due to the length of the contract (3+ years) and the level of strategic importance of it»And lastly, doing pure agile development: figuring out things at the last responsible moment and maintaining the equation that maximum flexibility equals maximum value. »Though still an aspirational goal to us, in our case, given everything that I just covered some parts of our roadmap will have very little flexibility around them
- I think that by now you should have a pretty good sense of the challenges that we’re trying. When we thought about how to tackle them, several options came to mind:
- one option was doing nothing. But, as I mentioned, we soon realized that this will decrease our chances of succeeding in this business so we decided to pass on that one.
- Another option was to try and make our customers more SaaS/agile-friendlyOne thing that we can do is try and influence regulation and policy that will allow our customers to be more agile»Trust is always key, and selling clients on our vision of the roadmap should reduce the number of odd requirements that they want, and we don’t really want to build»And sometimes, the barrier for them is purely mental, and evangelizing the benefits of the model and methodology can take us a long way.»These are all great. We do all of them. But they’re not enough
- We very soon realized that in order to succeed in this business we had to change and organize ourselves around the realities of our industryOne key insight that we gained is that we’re a SaaS company but: our clients are too different from one another to have a cookie-cutter approach for handling them; and it’s impossible to manage this variability completely within product management – therefore, » we have to put a process in place that will allow us to build things that clients demand and that we may not want to build at that time, or at all (in the worse case scenario) without letting it disrupt our more strategic product development – balancing strategy-driven and opportunity driven product development» and we have to build tools that will truly empower our account-specific experts and reduce their dependency on support from the product team
- Ok, we have an approach in place, a way to think about a solution to this problem, but things aren’t going to just solve themselves. We have to put in place processes and tools to make this a reality
- But before we started putting a process in place, we first had to overcome a language barrier: making sure that sales, product management and engineering are speaking the same language. We learned that people outside product development don’t care as much about individual user stories b/c they are too fine grained in most cases. So we had to expand our requirement model in a way that would allow us to talk about requirements of dramatically different scales. »It doesn’t make sense to talk about something that would take a few days to complete and something that would take several months to complete and call them both “user stories”.»A side effect, if you will, of having requirements of different scale is also different levels of confidence around their scope estimates. With user stories, we’re usually much more confident in our 1SP estimate than in our 64SP estimate. In our normal estimation process, we account for some of that change in uncertainty by using a fibonachy or a power of 2 scale but when the scale is that different, we need a different mechanism.Let me walk you through our requirement model to give you a better sense of what I’m talking about.
- Here is how it looks like at the very high level. I’ll use this as an opportunity to feature another plugin that we use: the gilffyplugin for confluence. Going left to right, at the highest level of planning we have “investment themes”, realized by something we call “initiatives”. Those in turn are realized by features, then it should start looking familiar to most of you: we have user stories and finally tasks
- Let’s talk about investment themes for a minute. They are not requirements per-se, they don’t go into a backlog or get estimated. They are a tool to ensure strategic alignment between our business objectives and the stuff that we build. At the highest level of planning, typically on an annual basis, we would decided that we want to dedicate, for example, 15% of our development capacity towards our “market expansion efforts”. Real requirements will later be mapped to those investment themes, using a JIRA custom field, so we can track “how what we’re building, maps to our strategic guidance”
- Initiatives are the highest level of requirements that we use. They are project-like in nature, but not exactly projects, since we do continuous product development. They are authored and decomposed by product managers and serve as the building blocks for our product roadmap. Engineering leads and Architects will typically be the ones estimating the scope of initiatives and they would do so using a unit we call DTI or Development Team Iteration. If an initiative is estimated at 1 DTI it means that we estimate that it would take a scrum team of 8, 1 iteration to complete.
- Below the initiative we have the most interesting requirement type: features. Since we started with the Mike Cohn model, if you will, we used to have something called “epics”. But epics tend to be rather poorly defined and we ran into some challenging anti-patterns with them: People would group an “iteration-worth of user stories” and call them an epic. Or a set of user stories needed to meet a client milestone and call that an epic. Or in the worth case, just group a bunch of unrelated user stories together and call them an epic, just to get a high level estimate for the.»So we decided to change the name to features, and refine the definition to a functional collection of user stories. Features are written and decomposed collaboratively by product managers and developers and we use them as the basic building block for our wave plan – the plan that details what we intend to build 2-3 iterations out. At OPOWER we release every iteration so there isn’t any release plan. In a particular iteration, a team may be working on user stories from different features at the same time. Leads will usually be the ones estimating features using story point ranges, 60-128 for example. The point is to convey the increased uncertainty in estimates in comparison to user stories.
- And then we have user stories. I’m assuming most of you are familiar with them so I won’t spend a lot of time talking about them. We’re pretty vanilla when it comes to user stories: the teams are the ones who estimate them and they are the basic building blocks in determining the iteration commitment and plan.
- Putting it all together, this is what we get. A set of requirement types, used in different planning horizons for different purposes using different estimation currencies to convey different confidence levels in the estimate.
- Let’s see how all of this really works using an example of the “rate analysis tool”.This initiative is aimed at helping customers figure out which rate plan would work best for them given what we know about the utility rate plans and their historic usage data.Some utilities have over 100 different rates which include: time-of-use components, peak days components and different charges, which makes this much more complicated that how it may initially sound.Below the initiative level, we have 4 different features: »the first is the minimum marketable feature; the second will allow customers to run a set of “hypotheticals”: if I’m willing to change my usage in way X, which will be the optimal rate for me then?”; the third is a feature that provides customers with further details on the different rates, estimates and other information; and lastly a feature for additional enhancements for things like more gracefully handling all sorts of unique edge cases.»Under each feature, we have a set of user stories. So under hypotheticals we have one for “willing to reduce the overall usage by x pct” for example, one for “willing to reduce usage only during peak days” as well as several others.
- Now we can start talking about process. And as I mentioned one of the biggest challenges that we are trying to overcome is focusing out product development around our product strategy, the things that we know will help drive energy efficiency and improve customer experience, and less about the things that we build opportunistically in order to win RFPs. In 2010 we could map about 25% of our product roadmap to strategic product initiatives that we were excited about, b/c we said “yes” to almost any crazy requirement that came through the door. Don’t get me wrong, it was the right thing to do at the time, but as we mature as a business, this is something that has to change. »In 2011 we want to be able to map about 85% of our product roadmap to strategic initiatives and this is something that will not happen on its own and we need to figure out a way to constrain the resources that are allocated towards opportunistic product development
- This is where the token system comes into place as a way empower our sales execs to make critical product decisions in a non-disruptive way.During our annual planning we have a unique investment theme which is a budget of DTIs given to the sales team touse at their discretion. There’s some string attached to it, they can’t use it for everything and it also can’t go negative.When a new RFP comes in, a lead PM is assigned to it and she helps decompose it into a set of product requirements. She then triages requirements, separating them into requirements that already exist or are on the roadmap and the timing works; ones that we absolutely won’t do; and ones that would require tokens. The sales team determines which requirements they’re willing to pay tokens for and the product team commits to delivering them. Once this internal hand-shake is established the sales team can move forward and sign the SOW.BothJira and Confluence play a key role in facilitating that system
- Generally speaking, we have a very open and transparent company culture that is reflected in our very simple workflow (start progress, resolve issue, close issue) and rather promiscuous permissions scheme. But this is one part of the workflow that has to be locked tight. All I’m really talking about is that tiny part of the workflow that has a red circle around it
- We use transition screens to limit access to the sensitive fields and only allow members of a “can commit” group to transition issues through that stage.»The transition screens allows us to fill the critical information that’s unique to commitments: first, marking the jira issue as a commitment. Then distinguishing between commitments that are still being discussed (in RFP stage) and commitments that we’ve already committed to (in SOW stage). The token budget that the commitment is coming from: is it for a new or an existing client? The DTI estimate, and the due date – when it’s supposed to be delivered. Since we do agile development, we don’t populate the standard “due date” field for all other issues.
- Once we have a set of commitments in place, we want to make sure that we can easily track themWe do that through a confluence page utilizing the “run” macro (for some basic filtering) and the “jira issues” macro rendering a view with the fields that are unique to client commitments (due date, token budget, DTI, etc.)
- Let me share with you one case study on how we actually put the token system to use and what the result ended up being Several months ago, we received and RFP, from a company that I will not name. The total scope of the work was several millions of dollars that I think you’ll all agree is a lot of money. Clearly a case in which you want to make the extra effort to win the deal, but you want to be smart about it and not say yes for everything. » Because in this particular case, saying yes to everything meant committing to about 25 DTIs of custom work or about 15% of our development capacity for a period of 18 months. Which is a very steep price to pay and meant giving up on strategic initiatives that we were very excited about. » So what did the token system give us? First, it facilitated very productive discussions around what’s more and less important, trying to come up with creative solutions for requirements that may be done without making changes to the product. » It made us make tough compromises which led to some very intense discussions, both internally and externally with the client » but finally, and perhaps most importantly, it enabled us to commit to doing work at a scope of about 10 DTIs or 60% less that what we would have committed to, had we not have the system in place. Now, this is really huge, we can build a lot of awesome features that we’re really excited about in the 15 DTIs that we just freed up without losing a multi-million dollar deal!
- So the token system is one of the key process changes that we made in order to adapt to our unique market but it is not enough. We also need tool to help us empower our account specific experts and reduce their dependency on support from the product team.
- So we now have a good process in place, but we still need to build tools that allow us to leverage some of its benefits. One of those tools is our live roadmap. We created it for our Sales and Engagement Managers addressing their need of seeing what’s coming up on the product side in the next few quarters, communicate this to clients if necessary and provide us with their feedback when we go about flushing each one of these initiatives out. It tries to take this view in Green Hopper, which is very development/product management oriented, and turn it into this. » Something that people on the business side can actually use. And all it takes is a simple Ruby app using a MongoDB that pulls an RSS feed off jira to displays our roadmap in a very visual, interactive way:
- »Each line represents a release and shows the release #, release date and for past releases, the release notes (in confluence). »Each card represents an initiative, the title is at the top and » a bunch of metadata is at the bottom such as: what product components are involved, what teams will be working on it, the DTI estimate, etc. All of these are being pulled out of custom fields in JIRA that we created »If you click on a card, it flips and shows add’l details, such as: the description field in jira, lead pm and other tickets that are connected to it.»Finally, an account exec can filter the roadmap by any metadata field and see, for example, only initiatives that affect her client’s web-portal that are coming up in the next 3 releases.
- I mentioned out release notes a few slides ago and they are another useful tool for empowering our account specific experts. They come out every three weeks as a page in confluence we call the Prodcast, with a lot of attention not only to what is being communicated but also to how it’s being communicated. It covers 4 major areas: what got done in the last iteration? Is there any difference between what we planned on building and what we ended up building? What will get done in the next iteration (i.e. what the teams committed to delivering) and what’s in the pipeline for the next few iterations beyond that
- A big challenge is crafting release notes that appeal to multiple stakeholders who are reading them for different purposes:»Our sales execs – just want to keep track, in real time, of any major changes to the product»Our Engagement managers and technical project managers – want to set expectations with our existing customers on what’s changing and help them plan their UAT period»Out professional services engineers – want to know how they need to configure the product to enable new features and make sure that they run properly»And other stakeholders just want to get a general sense of the stuff we’re working on.
- Here’s how we’re addressing this challenge using our 3.5 release as an example: it starts with the Code name (in a drawing) and the key features – a very high level viewThen our PSEs can dive into the weeds using the special section for implementation and customization.We also videotape our demo day and make the videos available as part of the release notes so people can get a more visual sense for what got done.Finally for our EM and TPMs who need more information on the recent features, we include a more detailed description for each feature with links to the actual tickets (so you can also go even deeper), screenshots of the actual changes (where appropriate) or more detailed “how stuff works” wiki pages for more back-end oriented features
- We keep our commitment section very simple and straight forward, trying to answer the basic questions that our stakeholders care about: what portion of the product will be affected? what’s the change? why are we doing this? who’s the lead pm? and which clients will benefit from having this in place?. So people can always skim it and get a rough sense of what’s going on, but if they need the details, the link to the particular JIRA issue is there.
- Finally, our pipeline section calls out the next two release dates and the candidate backlog for them shown in a similar fashion to the commitment part.We don’t show a separate backlog to each of those releases in order to properly communicate the level of uncertainty behind this plan. This helps our EMs and TPMs by preventing them from making promises that will put us in a bind later on.
- These artifacts: product roadmap, release notes, exist in almost every product company. For us, taking them to the next level and continuing the push the envelope in that regards, has been tremendously helpful and addressing some of our key business challenges.
- At OPOWER we believe in always maintaining a “proud but dissatisfied” approach when it comes to the work that we do. So now we’re training our sights on a visual tool that drives a lot of the back-end roadmapping and planning the feeds the tools that you just saw: On the one hand, we have an initiative backlog, estimated in DTIs and on the other hand we have our hiring plan that yields how many DTIs we have available each iteration. We need a tool that will allow us to reconcile the two, and the only way existing tools allow us to do so is by breaking everything down to a very fine grained level: people, hours, etc. which doesn’t make a lot of sense. So when I’m talking about a real agile-enterprise long-term capacity planning tool, I mean a tool that focuses on: »iterations, not dates or hours; on teams, rather than individuals; and on DTIs (in our case) not story-point velocities or man hours
- So, final recapI talked about the problem we were trying to solve: balancing client demands, product vision and engineering capacity – something all of you are experiencing and tackling one way or the other. I Talked about the approach that we adopted to balancing strategy-driven and opportunity-driven product development, and the role that empowerment plays in that dynamics. The process that we put in place through the token system, and the supporting tools that we created: the live roadmap and the awesome release notes. As I mentioned, in 2011 OPOWER hopes to help consumer save 400 gwh of energy, and all the things that I described in this presentation will play a key role in helping us reach that goal.hAnd with that, I want to say thank you to all of you for taking the time and being here today and open this sessions for any questions from the crowd