CPMN meeting from April 2011

1. Market research without totally screwing it up Colin Millerchip 18 April 2011

2. Agenda Introduction Before you start Market segments Looking for problems General techniques Discovery-validation cycle Research tools (qualitative, quantitative) Risk management Financial modelling Questions?

3. Before we start… This presentation is about market research, primarily to discover new product / service opportunities Market research is not the only way to find such opportunities For eg, technology-driven innovation, customer funded development Apologies in advance if this is overly biased towards “mass market B2B” … and apologies that these are very verbose slides They’re intended to be a resource that can be used off-line

4. Introduction Research isn’t a well-defined path you walk along Each piece of research will need its own path If you follow a checklist, you still need to keep your brain engaged Discovery-validation cycle will help you find some of the “unknown unknowns” You’ll never get to 100% knowledge If you try to do this, you’re going to be way too late There are no guarantees – deal with it!

5. Before you start What are you trying to find out? What questions you want to answer Who you want to ask (market segment) Make sure to think about the business justification What will the people making the go/no-go decision need to know? For eg: What’s the channel to market? Do people tend to spend money in this area? Can we capture the opportunity (core competence)? How long do you have? Use common sense: plan the phases of your research, track progress against plan, modify as you learn more

7. Market segments (cont.) Good segmentation criteria have clear boundaries are easy to determine are easy to estimate in the general population don’t change much over time Example criteria Geographic Business demographic (industry, size, etc) Adopter categories Product usage Purchasing approaches

8. Looking for problems (sorry, this is a bit “product-y”) Pragmatic Marketing call this “the 5th P” The traditional “four P’s of marketing”: product, price, place, promotion We’re trying to find a problem to solve for people… … that’s sufficiently painful, pervasive, pressing (urgent), and profitable (for us to solve) Typically* people want to do one of 3 things: make more money; reduce costs (time ~= money); or reduce risks. Qu: what are you looking to do for people? http://www.pragmaticmarketing.com/srpm * … in B2B; this isn’t the case with B2C

9. General techniques Face-to-face meetings are invaluable It’s costly, but you must do it – NIHITO Listening not talking Don’t end up doing a pitch of your ideas You’ll learn more from -ve than +ve feedback Thank them for the praise, but encourage criticism Don’t tell customers that they’re wrong … even if they are You’re researching, not selling (if necessary, you can gentle correct their misunderstandings at the end)

10. General techniques (cont.) Contextualise ie, understand the context of the comments eg, their job title, level of experience, job title, size of team, size of company, etc Will help you understand results eg, testers are like X, developers are like Y Will help you find target market segments eg, small companies don’t have this problem

11. General techniques (cont.) Open-ended questions Leading questions Unprompted vs. prompted feedback (both are useful) How painful is the problem? Frequency of use case How ‘workaroundable’ Need vs. want Repeatability Try to ask the same things, in the same way Document it (if you need to) Write up customer interviews straight away Try to record interview (dictaphone)

12. General techniques (cont.) Selection Bias 3 segments within any market segment: customers; evaluators (know of you, haven’t bought); prospects (don’t know about you). Make sure you’re not missing a major segment in your research Much easier said than done for ‘prospects’… For eg, your poll shows that 100% of database users back up databases… but could it be that only database administrators answered the poll? People lie! People tend to be nice to you They love your product ideas… but won’t part with cash when you ship the product Qu: how can you test their propensity to spend money to solve their problem?

13. Discovery-validation cycle Start with qualitative research Customer interviews, open-ended, 2-way discussions Learn more about your research topic Move to quantitative research Polls/surveys, specific question / answers Gather statistically-valid data This phase isn’t pure validation, there’s still some discovery going on “If you want to know what people think and do, choose quant; If you want to know why they think and behave that way, choose qual”

14. Qualitative research tools Customer interviews Usability tests Forum research Meet people at events (user groups, conferences, etc) Feedback via sales, support careful with this… … etc etc…

15. Quantitative research tools: polls Normally (but not always), you want the answers to be mutually exclusive, and complete (ie, cover the total range of responses) Make then engaging – for eg, sacrifice perfect response data for amusement Yes and No isn’t enough: what about “Don’t know”, or “What’s this?” or “Other” Incentivise people: less critical than for surveys (minimal time commitment), sharing current results is normally enough I’ve used: http://www.polldaddy.com/

16. Quantitative research tools: surveys Keep the survey fairly short Resist temptation to ask about secondary issues Find out about the respondent Demographics, company, experience Get their permission to follow up (“if you want to enter the prize draw, or are happy to be contacted about your responses, please leave your email address here”) Interesting questions “Spend 100 points” Forced choices (“pick one”; ranking (beware this can annoy people)) Incentivise people: freebees, prize draw, share results I’ve used: http://www.surveymonkey.com/ http://www.questionpro.com/

17. Quantitative research tools: gotcha’s Careful with your wording! Test it out before going live Monitor initial early responses, be prepared to change survey if required (eg, loads of people select “Other” option to a question) Use a web re-director can be helpful here, since not all changes can be made to an active/open survey Think about how you’ll react to various findings If survey response data to a question suggests X, will you believe it? Or will you doubt that people understood the question / answered it properly?

18. Other research tools Google adwords Alexa / compete.com traffic ranking? Question over whether these are biased Forum traffic stats (ours, competitors) Social networking sites Customer feedback mechanisms Usage data from your service / product (eg, download stats; quote / invoice stats; “dial home” pings; etc) … etc etc…

19. Analysts data Can cut down leg work Beware of… “Hockey-stick” curves (especially ones that are continually adjusted) Correlation between vendor spend with analysts, and their ranking in analyst’s “magic quadrant”

20. Risk management You can’t wait until you know for sure Identify the risks – technical, commercial, personnel, etc Try to quantify these Likelihood / Frequency Consequences Cost to mitigate Put in place a mitigation plan

21. Financial modelling FITA* may not be enough Financial modelling, & related jargon Monte Carlo simulation / PDFs (eg, RiskAMP, @RISK) Fully loaded costs, NPV, IRR, interest rate, cash flow forecast, revenue model, Tornado chart, spider chart Are you scared yet? :o) Can drive the model backwards “What revenue do I need to achieve to get my money back in n years?” * Finger in the air

22. Questions ? colin@millerchip.org @millerchip