This document summarizes a lecture on agile customer research design. It outlines the stages and milestones of qualitative and quantitative research, including secondary research, expert interviews, customer conversations, survey programming, pretesting and data collection. It emphasizes having a Directly Responsible Individual and deadlines for each task. The research is meant to answer hypotheses and SMART goals in order to inform the product vision and next steps.

1. Dawn of the Data Age Lecture Series
Interpreting Data Like a Pro

2. Hi. I’m Luciano Pesci…
Founder & CEO, EMPERITAS
● Team of economists and data scientists delivering bi-weekly Customer Lifetime Value
intelligence so our clients can beat their competitors for the best customers.
Founder & Director, Utah Community Research Group, Univ. of Utah
● Teach microeconomics, data science, applied research, & American economic history.
2

3. Today’s Lecture Outline
● Teach you an agile customer research design.
● Show you qualitative & quantitative research best practices.
● Explain how to use the research to achieve your product vision.
3

4. 4Agile Research Design

5. The 30k Foot View
● The agile customer research covered
today is part of a larger process:
○ Think Broad. Mine Deep. Explain Simply.
● It needs to be highly systematized.
○ Steps, Stages, & Milestones.
○ DRIs & Deadlines.
○ Meticulous file organization.
5

6. EXECUTION
Data Collection
Think Broad. Mine Deep. Explain Simply.*
6
ROADMAP
SMART Goals
ANALYSIS
Extracting Insights
REPORTING
Telling a Story
*Storytelling With Data: youtu.be/syjNCvGKPpQ
Today’s lecture focuses here.

7. Stages & Milestones
● The execution step of agile customer research can be divided into
two stages with their own milestones (based on tasks):
○ Qualitative Research
■ Secondary Research
■ Expert Interviews
■ Customer Conversations
○ Quantitative Research
■ Programming Survey
■ Pretest & Soft-Launch
■ Data Collection
7

8. 8
Program
Survey
Pretest &
Soft-Launch
Data
Collection
Secondary
Research
Expert
Interviews
Customer
Conversations
Two Research Stages*
QUALITATIVE QUANTITATIVE
Each of the research stages can be done in a week, but the milestones are
unequal. Customer conversations & data collection take the most time & effort.
*Sizes of each milestone indicates relative difficulty.

9. DRIs & Deadlines
● “If it’s everyone's job, it’s no one’s job.”*
● Assign a Directly Responsible Individual (DRI)
to each milestone/task and set hard
deadlines that are visible and enforced.
○ With these details you can create a research Gantt.
9
*Craig Freshley, “Everyone’s Job”: youtu.be/uXFDYAv40wg

10. Visualizing Agile Research Sprint
10
*Monthly Research Sprint Gantt: goo.gl/r4FPhq

11. PSA: File Organization
You’ll produce a large volume &
variety of files. A system is needed:
○ Foldering & subfoldering
○ Lexicon of critical terms
○ File naming convention
○ Access management rules
11

12. The 30 Second Rule
● The standard for organizing your
research related files is this:
○ If you can’t find something in 30 seconds
of searching you aren’t organized.
12

13. 13Qual & Quant Milestones

14. Secondary Research
● There’s a mountain of existing public
knowledge to help your research.
● Focus your attention on learning:
○ Market trends including your competitor’s
product, price, messaging, & channel mix
○ Customer chatter (reviews, social, etc)
○ Similar research that’s been done in the past
14
*Product Manager Secondary Research Sources: goo.gl/CY53fg

15. Expert Interviews
● It’s possible to greatly accelerate your
own knowledge by interviewing experts.
○ Some charge a fee (it’s usually worth it).
● Should be no more than 30-mins long and
narrowly focused on their niche expertise.
○ Ask if you can record it, talk about what you
discovered from your secondary research.
15

16. Customer Conversations*
● This should be a natural conversation
between two humans, not a survey.
● Prepare topics & questions, but probe
and be willing to go off script.**
○ A $50 incentive helps participation rates.
○ Solicitation takes a lot of time.
16
*Customer Conversations (00:09:05): youtu.be/CO_JlBDZgNs
**Product Manager Conversation Script: goo.gl/69YWVw

17. Programming The Survey*
● Surveys are a powerful tool for
generating custom quantitative data.
● Tools like Qualtrics** allow you to
personalize the survey experience.
○ This produces more (and better quality)
customer research data.
17
*Step Up Your Survey Research: youtu.be/WuBenXuVzqc
**Qualtrics Product Manager Survey: goo.gl/PExkms

18. Pretest & Soft-Launch
● NEVER send an untested survey to customers!
● Pretest the survey internally. When you “think”
it’s ready soft-launch to 20-30 customers.
○ Monitor the data and watch for issues, make
adjustments then fully launch the survey.
18

19. Data Collection (Fieldwork)
● With a good survey and a sufficient
customer list, you should be able to
collect data in a matter of days.
● Keep monitoring the survey for issues:
○ Read open-ended responses
○ Validate data coding is correct
○ Track time to complete & question drops
19

20. Another Way To Look At It
● With survey data in hand, the execution step
of the agile customer research process ends.
● More details about each of the qual & quant
milestones are laid out in my recent article.*
20
*Agile Customer Research For PMs: goo.gl/CCeZpb

21. 21Research-Driven Product Vision

22. Returning To Your SMART Goals
● Pause before diving into analysis.
● Revisit your SMART goals from
the earlier roadmap step of the
agile customer research process.
○ Analysis needs to answer your hypotheses.
22
*SMART Goals (00:01:15): youtu.be/VqMCK7Whyd4

23. Analytics Treasure Map
● The next step in the agile customer research
process is mining deeply with analytics.*
● “Mining” implies there’s value to be found in your
data. An analytics treasure map helps you find
evidence to answer your hypotheses.*
23
*Product Manager Analytics Treasure Map: goo.gl/p8XD9c

24. Storytelling With Data
● Explaining simply is the final step of
the agile customer research process.
● If you don’t get this step right all of
your hard work will be wasted!
○ You need a story and lots of visuals.*
24
*Visualizing Complex Data (00:08:50): youtu.be/JH-MCo-Xh5g

25. Next Step is Action
● With answers to your original
hypotheses, you’re ready to act.
○ What you’ll do with the answers was
also outlined in the SMART goals.
● Iterative waves of research will give
confidence for larger decisions.
25

26. 26Summing Up

27. What We Covered Today...
● The agile customer research design.
● Qualitative & quantitative research best practices.
● Using research to achieve your product vision.
27

28. Thursday March 15th, 2018
emperitas.com/lecture