How to Build Good Products Well: The Product Management Manual

How to Build Good Products
Well
The Product Management Manual
Geoff Charles
© Geoff Charles @ FinTechWorldTour.com

A Primer on
everything you
need to know
about Product
Management
In 132 slides...
1. Overview
2. Roles and responsibility
3. The Process:
a. Aim: Vision, Objectives, Key Results, Roadmap
b. Identify: Hypothesis, Design Thinking, Prioritization
c. Define: Product Spec, Aligning Stakeholders
d. Plan: Effort Estimate, Velocity, Capacity Planning
e. Execute: Agile, Scrum, Sprints, QA, Release Management
f. Measure: A/B Testing, Monitoring, Issue Management

Overview of Product
Development

Product Development Process in a nutshell

6 phases of Product Development
2. Identify
3. Define
4. Plan
5. Execute
6. Measure
1. Aim Define your company’s financial and strategic goals
Identify opportunities that would help you reach your goals
Define product hypothesis that would harness these opportunities
Plan how to deliver on these product hypothesis
Execute on the plan and build the product
Measure the impact of the hypothesis and learn

6 phases of Product Development
2. Identify
3. Define
4. Plan
5. Execute
6. Measure
1. Aim

Product Development Process Details
PhaseActivityDeliverable
2. Identify 3. Define 4. Plan 5. Execute 6. Measure
Goal Setting
Quarterly
Planning
Product Vision
OKRs
1. Aim
Ideation
Prioritization
High Level
Requirements
Roadmap
Customer
Research
Requirements
Gathering
Risk
Assessment
Business
Requirement
Document
Technical
Requirements
Design
Effort
Estimation
Velocity
Tracking
PMO plan
Staffing
PMO Kick Off
Scrum
Development &
Testing
Working
Software
Documentation
Release
Management
Data Collection
Monitoring
Impact Analysis
Health Report
Issue
Management
Key Questions

Product Management 101
Roles & Responsibilities

What do Product Managers do?
1. Customer centricity: They deeply understand customer needs
2. Strategy: They understand the business and think innovatively about how to solve
problems
3. Alignment & communication: They are the glue between the business, the
technology team, and the customer
4. Prioritization: They know how to make trade off decisions and keep teams focused
5. Execution: They can execute complex projects with cross functional teams to
deliver value

1. Customer Centricity: Obsess about the customer
PM
Support
Team
Customer
Research
B2B
Customer
Success
Sales
Marketing
Customer
Feedback
Tips:
1. Get input from customers
and through customer facing
teams
2. Bring the voice of the
customer inside the
organization to increase
empathy

2. Strategy: have a vision and roadmap
What products are we going to
build to increase business value?
What features does this product
need to have to move a specific
metric?
CEO
Head of Product
Product Manager
What is the market opportunity
we are going after?
Tip: Be transparent.
Publish your roadmap
publicly. Constantly
communicate.

3. Alignment and Communication: be the glue
Tips:
1. Make no excuses. Cover
any gaps.
2. Take responsibility without
authority.
3. Facilitate. Build bridges
between teams.

4. Prioritization: make tough calls
Tip: Less is more. Ruthlessly
prioritize. Learn to say no.
Explain why.

5. Execution: Move Metrics depending on stage
Tip: Show results by owning
and moving metrics

Product Management Skills Covered
Customer & Market Research
Vision & Roadmap Crafting
Hypothesis and business
value evaluation
Prioritization & Panning
Product Requirements &
Negotiations
Stakeholder Management &
Communication
Quality Assurance
Release & Monitoring
Sales & Marketing Support

Common pitfalls of Product Management
1. “The Project Manager” → focuses on managing projects instead of product
2. “The Firefighter” → spends time turning off fires instead of being strategic
3. “The Engineering Manager” → manages engineers instead of leveraging EM
4. “The Needy PM” → wants it all, can’t prioritize
5. “The Technical PM” → tells engineers how to build things instead of what to build
6. “The Black Box Decider” → makes decisions without input from others or data

1.Aim
Key Concepts Covered:
● Vision
● Objectives and Key Results
● Defining & Measuring Metrics

Step 1: Aim
Goal Setting
Quarterly
Planning
Product Vision
OKRs
1. Aim
Ideation
Prioritization
High Level
Requirements
Roadmap
Customer
Research
Requirements
Gathering
Risk
Assessment
Business
Requirement
Document
Technical
Requirements
Design
Effort
Estimation
Velocity
Tracking
PMO plan
Staffing
PMO Kick Off
Scrum
Development &
Testing
Working
Software
Documentation
Release
Management
Data Collection
Monitoring
Impact Analysis
Health Report
Issue
Management
Key Questions

“Be stubborn on
vision but flexible
on details”
- Jeff Bezos
Metrics
Hypothesis
Project
execution
Results
Company goals
Order of company
alignment
Vision

What makes a good vision?
● Short
● Clear
● Specific
● Achievable
● Inspirational

Can you guess which company?
"To accelerate the world’s transition to sustainable energy."
“To connect the world’s professionals to make them more productive and successful.”
"To organize the world's information and make it universally accessible and useful."
“To refresh the world in mind, body and spirit. To inspire moments of optimism and happiness
through our brands and actions."
“To unlock the potential of human creativity—by giving a million creative artists the opportunity to
live off their art and billions of fans the opportunity to enjoy and be inspired by it."

Align on metrics
first, projects
second
“You can’t improve what you
can’t measure”
- Darren Hardy
Metrics
Hypothesis
Project
execution
Results
Company goals
Order of company
alignment

Leverage the OKR
process
Metrics
Hypothesis
Project
execution
Results
Company goals
OKR process:
What are our
team goals?Set yearly by
executive
team
Targets set
quarterly by cross
functional team

Objectives and Key Results (OKRs)
I will (Objective) as measured by (this set of Key Results).
Vision
Objective 1 Objective 2 Objective 3
Key Result 1
Key Result 2
Key Result 3
Project 1
Project 2
Project 3
Memorable qualitative
descriptions of what you
want to achieve.
Set of metrics that
measure your progress
towards the Objective.
Needs to be a number.
Hypothesis you want
to execute on to move
the metric

Tips on setting correct OKRs
1. Make it measurable. If KRs do not have numbers, they are not KRs.
2. Don’t prescribe. Define OKRs, not projects. Don’t reverse engineer KRs to fit
projects.
3. Set cross functionally, not at the team level. Organize teams around OKRs.
Cascade your objectives down to teams and individuals
4. Have stretch goals. Aim to achieve 70% of OKRs. Right balance between comfort
and being challenged.
5. Keep them top of mind. Come back to OKRs at every sprint / release.

Product
Management owns
metrics, not
projects
Metrics
Hypothesis
Project
execution
Results
Project
Management
Product
Management
Company goals
Scrum Team
Responsibility w/
Stakeholders© Geoff Charles @ FinTechWorldTour.com

Classic metrics to avoid:
● Number of features launched
● Number of story points completed
● Number of employees
● Number of lines of code
● Number of bugs
Pick the right metric
Right metric depends on business model
Company Metric Ideal frequency
Airbnb Bookings / stay Annually
Facebook Active users Daily
Ebay Gross Merchandise
Value
Daily
Uber Riders Weekly
Avoid vanity metrics
Focus on impact not output

Measure the metric
Operational Environment
Powers applications
Analytical Environment
Powers decision making
Extract,
Transform,
Load
Usage data
Business
Intelligence
Partner
Reporting
Financial
Accounting

2. Identify
● Ideation
● Hypothesis testing
● Prioritization
● Roadmapping

Step 2: Identify
Goal Setting
Quarterly
Planning
Product Vision
OKRs
1. Aim
Ideation
Prioritization
High Level
Requirements
Roadmap
Customer
Research
Requirements
Gathering
Risk
Assessment
Business
Requirement
Document
Technical
Requirements
Design
Effort
Estimation
Velocity
Tracking
PMO plan
Staffing
PMO Kick Off
Scrum
Development &
Testing
Working
Software
Documentation
Release
Management
Data Collection
Monitoring
Impact Analysis
Health Report
Issue
Management
Key Questions

1. Understand the
problem
“If I had only one hour to
save the world, I would
spend fifty-five minutes
defining the problem, and
only five minutes finding the
solution.”
- Albert Einstein
Metrics
Hypothesis
Project
execution
Results
Company goals
Ideation
process: “How
might we?”

Metrics
Hypothesis
Project
execution
Results
Company goals
Do not cross this line without:
1. Being aligned on the
metric
2. Deeply understanding
the problem
2. Hypothesize
once you
understood the
problem deeply

Hypothesize solutions leveraging design thinking
Common method:
1. Gather core group of
stakeholders
2. Recap problem
3. Diverge: individually
brainstorm ideas to solve the
problem
4. Capture ideas in 1 line on
sticky notes
5. Group ideas together in
themes
6. Converge on top ideas by
aligning on which move will
move the metric

3. Prioritize
“People think focus means
saying yes to the thing
you've got to focus on. But
that's not what it means at
all. It means saying no to the
hundred other good ideas
that there are. You have to
pick carefully.”
- Steve Jobs
Metrics
Hypothesis
Project
execution
Results
Company goals
Prioritization:
“What should
we do first?”

Input > Output → you need to prioritize
? ? ?
? ? ?
? ? ?
Bug!
New product idea!
New feature for my
customer!
Tech Debt!
Sales demo!
20 story points per
sprint
So many requests So much capacity

Deciding what not to build is as important as what to build
Learn to say “No”
? ? ?
? ? ?
? ? ?
? ?
? ?
The “No” Gate
Your
“Backlog”
Ideas
“No”
Not part of our
company goals
Not part of our
quarterly OKRs
Not aligned with
metrics we are
focusing on
Not properly
defined or
understood

Business Case Evaluation
? ? ?
? ? ?
? ? ?
Tips:
● Work with the business
teams!
● Market sizing
● Forecasting
● Benchmarking
$
$$$
$$ $
How much do we
estimate this will
move the metric?
? ?
? ?

Effort evaluation (high level t-shirt size)
? ? ?
? ? ?
? ? ?
? ?
? ? $
$$$
$$
$
$
$$$
$$ $
Is this feasible?
How much work
roughly is it?
Tips:
● This is not story
pointing, it’s high
level sizing
● XS, S, M, L, XL,
XXL
● Have tech lead
do this
● If you wait until
you have all
requirements
defined, it’s too
late

The ideal world
Effort
Business
Value
:|:)
:(:|

Break down big projects
Effort
Business
Value
:|:)
:(:|
Can you decrease
scope or
complexity?

Don’t just prioritize metric movers
Effort
Business
Value
:|:)
:(:|
● Metric Movers: These pay the
bills. If you don’t move metrics,
you will lose the ability to fund
yourself
● Customer or Employee Requests:
If you don’t listen to customers or
employees, they will hate you.
● Delights: If you don’t delight
customers, you won’t inspire
loyalty.
● Long term bets: If you don’t invest
in long term capabilities, you will© Geoff Charles @ FinTechWorldTour.com

Share priorities using a roadmap
● Simple
● Shared & Accessible
● Aligned
● Dynamic
● Accurate near term
● Aspirational long term

“What can I tell investors, the press and
board members?”
Roadmap at different levels
The CEO Roadmap
The Stakeholder
Roadmap
The Scrum Team
Roadmap
“What will we get to reach our goals?
“What will I need to build in the future?”
Increasing
detail

3. Define
3.1. Product Specs
● Product Specs
● Stakeholder input
● Testing design

Step 3: Define
Goal Setting
Quarterly
Planning
Product Vision
OKRs
1. Aim
Ideation
Prioritization
High Level
Requirements
Roadmap
Customer
Research
Requirements
Gathering
Risk
Assessment
Business
Requirement
Document
Technical
Requirements
Design
Effort
Estimation
Velocity
Tracking
PMO plan
Staffing
PMO Kick Off
Scrum
Development &
Testing
Working
Software
Documentation
Release
Management
Data Collection
Monitoring
Impact Analysis
Health Report
Issue
Management
Key Questions

5 reasons why Product Specs are critical
1. Explains why / business value
2. Contract with eng teams to build the right thing
3. Defines hypothesis to evaluate
4. De-risks projects by thinking ahead
5. Accelerates velocity

A good user story is:
Independent
Negotiable
Valuable
Estimable
Small
Testable
Guiding Principles
1. Focus on the user. Define the problem you are trying
to solve.
2. Define the what, not the how.
3. Be succinct.
4. Limit scope.
5. Keep it simple.
6. Get input.
7. Alway think: what could go wrong?

Get Input Early: Change Management
PM
Support &
Operations
Design
Engineering
Management
Legal &
Compliance
Business
Stakeholders
(Sales,
Marketing)
PMO
Aligned with feature functionality
to add value to the business
No business risks
Aligned with
customer experience
of the feature
Can support the feature once it
goes live (e.g. customer calls,
servicing, etc.)
Agree with the direction of the
product and can build the
capabilities needed
Ensure requirements
follow existing laws and
regulations with limited
risks
Defines and commits to deliver
and release the feature according
to a set plan

The press release!
Start from the press release
● Iterating on a press
release is a lot quicker
and less expensive than
iterating on the product
itself
● Ensures you understand
what impact you are
making
1. Heading — Name of product / feature
2. Subheading — One sentence summary of benefit
3. Summary — Summary of product and benefit.
4. Problem — Describe problem your product solves.
5. Solution — Describe how product solves problem.
6. Quote from You — A quote from a spokesperson
7. How to Get Started — Describe how easy it is to get
started.
8. Customer Quote — Provide a quote from a hypothetical
customer that describes how they experienced the
benefit.
9. Closing and Call to Action — Wrap it up and give
pointers where the reader should go next.

Content of a good product spec
1. User story: What feature do you want to build and for whom?
2. Business value: Why build this feature?
3. Acceptance Criteria: How is this feature meant to work?
4. Business Metrics: How do I know this feature is having an impact?
5. Data Model: What data should this feature generate?
6. Controls: How do I know this feature works after launch?
7. Testing: How do I ensure all of the above works before launch?

1. User Story
“As a ____, I want to ____ so that ____”
A user story has three parts:
1. Persona: what customer segment are you
solving for?
2. Action: what action do you want to
enable?
3. Outcome: what is the outcome of
enabling this action?
GGB: As a commuter from Marin, I want to drive
from Marin to SF in less than 1h, so that I can live
in Marin and work in SF.

GGB: Business value is to increase growth
rate of Marin county by 50%, which was low
given that Marin was only accessible by
Ferry.
2. Business Value
● What metric are you moving?
● By how much?
● How will this impact financials?
Company goals?

3. Acceptance Criteria
Written instructions for how the software
should behave, including the feature details
such as design wireframes, copy, flows, etc.
● What actions should occur
● What should be valid before
● What are the outcomes of each action or
set of actions (workflow)
● What are the error cases that can happen
for each action
GGB: “it must support a flow of up to 40,000 cars
/ hour on a day with wind up to 20mph. If wind is
higher than 20mph, it should close gates”

4. Business Metrics
1. The Business Metric
2. How to measure it
3. Its baseline value
4. If it is a metric you want to move: Its
target increase once the product launches
— this should be framed as a hypothesis
5. If it is a metric you want to monitor: Its
threshold after which you know the
product is not working (or being used) as
intended
GGB:
Number of people driving across the bridge in the
morning compared to using the Ferry → Output
Number of people living in Marin → Impact

GGB:
● Goal:
○ how many cars are on the bridge at
the same time
○ how many cars are serviced by the
bridge
● Data
○ Entrance of car (time of day)
○ Exit of car (time of day)
5. Data Model
Important for all stakeholders! Operations, Data
Science, Growth, Finance, Compliance, etc.
● Goal: what question do you want to
answer?
● Data you need to answer the question
○ Data Definition: Name of the field
and how it is defined
○ Data Values: Expected values in the
field
○ Data Validations: Rule sets that
must hold true for each field and
between fields

Controls are a set of rules that must not break
Controls can be detective or preventive.
● Detective controls are passive and alert
when an action happens that should not
have.
● Preventative controls prevent the action
from happening.
Controls should be operationally independent to
decrease risks in single point of failure.
6. Controls
GGB: Pressure
control system in
the pillars to close
gate if there is too
much weight on
the bridge.

3. Define
3.2. Aligning Stakeholders

Align top down
Metrics
Hypothesis
Project
execution
Results
Company goals
Order of
alignment

Weekly / Daily in stakeholder
meetings or offline
communication
Meeting frequency
Metrics
Hypothesis
Project
execution
Results
Company goals
Set yearly by executive team
Set quarterly by cross
functional team (including
PM!)
Weekly / Bi-Weekly in
stakeholder meetings or
brainstorming sessions
As soon as there are results!

Product vs. Business
Product Manager
Serves the customer
Owns product metrics
Builds features
Business Manager
Serves the business
Owns financial metrics
Sells features
Empowers
Advises

Relationships are built on trust
1. Build credibility - become expert on the product, customer, market
2. Empower & support teams - collaborate on key projects and listen
3. Be transparent - about progress and decision making
4. Deliver on commitments - don’t give them reasons to doubt you

Build credibility → Ex: Amplify
62
Hosts leading industry
conference!
Presents Product Management
as thought leaders
Publishes content that adds
value and generates leads
● Craft value proposition of product to the customer - case studies
● Focus on sales -- make it as easy as possible to sign up, demo, and onboard
● Define compelling roadmap with your sales teams and clients, and keep them updated

Empower & support teams → Ex: Affirm
63
Leads form
w/ FAQ
Case
studies per
industry
Business landing page with clear
value proposition

Empower & support teams → customer lifecycle
64
Sales Account Management
1 2 3 4 4 4
(+ Sales)
Product Management

Be transparent → Types of communication
Proactive - you initiate
Scheduled - you plan
Reactive - you receive
● Collect feedback: Give them a stake in your roadmap
● Share important: Risks? Wins? losses?
● Brainstorm: informal safe space to work together
● Current status: what’s currently being worked on?
● Next deliverables: what’s coming up?
● Retrospective: what’s going well? What can be improved?
● Take the time to assess: give SLA
● Assess: Which metric? Which problem? What assumptions?
● Modify roadmap OR decline & explain

Be transparent → Publish, Publish, Publish
Meeting notes (Square)
Centralized Project Status
Centralized Backlog &
Roadmap
Decision Log
Sprint Report
Sprint Retro
Release Log
(1/release)
Product Requirements
6 Page Memo (Amazon)

Be transparent → Over communicate
Frequency
Scope
Company
Individual
WeeklyDaily Monthly
1-1s
Slack rooms Sprint Demo
Sprint Report
Release blast
@productreleases
Scrum
Team
Cross
Functional
Team
Inception Meetings
Scrum
Quarterly
Email group updates
@product
Quarterly OKR review
& setting
Stakeholder OKR
Status Meetings

4. Plan
● Velocity
● Capacity
● Tips to increase velocity
● Planning in Agile

Step 4: Plan
Goal Setting
Quarterly
Planning
Product Vision
OKRs
1. Aim
Ideation
Prioritization
High Level
Requirements
Roadmap
Customer
Research
Requirements
Gathering
Risk
Assessment
Business
Requirement
Document
Technical
Requirements
Design
Effort
Estimation
Velocity
Tracking
PMO plan
Staffing
PMO Kick Off
Scrum
Development &
Testing
Working
Software
Documentation
Release
Management
Data Collection
Monitoring
Impact Analysis
Health Report
Issue
Management
Key Questions

Estimation
Principles
● Time building software > Time estimating and
planning
● 80/20 rule: effort invested in estimation leads to
minimal results after a certain point
● Debate is more important than accuracy. Involve
entire team. Team estimate, not specific to a
given developer.
● Story points (complexity) > Hours
Strategies
● Planning Poker
● Relative sizing

Velocity
● Velocity = previous velocity = what the
team delivered
● Capacity = velocity +/- change in team
size / vacations etc.
Velocity here is ~70
story points

8 Tips to improve velocity
1. Focus: Work on 1 thing at a time and drive
it to the finish. Reduce switching costs.
2. Less is more: Limit number of
simultaneous projects. Don’t cram too
much into a sprint. Prioritize ruthlessly.
3. Reduce dependencies: Ensure team is
autonomous. Focus on WIP.
4. Protect your time: protect individual
contributors from unnecessary meetings.
5. Align on norms: Align on rules for how
you work as a team
6. Commit: A sprint is a commitment.
Commit as a team and hold each other
accountable.
7. Keep is simple: Reduce complexity in
feature requirements and system design
as much as possible.
8. Learn continuously: Retrospect as a team
at end of every sprint. Never blame,
always seek to improve.

Project Management
Story
points
Time
1
3 2
What will get done
by <date>?
When will <feature>
get done?
Tip:
● Reduce scope first, don’t add
time.
● Let JIRA do this for you
● Update stakeholders regularly
Can I get <feature>
done by <date>?

5. Execute
5.1. Agile Process
● Agile
● Scrum
● Sprint Process
● Definition of Ready

Step 5: Execute
Goal Setting
Quarterly
Planning
Product Vision
OKRs
1. Aim
Ideation
Prioritization
High Level
Requirements
Roadmap
Customer
Research
Requirements
Gathering
Risk
Assessment
Business
Requirement
Document
Technical
Requirements
Design
Effort
Estimation
Velocity
Tracking
PMO plan
Staffing
PMO Kick Off
Scrum
Development &
Testing
Working
Software
Documentation
Release
Management
Data Collection
Monitoring
Impact Analysis
Health Report
Issue
Management
Key Questions

What is Agile?
Agile is a time boxed, iterative approach to
software delivery that builds software
incrementally from the start of the project,
instead of trying to deliver it all at once near
the end.
Agile Manifesto:
Individuals and interactions
over processes and tools
Working software
over comprehensive documentation
Customer collaboration
over contract negotiation
Responding to change
over following a plan

5 benefits with Agile Software Development
1. Shortening time to value: your customers only care about working software → 20% faster time
to market
2. Just in time: requirements get finalized right before starting projects → 20-80% more
productive, 30% decrease in cost
3. Continuous improvement: focus on launching small increments and iterating → 20-40% higher
customer satisfaction
4. Adaptive planning: requirements change over time because you learn → Flexibility in
responding to change
5. Continuous testing: testing is done in an automated fashion → Higher quality

Agile Process Summary
?
?
?
?
?
3
5
8
5
3
Sprint
Planning
Backlog
grooming
Sprint &
Scrum
Sprint
Review
Sprint
Retro
3
5
8
5
3
3
5
8
✓
✓
𐄂
Intake &
Prioritization
Release
Management

2 important gates: Definition of Ready & Done
Sprint
Planning
Backlog
grooming
Sprint &
Scrum
Sprint
Review
Sprint
Retro
Ticket should meet
Definition of Done
Definition of Ready =
Product Spec
?
?
?
?
?
?
3
5
8
5
3
1
3
5
8
5
3
1
3
5
8
✓
✓
𐄂

Definition of ready
Why does this matter?
● We build the right thing
● We build the thing right
● Common understanding
● Accurate estimates
● Accelates velocity
Tip: Don’t accept work that you have not
fully understood
Definition of ready:
❏ Product Specification Completed
❏ Stakeholders aligned
❏ Required sign offs completed
❏ Requirements understood
❏ Effort estimated
❏ Project plan crafted

5. Execute
5.2. Scrum Responsibilities

Scrum team: Independent and autonomous
Scrum team:
● Product Manager (1)
● Engineering Manager (1)
● Scrum Master (1)
● Engineers (5-10)
● Designers (1)
● DevOps (1)
Independent and
autonomous unit to
deliver value to the
customer

Agile responsibilities
Scrum Master
Responsible for the scrum
Product Manager
Responsible for the product
Engineering Manager
Responsible for the team
Individual Contributor
Responsible for the work

Product Manager
Responsibilities
● Understand customer needs
● Represent the customer
● Set vision for the product
● Define metrics
● Identify hypothesis
● Craft requirements / user story
● Prioritize / make trade-offs
● Communicate
● Build relationship with stakeholders
Pitfalls
● Define the what / why not the how
● Don’t manage the team, inspire them
● Over communicate to stakeholders &
solicit their input
● Less is more: keep team focused

Scrum Master
Responsibilities
● Project planning
● Execute sprints
● Facilitate meetings
● Update project information
● Identify blockers and unblock
● Improve process & tools continuously
● Assess project risks and mitigations
● Measure velocity
Pitfalls
● Include PM in key decision
● Solicit feedback from stakeholders
continuously
● Don’t obsess over process for process
sake

Engineering Manager
Responsibilities
● Ensure successful completion of
engineering work
● Manage the engineering team
● Input into priorities & hypothesis
● Staffing & assigning
● Resource planning
● Enforce best practices for quality
● Career development
● Ensure technical integrity with architect
● Manage technical debt
Tips
● Work closely with product managers to
align on technical and business roadmap
● Work closely with scrum master to
improve team efficiency
● Align individual career growth with
resource requirements
● Advocate for technical excellence in the
organization

Individual Contributor
Responsibilities
● Write working software
● Design solution
● Estimate effort
● Implement tests
● Code reviews
Tips
● IC is not only developers! Can include
designers, data scientists, etc.
● Protect your time - 80%+ no meetings!
● Work on 1 thing at a time
● Limit switching costs
● Ensure full review of specifications before
coding
● Test driven development!
● Pair programming

Other satellite scrum roles
● Architect: Leads technical
direction of systems
○ Functional designs
○ Technical norms and best practices
○ Design review & mentorship
○ Long term technical roadmap
● QA: Additional layer of testing to
ensure integrity of releases.
○ Provides engineering leverage in terms
of testing automation tools
○ Helps Product cover all edge cases of
the requirements
○ Executes exploritary testing
● Designers: Ensure high quality and
consistent user experience
○ Front end designs
○ Color schemes, font, assets, etc.
○ User research & usability testing
○ Prototyping
○ Interaction design
● Devops: Keep products running
○ Security
○ Deployments & hardware
○ Automation & developper productivity
○ Health & Monitoring

The team is more than a combination of roles
The scrum team is at its core one team. The
entire team is responsible for success and
failure
● Align on team norms
● Build trust
● Ensure open communication
● Continuously improve
○ Retrospectives
○ Post Mortem
○ 1-1s
○ Team building
○ Mentorship
○ Feedback

● Sprint planning
● Scrum
● Retro
● Demo
● Backlog grooming
5. Execute
5.3. Sprint Ceremonies

Overview of Scrum Ceremonies
Sprint Planning
What work needs to get done now?
Scrum
What work is being done?
Backlog Grooming
What is coming up?
Sprint Review
What work got done?
Sprint Retro
How did we do?

Typical 2 Week Sprint Cadence
Week 1 Week 2
M T W T F M T W T F
Sprint
Planning
(1-2h)
Scrum
(15mn)
Scrum
(15mn)
Scrum
(15mn)
Scrum
(15mn)
Scrum
(15mn)
Scrum
(15mn)
Scrum
(15mn)
Scrum
(15mn)
Scrum
(15mn)
Backlog
Grooming
(1-2h)
For next
sprint
Sprint
Review (1-
2h)
Sprint
Retro (1h)

Sprint Planning: What work needs to get done now?
Goal: Align team on objectives of a sprint and
ensure proper scope, understanding and focus
Logistics: every sprint of first day of sprint. 1-
2h (if 2 week sprints, then 2h). Led by PM. Full
scrum team present.
Runbook
1. Recap on OKRs / Roadmap
2. Recap on progress from last sprint
3. Define objective of sprint
4. Review velocity
5. Measure capacity
6. Go through tickets in scope
7. Commit to scope and objective
8. Assign first set of tickets
9. Start sprint
Tips: Ensure all tickets are groomed and meet
the definition of ready before entering sprint
planning. Get formal commitment from team.

Scrum: What work is being done?
Goal: Status update to the team to identify
blockers and ensure progress made
Logistics: Every morning for 15 minutes with
full team. Led by scrum master.
Runbook
1. Check burndown chart to measure
progress
2. Every member of the team updates
everyone else:
a. What did I do yesterday
b. What am I doing today
c. What are my impediments
3. Assign new tickets
4. Identify blockers or deeper dive
discussions needed
Tips:
- Dedicated space with video conferencing
to save time.
- Use the scrum board to give updates
- Save 15 minutes after scrum for follow
up conversations.

Keep your backlog sacred
Bad backlog
● Tickets with no descriptions
● Old tickets no one understands
● Tickets without any effort
estimates
● Not prioritized
● High priority items not understood
● Large tickets not broken down
Good backlog
● Prioritized
● Clear requirements
● Estimated
● Well broken down
● Understood
● Small, clear stories at the top
● Vague, large stories at the bottom

Backlog grooming: What is coming up?
Goal: Ensure understanding, feasibility and
estimate effort of potential work. Input into
prioritization.
Logistics: Full scrum team, every week or
other week for 1h. Every ticket that gets
presented in sprint planning should go through
backlog grooming. Not every ticket in
grooming is prioritized or will be worked on.
Runbook
1. Present tickets and go through
requirements in detail
2. Answer any questions on scope, value,
etc.
3. Get feedback on spec details from eng
team
4. Debate different approaches for how to
do this.
5. Estimate effort and debate if there are
disagreements
6. Breakdown of projects into digestible /
technical subtasks
Tips: Use a tag or status to indicated what
needs to be groomed

Sprint Review: What work got done?
Goal: Review outcome of sprint and get
feedback on features
Logistics: End of sprint meeting with full
scrum team + any relevant stakeholders (1h).
Led by scrum master + dev team showcasing
work to the product manager.
Runbook
1. Demo items that are done
2. Feedback from stakeholders
3. Align on acceptance criteria - determine
if ticket is closed or not
4. Update tickets / clean up board /
housekeeping
5. Close sprint
6. Recap on milestones
Tips: Have company wide demo of finalized
product features outside of sprint review.

Sprint Retrospective: How did we do?
Goal: Identify ways to improve as a team
Logistics: 1h at end of every sprint. Led by
scrum master. Entire team present.
Runbook
1. Go through last sprint’s action items and
get updates
2. Review sprint report
3. Go around the room sharing:
a. What have we done well this
sprint?
b. What can we improve?
4. Go through the list of improvements and
brainstorm solutions as a team
5. Align as a team on high priority solutions
and assign ownership
Tips:
- Take 5 minutes at the beginning for
everyone to write down their thoughts
- Share 1 thing per person and keep going
around until everyone finished
- Do not try to solve yet. Solve once
everyone had a chance to speak© Geoff Charles @ FinTechWorldTour.com

5. Execute
5.4. Quality Assurance
● Testing
● Automation
● BDD tests
● TDD

Quality is everyone’s responsibility
While there is no QA team in Agile...
● Quality is everyone’s responsibility → QA
embedded in team by engineers and PM
● Testing done throughout development
process, not simply at end → helps
increase velocity
● Separate QA team = more overhead, lead
time, delegation of critical responsibility
There is still Quality
● Invest in testing tooling and automation
for engineering
● Define standards for test coverage
● Collaborate on requirements and edge
cases

Common types of tests
Type Goal Automated? Responsible
Unit Test Make sure a function works Yes Engineer
Functional Test Make sure specific functionality works Yes Engineer w/ PM
Integration Test Make sure you don’t break other things Yes Engineer
End to End Test Make sure end to end feature is working Yes Engineer w/ PM
Smoke Test Make sure basic product features work Yes Engineer
Performance Test Make sure feature does not hurt performance Yes Engineer
Acceptance Test Make sure feature meets requirements No PM
Exploratory Testing Make sure broader product works No PM

BDD: Behavioral Driven Development
Tests behavior of the feature
● Given <context>
● When <action>
● Then <result>
Product Requirement → BDD tests
BDD tests should be automated
BDD tests should come from anyone
Example
● Requirement: Only users who log in for
the first time on a device should get a one
time password
● BDD tests:
○ Given user is using app for the first
time, when user logs in, then user
should get a one time password
○ Given user is using app for the
second time, when user logs in, then
user should not get a one time
password

TDD: Test Driven Development
Value:
● High code coverage & quality
● Higher velocity
Example: Password should have 7 length and at
least one number or symbol and capitalized
letter
● AAAAAAA → Fail
● aaaaaaa → Fail
● AAAAAA1 → Pass
● AAAAA1 →Fail
● aaaaaa1 → Fail
● aaaAaa! → Pass© Geoff Charles @ FinTechWorldTour.com

Testing & Release Management
Code Complete
& Tests Passing
Locally
Code Pushed &
Automated Tests
Passing
Staging
Environment for
Manual Testing
Any errors
Released to
Production
● Feature flags
● Beta release
● Shadow release
● Acceptance test
● Exploratory test
● Performance test
● Regression test
● Unit test
● Functional test
● Integration test

5. Execute
5.5. Release Management
● Definition of done
● Release cadence

Release Management: Definition of Done
Feature is done Pre Release Post Release
1. Acceptance Criteria is met
2. Unit test pass & coverage
3. Automated functional tests
4. No performance impact
5. Code reviewed
6. Documentation
7. Automated alerts & runbook
8. Product Owner accepts

Release Management: Pre Release
1. Release notes
2. Internal communication
3. Customer communication
4. Documentation for sales
5. Documentation for support
6. Training
7. Product metrics
8. DevOps deployment plan
9. Rollback

Release Management: Post Release
1. Release successful
2. Product metrics healthy
3. System metrics healthy
4. Hypothesis is validated
5. Customers are happy :)

Sprint Cadence ≠ Release Cadence
Feature A
Feature B
...
Sprint 1
Sprint 2
Sprint 3
Release 1
Release 2
How often should you release?
● Back end & B2C web →
release as frequently as
possible without risk (e.g. 2-3
/ week)
● B2C mobile → limit app
releases to 2-4/month given
need to upgrade / app
approval
● B2B → limit releases to 2-
4/month given need for
customer management /
training

6. Measure
6.1. Measuring Impact
● A/B testing
● Retrospecting
● Celebrating

Step 6: Measure
Goal Setting
Quarterly
Planning
Product Vision
OKRs
1. Aim
Ideation
Prioritization
High Level
Requirements
Roadmap
Customer
Research
Requirements
Gathering
Risk
Assessment
Business
Requirement
Document
Technical
Requirements
Design
Effort
Estimation
Velocity
Tracking
PMO plan
Staffing
PMO Kick Off
Scrum
Development &
Testing
Working
Software
Documentation
Release
Management
Data Collection
Monitoring
Impact Analysis
Health Report
Issue
Management
Key Questions

Measure
Metrics
Hypothesis
Project
execution
Results
Company goals
Measure: “Did
we move the
metric?”

Post release:
Did the metric move?
Was there statistical
significance?
Was my hypothesis correct?
Your opinion
does not
matter
(and our intuition is often
wrong)

A/B test everything
Key
Metric
Time
Yay, metric going
up!
Ship it!
What you think is happening What is actually happening
Key
Metric
Time
Test: not such
much
Control: things are
going up without
the launch

A/B testing tools
Google Optimize
Mixpanel
Optimizely
VWO

If you didn’t improve the metric…
Retrospect:
● Is our hypothesis wrong?
○ If so, was it due to a research
gap?
● Is our implementation wrong (e.g.
bad design / product / launch)?
● Is our experiment design wrong?
● Is our data wrong?
Learn and iterate: Determine what you will
do differently next time
The results are in...
If you improved the metric…
Celebrate your team & share broadly

6. Measure
6.2. Monitoring
● Dashboards
● Risk assessment
● Alerting

Maintaining functionality is just as important as
adding functionality
Google 20 years ago Google today

Strategy for monitoring
1. Understand: What is my product intended to do?
2. Measure: How do I know if my product is doing what it’s intended to do?
3. Assess: How well is my product doing what it is intended to do?
4. Act: How well do I respond when my product is not doing what it is
intended to do?

Bad vs. Good Product
Bad Product Good Product
Understand No understanding of how product works
No documentation
PM & Engineering team are experts
Great support documentation
Measure No metrics defined
No alerting
Limited visibility into how product is
performing / used
Metrics defined, implemented and visible
Automated controls are in place
Adequate testing pre and post launch
Tools to monitor system
User feedback
Assess Critical bugs
System performance
Poor user satisfaction
Not being used as intended
No critical bugs
Good performance
Good user experience
Act Issues not found quickly
Issues not fixed quickly
No progress made
Fast awareness & resolution of issues
Consistent retrospection / post mortems
Improvement over time© Geoff Charles @ FinTechWorldTour.com

Runbook for alerting & monitoring
1. Define the risk (e.g. users can’t log in)
2. Identify the measure (e.g. number of logins)
3. Define the threshold & time period (e.g. number of hourly logins
decreases by 50%)
4. Implement (e.g. BI tools like Chartio or Mixpanel or Google Analytics)
5. Learn & Improve (e.g. alert did not catch issue, tighten)

Look at metrics every day
Tips:
● Have different levels of
granularity
○ Hourly
○ Daily
○ Weekly
○ Monthly
● Cross functional teams
look at the same
dashboard (e.g. sales,
engineering, product)

Automate alerts with clear runbooks
Pick the right tool for the job Set the right thresholds
Define clear runbooks
● Ensure thresholds are tight enough to
catch issues but loose enough to not
be “noisy”
● Back test thresholds
● Invest in SRE model
● Distinguish warnings vs. critical failures
● Have clear debugging and resolutions
steps
● Define escalation paths

6. Measure
6.3. Issue Management
● Bug identification
● Bug triage

No
software is
perfect
1. Catch issues before releasing
2. Identify production issues quickly
3. Triage issues effectively and accurately
4. Communicate & escalate appropriately
5. Resolve high priority issues
6. Correctly balance quality and velocity

Bicycle instead of
scooter
Maintain healthy tension between forces
Speed
Quality
Scope
Scooter but will
break down
Good scooter but
will take time
● Manage technical debt
● Complexity kills
● Technology competitive
advantage
● Build the right thing
● Minimize scope to
what adds value
● Incrementally
improve
● Focus on outcomes
● Invest in automation
● Shorten time to market

The life of a bug
Bug identified Initial Triage
Issue
Management
Resolution
Bugs can come from
anywhere:
● Users
● Feature Testing
● Business
● Product
Management
● Monitoring
Need: strong
documentation of the bug
Initial triage answers:
● Reproducible?
● Material? (impact +
how many users)
● Workaround?
● Quick fix?
Need: resource to do
initial triage quickly
If it’s a real bug, PM
assesses priority:
● Business impact
● User impact
● Compliance / legal
impact
Need: prioritization &
escalation framework
Resolution depends on
the priority:
● P1 → immediate →
inject in sprint
● P2 → soon → next
sprint(s)
● P3 → no SLA → up
to team discretion
Need: SLA based on
priority© Geoff Charles @ FinTechWorldTour.com

Bug triage template
Category Description P3 P2 P1
Impact ● What is the customer impact?
● What is the business impact?
● What is the employee impact?
Low Medium High
Likelihood ● What is the likelihood this issue
happens again?
Low Medium High
Risk ● What are the legal or compliance
risks?
● What is the reputational risk?
Low Low Medium /
High
Mitigation ● Are there workarounds?
● Are there quick resolutions
available?
Yes Yes / No No

Recap

You got this!
Goal Setting
Quarterly
Planning
Product Vision
OKRs
1. Aim
Ideation
Prioritization
High Level
Requirements
Roadmap
Customer
Research
Requirements
Gathering
Risk
Assessment
Business
Requirement
Document
Technical
Requirements
Design
Effort
Estimation
Velocity
Tracking
PMO plan
Staffing
PMO Kick Off
Scrum
Development &
Testing
Working
Software
Documentation
Release
Management
Data Collection
Monitoring
Impact Analysis
Health Report
Issue
Management
Key Questions

● Product Manager Lead in Financial Technology
● 8+ years in financial services
○ Management Consultant at Oliver Wyman
○ B2B FinTech Nomis Solutions
○ B2C FinTech LendUp & Mission Lane
● Focus:
○ FinTech Strategy - unsecured lending
○ Product Management
○ Agile
○ Technology
● On sabbatical focusing on financial inclusion around the
world - fintechworldtour.com
About the Author - Geoff Charles

Thank you.

How to Build Good Products Well: The Product Management Manual

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie How to Build Good Products Well: The Product Management Manual

Ähnlich wie How to Build Good Products Well: The Product Management Manual (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

How to Build Good Products Well: The Product Management Manual

Hinweis der Redaktion