1. Product Optimization
Webinar 14-July-2016

2. VENKATRAMAN RM
Director of Products OLA CABS
Experience:
Manager, Product Management | PayPal
Product Marketing Manager | Synaptris
Senior Product Manager | Info Edge India
Ltd

3. WHAT WOULD YOU TAKE AWAY TODAY?
• PM Role Aspirants
– Understanding Product Management processes pertaining to
important area in Product Life cycle – Product Optimization
• PMs
– Framework to structure your backlog and roadmap
– Comprehensive way to approach optimization
In-depth learning of concepts like creating backlog,
A/B testing, Customer Driven Innovation etc
NOT IN TODAY’S SCOPE

4. TYPICAL PRODUCT ROADMAP POST
MVP LAUNCH
MVP DESCOPED
ITEMS
GROWTH OPTIMIZATION
TECH STACK
IMPROVEMENTS

5. WHAT IS PRODUCT OPTIMIZATION?
• Incremental improvements to the product to
– Deliver more of measurable value to your customers
– Improve Core (and/or Secondary) Product metrics
THESE ARE NOT CONSIDERED OPTIMIZATION
 Product Flow Revamps
 Big UI Refreshes
 Product extensions

6. TYPICAL PRODUCT LIFE CYCLE
Optimization
< 50%
Optimization
> 70%
Optimization
< 30%

7. STEP 0 – DEFINE YOUR OPTIMIZATION
THEME(S)
Agree on focus metric(s)
• Conversion Rate
• Transaction cost %
• Improve User experience
All themes to be quantified by a metric
• Abstract tracks such as ‘Improve User experience’ can be quantified by
parameters such as ‘Net Promoter Score’ or ‘Feedback score’
Set Targets and challenge yourself and the TEAM
• E.g. Conversion rate improvement by 1% this quarter
• Targets help keep focus ON

8. STEP 1: DATA, DATA, DATA
• What data you need?
– Baseline?
– Funnel?
– Drill downs?
• Validate Data for integrity
• Gaps?
– Prioritize instrumentation stories & bug fixes to roadmap

9. Product Optimization Cycle
Analyze
Identify
Size
Hypothesiz
e
Prioritize
Build
Test
Learn
* Data Integrity
* Go to your Customer
Go back to
Customer
* Pre-post
* A/B Test
Validation
Effort-Confidence-Impact

10. Example Analysis – Conversion Funnel (View Item
to Place Order)
100%
80%
70%
View Item
Add to Cart
Select/Add shipping
address
Payment options
58%
45%
Pay
39% Order confirmed
Did not like the item?
Enough info not available to make a
choice?
Did user divert to different page to
discover more things?
Cant find cart after diversion?
Guest vs. Member?
Issue with Enter address page?
Didn’t find payment method of
choice?
Not a serious buyer?
Integration issues with payment
provider?
Buyer Card not working?

11. Questions
• Which is the biggest opportunity in the funnel?
• What are possible hypothesis why user dropped after viewing
an item?
• Which part of the funnel is easier to improve?
• Crashes & Errors?
• Can I create separate funnels for different types of users? For
different product categories?

12. SIMPLE PRIORITIZATION FRAMEWORK
Feature Impact (1-9) Effort (1-9) 1-
High; 9-Low
effort
Confidence
factor (1-9)
Net score
I*E*C
F1 8 8 7 448
F2 5 6 9 270
F3 6 3 5 90
F4 3 1 3 6
How about extending the above?
- Adding weights to each of the factors
Why not I + E + C instead of I*E*C?
When can ordering be compromised?
- Context variables and real issues

13. Measure
• Measure under similar
circumstances
• Measure in isolation from other
projects
• Phased rollout based on results
• Preparation time required for
every project/test
A/B Test
• Hard to control external factors
• Cannot separate out impact of
parallel projects
• Almost impossible to measure
small improvements
Pre/Post
measurement
Impacted User
segment Drill
down analysis

14. OPTIMIZATION AS BOTTOM UP INNOVATION
• Data can provide Deeper Insights which is otherwise
unavailable
• Engineers and QA teams know more than what you think –
Edge use cases, dead-end flows, bugs, vulnerabilities. TAP
THEIR KNOWLEDGE

15. RECAP
• How to define your Optimization Theme – Metric
focus
• Importance of having Data & Quality Data
• Optimization Cycle
• Prioritization Framework (Impact-Effort-Confidence
Factor)
• Measurement techniques - A/B Testing, Pre/Post

16. Questions?

17. Description
Level 1
Description
Level 2
Description
Level 3
Description
Level 4
Goal