- Model building is only 10% of an ML Product, the rest is good Product Management. Focus on building good tracking and testing. Models will get better. - ML Product Roadmaps look very different. It may take 2-3 years to get true gain (test vs actual accuracy). Keep managing stakeholder misinformation. Educating is part of the job. - ML is more than a feature in your app. Relook at the whole canvas. You'll need expertise beyond data; Re-think tech design, UX and most importantly Business strategy.

1. www.productschool.com
Effective Tips for Building ML
Products by Rally Health Lead PM

2. Join 40,000+
Product Managers on
Free Resources
Discover great job
opportunities
Job Portal
prdct.school/PSJobPortalprdct.school/events-slack

3. CERTIFICATES
Your Product Management Certificate Path
Product Leadership
Certificate™
Full Stack Product
Management Certificate™
Product Management
Certificate™
20 HOURS40 HOURS40 HOURS

4. Corporate
Training
Level up your team’s Product
Management skills

5. Huzaifa Sial
T O N I G H T ’ S S P E A K E R

6. Disclaimers
(Or context)

7. Tools you will need
Common Sense
+
Intuition

8. Tools you will need
Common Sense
(abstraction of reality)
+
Intuition
(experience)

9. What is Machine Learning good for
Essentially pattern recognition.
Especially good when patterns are too complex or too fuzzy.
Things beyond rules-sets and basic statistics.

10. What is Machine Learning good for
Classifying
Grouping
Predicting
Responding

11. HammerWrench

14. 2, 4, 6, __

15. X
O
X
O

16. X
O
X
O X

17. X
O
X
O X
O

18. What is Machine Learning good for
Classifying Grouping
Predicting Responding
2, 4, 6, __ X
O
X
O X
O

19. Rate patients on cognitive impairment and dementia
and as a measure of spatial dysfunction and neglect.

20. Needles in a haystack of supply, categorized to 1 of 13,000
possible product categories. 19 of which are other needles.

21. The best time-range, sequence, care-type and provider
combinations to help a person recovery from their specific surgery.

22. Read stacks of published literature to compute a meta-analysis of
the benefits of switching to a new surgical procedure.

23. Personalize health actions to each user based on their health
conditions, benefits and interests so the most interesting and
useful actions come first.

24. Tools you will need
Common Sense
(abstraction of reality)
+
Intuition
(experience)

25. Product
ML
Model
Merger UXData
KPIs
Problem worth solving Solution worth a value

26. Product
ML
Model
Merger UXData
Reports
Real world is a distribution with loong tails
Common Sense +1
Team
● PM
Problem Space
Problem worth solving Solution worth a value

27. Expression Predictor
There are no absolutes in guesses (Machine Learning). That’s not good.
Intuition +1
HammerWrench

28. Product
ML
Model
Rule-based
Engine
UXData
Reports
Always have a rule based engine alongside ML models.
Intuition +1
Roadmap
Rule engine
Team
● PM
● Architect
● Engineers
Problem Space
Problem worth solving Solution worth a value

29. Each problem is broken into sub problems
Common Sense +1

30. Product
ML
Ensembler
(x models)
Rule-based
Engine
UXData
Reports
Plan for multiple models, releases, version, ensembling etc.
Intuition +1
Roadmap
Rules engine
Model Mgmt
Complexity for Rules vs. Model Mgmt
system depends on what your
problem space has more of
Team
● PM
● Architect
● Engineers
● ML Architect
Problem Space
Problem worth solving Solution worth a value

31. A kid learns to read Fonts through exposure
Data > Algos. Getting data right is a life saver.
Common Sense +1

32. Product
Rule-based
Engine
UXData
Reports
Most Data is about the common cases
Intuition +1
Roadmap
Rules engine
Model Mgmt Data Features Models
ML
Ensembler
(x models)
Team
● PM
● Architect
● Engineers
● ML Architect
● Data Scientists
Problem Space
Problem worth solving Solution worth a value

33. ML UX vs. Traditional UX
ML driven interfaces have huge opportunities of UX Improvement. Spend the time here.
Common Sense +1

34. Product
Rule-based
Engine
ML UXData
Reports
This is a multiplier to all your metrics. Get that low-hanging fruit!
Intuition +1
Roadmap
Rules engine
Model Mgmt Data Features Models
ML
Ensembler
(x models)
Team
● PM
● Architect
● Engineers
● ML Architect
● Data Scientists
● UX Designers
UX A/B + Actuals
Problem Space
Problem worth solving Solution worth a value

35. Lab vs. Real Life
What works in the lab will likely drop in real-life. Real life is a lot more chaotic and your data and features can’t map it all.
Intuition +1

36. Product
A/B Testing + Actual User Feedback
Rule-based
Engine
ML UXData
Reports
This may be costly and complicated. But this the compass for your journey
Intuition +1
Roadmap
Rules engine
Model Mgmt Data Features Models
ML
Ensembler
(x models)
Team
● PM
● Architect
● Engineers
● ML Architect
● Data Scientists
● UX Designers
UX A/B + Actuals
Problem Space
Problem worth solving Solution worth a value

37. Product
A/B Testing + Actual User Feedback
Rule-based
Engine
ML UXData
Reports
Start collecting data for future features / models. It takes a year.
Intuition +1
Roadmap
Rules engine
Model Mgmt Data Features Models
ML
Ensembler
(x models)
Team
● PM
● Architect
● Engineers
● ML Architect
● Data Scientists
● UX Designers
UX A/B + Actuals Future cases
Problem Space
Problem worth solving Solution worth a value

38. ML Gains
37x
10x
0 1 2 3
Eng work
Gains/Value
True gains will take a couple years. Year 1 is infrastructure and faking it. Year 2 is focused on Models. Year 3 is shooting for maturity.
Intuition +1

39. Product
ML UX
Reports
Data
Data
A/B Testing + Actual User Feedback
Rule-based
Engine
ML UXData
Reports
Roadmap
Rules engine
Model Mgmt Data Features Models
ML
Ensembler
(x models)
Team
● PM
● Architect
● Engineers
● ML Architect
● Data Scientists
● UX Designers
UX A/B + Actuals Future cases
Focus on building Year 1. Focus on Models Year 2. Maturity Year 3.
Intuition +1
Problem Space
Problem worth solving Solution worth a value

40. Business Side
● ML is often over-prescribed as solution
● ML solutions are often expected unrealistic results
● Expectations are often vaporware driven
● Capabilities are often over marketed
● Goals are often playing catch-up with public spin
● There is a lot of Naysayers
● With ML, data drives possibilities just as much as aspiration
○ From Product driven -> Data driven
● Attribution is hard on efforts. Models may help immensely, may not.

41. Education is part of the job.
Product
ML UX
Reports
Data
Data
A/B Testing + Actual User Feedback
Problem worth solving Solution worth a value
Rule-based
Engine
ML UXData
Reports
Roadmap
Rules engine
Model Mgmt Data Features Models
ML
Ensembler
(x models)
Team
● PM
● Architect
● Engineers
● ML Architect
● Data Scientists
● UX Designers
UX A/B + Actuals Future cases
Create a tons of shareable content. Education is a necessary part of your job.
Intuition +1
Problem Space

42. Redo Canvas when designing ML or bringin ML in Products
Quality of Action vs. Action. Get paid to perform, not do.
Intuition +1

43. Common Sense Lessons
● Real world is a distribution with long tails
● There are no absolutes in guesses (ML)
● Data > Algos. Getting data right is a life saver
● ML driven interfaces = huge UX opportunities
● A lab is an ideal subset of real-life
● Data is the fuel for solutioning
● True gains will take a couple years
● ML is exciting and confusing
● ML is predictive
Intuition Lessons
● Always have a rule based engine alongside ML models
● Plan for multiple models, releases, version, ensembling etc
● Most Data is about the common cases, hence the models are too
● This is a multiplier to all your metrics. Get that low-hanging fruit!
● Real-life = easy 10% drop. Manage expectations and goals
● Start collecting data for future features / models. It takes a year
● Focus on building Year 1. Focus on Models Year 2. Maturity Year 3
● Education is part of the job. Don’t let excitement set goals
● Get paid for performance not actions

44. Thanks for listening!
Huzaifa Sial
me@hsial.com
calendly.com/hsial
linkedin.com/in/hsial

45. www.productschool.com
Part-time Product Management Training Courses
and
Corporate Training