1.
Kubeflow, MLFlow and beyond -
augmenting ML delivery
STEPAN PUSHKAREV
ILNUR GARIFULLIN

2.
Outline
1. Intro
2. Why yet another Flow
3. Kubeflow overview
4. Hydrosphere.io overview
Practice
1. Defining a pipeline and underlying worker containers
2. Run train-test-deploy ML pipeline with Kubeflow
3. Simulating production traffic
4. Monitoring data and domain drift
5. Closing the loop with automated re-training

3.
Why

4.
Machine Learning 5 years ago
Business Problem
Data
High hopes

5.
Then somebody opened a black box….
High hopes

6.
Machine Learning Workflow - whitening the box
1. Research
2. Data Preparation
3. Model Training
4. Model Cataloguing
5. Model Deployment
6. Model Integration Testing
7. Production Inferencing
8. Model Performance Monitoring
9. Model Maintenance

7.
f(x)
ML Workflow as a pure function
Immutable Raw
Dataset
ML Service in prod
Repeatable | Scalable | Observable
Machine Learning Operations

8.
What is Kubeflow?
● Began as Kubernetes template / blueprint for running Tensorflow
● Evolved into “Toolkit” - loosely coupled tools and blueprints for ML on
Kubernetes

9.
Kubeflow Pipelines - the very first original contribution
Main components:
1. Python SDK
2. UI
3. Orchestrator
4. ML Metadata Service
5. Argo under the hood

10.
Kubeflow Pipelines

11.
Kubeflow brings ML Awareness into CD
1. METADATA store and visualization components
2. Kubernetes native - enforcement of Docker-only pipelines
3. Strong believe in Kubernetes community
Cons:
1. Still Low level. Needs Ops team to make it work.
2. No ML Workflow adapters, domain abstractions

12.
Alternatives
1. Jenkins / Gitlab may close the gap with ML plugins
2. AirFlow may close the gap with ML Operators
3. AWS Step Functions - do it yourself
1. ML Flow - simple architecture, easier to start, local mode,
experiments tracking is better.
Could be run in Kube not not as native as Kubeflow.
Databricks / Azure community.

13.
What is Hydrosphere.io?
Hydrosphere.io is a platform for ML Models Management.
- Started as an open source Microservices native ML Serving
- An exact value-add “tool” - a part of the Kubeflow toolkit
- Now covers model Cataloguing, Deployment, Inferencing,
Monitoring and Maintenance

14.
Hydrosphere.io

15.
Data Prep Training Cataloguing Deployment Integration
Testing
Production
Inferencing
Performance
Monitoring
Model
Maintenance
Tools Landscape
Orchestrate

16.
The Code
https://github.com/Hydrospheredata/workshop

17.
Workshop plan - Part 1 - Forward path
Data
Prep
Model
Training
Model
Release
Deploy to
Stage
Integration
Testing
Deploy to Prod

18.
Class plan - Part 2 - Production Flow
Simulate
Prod Load
Production
Inferencing
Accuracy
Monitoring
Performance
Monitoring
Predictions
Tracing

19.
Class plan - Part 3 - Maintenance Flow
Data
Prep
Model
Training
Model
Release
Deploy to
Stage
Integration
Testing
Deploy to Prod Production
Inferencing
Production
Metrics
feedback loop

20.
Practice

21.
Next Steps
TRY
hydrosphere.io
kubeflow.org
github.com/Hydrospheredata/workshop
ASK QUESTIONS
spushkarev@hydrosphere.io
info@hydrosphere.io
https://gitter.im/Hydrospheredata/hydro-serving

22.
Contact Us
GENERAL INQUIRIES
hydrosphere.io
info@hydrosphere.io
linkedin.com/company/hydrospherebigdata
twitter.com/hydrospheredata
facebook.com/hydrosphere.io
ADDRESS
125 University Avenue, Suite 290
Palo Alto, CA, 94301
tel: 650-521-7875
BUSINESS AND TECHNICAL
Stepan Pushkarev
spushkarev@hydrosphere.io
Ilnur Garifullin
igarifullin@provectus.com