This document discusses using MLFlow to train machine learning models and Seldon to deploy them in a Kubernetes environment. It provides an example of using a wine quality dataset to train two ElasticNet regression models with MLFlow and deploy them for an A/B test using Seldon. Key steps covered include tracking experiments and hyperparameters with MLFlow, defining the model interface with an MLproject file, and creating the inference graph in Seldon to route traffic between the two models and provide a feedback loop.
11. Keep track of experiments (e.g. versioning).
Optimise hyperparameters at scale.
Share models with the rest of the team.
Plain notebooks may not be the best solution!!
Training models at scale is hard
secret-project
├── eda.ipynb
├── requirements.txt
├── requirements-tf2.txt
└── experiments
├── attention-embedd.ipynb
├── attn.ipynb
├── hmm.ipynb
├── LSTM-2.ipynb
└── lstm.ipynb
12. MLtracking
API to track results and hyperparameters.
These can be stored remotely.
MLproject
Define environment, parameters and model’s interface.
Hyperparameter optimisation can be run Databricks or Kubernetes.
MLmodel
Snapshot/version of the model.
MLFlow can help!
13. MLFlow can help!
MLtracking
Track both hyperparameters:
Track metrics:
MLproject
Pip dependencies and entry point template.
MLmodel
Snapshot of regression coefficients and hyperparameters:
14. MLproject file and training
MLproject
name: mlflow-talk
conda_env: conda.yaml
entry_points:
main:
parameters:
alpha: float
l1_ratio: {type: float, default: 0.1}
command: "python train.py {alpha} {l1_ratio}"
$ mlflow run ./training -P alpha=0.5
$ mlflow run ./training -P alpha=1.0
20. Cloud Native
On-prem
Built on top of Kubernetes Cloud Native APIs.
All major cloud providers are supported.
On-prem providers such as OpenShift are also
supported.
https://docs.seldon.io/projects/seldon-core/en/latest/examples/notebooks.html
21. Seldon Architecture
Data scientists,
engineers and
managers
Deployment Controller
(kubectl, CI/CD, Seldon Deploy)
Business
Applications REST API or gRPC
Kubernetes clusters
running Seldon Core
Inference
Components
Kubernetes
API
Operator
Ingress
(Ambassador, Istio)
Public docker
registry
Client docker
registry
Seldon docker
registry
MLFlow models
(S3, GCS, etc.)