Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
End-to-End ML Pipelines
TFX + KubeFlow + Airflow
Chris Fregly
Founder @ .
Founder @ PipelineAI
Real-time Machine Learning and AI in Production
Former Databricks, Netflix
Apache Spark Contributor
O...
Advanced Spark and TensorFlow Meetup (Global, Monthly Events)
https://meetup.com/Advanced-Spark-and-TensorFlow-Meetup
Community Edition
https://community.pipeline.ai
1 OK with Command Line?
2 OK with Python?
3 OK with Linear Algebra?
Who are you?
4 OK with Docker?
6
5 OK with Jupyter Not...
Note #1 of 10
IGNORE WARNINGS & ERRORS
Everything will be OK!
Note #2 of 10
THERE IS A LOT OF MATERIAL HERE
Many opportunities to explore on your own.
Note #3 of 10
YOU HAVE YOUR OWN INSTANCE
16 CPU, 104 GB RAM, 200GB SSD
And access to TPUv3 Cluster
Note #4 of 10
DATASETS
Chicago Taxi Dataset
Fashion MNIST
(and various others)
Note #5 of 10
SOME NOTEBOOKS TAKE MINUTES
Please be patient.
We are using large datasets!
Note #6 of 10
QUESTIONS?
Post questions to Zoom chat or Q&A.
Antje and I will answer soon.
Antje Barth =>
Note #7 of 10
KUBEFLOW IS NOT A SILVER BULLET
There are still gaps in the pipeline.
(But the gaps are shrinking.)
Note #8 of 10
THIS IS NOT CLOUD DEPENDENT*
*Except for 2 small exceptions…
Patches are underway.
Note #9 of 10
PRIMARILY TENSORFLOW 1.x
TF 2.x is not yet fully supported by TFX
(We have a section on TF 2.x)
Note #10 of 10
SHUTDOWN EACH NOTEBOOK AFTER
We are using complex browser voo-doo.
System 6
System 5System 4
Training
At Scale
System 3
System 1
Data
Ingestion
Data
Analysis
Data
Transform
Data
Validation
...
1 Setup Environment with Kubernetes
TensorFlow Extended (TFX)
ML Pipelines with Airflow and KubeFlow
Agenda
Hyper-Paramete...
Bonus Extras!
6 TPUs
MLflow
RedisAI + TensorFlow + PyTorch
TensorFlow 2.0
TensorFlow Privacy Libraries
7
8
9
10
Hands On
00_Explore_Environment
1.1 Kubernetes
TensorFlow Extended (TFX)
Airflow ML Pipelines
1.0 Environment Overview
KubeFlow ML Pipelines
6
Hyper-Param...
1.1 Kubernetes
Kubernetes
NFS
Ceph
Cassandra
MySQL
Spark
Airflow
Tensorflow
Caffe
TF-Serving
Flask+Scikit
Operating system...
Hands On
01_Explore_Kubernetes_Cluster
1.2 TensorFlow Extended (TFX)
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evalute
Model
Deploy
Reprod...
1.3 Airflow ML Pipelines
1.4 KubeFlow ML Pipelines
1.5 Hyper-Parameter Tuning (Katib)
1.6 Prediction Traffic Routing (Istio)
1 Setup Environment with Kubernetes
TensorFlow Extended (TFX)
ML Pipelines with Airflow and KubeFlow
Agenda
Hyper-Paramete...
2.1 TFX Internals
2.0 TFX Components
6
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Dep...
2.1 TFX Internals
Driver/Publisher
Moves data to/from Metadata Store
Executor
Runs the Actual Processing Code
Metadata Sto...
2.2 TFX Libraries
2.2.1
TFX Components Use These Core TFX Libraries:
TensorFlow Data Validation (TFDV)
TensorFlow Transfor...
2.2.1 TFX Libraries - TFDV
TensorFlow Data Validation (TFDV)
Find Missing, Redundant & Important Features
Identify Feature...
Hands On
02_TensorFlow_Data_Validation
(TFDV)
2.2.2 TFX Libraries - TFT
TensorFlow Transform (TFT)
Preprocess `tf.Example` data with TensorFlow
Useful for data that req...
Hands On
03_TensorFlow_Transform
(TFT)
2.2.3 TFX Libraries - TFMA
TensorFlow Model Analysis (TFMA)
Analyze Model on Different Slices of Dataset
Track Metrics Ove...
Hands On
04_TensorFlow_Model_Analysis
(TFMA)
2.2.4 TFX Libraries – Metadata
TensorFlow Metadata (TFMD)
ML Metadata (MLMD)
Record and Retrieve Experiment Metadata
Artif...
2.3 TFX Components
ExampleGen
StatisticsGen
SchemaGen
ExampleValidator
Evaluator
Transform
ModelValidator
Trainer
Model Pu...
2.3.1 ExampleGen
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Training...
2.3.2 StatisticsGen
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Train...
2.3.3 SchemaGen
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Training
...
2.3.4 ExampleValidator
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Tr...
2.3.5 Transform
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Training
...
2.3.6 Trainer
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Training
Tr...
2.3.7 Evaluator
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Training
...
2.3.8 ModelValidator
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Trai...
2.3.9 Model Pusher (Deployer)
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Repro...
2.3.10 Slack Component (!!)
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reprodu...
1 Setup Environment with Kubernetes
TensorFlow Extended (TFX)
ML Pipelines with Airflow and KubeFlow
Agenda
Hyper-Paramete...
3.0 ML Pipelines with Airflow and KubeFlow
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model...
3.1 Airflow
6
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Training
Mo...
Hands On
05_Airflow_ML_Pipelines
(Chicago Taxi Dataset)
Hands On
06_Airflow_Feature_Analysis
Hands On
07_Airflow_Model_Analysis
3.2 KubeFlow
6
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
Training
P...
Hands On
08_Simple_KubeFlow_ML_Pipeline
Hands On
09_Advanced_KubeFlow_ML_Pipeline
(Chicago Taxi Dataset)
Hands On
10_Distributed_TensorFlow_Job
1 Setup Environment with Kubernetes
TensorFlow Extended (TFX)
ML Pipelines with Airflow and KubeFlow
Agenda
Hyper-Paramete...
4.0 Hyper-Parameter Tuning
6
Experiment
Single Optimization Run
Single Objective Function Across Runs
Contains Many Trials...
Hands On
11_Hyper_Parameter_Tuning
1 Setup Environment with Kubernetes
TensorFlow Extended (TFX)
ML Pipelines with Airflow and KubeFlow
Agenda
Hyper-Paramete...
5.0 Deploy Notebook as Job
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduc...
5.1 Create Docker Image
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduce
T...
5.2 Deploy Notebook as Job
Feature
Load
Feature
Analyze Feature
Transform
Model
Train
Model
Evaluate
Model
Deploy Reproduc...
Hands On
12_Deploy_Xgboost_Notebook
1 Setup Environment with Kubernetes
TensorFlow Extended (TFX)
ML Pipelines with TFX, Airflow, and KubeFlow
Agenda
Hyper-Pa...
Bonus Extras!
6 TPUs
MLflow
RedisAI + TensorFlow + PyTorch
TensorFlow 2.0
TensorFlow Privacy Libraries
7
8
9
10
Thank you!
https://pipeline.ai @cfregly @PipelineAI
Upcoming Hands-On Workshops
Nächste SlideShare
Wird geladen in …5
×

Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + Kubernetes + PyTorch + XGBoost + Airflow + MLflow + Spark + Jupyter + TPU

472 Aufrufe

Veröffentlicht am

Title
Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + Kubernetes + PyTorch + XGBoost + Airflow + MLflow + Spark + Jupyter + TPU

Video
https://youtu.be/vaB4IM6ySD0

Description
In this workshop, we build real-world machine learning pipelines using TensorFlow Extended (TFX), KubeFlow, and Airflow.

Described in the 2017 paper, TFX is used internally by thousands of Google data scientists and engineers across every major product line within Google.

KubeFlow is a modern, end-to-end pipeline orchestration framework that embraces the latest AI best practices including hyper-parameter tuning, distributed model training, and model tracking.

Airflow is the most-widely used pipeline orchestration framework in machine learning.

Pre-requisites
Modern browser - and that's it!
Every attendee will receive a cloud instance
Nothing will be installed on your local laptop
Everything can be downloaded at the end of the workshop

Location
Online Workshop

Agenda
1. Create a Kubernetes cluster
2. Install KubeFlow, Airflow, TFX, and Jupyter
3. Setup ML Training Pipelines with KubeFlow and Airflow
4. Transform Data with TFX Transform
5. Validate Training Data with TFX Data Validation
6. Train Models with Jupyter, Keras/TensorFlow 2.0, PyTorch, XGBoost, and KubeFlow
7. Run a Notebook Directly on Kubernetes Cluster with KubeFlow
8. Analyze Models using TFX Model Analysis and Jupyter
9. Perform Hyper-Parameter Tuning with KubeFlow
10. Select the Best Model using KubeFlow Experiment Tracking
11. Reproduce Model Training with TFX Metadata Store and Pachyderm
12. Deploy the Model to Production with TensorFlow Serving and Istio
13. Save and Download your Workspace

Key Takeaways
Attendees will gain experience training, analyzing, and serving real-world Keras/TensorFlow 2.0 models in production using model frameworks and open-source tools.

Related Links
1. PipelineAI Home: https://pipeline.ai
2. PipelineAI Community Edition: http://community.pipeline.ai
3. PipelineAI GitHub: https://github.com/PipelineAI/pipeline
4. Advanced Spark and TensorFlow Meetup (SF-based, Global Reach): https://www.meetup.com/Advanced-Spark-and-TensorFlow-Meetup
5. YouTube Videos: https://youtube.pipeline.ai
6. SlideShare Presentations: https://slideshare.pipeline.ai
7. Slack Support: https://joinslack.pipeline.ai
8. Web Support and Knowledge Base: https://support.pipeline.ai
9. Email Support: support@pipeline.ai

Veröffentlicht in: Software
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + Kubernetes + PyTorch + XGBoost + Airflow + MLflow + Spark + Jupyter + TPU

  1. 1. End-to-End ML Pipelines TFX + KubeFlow + Airflow Chris Fregly Founder @ .
  2. 2. Founder @ PipelineAI Real-time Machine Learning and AI in Production Former Databricks, Netflix Apache Spark Contributor O’Reilly Author High Performance TensorFlow in Production Meetup Organizer Advanced Spark and TensorFlow Meetup Who Am I? (@cfregly)
  3. 3. Advanced Spark and TensorFlow Meetup (Global, Monthly Events) https://meetup.com/Advanced-Spark-and-TensorFlow-Meetup
  4. 4. Community Edition https://community.pipeline.ai
  5. 5. 1 OK with Command Line? 2 OK with Python? 3 OK with Linear Algebra? Who are you? 4 OK with Docker? 6 5 OK with Jupyter Notebook?
  6. 6. Note #1 of 10 IGNORE WARNINGS & ERRORS Everything will be OK!
  7. 7. Note #2 of 10 THERE IS A LOT OF MATERIAL HERE Many opportunities to explore on your own.
  8. 8. Note #3 of 10 YOU HAVE YOUR OWN INSTANCE 16 CPU, 104 GB RAM, 200GB SSD And access to TPUv3 Cluster
  9. 9. Note #4 of 10 DATASETS Chicago Taxi Dataset Fashion MNIST (and various others)
  10. 10. Note #5 of 10 SOME NOTEBOOKS TAKE MINUTES Please be patient. We are using large datasets!
  11. 11. Note #6 of 10 QUESTIONS? Post questions to Zoom chat or Q&A. Antje and I will answer soon. Antje Barth =>
  12. 12. Note #7 of 10 KUBEFLOW IS NOT A SILVER BULLET There are still gaps in the pipeline. (But the gaps are shrinking.)
  13. 13. Note #8 of 10 THIS IS NOT CLOUD DEPENDENT* *Except for 2 small exceptions… Patches are underway.
  14. 14. Note #9 of 10 PRIMARILY TENSORFLOW 1.x TF 2.x is not yet fully supported by TFX (We have a section on TF 2.x)
  15. 15. Note #10 of 10 SHUTDOWN EACH NOTEBOOK AFTER We are using complex browser voo-doo.
  16. 16. System 6 System 5System 4 Training At Scale System 3 System 1 Data Ingestion Data Analysis Data Transform Data Validation System 2 Build Model Model Validation Serving Logging Monitoring Roll-out Data Splitting Ad-Hoc Training Why TFX and Why KubeFlow? Improve Training/Serving Consistency Unify Disparate Systems Manage Pipeline Complexity Improve Portability Wrangle Large Datasets Improve Model Quality Manage Versions Composability Distributed Training Configure
  17. 17. 1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  18. 18. Bonus Extras! 6 TPUs MLflow RedisAI + TensorFlow + PyTorch TensorFlow 2.0 TensorFlow Privacy Libraries 7 8 9 10
  19. 19. Hands On 00_Explore_Environment
  20. 20. 1.1 Kubernetes TensorFlow Extended (TFX) Airflow ML Pipelines 1.0 Environment Overview KubeFlow ML Pipelines 6 Hyper-Parameter Tuning (Katib) Prediction Traffic Router (Istio) 1.2 1.3 1.4 1.5 1.6
  21. 21. 1.1 Kubernetes Kubernetes NFS Ceph Cassandra MySQL Spark Airflow Tensorflow Caffe TF-Serving Flask+Scikit Operating system (Linux, Windows) CPU Memory DiskSSD GPU FPGA ASIC NIC Jupyter GCP AWS Azure On-prem Namespace Quota Logging Monitoring RBAC
  22. 22. Hands On 01_Explore_Kubernetes_Cluster
  23. 23. 1.2 TensorFlow Extended (TFX) Feature Load Feature Analyze Feature Transform Model Train Model Evalute Model Deploy Reproduce Training
  24. 24. 1.3 Airflow ML Pipelines
  25. 25. 1.4 KubeFlow ML Pipelines
  26. 26. 1.5 Hyper-Parameter Tuning (Katib)
  27. 27. 1.6 Prediction Traffic Routing (Istio)
  28. 28. 1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  29. 29. 2.1 TFX Internals 2.0 TFX Components 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training 2.2 TFX Libraries 2.2 TFX Components
  30. 30. 2.1 TFX Internals Driver/Publisher Moves data to/from Metadata Store Executor Runs the Actual Processing Code Metadata Store Artifact, execution, and lineage Info Track inputs & outputs of all components Stores training run including inputs & outputs Analysis, validation, and versioning resultsc Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  31. 31. 2.2 TFX Libraries 2.2.1 TFX Components Use These Core TFX Libraries: TensorFlow Data Validation (TFDV) TensorFlow Transform (TFT) TensorFlow Model Analysis (TFMA) TensorFlow Metadata (TFMD) + ML Metadata (MLMD) 2.2.2 2.2.3 2.2.4
  32. 32. 2.2.1 TFX Libraries - TFDV TensorFlow Data Validation (TFDV) Find Missing, Redundant & Important Features Identify Features with Unusually-Large Scale `infer_schema()` Generates Schema Describe Feature Ranges Detect Data Drift Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Uniformly Distributed Data è ç Non-Uniformly Distributed Data
  33. 33. Hands On 02_TensorFlow_Data_Validation (TFDV)
  34. 34. 2.2.2 TFX Libraries - TFT TensorFlow Transform (TFT) Preprocess `tf.Example` data with TensorFlow Useful for data that requires a full pass Normalize all inputs by mean and std dev Create vocabulary of strings è integers over all data Bucketize features based on entire data distribution Outputs a TensorFlow graph Re-used across both training and serving Uses Apache Beam (local mode) for Parallel Analysis Can also use distributed mode `preprocessing_fn(inputs)`: Primary Fn to Implement Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training import tensorflow as tf import tensorflow_transform as tft def preprocessing_fn(inputs): x = inputs['x'] y = inputs['y'] s = inputs['s'] x_centered = x - tft.mean(x) y_normalized = tft.scale_to_0_1(y) s_integerized = tft.compute_and_apply_vocabulary(s) x_centered_times_y_normalized = x_centered * y_normalized return { 'x_centered': x_centered, 'y_normalized': y_normalized, 'x_centered_times_y_normalized':x_centered_times_y_normalized, 's_integerized': s_integerized }
  35. 35. Hands On 03_TensorFlow_Transform (TFT)
  36. 36. 2.2.3 TFX Libraries - TFMA TensorFlow Model Analysis (TFMA) Analyze Model on Different Slices of Dataset Track Metrics Over Time (“Next Day Eval”) `EvalSavedModel` Contains Slicing Info TFMA Pipeline: Read, Extract, Evaluate, Write ie. Ensure Model Works Fairly Across All Users Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  37. 37. Hands On 04_TensorFlow_Model_Analysis (TFMA)
  38. 38. 2.2.4 TFX Libraries – Metadata TensorFlow Metadata (TFMD) ML Metadata (MLMD) Record and Retrieve Experiment Metadata Artifact, Execution, and Lineage Info Track Inputs / Outputs of All TFX Components Stores Training Run Info Analysis and Validation Results Model Versioning Info Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  39. 39. 2.3 TFX Components ExampleGen StatisticsGen SchemaGen ExampleValidator Evaluator Transform ModelValidator Trainer Model Pusher2.3.92.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.3.6 2.3.7 2.3.8 Slack (!!)2.3.10
  40. 40. 2.3.1 ExampleGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Load Training Data Into TFX Pipeline Supports External Data Sources Supports CSV and TFRecord Formats Converts Data to tf.Example Note: TFX Pipelines require tf.Example (?!) Difficult to use non-TensorFlow models like XGBoost from tfx.utils.dsl_utils import csv_input from tfx.components.example_gen.csv_example_gen.component import CsvExampleGen examples = csv_input(os.path.join(base_dir, 'data/simple')) example_gen = CsvExampleGen(input_base=examples)
  41. 41. 2.3.2 StatisticsGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Generates Statistics on Training Data Consumes tf.Example instances from tfx import components compute_eval_stats = components.StatisticsGen( input_data=examples_gen.outputs.eval_examples, name='compute-eval-stats' )
  42. 42. 2.3.3 SchemaGen Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Schema Needed by Some TFX Components Data Types, Value Ranges, Optional, Required Consumes Data from StatisticsGen Auto-gen `schema.proto` from training data Schema used by TFDV, TFT, TFMA Libraries Uses TFDV Library to infer schema Best effort and basic Human should verify feature { name: "age" value_count { min: 1 max: 1 } type: FLOAT presence { min_fraction: 1 min_count: 1 } } from tfx import components infer_schema = components.SchemaGen( stats=compute_training_stats.outputs.output)
  43. 43. 2.3.4 ExampleValidator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Identifies Anomalies in Training Data Used with serving data to detect drift / skew Uses StatisticsGen and SchemaGen Outputs Produces Validation Results Uses TFDV Library for Input Validation feature { name: "age" value_count { min: 1 max: 1 } type: FLOAT presence { min_fraction: 1 min_count: 1 } } from tfx import components infer_schema = components.SchemaGen( stats=compute_training_stats.outputs.output )
  44. 44. 2.3.5 Transform Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Uses Data from ExampleGen and SchemaGen Transformations Become Part of TF Graph (!!) Helps Avoid Training/Serving Skew Uses TFT Library for Transformations Designed for Transformations that Require Full Pass Through Entire Dataset Global Reduction Across All Batches Create Word Embeddings & Vocabs String <=> Index Conversion Normalize & Bucketize PCA Feature-Crossing def preprocessing_fn(inputs): # inputs: map from feature keys # to raw not-yet-transformed features # outputs: map from string feature key # to transformed feature operations
  45. 45. 2.3.6 Trainer Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Trains / Validates tf.Examples from Transform Uses schema.proto from SchemaGen Produces SavedModel and EvalSavedModel Uses Core TensorFlow Python API Works with TensorFlow 1.x Estimator API TensorFlow 2.0 Keras Support Coming Soon from tfx import components trainer = components.Trainer( module_file=taxi_pipeline_utils, train_files=transform_training.outputs.output, eval_files=transform_eval.outputs.output, schema=infer_schema.outputs.output, tf_transform_dir=transform_training.outputs.output, train_steps=10000, eval_steps=5000, warm_starting=True )
  46. 46. 2.3.7 Evaluator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Uses EvalSavedModel from Trainer Writes Analysis Results to ML Metadata Store Uses TFMA Library for Analysis TFMA Uses Apache Beam to Scale Analysis from tfx import components import tensorflow_model_analysis as tfma taxi_eval_spec = [ tfma.SingleSliceSpec(), tfma.SingleSliceSpec(columns=['trip_start_hour']) ] model_analyzer = components.Evaluator( examples=examples_gen.outputs.eval_examples, eval_spec=taxi_eval_spec, model_exports=trainer.outputs.output)
  47. 47. 2.3.8 ModelValidator Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Validate Models from Trainer Uses Data from SchemaGen and StatisticsGen Compares New Models to Baseline Baseline == current model in production New Model is Good if Meets/Exceeds Metrics If Good, Notify Pusher to Deploy New Model Writes Validation Results to Metadata Store import tensorflow_model_analysis as tfma taxi_mv_spec = [tfma.SingleSliceSpec()] model_validator = components.ModelValidator( examples=examples_gen.outputs.output, model=trainer.outputs.output)
  48. 48. 2.3.9 Model Pusher (Deployer) Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Push Good Model to Deployment Target Uses Trained SavedModel Writes Version Data to Metadata Store from tfx import components pusher = components.Pusher( model_export=trainer.outputs.output, model_blessing=model_validator.outputs.blessing, serving_model_dir=serving_model_dir)
  49. 49. 2.3.10 Slack Component (!!) Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Runs After ModelValidator Adds Human-in-the-Loop Step to Pipeline TFX Sends Message to Slack with Model URI Asks Human to Review the New Model Respond ‘LGTM’, ‘approve’, ‘decline’, ‘reject’ Requires Slack API Setup / Integration export SLACK_BOT_TOKEN={your_token} _channel_id = 'my-channel-id' _slack_token = os.environ['SLACK_BOT_TOKEN’] slack_validator = SlackComponent( model_export=trainer.outputs.output, model_blessing=model_validator.outputs.blessing, slack_token=_slack_token, channel_id=_channel_id, timeout_sec=3600, ) https://github.com/tensorflow/tfx/tree/master /tfx/examples/custom_components/slack/slack_component
  50. 50. 1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  51. 51. 3.0 ML Pipelines with Airflow and KubeFlow Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy 3.1 Airflow KubeFlow3.2
  52. 52. 3.1 Airflow 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Most Widely-Used Workflow Orchestrator Define Execution Graphs in Python Decent UI Good Community Support
  53. 53. Hands On 05_Airflow_ML_Pipelines (Chicago Taxi Dataset)
  54. 54. Hands On 06_Airflow_Feature_Analysis
  55. 55. Hands On 07_Airflow_Model_Analysis
  56. 56. 3.2 KubeFlow 6 Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training Pipelines Based on Argo CI/CD Project from Intuit TFJob Supports Distributed Training KubeFlow Fairing Project (!!) Run a notebook as a production job Deploy training code with dependencies
  57. 57. Hands On 08_Simple_KubeFlow_ML_Pipeline
  58. 58. Hands On 09_Advanced_KubeFlow_ML_Pipeline (Chicago Taxi Dataset)
  59. 59. Hands On 10_Distributed_TensorFlow_Job
  60. 60. 1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  61. 61. 4.0 Hyper-Parameter Tuning 6 Experiment Single Optimization Run Single Objective Function Across Runs Contains Many Trials Trial List of Param Values Suggestion Optimization Algorithm Job Evaluates a Trial Calculates Objective Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  62. 62. Hands On 11_Hyper_Parameter_Tuning
  63. 63. 1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with Airflow and KubeFlow Agenda Hyper-Parameter Tuning with KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  64. 64. 5.0 Deploy Notebook as Job Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training 6 5.1 Wrap Model in a Docker Image Deploy Job to Kubernetes5.2
  65. 65. 5.1 Create Docker Image Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  66. 66. 5.2 Deploy Notebook as Job Feature Load Feature Analyze Feature Transform Model Train Model Evaluate Model Deploy Reproduce Training
  67. 67. Hands On 12_Deploy_Xgboost_Notebook
  68. 68. 1 Setup Environment with Kubernetes TensorFlow Extended (TFX) ML Pipelines with TFX, Airflow, and KubeFlow Agenda Hyper-Parameter Tuning with TFX and KubeFlow Deploy Notebook with Kubernetes 2 3 4 5
  69. 69. Bonus Extras! 6 TPUs MLflow RedisAI + TensorFlow + PyTorch TensorFlow 2.0 TensorFlow Privacy Libraries 7 8 9 10
  70. 70. Thank you! https://pipeline.ai @cfregly @PipelineAI
  71. 71. Upcoming Hands-On Workshops

×