Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Machine Learning Platform Life-Cycle Management

567 Aufrufe

Veröffentlicht am

Presented at AI NEXTCon SF 4/10-13, 2018
http://aisv18.xnextcon.com

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

Machine Learning Platform Life-Cycle Management

  1. 1. AI NEXTCon Silicon Valley 18 SANTA CLARA | APRIL 10-13 #ainextcon Hope (Xinwei) Wang Software Engineer at Intuit
  2. 2. • What is a machine learning platform? • What is the ML platform lifecycle? • Why ML platform lifecycle management? • Artifacts and their associations • Use cases at Intuit Machine Learning Platform Life-Cycle Management
  3. 3. • Manages the entire lifecycle of an ML model • Includes automating and accelerating the delivery of ML applications
  4. 4. Data Discovery Feature Engineering Model Training Model Scoring Model Development Data Ingestion Life-Cycle Management
  5. 5. • Metadata/catalog tool • Accessible data source (raw attributes & data lineage) Metadata/catalog tool Data lake
  6. 6. • Output: features • Reproducible • Reusable Metadata/catalog tool Data lake
  7. 7. • Collaborative environment • Access data lake Notebooks Data lake
  8. 8. Support ability for: • Being triggered either manually/via automation • Creation and management of training sets • Retraining • Optimizing hyper parameter tuning through parallelization of model training execution
  9. 9. • Support online/offline (depend on use cases) • Ability to be triggered either manually/via automation
  10. 10. • No central artifact management solution • Hard to reuse existing features/data/algorithms/toolings • Inability to scale for large datasets • Lack of automation/orchestration across the ML lifecycle • Lack of rigor/discipline in the ML development lifecycle • Slow down delivery of machine learning applications
  11. 11. • Optimizing data scientists’ engineering process • Tie ML components together into a cohesive platform, support the lifecycle of ML artifacts end-to-end • Increase efficiency of delivering ML predictions at scale • Artifact management
  12. 12. • Data artifacts – Features – Training sets • Model artifacts – Model code – Trained models – Performance metrics – Hyper parameter values • Environment artifacts – Languages & language versions – Packages & package versions
  13. 13. Data Discovery Feature Engineering Model Training Model Scoring Model Development Data Ingestion Life-Cycle Management
  14. 14. Environment in container
  15. 15. • Flexibility: Model has specific environment • Consistency: Model has same behavior throughout the lifecycle
  16. 16. Environment metadata Container Trained models Model Features Scheduled retraining/ performance benchmarks Hyper- parameter values Metrics definition Performance metrics Feature set definition Training datasets Model artifacts Data artifacts Environment artifacts Model scoring
  17. 17. • Developed in notebooks • Multiple versions • Each version associated with an externalized environment artifact
  18. 18. • Environment must be consistent for development, training, scoring • Externalized as metadata • Model/execution environments constructed from metadata and deployed into containers (Docker, Yarn, Conda, etc.)
  19. 19. Container/virtual environment Usage Docker container • Model development • Model training • Online scoring Yarn container On Spark cluster • Distributed training • Batch offline scoring Conda environment Model development • Tailored to the environment (built based on externalized environment metadata) • Used for model development, training, execution
  20. 20. • Used as data input of the model • Stored in discoverable feature repository • Metadata defines the model specific feature sets
  21. 21. • Serialized, weighted model files • Associated with a version of model code and training set
  22. 22. • Datasets used to train, validate and test the model • Associated to a trained model
  23. 23. • Define what feature sets this model requires
  24. 24. • Set up values before learning process • Model specific
  25. 25. • Defines the metrics to collect and thresholds to evaluate models against.
  26. 26. • Metrics to evaluate model effectiveness • Model metrics including: ROC curve, confusion matrix, F1 score, precision, recall, etc.
  27. 27. § To automate the retraining and deployment of updated models § Model-specific
  28. 28. Data Discovery Feature Engineering Model Training Model Scoring Model Development Data Ingestion Life-Cycle Management
  29. 29. Personalization service Feature service Online scoring serviceSpark streamingKafkaBatch source Offline Spark job Online store Feature repository Lake Clickstream data Online features Offline features Score data Get contextual help Self-help service in QuickBooks
  30. 30. Serialized Model Environment Metadata Deploy Code Docker File Docker Base Image Model Specific Docker Image Example: Online scoring service deployment diagram Model Deploy Template Model Specific Deploy Code/metadata Trained Model Serialized Model
  31. 31. Data Discovery Feature Engineering Model Training Model Scoring Model Development Data Ingestion Life-Cycle Management
  32. 32. Serialized Model Environment Metadata Deploy Code Docker File Docker Base Image Model Specific Docker Image Model Deploy Template Model Specific Deploy Code/metadata Trained Model Serialized Model Example: Online scoring service deployment diagram
  33. 33. Serialized Model Environment Metadata Deploy Code Docker File Docker Base Image Model Specific Docker Image Model Deploy Template Model Specific Deploy Code/metadata Trained Model Serialized Model Example: Online scoring service deployment diagram
  34. 34. Serialized Model Environment Metadata Deploy Code Docker File Docker Base Image Model Specific Docker Image Model Deploy Template Model Specific Deploy Code/metadata Trained Model Serialized Model Example: Online scoring service deployment diagram
  35. 35. Serialized Model Environment Metadata Deploy Code Docker File Docker Base Image Model Specific Docker Image Model Deploy Template Model Specific Deploy Code/metadata Trained Model Serialized Model Example: Online scoring service deployment diagram
  36. 36. Serialized Model Environment Metadata Deploy Code Docker File Docker Base Image Model Specific Docker Image Model Deploy Template Model Specific Deploy Code/metadata Trained Model Serialized Model Example: Online scoring service deployment diagram
  37. 37. Environment in container
  38. 38. Thank you

×