SlideShare ist ein Scribd-Unternehmen logo

Databricks Overview for MLOps

•
•
Databricks
Databricks

Databricks Overview for MLOps by Clemend Mewald, Director of Product Management, Databricks

Daten & Analysen
1 von 30
Downloaden Sie, um offline zu lesen
Databricks Overview for MLOps Clemens Mewald Director of Product Management
MLOps / Governance The Databricks ML Platform Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
DATA ENGINEERS DATA SCIENTISTS ML ENGINEERS DATA ANALYSTS Collaborative Data Science Workspace MLOps / Governance Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
Data Science Workspace DATA ENGINEERS DATA SCIENTISTS Cloud-native Collaboration Features Commenting Co-Presence Co-Editing Multi-Language Scala, SQL, Python, R: All in one notebook. Collaborative Realtime co-presence, co-editing, and commenting. Databricks Notebooks ML ENGINEERS DATA ANALYSTS
(Git-based) Projects Version Review Test Development / Experimentatio n Production Jobs Git / CI/CD Systems CI/CD Integration ▲ ▼ Supported Git Providers
MLOps / Governance High Quality Data at Scale Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
High Quality Data at Scale Structured, Semi-Structured and Unstructured Data Business Intelligence Data Science Machine Learning Delta Lake Data Science Workspace MLflow Workspace SQL Analytics Ingest any format at any scale from any source ACID transactions guarantee data validity Versioning and time-travel built-in Automated logging of data + version information
Turnkey ML Training at Scale MLOps / Governance Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
ML Runtime: DevOps-free Environment optimized for Machine Learning Packages up the most popular ML Toolkits Simplifies Distributed ML/DL Distribute and scale any single-machine ML code to 1,000’s of machines. Built-in AutoML and Auto-Logging Hyperparameter tuning, AutoML, automated tracking, and visualizations with MLflow Turnkey ML Training at Scale
Distributed Training ▪ Built-in support in the ML Runtime TensorFlow native Distribution Strategy (Spark TensorFlow Distributor) HorovodRunner (Keras, TensorFlow, and PyTorch) Worker Nodes Driver Training Tasks
Distributed Tuning ▪ Built-in support in the ML Runtime Worker Nodes Driver Trials Integration
Support for all Deployment Modes MLOps / Governance Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
Models Tracking Flavor 2 Flavor 1 Custom Models In-Line Code Containers Batch & Stream Scoring Cloud Inference Services OSS Serving Solutions Parameters Metrics Artifacts Models Metadata Deployment Options Staging Production Archived Data Scientists Deployment Engineers v2 v3 v1 Model Registry Support for all Deployment Modes
Support for all Deployment Modes Deploying an MLLib model as a Spark UDF
Support for all Deployment Modes Deploying an MLLib model as a Spark UDF Deploying a Scikit Learn model as a Spark UDF
Support for all Deployment Modes Deploying an MLLib model as a Spark UDF Deploying a Scikit Learn model as a Spark UDF Deploying a TensorFlow model as a Spark UDF
Support for all Deployment Modes Deploying an MLLib model as a Spark UDF Deploying a Scikit Learn model as a Spark UDF Deploying a TensorFlow model as a Spark UDF Yes, they’re all the same! As are the commands to deploy these models as Docker containers, etc.
Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving Data Governance Powered by Experiment Tracking Reproducibility Model Governance End-to-end MLOps / Governance
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Data Source / Lineage Data Versioning Automated Data Source capture and Versioning
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Feature-Level Data Lineage / Usage Automated capture of Feature Usage
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Parameters Metrics Models Artifacts Automated capture of ML metrics, parameters, artifacts, etc.
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Trials Automated capture of Hyperparameter Search
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Model Interpretability Automated Model Interpretability
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Code Versioning Cluster Configuration Environment Configuration Automated capture of Code, Environment and Cluster Specification
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Model Discoverability Model Stage-Based ACLs Model Sharing, Reuse, and ACLs
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Approval Process for Stage Transitions Audit Log of Model Changes Automated Model Lineage and Governance
Powered by Data Governance Experiment Tracking Reproducibility Model Governance Turnkey Serving integrated with Model Versions and Stages Turnkey Model Serving
Data Governance Experiment Tracking Reproducibility Model Governance Quality / Performance Metric Monitoring Powered by Model Quality monitoring
Code versioning Data versioning Cluster configuration Environment specification Auto-Logging Reproducibility Checklist Reproduce Run Feature Data Governance Experiment Tracking Reproducibility Model Governance Powered by ✓ ✓ ✓ ✓ The Result: Full End-to-End Governance and Reproducibility
MLOps / Governance The Databricks ML Platform Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving

Empfohlen

MLOps - The Assembly Line of ML
MLOps - The Assembly Line of MLMLOps - The Assembly Line of ML
MLOps - The Assembly Line of ML
Jordan Birdsell
 
MLOps in action
MLOps in actionMLOps in action
MLOps in action
Pieter de Bruin
 
From Data Science to MLOps
From Data Science to MLOpsFrom Data Science to MLOps
From Data Science to MLOps
Carl W. Handlin
 
What is MLOps
What is MLOpsWhat is MLOps
What is MLOps
Henrik Skogström
 
MLOps for production-level machine learning
MLOps for production-level machine learningMLOps for production-level machine learning
MLOps for production-level machine learning
cnvrg.io AI OS - Hands-on ML Workshops
 
MLOps Bridging the gap between Data Scientists and Ops.
MLOps Bridging the gap between Data Scientists and Ops.MLOps Bridging the gap between Data Scientists and Ops.
MLOps Bridging the gap between Data Scientists and Ops.
Knoldus Inc.
 
MLOps Using MLflow
MLOps Using MLflowMLOps Using MLflow
MLOps Using MLflow
Databricks
 
MLOps Virtual Event: Automating ML at Scale
MLOps Virtual Event: Automating ML at ScaleMLOps Virtual Event: Automating ML at Scale
MLOps Virtual Event: Automating ML at Scale
Databricks
 

Empfohlen

MLOps - The Assembly Line of ML
MLOps - The Assembly Line of MLMLOps - The Assembly Line of ML
MLOps - The Assembly Line of ML
Jordan Birdsell
 
MLOps in action
MLOps in actionMLOps in action
MLOps in action
Pieter de Bruin
 
From Data Science to MLOps
From Data Science to MLOpsFrom Data Science to MLOps
From Data Science to MLOps
Carl W. Handlin
 
What is MLOps
What is MLOpsWhat is MLOps
What is MLOps
Henrik Skogström
 
MLOps for production-level machine learning
MLOps for production-level machine learningMLOps for production-level machine learning
MLOps for production-level machine learning
cnvrg.io AI OS - Hands-on ML Workshops
 
MLOps Bridging the gap between Data Scientists and Ops.
MLOps Bridging the gap between Data Scientists and Ops.MLOps Bridging the gap between Data Scientists and Ops.
MLOps Bridging the gap between Data Scientists and Ops.
Knoldus Inc.
 
MLOps Using MLflow
MLOps Using MLflowMLOps Using MLflow
MLOps Using MLflow
Databricks
 
MLOps Virtual Event: Automating ML at Scale
MLOps Virtual Event: Automating ML at ScaleMLOps Virtual Event: Automating ML at Scale
MLOps Virtual Event: Automating ML at Scale
Databricks
 
Using MLOps to Bring ML to Production/The Promise of MLOps
Using MLOps to Bring ML to Production/The Promise of MLOpsUsing MLOps to Bring ML to Production/The Promise of MLOps
Using MLOps to Bring ML to Production/The Promise of MLOps
Weaveworks
 
MLOps by Sasha Rosenbaum
MLOps by Sasha RosenbaumMLOps by Sasha Rosenbaum
MLOps by Sasha Rosenbaum
Sasha Rosenbaum
 
What’s New with Databricks Machine Learning
What’s New with Databricks Machine LearningWhat’s New with Databricks Machine Learning
What’s New with Databricks Machine Learning
Databricks
 
MLops workshop AWS
MLops workshop AWSMLops workshop AWS
MLops workshop AWS
Gili Nachum
 
Ml ops intro session
Ml ops intro sessionMl ops intro session
Ml ops intro session
Avinash Patil
 
Introduction to MLflow
Introduction to MLflowIntroduction to MLflow
Introduction to MLflow
Databricks
 
MLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageMLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive Advantage
DATAVERSITY
 
Pythonsevilla2019 - Introduction to MLFlow
Pythonsevilla2019 - Introduction to MLFlowPythonsevilla2019 - Introduction to MLFlow
Pythonsevilla2019 - Introduction to MLFlow
Fernando Ortega Gallego
 
Learn to Use Databricks for the Full ML Lifecycle
Learn to Use Databricks for the Full ML LifecycleLearn to Use Databricks for the Full ML Lifecycle
Learn to Use Databricks for the Full ML Lifecycle
Databricks
 
Introdution to Dataops and AIOps (or MLOps)
Introdution to Dataops and AIOps (or MLOps)Introdution to Dataops and AIOps (or MLOps)
Introdution to Dataops and AIOps (or MLOps)
Adrien Blind
 
MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
MLOps Virtual Event | Building Machine Learning Platforms for the Full LifecycleMLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
Databricks
 
Apply MLOps at Scale by H&M
Apply MLOps at Scale by H&MApply MLOps at Scale by H&M
Apply MLOps at Scale by H&M
Databricks
 
MLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in ProductionMLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in Production
Provectus
 
Ml ops past_present_future
Ml ops past_present_futureMl ops past_present_future
Ml ops past_present_future
Nisha Talagala
 
MLOps with Kubeflow
MLOps with Kubeflow MLOps with Kubeflow
MLOps with Kubeflow
Saurabh Kaushik
 
Introducing MLOps.pdf
Introducing MLOps.pdfIntroducing MLOps.pdf
Introducing MLOps.pdf
Dr. Anish Cheriyan (PhD)
 
"Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow""Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow"
Databricks
 
Vertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflowsVertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflows
Márton Kodok
 
“Houston, we have a model...” Introduction to MLOps
“Houston, we have a model...” Introduction to MLOps“Houston, we have a model...” Introduction to MLOps
“Houston, we have a model...” Introduction to MLOps
Rui Quintino
 
MLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
MLOps journey at Swisscom: AI Use Cases, Architecture and Future VisionMLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
MLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
BATbern
 
Machine Learning Models in Production
Machine Learning Models in ProductionMachine Learning Models in Production
Machine Learning Models in Production
DataWorks Summit
 
Certification Study Group - NLP & Recommendation Systems on GCP Session 5
Certification Study Group - NLP & Recommendation Systems on GCP Session 5Certification Study Group - NLP & Recommendation Systems on GCP Session 5
Certification Study Group - NLP & Recommendation Systems on GCP Session 5
gdgsurrey
 

Weitere ähnliche Inhalte

Was ist angesagt?

Using MLOps to Bring ML to Production/The Promise of MLOps
Using MLOps to Bring ML to Production/The Promise of MLOpsUsing MLOps to Bring ML to Production/The Promise of MLOps
Using MLOps to Bring ML to Production/The Promise of MLOps
Weaveworks
 
Introduction to MLflow
Introduction to MLflowIntroduction to MLflow
Introduction to MLflow
Databricks
 
MLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageMLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive Advantage
DATAVERSITY
 
MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
MLOps Virtual Event | Building Machine Learning Platforms for the Full LifecycleMLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
Databricks
 
MLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in ProductionMLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in Production
Provectus
 
"Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow""Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow"
Databricks
 
Vertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflowsVertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflows
Márton Kodok
 

Was ist angesagt? (20)

Using MLOps to Bring ML to Production/The Promise of MLOps
Using MLOps to Bring ML to Production/The Promise of MLOpsUsing MLOps to Bring ML to Production/The Promise of MLOps
Using MLOps to Bring ML to Production/The Promise of MLOps
 
MLOps by Sasha Rosenbaum
MLOps by Sasha RosenbaumMLOps by Sasha Rosenbaum
MLOps by Sasha Rosenbaum
 
What’s New with Databricks Machine Learning
What’s New with Databricks Machine LearningWhat’s New with Databricks Machine Learning
What’s New with Databricks Machine Learning
 
MLops workshop AWS
MLops workshop AWSMLops workshop AWS
MLops workshop AWS
 
Ml ops intro session
Ml ops intro sessionMl ops intro session
Ml ops intro session
 
Introduction to MLflow
Introduction to MLflowIntroduction to MLflow
Introduction to MLflow
 
MLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageMLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive Advantage
 
Pythonsevilla2019 - Introduction to MLFlow
Pythonsevilla2019 - Introduction to MLFlowPythonsevilla2019 - Introduction to MLFlow
Pythonsevilla2019 - Introduction to MLFlow
 
Learn to Use Databricks for the Full ML Lifecycle
Learn to Use Databricks for the Full ML LifecycleLearn to Use Databricks for the Full ML Lifecycle
Learn to Use Databricks for the Full ML Lifecycle
 
Introdution to Dataops and AIOps (or MLOps)
Introdution to Dataops and AIOps (or MLOps)Introdution to Dataops and AIOps (or MLOps)
Introdution to Dataops and AIOps (or MLOps)
 
MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
MLOps Virtual Event | Building Machine Learning Platforms for the Full LifecycleMLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle
 
Apply MLOps at Scale by H&M
Apply MLOps at Scale by H&MApply MLOps at Scale by H&M
Apply MLOps at Scale by H&M
 
MLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in ProductionMLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in Production
 
Ml ops past_present_future
Ml ops past_present_futureMl ops past_present_future
Ml ops past_present_future
 
MLOps with Kubeflow
MLOps with Kubeflow MLOps with Kubeflow
MLOps with Kubeflow
 
Introducing MLOps.pdf
Introducing MLOps.pdfIntroducing MLOps.pdf
Introducing MLOps.pdf
 
"Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow""Managing the Complete Machine Learning Lifecycle with MLflow"
"Managing the Complete Machine Learning Lifecycle with MLflow"
 
Vertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflowsVertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflows
 
“Houston, we have a model...” Introduction to MLOps
“Houston, we have a model...” Introduction to MLOps“Houston, we have a model...” Introduction to MLOps
“Houston, we have a model...” Introduction to MLOps
 
MLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
MLOps journey at Swisscom: AI Use Cases, Architecture and Future VisionMLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
MLOps journey at Swisscom: AI Use Cases, Architecture and Future Vision
 

Ähnlich wie Databricks Overview for MLOps

Machine Learning Models in Production
Machine Learning Models in ProductionMachine Learning Models in Production
Machine Learning Models in Production
DataWorks Summit
 
Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...
Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...
Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...
DataWorks Summit
 
AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)
AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)
AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)
dtz001
 
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
James Anderson
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
Databricks
 

Ähnlich wie Databricks Overview for MLOps (20)

Machine Learning Models in Production
Machine Learning Models in ProductionMachine Learning Models in Production
Machine Learning Models in Production
 
Certification Study Group - NLP & Recommendation Systems on GCP Session 5
Certification Study Group - NLP & Recommendation Systems on GCP Session 5Certification Study Group - NLP & Recommendation Systems on GCP Session 5
Certification Study Group - NLP & Recommendation Systems on GCP Session 5
 
Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...
Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...
Introducing MlFlow: An Open Source Platform for the Machine Learning Lifecycl...
 
Deployment Design Patterns - Deploying Machine Learning and Deep Learning Mod...
Deployment Design Patterns - Deploying Machine Learning and Deep Learning Mod...Deployment Design Patterns - Deploying Machine Learning and Deep Learning Mod...
Deployment Design Patterns - Deploying Machine Learning and Deep Learning Mod...
 
AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)
AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)
AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)
 
DevOps for Machine Learning overview en-us
DevOps for Machine Learning overview en-usDevOps for Machine Learning overview en-us
DevOps for Machine Learning overview en-us
 
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...
 
Managing the Machine Learning Lifecycle with MLflow
Managing the Machine Learning Lifecycle with MLflowManaging the Machine Learning Lifecycle with MLflow
Managing the Machine Learning Lifecycle with MLflow
 
Lviv Data Science Club (Sergiy Lunyakin)
Lviv Data Science Club (Sergiy Lunyakin)Lviv Data Science Club (Sergiy Lunyakin)
Lviv Data Science Club (Sergiy Lunyakin)
 
Machine Learning Operations & Azure
Machine Learning Operations & AzureMachine Learning Operations & Azure
Machine Learning Operations & Azure
 
Building ML Pipelines with DCOS
Building ML Pipelines with DCOSBuilding ML Pipelines with DCOS
Building ML Pipelines with DCOS
 
Paige Roberts: Shortcut MLOps with In-Database Machine Learning
Paige Roberts: Shortcut MLOps with In-Database Machine LearningPaige Roberts: Shortcut MLOps with In-Database Machine Learning
Paige Roberts: Shortcut MLOps with In-Database Machine Learning
 
MLops on Vertex AI Presentation (AI/ML).pptx
MLops on Vertex AI Presentation (AI/ML).pptxMLops on Vertex AI Presentation (AI/ML).pptx
MLops on Vertex AI Presentation (AI/ML).pptx
 
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
GDG Cloud Southlake #16: Priyanka Vergadia: Scalable Data Analytics in Google...
 
Processing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache SparkProcessing Large Datasets for ADAS Applications using Apache Spark
Processing Large Datasets for ADAS Applications using Apache Spark
 
Whats New In 2010 (Msdn & Visual Studio)
Whats New In 2010 (Msdn & Visual Studio)Whats New In 2010 (Msdn & Visual Studio)
Whats New In 2010 (Msdn & Visual Studio)
 
Why is dev ops for machine learning so different
Why is dev ops for machine learning so differentWhy is dev ops for machine learning so different
Why is dev ops for machine learning so different
 
EPAM ML/AI Accelerator - ODAHU
EPAM ML/AI Accelerator - ODAHUEPAM ML/AI Accelerator - ODAHU
EPAM ML/AI Accelerator - ODAHU
 
Innovation morning data analytics + ai
Innovation morning data analytics + ai Innovation morning data analytics + ai
Innovation morning data analytics + ai
 
AML_service.pptx
AML_service.pptxAML_service.pptx
AML_service.pptx
 

Mehr von Databricks

Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1
Databricks
 
Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2
Databricks
 
Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2
Databricks
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
Databricks
 
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDemocratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized Platform
Databricks
 
Why APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringWhy APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML Monitoring
Databricks
 
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch FixThe Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
Databricks
 
Stage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationStage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI Integration
Databricks
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchSimplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Databricks
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on Kubernetes
Databricks
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesScaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Databricks
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
 
Re-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkRe-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and Spark
Databricks
 
Raven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction QueriesRaven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction Queries
Databricks
 
Massive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeMassive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta Lake
Databricks
 
Machine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack DetectionMachine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack Detection
Databricks
 

Mehr von Databricks (20)

DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptxDW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptx
 
Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1Data Lakehouse Symposium | Day 1 | Part 1
Data Lakehouse Symposium | Day 1 | Part 1
 
Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2Data Lakehouse Symposium | Day 1 | Part 2
Data Lakehouse Symposium | Day 1 | Part 2
 
Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2Data Lakehouse Symposium | Day 2
Data Lakehouse Symposium | Day 2
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
 
Democratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized PlatformDemocratizing Data Quality Through a Centralized Platform
Democratizing Data Quality Through a Centralized Platform
 
Learn to Use Databricks for Data Science
Learn to Use Databricks for Data ScienceLearn to Use Databricks for Data Science
Learn to Use Databricks for Data Science
 
Why APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML MonitoringWhy APM Is Not the Same As ML Monitoring
Why APM Is Not the Same As ML Monitoring
 
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch FixThe Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
The Function, the Context, and the Data—Enabling ML Ops at Stitch Fix
 
Stage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI IntegrationStage Level Scheduling Improving Big Data and AI Integration
Stage Level Scheduling Improving Big Data and AI Integration
 
Simplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorchSimplify Data Conversion from Spark to TensorFlow and PyTorch
Simplify Data Conversion from Spark to TensorFlow and PyTorch
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on Kubernetes
 
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark PipelinesScaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
 
Sawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature AggregationsSawtooth Windows for Feature Aggregations
Sawtooth Windows for Feature Aggregations
 
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen SinkRedis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
 
Re-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and SparkRe-imagine Data Monitoring with whylogs and Spark
Re-imagine Data Monitoring with whylogs and Spark
 
Raven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction QueriesRaven: End-to-end Optimization of ML Prediction Queries
Raven: End-to-end Optimization of ML Prediction Queries
 
Massive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta LakeMassive Data Processing in Adobe Using Delta Lake
Massive Data Processing in Adobe Using Delta Lake
 
Machine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack DetectionMachine Learning CI/CD for Email Attack Detection
Machine Learning CI/CD for Email Attack Detection
 

Kürzlich hochgeladen

Jual Cytotec Asli Obat Aborsi No. 1 Paling Manjur
Jual Cytotec Asli Obat Aborsi No. 1 Paling ManjurJual Cytotec Asli Obat Aborsi No. 1 Paling Manjur
Jual Cytotec Asli Obat Aborsi No. 1 Paling Manjur
ptikerjasaptiker
 
PLE-statistics document for primary schs
PLE-statistics document for primary schsPLE-statistics document for primary schs
PLE-statistics document for primary schs
cnajjemba
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
gajnagarg
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
怎样办理旧金山城市学院毕业证（CCSF毕业证书）成绩单学校原版复制
怎样办理旧金山城市学院毕业证（CCSF毕业证书）成绩单学校原版复制怎样办理旧金山城市学院毕业证（CCSF毕业证书）成绩单学校原版复制
怎样办理旧金山城市学院毕业证（CCSF毕业证书）成绩单学校原版复制
vexqp
 
Sequential and reinforcement learning for demand side management by Margaux B...
Sequential and reinforcement learning for demand side management by Margaux B...Sequential and reinforcement learning for demand side management by Margaux B...
Sequential and reinforcement learning for demand side management by Margaux B...
Paris Women in Machine Learning and Data Science
 
Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...
nirzagarg
 
一比一原版(UCD毕业证书）加州大学戴维斯分校毕业证成绩单原件一模一样
一比一原版(UCD毕业证书）加州大学戴维斯分校毕业证成绩单原件一模一样一比一原版(UCD毕业证书）加州大学戴维斯分校毕业证成绩单原件一模一样
一比一原版(UCD毕业证书）加州大学戴维斯分校毕业证成绩单原件一模一样
wsppdmt
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制
怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制
怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制
vexqp
 
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
gajnagarg
 
怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制
vexqp
 
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptxThe-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
Vivek487417
 
如何办理英国诺森比亚大学毕业证（NU毕业证书）成绩单原件一模一样
如何办理英国诺森比亚大学毕业证（NU毕业证书）成绩单原件一模一样如何办理英国诺森比亚大学毕业证（NU毕业证书）成绩单原件一模一样
如何办理英国诺森比亚大学毕业证（NU毕业证书）成绩单原件一模一样
wsppdmt
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
gajnagarg
 

Kürzlich hochgeladen (20)

Jual Cytotec Asli Obat Aborsi No. 1 Paling Manjur
Jual Cytotec Asli Obat Aborsi No. 1 Paling ManjurJual Cytotec Asli Obat Aborsi No. 1 Paling Manjur
Jual Cytotec Asli Obat Aborsi No. 1 Paling Manjur
 
PLE-statistics document for primary schs
PLE-statistics document for primary schsPLE-statistics document for primary schs
PLE-statistics document for primary schs
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
 
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangePredicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
 
怎样办理旧金山城市学院毕业证（CCSF毕业证书）成绩单学校原版复制
怎样办理旧金山城市学院毕业证（CCSF毕业证书）成绩单学校原版复制怎样办理旧金山城市学院毕业证（CCSF毕业证书）成绩单学校原版复制
怎样办理旧金山城市学院毕业证（CCSF毕业证书）成绩单学校原版复制
 
Sequential and reinforcement learning for demand side management by Margaux B...
Sequential and reinforcement learning for demand side management by Margaux B...Sequential and reinforcement learning for demand side management by Margaux B...
Sequential and reinforcement learning for demand side management by Margaux B...
 
Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Tumkur [ 7014168258 ] Call Me For Genuine Models We...
 
一比一原版(UCD毕业证书）加州大学戴维斯分校毕业证成绩单原件一模一样
一比一原版(UCD毕业证书）加州大学戴维斯分校毕业证成绩单原件一模一样一比一原版(UCD毕业证书）加州大学戴维斯分校毕业证成绩单原件一模一样
一比一原版(UCD毕业证书）加州大学戴维斯分校毕业证成绩单原件一模一样
 
Switzerland Constitution 2002.pdf.........
Switzerland Constitution 2002.pdf.........Switzerland Constitution 2002.pdf.........
Switzerland Constitution 2002.pdf.........
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制
怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制
怎样办理伦敦大学毕业证（UoL毕业证书）成绩单学校原版复制
 
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
 
怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制
怎样办理伦敦大学城市学院毕业证（CITY毕业证书）成绩单学校原版复制
 
Aspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraAspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - Almora
 
Data Analyst Tasks to do the internship.pdf
Data Analyst Tasks to do the internship.pdfData Analyst Tasks to do the internship.pdf
Data Analyst Tasks to do the internship.pdf
 
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptxThe-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
The-boAt-Story-Navigating-the-Waves-of-Innovation.pptx
 
如何办理英国诺森比亚大学毕业证（NU毕业证书）成绩单原件一模一样
如何办理英国诺森比亚大学毕业证（NU毕业证书）成绩单原件一模一样如何办理英国诺森比亚大学毕业证（NU毕业证书）成绩单原件一模一样
如何办理英国诺森比亚大学毕业证（NU毕业证书）成绩单原件一模一样
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
 

Databricks Overview for MLOps

  • 1. Databricks Overview for MLOps Clemens Mewald Director of Product Management
  • 2. MLOps / Governance The Databricks ML Platform Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
  • 3. DATA ENGINEERS DATA SCIENTISTS ML ENGINEERS DATA ANALYSTS Collaborative Data Science Workspace MLOps / Governance Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
  • 4. Data Science Workspace DATA ENGINEERS DATA SCIENTISTS Cloud-native Collaboration Features Commenting Co-Presence Co-Editing Multi-Language Scala, SQL, Python, R: All in one notebook. Collaborative Realtime co-presence, co-editing, and commenting. Databricks Notebooks ML ENGINEERS DATA ANALYSTS
  • 5. (Git-based) Projects Version Review Test Development / Experimentatio n Production Jobs Git / CI/CD Systems CI/CD Integration ▲ ▼ Supported Git Providers
  • 6. MLOps / Governance High Quality Data at Scale Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
  • 7. High Quality Data at Scale Structured, Semi-Structured and Unstructured Data Business Intelligence Data Science Machine Learning Delta Lake Data Science Workspace MLflow Workspace SQL Analytics Ingest any format at any scale from any source ACID transactions guarantee data validity Versioning and time-travel built-in Automated logging of data + version information
  • 8. Turnkey ML Training at Scale MLOps / Governance Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
  • 9. ML Runtime: DevOps-free Environment optimized for Machine Learning Packages up the most popular ML Toolkits Simplifies Distributed ML/DL Distribute and scale any single-machine ML code to 1,000’s of machines. Built-in AutoML and Auto-Logging Hyperparameter tuning, AutoML, automated tracking, and visualizations with MLflow Turnkey ML Training at Scale
  • 10. Distributed Training ▪ Built-in support in the ML Runtime TensorFlow native Distribution Strategy (Spark TensorFlow Distributor) HorovodRunner (Keras, TensorFlow, and PyTorch) Worker Nodes Driver Training Tasks
  • 11. Distributed Tuning ▪ Built-in support in the ML Runtime Worker Nodes Driver Trials Integration
  • 12. Support for all Deployment Modes MLOps / Governance Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving
  • 13. Models Tracking Flavor 2 Flavor 1 Custom Models In-Line Code Containers Batch & Stream Scoring Cloud Inference Services OSS Serving Solutions Parameters Metrics Artifacts Models Metadata Deployment Options Staging Production Archived Data Scientists Deployment Engineers v2 v3 v1 Model Registry Support for all Deployment Modes
  • 14. Support for all Deployment Modes Deploying an MLLib model as a Spark UDF
  • 15. Support for all Deployment Modes Deploying an MLLib model as a Spark UDF Deploying a Scikit Learn model as a Spark UDF
  • 16. Support for all Deployment Modes Deploying an MLLib model as a Spark UDF Deploying a Scikit Learn model as a Spark UDF Deploying a TensorFlow model as a Spark UDF
  • 17. Support for all Deployment Modes Deploying an MLLib model as a Spark UDF Deploying a Scikit Learn model as a Spark UDF Deploying a TensorFlow model as a Spark UDF Yes, they’re all the same! As are the commands to deploy these models as Docker containers, etc.
  • 18. Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving Data Governance Powered by Experiment Tracking Reproducibility Model Governance End-to-end MLOps / Governance
  • 19. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Data Source / Lineage Data Versioning Automated Data Source capture and Versioning
  • 20. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Feature-Level Data Lineage / Usage Automated capture of Feature Usage
  • 21. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Parameters Metrics Models Artifacts Automated capture of ML metrics, parameters, artifacts, etc.
  • 22. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Trials Automated capture of Hyperparameter Search
  • 23. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Model Interpretability Automated Model Interpretability
  • 24. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Code Versioning Cluster Configuration Environment Configuration Automated capture of Code, Environment and Cluster Specification
  • 25. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Model Discoverability Model Stage-Based ACLs Model Sharing, Reuse, and ACLs
  • 26. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Approval Process for Stage Transitions Audit Log of Model Changes Automated Model Lineage and Governance
  • 27. Powered by Data Governance Experiment Tracking Reproducibility Model Governance Turnkey Serving integrated with Model Versions and Stages Turnkey Model Serving
  • 28. Data Governance Experiment Tracking Reproducibility Model Governance Quality / Performance Metric Monitoring Powered by Model Quality monitoring
  • 29. Code versioning Data versioning Cluster configuration Environment specification Auto-Logging Reproducibility Checklist Reproduce Run Feature Data Governance Experiment Tracking Reproducibility Model Governance Powered by ✓ ✓ ✓ ✓ The Result: Full End-to-End Governance and Reproducibility
  • 30. MLOps / Governance The Databricks ML Platform Data Science Workspace Data Ingestion Data Versioning Model Training Model Tuning Runtime and Environments Monitoring Batch Scoring Online Serving