SlideShare ist ein Scribd-Unternehmen logo
1 von 8
Downloaden Sie, um offline zu lesen
Working with Data
Eng Teong Cheah
Microsoft MVP
Working with Datastores
In Azure Machine Learning, datastores are abstractions for cloud data sources. They
encapsulate the information required to connect to data sources. You can access
datastores directly in code by using the the Azure Machine Learning.
Types of Datastore
Azure Machine Learning supports the creation of datastores for multiple kinds of Azure
data source, including:
- Azure Storage (blob and file containers)
- Azure Data Lake stores
- Azure SQL Database
- Azure Databricks file system (DBFS)
Working with Datasets
Datasets are versioned packaged data objects that can be easily consumed in
experiments and pipelines. Datasets are the recommended what to work with data, and re
the primary mechanism for advanced Azure Machine Learning capabilities like data
labeling and data drift monitoring.
Types of Datasets
Datasets are typically on files in a datastores, though they can also be based on URLs and
other sources. You can create the following types of datasets:
- Tabular
The data is read from the dataset as a table. You should use this type of dataset when
your data is consistently structured and you want to work with it in common tabular
data structures, such as Pandas dataframes.
- File
The dataset presents a list of file paths that can be read as though from the file system.
Use this type of dataset when your data is unstructured, or when you need to process
the data at the file level(for example, to train a convolutional neural network from a set
of image files).
Demo
Work with Data
References
Microsoft Docs

Weitere Àhnliche Inhalte

Ähnlich wie Working with Data

ADF Demo_ppt.pptx
ADF Demo_ppt.pptxADF Demo_ppt.pptx
ADF Demo_ppt.pptxvamsytaurus
 
Afternoons with Azure - Azure Data Services
Afternoons with Azure - Azure Data ServicesAfternoons with Azure - Azure Data Services
Afternoons with Azure - Azure Data ServicesCCG
 
Data Analytics Meetup: Introduction to Azure Data Lake Storage
Data Analytics Meetup: Introduction to Azure Data Lake Storage Data Analytics Meetup: Introduction to Azure Data Lake Storage
Data Analytics Meetup: Introduction to Azure Data Lake Storage CCG
 
Introduction to Azure Data Lake
Introduction to Azure Data LakeIntroduction to Azure Data Lake
Introduction to Azure Data LakeAntonios Chatzipavlis
 
Azure fundamental -Introduction
Azure fundamental -IntroductionAzure fundamental -Introduction
Azure fundamental -IntroductionManishK55
 
DATABASE Lecture 1 and 2.pptx
DATABASE Lecture 1 and 2.pptxDATABASE Lecture 1 and 2.pptx
DATABASE Lecture 1 and 2.pptxRUBAB79
 
Modern Analytics Academy - Data Modeling (1).pptx
Modern Analytics Academy - Data Modeling (1).pptxModern Analytics Academy - Data Modeling (1).pptx
Modern Analytics Academy - Data Modeling (1).pptxssuser290967
 
Transform your data with Azure Data factory
Transform your data with Azure Data factoryTransform your data with Azure Data factory
Transform your data with Azure Data factoryPrometix Pty Ltd
 
Exploring Microsoft Azure Infrastructures
Exploring Microsoft Azure InfrastructuresExploring Microsoft Azure Infrastructures
Exploring Microsoft Azure InfrastructuresCCG
 
2.Introduction to NOSQL (Core concepts).pptx
2.Introduction to NOSQL (Core concepts).pptx2.Introduction to NOSQL (Core concepts).pptx
2.Introduction to NOSQL (Core concepts).pptxRushikeshChikane2
 
Azure Data serices and databricks architecture
Azure Data serices and databricks architectureAzure Data serices and databricks architecture
Azure Data serices and databricks architectureAdventureWorld5
 
Azure data bricks by Eugene Polonichko
Azure data bricks by Eugene PolonichkoAzure data bricks by Eugene Polonichko
Azure data bricks by Eugene PolonichkoAlex Tumanoff
 
Azure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene PolonichkoAzure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene PolonichkoDimko Zhluktenko
 
Cloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsCloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsPushkar Chivate
 
Integrating with Azure Data Lake
Integrating with Azure Data LakeIntegrating with Azure Data Lake
Integrating with Azure Data LakeRuman Khan
 
Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)James Serra
 

Ähnlich wie Working with Data (20)

ADF Demo_ppt.pptx
ADF Demo_ppt.pptxADF Demo_ppt.pptx
ADF Demo_ppt.pptx
 
Azure Data Engineering.pptx
Azure Data Engineering.pptxAzure Data Engineering.pptx
Azure Data Engineering.pptx
 
Afternoons with Azure - Azure Data Services
Afternoons with Azure - Azure Data ServicesAfternoons with Azure - Azure Data Services
Afternoons with Azure - Azure Data Services
 
Data Analytics Meetup: Introduction to Azure Data Lake Storage
Data Analytics Meetup: Introduction to Azure Data Lake Storage Data Analytics Meetup: Introduction to Azure Data Lake Storage
Data Analytics Meetup: Introduction to Azure Data Lake Storage
 
Introduction to Azure Data Lake
Introduction to Azure Data LakeIntroduction to Azure Data Lake
Introduction to Azure Data Lake
 
Azure fundamental -Introduction
Azure fundamental -IntroductionAzure fundamental -Introduction
Azure fundamental -Introduction
 
DATABASE Lecture 1 and 2.pptx
DATABASE Lecture 1 and 2.pptxDATABASE Lecture 1 and 2.pptx
DATABASE Lecture 1 and 2.pptx
 
Modern Analytics Academy - Data Modeling (1).pptx
Modern Analytics Academy - Data Modeling (1).pptxModern Analytics Academy - Data Modeling (1).pptx
Modern Analytics Academy - Data Modeling (1).pptx
 
Transform your data with Azure Data factory
Transform your data with Azure Data factoryTransform your data with Azure Data factory
Transform your data with Azure Data factory
 
Exploring Microsoft Azure Infrastructures
Exploring Microsoft Azure InfrastructuresExploring Microsoft Azure Infrastructures
Exploring Microsoft Azure Infrastructures
 
2.Introduction to NOSQL (Core concepts).pptx
2.Introduction to NOSQL (Core concepts).pptx2.Introduction to NOSQL (Core concepts).pptx
2.Introduction to NOSQL (Core concepts).pptx
 
Azure Data serices and databricks architecture
Azure Data serices and databricks architectureAzure Data serices and databricks architecture
Azure Data serices and databricks architecture
 
slides.pptx
slides.pptxslides.pptx
slides.pptx
 
slides.pptx
slides.pptxslides.pptx
slides.pptx
 
Azure data bricks by Eugene Polonichko
Azure data bricks by Eugene PolonichkoAzure data bricks by Eugene Polonichko
Azure data bricks by Eugene Polonichko
 
Azure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene PolonichkoAzure DataBricks for Data Engineering by Eugene Polonichko
Azure DataBricks for Data Engineering by Eugene Polonichko
 
Cloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsCloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure tools
 
Integrating with Azure Data Lake
Integrating with Azure Data LakeIntegrating with Azure Data Lake
Integrating with Azure Data Lake
 
Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)
 
CC -Unit4.pptx
CC -Unit4.pptxCC -Unit4.pptx
CC -Unit4.pptx
 

Mehr von Eng Teong Cheah

Responsible Machine Learning
Responsible Machine LearningResponsible Machine Learning
Responsible Machine LearningEng Teong Cheah
 
Training Optimal Models
Training Optimal ModelsTraining Optimal Models
Training Optimal ModelsEng Teong Cheah
 
Machine Learning Workflows
Machine Learning WorkflowsMachine Learning Workflows
Machine Learning WorkflowsEng Teong Cheah
 
Working with Compute
Working with ComputeWorking with Compute
Working with ComputeEng Teong Cheah
 
Experiments & TrainingModels
Experiments & TrainingModelsExperiments & TrainingModels
Experiments & TrainingModelsEng Teong Cheah
 
Automated Machine Learning
Automated Machine LearningAutomated Machine Learning
Automated Machine LearningEng Teong Cheah
 
Getting Started with Azure Machine Learning
Getting Started with Azure Machine LearningGetting Started with Azure Machine Learning
Getting Started with Azure Machine LearningEng Teong Cheah
 
Hacking Containers - Container Storage
Hacking Containers - Container StorageHacking Containers - Container Storage
Hacking Containers - Container StorageEng Teong Cheah
 
Hacking Containers - Looking at Cgroups
Hacking Containers - Looking at CgroupsHacking Containers - Looking at Cgroups
Hacking Containers - Looking at CgroupsEng Teong Cheah
 
Hacking Containers - Linux Containers
Hacking Containers - Linux ContainersHacking Containers - Linux Containers
Hacking Containers - Linux ContainersEng Teong Cheah
 
Data Security - Storage Security
Data Security - Storage SecurityData Security - Storage Security
Data Security - Storage SecurityEng Teong Cheah
 
Application Security- App security
Application Security- App securityApplication Security- App security
Application Security- App securityEng Teong Cheah
 
Application Security - Key Vault
Application Security - Key VaultApplication Security - Key Vault
Application Security - Key VaultEng Teong Cheah
 
Compute Security - Container Security
Compute Security - Container SecurityCompute Security - Container Security
Compute Security - Container SecurityEng Teong Cheah
 
Compute Security - Host Security
Compute Security - Host SecurityCompute Security - Host Security
Compute Security - Host SecurityEng Teong Cheah
 
Virtual Networking Security - Network Security
Virtual Networking Security - Network SecurityVirtual Networking Security - Network Security
Virtual Networking Security - Network SecurityEng Teong Cheah
 
Virtual Networking Security - Perimeter Security
Virtual Networking Security - Perimeter SecurityVirtual Networking Security - Perimeter Security
Virtual Networking Security - Perimeter SecurityEng Teong Cheah
 
Access Security - Hybrid Identity
Access Security - Hybrid IdentityAccess Security - Hybrid Identity
Access Security - Hybrid IdentityEng Teong Cheah
 

Mehr von Eng Teong Cheah (20)

Monitoring Models
Monitoring ModelsMonitoring Models
Monitoring Models
 
Responsible Machine Learning
Responsible Machine LearningResponsible Machine Learning
Responsible Machine Learning
 
Training Optimal Models
Training Optimal ModelsTraining Optimal Models
Training Optimal Models
 
Deploying Models
Deploying ModelsDeploying Models
Deploying Models
 
Machine Learning Workflows
Machine Learning WorkflowsMachine Learning Workflows
Machine Learning Workflows
 
Working with Compute
Working with ComputeWorking with Compute
Working with Compute
 
Experiments & TrainingModels
Experiments & TrainingModelsExperiments & TrainingModels
Experiments & TrainingModels
 
Automated Machine Learning
Automated Machine LearningAutomated Machine Learning
Automated Machine Learning
 
Getting Started with Azure Machine Learning
Getting Started with Azure Machine LearningGetting Started with Azure Machine Learning
Getting Started with Azure Machine Learning
 
Hacking Containers - Container Storage
Hacking Containers - Container StorageHacking Containers - Container Storage
Hacking Containers - Container Storage
 
Hacking Containers - Looking at Cgroups
Hacking Containers - Looking at CgroupsHacking Containers - Looking at Cgroups
Hacking Containers - Looking at Cgroups
 
Hacking Containers - Linux Containers
Hacking Containers - Linux ContainersHacking Containers - Linux Containers
Hacking Containers - Linux Containers
 
Data Security - Storage Security
Data Security - Storage SecurityData Security - Storage Security
Data Security - Storage Security
 
Application Security- App security
Application Security- App securityApplication Security- App security
Application Security- App security
 
Application Security - Key Vault
Application Security - Key VaultApplication Security - Key Vault
Application Security - Key Vault
 
Compute Security - Container Security
Compute Security - Container SecurityCompute Security - Container Security
Compute Security - Container Security
 
Compute Security - Host Security
Compute Security - Host SecurityCompute Security - Host Security
Compute Security - Host Security
 
Virtual Networking Security - Network Security
Virtual Networking Security - Network SecurityVirtual Networking Security - Network Security
Virtual Networking Security - Network Security
 
Virtual Networking Security - Perimeter Security
Virtual Networking Security - Perimeter SecurityVirtual Networking Security - Perimeter Security
Virtual Networking Security - Perimeter Security
 
Access Security - Hybrid Identity
Access Security - Hybrid IdentityAccess Security - Hybrid Identity
Access Security - Hybrid Identity
 

KĂŒrzlich hochgeladen

A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 

KĂŒrzlich hochgeladen (20)

A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 

Working with Data

  • 3. Working with Datastores In Azure Machine Learning, datastores are abstractions for cloud data sources. They encapsulate the information required to connect to data sources. You can access datastores directly in code by using the the Azure Machine Learning.
  • 4. Types of Datastore Azure Machine Learning supports the creation of datastores for multiple kinds of Azure data source, including: - Azure Storage (blob and file containers) - Azure Data Lake stores - Azure SQL Database - Azure Databricks file system (DBFS)
  • 5. Working with Datasets Datasets are versioned packaged data objects that can be easily consumed in experiments and pipelines. Datasets are the recommended what to work with data, and re the primary mechanism for advanced Azure Machine Learning capabilities like data labeling and data drift monitoring.
  • 6. Types of Datasets Datasets are typically on files in a datastores, though they can also be based on URLs and other sources. You can create the following types of datasets: - Tabular The data is read from the dataset as a table. You should use this type of dataset when your data is consistently structured and you want to work with it in common tabular data structures, such as Pandas dataframes. - File The dataset presents a list of file paths that can be read as though from the file system. Use this type of dataset when your data is unstructured, or when you need to process the data at the file level(for example, to train a convolutional neural network from a set of image files).