SlideShare ist ein Scribd-Unternehmen logo
1 von 5
32- Azure Data Factory
 Validation Activity in azure data factory
 Demo- Validation Activity
Welcome in BPCloudLearningInHindi
1
Validation Activity:
2
Welcome in BPCloudLearningInHindi
You can use a Validation in a pipeline to ensure the pipeline only continues execution once
it has validated the attached dataset reference exists, that it meets the specified criteria,
or timeout has been reached
In Azure Data Factory, the Validation activity is used to verify that the data in a source
data store meets certain criteria before it is processed further. This activity can help
ensure data quality and prevent errors downstream.
The Validation activity can be used to perform a wide range of data validation checks,
including:
1. Checking that the data confirms to a specific schema or data model.
2. Verifying that required columns or fields are present and contain valid data.
3. Checking for duplicate or missing data.
4. Ensuring that data falls within acceptable ranges or thresholds.
5. Validating that data meets certain business rules or constraints.
3
Welcome in BPCloudLearningInHindi
To use the Validation activity, you need to define a validation rule using either a JSON
schema or an Azure Data Factory expression. The validation rule specifies the criteria that
the data must meet in order to pass validation.
Overall, the Validation activity is a useful tool for ensuring data quality and accuracy in
Azure Data Factory pipelines.
The "Validation" activity is an important tool for ensuring data quality and accuracy in your
Azure Data Factory pipelines, and can be used in conjunction with other activities such as
the "Copy" and "Transform" activities to create a comprehensive data processing workflow.
Welcome in BPCloudLearningInHindi 4
Welcome in BPCloudLearningInHindi 5

Weitere Àhnliche Inhalte

Was ist angesagt?

Fully Utilizing Spark for Data Validation
Fully Utilizing Spark for Data ValidationFully Utilizing Spark for Data Validation
Fully Utilizing Spark for Data Validation
Databricks
 

Was ist angesagt? (20)

Snowflake Architecture.pptx
Snowflake Architecture.pptxSnowflake Architecture.pptx
Snowflake Architecture.pptx
 
Fully Utilizing Spark for Data Validation
Fully Utilizing Spark for Data ValidationFully Utilizing Spark for Data Validation
Fully Utilizing Spark for Data Validation
 
Speeding Time to Insight with a Modern ELT Approach
Speeding Time to Insight with a Modern ELT ApproachSpeeding Time to Insight with a Modern ELT Approach
Speeding Time to Insight with a Modern ELT Approach
 
Azure data platform overview
Azure data platform overviewAzure data platform overview
Azure data platform overview
 
Implement SQL Server on an Azure VM
Implement SQL Server on an Azure VMImplement SQL Server on an Azure VM
Implement SQL Server on an Azure VM
 
Azure+Databricks+Course+Slide+Deck+V4.pdf
Azure+Databricks+Course+Slide+Deck+V4.pdfAzure+Databricks+Course+Slide+Deck+V4.pdf
Azure+Databricks+Course+Slide+Deck+V4.pdf
 
Intro to Azure Data Factory v1
Intro to Azure Data Factory v1Intro to Azure Data Factory v1
Intro to Azure Data Factory v1
 
Azure Data Factory for Azure Data Week
Azure Data Factory for Azure Data WeekAzure Data Factory for Azure Data Week
Azure Data Factory for Azure Data Week
 
Azure Data Factory Introduction.pdf
Azure Data Factory Introduction.pdfAzure Data Factory Introduction.pdf
Azure Data Factory Introduction.pdf
 
Databricks Fundamentals
Databricks FundamentalsDatabricks Fundamentals
Databricks Fundamentals
 
HA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybridHA/DR options with SQL Server in Azure and hybrid
HA/DR options with SQL Server in Azure and hybrid
 
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on DatabricksCI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks
 
1- Introduction of Azure data factory.pptx
1- Introduction of Azure data factory.pptx1- Introduction of Azure data factory.pptx
1- Introduction of Azure data factory.pptx
 
Technical Deck Delta Live Tables.pdf
Technical Deck Delta Live Tables.pdfTechnical Deck Delta Live Tables.pdf
Technical Deck Delta Live Tables.pdf
 
Azure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the CloudAzure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the Cloud
 
PL/SQL All the Things in Oracle SQL Developer
PL/SQL All the Things in Oracle SQL DeveloperPL/SQL All the Things in Oracle SQL Developer
PL/SQL All the Things in Oracle SQL Developer
 
Azure Databricks - An Introduction (by Kris Bock)
Azure Databricks - An Introduction (by Kris Bock)Azure Databricks - An Introduction (by Kris Bock)
Azure Databricks - An Introduction (by Kris Bock)
 
Azure Data Factory
Azure Data FactoryAzure Data Factory
Azure Data Factory
 
Amazon Redshift Tutorial | AWS Tutorial for Beginners | AWS Certification Tra...
Amazon Redshift Tutorial | AWS Tutorial for Beginners | AWS Certification Tra...Amazon Redshift Tutorial | AWS Tutorial for Beginners | AWS Certification Tra...
Amazon Redshift Tutorial | AWS Tutorial for Beginners | AWS Certification Tra...
 
Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics Overview (r2)
 

Ähnlich wie 32- Validation Activity in Azure Data Factory.pptx

Less11 Security
Less11 SecurityLess11 Security
Less11 Security
vivaankumar
 
DB2 LUW Auditing
DB2 LUW AuditingDB2 LUW Auditing
DB2 LUW Auditing
DB2Locksmith
 

Ähnlich wie 32- Validation Activity in Azure Data Factory.pptx (20)

Testing insights from data lakes
Testing insights from data lakesTesting insights from data lakes
Testing insights from data lakes
 
Less11 Security
Less11 SecurityLess11 Security
Less11 Security
 
Microsoft sc 900 exam dumps 2021
Microsoft sc 900 exam dumps 2021Microsoft sc 900 exam dumps 2021
Microsoft sc 900 exam dumps 2021
 
IRJET- Accomplishing Secure, Widespread, and Fine-Grained Question Results Ch...
IRJET- Accomplishing Secure, Widespread, and Fine-Grained Question Results Ch...IRJET- Accomplishing Secure, Widespread, and Fine-Grained Question Results Ch...
IRJET- Accomplishing Secure, Widespread, and Fine-Grained Question Results Ch...
 
Periodic Auditing of Data in Cloud Using Random Bits
Periodic Auditing of Data in Cloud Using Random BitsPeriodic Auditing of Data in Cloud Using Random Bits
Periodic Auditing of Data in Cloud Using Random Bits
 
34- Fail Activity in Azure Data Factory.pptx
34- Fail Activity in Azure Data Factory.pptx34- Fail Activity in Azure Data Factory.pptx
34- Fail Activity in Azure Data Factory.pptx
 
Data Warehouse (ETL) testing process
Data Warehouse (ETL) testing processData Warehouse (ETL) testing process
Data Warehouse (ETL) testing process
 
Deliver Trusted Data by Leveraging ETL Testing
Deliver Trusted Data by Leveraging ETL TestingDeliver Trusted Data by Leveraging ETL Testing
Deliver Trusted Data by Leveraging ETL Testing
 
DB2 LUW Auditing
DB2 LUW AuditingDB2 LUW Auditing
DB2 LUW Auditing
 
Deep Dive on Amazon S3
Deep Dive on Amazon S3Deep Dive on Amazon S3
Deep Dive on Amazon S3
 
Enhancing Data Security in Cloud Storage Auditing With Key Abstraction
Enhancing Data Security in Cloud Storage Auditing With Key AbstractionEnhancing Data Security in Cloud Storage Auditing With Key Abstraction
Enhancing Data Security in Cloud Storage Auditing With Key Abstraction
 
OER UNIT 5 Audit
OER UNIT  5 AuditOER UNIT  5 Audit
OER UNIT 5 Audit
 
IRJET - Providing High Securtiy for Encrypted Data in Cloud
IRJET -  	  Providing High Securtiy for Encrypted Data in CloudIRJET -  	  Providing High Securtiy for Encrypted Data in Cloud
IRJET - Providing High Securtiy for Encrypted Data in Cloud
 
Etl testing strategies
Etl testing strategiesEtl testing strategies
Etl testing strategies
 
Big query
Big queryBig query
Big query
 
SAP BI 7 security concepts
SAP BI 7 security conceptsSAP BI 7 security concepts
SAP BI 7 security concepts
 
Test data management
Test data managementTest data management
Test data management
 
Quality Inspection.pdf
Quality Inspection.pdfQuality Inspection.pdf
Quality Inspection.pdf
 
Etl testing
Etl testingEtl testing
Etl testing
 
All You Need To Know About Big Data Testing - Bahaa Al Zubaidi.pdf
All You Need To Know About Big Data Testing - Bahaa Al Zubaidi.pdfAll You Need To Know About Big Data Testing - Bahaa Al Zubaidi.pdf
All You Need To Know About Big Data Testing - Bahaa Al Zubaidi.pdf
 

Mehr von BRIJESH KUMAR

Mehr von BRIJESH KUMAR (20)

53-Dataset Source and Sink Data flow in Azure Data Factory.pptx
53-Dataset Source and Sink Data flow in Azure Data Factory.pptx53-Dataset Source and Sink Data flow in Azure Data Factory.pptx
53-Dataset Source and Sink Data flow in Azure Data Factory.pptx
 
52- Source and Sink Data flow in Azure Data Factory.pptx
52- Source and Sink Data flow in Azure Data Factory.pptx52- Source and Sink Data flow in Azure Data Factory.pptx
52- Source and Sink Data flow in Azure Data Factory.pptx
 
51- Data flow in Azure Data Factory.pptx
51- Data flow in Azure Data Factory.pptx51- Data flow in Azure Data Factory.pptx
51- Data flow in Azure Data Factory.pptx
 
3- Azure Resource Group.pptx
3- Azure Resource Group.pptx3- Azure Resource Group.pptx
3- Azure Resource Group.pptx
 
1- Introduction of Azure Cloud.pptx
1- Introduction of Azure Cloud.pptx1- Introduction of Azure Cloud.pptx
1- Introduction of Azure Cloud.pptx
 
47- Web Hook Activity in Azure Data Factory.pptx
47- Web Hook Activity in Azure Data Factory.pptx47- Web Hook Activity in Azure Data Factory.pptx
47- Web Hook Activity in Azure Data Factory.pptx
 
46- Web Activity in Azure Data Factory.pptx
46- Web Activity in Azure Data Factory.pptx46- Web Activity in Azure Data Factory.pptx
46- Web Activity in Azure Data Factory.pptx
 
44- Filter Activity in Azure Data Factory.pptx
44- Filter Activity in Azure Data Factory.pptx44- Filter Activity in Azure Data Factory.pptx
44- Filter Activity in Azure Data Factory.pptx
 
43- Wait Activity in Azure Data Factory.pptx
43- Wait Activity in Azure Data Factory.pptx43- Wait Activity in Azure Data Factory.pptx
43- Wait Activity in Azure Data Factory.pptx
 
41- Scripts Activity in Azure Data Factory.pptx
41- Scripts Activity in Azure Data Factory.pptx41- Scripts Activity in Azure Data Factory.pptx
41- Scripts Activity in Azure Data Factory.pptx
 
39- Lookup Activity in Azure Data Factory.pptx
39- Lookup Activity in Azure Data Factory.pptx39- Lookup Activity in Azure Data Factory.pptx
39- Lookup Activity in Azure Data Factory.pptx
 
40 Stored Procedure Activity in Azure Data Factory.pptx
40 Stored Procedure Activity in Azure Data Factory.pptx40 Stored Procedure Activity in Azure Data Factory.pptx
40 Stored Procedure Activity in Azure Data Factory.pptx
 
38- Get Metadata Activity in Azure Data Factory.pptx
38- Get Metadata Activity in Azure Data Factory.pptx38- Get Metadata Activity in Azure Data Factory.pptx
38- Get Metadata Activity in Azure Data Factory.pptx
 
37- User Properties in Activity in Azure Data Factory.pptx
37- User Properties in Activity in Azure Data Factory.pptx37- User Properties in Activity in Azure Data Factory.pptx
37- User Properties in Activity in Azure Data Factory.pptx
 
36- Copy Activity Setting in Azure Data Factory.pptx
36- Copy Activity Setting in Azure Data Factory.pptx36- Copy Activity Setting in Azure Data Factory.pptx
36- Copy Activity Setting in Azure Data Factory.pptx
 
35- Copy Activity in Azure Data Factory.pptx
35- Copy Activity in Azure Data Factory.pptx35- Copy Activity in Azure Data Factory.pptx
35- Copy Activity in Azure Data Factory.pptx
 
33- If Condition Activity in Azure Data Factory.pptx
33- If Condition Activity in Azure Data Factory.pptx33- If Condition Activity in Azure Data Factory.pptx
33- If Condition Activity in Azure Data Factory.pptx
 
31- Execute Pipeline Activity in Azure Data Factory.pptx
31- Execute Pipeline Activity in Azure Data Factory.pptx31- Execute Pipeline Activity in Azure Data Factory.pptx
31- Execute Pipeline Activity in Azure Data Factory.pptx
 
30- Append Variable Activity in Azure Data Factory.pptx
30- Append Variable Activity in Azure Data Factory.pptx30- Append Variable Activity in Azure Data Factory.pptx
30- Append Variable Activity in Azure Data Factory.pptx
 
27- System Funciton in Azure Data Factory.pptx
27- System Funciton in Azure Data Factory.pptx27- System Funciton in Azure Data Factory.pptx
27- System Funciton in Azure Data Factory.pptx
 

KĂŒrzlich hochgeladen

The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptx
heathfieldcps1
 

KĂŒrzlich hochgeladen (20)

REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptx
 
General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual  Proper...General Principles of Intellectual Property: Concepts of Intellectual  Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...
 
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptxHMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
 
SOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning PresentationSOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning Presentation
 
FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024
 
Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)
 
Unit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptxUnit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptx
 
Single or Multiple melodic lines structure
Single or Multiple melodic lines structureSingle or Multiple melodic lines structure
Single or Multiple melodic lines structure
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 
Interdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptxInterdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptx
 
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
 
Google Gemini An AI Revolution in Education.pptx
Google Gemini An AI Revolution in Education.pptxGoogle Gemini An AI Revolution in Education.pptx
Google Gemini An AI Revolution in Education.pptx
 
Food safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdfFood safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdf
 
The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptx
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.ppt
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
 
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxBasic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
 
Understanding Accommodations and Modifications
Understanding  Accommodations and ModificationsUnderstanding  Accommodations and Modifications
Understanding Accommodations and Modifications
 
Making communications land - Are they received and understood as intended? we...
Making communications land - Are they received and understood as intended? we...Making communications land - Are they received and understood as intended? we...
Making communications land - Are they received and understood as intended? we...
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptx
 

32- Validation Activity in Azure Data Factory.pptx

  • 1. 32- Azure Data Factory  Validation Activity in azure data factory  Demo- Validation Activity Welcome in BPCloudLearningInHindi 1
  • 2. Validation Activity: 2 Welcome in BPCloudLearningInHindi You can use a Validation in a pipeline to ensure the pipeline only continues execution once it has validated the attached dataset reference exists, that it meets the specified criteria, or timeout has been reached In Azure Data Factory, the Validation activity is used to verify that the data in a source data store meets certain criteria before it is processed further. This activity can help ensure data quality and prevent errors downstream. The Validation activity can be used to perform a wide range of data validation checks, including: 1. Checking that the data confirms to a specific schema or data model. 2. Verifying that required columns or fields are present and contain valid data. 3. Checking for duplicate or missing data. 4. Ensuring that data falls within acceptable ranges or thresholds. 5. Validating that data meets certain business rules or constraints.
  • 3. 3 Welcome in BPCloudLearningInHindi To use the Validation activity, you need to define a validation rule using either a JSON schema or an Azure Data Factory expression. The validation rule specifies the criteria that the data must meet in order to pass validation. Overall, the Validation activity is a useful tool for ensuring data quality and accuracy in Azure Data Factory pipelines. The "Validation" activity is an important tool for ensuring data quality and accuracy in your Azure Data Factory pipelines, and can be used in conjunction with other activities such as the "Copy" and "Transform" activities to create a comprehensive data processing workflow.