This document discusses error handling for an MKT_ETL_Load job. It analyzes how structural changes in fields between the MSBI and EBS sources cause failures, mainly due to NULLs, duplicates and invalid date formats. It also provides examples of four cases where mismatches between the source and target schemas could cause failures: 1) NULL values, 2) data types, 3) primary keys, 4) date formatting. The document recommends implementing error handling by adding a data conversion task before the target to redirect invalid records to a secondary destination like a flat file or table.

1. Error Handling Approach for MKT_ETL_Load
Our key analysis is mainly focusing on structural changes in MSBI fields against EBS source as job failures mostly
happened due to restriction of NULLs, Duplicates and Invalid format mainly on date fields.
Below Lineage showcases on how the load packages pull data from EBS source into ATD warehouse
Load Transform Extract
loadDim_ProductOrganizationBridge transformDim_ProductOrganizationBridge extractPRDINV_MTL_SYSTEM_ITEMS_B
loadFact_ComplianceSnapshot loadDim_Program
transformFact_ComplianceSnapshot extractPRDAR_HZ_CUST_ACCOUNTS
extractPRDAR_HZ_CUST_ACCT_SITES_ALL
extractPRDAR_HZ_CUST_SITE_USES_ALL
extractPRDXXATDAR_XXATDAR_EARNINGS_SUMMAR
Y_STG
extractPRDXXATDAR_XXATDAR_SITE_MKT_PROGRAM
loadFact_ProductEligibility loadDim_Program
transformFact_ProductEligibility(disabled)
loadFact_ProgramMembershipSum
mary loadDim_Program
transformFact_ProgramMembershipSum
mary
extractPRDXXATDAR_XXATDAR_EARNINGS_SUMMAR
Y_STG
loadFact_ProgramPayment loadDim_Program
transformFact_ProgramPayment extractPRDAR_RA_CUSTOMER_TRX_ALL
extractPRDXXATDAR_XXATDAR_ASPIRE_PAYMENT_ST
G
extractPRDXXATDAR_XXATDAR_SITE_MKT_PROGRAM
loadREF_MarketingComplianceProgr
ams
transformFact_MarketingComplianceProg
rams extractSharepoint_MarketingCompliancePrograms
Based on below key points We noted the appropriate fields that may cause the job to fail.
Case 1: If oracle source has accepting null and MSBI has not null in it.
Case 2: If oracle source has datatype of varchar/float and MSBI has int.
Case 3: If MSBI has Primary Key but not in oracle.
Case 4: If date fields in MSBI has TO_CHAR, TO_DATE, CAST & CONVERT functions used.
Load Transform Extract
loadDim_ProductOrganizationBridge
transformDim_ProductOrganizationBrid
ge extractPRDINV_MTL_SYSTEM_ITEMS_B
ProductID-PK INVENTORY_ITEM_ID NOT NULL NUMBER
OrganizationID-PK ORGANIZATION_ID NOT NULL NUMBER

2. loadFact_ComplianceSnapshot loadDim_Program
transformFact_ComplianceSnapshot extractPRDAR_HZ_CUST_ACCOUNTS
extractPRDAR_HZ_CUST_ACCT_SITES_ALL
extractPRDAR_HZ_CUST_SITE_USES_ALL
extractPRDXXATDAR_XXATDAR_EARNINGS_SUMM
ARY_STG
extractPRDXXATDAR_XXATDAR_SITE_MKT_PROGR
AM
[SnapshotDate_id] [int] NOT NULL,-PK
[SnapshotDate_dt] [datetime] NOT
NULL,-PK
[Customer_id] [int] NOT NULL,-PK
[Program_cd] [varchar](50) NOT NULL,-
PK PROGRAM_CODE VARCHAR2(50)
[Program_id][int]NOT NULL,-PK [Program_nm] [varchar](255)NOT NULL, PROGRAM_NAME VARCHAR2(240)
[Organization_id] [int] NOT NULL,
[Customer_cd][varchar](30)NOT NULL,-
PK ACCOUNT_NUMBER VARCHAR2(30)
[AssignedSalespersonUser_id] [int]
NOT NULL,
[Location_cd][varchar](40) NOT NULL,-
PK LOCATION_NUMBER VARCHAR2(40)
[PeriodGoalAllLocations][int]NOT
NULL, [PeriodGoalAllLocations][int]NULL,
[YearGoalAllLocations][int]NOT NULL, [YearGoalAllLocations][int]NULL,
[ProgramStartDate] [datetime]NOT
NULL,
[ProgramStartDate] [datetime]NOT
NULL-PK
loadREF_MarketingComplianceProgra
ms
transformFact_MarketingCompliancePr
ograms extractSharepoint_MarketingCompliancePrograms
loadFact_ProgramPayment loadDim_Program
transformFact_ProgramPayment extractPRDAR_RA_CUSTOMER_TRX_ALL
extractPRDXXATDAR_XXATDAR_ASPIRE_PAYMENT
_STG
[Invoice_cd] NOT NULL, PK [INVOICENO] NULL
[Invoice_line_no] NOT NULL, PK [LINE_NUMBER NULL
[Customer_cd] NOT NULL [CUSTOMER_NUMBER] NULL
[Location_cd] NOT NULL [SHIP_TO_LOCATION_NUMBER] NULL
[Program_cd] NOT NULL [PROGRAM_CODE] NULL
[Units] NOT NULL [UNITS] NULL
[Sales] NOT NULL [SALES_TOTAL] NULL
[Payment] NOT NULL [PAYMENT] NULL
[CreditAmount] NOT NULL [CREDIT_AMOUNT] NULL
[DebitAmount] NOT NULL [DEBIT_AMOUNT] NULL
[CreationDate] NOT NULL [CREATION_DATE] NULL
extractPRDXXATDAR_XXATDAR_SITE_MKT_PROGR
AM
loadFact_ProgramMembershipSumma
ry loadDim_Program
transformFact_ProgramMembershipSu
mmary
extractPRDXXATDAR_XXATDAR_EARNINGS_SUMM
ARY_STG
ProgramStartDate] NOT NULL,PK PROGRAM_START_DATE NULL
Program_cd NOT NULL, PK PROGRAM_CODE NULL
Customer_cd NOT NULL, PK ACCOUNT_NUMBER NULL
[Location_cd] NOT NULL, PK LOCATION_NUMBER NULL
[MembershipNumber] NOT NULL,PK MEMBERSHIP_NUMBER NULL
MembershipType NOT NULL,PK MEMBERSHIP_ID NULL

3. How to Implement Error Handling
In order to implement error handling on SSIS(ETL), We must need to point as "Row redirect" on required fields
under newly created Data Conversion task prior to the target, it diverts the bad records into secondary
destination as flat file or table.