6. Master Package At midnight every day the Master Package runs all the other packages contained in the ETL container. The ETL container has execute package tasks; each package is deployed to the SSIS stored packages. Upon completion it launches the Maintenance container comprising of a Back-up, Rebuild Index and Shrink database tasks. Separate failure notices are email if either container fails and separate progress email are sent when each container succeeds.
7. Master Package The ETL container has execute package tasks; each package has been deployed to the SSIS stored packages area of the server.
8. Employee Master Package Control Flow This package populates or updated a SQL Server table and notified the relevant parties whether it was successful in its tasks. Conditional record counts are captured by variables used to construct the email message reporting Error rows, New Rows and Updated rows if the package is successful. If the package fails then an email is sent reporting that this package failed.
9. Employee Master Package Data Flow The data is imported from an Excel Source, record count is captured by RawDataRows variable, converted to DB data types, columns combined to create FullName, EmployeeID looked-up to find matching records and then split. In order of condition evaluation: Records containing any null attribute values are routed as Error, record count is captured by ErrorRows variable and exported to a flat file error log. Records with EmployeeID that do not match in existing staging table are routed as New, record count is captured by InsertedRows variable and inserted into staging table. Records with matching EmployeeID are routed as Changed, record count is captured by UpdatedRows variable and all attributes are updated in staging table.
10. Project Time Sheets Package Control Flow This package reads multiple CSV files in a controlled folder and populates the JobTimeSheets table in the ETL staging area on the server, then summarizes all the rows counts captured by the variables using a VB script, creating a new set of row count variables for cumulative totals for all sheets processed in that package execution. Summary rows counts are composed into a message/progress report and emailed with attached error log.
11. Project Time Sheets Package Control Flow The Script Editor properties show the incoming read only parameters and the outgoing parameters with cumulative total rows counts. This is the VB code used to calculate cumulative totals for each type of rows count. These calculations are performed with each loop through individual employee time data sheet contained in the time folder. Final tallies are emailed in progress report. Public Sub Main() Dim InsertedRows As Integer = CInt (Dts.Variables( "InsertedRows" ).Value) Dim ErrorRows As Integer = CInt (Dts.Variables( "ErrorRows" ).Value) Dim RawDataRows As Integer = CInt (Dts.Variables( "RawDataRows" ).Value) Dim UpdatedRows As Integer = CInt (Dts.Variables( "UpdatedRows" ).Value) Dim InsertedRows_fel As Integer = CInt (Dts.Variables( "InsertedRows_fel" ).Value) Dim ErrorRows_fel As Integer = CInt (Dts.Variables( "ErrorRows_fel" ).Value) Dim RawDataRows_fel As Integer = CInt (Dts.Variables( "RawDataRows_fel" ).Value) Dim UpdatedRows_fel As Integer = CInt (Dts.Variables( "UpdatedRows_fel" ).Value) Dts.Variables( "InsertedRows" ).Value = InsertedRows + InsertedRows_fel Dts.Variables( "ErrorRows" ).Value = ErrorRows + ErrorRows_fel Dts.Variables( "RawDataRows" ).Value = RawDataRows + RawDataRows_fel Dts.Variables( "UpdatedRows" ).Value = UpdatedRows + UpdatedRows_fel Dts.TaskResult = Dts.Results.Success End Sub End Class
12. Project Time Sheets Package Data Flow Like the previous package this package also imports, converts, counts, matches FKs to the existing staging tables (Employees, JobMaster & JobTimeSheets). The data is also checked for missing information and logged. Error logic is: ISNULL(ConvertedEmployeeID) || ISNULL(ConvertedWorkDate) || ISNULL(ConvertedJobNumber) || ISNULL(ConvertedRegHours) || ISNULL(MatchedEmployeesEmployeePK) || ISNULL(MatchedJobMasterJobMasterPK) || (MatchedJobMasterJobClosed == TRUE && MatchedJobMasterJobClosedDate < ConvertedWorkDate)
13.
14. Data Source View This DSV was created in BIDS, it shows the 4 fact tables