This project involved designing a SQL Server 2005 database for a construction company, updating the database from various sources using Integration Services, creating cubes in Analysis Services, developing reports in Reporting Services, and setting up dashboards and KPIs using PerformancePoint and SharePoint. Key aspects of the project included ETL packages to load and transform data, MDX queries against the cubes, parameterized reports published to a report server, and dashboards in PerformancePoint to visualize key metrics. The goal was to provide integrated business intelligence capabilities for analyzing employee, project, and financial data.
1. Portfolio for Charles Tontz
Business Intelligence Masters Program
SQL Server 2005
Integration/Analysis/Reporting Services
Project
Introduction: Project Summary
Design and build a SQL Server 2005 database to track employee and customer information,
timesheet and labor rate data as well as job order information, job materials, and customer
invoices for a simulated construction company called AllWorks, Inc. Update the database from
various Excel and flat file sources. Create four cubes in Analysis Services and design and
implement numerous MDX queries to produce datasets sliced and sorted various ways. Create
five Key Performance Indicators (KPI’s) which are then linked to Excel 2007. Setup up several
Excel 2007 workbooks to provide the ability to use data in the cubes in almost any view, including
pertinent KPI’s so that one instantly knows if the trend for various categories exceeds a
predefined goal, is within an acceptable range, or is falling short. Produce reports from the
provided specifications and put them on the report server where anyone with access rights can
open and view the reports, including two with drop down selections for category and time
periods. Generate sample reports to a PDF file and save them to a file on the drive.
Audience:
This project was a collaboration of IT and department managers, and directly responsible to one
of the senior executives.
2. Project Goals:
** Integration Services **
Review source data from three different source types and build packages that process and
validate the data. This data was used to update the database with any new or changed records.
Emails were generated with the results for each package, including Error Event Handlers if any
part of the process failed, and writing error logs where appropriate. Otherwise, the email
indicated successful completion with the total number of records processed, inserted, and
updated, and any error records written to an error file. A Master package was created as a
container for the eight packages and was scheduled to run each night. Two other packages were
created to handle nightly database backups, then re-index all files and shrink the database.
SSIS Project db sample Package Data Flow
Eleven packages were created for the entire project. This package grabs the most recent client data
from and Excel file, converts and validates it, then identifies new, existing, and error records, and
proceeds accordingly to insert or update the SQL Server database, and write any errors to a file
which can be reviewed by the appropriate person for correction
.
(Click on image to view -- then again on enlarged image, depending on your monitor settings)
VB Script Task
3. One of the packages required reading multiple Excel files and totalling record counts for all files. To allow this,
the following VB code was entered in a task to keep track of the totals:
VB Script for tracking total counts in a For Each Loop Container:
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
' The execution engine calls this method when the task executes.
' To access the object model, use the Dts object. Connections, variables,
events,
' and logging features are available as static members of the Dts class.
' Before returning from this method, set the value of Dts.TaskResult to
indicate success or failure.
'
' To open Code and Text Editor Help, press F1.
' To open Object Browser, press Ctrl+Alt+J.
Public Sub Main()
Dim InitialRecordCount As Integer =
CInt(Dts.Variables(quot;InitialRecordCountquot;).Value)
Dim UpdateRecordCount As Integer =
CInt(Dts.Variables(quot;UpdateRecordCountquot;).Value)
Dim ErrorRecordCount As Integer =
CInt(Dts.Variables(quot;ErrorRecordCountquot;).Value)
Dim InsertRecordCount As Integer =
CInt(Dts.Variables(quot;InsertRecordCountquot;).Value)
Dim InitialRecordCount_fel As Integer =
CInt(Dts.Variables(quot;InitialRecordCount_felquot;).Value)
Dim UpdateRecordCount_fel As Integer =
CInt(Dts.Variables(quot;UpdateRecordCount_felquot;).Value)
Dim ErrorRecordCount_fel As Integer =
CInt(Dts.Variables(quot;ErrorRecordCount_felquot;).Value)
Dim InsertRecordCount_fel As Integer =
CInt(Dts.Variables(quot;InsertRecordCount_felquot;).Value)
Dts.Variables(quot;InitialRecordCountquot;).Value = InitialRecordCount +
InitialRecordCount_fel
Dts.Variables(quot;UpdateRecordCountquot;).Value = UpdateRecordCount +
UpdateRecordCount_fel
Dts.Variables(quot;ErrorRecordCountquot;).Value = ErrorRecordCount +
ErrorRecordCount_fel
Dts.Variables(quot;InsertRecordCountquot;).Value = InsertRecordCount +
InsertRecordCount_fel
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
4. The variables are initialized, and the _fel (for each loop) variables increment the
Initial/Update/Error/Insert RecordCount variables as each file is processed. The
accumulated totals are reported via email when the package completes all processing
tasks.
** Analysis Services **
The next piece of the project involved creating four cubes in Analysis Server, with a
sample structure represented below. This cube (a star-schema) was created to allow
analysis of job categories by just about any measure of which one could conceive.
Several calculated members were added to provide total cost, open invoice amounts,
etc.
(Click on image to view -- then again on enlarged image, depending on your monitor settings)
This is one example of the cube as viewed from Excel. Dimensions and Measures are loaded into
an Excel 'Pivot Table' for analysis. The data can be sorted by any column, even the stoplights, for
review and analysis. Rows, columns, and values can be changed at will, allowing enormous
flexibility, providing aggregation at higher levels with supplied drill-down to detail levels, if
needed.
(Click on image to view -- then again on enlarged image, depending on your monitor settings)
5. Sample MDX query:
-- Chuck Tontz
-- 7/09/2008
-- ProjectOverheadView cube: Overhead by category for Q3 & Q4 of 2005, with Change Percentage
WITH
MEMBER [Overhead Amount] AS
[Measures].[Weekly Over Head], format_string='currency'
MEMBER [Change Pct] AS
IIF(([Overhead Amount],[FY Qtr].PrevMember) = NULL,
NULL,
([Overhead Amount] - ([Overhead Amount],[FY Qtr].PrevMember))
/ ([Overhead Amount],[FY Qtr].PrevMember)), format_string = 'percent'
SELECT {
([Qry Overhead Summary].[FY Qtr].&[2005 Q3], [Overhead Amount])
,([Qry Overhead Summary].[FY Qtr].&[2005 Q4],[Overhead Amount])
6. , ( [Qry Overhead Summary].[FY Qtr].&[2005 Q4], [Change Pct])
} ON COLUMNS,
NON EMPTY [Overhead].[Description].Members ON ROWS
FROM ProjectOverheadView
Results: Overhead amounts for current and prior quarter with the change percentage.
2005 Q3 2005 Q4 2005 Q4
Overhead Overhead Change
Amount Amount Pct
All $7,923.24 $8,706.71 9.89%
ACCOUNTANTS FEES (null) $140.00 (null)
ADVERTISING (null) $44.00 (null)
ESTIMATED (null) $216.00 (null)
TAX/QTRLY235
EXTRA: FUEL $537.12 $528.71 -1.57%
HEALTH INSURANCE $4,163.40 $3,577.77 -14.07%
LIABILITY INSURANCE (null) $38.00 (null)
MISC EQUIPMENT (null) $44.00 (null)
MOBIL PHONES (null) $86.00 (null)
RENT (null) $104.00 (null)
TRUCKS $3,222.72 $3,712.23 15.19%
WORKMANS COMP (null) $216.00 (null)
** Reporting Services **
Six reports were produced from existing specification requirements. These included a
simple table-style report from a SQL Server 2005 database, with the remaining reports
using datasets from the cubes that were built in Analysis Services. Several of these
were graphs, including one with a graph and summary table. These were then
published to the network ReportServer so that anyone with access rights could view
the report, and print to a PDF file, or several other formats, including Excel, if needed.
7. This report is an example of a table report with multiple parameters available to the
user. Specifically, the report runs Timesheets for the selected Employee, with a
from/to date range, summarized by week, by job. It was created from an OLAP cube in
the data warehouse. (If the total hours for each job/week look suspicious, it's because
the fictitious data for this report is repetitive for some periods. If you view following
pages in the PDF version below, it becomes apparent that the total job hours is
not 10.5 times the hours worked by the employee.)
(Click on image to view -- then again on enlarged image, depending on your monitor settings)
The MDX code is generated automatically in BIDS, but the report dataset was modified after
creation to add the parameters.
Report Dataset:
SELECT NON EMPTY { [Measures].[Total Labor], [Measures].[Hoursworked] } ON COLUMNS,
NON EMPTY { (STRTOMEMBER(@FromTimeDay, CONSTRAINED) : STRTOMEMBER(@ToTimeDay,
CONSTRAINED))* [Job Master].[Description].[Description] }
ON ROWS FROM [Project Labor Cube]
where STRTOSET(@parmEmployee)
List Employees for parameter selection:
WITH MEMBER [Measures].[ParameterCaption] AS '[Employees].[Full
Name].CURRENTMEMBER.MEMBER_CAPTION' MEMBER [Measures].[ParameterValue] AS '[Employees].
8. [Full Name].CURRENTMEMBER.UNIQUENAME' MEMBER [Measures].[ParameterLevel] AS '[Employees].
[Full Name].CURRENTMEMBER.LEVEL.ORDINAL' SELECT {[Measures].[ParameterCaption], [Measures].
[ParameterValue], [Measures].[ParameterLevel]} ON COLUMNS , [Employees].[Full Name].Children ON ROWS
FROM [Project Labor Cube]
FromTimeDay dataset:
WITH MEMBER [Measures].[ParameterCaption] AS '[Project Labor View].[Week End
Date].CURRENTMEMBER.MEMBER_CAPTION' MEMBER [Measures].[ParameterValue] AS '[Project Labor
View].[Week End Date].CURRENTMEMBER.UNIQUENAME' MEMBER [Measures].[ParameterLevel] AS
'[Project Labor View].[Week End Date].CURRENTMEMBER.LEVEL.ORDINAL' SELECT {[Measures].
[ParameterCaption], [Measures].[ParameterValue], [Measures].[ParameterLevel]} ON COLUMNS , [Project Labor
View].[Week End Date].Children ON ROWS FROM [Project Labor Cube]
ToTimeDay dataset:
WITH MEMBER [Measures].[ParameterCaption] AS '[Project Labor View].[Week End
Date].CURRENTMEMBER.MEMBER_CAPTION' MEMBER [Measures].[ParameterValue] AS '[Project Labor
View].[Week End Date].CURRENTMEMBER.UNIQUENAME' MEMBER [Measures].[ParameterLevel] AS
'[Project Labor View].[Week End Date].CURRENTMEMBER.LEVEL.ORDINAL' SELECT {[Measures].
[ParameterCaption], [Measures].[ParameterValue], [Measures].[ParameterLevel]} ON COLUMNS , [Project Labor
View].[Week End Date].Children ON ROWS FROM [Project Labor Cube]
All reports were then deployed to the intranet report server. (The creative naming scheme
was simply to comply with the report specifications. Normally, a more useful name is
provided.)
(Click on image to view -- then again on enlarged image, depending on your monitor
settings)
9. The same report after being exported to a PDF file:
(Click on link to view) Report5.pdf
These reports can be made available to groups, or specific individuals, as needed.
** SharePoint Server **
The SharePoint project was an excercise in familiarity, rather than mastery. It quickly
becomes clear why a SharePoint Administrator is a requirement! The only requirement was
to set up a dedicated reports page and KPI List page and publish reports there. A very
convenient feature of SharePoint is that it allows anyone to output the report as an Adobe
PDF document or an Excel workbook.
(Click on image to view -- then again on enlarged image, depending on your monitor
settings)
10. Any of the listed reports can be viewed in browser, or exported to a PDF or Excel file,
among other choices.
** PerformancePoint Server **
PerformancePoint Server's Dashboard Designer provides an exellent tool for designing
dashboards and business scorecards which can be pushed out to a SharePoint page. As the
attached example pages show, summarized data can be expressed several ways, and still
allow for drill-down capability in most cases. The project specified four reports with
requirements for charts, summary data reports, KPI's and business scorecards. All reports
were created within Dashboard Designer as an analytic chart or grid, but reports created in
SQL Server Reporting Services can be imported as well.
(Click on images to view -- then again on enlarged image, depending on your monitor
settings)
ScoreCards:
Report of Materials used summarized quarterly by selected client:
11. Chart of weekly overhead for multiply-selectable accounts:
Labor hours and $ total for Top Ten jobs and Top Five workers by selected quarter: