7. Storage
Data Marts
Extract, Transform,
and Load (ETL)
Middleware
Server(s)
Data
Warehouse
Design and Visualization
Data Cubes and Tabular Models
E
T
L
Reporting
Server(s)
BI and Designer
Clients
Source
data
8.
9. The conundrum
Business
Knows the requirement
A little about technology
Frustration leads to governance
violations
I’ll get it myself!
Knows the technology
A little about the business
Frustrations, leads to invalid
deployments of technology
I’ll just build it!
10.
11. “Traditional” BI
(Waterfall)
• Monolithic
projects
• Expensive tools
• Few users
(execs)
• Results come
over years
Agile BI
(Agile/Lean)
• Small targeted
projects
• Inexpensive
tools
• Many users
(execs & field)
• Quick targeted
results
12.
The Forrester Wave™: Agile Business Intelligence
Platforms, Q3 2014
21. SQL Server Data Tools
Power View
Power Q&A
Power Query
PerformancePoint Services
PowerPivot for SharePoint
Excel Services
22. On Premises Cloud
Power BI PerformancePoint
SharePoint
Mashup
Azure VMs
23. ONE DOES NOT SIMPLY
RETURN HIS RAW DATA FROM THE DATABASE
24. EE
Storage
SQL Server DB
SQL Server
Integration Services
(SSIS)
SharePoint (with)
• Excel Services
• PowerPivot for SharePoint
• SSRS SharePoint Mode
• PerformancePoint
SQL Server
DB
Design and Visualization
SQL Server Analysis
Services
Multidimensional and
Tabular modes
L
SQL Server
Reporting
Services
(SSRS)
Excel
SQL Data Tools
Report Builder
3rd party tools
ETL
E
T
Source
data
25. Worksheets
Tabular Data Model
(xVelocity)
Pivot Charts and
Tables
Power View
(Analytic reports)
Power Map
(Geospatial and time series data)
Power Pivot
(Model design)
Power Query (ETL)
Power Pivot Import (EL)
26. Power Pivot Worksheets
• Pivot Tables and Charts
• Power View
Data Marts and
other
Data Cubes and
Tabular Models
Standard Worksheets
• Pivot Tables and Charts
PerformancePoint Scorecards and KPIs
PerformancePoint Reports
• Analytic Charts and Grids
• Decomposition trees
SQL Server Reporting Services Reports
• Standard
• Power View
Show of Hands –BI Projects?
About me.
President & CoFounder of UVI
BI PAC member for +3 years
Guide the projects
Every BI talk needs to start with a graph
We used to call if decision support. –I still believe in that.
Business Intelligence is a bit of an oxymoron
Instead of querying our source systems directly, we want to take our data and move it into Data Warehouses and data marts, which are optimized for the sorts of analysis that we want to perform. This is done through an ETL operation.
CLICK
The data is extracted from the source system, CLICK transformed into the shape we need it, CLICK then loaded into the data warehouse. CLICK Other ETL processes or cube process will load the data into any necessary marts, cubes or models.
From here various servers and client will access the data, usually from the data marts of cubes, but occasionally from the warehouse directly.
So how does this translate to the Microsoft stack? There are two ways. The Enterprise, or “classic” BI method, or the Power (personal) method.
Gartner does a good job of describing the continuum
Natural tension between business and IT.
Nephew -Actuary Example. White Lab Coat people.
Manufacturer –the problem they were trying to solve –MRP
-Sales data with manufacturing velocity to minimize waste.
-worked for months at great expense.
-many interviews
-developed exactly the reports they asked for –pixel perfect
-CEO has a completely different need. Start over.
We’ve changed our entire model b/c of this. Going to talk about the agile approach and give some practical examples.
Customers are empowered to switch at will, have ever changing needs, and business needs to respond.
Business Agility –Grassroots approach. People in the trenches able to make decisions. People who are able to touch the customer
Information Agility –Empowerment through easy access and BI tools
Most organizations are nowhere near this. So how do you get there?
Do not skip this step. Everyone is busy and need to be grounded to your focus.
Need to constantly remind people why you are doing this
Pitch deck creates a framework from where to talk from
How does your sponsor want to com
Decisions makers need just enough information to make a decision. Kill slides that don’t work.
Make them tell you what keeps them up at night.
Help me understand the pain.
Need to get them to open-up a vein.
Get ugly early
This is a whole topic in itself, I just want to draw it out as a discussion point.
Understand what kind of ore you are mining for before you start digging holes
Existing tools –this can be a bottle neck. Will discuss how you get around it.
Be brutality honest. Lying to yourself will only hurt you long term.
Only build what is necessary to bring customer value
IT will want to build everything. Stay focused
There are many products that make up the BI stack and many of them will be used one way or another in any dashboard project. What’s important is to understand what they are, and how they fit. In some cases, there are multiple ways to achieve the same goal, so the implications of using any should be understood.
The good news is that these all fit together into a few product groups. We’ll got through these today.
Keep in mind that in every case, SharePoint is the preferred publishing platform
SSRS is the only exception – it can be hosted on its own, but significant advantages are there for SharePoint
These approaches are not mutually exclusive, and in some cases can’t be. For example, SharePoint Mashups will depend on content from any or all of Excel, SSRS, PerformancePoint, or other SharePoint content.
We’re going to walk through all of these today with the exception of Azure VMs. Azure VMs take the same approach as on-prem, they’re just hosted in the cloud – the “private cloud”
No matter which approach we take, they all have the same basic underpinnings. These are based on fundamental BI concepts that need to be understood. So let’s dive in.
First and foremost, there is one fundamental Business Intelligence rule that must be adhered to.
Business Intelligence is all about the data, but that doesn’t mean that you just wire up Excel to source data and start Extracting (although far too many people do). This is bad for a number of reason
- Security – data level access to production data
- Usability – difficult to understand constructs (Great Plains anyone?)
- Performance – reporting against the production data concentrates the load.
- Organization – data optimized for transactions, not reporting
Starting with the classic method, SQL Server Integration Services is the tool that performs our ETL.
SQL Server Database Engine is used for the storage of the data warehouses and data marts
SQL Server Analysis Services is the multidimensional engine (traditional OLAP cubes) and now is the engine for enterprise tabular models (xVelocity).
SSRS is the traditional server engine for serving reports, and can be deployed either standalone, or through SharePoint.
These tools all ship on SQL server media, but some (SSRS and PowerPivot for SharePoint) may be deployed to SharePoint
Clients of this infrastructure may be servers themselves, or designers and Power Users. Consuming tools include Excel, SQL Server Data Tools, Excel Services, PowerPivot for SharePoint, or a host of other tools.
Recently, there has been a lot of work in the Personal BI space – so how does that compare to this approach? Fundamental BI concepts still apply.
To start with, we have an Excel Workbook. Excel is the personal BI client from Microsoft. As of the 2010 version (through an add-in), or Excel 2013 directly we have access to an embedded xVelocity data model.
CLICK
Using the PowerPivot add-in (needs to be enabled) we can import data directly from the source data systems, and then manipulate the structure, but the data is read only. It can be refreshed, but not edited. Really, we have the E and the L of an ETL system.
CLICK
More recently, Power Query has been introduced. It’s a part of Power BI, but in this context it’s just a free Excel add-in that brings more elegance to the import. It puts the T back into ETL on the personal side. It has a host of other features, and different data source options, but that’s fundamentally what it is. Power Query can also load data directly into the workbook, into the model, or both.
CLICK
Once the data has been loaded it can be consumed through a number of Excel tools. The traditional multi dimensional tools are the Pivot chart and Pivot table, but we now also have Power View for analytical reporting, and Power Map for geospatial analysis. PowerPivot is the model editor.
Be careful with Power Query. It cant be automatically refreshed. Yet.
All of these approaches, both enterprise and personal converge through SharePoint and dashboards.
Within SharePoint, we can publish reports and data models, and establish connections to the relevant back end systems. These components can then be used to construct dashboards, or used on their own as dashboards.
Dashboards can contain, but are not necessarily limited to
Worksheets and worksheet components through Excel Services, either directly connected or via PowerPivot
SSRS Reports
PerformancePoint scorecards and KPIs
PerformancePoint reports
People don’t even know why
MVP – helps users re-define their requirements
People continue to do tasks that no longer add value to the project.
Past – Economics is the art of predicting the past. Useful, but like driving while looking in the rear view mirror
Present – KPI, real time vs real enough time
Future – Target story about father that used Visa card and got a coupon for baby formula because his daughter had bought a pregnancy test kit with his credit card…
As an example – what documents are relevant to a manufacturing operation?
We generally focus on search, and attempt to derive relevance from usage, term matching etc, but why not be explicit as at our customer?
Mashup production ERP data with document data to present a report in context of what is important on a particular day
Problems don’t care how they solve them.
Right tool, right problem
What is the minimum I can do to build customer value.