A great looking, relevant, informative dashboard for the executive is the ultimate end goal for most Business Intelligence projects. Microsoft has all of the tools that are necessary to get data from your transactional systems into an effective dashboard. What is less clear is to how to go about achieving it. Which tools should you use? As with most things, we have different tools for different requirements, and it isn't always clear as to how they match up.
This session will focus on what the various reporting and dashboarding tools from Microsoft can do for you, where you should use them, and how to get them working for you. Both on premises and cloud based scenarios will be discussed. At the conclusion of this session, you should have a fundamental understanding of the Microsoft products in this space fit together, including Excel, Reporting Services, SharePoint, Power View, and PerformancePoint.
13. ONE DOES NOT SIMPLY
RETURN HIS RAW DATA FROM THE DATABASE
14. Data Marts
Extract, Transform,
and Load (ETL)
Middleware
Server(s)
Data
Warehouse
Storage Design and
Visualization
Data Cubes and
Tabular Models
E
T
L
Reporting
Server(s)
BI and
Designer
Clients
Source
data
15. EE
SQL Server DB
SQL Server
Integration
Services (SSIS)
SharePoint (with)
• Excel Services
• PowerPivot for SharePoint
• SSRS SharePoint Mode
• PerformancePoint
SQL Server
DB
Storage Design and Visualization
SQL Server Analysis
Services
Multidimensional and
Tabular modes
L
SQL Server
Reporting
Services
(SSRS)
Excel
SQL Data Tools
Report Builder
3rd party tools
ETL
E
T
Source
data
16. Worksheets
Tabular Data Model
(xVelocity)
Pivot Charts and
Tables
Power View
(Analytic reports)
Power Map
(Geospatial and time series data)
Power Pivot
(Model design)
Power Query (ETL)
Power Pivot Import (EL)
17. Power Pivot Worksheets
• Pivot Tables and Charts
• Power View
Data Marts and
other
Data Cubes and
Tabular Models
Standard Worksheets
• Pivot Tables and Charts
PerformancePoint Reports
• Analytic Charts and Grids
• Decomposition trees
SQL Server Reporting Services Reports
• Standard
• Power View
PerformancePoint Scorecards and
KPIs
18.
19. Power BI
Excel, Power X, Mobile, Data Management
Gateway
Dashboard tools
PerformancePoi
nt
(Dashboard
Designer)
SharePoint
(Pages and
Filters)
On Premises Office 365
20. Business
Intelligence Center
2007 - Excel and Connections
2010 - PerformancePoint
2013 - PerformancePoint and
Power Pivot
Reports
Excel and Connections
Web Part Pages
SharePoint KPIs
A dashboard is a key piece of equipment. There’s one in this Formula car – even though there’s no dash. Driver is interested in a few things. In order, they are his location, his speed, his gear, and the condition of the critical parts of his vehicleFrom the drivers viewport the “UI” prioritizes these items in order of importanceAt a glance, these things can be discerned But dashboards mean different things to different people
This is an old one from an old car, and there’s nothing wrong with it. At a glance, you can determine the speed of the vehicle, how much fuel is available, and if there is something wrong.This is likely about the oldest example of a KPI that I’ve ever seen as well
This dashboard is a little differentGranted, on this airplane, there are many more parameters to measure than on the old car. All of that information is available, but it’s hard to spotIt’s not hard to spot if you have a trained eyePilots need immediate access to aircraft conditions, hence this complicated setup, but they are trained. It’s not complicated to them.Dashboards have different design points for different audiences.
Here’s a newer one from a newer carSpeed is still the most important thing, but fuel is only important when there isn’t enough, so there is an alert.The amount of fuel can still be seen, as can RPM, or optionally through configuration, a series of other parameters that the driver may be interested in.We can also show other interesting information customized to the driver – in this case, the song that is currently playing. The points are that - we are monitoring the status of something - We are (or should be) limited by the operator’s ability to absorb information or by the user interface - We may be limited by the available data
Open the Dashboard SiteOverview DashboardNote the different measuresOpen a few of the KPIsOpen GDP and KPI for AmericasSelect Canada and note how the 3 graphs updateSelect Financial Dashboard and select CanadaNext, select China Note the increase in GDP per Capita, and povertyOpen health dashboard. Select China - note the change in Child MortalityOpen education dashboard - use Europe dataNote that these were all built with different tools, and they all have different strengths. We will show how it's been done..
There are many products that make up the BI stack and many of them will be used one way or another in any dashboard project. What’s important is to understand what they are, and how they fit. In some cases, there are multiple ways to achieve the same goal, so the implications of using any should be understood.The good news is that these all fit together into a few product groups. We’ll got through these today.
Keep in mind that in every case, SharePoint is the preferred publishing platformSSRS is the only exception – it can be hosted on its own, but significant advantages are there for SharePointThese approaches are not mutually exclusive, and in some cases can’t be. For example, SharePoint Mashups will depend on content from any or all of Excel, SSRS, PerformancePoint, or other SharePoint content.We’re going to walk through all of these today with the exception of Azure VMs. Azure VMs take the same approach as on-prem, they’re just hosted in the cloud – the “private cloud”No matter which approach we take, they all have the same basic underpinnings. These are based on fundamental BI concepts that need to be understood. So let’s dive in. First and foremost, there is one fundamental Business Intelligence rule that must be adhered to.
Business Intelligence is all about the data, but that doesn’t mean that you just wire up Excel to source data and start Extracting (although far too many people do). This is bad for a number of reason - Security – data level access to production data - Usability – difficult to understand constructs (Great Plains anyone?) - Performance – reporting against the production data concentrates the load. - Organization – data optimized for transactions, not reporting
Instead of querying our source systems directly, we want to take our data and move it into Data Warehouses and data marts, which are optimized for the sorts of analysis that we want to perform. This is done through an ETL operation.CLICKThe data is extracted from the source system, CLICK transformed into the shape we need it, CLICK then loaded into the data warehouse. CLICK Other ETL processes or cube process will load the data into any necessary marts, cubes or models.From here various servers and client will access the data, usually from the data marts of cubes, but occasionally from the warehouse directly. So how does this translate to the Microsoft stack? There are two ways. The Enterprise, or “classic” BI method, or the Power (personal) method.
Starting with the classic method, SQL Server Integration Services is the tool that performs our ETL. SQL Server Database Engine is used for the storage of the data warehouses and data martsSQL Server Analysis Services is the multidimensional engine (traditional OLAP cubes) and now is the engine for enterprise tabular models (xVelocity). SSRS is the traditional server engine for serving reports, and can be deployed either standalone, or through SharePoint. These tools all ship on SQL server media, but some (SSRS and PowerPivot for SharePoint) may be deployed to SharePointClients of this infrastructure may be servers themselves, or designers and Power Users. Consuming tools include Excel, SQL Server Data Tools, Excel Services, PowerPivot for SharePoint, or a host of other tools. Recently, there has been a lot of work in the Personal BI space – so how does that compare to this approach? Fundamental BI concepts still apply.
To start with, we have an Excel Workbook. Excel is the personal BI client from Microsoft. As of the 2010 version (through an add-in), or Excel 2013 directly we have access to an embedded xVelocity data model. CLICKUsing the PowerPivot add-in (needs to be enabled) we can import data directly from the source data systems, and then manipulate the structure, but the data is read only. It can be refreshed, but not edited. Really, we have the E and the L of an ETL system. CLICKMore recently, Power Query has been introduced. It’s a part of Power BI, but in this context it’s just a free Excel add-in that brings more elegance to the import. It puts the T back into ETL on the personal side. It has a host of other features, and different data source options, but that’s fundamentally what it is. Power Query can also load data directly into the workbook, into the model, or both. CLICKOnce the data has been loaded it can be consumed through a number of Excel tools. The traditional multi dimensional tools are the Pivot chart and Pivot table, but we now also have Power View for analytical reporting, and Power Map for geospatial analysis. PowerPivot is the model editor.Be careful with Power Query. It cant be automatically refreshed. Yet.All of these approaches, both enterprise and personal converge through SharePoint and dashboards.
Within SharePoint, we can publish reports and data models, and establish connections to the relevant back end systems. These components can then be used to construct dashboards, or used on their own as dashboards. Dashboards can contain, but are not necessarily limited to Worksheets and worksheet components through Excel Services, either directly connected or via PowerPivotSSRS ReportsPerformancePoint scorecards and KPIsPerformancePoint reports
There are different tools for different tasks, and SharePoint is no exception. Given that we can mashup resources from a number of different products, its no surprise that we need to use several tools to accomplish it. We’re going to have a quick look at most of these in turn, and we’ll start with Reporting Services
What SharePoint thinks BI is, isn’t always what you want. In 2007 SP had a BI Center – it was centred around Excel and SharePoint KPIsIn 2010 this was changed, and PerformancePoint became the centre of the universe- Connections library ONLY supported PP ConnectionsIn 2013 Everything same, but more PowerPivot (back to Excel…). Normal Connections returnedEven with the PowerPivot focus – PP feature not enabled on BI site by default- Now we get both types of Library….A Report is an Excel File (Now with Power View!)Or a Dashboard. Viable – esp if you don’t have publishing.
SSRS ForMature – Feature RichStructured Reporting (vs analytical)Formatted Reports (pagination, etc)Scheduled or batch reportingParameter Driven
Create new Doc Library - enable for SSRSShow Report Builder for new, but don't launchEdit the GDP per capita reportShow the data sources, Country list and mainExplain parameters and how they work. Run the report in test modeAdd the report to the test page and run itSelect ChinaAdd a Current User Filter to the page and connect it - don't forget to select a fieldEdit the user profile - change the country fieldShow multiple charts - My Financial Dashboard.
Excel is familiarNow uses the xVelocity EnginePower Pivot allows scheduled refreshPower View for Analytical ReportingCan Use as a “cube” data sourceDirect connect allows refresh on open (both good and bad)Compatible with Power BICan use EffectiveUserName for per user authentication
Open up the Health spreadsheet and open Power PivotNote the tables and calculated measuresDo a quick pivot chart using life expectancyShow the main page in ExcelShow the Health dashboard in the browserGo back to Excel and discuss publishing items - named items and parameters- for slicersShow Power View in the browser. Note how it's a dashboard itself!Create a new Power View from the embedded model
Assembly of several products initially including Scorecard Manager, ProclarityHas a “Dashboard designer” to build dashboards using SSRS, PP, Excel, Scorecards, Analytic Charts/gridsOnly 2 truly unique features. Scorecarding and Analytic Charts/gridsCan use EffectiveUserName for per user authentication
Go to the perfpoint library and look around at the artifacts. Launch Dashboard designerOpen Education dashboardRich editor for web partsCreate new report but don't saveOpen Internet penetrationShow literacy rates KPI and definition Data sources, target, etcShow ScorecardsShow filter and connectionPublishPoint out that we can have pure PerfPoint or a mix.Show pivots - pick a completely different measureReset - drill down onto Country from Internet penetration rates.Show the overboard and point out the filter connections.
Everything previously discussed is available as a component to a pagePublishing strongly recommendedFilter web parts a big plus.
Create New pageAdd the Education Scorecard, and the GDP ReportShow the SP mashup dashboard
For outside of the firewallExcel onlyPower Q&A Power Query (for query sharing)Power Q&AData Management GatewayPower BI Mobile client
Upload the file into the SPC14 folderEnable itOpen in browserOpen in Power BI application
There are lots of things to worry about… this is what comes up frequently that doesn’t fit into a technical demo
My opinion is that requirements gathering is the hardest part of a BI projects. Even when the business users know what they want, communication is a real problem. BI projects tend to be monsters. They are big, and expensive, and often there is no product until the very end. The problem tends to be “boiling the ocean” – being more concerned with building the infrastructure to fix a problem than actually fixing the problem. The trick is to find a single question that’s worth answering, answer it, and then move on to the next question.
I would encourage those of you so inclined to look at the Lean methodology. The central tenet of Lean is to start small, and build on successes. Don’t do anything that doesn’t bring tangible value of some sort. The idea is MVP – Minimum Viable Product. In essence, how little can we do to have something worthwhile? Then build on it. Our company has built BIT…. Everything you need for Personal and Team BI staged on a tabletIt helps facilitate:Rapid PrototypingRapid Data DiscoveryModeling ToolRapid InsightsWhich leads to better requirements gathering – shortens the feedback loop
- People sometimes get hung up on per user authentication- “You can’t do BI without Kerberos” – which is wrong- In many cases, this is simply not true. Service accounts can impersonate end users down the chain. SSRS has SetUser(), Excel Services and PerformancePoint support EffectiveuserName against multidimensionalBISM supports impersonation in SSRS Power ViewNo per User auth of any sort with PowerBI – control through document security
- Depending on the tool, data freshness may be an issueLatency between data source and data martLatency between data mart and data modelsWhat is “Real enough” time?There is a trade off between freshness and costThere are BI Tools for Real Time
Data is like food. Not everything needs to be fresh….SSRS, which we use to power the ETL, uses SQL ‘s agent jobs to run on a schedule. These are quite granular, down to the minute level. Any tools leveraging the data marts/cubes directly can be that fresh. However, constant runs are resource intensivePowerPivot for SharePoint can be scheduled for refresh daily. This is also true for refreshes in Power BI using the Data Management Gateway, although both can be refreshed on demand. There is a hack for PP4 SPIf Real Time analysis is actually necessary, there are tools available. StreamInsight for ETL, and DirectQuery for tabular models.
So what do we need to make all of this work?
Just because it comes from SQL doesn’t mean it belongs there.Power BI is an odd duck. It’s from 2 quadrants.
Reporting Services is the only one that works well with Standard Edition SharePointExcel and PerformancePoint require SharePoint EnterprisePowerPivot requires BI Edition or Enterprise SQL ServerPower BI is an additional Office 365 licence on top of E3
- What does the future hold? Almost all development efforts are going into the Power stackReporting Services still fills and important niche, and is relatively completePerformancePoint? If Excel gets features, it’s redundantI like the model deployment from Excel to SSAS. Is Power Query far behind?