Watch the full webinar here: http://bit.ly/1RnbVib
This session will discuss the fantastic new capabilities of Query Store with SQL Server 2016. It will illustrate what it is, why we want it, how we would use it, and how it is BETTER than any other available performance tuning tools. This session will walk through enabling this feature, leveraging its capabilities, and modifying some of the available options. If attendees walk away with nothing else from this session, remember the LIVE Query Statistics (or Execution Plan). This is a game changer and will greatly impact any on-demand query optimization.
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Shopping Around for Query Optimizations? Try the Query Store!
1. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
SQL Server 2016
The Query Store
Author: Sean Werick
Company: Pragmatic Works
2. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Who is Pragmatic Works?
o 100% Microsoft SQL Server Focused
o Team of Microsoft MVPs, Authors, Presenters
o Concentrated on Optimizing your Data Lifecycle
What is Pragmatic Works?
o Software Solutions
o Training Offerings
o Professional Services
Why Pragmatic Works?
o Microsoft Gold Partner – Data Platform & Business Intelligence
o Microsoft’s only “National Solution Provider” for Business Intelligence
o Microsoft Analytics Platform System Implementation Partner of the Year
o INC Magazine’s Fastest 5000 Growing Companies
o Leader in offerings centered around Microsoft’s emerging technologies
About Pragmatic Works
3. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
• Principal Consultant
• Community Speaker
• Blogs: www.seanwerick.com
• Email: SWerick@pragmaticworks.com
About Sean
9. 0100101010110 SQL Server
OLTP
SQL Server
data warehouse
ETL
In-memory
ColumnStore
In-memory
OLTP
Real-time fraud
detection
In-memory enhancements
Operational analytics & enhanced performance
Fraud detected
2-24
hrs
10. dbo.Patients
Jane Doe
Name
243-24-9812
SSN
USA
Country
Jim Gray 198-33-0987 USA
John Smith 123-82-1095 USA
dbo.Patients
Jane Doe
Name
1x7fg655se2e
SSN
USA
Jim Gray 0x7ff654ae6d USA
John Smith 0y8fj754ea2c USA
Country
Result Set
Jim Gray
Name
Jane Doe
Name
1x7fg655se2e
SSN
USA
Country
Jim Gray 0x7ff654ae6d USA
John Smith 0y8fj754ea2c USA
dbo.Patients
SQL Server
ciphertext
Query
Always Encrypted
Help protect data at rest and in motion, on-premises & cloud
TrustedApps
SELECT Name FROM
Patients WHERE SSN=@SSN
@SSN='198-33-0987'
Result Set
Jim Gray
Name
SELECT Name FROM
Patients WHERE SSN=@SSN
@SSN=0x7ff654ae6d
Column
Encryption
Key
Enhanced
ADO.NET
Library
Column
Master
Key
Client side
11. Mission critical performance
Performance
Operational analytics
In-memory OLTP for
more applications
Query data store
Native JSON
Temporal database
support
Security
Always Encrypted
Row Level
Security
Dynamic Data
Masking
Availability
Enhanced AlwaysOn
• 3synchronousreplicasfor
autofailoveracrossdomains
• Roundrobinloadbalancing
ofreplicas
• DTCfortransactionalintegrity
acrossdatabaseinstances
withAlwaysOn
Enhanced online
operations
Scalability
Enhanced database
caching
Support for Windows
Server 2016
• 12TBmemorysupport
12. PolyBase
Query relational and non-relational data with T-SQL
T-SQL query
SQL Server Hadoop
Quote:
************************
**********************
*********************
**********************
***********************
$658.39
Jim Gray
Name
11/13/58
DOB
WA
State
Ann Smith 04/29/76 ME
13. Data Scientist
Interact directly with data
Built-in to SQL Server
Data Developer/DBA
Manage data and
analytics together
Built-in advanced analytics
In-database analytics at massive scale
Example Solutions
• Salesforecasting
• Warehouse efficiency
• Predictive maintenance
Relational Data
Analytic Library
T-SQL Interface
Extensibility
?
R
RIntegration
010010
100100
010101
Microsoft Azure
Marketplace
New R scripts
010010
100100
010101
010010
100100
010101
010010
100100
010101
010010
100100
010101
010010
100100
010101
• Credit risk protection
14. SQL Server
Report Builder
SQL Server BI “Reporting
Services”
• Power BI Desktop
• Mobile reports
• Paginated reports
Power BI
• Power BI Desktop
• Microsoft Excel
• Mobile reports
• Paginated reports
Power BI Web Portal
Power BI Desktop
CONSUMEMANAGECREATE
Microsoft Excel
“Datazen” Publisher**
Report Designer in
SQL Server Data Tools
Windows App
Android App
iOS App
Report Server Web portal
(all BI report types)
SharePoint Web
15. Deeper insights across data
Access any data
PolyBase
Power Query for analytics
and reporting
Enhanced SSIS
• Designersupportforprevious
SSISversions
• SupportforPowerQuery
Scale and manage
Enterprise-grade
Analysis Services
New single SSDT in
Visual Studio 2015
Enhanced MDS
• Exceladd-in15xfaster
• Moregranularsecurityroles
• Archivaloptionsfortransactionlogs
• Reuseentitiesacrossmodels
Powerful insights
Built-in advanced analytics
Business insights through
rich visualizations on
mobile devices
Enhanced Reporting
Services
16. Order history
Name SSN Date
Jane Doe cm61ba906fd
2/28/200
5
Jim Gray ox7ff654ae6d
3/18/200
5
John Smith i2y36cg776rg
4/10/200
5
Bill Brown nx290pldo90l
4/27/200
5
5/12/200
Order history
Name SSN Date
Jane Doe cm61ba906fd
2/28/200
5
Jim Gray ox7ff654ae6d
3/18/200
5
John Smith i2y36cg776rg
4/10/200
5
Bill Brown nx290pldo90l
4/27/200
5
Customer data
Product data
Order History
Stretch to cloud
Stretch SQL Server into Azure
Stretch warm and cold tables to Azure with remote query
processing
App
Query
Microsoft Azure
Jim Gray ox7ff654ae6d 3/18/2005
17. Hyperscale cloud
Hybrid solutions
Stretch tables into Azure
Power BI with on-premises data
Hybrid scenarios with SSIS
Enhanced backup to Azure
Simplicity
Easy migration of on-premises
SQL Server
Simplified Add Azure
Replica Wizard
Consistency
Common development,
management and
identity tools
Consistent experience
from on-premises to Azure
19. Introducing Azure SQL Data Warehouse
Integration with Power BI, ADF,
and Machine Learning services
Separate storage & compute
Elastic scale
Scale-out relational
data warehouse
20. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
What is the Query Store?
A SQL Server 2016 (and SQL DB) Feature that can be Enabled
• Provides insight on query plan choice and performance
• Simplifies performance troubleshooting by quickly finding performance differences
caused by query plan changes
• Automatically captures a history of queries, plans, and runtime statistics, and retains
these for review
• Separates data by time windows to better illustrate database usage patterns and
understand when query plan changes happened on the server
21. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Why Use the Query Store?
Query Performance Insight Helps Satisfy 2 Main Goals
Performance Cost
22. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
WHAT IS THE REAL COST?
Why Use the Query Store?
All about the DTUs (PaaS)
A Database Transaction Unit (DTU) is a
blended measure of the resources allocated at
each performance level.
• Processor - % Processor Time
• Logical Disk - Disk Reads/sec
• Logical Disk - Disk Writes/sec
• Database - Log Bytes Flushed/sec
• Review the list of top resource-consuming
queries
• Select an individual query to view its details
• Click Settings to customize how data is
displayed or to show a different time period
• Open Index Advisor* (PaaS) and check if any
recommendations are available
Performance Cost
* Index Advisor - https://azure.microsoft.com/en-us/documentation/articles/sql-database-index-advisor/
24. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Does It Work with In-Memory OLTP?
• SQL Server Query Store allows you to monitor the
performance of natively compiled code for workloads
running in-memory OLTP
• Compile and runtime statistics are collected and
exposed the same way as for disk-based workloads
• When you migrate to in-memory OLTP you can
continue using Query Store views in SQL Server
Management Studio as well as custom scripts you have
developed for disk-based workloads before migration
• This saves your investment in learning Query Store
technology and makes it generally usable for
troubleshooting all type of workloads
25. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
How to Use the Query Store
In Just 3 Easy Steps, You TOO can be a DBA!!
Okay, okay. This may take a bit more effort and knowledge.
26. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
There are 2 Ways to Enable Query Store
By Using the Query Store Page in SSMS
Setting up the Query Store
In Object Explorer, right-click a database, and then click
Properties
27. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
By Using the Query Store Page in SSMS
Setting up the Query Store
In the Database Properties dialog box, select the Query
Store page
Select related attributes
• General
o Operation Mode (Read or Read/Write)
• Monitoring
o Data Flush Interval (Manual Entry in Minutes)
o Statistics Collection (5 mins, 10 mins, 15 mins, 30 mins, 1 hr,
1 day)
• Max Size
o Query Store Capture Mode (All or Auto)
o Size Based Cleanup Mode (Off or Auto)
o Stale Query Threshold (Manual Entry in Days)
28. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
By Using T-SQL Statements
• Use the ALTER DATABASE statement
ALTER DATABASE AdventureWorks2012
SET QUERY_STORE = ON;
Ability to
Setting up the Query Store
29. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
GUI Option T-SQL Option
Operation Mode (Requested) OPERATION_MODE
Data Flush Interval (Minutes) DATA_FLUSH_INTERVAL_SECONDS
Statistics Collection Interval INTERVAL_LENGTH_MINUTES
Max Size (MB) MAX_STORAGE_SIZE_MB
Query Store Capture Mode QUERY_CAPTURE_MODE
Size Based Cleanup Mode SIZE_BASED_CLEANUP_MODE
Stale Query Threshold (Days) CLEANUP_POLICY (STALE_QUERY_THRESHOLD_DAYS)
MAX_PLANS_PER_QUERY
Query Store Options with T-SQL
ALTER DATABASE [db_name]
SET QUERY_STORE
(
[Query Store options]
);
-- Example
ALTER DATABASE AdventureWorks
SET QUERY_STORE
(
MAX_STORAGE_SIZE_MB = 250
);
30. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
sys.query_store_plansys.query_store_runtime_stats
sys.query_context_settings
sys.query_store_runtime_stats_interval
sys.query_store_query
sys.query_store_query_text
plan_id
runtime_stats_interval_id
query_id
context_settings_id
query_text_id
DATA MODEL
sys.database_query_store_options
What Information Does It Provide?
31. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
What Information Does It Provide?
*Additional Detail - https://msdn.microsoft.com/en-us/library/dn818146.aspx
Query Store View Description
sys.database_query_store_options Query Store options for this database
sys.query_context_settings Semantics affecting context settings associated with a query (The same
query text compiled under different settings may produce different
results [depending on the underlying data])
sys.query_store_plan Each execution plan associated with a query
sys.query_store_query Information about the query and its associated overall aggregated
runtime execution statistics
sys.query_store_query_text Transact-SQL text and the SQL handle of the query
sys.query_store_runtime_stats Information about runtime execution statistics information for the query
sys.query_store_runtime_stats_interval Information about the start and end time of each interval over which
runtime execution statistics information for a query has been collected
32. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Query Store Best Practices
Information and performance are generally a give and take, but there ARE some
best practices
Every single setting in the Query Store has associated costs and benefits
• Treat Query Store performance tuning like an agile project with Sprints
o Sprint 1 – Keep Operation Mode in READ/WRITE mode for information collection
This reduces write time, and subsequent overall database performance, for data
collection as no reporting is occurring
o Sprint 2 – Change Operation Mode to READ mode for analysis and actual performance
tuning
This it increases report performance because no writes are occurring. This also results
in no “dirty reads” because of the asynchronous writes
33. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Query Store Best Practices
• Flush the data at specific business case intervals. For example, if performance of the DB
seems to be poor around end of the month due financial reporting, consider using the Agile
methodology to complete one iteration, then flush the data before the next iteration.
• Keep the Max Size small enough to keep the reporting relatively snappy, however, keep
enough data to capture all of the business needs
• Keep an accurate Cleanup Policy as the business and reporting solution evolves
• Collect statistics as frequently as possible to see exactly which execution plans are being
executed. If the statistics are not updated for some time, you may be looking at an old
execution plan because the statistics have changed
Information and performance are generally a give and take, and it ultimately
relies on each and every business and its processes
34. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Query Store Reporting
Azure SQL DB
• Cloud offering (PaaS)
• Learn and adapt dynamically with your app
• Scale performance on the fly, without app
downtime
• Build multitenant apps with customer isolation
and efficiency
• Work within your preferred development
environments
• Help protect and secure your app data
35. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Query Store Reporting
The top queries view opens and the
top CPU consuming queries are listed
• The top line shows overall DTU% for the
database, while the bars show CPU%
consumed by the selected queries
during the selected interval
• The bottom grid represents aggregated
information for the visible queries
o Average CPU per query during
observable interval
o Total duration per query
o Total number of executions for a
particular query
36. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Query Store Reporting
Viewing Individual Query Details
Click any query in the list of top queries.
The details view opens and the queries CPU consumption is broken
down over time.
Click around the chart for details.
The top line is overall DTU%, and the bars are CPU% consumed by
the selected query.
Review the data to see detailed metrics including duration, number
of executions, and resource utilization percentage for each interval
the query was running.
40. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Query Store Admin with T-SQL
Stored Procedure Description
sp_query_store_flush_db Flushes the in-memory portion of the Query Store data to disk
sp_query_store_reset_exec_stats Clears the runtime stats for a specific query plan from the query store
sp_query_store_force_plan Enables forcing a particular plan for a particular query.
When a plan is forced for a particular query, every time SQL Server encounters
the query, it tries to force the plan in the optimizer. If plan forcing fails, a
XEvent is fired and the optimizer is instructed to optimize in the normal way.
sp_query_store_unforce_plan Enables unforcing a particular plan for a particular query
sp_query_store_remove_plan Removes a single plan from the query store
sp_query_store_remove_query Removes the query, as well as all associated plans and runtime stats from the
query store
41. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Forcing Execution Plans
OLD WAY
1. Run SET SHOWPLAN_XML ON to return the XML
Execution Plan, or alternatively, grab the XML Plan
from the sys.dm_exec_text_query_plan DMF)
2. Create the Plan Guide using the
sp_create_plan_guide Stored Procedure
42. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Forcing Execution Plans
NEW WAY
1. Click the “Force Plan” button inside one of the
Query Store reports
2. Use the sp_query_store_force_plan SP.
EXEC sp_query_store_force_plan 136, 146;
43. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Error State
Remember to verify intermittently that the Query Store is still collecting data
SELECT actual_state_desc, desired_state_desc, current_storage_size_mb,
max_storage_size_mb, readonly_reason, interval_length_minutes,
stale_query_threshold_days, size_based_cleanup_mode_desc,
query_capture_mode_desc
FROM sys.database_query_store_options;
If there is a difference between the actual_state_desc and desired_state_desc, this indicates that a change of
operations mode occurred automatically.
The most common change is for the Query Store to silently switch to read-only mode. In extremely rarely
circumstances, Query Store can end up in the ERROR state because of internal errors.
When the actual state is read-only, use the readonly_reason column to determine the root cause. Typically, the
Query Store transitioned to read-only mode because the size quota was exceeded. In that case the
readonly_reason is set to 65536.
44. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM
Error State
How to Fix
Reactive
Increase the maximum storage size by using the
MAX_STORAGE_SIZE_MB option of ALTER DATABASE
Clean up Query Store data
ALTER DATABASE [QueryStoreDB]
SET QUERY_STORE CLEAR;
Change Operation Mode back to READ/WRITE
ALTER DATABASE [QueryStoreDB]
SET QUERY_STORE (OPERATION_MODE = READ_WRITE);
Proactive
Apply best practices, specifically related to Query
Store size
Make sure that most recent data is retained,
configure time-based policy to remove stale
information regularly
Consider setting Query Capture Mode to Auto as it
filters out queries that are usually less relevant for
your workload
Key points: Roadmap slide for SQL Server 2016 BI – SQL Server Reporting Services for paginated and mobile reporting.
Talking points:
Today, customers who have a qualifying SQL Server 2008+ Enterprise Edition license and active Software Assurance can download Datazen Server (http://www.microsoft.com/sqlmobilebi) and use it as long as they have active Software assurance.
In SQL Server 2016 our plans are to provide a better integrated experience and enable both mobile and paginated reporting with SQL Server Reporting Services (native mode).
Datazen is expected to be available as a component of SQL Server Reporting Services enabling IT organizations to create and manage paginated reports as well as mobile reports for customers who need a mobile reporting solution that is implemented on-premises. SQL Server Reporting Services Report Server can run in one of two deployment modes, native mode and SharePoint Mode. Mobile reports will only be supported in native mode.
What changes are expected for paginated reports in SQL Server 2016?
IT can continue to use SQL Server Report Builder and Report Designer to create paginated reports and manage them in SQL Server Reporting Services. SQL Server 2016 Reporting Services (native mode) allows you to view these repots in the Report Server Web portal, render as a PDF as well as pin paginated report elements to Power BI dashboards enabling business users to get access to relevant information from one singe place.
Mobile reports are created in Datazen Publisher, based on data sources that are set up in SQL Server Reporting Services 2016. These mobile reports can be consumed from the mobiles apps and on the Report Server Web Portal, enabling business users to get access to mobile and paginated reports from on web portal.
SQL Server Reporting Services in integrated mode will continue to support SQL Server Reporting Services 2016 for paginated reports.
Key points: Provide an overview of the future roadmap of SQL Server BI and Power BI beyond SQL Server 2016
Talking points:
As previously mentioned we plan to integrate Datazen Server into SQL Server 2016 Reporting Services to provide IT with one solution that can be deploy on premises, for both, paginated and mobile reporting. After general availability of SQL Server 2016 we will continue to deliver on our vision to provide a set of client tools that can be used to create paginated and mobile reports that then can be managed both in the cloud using the Power BI service and on premises for customers, who are not yet ready to move to the cloud.
This means that business users will be able to explore data from Excel workbooks, Power BI Desktop reports, “Datazen” Publisher, Report Builder and Report Designer in one place, from one single pane of glass using the Power BI Service. During this timeframe we expect to converge the capability of Datazen Publisher into Power BI Desktop
Customers who need a solution that is implemented on-premises can use the same tools (Power BI Desktop, “Datazen” Publisher, Report Builder and Report Designer) and publish them to the on-premises server, without requiring SharePoint.
Business users who require mobile access to their data can consume all reports using the same mobile apps.
We will continue to enable organizations to embed BI content into SharePoint pages.
This means that no mater which tool you use to create reports you can consume them either from the cloud or on premises. Enabling organizations to be more productive, create a report once and consume it in the cloud, on-premises or both if required both.
Cortana Analytics Suite delivers an end-to-end platform with integrated and comprehensive set of tools and services to help you build intelligent applications that let you easily take advantage of Advanced Analytics.
First Cortana Analytics Suite provides services to bring data in, so that you can analyze it. It provides information management capabilities like Azure Data Factory so that you can pull data from any source (relational DB like SQL or non-relational ones like your Hadoop cluster) in an automated and scheduled way, while performing the necessary data transforms (like setting certain data colums as dates vs. currency etc). Think ETL (Extract, Transform, Load) in the cloud. Event hub does the same for IoT type ingestion of data that streams in from lots of end points.
The data brought in then can be persisted in flexible big data storage services like Data Lake and Azure SQL DW.
You can then use a wide range of analytics services from Azure ML to Azure HDInsight to Azure Stream Analytics to analyze the data that are stored in the big data storage. This means you can create analytics services and models specific to your business need (say real time demand forecasting).
The resultant analytics services and models created by taking these steps can then be surfaced as interactive dashboards and visualizations via Power BI
These same analytics services and models created can also be integrated into various different UI (web apps or mobile apps or rich client apps) as well as via integrations with Cortana, so end users can naturally interact with them via speech etc., and so that end users can get proactively be notified by Cortana if the analytics model finds a new anomaly (unusual growth in certain product purchases- in the case of real time demand forecasting example given above) or whatever deserves the attention of the business users.
As a DBA, you’ll obviously be more focused on application and database performance.
As an executive, you’ll be equally focused on keeping costs at a minimum.
For the Index Advisor, To get index recommendations a database needs to have about a week of usage, and within that week there needs to be some activity. There also needs to be some consistent activity as well. The index advisor can more easily optimize for consistent query patterns than it can for random spotty bursts of activity. If recommendations are not available theIndex recommendations page should provide a message explaining why.
Select the Query Store page
Select related attributes
Operation Mode
Off, Read Only, Read Write
Define Monitoring
Data Flush Interval (Minutes) – User Defined
Stats Collection Interval – 1 minute, 5 minutes, 10 minutes, 15 minutes, 30 minutes, an hour, or 1 day
Define the Statistics Collection
What are statistics?
Define the Max Size MB
Define the Query Store Capture Mode
Demo Here
Go to VM and show the custom reports you can write.
Show how to Force Execution Plan in Query Store via SSMS