http://pa.summit.solidq.com Presentación durante el SolidQ Summit Panama City 2016

1. #SQSummit
Novedades SQL 2016
Javier Loria
JLoria@solidq.com
/in/JavierSQL
@JavierSQL
In-Memory y ColumnStore
Index

2. Sponsors del SolidQ Summit Panamá

3. Agenda
• 09.00AM - 10.15AM
• Overview, In-Memory y ColumnStore
• 10-30AM - 11.45AM.
• Query Store-Always Encrypted-Row Level Security,
Dynamic Data Masking y StrechDb
• 12.00AM - 1.15PM.
• Polybase y Novedades SQL Azure y
• 2.30PM - 3.45PM
• HA & DR in SQL Server 2016
• 4.00PM - 5.15PM
• Novedades de SSRS-SSIS-SSAS

4. Microsoft
Data Platform
Relational Beyond Relational
On-premisesCloud
SQLServerAzure VM
Azure SQL DB
Azure SQL DW
AzureData Lake
HDInsight
DocumentDB
SQLServer
APS
SQLServer
PowerBI
Azure MachineLearning
Azure Data Factory
FederatedQuery

5. at massive scale
SQL Server 2016: Everything built-in
The above graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Microsoft. Gartner does not endorse any
vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's
research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
Consistent experience from on-premises to cloud
In-memoryacrossall workloads
built-inbuilt-in built-in built-in
TPC-H non-clustered results as of 04/06/15, 5/04/15, 4/15/14 and 11/25/13, respectively. http://www.tpc.org/tpch/results/tpch_perf_results.asp?resulttype=noncluster
National Institute of Standards and Technology Comprehensive Vulnerability Database update 10/2015
0
1
4
0 0
3
34
29
22
15
5
22
6
43
20
69
18
49
3
-80
-70
-60
-50
-40
-30
-20
-10
0
2010 2011 2012 2013 2014 2015
SQL Server Oracle MySQL2 SAP HANA
built-in
Microsoft Tableau Oracle
$120
$480
$2,230
Self-service BI per userTPC-H non-clustered 10TB
Oracle
is #4#2
SQL Server
#1
SQL Server
#3
SQL Server
built-in

7. Key issues
Complex implementation
Requirement of two servers
(both capital and operational
expenditures)
Data latency in analytics
More business demand that requires
real-time analytics
IIS Server
BI analysts

8. Capabilities and benefits
IIS Server
Capabilities
Runs analytics queries concurrently with operational
workload using same schema
Requires no application changes to run data
warehouse queries on In-Memory OLTP workload
Benefits
Minimal impact on OLTP workload
Offers best available performance and scalability
Offloads analytics workload to readable secondary
BI analysts

9. Real-time Operational Analytics targets the scenario
that has a single data source, such as an enterprise
resource planning (ERP) application, and makes it
possible to run both operational and analytics
workloads.
This doesn’t replace the need for
a separate data warehouse when:
Data from multiple sources must be integrated before
running the analytics workload
Extreme analytics performance using pre-aggregated
data, such as cubes, is required
IIS Server
OLTP workload

10. Demo
DISK Based tables

11. In-Memory OLTP integration and
application migration
Microsoft Confidential 1
Memory-optimized table
filegroup Data filegroup
SQL Server.exe
In-memory OLTP engine for memory-
optimized tables and indexes
TDS handler and session management
Natively compiled SPs
and schema
Buffer pool for tables and indexes
Proc/plan cache for ad-hoc T-SQL
and SPs
Transaction log
Query
interoperability
Non-durable
table T1 T4T3T2
T1 T4T3T2
T1 T4T3T2
T1 T4T3T2
Tables
Indexes
Interpreter for TSQL, query plans,
expressions
T1 T4T3T2
T1 T4T3T2
Checkpoint and recovery
Access methods
Parser,
Catalog,
Algebrizer,
Optimizer
In-Memory
OLTP Compiler
In-Memory
OLTP
component
Key
Existing SQL
component
Generated .dll
Client app

12. Create table DDL
CREATE TABLE [Customer](
[CustomerID] INT NOT NULL
PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT = 1000000),
[Name] NVARCHAR(250) NOT NULL
INDEX [IName] HASH WITH (BUCKET_COUNT = 1000000),
[CustomerSince] DATETIME NULL
)
WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);
This table is memory
optimized
This table is durable
Secondary indexes are
specified inline
Hash index

13. Memory-optimized table creation
CREATE TABLE DDL
Generate and compile
code
Produce DLL table
Load DLL table

14. Create procedure DDL
CREATE PROCEDURE [dbo].[InsertOrder] @id INT, @date DATETIME
WITH
NATIVE_COMPILATION,
SCHEMABINDING,
EXECUTE AS OWNER
AS
BEGIN ATOMIC
WITH
(TRANSACTION
ISOLATION LEVEL = SNAPSHOT,
LANGUAGE = 'us_english')
-- insert T-SQL here
END
This proc is natively
compiled
Native procs must be
schema-bound
Atomic blocks
• Create a transaction if
there is none
• Otherwise, create a
savepoint
Execution context is
required
Session settings are fixed at
create time

15. Procedure creation
CREATE PROC DDL
Optimize query
Generate and compile code
Produce procedure DLL
Load procedure DLL

16. Demo
In-Memory based tables

17. Performance gains
Microsoft Confidential 2
In-
Memory
OLTP
Compiler In-Memory
OLTP
component
Memory-optimized
table filegroup
Data filegroup
SQL Server.exe
In-Memory OLTP engine for
memory-optimized tables and
indexes
TDS handler and session management
Natively compiled
SPs and schema
Buffer pool for tables and
indexes
Proc/plan cache for ad-hoc T-
SQL and SPs
Client app
Transaction log
Interpreter for TSQL, query
plans, and expressions
Query
interop
Access methods
Parser,
Catalog,
Algebrizer,
Optimizer
10-30x more efficient
Reduced log bandwidth
and contention; log
latency remains
Checkpoints are
background sequential
IO
No improvements in
communication stack,
parameter passing, or
result set generation
Key
Existing SQL
component
Generated .dll

18. Create updatable non-clustered columnstore
index (NCCI) for analytics queries
Drop all other indexes that were created for
analytics
Maintain columnstore like any other index
Use Query Optimizer to choose columnstore
index where needed
Requires no application changes
Non-clustered columnstore index
B-tree index
Delta rowgroups
Non-clustered columnstore index
Deletebitmap
B-tree index

19. B-tree index
HOT
Delta
rowgroups
Non-clustered columnstore index
Control how long the data stays in delta
rowgroups
Example: for HOT/WARM (time based)
• < 6 hours – HOT (delta store)
OLTP workload
Use analytics queries to access both
columnstore and hot data transparently
Deletebitmap
Minimizing columnstore index overhead:
Compression Delay

20. B-tree index
HOT
Non-clustered columnstore index – filtered index
Deletebitmap
Delta rowgroups
Create columnstore only on cold data, using
filtered predicates to minimize maintenance
Minimizing columnstore index overhead:
Filtered Index
OLTP workload
Use analytics queries to access both
columnstore and hot data transparently

21. Using columnstore on in-memory tables
Requires no explicit delta rowgroup
Keeps rows (tail) not in columnstore In-Memory OLTP table
Requires no columnstore index overhead when operating on
tail
Uses background task to migrate rows from tail to
columnstore in chunks of 1 million rows
Uses Deleted Rows Table (DRT) to track deleted rows
Keeps columnstore data fully resident in memory and
persisted together with operational data
You can use Compression Delay to keep data longer in the
“tail” to eliminate impact on OLTP workload operating on the
“hot” data
Requires no application changes
nonclustered index
Hash index
In-Memory OLTP table
Deleted rows table
tail

22. 28
Minimizing columnstore index overhead:
Readable secondary
Key points
Mission-critical real-time workloads typically configured
for high availability using AlwaysOn Availability Group
You can offload analytics to readable secondary replicas
AlwaysOn Availability GroupReal-time workload Analytics workload

23. También puedes preguntar tus
dudas con el hashtag
#SQSummit en Twitter
ADAPTIVE BI FRAMEWORK
Te ayudaremos a mejorar la velocidad de desarrollo de tu plataforma de
analítica de negocio basada en nuestra experiencia:
•Diseña antes de construir
•Automatización de procesos por ETL
•Servicios de mentoring para ayudarte a conseguir mejores prácticas para la construcción
de procesos específicos y plataformas de analítica de negocio
•Muy fácil de mantener
SOLIDQ FLEX SERVICES
Con SolidQ Flex Services evitarás sustos, consiguiendo que tus sistemas
sean estables. Desde una solución sencilla de monitorización, hasta un
servicio de atención de incidencias 24/7, mantenimiento proactivo,
resolución de problemas y línea de soporte.
Todo con un coste fijo mensual… y tú dedica el tiempo a las cosas
importantes.
¡Gracias!