SlideShare ist ein Scribd-Unternehmen logo
1 von 132
Downloaden Sie, um offline zu lesen
SQL PASS Summit 2017
Hiram Fleitas
Universal Property & Casualty Insurance Company
UPCIC Internal: hiram/summit2017
Public: goo.gl/BqfYSi
30 Minute Summary
Please silence
cell phones
Explore everything PASS has to offer
Free online webinars Local user groups
around the world
Free 1-day local
training events
Online special
interest user groups
Business analytics
training
Get involved
Free Online Resources
PASS Blog
White Papers
Session Recordings
Newsletter
PASS Connector
BA Insights
www.pass.org
Session evaluations
Your feedback is important and valuable.
Submit your feedback by the end of this session to win a prize.
http://hiram/evalsgoo.gl/u3eN9m
BIO
Father, SQL Server DBA/Dev, IT Pro,
GRT, USCG Auxiliary.
*Dev since 1995.
*SQL since 1999.
FUN
Wakeboard, Wakesurf, Snowboard,
GORUCK, Family.
/HiramFleitas
hiramfleitas@hotmail.com
dba2o.wordpress.comHiramFleitas
@HiramFleitas
Hiram Fleitas
Database Administrator (DBA), @UPCIC
/hfleitas
Agenda
• Schedule
• Keynotes
• Session notes and take-aways
• Demos
• Follow-up resources
Schedule Attended
Time Monday Tuesday Wednesday Thursday Friday
8 am Data Modeling
with Power BI
Columnstore
Indexes -
Practical
Solutions &
Techniques
Keynote Day 1 Keynote Day 2 2017 Intelligent Diagnostics
10 am Benchmarking & Testing
IMOLTP Performance
Making Developers Lives
Easier with SQL 2016
3 Hard Performance
Problems
11 am T-SQL Tips and Tricks BI Toolbelt Statistics & Power BI
1 pm Hacking SQL Server 2017 Deep Dive
3 pm Unlock your Data by
Integrating Analytics into
your Business Apps
2017 The Fast and the
Furious
Avoid Common Database
Development Pitfalls
4 pm Linux Administration for
the SQL Server DBA
5 pm Preparing to be Hacked Everything about Data
Partitioning
*Highlighted favorites.
**See full summit schedule here: http://www.pass.org/summit/2017/Sessions/Schedule.aspx
Columnstore Indexes
Notes:
• Columnstore Indexes: Practical Solutions & Techniques.
• 2016 Brings SIMD for new hardware.
• “Rowstore is dead” – some say in the next 5yrs.
• Biggest pain is lack of parallelism.
• 2 Main components:
• Columnar Storage & Batch Execution
• Rowgroups size: 1-1mil (just over 1 million).
• 10K rows ignore CIDX. 10K is not big data.
• Rowstore all values stored together.
• Columnstore index stores columns together selected.
• If you want to modify data don’t use column store.
• Phases:
1. Shuffling (no order)
2. Rowgroups creation (split the data) horizontal.
3. Segment separation (split the data) vertical.
4. Compress (some segments may compress w/ better ratio).
• Very aggressive “read-ahead”.
• Standard edition is 64KB extents.
• Enterprise edition is 512KB.
• Columnstore indexes is 16MB extents.
• 2014 CIDX 10% of Buffer Pool.
• 64GB of RAM = 6GB for CIDX only.
• Get SP1+ Updates.
Niko Neugebauer
Full Day
Performance
Notes:
• Must use Partitioning if 10+ mil rows. If not kittens are suffering.
• Must use Resource Governor.
• Every single partition is its very own column store.
• Partitioning solves dictionary problem.
• Execution plan decompresses after aggregation.
• Data compression columnstore 130MB vs. columnstore_archive 60MB.
• Backup will be faster, takes LOB & shifts it.
• Restore will be faster.
• Tuple mover shutdown Trace Flag 634.
• Hunts for delta-stores & destroys them every 5 minutes.
• Alter index reorg with (compress_all_row_groups = on)
• Closes all delta-stores.
• Deleted buffer stores IDs of deleted Bitmaps. (2016-2017)
• Natively compiled stored procs ignore column store indexes.
• Schema only.
• IMOLTP freezes schema.
• 2017 Adaptive Query Processing solving hash joins vs nested loops for cached
execution plans.
• Trivial Plans auto upgrade for CIDX to full optimized plans.
• See all new extended events on 2017 RTM.
• Batch Mode uses 64 - 964 rows at a time (shifts rows) vs Row Mode uses
GetNextRow 1 row at a time per operator in execution plan.
• Paul White has a list of conditions for not getting a full optimized plan.
• Trivial means row execution.
Columnstore Indexes
Notes:
• Where 1 = (select 1). Forces full optimized plan. (Parallelism)
• Dbcc CSIndex – Never use anywhere else besides Dev.
• Internals: sys.dm_column_store_object_pool
• Do not use columnstore in OLTP. Updates will lock rowgroups.
• (Tablock) in 2016 uses bulk load API. Insert … Select.
• If you have too many bulk load running, if memory pressure >25 seconds goes to
delta store.
• “If a feature works as advertised, you didn’t test it well enough.”
• Change Buffer Size in SSIS Data Flow. 100MB (2014)
• See docs: Data Loading Performance Guide.
• Never update the columnstore index.
• Use delete + insert instead.
• Force Batch mode: Create fake table with CSIDX, left outer join on 1=0.
Problems / Fixes:
1. Delta Stores: Alter index reorg to compress. (compress_all_row_groups = on). TF 634
disables tuple mover.
2. If VM use L2 Cache Size.
3. Don’t multiply within aggregation.
Key Take-aways:
• Must use Partitioning
• Must use Resource Governor
• Force batch mode with fake table.
• Batch Mode does not support Nested Loops, avoid them at almost all cost.
• Columnstore Blogpost: http://nikoport.com/columnstore
• Columnstore Library: http://github.com/NikoNeugebauer/CISL
• Slides and Scripts: https://goo.gl/CeaycW
Niko Neugebauer
Full Day
Performance
T-SQL Tips and Tricks
Itzik Ben-Gan
Wed, Nov 1
10:45 AM - 12 PM
Performance
Notes:
• Must have Columnstore to enable batch mode processing.
• Use a filtered nonclustered columnstore index.
• See also windowing functions.
sum(val) over (partition by ActID order by TranID) as Balance
from transactions
• 74 sec - 5 sec.
create nonclustered columnstore index idx_yesplease
on transactions (ActID) where ActID = -1 and ActID = -2
• Distinct window aggregates: dense_rank()
• Joins: Use () around left outer joins. With on a.col = c.col outside the ().
• Option (force order). Forces order of joined tables.
• Merge tip and exists. (select … except …)
Key Take-aways:
• Source Code: http://tsql.solidq.com (go to Resources, Downloads)
• Slides: Itzik_Ben-Gan_T-SQL_Tips_and_Tricks.pdf
• Scripts: Itzik_Ben-Gan_T-SQL_Tips_and_Tricks.sql
DEMOColumnstore Indexes
Columnstore Indexes
Keynote Day 1
Rohan:
• Data + Cloud + AI
• Azure has the world largest cloud database footprint.
• Modern Data Estate: Hybrid.
• SQL Server 2017 downloads: 2M+ in 17 months. Post GA 500K in 1 month.
• 30% discount for SQL Server 2017 and Red Hat Linux.
• Azure: 700TB daily telemetry data helps anticipate needs for cloud and on-prem.
• Azure Data Factory. Hourly pricing with no SQL Server license required.
• Power BI Premium capacity, scale, extends on-prem, embedded analytics. Power BI
Report Server on-prem.
Bob / Conor:
• HP ProLiant DL580 Gen10 Server: Read 6.5B rows (15sec on SSD), (2sec on Scalable
Persistent Memory).
• Automatic Tuning, blip in the matrix, fixes parameter sniffing problems.
Tobias / Mihaela:
• docker ps
• git clone https://github.com/erickangMSFT/wwi-db
• cd wwi-db
• docker-compose build
• docker-compose up
• docker ps
• Microsoft SQL Operations Studio
Danielle:
• 1.4M rows insert/sec. Real-time scoring using native SQL Machine Learning and
Azure Machine Learning. In-Memory OLTP + Columnstore Index.
• Healthcare IoT demo length of stay. Demo patient 45 yr old, male, 192lbs, 29.8bmi,
surgery 2015, allergic to fish and tree nuts, complaints of asthma from forest fire,
latest lab results, admits patient and estimated 4 days.
• Context: Recent car company launched IoT vehicle that needed 300K tps.
• Azure Machine Learning Workbench + Jupyter Notebook to build ML model that
predicts Length of Stay which imports into SQL Server using utils.write_rts_model.
use Ignite_Demo;
select @nativeModel = value from rts where id = 'RF'
--(random forests)
select LengthOfStay_Pred
from PREDICT (model=@nativeModel, data=dbo.LoS as input) with
(LengthOfStay_Pred float) p
where input.eid=@eid;
• Predictions latency <20 ms using native SQL Machine Learning functions.
Scott:
• Biml: Business Intelligence Markup Language. DW automation.
• Automatic generation a cloud hybrid solution in ~300 lines of biml code.
• Starts with inspecting on-prem source schema.
• Creates azure datalakes target tables.
• Creates an azure data factory to load.
• Uses an SSIS package for address scrubbing.
• Runs PowerShell to deploy it all automatically to the cloud so its ready for execution.
Rohan Kumar
Bob Ward
Conor Cunningham
Tobias Ternstrom
Mihaela Blendea
Danielle Dean
Scott Currie
Julie Strauss
Christian Wade
Riccardo Muti
The Rolling Stones and
Beatles of All Data and
Cloud
Keynote Day 1
Julie:
• 100TBs read from disk, folded multiple times, returned in 6 seconds. Processing
Azure SQL telemetry funnels and cohorts.
• 100x faster queries. 4.5B rows/sec. 360GB/sec.
• 5x more compute power. 30K DW units. 4K vCPUs provisioned in minutes.
• 1st Cloud DW with NVMe SSD.
Christian:
• Power BI: NYC Taxi Interactive 10 billion. NY Taxi data model. Each row in the trip
table represents a NY taxi ride across a 10yr period. 10bn rows (9TBs of data).
• Revenue per trip by the top 10 zip codes. Visualize it on a map.
• Instant gratification. Clicky clicky draggy-droppy data analysis over massive datasets.
• Interactive charts, click one of the bars in the chart, we can see the highest revenue
pickup location is JFK. Zoom map into the individual pickup locations.
• Build models in Visual Studio 2017 and SSDTS. This is the crown jewel for business
calculations. Shares same connectivity stack as Power BI.
• Deploy model to on-prem SQL Server or Azure Analysis Service - completely hybrid.
• Azure Analysis Services scale out by simply dragging a slider bar across the number
of read-only replicas with copies of the data you need (max: 7).
• Queries get load balanced across replicas via Round Robin.
Ricardo:
• NYC Taxi Interactive 10 billion demo.
• Imports ZipCodes demographic info: med income, avg income, population.
• Models DropoffGeography to ZipCodes within Power BI.
• Tipping by zip code to correlate median income to how much ppl tip (avg tip %). Drill
through huge outliers to see details of specific metrics and trends. For example, 0
miles, 1 minute, fare $2.50, tip was $197.
• Drill back up, graphical filtering, get rid of the outliers, right click exclude, graph auto
adjusts.
• See optimized view on phone or from any device.
Key Take-aways:
• Automatic Tuning
• Docker
• Length of stay
• Biml
• Power BI NYC Taxi Demo
• Video: https://youtu.be/NGbk9XGWTHI
Rohan Kumar
Bob Ward
Conor Cunningham
Tobias Ternstrom
Mihaela Blendea
Danielle Dean
Scott Currie
Julie Strauss
Christian Wade
Riccardo Muti
The Rolling Stones and
Beatles of All Data and
Cloud
DEMO
• Performance 2017 vs 2016
• 2017 Automatic Tuning
• Docker
• Power BI
Performance 2017 vs 2016
Sponsor
3140 Northwoods Parkway, Suite 700, Norcross, Georgia 30071 // Phone: 1 (877) 531-7466 // Fax: (678) 495-1302 // www.CentricsIT.com
United States // Canada // United Arab Emirates // United Kingdom // Czech Republic // Australia
2017 Automatic Tuning
Docker
Power BI
Hacking SQL Server
André Melancia
Wed, Nov 1
1:30 PM - 2:45 PM
Support
Notes:
• Man In The Middle Attack: Adds Aliases, edits hosts file on clients.
• TCP Proxy, Wireshark. Change data on the fly presented to the victim client.
• Since SQL Server 2005 Login packet is encrypted, all other packets are not.
• Force Encryption: YES!
• AE: issue still see packet sizes so MITM can destroy them.
• Deterministic: allows equality lookups. Cypher is always the same for a given value.
• Instant File Initialization: information is not cleaned from HD, can be recovered/read.
• TDE: Server has the keys. Data is decrypted before leaving the server.
• DDM: is not a security feature. Requires unmask permission by database – not per
column or table. Inner join the masked column character by character to a letters
table. Scooby Doo!
• RLS: Field combination division by value in the where clause still generates division
by zero error. Create alert/xevent for continuous division by zero msg/sec.
Key Take-aways:
• In 2003 SQL Slammer: 75K victims in 10 minutes causing a DDoS general Internet
outage. Patch had been released 6 months earlier. Must patch servers!
• SQL Network Config, Protocols for MSSQLServer, Force Encryption: Yes (Always)
• Enable Encrypted Connections: http://msdn.microsoft.com/library/ms191192.aspx
• Encryption Without Validation: http://msdn.microsoft.com/library/ms131691.aspx
• Connection String Keywords: http://msdn.microsoft.com/library/ms130822.aspx
• Encrypt, Trust Server Certificate.
• Slides and Scripts: http://andy.pt/doc
• Video: https://youtu.be/VeGtD0Wf3mg
DEMO
• Man-In-The-Middle (MITM)
• Hack DDM
• Hack RLS
MITM
Hack DDM
Hack RLS
Keynote Day 2
Notes:
• Globally Distributed Made Simple: Azure Cosmos DB.
• 90% of the world’s data was created in the last 2 years.
• 50-150x growth from 2010 to 2020 (9ZB). Sensors, Social, VoIP, Enterprise.
• Emails: 204mil/min, Google Search: 4mil/min, WhatsApp Photos: 347k/min.
• Butterfly-effect of data are consequences across continents.
• Data is growing: Currently 125EB 79.2% is unstructured, 20.8% is structured.
• 80% of business-relevant originates unstructured.
• Push computation to where data is produced.
• Address the needs of 1st-party customers: Xbox, Bing, O365, Store.
• Reqs: turnkey global, guarantee low latency, HA, consistency, elastic, SLA, low cost,
indexing, APIs, schema-agnostic, etc. VERY HARD!
• Millions of lines of code (mainly C++)
• Ring 0 service. Means in every region. 36operational/42built regions.
• Database Account, Database, User, Permission, Container, item, attachment, sproc,
trigger, udf, conflicts. Database scales in terms of containers.
• Regions > DCs > Stamps (Floors/Suites) > Fault Domains (racks) > Clusters (10-20 FDs,
1K tenants/cluster) > Machine (200-400 replicas/tenant, hundreds of
tenants/machine) > Containers > Replicas > [Resource Governor, Transport,
Admission control, DB Engine] > DB Engine Instance [Secret Sorcery: Resource
Manager, Language Runtime Hosts, Query Proc, RSM, Index Mgr, Bw-
tree++/LLAMA++, Log Mgr, IO Mgr].
• Partitioning: Teants > [Tables, Collections, Graphs] > Containers > Resource
Partitions > Replica set: Leader (writes), (reads) 2 followers + 1 forwarder -> to
remote partition(s). Sets allows for many topologies: Chain, Star, Ring, Tree.
Notes:
• Partitioning BP: Even distribution by partition key. Location aware key for access
locality and transaction scope. ACID at the scope of a partition. Time ranges for
heavy workloads for more even distribution.
• Choose failover priority by region. Simulate a region failover.
• Polcy-based Geo-fencing. Replicate data with control.
• Data is AlwaysOn. Active writers and Active Readers.
• Elastic: Independently scale storage & throughput. Transparent server side partition
management and routing. Auto indexed SSD storage. Evict old data using built-in
support for TTL. How do you deal with AsyncIO? More SSD…, is there Cosmos
App/Client?
• Scale from 10-100mil trans/sec across regions. Responsive partition management.
Modular, resource governed nested consensus. (Time zones: LA vs CHI)
• Request Unit/sec (RU) is the normalized currency (%RAM, %CPU, %IOPS). Replica
gets a fixed bud of RUs. Ops consume RUs: Get(Read), Post(Insert), Put(Replace),
Post(Query). Min RUs – Replica Quiescence, Max RUs – No throttling, Above – Rate
limit. Customers pay for reserved RUs.
• Query Cost: Scans, Index Lookups, Query Complexity, Instruction overhead.
• As partitions are split/merged tran/sec are evenly distributed per partition. If you
add regions, you should double RUs per region to avoid starving a particular region.
• Lower throughput when not needed ie. after Christmas shopping reduced
throughput from 1 trillion RUs to minimum vs on-prem having to incur all the cap-ex.
• Internal Microsoft workloads 3 trillion RUs across 20 regions in 3 days. 2017 Sept 7-
10.
Rimma Nehme
GPM, Azure Cosmos DB, Microsoft
Keynote Day 2
Notes:
• Guaranteed Low Latency: Read/Writes served from local region.
• 99th percentile (R. 1KB) <10ms, (W. 1KB) <15ms.
• 50th percentile (R. 1KB) <2ms, (W. 1KB) <6ms.
• Automatically indexed SSD storage.
• Centralized database in Chicago is read from London in 80ms. CosmosDB 10ms!
CosmosDB beats physics of speed of light. Lol
• Constancy problems happen, what if A & B can communicate but cant connect to C
which customers are writing to C.
• Strong consistency high latency means until the whole quorum agrees you cant see
the data. (Blue Pill)
• NoSQL focuses availability, no guarantees about consistency but you’ll get your data
no matter what. Eventual consistency low latency. (Red Pill)
• Most real-life applications do not fall into these extremes, they are somewhere in
between. Real world consistency is not binary choice.
• Checkout Paper “Replicated Data Consistency Explained Through Baseball” by Doug
Terry 2011. Relates real word examples depending who you are in the game.
• Consistency models in Azure CosmosDB: Bounded-stateless (bound for your
staleness of your data, how many writes your reads will lag behind), Session
(monotonic reads & writes in your local region, perfect consistent data to your local
region), and Consistent prefix (guarantees when updates are applied there’s no gaps
in prefix return).
• Using TLA+ specs to precisely specify consistency models to describe what would
happen and how to address it. CosmosDB credits TLA+ by Dr. Leslie Lamport for
flexibility in consistency offerings and famous for paxos algorithm.
Notes:
• Consistency usage: Session 73%, Bounded Staleness 18%, Strong 4%, Eventual 3%,
Consistent Prefix 2%. Tradeoffs can be monetized.
• High Availability SLA is no longer good enough! A true measure in a service business
is it’s SLAs. 1. Latency @99th percentile. 2. Throughput. 3. Consistency. 4. Availability.
• Azure portal Metrics blade shows runtime telemetry statistics against SLAs within
region and across regions.
• DB engine operates on (ARS) Atom-Record-Sequence data model. Onboards other
models: Key-Value, Column-Family, Document and Graph. Single client can connect
to same container and extrapolate data in different ways. Ie. Document vs Graph.
• Schema agnostic leads to store the data as is not compartmentalized. All parameters
of the data are stored in one document. Humanly readable as a JSON document.
• Query represented as a tree and results yet as another tree as a JSON doc.
• Indexes are union of all trees keeping only unique values to each of the documents.
So its an inverted index for JSON-path. For more detail see: Schema-Agnostic
Indexing with Azure Cosmos DB.
• Updates are stored as deltas so they never block or conflict.
• Create udfs and stored procs using JavaScript integration with the DB Engine. Runs
natively by the hosted runtime.
• Lift and shift across platforms or vendors are easier to cloud and from on-prem.
• Running the Service: Weekly deployments of entire stack world wide. Automated
performance, RG and consistency runs every 4hrs. 16+ hrs of stress test runs daily.
Full stack upgrades with customer workloads. Chaos tests. Bots automatically
identify problems and fix nodes that might go down. Hot fix all invariant violations
within 5 days. Transparently making all important metrics available to customers.
Rimma Nehme
GPM, Azure Cosmos DB, Microsoft
Keynote Day 2
Notes:
• You don’t get pearls without irritating the grid.
• The world of data is divided in 2 groups, those who use Cosmos DB and those who
will eventually use Cosmos DB.
• Its not the strongest species that will survive nor the most intelligent, it’s the one
that’s most adaptable to change.- Darwin
• Try Cosmos DB for free, you don’t have to have an azure account, you don’t have to
have a credit card. You can just come in and experience the service.
• Your childhood dream of going into Cosmos will finally be fulfilled. Become a Cosmos
DBA.
Key Take-aways:
• NoSQL database built by Microsoft. Addresses pinpoints when building large scale
applications. Single digit millisecond latency.
• Replication data world-wide with a single click.
• Legit trial – no bs.
• Global distribution, horizontal partitioning, fine grain multitenancy cannot be an
after thought. These things are built in from the ground up.
• Schema agnostic database engine design is crucial for globally distributed databases.
Create/Alter index across regions, datacenters and planet becomes too rigid.
• Intermediate consistency models are extremely useful. You can make the right trade-
offs. With the SLAs the system gives you can guarantee the correctness of the
approach.
• A globally distributed database must provide comprehensive SLAs, fundamentally in
a service business.
• Ask yourself how would you build a database if we were starting today? Without
none of the legacy. Would we look at it differently, limitations, constraints, biases.
• Life is too short to build something nobody wants.
• If we focus on real pain points not nice to have, abandon the constraints and self-
imposed dogmas we can have a real modern database system.
• Video: https://youtu.be/7SRi9vyDtWY
Rimma Nehme
GPM, Azure Cosmos DB, Microsoft
SQL Server 2017 Deep Dive
Tobias Trenstrom
Travis Wright
Shreya Verma
Nellie Gustafsson
Thurs, Nov 2
1:30 PM - 2:45 PM
Build
Notes:
• Choice of platform (Linux, Windows, Docker) and language (T-SQL, Java, C/C++,
C#/VB.NET, PHP, Node.js, Python, Ruby).
• Industry-leading performance and security now on Linux and Docker.
• Most secure over the last 7 years. MySQL and Oracle had the most vulnerabilities.
• Only commercial DB with AI built-in. R + Python + T-SQL at massive scale.
• End-to-end mobile BI on any device at a fraction of the cost.
• In-memory across all workloads. Most consistent data platform on-prem and cloud.
• HA and DR for Linux and Windows. Clusterless AGs. Ultimate HA with OS-level
redundancy and low-downtime migrations. Load balancing of readable secondaries.
• AG enhancements: DTC_Support=Per_DB,
Required_Synchronous_Secondaries_To_Commit=1
Notes:
• Partner Ecosystem: Azure Site Recovery, DH2i nearest-to-zero downtime, HPE
Serviceguard (SGLX) monitors and recovery options, IDERA SQL Safe Backup,
Portwork for containers, Veeam Backup & Replication.
• ML: Bringing intelligence to where the data lives. Native scoring using PREDICT.
Sentiment Analysis demo.
• Graph Database: Expresses pattern matching. MATCH, to support pattern matching
and traversals. Create table () as node --entities. Create table () as edge --
relationships. Solve problems more intuitively and with less code. Scenarios:
Recommendations, Fraud Detection, Content Management, Bill of Materials, CRM.
• Resumable Online Index Rebuild. Alter index … Resumable=On, Pause/Resume.
• Automatic Plan Regression Correction. sys.dm_db_tunning_recommendations. Alter
database name set Automatic_Tunning (Force_Last_Good_Plan=On).
• MAXDOP option for statistics create/update. Faster backups +other improvements.
• Adaptive Query Processing: Interleaved Execution for MSTVFs - Wohoo!! Batch
Mode Memory Grant Feedback (MGF) adjust grants based on execution feedback
and remove spills improving concurrency for repeating queries. Batch Mode
Adaptive Joins (AJ) will defer the choice of hash join or nested loop until after the
first join input has been scanned. AJ uses nested loop for small inputs, hash joins for
large inputs.
• New TPC-H 10TB world record! New TPC-H 3TB $/perf world record!
Key Take-aways:
• Slides: PASSSummit-SQLServer2017DeepDive.pdf
• Blog: blogs.msdn.microsoft.com/sqlserverstorageengine
• GitHub: sentiment-analysis, sql-graph
• ML Tutorials: aka.ms/mlsqldev
• Data Science Tutorials and Samples: Solution templates
DEMO
• Multi-OS AG Failover
• Get Sentiment
• Get Length Of Stay
• Fraud Detection with Graph
• Is It Bacon?
Resources
Slides, Scripts, Recordings, Notes:
• Recordings (UPCIC Internal): http://hiram/summit2017
• Public: https://goo.gl/BqfYSi
• Sneak Peeks: http://pass.org/summit/2017/Sessions/SneakPeeks.aspx
• PASStv: http://pass.org/summit/2017/PASStv.aspx
• Keynote Day 1: https://youtu.be/NGbk9XGWTHI
• Microsoft BI - An Integrated Modern Solution: https://youtu.be/T0If-SNl08A
• Keynote Day 2: https://youtu.be/7SRi9vyDtWY
• Keynote Day 3 (WIT): https://youtu.be/8JnXi0gnTb8
• Speakers: http://pass.org/summit/2017/Speakers.aspx
• Bob Ward’s slides & scripts: http://aka.ms/bobwardms
• Performance: Adaptive Query Processing, Automatic Tuning.
• Docker: https://docs.microsoft.com/sql/linux/quickstart-install-connect-docker
• Docker SQL, App, CI/CD: https://channel9.msdn.com/events/Connect/2017/T152
• Power BI galleries: https://community.powerbi.com/t5/Galleries/ct-p/Galleries
• Power BI webinars: https://docs.microsoft.com/power-bi/webinars
• Apply Forecasting: https://docs.microsoft.com/power-bi/desktop-analytics-pane
More:
• Twitter: @sqlpass, #PASSSummit, #powerbihelp, #sqlhelp, @MSPowerBI
• Join Slack workspace: http://dbatools.io/slack
• Slack channels: #dbatools, #firstresponderkit, #general, #pass-summit, #query-
tuning, #sqlhelp, #git-help
• Pass Virtual Groups: http://pass.org/Community/Groups.aspx (App Dev, BI, DBA,
DevOps, Global Spanish, HA/DR, Performance, PowerShell, Virtualization, Linux)
• More webinars: Channel9, GroupBy, SQLWorkBooks
Session evaluations
Your feedback is important and valuable.
Submit your feedback by the end of this session to win a prize.
http://hiram/evalsgoo.gl/u3eN9m
Thank You
Hit me up!
hiramfleitas@hotmail.com
dba2o.wordpress.comHiramFleitas
@HiramFleitas
/hfleitas/HiramFleitas

Weitere ähnliche Inhalte

Was ist angesagt?

Understanding Storage I/O Under Load
Understanding Storage I/O Under LoadUnderstanding Storage I/O Under Load
Understanding Storage I/O Under LoadScyllaDB
 
Eliminating Volatile Latencies Inside Rakuten’s NoSQL Migration
Eliminating  Volatile Latencies Inside Rakuten’s NoSQL MigrationEliminating  Volatile Latencies Inside Rakuten’s NoSQL Migration
Eliminating Volatile Latencies Inside Rakuten’s NoSQL MigrationScyllaDB
 
Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...
Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...
Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...huguk
 
AWS Redshift Introduction - Big Data Analytics
AWS Redshift Introduction - Big Data AnalyticsAWS Redshift Introduction - Big Data Analytics
AWS Redshift Introduction - Big Data AnalyticsKeeyong Han
 
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDBScylla Summit 2022: New AWS Instances Perfect for ScyllaDB
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDBScyllaDB
 
Zeotap: Moving to ScyllaDB - A Graph of Billions Scale
Zeotap: Moving to ScyllaDB - A Graph of Billions ScaleZeotap: Moving to ScyllaDB - A Graph of Billions Scale
Zeotap: Moving to ScyllaDB - A Graph of Billions ScaleScyllaDB
 
Sparkling Water Workshop
Sparkling Water WorkshopSparkling Water Workshop
Sparkling Water WorkshopSri Ambati
 
MySQL in the Hosted Cloud
MySQL in the Hosted CloudMySQL in the Hosted Cloud
MySQL in the Hosted CloudColin Charles
 
Real-time Cassandra
Real-time CassandraReal-time Cassandra
Real-time CassandraAcunu
 
Scaling Pinterest
Scaling PinterestScaling Pinterest
Scaling PinterestC4Media
 
How to overcome mysterious problems caused by large and multi-tenancy Hadoop ...
How to overcome mysterious problems caused by large and multi-tenancy Hadoop ...How to overcome mysterious problems caused by large and multi-tenancy Hadoop ...
How to overcome mysterious problems caused by large and multi-tenancy Hadoop ...DataWorks Summit/Hadoop Summit
 
Latency and Consistency Tradeoffs in Modern Distributed Databases
Latency and Consistency Tradeoffs in Modern Distributed DatabasesLatency and Consistency Tradeoffs in Modern Distributed Databases
Latency and Consistency Tradeoffs in Modern Distributed DatabasesScyllaDB
 
Scylla Summit 2016: Graph Processing with Titan and Scylla
Scylla Summit 2016: Graph Processing with Titan and ScyllaScylla Summit 2016: Graph Processing with Titan and Scylla
Scylla Summit 2016: Graph Processing with Titan and ScyllaScyllaDB
 
Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...
Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...
Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...DataStax
 
MySQL NDB Cluster 8.0 SQL faster than NoSQL
MySQL NDB Cluster 8.0 SQL faster than NoSQL MySQL NDB Cluster 8.0 SQL faster than NoSQL
MySQL NDB Cluster 8.0 SQL faster than NoSQL Bernd Ocklin
 
Scylla Summit 2018: Consensus in Eventually Consistent Databases
Scylla Summit 2018: Consensus in Eventually Consistent DatabasesScylla Summit 2018: Consensus in Eventually Consistent Databases
Scylla Summit 2018: Consensus in Eventually Consistent DatabasesScyllaDB
 
(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems
(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems
(GAM403) From 0 to 60 Million Player Hours in 400B Star SystemsAmazon Web Services
 
FOSDEM MySQL and Friends Devroom
FOSDEM MySQL and Friends DevroomFOSDEM MySQL and Friends Devroom
FOSDEM MySQL and Friends DevroomMorgan Tocker
 
What Kiwi.com Has Learned Running ScyllaDB and Go
What Kiwi.com Has Learned Running ScyllaDB and GoWhat Kiwi.com Has Learned Running ScyllaDB and Go
What Kiwi.com Has Learned Running ScyllaDB and GoScyllaDB
 
LesFurets.com: From 0 to Cassandra on AWS in 30 days - Tsunami Alerting Syste...
LesFurets.com: From 0 to Cassandra on AWS in 30 days - Tsunami Alerting Syste...LesFurets.com: From 0 to Cassandra on AWS in 30 days - Tsunami Alerting Syste...
LesFurets.com: From 0 to Cassandra on AWS in 30 days - Tsunami Alerting Syste...DataStax Academy
 

Was ist angesagt? (20)

Understanding Storage I/O Under Load
Understanding Storage I/O Under LoadUnderstanding Storage I/O Under Load
Understanding Storage I/O Under Load
 
Eliminating Volatile Latencies Inside Rakuten’s NoSQL Migration
Eliminating  Volatile Latencies Inside Rakuten’s NoSQL MigrationEliminating  Volatile Latencies Inside Rakuten’s NoSQL Migration
Eliminating Volatile Latencies Inside Rakuten’s NoSQL Migration
 
Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...
Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...
Using Big Data techniques to query and store OpenStreetMap data. Stephen Knox...
 
AWS Redshift Introduction - Big Data Analytics
AWS Redshift Introduction - Big Data AnalyticsAWS Redshift Introduction - Big Data Analytics
AWS Redshift Introduction - Big Data Analytics
 
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDBScylla Summit 2022: New AWS Instances Perfect for ScyllaDB
Scylla Summit 2022: New AWS Instances Perfect for ScyllaDB
 
Zeotap: Moving to ScyllaDB - A Graph of Billions Scale
Zeotap: Moving to ScyllaDB - A Graph of Billions ScaleZeotap: Moving to ScyllaDB - A Graph of Billions Scale
Zeotap: Moving to ScyllaDB - A Graph of Billions Scale
 
Sparkling Water Workshop
Sparkling Water WorkshopSparkling Water Workshop
Sparkling Water Workshop
 
MySQL in the Hosted Cloud
MySQL in the Hosted CloudMySQL in the Hosted Cloud
MySQL in the Hosted Cloud
 
Real-time Cassandra
Real-time CassandraReal-time Cassandra
Real-time Cassandra
 
Scaling Pinterest
Scaling PinterestScaling Pinterest
Scaling Pinterest
 
How to overcome mysterious problems caused by large and multi-tenancy Hadoop ...
How to overcome mysterious problems caused by large and multi-tenancy Hadoop ...How to overcome mysterious problems caused by large and multi-tenancy Hadoop ...
How to overcome mysterious problems caused by large and multi-tenancy Hadoop ...
 
Latency and Consistency Tradeoffs in Modern Distributed Databases
Latency and Consistency Tradeoffs in Modern Distributed DatabasesLatency and Consistency Tradeoffs in Modern Distributed Databases
Latency and Consistency Tradeoffs in Modern Distributed Databases
 
Scylla Summit 2016: Graph Processing with Titan and Scylla
Scylla Summit 2016: Graph Processing with Titan and ScyllaScylla Summit 2016: Graph Processing with Titan and Scylla
Scylla Summit 2016: Graph Processing with Titan and Scylla
 
Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...
Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...
Lessons Learned on Java Tuning for Our Cassandra Clusters (Carlos Monroy, Kne...
 
MySQL NDB Cluster 8.0 SQL faster than NoSQL
MySQL NDB Cluster 8.0 SQL faster than NoSQL MySQL NDB Cluster 8.0 SQL faster than NoSQL
MySQL NDB Cluster 8.0 SQL faster than NoSQL
 
Scylla Summit 2018: Consensus in Eventually Consistent Databases
Scylla Summit 2018: Consensus in Eventually Consistent DatabasesScylla Summit 2018: Consensus in Eventually Consistent Databases
Scylla Summit 2018: Consensus in Eventually Consistent Databases
 
(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems
(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems
(GAM403) From 0 to 60 Million Player Hours in 400B Star Systems
 
FOSDEM MySQL and Friends Devroom
FOSDEM MySQL and Friends DevroomFOSDEM MySQL and Friends Devroom
FOSDEM MySQL and Friends Devroom
 
What Kiwi.com Has Learned Running ScyllaDB and Go
What Kiwi.com Has Learned Running ScyllaDB and GoWhat Kiwi.com Has Learned Running ScyllaDB and Go
What Kiwi.com Has Learned Running ScyllaDB and Go
 
LesFurets.com: From 0 to Cassandra on AWS in 30 days - Tsunami Alerting Syste...
LesFurets.com: From 0 to Cassandra on AWS in 30 days - Tsunami Alerting Syste...LesFurets.com: From 0 to Cassandra on AWS in 30 days - Tsunami Alerting Syste...
LesFurets.com: From 0 to Cassandra on AWS in 30 days - Tsunami Alerting Syste...
 

Ähnlich wie [DBA]_HiramFleitas_SQL_PASS_Summit_2017_Summary

Designing, Building, and Maintaining Large Cubes using Lessons Learned
Designing, Building, and Maintaining Large Cubes using Lessons LearnedDesigning, Building, and Maintaining Large Cubes using Lessons Learned
Designing, Building, and Maintaining Large Cubes using Lessons LearnedDenny Lee
 
Migration to ClickHouse. Practical guide, by Alexander Zaitsev
Migration to ClickHouse. Practical guide, by Alexander ZaitsevMigration to ClickHouse. Practical guide, by Alexander Zaitsev
Migration to ClickHouse. Practical guide, by Alexander ZaitsevAltinity Ltd
 
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...Chester Chen
 
The state of Hive and Spark in the Cloud (July 2017)
The state of Hive and Spark in the Cloud (July 2017)The state of Hive and Spark in the Cloud (July 2017)
The state of Hive and Spark in the Cloud (July 2017)Nicolas Poggi
 
Dynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldDynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldStéphane Dorrekens
 
Benchmarking Solr Performance at Scale
Benchmarking Solr Performance at ScaleBenchmarking Solr Performance at Scale
Benchmarking Solr Performance at Scalethelabdude
 
[JSS2015] In memory and operational analytics
[JSS2015] In memory and operational analytics[JSS2015] In memory and operational analytics
[JSS2015] In memory and operational analyticsGUSS
 
Jss 2015 in memory and operational analytics
Jss 2015   in memory and operational analyticsJss 2015   in memory and operational analytics
Jss 2015 in memory and operational analyticsDavid Barbarin
 
First Hive Meetup London 2012-07-10 - Tomas Cervenka - VisualDNA
First Hive Meetup London 2012-07-10 - Tomas Cervenka - VisualDNAFirst Hive Meetup London 2012-07-10 - Tomas Cervenka - VisualDNA
First Hive Meetup London 2012-07-10 - Tomas Cervenka - VisualDNATomas Cervenka
 
Agility and Scalability with MongoDB
Agility and Scalability with MongoDBAgility and Scalability with MongoDB
Agility and Scalability with MongoDBMongoDB
 
Performance Benchmarking: Tips, Tricks, and Lessons Learned
Performance Benchmarking: Tips, Tricks, and Lessons LearnedPerformance Benchmarking: Tips, Tricks, and Lessons Learned
Performance Benchmarking: Tips, Tricks, and Lessons LearnedTim Callaghan
 
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 20197 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019Dave Stokes
 
AWS re:Invent 2016| DAT318 | Migrating from RDBMS to NoSQL: How Sony Moved fr...
AWS re:Invent 2016| DAT318 | Migrating from RDBMS to NoSQL: How Sony Moved fr...AWS re:Invent 2016| DAT318 | Migrating from RDBMS to NoSQL: How Sony Moved fr...
AWS re:Invent 2016| DAT318 | Migrating from RDBMS to NoSQL: How Sony Moved fr...Amazon Web Services
 
Deep Dive into DynamoDB
Deep Dive into DynamoDBDeep Dive into DynamoDB
Deep Dive into DynamoDBAWS Germany
 
Latest (storage IO) patterns for cloud-native applications
Latest (storage IO) patterns for cloud-native applications Latest (storage IO) patterns for cloud-native applications
Latest (storage IO) patterns for cloud-native applications OpenEBS
 
Data Science in the Cloud @StitchFix
Data Science in the Cloud @StitchFixData Science in the Cloud @StitchFix
Data Science in the Cloud @StitchFixC4Media
 
Solving Large Scale Optimization Problems using CPLEX Optimization Studio
Solving Large Scale Optimization Problems using CPLEX Optimization StudioSolving Large Scale Optimization Problems using CPLEX Optimization Studio
Solving Large Scale Optimization Problems using CPLEX Optimization Studiooptimizatiodirectdirect
 

Ähnlich wie [DBA]_HiramFleitas_SQL_PASS_Summit_2017_Summary (20)

Designing, Building, and Maintaining Large Cubes using Lessons Learned
Designing, Building, and Maintaining Large Cubes using Lessons LearnedDesigning, Building, and Maintaining Large Cubes using Lessons Learned
Designing, Building, and Maintaining Large Cubes using Lessons Learned
 
Migration to ClickHouse. Practical guide, by Alexander Zaitsev
Migration to ClickHouse. Practical guide, by Alexander ZaitsevMigration to ClickHouse. Practical guide, by Alexander Zaitsev
Migration to ClickHouse. Practical guide, by Alexander Zaitsev
 
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
 
The state of Hive and Spark in the Cloud (July 2017)
The state of Hive and Spark in the Cloud (July 2017)The state of Hive and Spark in the Cloud (July 2017)
The state of Hive and Spark in the Cloud (July 2017)
 
Dynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the fieldDynamics CRM high volume systems - lessons from the field
Dynamics CRM high volume systems - lessons from the field
 
Benchmarking Solr Performance at Scale
Benchmarking Solr Performance at ScaleBenchmarking Solr Performance at Scale
Benchmarking Solr Performance at Scale
 
[JSS2015] In memory and operational analytics
[JSS2015] In memory and operational analytics[JSS2015] In memory and operational analytics
[JSS2015] In memory and operational analytics
 
Jss 2015 in memory and operational analytics
Jss 2015   in memory and operational analyticsJss 2015   in memory and operational analytics
Jss 2015 in memory and operational analytics
 
First Hive Meetup London 2012-07-10 - Tomas Cervenka - VisualDNA
First Hive Meetup London 2012-07-10 - Tomas Cervenka - VisualDNAFirst Hive Meetup London 2012-07-10 - Tomas Cervenka - VisualDNA
First Hive Meetup London 2012-07-10 - Tomas Cervenka - VisualDNA
 
Agility and Scalability with MongoDB
Agility and Scalability with MongoDBAgility and Scalability with MongoDB
Agility and Scalability with MongoDB
 
Performance Benchmarking: Tips, Tricks, and Lessons Learned
Performance Benchmarking: Tips, Tricks, and Lessons LearnedPerformance Benchmarking: Tips, Tricks, and Lessons Learned
Performance Benchmarking: Tips, Tricks, and Lessons Learned
 
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 20197 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
 
AWS re:Invent 2016| DAT318 | Migrating from RDBMS to NoSQL: How Sony Moved fr...
AWS re:Invent 2016| DAT318 | Migrating from RDBMS to NoSQL: How Sony Moved fr...AWS re:Invent 2016| DAT318 | Migrating from RDBMS to NoSQL: How Sony Moved fr...
AWS re:Invent 2016| DAT318 | Migrating from RDBMS to NoSQL: How Sony Moved fr...
 
Large scalecplex
Large scalecplexLarge scalecplex
Large scalecplex
 
Deep Dive into DynamoDB
Deep Dive into DynamoDBDeep Dive into DynamoDB
Deep Dive into DynamoDB
 
GCP for AWS Professionals
GCP for AWS ProfessionalsGCP for AWS Professionals
GCP for AWS Professionals
 
Latest (storage IO) patterns for cloud-native applications
Latest (storage IO) patterns for cloud-native applications Latest (storage IO) patterns for cloud-native applications
Latest (storage IO) patterns for cloud-native applications
 
The state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the CloudThe state of SQL-on-Hadoop in the Cloud
The state of SQL-on-Hadoop in the Cloud
 
Data Science in the Cloud @StitchFix
Data Science in the Cloud @StitchFixData Science in the Cloud @StitchFix
Data Science in the Cloud @StitchFix
 
Solving Large Scale Optimization Problems using CPLEX Optimization Studio
Solving Large Scale Optimization Problems using CPLEX Optimization StudioSolving Large Scale Optimization Problems using CPLEX Optimization Studio
Solving Large Scale Optimization Problems using CPLEX Optimization Studio
 

Mehr von Hiram Fleitas León

(Dev) Hiram Fleitas - Data Viz & Analytics using Python and JavaScript
(Dev) Hiram Fleitas - Data Viz & Analytics using Python and JavaScript(Dev) Hiram Fleitas - Data Viz & Analytics using Python and JavaScript
(Dev) Hiram Fleitas - Data Viz & Analytics using Python and JavaScriptHiram Fleitas León
 
(SQL ML) Hiram Fleitas - Real-time Sentiment Prediction in SQL Server
(SQL ML) Hiram Fleitas - Real-time Sentiment Prediction in SQL Server(SQL ML) Hiram Fleitas - Real-time Sentiment Prediction in SQL Server
(SQL ML) Hiram Fleitas - Real-time Sentiment Prediction in SQL ServerHiram Fleitas León
 
(AI Bot - Azure Cloud) Hiram Fleitas - It's a'Bot Time
(AI Bot - Azure Cloud) Hiram Fleitas - It's a'Bot Time(AI Bot - Azure Cloud) Hiram Fleitas - It's a'Bot Time
(AI Bot - Azure Cloud) Hiram Fleitas - It's a'Bot TimeHiram Fleitas León
 
(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...
(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...
(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...Hiram Fleitas León
 
[DBA-Adv]_HiramFleitas_Hack_Protect_Predict_SQLServer_LearnThem
[DBA-Adv]_HiramFleitas_Hack_Protect_Predict_SQLServer_LearnThem[DBA-Adv]_HiramFleitas_Hack_Protect_Predict_SQLServer_LearnThem
[DBA-Adv]_HiramFleitas_Hack_Protect_Predict_SQLServer_LearnThemHiram Fleitas León
 
DBA Advanced - Hiram Fleitas - SQL ML / AI
DBA Advanced - Hiram Fleitas - SQL ML / AIDBA Advanced - Hiram Fleitas - SQL ML / AI
DBA Advanced - Hiram Fleitas - SQL ML / AIHiram Fleitas León
 
[DBA]_HiramFleitas_Encryption_And_Database_Source_Control
[DBA]_HiramFleitas_Encryption_And_Database_Source_Control[DBA]_HiramFleitas_Encryption_And_Database_Source_Control
[DBA]_HiramFleitas_Encryption_And_Database_Source_ControlHiram Fleitas León
 

Mehr von Hiram Fleitas León (7)

(Dev) Hiram Fleitas - Data Viz & Analytics using Python and JavaScript
(Dev) Hiram Fleitas - Data Viz & Analytics using Python and JavaScript(Dev) Hiram Fleitas - Data Viz & Analytics using Python and JavaScript
(Dev) Hiram Fleitas - Data Viz & Analytics using Python and JavaScript
 
(SQL ML) Hiram Fleitas - Real-time Sentiment Prediction in SQL Server
(SQL ML) Hiram Fleitas - Real-time Sentiment Prediction in SQL Server(SQL ML) Hiram Fleitas - Real-time Sentiment Prediction in SQL Server
(SQL ML) Hiram Fleitas - Real-time Sentiment Prediction in SQL Server
 
(AI Bot - Azure Cloud) Hiram Fleitas - It's a'Bot Time
(AI Bot - Azure Cloud) Hiram Fleitas - It's a'Bot Time(AI Bot - Azure Cloud) Hiram Fleitas - It's a'Bot Time
(AI Bot - Azure Cloud) Hiram Fleitas - It's a'Bot Time
 
(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...
(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...
(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...
 
[DBA-Adv]_HiramFleitas_Hack_Protect_Predict_SQLServer_LearnThem
[DBA-Adv]_HiramFleitas_Hack_Protect_Predict_SQLServer_LearnThem[DBA-Adv]_HiramFleitas_Hack_Protect_Predict_SQLServer_LearnThem
[DBA-Adv]_HiramFleitas_Hack_Protect_Predict_SQLServer_LearnThem
 
DBA Advanced - Hiram Fleitas - SQL ML / AI
DBA Advanced - Hiram Fleitas - SQL ML / AIDBA Advanced - Hiram Fleitas - SQL ML / AI
DBA Advanced - Hiram Fleitas - SQL ML / AI
 
[DBA]_HiramFleitas_Encryption_And_Database_Source_Control
[DBA]_HiramFleitas_Encryption_And_Database_Source_Control[DBA]_HiramFleitas_Encryption_And_Database_Source_Control
[DBA]_HiramFleitas_Encryption_And_Database_Source_Control
 

Kürzlich hochgeladen

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfonteinmasabamasaba
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park masabamasaba
 
AI & Machine Learning Presentation Template
AI & Machine Learning Presentation TemplateAI & Machine Learning Presentation Template
AI & Machine Learning Presentation TemplatePresentation.STUDIO
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024VictoriaMetrics
 
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisamasabamasaba
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...masabamasaba
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...Health
 
Harnessing ChatGPT - Elevating Productivity in Today's Agile Environment
Harnessing ChatGPT  - Elevating Productivity in Today's Agile EnvironmentHarnessing ChatGPT  - Elevating Productivity in Today's Agile Environment
Harnessing ChatGPT - Elevating Productivity in Today's Agile EnvironmentVictorSzoltysek
 
Software Quality Assurance Interview Questions
Software Quality Assurance Interview QuestionsSoftware Quality Assurance Interview Questions
Software Quality Assurance Interview QuestionsArshad QA
 
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyviewmasabamasaba
 
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...SelfMade bd
 
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Steffen Staab
 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...panagenda
 
Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the pastArchitecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the pastPapp Krisztián
 
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareAnnouncing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareJim McKeeth
 
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...masabamasaba
 
%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in sowetomasabamasaba
 
Define the academic and professional writing..pdf
Define the academic and professional writing..pdfDefine the academic and professional writing..pdf
Define the academic and professional writing..pdfPearlKirahMaeRagusta1
 
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfPayment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfkalichargn70th171
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsJhone kinadey
 

Kürzlich hochgeladen (20)

%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
%in kaalfontein+277-882-255-28 abortion pills for sale in kaalfontein
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
 
AI & Machine Learning Presentation Template
AI & Machine Learning Presentation TemplateAI & Machine Learning Presentation Template
AI & Machine Learning Presentation Template
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
 
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
 
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...
 
Harnessing ChatGPT - Elevating Productivity in Today's Agile Environment
Harnessing ChatGPT  - Elevating Productivity in Today's Agile EnvironmentHarnessing ChatGPT  - Elevating Productivity in Today's Agile Environment
Harnessing ChatGPT - Elevating Productivity in Today's Agile Environment
 
Software Quality Assurance Interview Questions
Software Quality Assurance Interview QuestionsSoftware Quality Assurance Interview Questions
Software Quality Assurance Interview Questions
 
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
 
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
 
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
 
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...
 
Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the pastArchitecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
 
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareAnnouncing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
 
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...
 
%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto
 
Define the academic and professional writing..pdf
Define the academic and professional writing..pdfDefine the academic and professional writing..pdf
Define the academic and professional writing..pdf
 
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfPayment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
 

[DBA]_HiramFleitas_SQL_PASS_Summit_2017_Summary

  • 1. SQL PASS Summit 2017 Hiram Fleitas Universal Property & Casualty Insurance Company UPCIC Internal: hiram/summit2017 Public: goo.gl/BqfYSi 30 Minute Summary
  • 3.
  • 4. Explore everything PASS has to offer Free online webinars Local user groups around the world Free 1-day local training events Online special interest user groups Business analytics training Get involved Free Online Resources PASS Blog White Papers Session Recordings Newsletter PASS Connector BA Insights www.pass.org
  • 5. Session evaluations Your feedback is important and valuable. Submit your feedback by the end of this session to win a prize. http://hiram/evalsgoo.gl/u3eN9m
  • 6. BIO Father, SQL Server DBA/Dev, IT Pro, GRT, USCG Auxiliary. *Dev since 1995. *SQL since 1999. FUN Wakeboard, Wakesurf, Snowboard, GORUCK, Family. /HiramFleitas hiramfleitas@hotmail.com dba2o.wordpress.comHiramFleitas @HiramFleitas Hiram Fleitas Database Administrator (DBA), @UPCIC /hfleitas
  • 7. Agenda • Schedule • Keynotes • Session notes and take-aways • Demos • Follow-up resources
  • 8. Schedule Attended Time Monday Tuesday Wednesday Thursday Friday 8 am Data Modeling with Power BI Columnstore Indexes - Practical Solutions & Techniques Keynote Day 1 Keynote Day 2 2017 Intelligent Diagnostics 10 am Benchmarking & Testing IMOLTP Performance Making Developers Lives Easier with SQL 2016 3 Hard Performance Problems 11 am T-SQL Tips and Tricks BI Toolbelt Statistics & Power BI 1 pm Hacking SQL Server 2017 Deep Dive 3 pm Unlock your Data by Integrating Analytics into your Business Apps 2017 The Fast and the Furious Avoid Common Database Development Pitfalls 4 pm Linux Administration for the SQL Server DBA 5 pm Preparing to be Hacked Everything about Data Partitioning *Highlighted favorites. **See full summit schedule here: http://www.pass.org/summit/2017/Sessions/Schedule.aspx
  • 9. Columnstore Indexes Notes: • Columnstore Indexes: Practical Solutions & Techniques. • 2016 Brings SIMD for new hardware. • “Rowstore is dead” – some say in the next 5yrs. • Biggest pain is lack of parallelism. • 2 Main components: • Columnar Storage & Batch Execution • Rowgroups size: 1-1mil (just over 1 million). • 10K rows ignore CIDX. 10K is not big data. • Rowstore all values stored together. • Columnstore index stores columns together selected. • If you want to modify data don’t use column store. • Phases: 1. Shuffling (no order) 2. Rowgroups creation (split the data) horizontal. 3. Segment separation (split the data) vertical. 4. Compress (some segments may compress w/ better ratio). • Very aggressive “read-ahead”. • Standard edition is 64KB extents. • Enterprise edition is 512KB. • Columnstore indexes is 16MB extents. • 2014 CIDX 10% of Buffer Pool. • 64GB of RAM = 6GB for CIDX only. • Get SP1+ Updates. Niko Neugebauer Full Day Performance Notes: • Must use Partitioning if 10+ mil rows. If not kittens are suffering. • Must use Resource Governor. • Every single partition is its very own column store. • Partitioning solves dictionary problem. • Execution plan decompresses after aggregation. • Data compression columnstore 130MB vs. columnstore_archive 60MB. • Backup will be faster, takes LOB & shifts it. • Restore will be faster. • Tuple mover shutdown Trace Flag 634. • Hunts for delta-stores & destroys them every 5 minutes. • Alter index reorg with (compress_all_row_groups = on) • Closes all delta-stores. • Deleted buffer stores IDs of deleted Bitmaps. (2016-2017) • Natively compiled stored procs ignore column store indexes. • Schema only. • IMOLTP freezes schema. • 2017 Adaptive Query Processing solving hash joins vs nested loops for cached execution plans. • Trivial Plans auto upgrade for CIDX to full optimized plans. • See all new extended events on 2017 RTM. • Batch Mode uses 64 - 964 rows at a time (shifts rows) vs Row Mode uses GetNextRow 1 row at a time per operator in execution plan. • Paul White has a list of conditions for not getting a full optimized plan. • Trivial means row execution.
  • 10. Columnstore Indexes Notes: • Where 1 = (select 1). Forces full optimized plan. (Parallelism) • Dbcc CSIndex – Never use anywhere else besides Dev. • Internals: sys.dm_column_store_object_pool • Do not use columnstore in OLTP. Updates will lock rowgroups. • (Tablock) in 2016 uses bulk load API. Insert … Select. • If you have too many bulk load running, if memory pressure >25 seconds goes to delta store. • “If a feature works as advertised, you didn’t test it well enough.” • Change Buffer Size in SSIS Data Flow. 100MB (2014) • See docs: Data Loading Performance Guide. • Never update the columnstore index. • Use delete + insert instead. • Force Batch mode: Create fake table with CSIDX, left outer join on 1=0. Problems / Fixes: 1. Delta Stores: Alter index reorg to compress. (compress_all_row_groups = on). TF 634 disables tuple mover. 2. If VM use L2 Cache Size. 3. Don’t multiply within aggregation. Key Take-aways: • Must use Partitioning • Must use Resource Governor • Force batch mode with fake table. • Batch Mode does not support Nested Loops, avoid them at almost all cost. • Columnstore Blogpost: http://nikoport.com/columnstore • Columnstore Library: http://github.com/NikoNeugebauer/CISL • Slides and Scripts: https://goo.gl/CeaycW Niko Neugebauer Full Day Performance
  • 11. T-SQL Tips and Tricks Itzik Ben-Gan Wed, Nov 1 10:45 AM - 12 PM Performance Notes: • Must have Columnstore to enable batch mode processing. • Use a filtered nonclustered columnstore index. • See also windowing functions. sum(val) over (partition by ActID order by TranID) as Balance from transactions • 74 sec - 5 sec. create nonclustered columnstore index idx_yesplease on transactions (ActID) where ActID = -1 and ActID = -2 • Distinct window aggregates: dense_rank() • Joins: Use () around left outer joins. With on a.col = c.col outside the (). • Option (force order). Forces order of joined tables. • Merge tip and exists. (select … except …) Key Take-aways: • Source Code: http://tsql.solidq.com (go to Resources, Downloads) • Slides: Itzik_Ben-Gan_T-SQL_Tips_and_Tricks.pdf • Scripts: Itzik_Ben-Gan_T-SQL_Tips_and_Tricks.sql
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27. Keynote Day 1 Rohan: • Data + Cloud + AI • Azure has the world largest cloud database footprint. • Modern Data Estate: Hybrid. • SQL Server 2017 downloads: 2M+ in 17 months. Post GA 500K in 1 month. • 30% discount for SQL Server 2017 and Red Hat Linux. • Azure: 700TB daily telemetry data helps anticipate needs for cloud and on-prem. • Azure Data Factory. Hourly pricing with no SQL Server license required. • Power BI Premium capacity, scale, extends on-prem, embedded analytics. Power BI Report Server on-prem. Bob / Conor: • HP ProLiant DL580 Gen10 Server: Read 6.5B rows (15sec on SSD), (2sec on Scalable Persistent Memory). • Automatic Tuning, blip in the matrix, fixes parameter sniffing problems. Tobias / Mihaela: • docker ps • git clone https://github.com/erickangMSFT/wwi-db • cd wwi-db • docker-compose build • docker-compose up • docker ps • Microsoft SQL Operations Studio Danielle: • 1.4M rows insert/sec. Real-time scoring using native SQL Machine Learning and Azure Machine Learning. In-Memory OLTP + Columnstore Index. • Healthcare IoT demo length of stay. Demo patient 45 yr old, male, 192lbs, 29.8bmi, surgery 2015, allergic to fish and tree nuts, complaints of asthma from forest fire, latest lab results, admits patient and estimated 4 days. • Context: Recent car company launched IoT vehicle that needed 300K tps. • Azure Machine Learning Workbench + Jupyter Notebook to build ML model that predicts Length of Stay which imports into SQL Server using utils.write_rts_model. use Ignite_Demo; select @nativeModel = value from rts where id = 'RF' --(random forests) select LengthOfStay_Pred from PREDICT (model=@nativeModel, data=dbo.LoS as input) with (LengthOfStay_Pred float) p where input.eid=@eid; • Predictions latency <20 ms using native SQL Machine Learning functions. Scott: • Biml: Business Intelligence Markup Language. DW automation. • Automatic generation a cloud hybrid solution in ~300 lines of biml code. • Starts with inspecting on-prem source schema. • Creates azure datalakes target tables. • Creates an azure data factory to load. • Uses an SSIS package for address scrubbing. • Runs PowerShell to deploy it all automatically to the cloud so its ready for execution. Rohan Kumar Bob Ward Conor Cunningham Tobias Ternstrom Mihaela Blendea Danielle Dean Scott Currie Julie Strauss Christian Wade Riccardo Muti The Rolling Stones and Beatles of All Data and Cloud
  • 28. Keynote Day 1 Julie: • 100TBs read from disk, folded multiple times, returned in 6 seconds. Processing Azure SQL telemetry funnels and cohorts. • 100x faster queries. 4.5B rows/sec. 360GB/sec. • 5x more compute power. 30K DW units. 4K vCPUs provisioned in minutes. • 1st Cloud DW with NVMe SSD. Christian: • Power BI: NYC Taxi Interactive 10 billion. NY Taxi data model. Each row in the trip table represents a NY taxi ride across a 10yr period. 10bn rows (9TBs of data). • Revenue per trip by the top 10 zip codes. Visualize it on a map. • Instant gratification. Clicky clicky draggy-droppy data analysis over massive datasets. • Interactive charts, click one of the bars in the chart, we can see the highest revenue pickup location is JFK. Zoom map into the individual pickup locations. • Build models in Visual Studio 2017 and SSDTS. This is the crown jewel for business calculations. Shares same connectivity stack as Power BI. • Deploy model to on-prem SQL Server or Azure Analysis Service - completely hybrid. • Azure Analysis Services scale out by simply dragging a slider bar across the number of read-only replicas with copies of the data you need (max: 7). • Queries get load balanced across replicas via Round Robin. Ricardo: • NYC Taxi Interactive 10 billion demo. • Imports ZipCodes demographic info: med income, avg income, population. • Models DropoffGeography to ZipCodes within Power BI. • Tipping by zip code to correlate median income to how much ppl tip (avg tip %). Drill through huge outliers to see details of specific metrics and trends. For example, 0 miles, 1 minute, fare $2.50, tip was $197. • Drill back up, graphical filtering, get rid of the outliers, right click exclude, graph auto adjusts. • See optimized view on phone or from any device. Key Take-aways: • Automatic Tuning • Docker • Length of stay • Biml • Power BI NYC Taxi Demo • Video: https://youtu.be/NGbk9XGWTHI Rohan Kumar Bob Ward Conor Cunningham Tobias Ternstrom Mihaela Blendea Danielle Dean Scott Currie Julie Strauss Christian Wade Riccardo Muti The Rolling Stones and Beatles of All Data and Cloud
  • 29. DEMO • Performance 2017 vs 2016 • 2017 Automatic Tuning • Docker • Power BI
  • 31.
  • 32.
  • 33. Sponsor 3140 Northwoods Parkway, Suite 700, Norcross, Georgia 30071 // Phone: 1 (877) 531-7466 // Fax: (678) 495-1302 // www.CentricsIT.com United States // Canada // United Arab Emirates // United Kingdom // Czech Republic // Australia
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
  • 41.
  • 42.
  • 43.
  • 44.
  • 45.
  • 47.
  • 48.
  • 49.
  • 50.
  • 51.
  • 52.
  • 53.
  • 54.
  • 55.
  • 56.
  • 57.
  • 58.
  • 59.
  • 60.
  • 61.
  • 62.
  • 63.
  • 64.
  • 65.
  • 66.
  • 67.
  • 68.
  • 69. Hacking SQL Server André Melancia Wed, Nov 1 1:30 PM - 2:45 PM Support Notes: • Man In The Middle Attack: Adds Aliases, edits hosts file on clients. • TCP Proxy, Wireshark. Change data on the fly presented to the victim client. • Since SQL Server 2005 Login packet is encrypted, all other packets are not. • Force Encryption: YES! • AE: issue still see packet sizes so MITM can destroy them. • Deterministic: allows equality lookups. Cypher is always the same for a given value. • Instant File Initialization: information is not cleaned from HD, can be recovered/read. • TDE: Server has the keys. Data is decrypted before leaving the server. • DDM: is not a security feature. Requires unmask permission by database – not per column or table. Inner join the masked column character by character to a letters table. Scooby Doo! • RLS: Field combination division by value in the where clause still generates division by zero error. Create alert/xevent for continuous division by zero msg/sec. Key Take-aways: • In 2003 SQL Slammer: 75K victims in 10 minutes causing a DDoS general Internet outage. Patch had been released 6 months earlier. Must patch servers! • SQL Network Config, Protocols for MSSQLServer, Force Encryption: Yes (Always) • Enable Encrypted Connections: http://msdn.microsoft.com/library/ms191192.aspx • Encryption Without Validation: http://msdn.microsoft.com/library/ms131691.aspx • Connection String Keywords: http://msdn.microsoft.com/library/ms130822.aspx • Encrypt, Trust Server Certificate. • Slides and Scripts: http://andy.pt/doc • Video: https://youtu.be/VeGtD0Wf3mg
  • 70. DEMO • Man-In-The-Middle (MITM) • Hack DDM • Hack RLS
  • 71. MITM
  • 72.
  • 73.
  • 74.
  • 75.
  • 76.
  • 77.
  • 78.
  • 79.
  • 80.
  • 81.
  • 82.
  • 83.
  • 84.
  • 85.
  • 86.
  • 87.
  • 88.
  • 89.
  • 90.
  • 91.
  • 92.
  • 93.
  • 94.
  • 95.
  • 96.
  • 97.
  • 99.
  • 100.
  • 101.
  • 102.
  • 103.
  • 104.
  • 106.
  • 107.
  • 108.
  • 109.
  • 110.
  • 111.
  • 112.
  • 113.
  • 114.
  • 115.
  • 116.
  • 117.
  • 118.
  • 119.
  • 120.
  • 121.
  • 122.
  • 123.
  • 124. Keynote Day 2 Notes: • Globally Distributed Made Simple: Azure Cosmos DB. • 90% of the world’s data was created in the last 2 years. • 50-150x growth from 2010 to 2020 (9ZB). Sensors, Social, VoIP, Enterprise. • Emails: 204mil/min, Google Search: 4mil/min, WhatsApp Photos: 347k/min. • Butterfly-effect of data are consequences across continents. • Data is growing: Currently 125EB 79.2% is unstructured, 20.8% is structured. • 80% of business-relevant originates unstructured. • Push computation to where data is produced. • Address the needs of 1st-party customers: Xbox, Bing, O365, Store. • Reqs: turnkey global, guarantee low latency, HA, consistency, elastic, SLA, low cost, indexing, APIs, schema-agnostic, etc. VERY HARD! • Millions of lines of code (mainly C++) • Ring 0 service. Means in every region. 36operational/42built regions. • Database Account, Database, User, Permission, Container, item, attachment, sproc, trigger, udf, conflicts. Database scales in terms of containers. • Regions > DCs > Stamps (Floors/Suites) > Fault Domains (racks) > Clusters (10-20 FDs, 1K tenants/cluster) > Machine (200-400 replicas/tenant, hundreds of tenants/machine) > Containers > Replicas > [Resource Governor, Transport, Admission control, DB Engine] > DB Engine Instance [Secret Sorcery: Resource Manager, Language Runtime Hosts, Query Proc, RSM, Index Mgr, Bw- tree++/LLAMA++, Log Mgr, IO Mgr]. • Partitioning: Teants > [Tables, Collections, Graphs] > Containers > Resource Partitions > Replica set: Leader (writes), (reads) 2 followers + 1 forwarder -> to remote partition(s). Sets allows for many topologies: Chain, Star, Ring, Tree. Notes: • Partitioning BP: Even distribution by partition key. Location aware key for access locality and transaction scope. ACID at the scope of a partition. Time ranges for heavy workloads for more even distribution. • Choose failover priority by region. Simulate a region failover. • Polcy-based Geo-fencing. Replicate data with control. • Data is AlwaysOn. Active writers and Active Readers. • Elastic: Independently scale storage & throughput. Transparent server side partition management and routing. Auto indexed SSD storage. Evict old data using built-in support for TTL. How do you deal with AsyncIO? More SSD…, is there Cosmos App/Client? • Scale from 10-100mil trans/sec across regions. Responsive partition management. Modular, resource governed nested consensus. (Time zones: LA vs CHI) • Request Unit/sec (RU) is the normalized currency (%RAM, %CPU, %IOPS). Replica gets a fixed bud of RUs. Ops consume RUs: Get(Read), Post(Insert), Put(Replace), Post(Query). Min RUs – Replica Quiescence, Max RUs – No throttling, Above – Rate limit. Customers pay for reserved RUs. • Query Cost: Scans, Index Lookups, Query Complexity, Instruction overhead. • As partitions are split/merged tran/sec are evenly distributed per partition. If you add regions, you should double RUs per region to avoid starving a particular region. • Lower throughput when not needed ie. after Christmas shopping reduced throughput from 1 trillion RUs to minimum vs on-prem having to incur all the cap-ex. • Internal Microsoft workloads 3 trillion RUs across 20 regions in 3 days. 2017 Sept 7- 10. Rimma Nehme GPM, Azure Cosmos DB, Microsoft
  • 125. Keynote Day 2 Notes: • Guaranteed Low Latency: Read/Writes served from local region. • 99th percentile (R. 1KB) <10ms, (W. 1KB) <15ms. • 50th percentile (R. 1KB) <2ms, (W. 1KB) <6ms. • Automatically indexed SSD storage. • Centralized database in Chicago is read from London in 80ms. CosmosDB 10ms! CosmosDB beats physics of speed of light. Lol • Constancy problems happen, what if A & B can communicate but cant connect to C which customers are writing to C. • Strong consistency high latency means until the whole quorum agrees you cant see the data. (Blue Pill) • NoSQL focuses availability, no guarantees about consistency but you’ll get your data no matter what. Eventual consistency low latency. (Red Pill) • Most real-life applications do not fall into these extremes, they are somewhere in between. Real world consistency is not binary choice. • Checkout Paper “Replicated Data Consistency Explained Through Baseball” by Doug Terry 2011. Relates real word examples depending who you are in the game. • Consistency models in Azure CosmosDB: Bounded-stateless (bound for your staleness of your data, how many writes your reads will lag behind), Session (monotonic reads & writes in your local region, perfect consistent data to your local region), and Consistent prefix (guarantees when updates are applied there’s no gaps in prefix return). • Using TLA+ specs to precisely specify consistency models to describe what would happen and how to address it. CosmosDB credits TLA+ by Dr. Leslie Lamport for flexibility in consistency offerings and famous for paxos algorithm. Notes: • Consistency usage: Session 73%, Bounded Staleness 18%, Strong 4%, Eventual 3%, Consistent Prefix 2%. Tradeoffs can be monetized. • High Availability SLA is no longer good enough! A true measure in a service business is it’s SLAs. 1. Latency @99th percentile. 2. Throughput. 3. Consistency. 4. Availability. • Azure portal Metrics blade shows runtime telemetry statistics against SLAs within region and across regions. • DB engine operates on (ARS) Atom-Record-Sequence data model. Onboards other models: Key-Value, Column-Family, Document and Graph. Single client can connect to same container and extrapolate data in different ways. Ie. Document vs Graph. • Schema agnostic leads to store the data as is not compartmentalized. All parameters of the data are stored in one document. Humanly readable as a JSON document. • Query represented as a tree and results yet as another tree as a JSON doc. • Indexes are union of all trees keeping only unique values to each of the documents. So its an inverted index for JSON-path. For more detail see: Schema-Agnostic Indexing with Azure Cosmos DB. • Updates are stored as deltas so they never block or conflict. • Create udfs and stored procs using JavaScript integration with the DB Engine. Runs natively by the hosted runtime. • Lift and shift across platforms or vendors are easier to cloud and from on-prem. • Running the Service: Weekly deployments of entire stack world wide. Automated performance, RG and consistency runs every 4hrs. 16+ hrs of stress test runs daily. Full stack upgrades with customer workloads. Chaos tests. Bots automatically identify problems and fix nodes that might go down. Hot fix all invariant violations within 5 days. Transparently making all important metrics available to customers. Rimma Nehme GPM, Azure Cosmos DB, Microsoft
  • 126.
  • 127. Keynote Day 2 Notes: • You don’t get pearls without irritating the grid. • The world of data is divided in 2 groups, those who use Cosmos DB and those who will eventually use Cosmos DB. • Its not the strongest species that will survive nor the most intelligent, it’s the one that’s most adaptable to change.- Darwin • Try Cosmos DB for free, you don’t have to have an azure account, you don’t have to have a credit card. You can just come in and experience the service. • Your childhood dream of going into Cosmos will finally be fulfilled. Become a Cosmos DBA. Key Take-aways: • NoSQL database built by Microsoft. Addresses pinpoints when building large scale applications. Single digit millisecond latency. • Replication data world-wide with a single click. • Legit trial – no bs. • Global distribution, horizontal partitioning, fine grain multitenancy cannot be an after thought. These things are built in from the ground up. • Schema agnostic database engine design is crucial for globally distributed databases. Create/Alter index across regions, datacenters and planet becomes too rigid. • Intermediate consistency models are extremely useful. You can make the right trade- offs. With the SLAs the system gives you can guarantee the correctness of the approach. • A globally distributed database must provide comprehensive SLAs, fundamentally in a service business. • Ask yourself how would you build a database if we were starting today? Without none of the legacy. Would we look at it differently, limitations, constraints, biases. • Life is too short to build something nobody wants. • If we focus on real pain points not nice to have, abandon the constraints and self- imposed dogmas we can have a real modern database system. • Video: https://youtu.be/7SRi9vyDtWY Rimma Nehme GPM, Azure Cosmos DB, Microsoft
  • 128. SQL Server 2017 Deep Dive Tobias Trenstrom Travis Wright Shreya Verma Nellie Gustafsson Thurs, Nov 2 1:30 PM - 2:45 PM Build Notes: • Choice of platform (Linux, Windows, Docker) and language (T-SQL, Java, C/C++, C#/VB.NET, PHP, Node.js, Python, Ruby). • Industry-leading performance and security now on Linux and Docker. • Most secure over the last 7 years. MySQL and Oracle had the most vulnerabilities. • Only commercial DB with AI built-in. R + Python + T-SQL at massive scale. • End-to-end mobile BI on any device at a fraction of the cost. • In-memory across all workloads. Most consistent data platform on-prem and cloud. • HA and DR for Linux and Windows. Clusterless AGs. Ultimate HA with OS-level redundancy and low-downtime migrations. Load balancing of readable secondaries. • AG enhancements: DTC_Support=Per_DB, Required_Synchronous_Secondaries_To_Commit=1 Notes: • Partner Ecosystem: Azure Site Recovery, DH2i nearest-to-zero downtime, HPE Serviceguard (SGLX) monitors and recovery options, IDERA SQL Safe Backup, Portwork for containers, Veeam Backup & Replication. • ML: Bringing intelligence to where the data lives. Native scoring using PREDICT. Sentiment Analysis demo. • Graph Database: Expresses pattern matching. MATCH, to support pattern matching and traversals. Create table () as node --entities. Create table () as edge -- relationships. Solve problems more intuitively and with less code. Scenarios: Recommendations, Fraud Detection, Content Management, Bill of Materials, CRM. • Resumable Online Index Rebuild. Alter index … Resumable=On, Pause/Resume. • Automatic Plan Regression Correction. sys.dm_db_tunning_recommendations. Alter database name set Automatic_Tunning (Force_Last_Good_Plan=On). • MAXDOP option for statistics create/update. Faster backups +other improvements. • Adaptive Query Processing: Interleaved Execution for MSTVFs - Wohoo!! Batch Mode Memory Grant Feedback (MGF) adjust grants based on execution feedback and remove spills improving concurrency for repeating queries. Batch Mode Adaptive Joins (AJ) will defer the choice of hash join or nested loop until after the first join input has been scanned. AJ uses nested loop for small inputs, hash joins for large inputs. • New TPC-H 10TB world record! New TPC-H 3TB $/perf world record! Key Take-aways: • Slides: PASSSummit-SQLServer2017DeepDive.pdf • Blog: blogs.msdn.microsoft.com/sqlserverstorageengine • GitHub: sentiment-analysis, sql-graph • ML Tutorials: aka.ms/mlsqldev • Data Science Tutorials and Samples: Solution templates
  • 129. DEMO • Multi-OS AG Failover • Get Sentiment • Get Length Of Stay • Fraud Detection with Graph • Is It Bacon?
  • 130. Resources Slides, Scripts, Recordings, Notes: • Recordings (UPCIC Internal): http://hiram/summit2017 • Public: https://goo.gl/BqfYSi • Sneak Peeks: http://pass.org/summit/2017/Sessions/SneakPeeks.aspx • PASStv: http://pass.org/summit/2017/PASStv.aspx • Keynote Day 1: https://youtu.be/NGbk9XGWTHI • Microsoft BI - An Integrated Modern Solution: https://youtu.be/T0If-SNl08A • Keynote Day 2: https://youtu.be/7SRi9vyDtWY • Keynote Day 3 (WIT): https://youtu.be/8JnXi0gnTb8 • Speakers: http://pass.org/summit/2017/Speakers.aspx • Bob Ward’s slides & scripts: http://aka.ms/bobwardms • Performance: Adaptive Query Processing, Automatic Tuning. • Docker: https://docs.microsoft.com/sql/linux/quickstart-install-connect-docker • Docker SQL, App, CI/CD: https://channel9.msdn.com/events/Connect/2017/T152 • Power BI galleries: https://community.powerbi.com/t5/Galleries/ct-p/Galleries • Power BI webinars: https://docs.microsoft.com/power-bi/webinars • Apply Forecasting: https://docs.microsoft.com/power-bi/desktop-analytics-pane More: • Twitter: @sqlpass, #PASSSummit, #powerbihelp, #sqlhelp, @MSPowerBI • Join Slack workspace: http://dbatools.io/slack • Slack channels: #dbatools, #firstresponderkit, #general, #pass-summit, #query- tuning, #sqlhelp, #git-help • Pass Virtual Groups: http://pass.org/Community/Groups.aspx (App Dev, BI, DBA, DevOps, Global Spanish, HA/DR, Performance, PowerShell, Virtualization, Linux) • More webinars: Channel9, GroupBy, SQLWorkBooks
  • 131. Session evaluations Your feedback is important and valuable. Submit your feedback by the end of this session to win a prize. http://hiram/evalsgoo.gl/u3eN9m
  • 132. Thank You Hit me up! hiramfleitas@hotmail.com dba2o.wordpress.comHiramFleitas @HiramFleitas /hfleitas/HiramFleitas