RML Utilities provide command line tools and interactive reports enabling you to:
Take SQL trace files (captured with SQL Profiler, sp_trace or extended events in SQL 2012+),
Process them into replayable RML files (using readtrace.exe),
Play them back in a different SQL environment (using ostress.exe),
And compare the performance at a granular level (using reporter.exe or custom queries).
Watch Re-runs on your SQL Server with RML Utilities
1. Watch Re-runs
OnYour SQL Server!
David Cobb
sql@davidcobb.net
All slides and content available
on my blog:
http://bit.ly/1HicQw9http://www.imdb.com/find?s=all&q=good%20times
2. David Cobb
SQLTrainer / Consultant
MCT, MCSE Data Platform for SQL 2012
sql@davidcobb.net
daveslog.com
Presenter
Past Roles:
Tech Support, Network Admin, Web
Developer, DBA
Current:
Systems Architect for CheckAlt.com
Training SQL Since: 2002
FavoriteTechs of the Moment:
PowerShell & Azure
5. What Are RML Utilities
Microsoft internal tool used by the SQL support team
Precursor to Distributed Replay Client introduced in SQL 2012
Has both replay and stress test modes.
Supports SQL Server versions 2005 thru SQL 2014.
Not officially supported! Good email responsiveness from the support team
though. sstlbugs@microsoft.com
6. WhyWould I UseThem?
PRE-SQL 2012 (otherwise look at the Distributed Replay Client,
it IS supported)
Testing effect on SQL statement performance from
environment changes using real production query load
(including inserts/updates/deletes) .
Test with new version of SQL (or hotfix, CU, or SP)
Test with different server or database settings (MAXDOP changes, RCSI)
Test with different files or disks (additional files, filegroups, different
RAID or LUN, SSDs)
Test index changes
Etc.
7. Where Do I Start?
RML Utilities Quickstart - Install the tool, (http://bit.ly/1IPGGfJ)locate the RML
Help.pdf and walk through the quickstart in a test environment to get familiar
with it.
Documentation is thorough, covering many use cases, but often at a high level of
technical detail.
8. How Would I UseThisTo Capture And Replay
Production Workloads?
Workflow diagram
Before you start:
Install RML Utilities and requirements (ReportViewer 2008) on test server
Create folders for config files, scripts, backups, traces and RML files on test server and
production
Optional: Install SQLNexus, PSSDIAG
(To be covered in a future presentation )
11. BASELINE CAPTURE:
On Production Server (SQL1)
Take a full backup in advance, plus tlog backups of all relevant user databases databases.
Note the time
Using a customized copy of SD_Detailed.XML (which is generated by running SQLDIAG ), with
ProfilerCollector disabled, start a SQLDIAG session to newly created folder.
sqldiag.exe /OC:RMLDemooutputsqldiagrun01
/IC:RMLDemoconfigSD_Detailed.XML
Also start capture of a SQL trace using sp_trace to a newly created folder.
using TraceCaptureDef.sql saving to a named folder (i.e. trace01 or trace0521).
exec @rc = sp_trace_create @TraceID output, 2 /* rollover*/,
N'C:RMLDemooutputtracerun01replay01' , @maxfilesize, NULL
Alternately, use StartSQLDiagForReplay2008.cmd from PSSDIAG
DON'T USE REPLAYTRACETEMPLATE IN SQL PROFILER!
12. BASELINE CAPTURE : Continued..
Start simulated workload if testing, or wait for production activity
When capture of production load is complete (timeframe up to you, 15 minutes, 1
hour?) stop the trace and SQLDIAG (ctrl-C in SQLDIAG window)
Take final tlog backups, then copy the following from production to the test server
(or other server that will analyze the results):
Trace files
SQLDIAG output
Full andTLog backups
14. DEMO : ReadTrace
I’m performing the demos that take some time so I can talk while they run..
15. PROCESS : OnTest Server (SQL2)
Using the ReadTrace tool
Use readtrace.exe from RML Cmd Prompt on the test server (or
any other), to process the trace files into replayable RML files
ReadTrace consumes as input trace (.TRC) or (.XEL) files and
.CAB or .ZIP files which contain .TRC files from a SQLTrace or
Extended Events
It generates .RML (Replay Mark-up Language) files and the
Performance Analysis (PerfAnalysis) database. (A populated
performance analysis database is required for reporter.exe
functionality.)
TIP! Save your command line statements in text file and paste
to console.
Track what statements and parameters were used exactly, and when.
Minimize typos.
16. ReadTrace Continued..
ReadTrace -I<PATH_TO_FIRST_TRACE_FILE> -o<PATH_TO_RML_OUTPUT_FOLDER>
-S<DBSERVER> -d<NEW_PERF_ANALYSIS_DB_NAME>
Note: No space between the parameter flag and the value in any RML Utilities commands!
Using ReadTrace this way creates RML files and stores performance in a new performance analysis
database
Many options for ReadTrace.exe
limit start time (-b) or end time (-e)
limit databases (-C), application (-A) or host (-H)
trace flags, MARS support, etc.. 5 pages of parameters in RML Help pdf, pg 35-40
ReadTrace –IC:RMLDemoFROMSQL1outputtracetrace01trace01.trc
–oC:RMLDemooutputrmlrun01-1 –SSQL2 –dPerfAnalysis_run01_1
-CAdventureworks2008R2
Upon completion, will open the result report using reporter.exe
Also review contents of RML output folder
17. REPLAY : OnTest Server (SQL2)
Prepare environment on test server
Ensure you won't have access to / affect production!
Restore databases up to point in time that you began the trace
Start SQLDIAG Session, just like on production
sqldiag.exe /OC:RMLDemooutputsqldiagrun01replay01
/IC:RMLDemoconfigSD_Detailed.XML
Also start capture of a replay trace using sp_trace, just like on production
exec @rc = sp_trace_create @TraceID output, 2 /rollover*/,
N'C:RMLDemooutputtracerun01' , @maxfilesize, NULL
18. DEMO : Ostress
Again, I’m performing the demos that take some time so I can talk while they run..
19. OSTRESS :The Replay Command
OStress is a scalable, ODBC based application which can replay database commands in two modes:
Replay mode orchestrates the order of the commands, to accurately simulate OLTP activity.
(i.e. Insert table 1, when done then update table 2, then delete table 3..)
Note that replay mode can take around 5 times longer that original capture, due to orchestration
overhead
Stress mode plays back SQL statements as quickly as possible, based on parameters you provide.
(i.e. Firehose mode)
No respecting order of statements, not useful for Inserts/Updates/Deletes with Foreign Key
relationships.
Many parameters for this tool as well
RML Help.pdf pg 93
20. Use OstressTo Start Replay
Start replay with ostress.exe from RML Cmd Prompt
ostress -E -SSQL2 -iC:RMLDemooutputrmlrun01*.rml
-cC:RMLDemoconfigostress_config.ini
-oC:RMLDemooutputostressrun01replay01 -mreplay
(again, no spaces between parameter flags and their values)
The ostress command and its parameters:
-E,-S server info
-i path to RML files to replay
-c Configuration file (copied from RML samples)
-o Output path for replay result
-m Mode is replay.This plays back in order. Also can be stress mode, aka fire hose mode
Once replay complete, stop the trace and SQLDIAG (ctrl-C in SQLDIAG window)
21. PROCESS : OnTest Server (SQL2)
Now we need to analyze the results of the replay on the test server.
We use readtrace.exe from RML Cmd Prompt again, but this time use -f
parameter just to create the performance analysis database, not to create
another set of replay files.
ReadTrace -IC:RMLDemooutputtracerun01replay01.trc -f
-SSQL2 -dPerfAnalysis_run01_replay01 -C"AdventureWorks2008R2"
22. ANALYZE : OnTest Server (SQL2)
reporter.exe runs once readtrace.exe is
complete. From Properties tab, enter the names
of both the performance analysis databases
(Baseline and test) for reports that compare
results.
Review ER Schema of the PerfAnalysis database.
Understand how we can query these tables for
our own analysis
Use Performance Analysis Comparison Script.sql
Dave is on github!
https://github.com/dave-007/RML-Utilities-Demo
23. BUTWAIT,THERE'S MORE!
SQLNexus
Visual reports from SQL traces (generated using PSSDIAG script) and SQLDIAG
http://sqlnexus.codeplex.com/
PSSDIAG Manager
RML Utilities big brother. More features
https://diagmanager.codeplex.com/
PAL
Gathers performance monitor logs and generates visual reports.
Compares metrics to thresholds defined by Microsoft geniuses!
https://pal.codeplex.com/
25. RESOURCE LINKS
Description of the Replay Markup Language (RML) Utilities for SQL Server -
https://support.microsoft.com/en-us/kb/944837/
Download RML Utilities for SQL Server (x64) CU4 -
https://www.microsoft.com/en-us/download/details.aspx?id=4511
Dependency: Download ReportViewer 2008 SP1 -
https://www.microsoft.com/en-us/download/confirmation.aspx?id=3841
SQL Nexus - http://sqlnexus.codeplex.com/
SQL Server 2012: RML, XEventViewer and Distributed Replay -
http://blogs.msdn.com/b/psssql/archive/2012/04/06/sql-server-2012-rml-xevent-viewer-and-distributed-
replay.aspx
AdventureWorks RandomWorkload from Jonathan Kehayias -
https://www.sqlskills.com/blogs/jonathan/the-adventureworks2008r2-books-online-random-workload-
generator/
Distributed Replay walkthrough -
http://blogs.msdn.com/b/mspfe/archive/2012/11/08/using-distributed-replay-to-load-test-your-sql-server-part-1.aspx
RML Demo files on Github: https://github.com/dave-007/RML-Utilities-Demo
Hinweis der Redaktion
OVERVIEW:
Have an idea for a hardware product, but not sure where to start? Join us at the Microsoft Innovation Center at Venture Hive for a crash course in both building a hardware product & a business!
The course will last for eight weeks, geared for students & local community members alike. There will be weekly sessions featuring industry leaders in both hardware & business development. We will cover topics all across the spectrum, from internet of things to key performance indicators in a business, from wearables to legal & IP considerations in a business.
Come to one or all of the sessions.
