Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Running SSIS 2017 at Scale Everywhere
Ähnlich wie Running SSIS 2017 at Scale Everywhere (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Running SSIS 2017 at Scale Everywhere
Hinweis der Redaktion
- A request to execute multiple packages will automatically distribute the packages across multiple machines for execution. Scale-Out Master and Scale-Out Worker can be installed side-by-side on a machine, while Scale-Out Workers can be installed each on a different machine. SSISDB (Catalog) is now extended to store info about Scale-Out Workers and parallel package executions. Familiar + new interface via SSMS for customers to enable Scale-Out Workers and execute packages in parallel.
- Master Service retrieves package execution requests submitted by customers via SSMS to SSISDB. Worker Services pull package execution tasks + contexts (packages + runtime info) from Master Service, initiate SSIS engine to execute packages locally in parallel, and write execution logs into SSISDB, so customers can monitor/diagnose their executions via SSMS. By default, new Windows accounts will be created at installation on host machines to launch Master Service and Worker Services.
- Customers submit a request to execute Package A – Master Service stores “Execute Package A” task Worker Service requests a new task to execute from Master Service Master Service identifies “Execute Package A” task as the next one scheduled for execution Master Service assigns “Execute Package A” task to Worker Service 5. 6. 7. Worker Service requests and receives the task execution context (Package A + runtime info) Worker Service initiates SSIS engine to execute Package A with its runtime info 9. 10. 11. Worker Service monitors the task execution, updates its status regularly, and writes execution logs After the task is completed, Worker Service cleans up the execution environment, updates the final status, and pulls another task.
- On SQL Server Installation Wizard’s “Feature Selection” page: Side-by-side installation on one machine is optimal, since Master Service is a light-weight process that needs little compute resource. Side-by-side installation will automatically generate Scale-Out Master’s connection endpoint for Worker Service and omit SSL client certificate exchange to secure Master-Worker communications.
- On SQL Server Installation Wizard’s “Server Configuration” page: Master Service is ”SQL Server Integration Services Scale Out Master 14.0” and its default Windows account is “NT Service\SSISScaleOutMaster140”. On SQL Server Installation Wizard’s “Database Engine Configuration” page: Select “Mixed Mode (SQL Server authentication and Windows authentication)” to enable SQL Server authentication and click "Add Current User" button, otherwise Worker Services will not be able to write execution logs into SSISDB.
- On SQL Server Installation Wizard’s “Integration Services Scale Out Configuration – Master Node” page: Selecting “Create a new SSL certificate” will create a default, self-signed server certificate and install it under Trusted Root Certification Authorities, Local Machine. By default, Scale-Out Master name and IP address are included in the self-signed certificate CNs (Common Names). Selecting “Use an existing SSL certificate” and clicking ”Browse” will display server certificates that are already installed under Trusted Root Certification Authorities, Local Machine. Once an existing certificate is selected, its thumbprint is displayed. Either way, the corresponding client certificate should be installed on all Scale-Out Workers for Worker Services to authenticate Scale-Out Master.
- On SQL Server Installation Wizard’s “Server Configuration” page: Worker Service is ”SQL Server Integration Services Scale Out Worker 14.0” and its default Windows account is “NT Service\SSISScaleOutWorker140”. During Scale-Out Worker installation on multiple machines, a default, self-signed SSL server certificate will be automatically created and installed under Trusted Root Certification Authorities, Local Machine. All corresponding client certificates should be installed on Scale-Out Master for Master Service to authenticate Scale-Out Workers.
- On Windows Firewall app: Open Scale-Out Master’s ports for Worker Services to communicate with Master Service (8391 by default) and write execution logs into SSISDB (1433 by default). Add predefined Remote Service Management (NP-in) inbound traffic rule/open Scale-Out Workers’ port 445 for Scale-Out Manager to access Scale-Out Workers’ certificate store.
- On Windows Services app: Make sure Master/Worker Services are started on Scale-Out Master/Workers, so each Scale-Out Worker can register to Scale-Out Master and get a unique WorkerAgentId stored in [catalog].[worker_agents] view of SSISDB.
- On SQL Server Installation Wizard’s “Integration Services Scale Out Configuration – Worker Node” page: Scale-Out Master’s connection endpoint consists of Scale-Out Master’s name/IP address + port specified during Scale-Out Master installation. If the default, self-signed SSL server certificate has been selected during Scale-Out Master installation, the corresponding client certificate (SSISScaleOutMaster.cer) can be found on Scale-Out Master in <drive>:\Program Files\Microsoft SQL Server\140\DTS\Binn\ folder. It should be copied to a folder on Scale-Out Workers, so it can be referenced and installed under Trusted Root Certification Authorities, Local Machine during Scale-Out Worker installations. During Scale-Out Worker installation on multiple machines, a default, self-signed SSL server certificate will be automatically created and installed, while the corresponding client certificate (SSISScaleOutWorker.cer) can be found on each Scale-Out Worker in <drive>:\Program Files\Microsoft SQL Server\140\DTS\Binn\ folder. All of them should be copied to a folder on Scale-Out Master, so they can be installed under Trusted Root Certification Authorities, Local Machine. Alternatively, you can use Microsoft Management Console (MMC) to import/export certificates from/to Scale-Out Master/Workers or Scale-Out Manager to set Scale-Out Master’s connection endpoint in Scale-Out Worker’s config file and exchange Scale-Out Master’s/Worker’s SSL client certificates automatically (SSIS 2017).
- On SSMS connected to Scale-Out Master: Create SSISDB and select “Enable this server as SSIS scale out master” on “Create Catalog” dialog window. Enable each Scale-Out Worker by using its WorkerAgentId from [catalog].[worker_agents] view of SSISDB as a parameter to execute [catalog].[enable_worker_agent] sproc Alternatively, you can use Scale-Out Manager to enable Scale-Out Master/Workers automatically (SSIS 2017).
- Run SSMS as administrator, since some operations on Scale-Out Manager, e.g. adding Scale-Out Workers, require administrative privileges. On SSMS connected to Scale-Out Master: Right click on SSISDB under Integration Services Catalogs and select “Manage Scale Out” to open “Scale-Out Manager”.
- Scale-Out Worker’s config file is stored in <drive>:\Program Files\Microsoft SQL Server\140\DTS\Binn\WorkerSettings.config folder. Worker Service must be restarted after updating the config file.
- On SSMS connected to Scale-Out Master: Right click on SSISDB under Integration Services Catalogs and select “Execute in Scale Out”. On “Package Selection” page: Select any package to set its run-time parameters/connection managers/advanced options, e.g. “Logging level”, “Retry count”, “Dump on errors”, etc.
- On “Machine Selection” page: By default, all Scale-Out Workers are allowed to execute selected packages in parallel.
- How to do this: https://docs.microsoft.com/en-us/sql/integration-services/scale-out/scale-out-support-for-high-availability.