2014.11.22 Azure for Sql Server Developer - SQLSAT355 Parma

Azure for SQL Server Developer
Marco Parenzan
@marco_parenzan
marco.parenzan@live.it
#sqlsatParma
#sqlsat355 November 22nd, 2014

Sponsors
#sqlsatParma

Organizers
#sqlsatParma

Speaker info/Marco Parenzan
 Formazione & Divulgazione con 1nn0va
 Microsoft MVP 2014 for Microsoft Azure
 Formazione & Progettazione con ITS
Kennedy
 Ricerca e Innovazione con Servizi Cgn
 www.slideshare.net/marco.parenzan
 www.github.com/marcoparenzan
 marco [dot] parenzan [at] live [dot] it
 www.innovazionefvg.net
 @marco_parenzan
Cloud Architect
SQL Server Friend
Loves Functional Programming, Html5 Game Programming and Internet of Things
#sqlsatParma

Agenda
 Why Azure?
 OPS Experience
 DEV Experience
 BI Experience
 Conclusions
#sqlsatParma

WHY AZURE?
#sqlsatParma

Trends driving IT pressures & opportunities
#sqlsatParma

Utility IT
Example scenarios
Data storage
Cloud identity
Strategic
Utility IT
VMs on demand
IT
Disaster recovery
Deploying packaged applications
Moving existing applications to the public cloud
#sqlsatParma

The Microsoft data platform

#sqlsatParma

SQL Server 2014 built-in hybrid technologies
#sqlsatParma

Azure Data Services for Modern App Design
#sqlsatParma

Barriers to Public Cloud Adoption
Security
Can a public cloud platform
keep my data and applications
safe? ?
ANSWER
You must learn to trust your
public cloud provider
#sqlsatParma

Compliance
Can I still meet my regulatory
requirements in the public
cloud? ?
ANSWER
You must understand the rules
that apply to you
#sqlsatParma

Availability
Will public cloud datacenters
? be up when they’re needed?
ANSWER
They’ll be at least as good as
your own datacenter, and there
are SLAs
#sqlsatParma

The unspoken worry
Will public cloud platforms
? make me lose my job?
ANSWER
Probably not, but you’ll need to
learn some new skills
People who don’t like change shouldn’t work in the technology industry
#sqlsatParma

OPS EXPERIENCE
#sqlsatParma

Handle relational data with IaaS
 You can create a VM with Windows Server
 You can install any software you want
 if you have your licence
 If you need a licence
 In case you don’t have a licence, you have VM
images ready with software and pay as you go
(hour/minute basis)
 You can scale in “minutes”
 …or in “real” hours…
 …but not weeks (or at least days!)
#sqlsatParma

Virtual Machine Storage Architecture
Azure Virtual Machine
C:
OS Disk
E:, F:, etc.
Data Disks
D:
Temporary Disk
Disk Cache (Contents can be lost)
#sqlsatParma

The G family
Optimized for data workloads
Up to 32 CPU cores, 448 GB RAM,
6.5 TB local SSD
Latest generation Intel processor
G
#sqlsatParma

Handle relational data with IaaS
Pro
 Full power of a Relational DB
 Simple Scaling Up
 No licencing
 Pay Per Use (no CAPEX)
 Full power control
 Ready images (no installation)
 With specific configurations
 Just Click, Name, choose,
deploy
Cons
 OPEX: highest
 Relational (non open) DBMS
are expensive
 Bits+Management
 You have to be a real
Administrator
 Scalability: you can ONLY
scale up
 Current sharding needs
application to be updated
 Current sharding is in
experimentation stage
#sqlsatParma

DEMO
#sqlsatParma

One of the first two services (the other were Cloud Services)
AZURE STORAGE
#sqlsatParma

Azure Storage Architecture
“Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency”, ACM
Symposium on Operating System Principals (SOSP), Oct. 2011
#sqlsatParma

Durability Options
 Locally Redundant Storage (LRS)
 Stores 3 replicas of the data within a single zone (facility) in a single region
 Provides data durability for disk, node and rack failures
 Geo Redundant Storage (GRS)
 Stores 6 replicas of the data across two regions (3 in each region)
 Provides additional durability to protect data against major regional natural disasters
(e.g., tornado, hurricane, fire, etc, destroying a whole region)
 Updates across regions are performed asynchronously
 Zone Redundant Storage (ZRS)
 Stores 3 replicas of the data across multiple zones (facilities) within a single region or
across regions
 Provides additional durability to protect data against zone failures (e.g., fire burning
down a facility)
#sqlsatParma

Premium storage
Up to 32 TB of storage per VM
>50,000 IOPS per VM
Less than 1ms read latency
#sqlsatParma

Abstractions – Tables
 Tables – Massively scalable NoSQL cloud store
 Key/Attribute(s) store at scale
 Store user, device or any type of metadata for your service
 Auto load balances partitions to meet traffic needs
 OData protocol (AtomPub or JSON)
#sqlsatParma

Abstractions – Azure Storage Files
 Move on-premises
applications to cloud
 VMs can net use an SMB
share using standard file
APIs and semantics
 SMB 2.1 protocol
 VM and storage account
within same region
 Supports REST and SMB
protocol access to same
file share
Microsoft Azure
Share data stored in Azure Files
among Azure VMs via SMB
Azure Storage
Blobs
Tables
Queues
Files
SMB
RES
T
API
#sqlsatParma
RES
T
API

Abstractions – Blobs
 Blobs – Massively scalable object store in the cloud
 Simple REST interface (Put, Get, Delete)
 Data sharing – share documents, pictures, video, music, etc.
 Big Data – store raw data/logs and compute/map reduce over
data
 Backups – data and device backups
#sqlsatParma

Virtual Machine Storage Architecture
Azure Virtual Machine
C:
OS Disk
E:, F:, etc.
D: Data Disks
Temporary Disk
Disk Cache
G:, H:, etc.
SMB Share
#sqlsatParma

Make a SQL backup over Azure
 Backup works if
 move data in a far and safe place
 You do it on a daily basis
 It can be tested
 SQL Server 2012/2014 can backup over Azure
Storage
 Set azure credentials in SQL Server to authenticate on-premise
SQL over Azure (share a certificate between
Azure and on-premise host)
 Indicate the container and the blob you configure for this
activity
 Do the backup
 Restore backup to the new host
#sqlsatParma

Breaking News – Premium Storage
 Consistent low latency SSD
based with predictable IO
throughput
 Suitable for high-performance
IO-intensive database workloads
 Single digit milliseconds
latencies
 Supports up to 1 TB blob/disk
size
 Stripe up to 32 disks for a total
of
32TB and more than 50,000
IOPS
 Premium Storage Disks work in
conjunction with a new VM series
#sqlsatParma

Handle relational data with SaaS
 You don’t want to handle management issues
 You don’t want vertical scalability-related
issues
 You don’t need special infrastructural
architecture
#sqlsatParma

Microsoft SQL Platform Continuum
On Premises
Physical
Virtual
Platform as a Service
SQL
Hybrid
Cloud
SQL SQL
Infrastructure as a Service
SQL SQL SQL
SQL Server
Physical Machines (raw iron)
Azure SQL Database
Virtualized Databases
SQL Server in Azure VM
Virtualized Machines
SQL SQL SQL
SQL Server Private Cloud
Virtualized Machines + Appliances
Shared
Lower
Cost
Dedicated
Higher Cost
Off Premises
SQL
Higher Administration Lower Administration
#sqlsatParma

Azure SQL Database Service Tiers
• •• •••
• •• •••
#sqlsatParma

Scalable Performance
 Basic, Standard, and Premium provide increasing performance levels
 Relative performance is expressed in database throughput units (DTUs)
 DTU is a blended measure of CPU, read IO, write IO, and memory
 Scale performance up/down via portal, APIs, PS, or T-SQL
to reflect actual or anticipated demand
 Database remains online while scaling
 Hourly billing improves cost efficiency
B S0
S1
S2
P1
P2
P3
Performance is easily scaled up or down to
meet changing workload and business needs
#sqlsatParma

Data Protection
 Automatic, continuous backups
 Retention varies by service tier: Basic 7 days, Standard 14 days, Premium 35
days
 Daily backups also copied to a remote datacenter
 “Oops” recovery
 Self-service point-in-time restore from local backups, including deleted
databases
 Data center disaster
 Self-service restore from remote backups to any region at any time
 No charge for backups, normal database rates apply for restored databases
Basic, Standard & Premium databases are
protected from accidental data corruption, deletion
& disaster
#sqlsatParma

Handle relational data with SaaS
Pro
 OPEX: lower than IaaS
 Bits+Management
 Full power of a Relational DB
 Editions
 Clustering
 CPU+Memory
 No licencing (No CAPEX)
 Pay Per Use (OPEX)
 No need for images
Cons
 OPEX: lower, than IaaS, but
more than other
 Partial power of a Relational
DB
 …
 Configure what is just
available
 Scalability: can ONLY scale up
#sqlsatParma

Migrating your on-premise SQL to SQL
Azure
 Not all features of On-Premise SQL are
availavable in SQL Azure
 A “backup” is not enough to bring DB to the cloud
 Exporting a script
 Some tools are available, updated
 Developer Way: Database Project, targeting SQL
Azure
 Operations Way: Database | Tasks | Deploy to
Windows Azure SQL Database
 https://sqlazuremw.codeplex.com/
#sqlsatParma

SQL Azure performance considerations
 Microsoft is making huge investments
 Compatibility with full t-sql will improve
 DB per customer
 Hundreds/thousands of databases
#sqlsatParma

Process automation that simplifies cloud
management
Lower costs and
improve predictability
Automation
Enable service owners to focus
on work that adds business
value
Reduce error-prone manual
activities while lowering costs
Optimize and extend
existing investments
Integration
Integrate into existing systems with
PowerShell integration modules
Build additional PS modules to
enable integrating into other
systems
Deliver flexible and
reliable services
Orchestration
Accelerate time to value
with flexible process
workflows
Improve service reliability
across multiple tools,
systems, and department
silos
#sqlsatParma

Built on PowerShell Workflow
PowerShell
Workflow
•Use Windows PowerShell syntax
•Multi-device management
•Running a single task to manage complex,
end-to-end processes
•Automated failure recovery
•Connection and activity retries
Centralized
store
•Credentials / certificates
•Global variable
•Global connection for runbooks
•Modules
•Runbooks (draft / published versioning)
•Scheduling
Highly
Available
•Runbook servers to process jobs
•Odata Web service to submit / retrieve status
•SQL Server clustering / always on
Historical
Analysis
•Historical view of runbook jobs
•Reporting through Excel PowerPivot for ROI
•View runbook used for all jobs
PowerShell
Workflow
Centralized
store
Highly
Available
Historical
Analysis
Microsoft
Azure
Automation
#sqlsatParma

Runbook Authoring
 Author
 Create PowerShell Workflow
runbooks
 Call existing runbooks in library
 Manage and browse and insert
assets in runbooks
 Modules and Activities
 Credentials (PowerShell Credentials
and Certificates)
 Variables (standard and encrypted)
 Connections
 Schedules
 Test
 Run runbook and see results within
authoring window
 Troubleshoot issues
 Publish
 Edit draft before publishing
#sqlsatParma

Manage Runbooks & Jobs
 Dashboard View
 View runbook jobs over time
 Find jobs that may need attention
 Runbooks View
 Filtering of jobs based on status and
date
 Identify authoring state (New, In edit,
Published)
 Filter by tags to group related
runbooks
 Jobs View
 History of jobs per runbook
 Who last updated and when
 Input parameters and output
 Drill into each job to view streams
generated to help troubleshooting
 Stop, Suspend and Resume job
#sqlsatParma

Resource Manager
 Today Challenge
 Deploy or update a group of resources, repeatedly
 Manage permissions on a group of resources
 Visualize a group of resources in a logical view,
including monitoring/billing
 Resource Manager
 Application Lifecycle Container
 Declarative solution for Deployment and
Configuration
 Consistent Management Layer
#sqlsatParma

Resource Groups
 Tightly coupled containers of multiple resources of similar
or different types
 Every resource *must* exist in one and only one resource
group
 Resource groups can span regions
 Resource Group is a unit of management
 Lifecycle: deployment, update, delete, status
 Grouping: metering, billing, quota: applied & rolled up to
group
#sqlsatParma

Instantiation of repeatable config.
Configuration  Resource Group
Power of Repeatability
 Azure Templates can:
 Ensure Idempotency
 Simplify Orchestration
 Provide Cross-
Resource
Configuration and
Update Support
 Azure Templates are:
 Source file, checked-in
 Specifies resources
and dependencies
(VMs, WebSites, DBs)
and connections
(config, LB sets)
 Parametized
input/output
SQL-A
Website
[SQL CONFIG] VM (2x)
DEPENDS ON SQL DEPENDS ON SQL
SQL - A Website Virtual
Machines
SQL CONFIG
#sqlsatParma

DEV EXPERIENCE
#sqlsatParma

The big issue leading to CQ(R)S
 A single model (tipically
from a Database
schema) caring about
all aspects of the
domain is hard
 Complex aggregate
roots that fits costly into
a relational db
 mapping cost is higher
than access cost (crud)
 Eventual consistency is
a must
 Acid costs
Presentation layer
Application layer
Domain layer
Infrastructure layer
#sqlsatParma

Transaction Script
Data Storage
Application Services
Remote Facade
Client
Request DTO
DTO Returned
Send DTO
Status Returned
#sqlsatParma

Active Record
Data Storage
Entity Entity
Application Services
Remote Facade
Client
Request DTO
DTO Returned
Send DTO
Status Returned
#sqlsatParma

Command/Query Responsibility Segregation
 Domain Model in the Command stack
 Domain Model focused on behavior that alters state
 Ad-hoc storage for queries in the Query stack
 Table-per-view strategy
 Query through LET
Command/Query Separation
(cit. Bertrand Meyer – 1980s)
Query
> Returns data
> Doesn’t alter state
Command
> Alter state
> Doesn’t return data
#sqlsatParma

CQRS in a sketch
Domai
n
Object
Repository
Domain
Object
Command Handlers
Event Bus
Client
Send Command
Ack/Nak
Response
Data Storage
Event
Handler
Query Handlers
s
Request DTO
DTO Returned
#sqlsatParma

Write DB with DocumentDB
 Produce and consume data at
a staggering rate while
requiring near instantaneous
response times to match user
expectations
 Developed iteratively with
many versions supported
concurrently and continuously
evolving data models
 Increasingly complex,
compensating for the lack of
query and transactional
processing in today’s NoSQL
database systems
 Experience unpredictable and
explosive growth
 It is a fully managed, highly
scalable, queryable, truly
schema-free JSON document
database, delivered as a
service for modern
applications
SQL
query over schema-free
JSON
multi-document
transactions
tunable, high
performance
designed
cloud first
#sqlsatParma

The CAP Theorem
Theorem: You can
have at most two of
these properties for
any shared-data
system
Consistency
Availability
Partition
tolerance
#sqlsatParma

Consistency
 Two kinds of consistency:
 strong consistency – ACID(Atomicity Consistency
Isolation Durability)
 weak consistency – BASE(Basically Available
Soft-state Eventual consistency )
#sqlsatParma

Eventual Consistency
 When no updates occur for a long period of
time, eventually all updates will propagate
through the system and all the nodes will be
consistent
 For a given accepted update and a given
node, eventually either the update reaches
the node or the node is removed from service
 Known as BASE (Basically Available, Soft
state, Eventual consistency), as opposed to
ACID
#sqlsatParma

Availability
 Traditionally, thought of as the server/process
available five 9’s (99.999 %).
 However, for large node system, at almost
any point in time there’s a good chance that a
node is either down or there is a network
disruption among the nodes.
 Want a system that is resilient in the face of
network disruption
#sqlsatParma

Asynchronous Geo-replication with SQL
Azure
 Standard geo-replication (Standard and Premium)
 Optional non-readable secondary in paired region
 Secondary database is charged at discounted rate
 Failover enabled by Microsoft if disaster occurs
 Active geo-replication (Premium)
 Up to 4 readable secondaries
 Full control over secondary location and failover
 Supports load balancing, application upgrade and
relocation scenarios
 Can be combined with a non-readable secondary
DB
1
DB
1
DB
DB
1
DB 1
1
Geo-replication minimizes business
interruption from possible disasters
#sqlsatParma

Azure Caching
 Opportunity
 Lower accesses to SQL Azure and cost less
 Cache the page?
 Cache the content of a Page
#sqlsatParma

Handle your “repeatable data” with Redis
 Declarative approach
 Configure ASP.NET MVC to use Redis
 Session State or Output cache
 Look At Andrea Dottor presentazione about it
 Redis as an efficient in memory key/value store
 Find many .NET clients in nuget
 Example Stack Exchange implementation
 Implements a cache-aside pattern
 Query cache first
 If not found, Query datasource and storie info incache
 Cache invalidaton rules have to be handled
 Explicit
 Timeout
#sqlsatParma

Azure Search
 A search-as-a-service solution allowing
developers to incorporate great search
experiences into applications without
managing infrastructure or needing to
become search experts.
 It is based on Elastic Search
#sqlsatParma

New development challenges
 None of the following is true:
 “Network is reliable”
 “Latency is near to zero or irrelevant”
 “Bandwidth is unlimited”
 “Network is secure”
 “Topology doesn’t change”
 “Transport cost is irrelevant”
 “Network is homogeneous”
From Mauro Servienti presentation
http://www.slideshare.net/mauroservienti/in-the-brain-designing-distributed-scalable-and-reliable-systems
#sqlsatParma

Latency and Retries
 SQL Server
 Long latencies and timeouts
 Transient errors
 Retry strategy
#sqlsatParma

Q&A
 Questions?
 Email: marco.parenzan@live.it
 Slideshare:
http://www.slideshare.net/marcoparenzan
 GitHub:
https://www.gihtub.cob/marcoparenzan
 Twitter: http://twitter.com/marco_parenzan
#sqlsatParma

Thanks!
#sqlsatParma
#sqlsat355
#sqlsatParma

2014.11.22 Azure for Sql Server Developer - SQLSAT355 Parma

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie 2014.11.22 Azure for Sql Server Developer - SQLSAT355 Parma

Ähnlich wie 2014.11.22 Azure for Sql Server Developer - SQLSAT355 Parma (20)

Mehr von Marco Parenzan

Mehr von Marco Parenzan (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

2014.11.22 Azure for Sql Server Developer - SQLSAT355 Parma