Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
The Last Frontier- Virtualization, Hybrid Management and the Cloud
1. The Next FrontierVirtualization, Hybrid Management and the Cloud
Kellyn Pot’vin-Gorman | Technical Intelligence Manager for the Office of CTO, Delphix
2. Kellyn Pot’Vin-Gorman
Technical Intelligence Manager for the Office of CTO,
Delphix
• Multi-platform DBA, (Oracle, MSSQL, MySQL, Sybase,
PostgreSQL…..)
• Oracle ACE Director, (Alumni), Oak Table
• APEX Women in Technology Award, CTA
• STEM education with Raspberry Pi and Python
• Liaison for Denver SQL Server User Group
• Rocky Mountain Oracle Training Days Conference
Director and Board Director
• Author, blogger, (http://dbakevlar.com)
3. Virtualization is the creation of a virtual (rather than
actual) version of something, such as an operating
system, a server, a storage device, database, files or
network resources.
What is Virtualization?
4. Virtualization Adoption Trend
.
Source: Gartner Forecasts
Public cloud infrastructure as a Service, (IaaS) hardware and software Spending
from 2015 to 2026, by segment, (in billions of US dollars)
0
20
40
60
80
100
120
140
160
180
200
2017 2018 2019 2020 2021 2022 2023 2024 2025 2026
SaaS and PaaS
Public Cloud with IaaS
5.
6. Source: Right Scale State of the Cloud 2016
0%
10%
20%
30%
40%
50%
60%
Physical Machine Virtual Machine Private Cloud Public Cloud
2017
2018
2019
Migration Plans- 2017-19
8. An overall trend in companies that includes autonomic computing,
where the IT environment will be able to manage itself based on
perceived activity, and resource shared computing, in which computer
processing power is utilized and/or paid for only as needed. The usual
goal of virtualization is to centralize administrative tasks while
improving scalability and work loads.
The Future of Virtualization
10. Savings!!
• Cost
• Resource
• Storage
• Time
Well, that depends on….
• Price per Gb
• Complexity
• SLAs and the
biggest savings….
11. DBAs Can be Viewed as the Bottleneck
3/25/2017 11 |
• We are the gatekeepers
• Archaic tools
• Large amounts of data and complex migration steps
• Fear of loss of role
12. 1. Just copy data and applications into the cloud…
Straightforward approach
Inefficient, non-incremental for large environments
Open-source “bcp” uses encryption, compresses, and
multi-threads
Archaic processes recommended by vendors
2. Start with backups to IaaS storage, then populate re-hosted
applications by restoring from those backups
Cloud backups are easy, known technology
How Are Companies Migrating to
the Cloud?
13. Cost Estimates for Azure
https://azure.microsoft.com/en-us/pricing/details/storage/blobs/
Storage Capacity LRS ZRS
First 1 TB / Month $0.024 per GB $0.03 per GB
Next 49 TB (1 to 50 TB) / Month $0.0236 per GB $0.0295 per GB
Next 450 TB (50 to 500 TB) / Month $0.0232 per GB $0.029 per GB
Next 500 TB (500 to 1,000 TB) / Month $0.0228 per GB $0.0285 per GB
16. Migration Complete…Not so Fast…
• What if you only want dev and test in the cloud.
• Data is migrated, but this doesn’t count for ongoing data loads,
application connectivity across the network.
• To refresh will take considerable time to perform with traditional
tools or cloning methods.
• Rarely a consideration for the difference in cost structure for
processing large amounts of data from on-premise to the cloud.
19. https://docs.microsoft.com/en-us/azure/sql-database/sql-database-cloud-migrate
• Database must be MSSQL 2005 or higher, (easy)
• Ensure that the database is compatible with Azure SQL DB, (correct any
incompatible functions, etc.)
• Must have identified all performance issues that will be impacted
beforehand.
• Ensure there is as little physical distance between the cloud data center
and bacpac files to be used for migration.
• Disable management jobs that will hinder migration processing.
• Drop any objects or historical data that can impact migration time and
can be performed post migration.
21. From on-premises to the Cloud, then
restore- Assumption
Database server
Cloud Storage
Database server
22. Much Improved Option-
1. Data virtualization
Easy, secure, revolutionary
Optimized for faster agile development and testing
Optimized for new costing structures in IaaS
With a software appliance, no concerns about hardware or
software outside of virtualization lock-in.
Migrating from on-prem to IaaS with
Virtualization
23. Storage costs
Thin-provisioning storage avoidance
Data transfer costs
Far less data transferred during provisioning/refresh operations
• As opposed to the volume of data transferred using traditional
cloning techniques
Network-attached storage data transfers offset somewhat
• Most non-production usage modifies or accesses only a tiny
proportion of the total database during lifetime
28. Data Virtualization: How is it
possible?
Delphix Virtualization Engine
Virtual Database
Any Storage
29. Data Virtualization: on-premise
Environment
Source DB server
SQL Server 2008-2016
1 TB
Delphix Virtualization Engine
2 TB storage
Target DB server
SQL Server 1008-2016
No database storage
SCCI/SSL
Backups
via SMB
30. Data virtualization: from on-premises
into the Cloud
Source DB server
SQL Server 2008-2016
2TB storage
Delphix Virtualization Engine
2 TB storage
Target DB server
SQL Server 2008-2016
No database storage
31. Optimized for the cloud in the first place…not after!
Different cost structures
Much smaller storage footprint, much less data-transfer
How Does Data Virtualization
Enhance this?
32. Traditional copy data management techniques
Developed without concerns about infrastructure chargeback
This corresponds to higher cost.
IaaS vendors monitor storage and data transfers
Help meet SLAs, garner profits
It’s not just the data that exists in the end, so transformations
can equal big money for cloud vendors.
Different cost structures
33. Know Thy Enemy…
• Tune SQL and Apps to perform efficiently as possible- natural
life of database is growth, (in processes, resources, etc.) before
migrating.
• The less network latency, the better- network tracing to
eliminate database blame is important.
• Many of the same tools and data provides value- DMVs provide
data internally to SQL Server.
• Look at management tools such as Cloudmonix, (formerly
AzureWatch) AppDynamics, Dynatrace, Zabbix or Logic
Monitor.
34. For Non-Production Systems…
Change the way you’ve always performed tasks.
Performing common tasks the same way as previously
might end up costing more.
Secure Data
All IaaS alternatives promote encryption for data in-
flight and for data at-rest, but encryption may not be the
right answer…
Secondary Considerations
35. All IaaS solutions provide encryption in-flight and encryption at-rest
But encryption doesn’t protect data as much as it needs to be .
Europe already requires data masking, not just data encryption for
any confindential data, (GDPR):
http://ec.europa.eu/justice/data-protection/article-
29/documentation/opinion-
recommendation/files/2014/wp216_en.pdf
Confidential data
36. Encryption is reversible data obfuscation, which is very different
from masking data.
• Data masking is non-reversible.
It solves the issue at the data level.
Is authentication and authorization in non-production in compliance
with security goals?
All organizations will soon need to review if critical data in non-
production environments be accessible to developers, testers
and users.
Confidential data
37. Masking personally-identifiable, (PII, HIPPA, PCI, etc.)
information renders it useless from a security standpoint
Resolves both the technical and personal responsibility issue.
The data can be masked before it moves to non-production,
removing unnecessary risk.
Why Masking is Part of the Answer
40. Data virtualization: transformation by
masking
SQL Server
Validated Sync
Environment
Delphix Masking Engine
Delphix Virtualization Engine
2TB storage
SQL Server Target
No database storage
41. Data virtualization: Masking and then to
the Cloud
Delphix Virtualization Engine
2TB storage
Delphix Masking Engine
Delphix Virtualization Engine
2 TB storage
SQL Server Target
Storage Only for
Masked Data
SQL Server
Validated Sync
Environment
SQL Server Target
Storage Only for
Masked Data
42. Replication ensures that ongoing data refreshes aren’t
costly and are simple
For data protection, for scalability, and for migration
Virtualized data is already compressed at-rest, further
compressed and encrypted in-flight, incrementally
replicated to another engine
Selective Replications to VDBs
43. Once data has been virtualized on-premises…
It can be used for simplifying patching and upgrading on-
prem or in preparation for transforming to Azure SQL
Database
Securely mask data, which is essential in today’s cloud world
Then...
securely virtualized data can be safely replicated to IaaS
Data Virtualization + Upgrades = $$
44. On-Prem, Masked, then Replicated to
IaaS
Delphix Masking Engine
Delphix Virtualization Engine
4 TB storage
Delphix Virtualization Engine
4 TB storage
SQL Server
Validated Sync
Environment
SQL Server Target
Storage Only for Masked
Data
SQL Server Target
Storage Only for Masked
Data
SQL Server Target
Storage Only for Masked
Data
45. • Watch for Chargebacks
Data virtualization minimizes storage and data transfer and
transformation costs.
• Confidential Data
Data masking integrated with data virtualization truly
secures PCI, PII, HIPPA and other confidential data.
• Optimize and tune BEFORE and AFTER migrating to the cloud.
Summary: Solutions for Cloud
Migrations using Virtualization
46. Want to try it out, download the Delphix AWS Trial- Available for Oracle, MSSQL
Soon! https://www.delphix.com/products/free-trial-request
Twittter: @DBAKevlar
Linked in: http://linkedin.com/in/kellynpotvin
Blog: http://dbakevlar.com
Hinweis der Redaktion
We all are quite familiar with Virtualization in the form of hosts or servers, but there are a number of areas we hear less about- virtualized networks and an area I specialize in- virtualized databases, apps and flatfiles.
They may have a similar term identified with them, but the technology behind each offering and each vendor can be slightly to very different.
Microsoft ( MSFT ) announced its earnings for Q1 FY17 on October 20th. (Fiscal years end with June.) The company posted a 3% year-over-year growth (5% in constant currency) in revenues to $20.453 billion. In our pre-earnings note , we noted that cloud services would boost revenues across productivity and business processes and intelligent cloud divisions. Commercial cloud annualized revenue run rate exceeded $13 billion, and the company is on course to achieve $20 billion annual run rate by fiscal year 2018. As a result, the stock was trading at all-time high on Thursday (after market hours)
Read more: http://www.nasdaq.com/article/microsoft-earnings-cloud-adoption-takes-center-stage-as-revenue-improve-slightly-cm697401#ixzz4afv45v4t
OK, who’s heard these words: Disruptions, catalyst, epic change, visionary, paradigm shift, machine learning, monetization and pivot
Hybrid CLOUD and CLOUD
Over 60% of companies stated they are on 2-6 clouds, between databases, apps and files.
Your clouds will become very much like your smart phone provider.
Amazon and Azure own this race, but a price war could shake up the future of which cloud our data resides.
Almost 70% of the market is owned by Amazon and Microsoft
During Amazon S3 outage, it wasn’t Azure or Oracle that benefited, but Google, which increased its business over 12% in just two days.
Yeah, those mundane buzzwords of what you did 5-10 years ago is now going away…what you did just 2 years ago is going to transition.
You may be a “cloud administrator” or a database architect without a single database platform and on multiple cloud platforms.
More likely multi-platform and multi-cloud.
We’re continually pushed that it once was about savings, but no longer. Where RDS was there to save the day, we now know that price per gb, it often comes out to the same or more than it once was on-prem.
When license costs are increased and no limit on usage, budgets can be stretched.
Do any of you see the problem with the high level project steps?
We commonly leave optimizing the environment until after we’ve migrated to the cloud.
Standard backup and recovery methods
Replication
Cloning
Continual feed to keep up to date or refresh on regular basis, via archaic tools- bcp, log shipping or paid replication tools.
And if you choose wrong or use more resources than expected, you can experience severe performance issues.
This is for Azure migrations- the requirements
Create this report- what can be migrated and what can’t?
90% of environments won’t fulfill the requirements and won’t migrate with the cloud migration asst.
Once final tests are done- you are testing.
Perform final migration, final sync to prod and downtime to switch from on-prem to cloud.
80-90% storage savings from traditional migration methods.
Data In flight can be significant cost for many cloud vendors
Network is the new bottleneck. You can avoid that with less copies- one golden copy- we call it the “validated sync environment”
By going to a single source, loading to a single source and maintaining a single source, a smaller footprint is attained.
Cost savings in the way of less storage required results in even bigger savings.
Different cloud manufacturers have different pricing structures- verify what you are being charged for and make sure those costs aren’t in contrast with your environment.
Many avoid RDS on Amazon- we don’t support it. For our Oracle customers, too limited.
Before you start, tune SQL instaead of after.
Use network tools like Nagios network analyzer or. Solarwinds Network Performance Monitor, (NPM)
Your performance data can assist you in identifying huge IO, CPU and remote resource work that should be minimized beforehand.
Data in flight can cost you and data processing that was normal on-prem, may need to be redesigned post cloud migration.
Inspect pricing small print carefully and know you’re final decision on *what* choice in cloud and type of service will determine.
Encryption is important for production.
SQL 2016 dynamic data masking isn’t production ready- three steps and I had ‘un-masked
data!
Or does it shift the problem toward authentication and authorization?
DMVs are your friend. Embrace them and know each databases pain points and how they will effect chargebacks once the database, application and wprkload is migrated to the cloud.
What may not have been an issue while on-prem, may become an expensive cost if overlooked post-project.