Today, we have proven techniques for many DevOps practices. For provisioning a new environment, we apply file based environment definitions to dynamic infrastructure: Helm for Kubernetes, Heat for OpenStack or Terraform for other clouds. For automating an application deployment, we can turn to basic pipelines like Jenkins provides or release automation tools like IBM UrbanCode deploy.
However, one area is a constant sticking point. Data. A provisioned test lab is useless without test data. Automated deployment tied to a manual schema update is only as fast as good as the DBA's working by hand. Meanwhile, data is different. There can be a lot of it. It's often sensitive. Changes to schema are generally incremental. Naively applying something like Terraform to the data problem is a recipe for trouble.
There is good news. Tools that specialize in managing databases are easy to integrate into your DevOps toolchain. Join Actifio's Jay Livens, DBmaestro's Chris Lucca and IBM's Eric Minick for a lively conversation examining how to overcome this stumbling block.
Don't Fumble the Data! Integrate Database Automation into your DevOps Toolchain
1. Don't Fumble the Data!
Integrate Database Automation into
your DevOps Toolchain
2. 2
Who are we?
Eric Minick
IBM
Product Management Lead
Continuous Delivery
Yaniv Yahuda
DBmaestro
Co-Founder and CTO
Jay Livens
Actifio
Senior Director
Product Marketing
3. 3
High Performing IT organizations are better:
46x
More deploys
440x
Faster lead time
96x
Faster MTTR
2017 State of DevOps Report – Puppet/DORA
5. We know how to do CD
Develop SCM Build Package
Repo
Test Perf Prod
5
6. Systems of Systems. Pipelines of Pipelines
6
• Test the System
• Deploy the System
• Release the System
Development SCM Build
Package
Repo
Test Stage Production
Development SCM Build
Package
Repo
Test Stage Production
Development SCM Build
Package
Repo
Test Stage Production
Development SCM Build
Package
Repo
Test Stage Production
Development SCM Build
Package
Repo
Test Stage Production
7. We know how to do CD
7
Most of the time is spent testing.
Develop SCM Build Package
Repo
Test Perf Prod
8. Data appears to be a different beast
8
Test data comes from Prod
Prod is big and full of secrets
There’s no build
Changes are incremental
Develop SCM Build Package
Repo
Test Perf Prod
10. 10
UrbanCode Deploy
Key Capabilities
• Continuous Delivery for multi-tier or
multi-service applications
• Pipeline views of which changes are where
• Push-button deployment of applications
into environments from Dev to Test to Prod
• 200 Integrations (plugins) with middleware
and other tools
• Easy environment creation (provisioning)
with the included Blueprint designer.
• Automation to go fast
• Automation to be correct
• Governance to be allowed
to do it at all
Easy Automation
graphical design
not scripts
10
15. Headline
15
“Move fast and break
things?”
Sure, but don’t break my
database!
Introducing DBmaestro Database DevOps
Platform.
16. 16
Force check in
& out Track
changes
Prevent:
• Code overrides
• Team conflicts
• Partial updates
• Undocumented changes
Integrated
version control
17. 17
DevOps Best
practices:
• Build once, deploy many
• Deploy the same way to all
environments
• Immutable packages
• Automate DB releases
• Risk management
• Traceability
Release pipelines:
fast, safe,
repeatable,
scalable
18. 18
• Identify and alert about
configuration drifts –
before you break things
• Catch invalid code before it
is executed
in your DB (syntax/SQL
error etc.)
Auto drift
detection &
code validation
19. 19
Determine what
types of activities
can be deployed,
where and when.
What’s not OK?
• Prevent drops
• Security risks
• Non standard actions
• And more…
Policy
management
20. 20
Control who can do
what and where
• Project segmentation
• Separation of duties
• Roles
• Rights of processing
− DEV…
− QA
− Pre-PROD?
− PROD!?!
Roles and
permissions
21. Keep a complete
history of all changes
made
Who?
What?
When?
Where?
HIPAA, SOX…
Audit trails for
changes
21
28. 28
Actifio: How It Works
Virtual Data Pipeline™
MANAGE
Lifecycle and Mobility as Single
System of Record across
Hybrid Cloud
USE
Unlimited virtual copies for
instant access and protection
UAT
Prototype Unit Test
Build Integration QA
COPY DATA
MANAGEMENT
COPYRIGHT 2017 ACTIFIO
CAPTURE
Data at block level, in native format,
Incremental, according
to your SLA
Cloud, Container, VM, Physical
hs
Multi-cloud, multi-directional
Private
Cloud
29. Feature Summary
29
Provision Multi-TB, Current Datasets
Instantly without Sub-setting
Reduces Wait Time, Improves Productivity,
Accelerates Release Cycles
Self-service, API-driven
Automated Data Masking,
Role-Based Access Control
CAPABILITIES WHY IT MATTERSProvision 1 to 40 copies instantly
10 TB
0 TB 0 TB 0 TBANY PRIVATE CLOUD:
(VMware, Hyper-V)
ANY PUBLIC CLOUD:
(AWS, Azure, Google, IBM Cloud)
Functionality
Testing
10 TB
0 TB
10 TB
10 TB
0 TB
0 TB0 TB
Low Tier
Performance
Testing
High Tier
CI CD Pipeline
Provision
Compute
Deploy
Build
0 TB
0 TB
0 TB
Provision
Virtual DBs
10 TB
MASKED VIRTUAL DATABASES
10 TB
0 TB 0 TB 0 TB
Sensitive Data
Masked
Automation with DevOps Tools: UrbanCode, DB
Maestro and others
Control & Security
Use Different Storage Tiers
for Different Test Profiles
Reduces Costs
Improves Quality of Testing
Any Cloud Flexibility & Freedom
30. How the components fit to deliver
dbMaestro
IBM UrbanCode
Deploy
IBM Rational
Test Workbench
Integrated with
Databases
Internal
Messages
Third-party
Services
virtual components
Test Environments
Dynamic Infrastructure
Apply database updates,
especially schema
Continuously test in
production-like env.
Quickly deliver test dataActifio
UC Deploy Plugins
Provide test data masking and
security
IBM Cloud Private
IBM InfoSphere
Optim Test Data Mgmt
Dev QA
3030