DevOps and the DBA

DevOps and the Agile DBA
Efficiency Out of Chaos
Kellyn Pot’Vin-Gorman | Technical Intelligence Manager for the Office of CTO

2© 2017 Delphix Corporation
Kellyn Pot’Vin-Gorman
Technical Intelligence Manager, Delphix
• Multi-platform DBA, (Oracle, MSSQL, MySQL,
Sybase, Postgres…..)
• Oracle ACE Director, (Alumni)
• Oak Table Network
• APEX Women in Technology Award, CTA 2014
• STEM education with Raspberry Pi and Python
• Board of Director, RMOUG
• Training Days Conference Director
• Author, blogger, (http://dbakevlar.com)

This session is an introduction to DevOps for Database
Administrators. The focus is not only to discuss the concepts,
tools and processes, but cultural changes for DBAs to
become more agile in a DevOps world.

© 2017 Delphix Corporation 4
Agenda
Story Time1
What is DevOps2
Agility- Speed is Everything3
Tools of the Trade4
Summary5

Story Time
• Large company
• 4TB transactional database, (small by today’s standards)
• Financial data, aggregated to other financial systems.
• Agile development released, (most often) directly to production,
without formal processes, (DevOps) in place.
• Archaic development, (1/3 of size of production), rarely, if ever
testing outside of production queries.

After 4-6 Weeks of Research…
• Over 40% data corruption to main transactional system that feeds
into datamarts.
• Even higher percentage of corruption impact in marts due to poor
agile development practices and percentage of highly volatile
source data.
I now have to reveal my findings to senior management as the
new Lead DBA…

“Accuracy is over-rated…”

Last Invited to the Party…
• DevOps is for Development and Operations- where do we, DBAs
fit?
• When we aren’t included, groups attempt to work around us and
the business pays the cost.
• Feeling alienated, some are content to promote this scenario.
• Results in our education on DevOps lagging behind other groups
and high cost to efficiency and productivity.

https://en.wikipedia.org/wiki/DevOps
DevOps
DevOps is a short for DEVelopment and IT OPerationS. The term is used to
refer to a set of practices that emphasize the collaboration and communication of
both software developers and information technology (IT) professionals while
automating the process of software delivery and infrastructure changes. It aims
at establishing a culture and environment where building, testing, and releasing
software can happen rapidly, frequently, and more reliably

History of DevOps
• First called “Agile Infrastructure” in 2008
• 2009, the first time DevOps was coined in Belgium
• Has taken over most of the fellow terms of WinOps and in some cases,
Agile.
• Introduced at relational database events in 2010, but rarely was given
much credit until the last five years in the DBA community.
https://www.versionone.com/devops-101/what-is-devops/

Common Terms, (Alphabetically)
• Agile
• Build Automation
• Configuration Management
• Continuous Delivery
• Functional Testing/Unit Testing
• Packages/Containers
• Rapid Deploy
• Release Coordination/Orchestration
• Test Driven Development
• Virtualization
https://xebialabs.com/glossary/

DevOps Relationship to Agile
• Agile’s success pushed the need for
DevOps
• Agile and DevOps have similarities,
Agile focuses more on culture and
DevOps focuses more on
organizational changes to create
success.
• DevOps focuses on the success of
releases as the scoring method of
success of implementation.
We’re VERY
close…

Build Automation
Tools or framework that allow source code to be automatically compiled into
releasable binaries. Usually includes code-level unit testing to ensure
individual pieces of code behave as expected.

Configuration Management
Concept &
Development
Testing &
Verification
Operations &
Support
Deployment to
Production

Continuous Delivery
Continuous Delivery is the ability to get changes of all types—
including new features, configuration changes, bug fixes and
experiments—into production, or into the hands of users, safely and
quickly in a sustainable way.
https://continuousdelivery.com/

Functional Testing/Unit Testing
Testing
Requirement
Analysis
Test Planning
Traceability
Matrix
Test Execution
Defect
Management
Coverage
Analysis
Test Case
Design

Rapid Deploy
Software Content
Enablement Service Need or
Focus
Area
All Tiers of Specific
Environment

Release Coordination/Orchestration
Release Orchestration focuses on the use of tools which manage software
releases from the development stage through the deployment to production.

Test Driven Development
Red
GreenRefactor
TDD
Create Test
that Fails
Update the Test
to Succeed
Eliminate
Redundancy

Timeline in a Project
Waterfall- 5 Releases
January February March April
Agile with Virtualization- 12 Releases
January February March April
Database Refresh
Development Production Release
Testing Cycle
Refresh and
Retest
Skip Test and
Release

Virtualization
A technical approach in which users and applications do not use physical
machines, but simulated systems running on actual, “real” hardware.
Virtualization can be used to eliminate resource usage and enable savings
for databases, network, file and application management, along with server
infrastructure.

▶▶▶
Virtualize and Deployed▶ ▶ ▶
600GB
Many Types of Virtualization
Storage Pool for Delphix
QA
DEV PATCH TEST
PRODUCTION
Database/App Tier
1
TB
1 TB
Read From Production
Each Virtual Database takes up around 5-10Gb upon creation, (dependent upon parameters)
TEST
Read AND Write

Why Database Virtualization is Different
• DBA is “only as good as their last backup”.
• Many database tools take considerable time to recover.
• DevOps is often about automation- automate an “undo” for
development and testing that includes data rewind.
• Include a self-service tool that will allow for rewind without DBA
intervention.

Packages/Containers
Containers offer the ability to isolate application code and/or the whole
infrastructure stack into a package able entity to ease deployment, even
from the same kernel. This is a powerful tool for DevOps to ease
deployment for complex tiers, applications and multiple data stores.

“Containerize” for Ease of Delivery
Development
Delphix Virtualization Engine
Same Storage Size as Production
NFS
iSCSI
Testing
NFS
iSCSI
Reporting
NFS
iSCSI
Create “Container”
Rsync (UNIX/Linux)
Robocopy (Windows)
Deployed using Jenkins

The DevOps Toolchain
 Code
 Build
 Test
 Package
 Release
 Configure
 Monitor
 Repeat
Areas that will benefit from
DBA involvement
 Code
 Build
 Test
 Package
 Release
 Configure
 Monitor
 Repeat
Areas where involvement
will benefit the DBAs
career

A Team Effort
Between the following groups:
• Development
• Management
• Business
• Database Administration
• Testing
• Operations
• Infrastructure

Continuous Delivery
Involves all these groups and…
• Orchestrates processes from different tiers.
• Allows for deployments more often and with more
success.
• Removes latency and adds automation.
• Improves communication between all groups
involved.

Goals Are All the Same…
• Shorter Delivery Cycles
• Higher Success Rate on Releases
• Shorten Time to Address Issues
• Continued Improvement in Communication
• Increased Revenue

Collaboration Between All Stakeholders
• In Agile shops, stakeholders meet on a regular basis.
• Those involved in the product development, testing, deployment and
support may meet as often as once per day or more.
Can you see the value of having the DBA in the room when the
requirements, infrastructure, code and other decisions are being made?
Yet, we still ask, “What’s in it for me?”

Cultural Challenges for DBAs
• Natural Gatekeepers
• Seek Stability
• Guardians of company data
• Trust archaic processes that ensure
success, may not ensure speed of
completion.
• Viewed as roadblocks.

Cultural Challenges for Operations
• Viewed as security guards
• Removers of “waste”
• Answer to management and business
• Also want stability, (often have similar concerns as the DBA Team)

Development
• Seeking change
• Need to meet requirements
of the business
• Need more access to meet
demands and deadlines
• Desire the path of least
resistance

Business
• Have Needs
• Want them fulfilled
• Often don’t understand requirements or
resource constraints
• Pivotal to End User experience and
requirement gathering
• Often can be the least concerned about
security

Where Can the DBA offer the Most Value?
• Update skills to remove bottlenecks
• Enhance monitoring utilities that take infrastructure
and development into overall view
• Recognize a need to become more skilled in scripting
and deployment processes
• Become familiar with DevOps toolchain
DBA
DevOpsSkills

Collaboration Practices/Tools
• Methods- Scrum, KanBan, Extreme Programming, Continuous Delivery
• Hybrid approaches combine, such as Scrumban and KanPlan
• Collaboration- Jira, SlackStorm, Pivotal Tracker, HipChat, Rally, (now Agile
Central)
• Deployment- RapidDeploy, ElasticBox, JuJu, Otto
• Automation- Jenkins, Chef, Ansible
• Build Tools- Maven, Gradle, Make, Packer, MSBuild

Agile Methodologies
Scrum- A lightweight PM framework. Benefits include a flexible set of
management and project protocols and strong collaboration focus.
Lean/Kanban- Strong focus on team and deliverables. Efficiency and
ability to see the “whole”.
Extreme Programming, (XP)- Now the most popular, with continuous
delivery key.
Crystal- A lightweight agile framework. A removal of hurdles, including
bureaucracy, obstacles and other impediments to the development cycle.
Feature Driven Development, (FDD)- client and architecture centric
software development, focused on the feature deliverable.

Example of a Scrum Sprint
https://www.scrumalliance.org/community/articles/2014/april/devops-and-agile

Know Your Tools
Incredible variation of tools and knowing tool classifications doesn’t
mean that the tools are alike. Many groups are often comparing
one feature of a given tool.

The Periodic Table of DevOps Tools
https://xebialabs.com/periodic-table-of-devops-tools/

The Last Piece- Security!

Confidential data
Production
Non-
Production
Exposure

As 80% of data in a company are copies, then 80% of data won’t be subject to
security like a production environment. Securing this data is not just a priority,
but in many cases, subject to legal ramifications, (i.e. PCI/PII)
• Masking Requirements
• Masking shouldn’t be reversible
• The masked data should be representative of the original data type to ensure
performance is consistent.
• Referential Integrity should be maintained as part of the masking process.
Masking should be a simple, repeatable process with a user interface
that ensures it is simple.
Masking in the Picture

Do I Have to Mask Data? Nah….
Type of Data Year Passed Ruling
Data Masking in
the EU
2014 ARTICLE 29 DATA PROTECTION,
(GDPR)
HIPAA 1996 Health Insurance Portability and
Accountability Act
PCI 2016,
(Updated)
Payment Card Industry Standards
PII Personably Identifiable Information
SOX 2002 Sarbanes-Oxley Act

Confidential data
Production
Non-
Production
Exposure
Encryption
Masking
Solution

Source/Version Control
“A component of software configuration management, version control,
also known as revision control or source control, is the management of
changes to documents, computer programs, large web sites, and other
collections of information.”

Branching and Bookmarking
• The ability to mark each iteration of development with a
bookmark
• Simplify to lock and deliver while testing a consistent
image via a virtual database, (VDB)
• If a something goes wrong, the ability to “bookmark”,
(and subsequent snapshot) to deliver to development to
address.

DevOps and Source Control- The Delphix Way

From the DBA Perspective
DevOps include the following focus areas for Database
Administrators:
• Automating repeatable and complex processes
• Ensuring the databases and data stores aren’t the bottleneck to
the development cycle and critical data is protected.
• Locating heterogeneous tools/scripting languages that can
support multiple tiers of technology.
• Understanding that automation does not lessen your value.
• Embracing new skills to add more value to the business and to
your career.

How This All Comes Together…
• The DBA, with their knowledge, natural “gate keeper” mindset and area
of expertise, should be a pivotal member of the DevOps solution.
• There are a number of stakeholders and groups that make up DevOps
and DBAs need to find a strong balance between securing the
environment and not being viewed as a roadblock.
• Understand the terms, the process and methodologies of DevOps to be
more integrated into the new age of DevOps!
• Collaboration and culture change are always challenging, but the benefit
in the end is more productivity, product satisfaction and shorter
development cycles.

Kellyn Pot’Vin-Gorman
Technical Intelligence Manager
kellyn@delphix.com
http://dbakevlar.com

DevOps and the DBA

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie DevOps and the DBA

Ähnlich wie DevOps and the DBA (20)

Mehr von Kellyn Pot'Vin-Gorman

Mehr von Kellyn Pot'Vin-Gorman (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

DevOps and the DBA

Hinweis der Redaktion