How to avoid boring workAutomation for DBAs. How to use OEM 12c and Ansible in day by day DBA worj

1. © 2015 Delphix. All Rights Reserved. Private & Confidential.!
How to avoid boring work!
Automation for DBAs!
Irish Oracle User Group!
!
Marcin Przepiorowski!
!
!
March 2015!

2. Delphix
at
a
Glance
Enterprise
so2ware
(private,
hybrid,
public
cloud):
radically
improves
data
delivery
Data
as
a
Service
Pla6orm
for
applica<ons,
databases,
data
warehouses,
and
files
• Con<nuously
collects
data,
stores
in
<me
machine,
and
shares
data
blocks
• Virtualizes
data:
99%
faster
delivery
(months
to
minutes),
90%
smaller
size
• Improves
return
on
applica<on
project
investments
by
50%
on
average
Founded
in
2008,
HQ
in
Menlo
Park,
California,
300
employees
Select
Customers
n >100%
renewal
rate
n Subscrip<on
so2ware
n >$100M
life<me
sales
Financial
Highlights
Investors

3. Jedidiah Yueh, President & CEO
• Founder of Avamar (acquired by EMC), !
• >$2B sales, 10,000+ customers!
Rick Caccia, VP Marketing
• VP Product Marketing, HP/ArcSight!
• Senior Director Product Management, Symantec!
Dan Graves, VP Product Management
• Symantec Cloud CTO ($1B product line)!
Mark Boullie, VP Sales Americas
• SVP WW Field Ops, Perfecto Mobile!
• ArcSight VP Americas, scaled to $200M!
Iain Chidgey, VP Sales EMEA
• ArcSight VP EMEA, scaled to $100M!
Chris Poulos, VP Sales APJ
• ArcSight, HP VP APJ , scaled to $100M!
Marco Aurelio
• VP Professional Services Oracle!
• VP PS SAP, launched customer success practice!
MANAGEMENT
Adam Leventhal, CTO
• Co-founder, inventor D-Trace!
• Wall Street Journal Gold Innovation Award!
Eric Schrock, VP Engineering
• Lead Architect Sun 7000 storage appliance!
• 5,000+ customers!
Vinay Srihari
• Oracle Senior Director of Engineering, Active Data Guard
(50,000+ Oracle Enterprise customers)!
Matthew Ahrens, Architect
• Co-founder, inventor ZFS, advanced file system with over $400M
sales ,10,000+ customers!
Mike Stewart
• Architect Oracle Flashback (50,000+ customers)!
Francisco Sanchez
• Architect Oracle RMAN (250,000+ customers)!
Kyle Hailey
• Principal architect Oracle Enterprise Manager performance
pages!
• Designer of Embarcadero’s DB Optimizer!
ENGINEERING

4. About
me
Oracle
consultant
since
2000
co-­‐developer
of
OraSASH
–
free
ASH/AWR
like
repository
Blogger

5. Agenda
What
is
a
boring
work
?
Why
we
should
avoid
it
?
How
to
avoid
boring
work
?

6. What
is
a
boring
work
?
Set
of
manual
task
done
daily
6

7. What
is
a
boring
work
?
Repor<ng
7

8. What
is
a
boring
work
?
Repea<ng
same
task
on
number
of
devices
8

9. How
to
avoid
a
boring
work
?
Set
of
manual
task
done
daily
9

10. How
to
avoid
a
boring
work
?
Automa<c
/
intelligent
repor<ng
10

11. How
to
avoid
a
boring
work
?
Repea<ng
same
task
on
number
of
devices
11

12. AUTOMATION
12

13. AutomaFon
Prevent
from
complex
manual
tasks
Help
to
keep
all
environment
configured
in
the
same
way
Improve
person
/
team
performance
and
liability
–
especially
ops
/
on-­‐call
person
13

14. On
call
DBA
working
during
night
14

15. Typing
long
instruc<on
15

16. AutomaFon
for
DBAs
–
tasks:
• Daily
ops
• Monitoring
• Changes
• Patching
• New
installa<on
• Cloning
16

17. Avoid
manual
work
(
when
possible
)
17

18. Examples:
Data
Guard
configuraFon
Use:
• RMAN
duplicate
from
ac<ve
(especially
in
12c)
• Data
Guard
Broker
18

19. Examples:
Space
management:
Use:
• ASM
/
big
files
• OMF
If
need
more
control:
• Jobs
to
add
files
• Triggers
to
check
files
names
19

20. AutomaFon
for
DBAs
-­‐
tools
• Oracle
Enterprise
Manager
12c
• Ansible
• Old
fashion
scripts
• Puppet
/
Chef
/
cfengine
/
etc
20

21. Oracle
Enterprise
Manager
12c
EMCLI
/
GUI
• Provisioning
• Patching
• Monitoring
/
aler<ng
• Jobs
• Daily
opera<on
• Changes
21

22. Oracle
Enterprise
Manager
12c
Provisioning
22

23. 23

24. 24

25. Oracle
Enterprise
Manager
12c
Patching
• Nice
and
easy
when
EM
connected
to
MOS
• Good
integra<on
with
EM
What
• Oracle
DB
• Middleware
(with
some
caveats)
25

26. 26

27. 27

28. 28

29. OEM
12c
Monitoring
/
alerFng:
• Monitor
on
system
level
not
target
level
• Avoid
OK
messages
• Avoid
spam
29

30. OEM
12c
Monitoring
/
alerFng:
Backup
alerts
• if
there
is
no
backup
status
at
cutover
<me
• when
is
no
successful
backup
No
alerts
–
Backup
OK
–
no
ac<on
needed
(so
no
need
to
send
mail
/
alert)
30

31. OEM
12c
Jobs:
• GUI
Disable
20
jobs
–
20
clicks
• EMCLI
emcli
login
-­‐username=sysman
emcli
suspend_job
–name=<pamern>
31

32. Ansible
• Open
Source
• Agent
less
–
using
SSH
• Fast
learning
curve
hmp://www.ansible.com/home
hmps://github.com/ansible/ansible
“Ansible
is
a
radically
simple
IT
automa<on
system.
It
handles
configura<on-­‐
management,
applica<on
deployment,
cloud
provisioning,
ad-­‐hoc
task-­‐execu<on,
and
mul<node
orchestra<on”
32

33. Ansible
• Wrimen
in
Python
• Minimal
requirements
on
target
• No
official
Oracle/Weblogic
module
• Search
GitHub
for
community
work
33

34. Ansible
ansible
one
command/script
for
set
of
hosts
ansible-­‐playbook
set
of
ac<ons
for
set
of
hosts
34

35. Ansible
• Add
SSH
public
key
into
authorized_keys
• Create
template
• Run
script
35

36. Add
SSH
public
key
into
authorized_keys
ansible
-k -i inventory/mac all
-m authorized_key
-a "user=oracle key="{{ lookup('file’,‘id_rsa.pub') }}"
”
---
- hosts: all
tasks:
- name: Add ssh key
authorized_key:
user=oracle
key="{{ lookup('file', ’/id_rsa.pub') }}"
36

37. Create
template
–
Jinja2
List of databases SID for {{ ansible_hostname }}
{% for item in databases %}
- {{ item }}
{% endfor %}
---
- hosts: all
tasks:
- name: generate scripts
template: src=template.j2 dest=/tmp/output mode=644
37

38. Run
script
ORACLE_SID=$1
export ORAENV_ASK=NO
. oraenv > /dev/null
sqlplus -s / as sysdba << EOF
set head off
select decode(cnt, 0, 'OK', 'FAILURE') from (
select count(*) cnt from dba_users where
account_status like '%LOCKED%' and username = 'SCOTT'
);
exit
EOF
38

39. Run
script
---
- hosts: all
tasks:
- name: check status
script: check_user.sh {{ item }}
with_items: databases
register: result
failed_when: "'FAILURE' in result.stdout"
39

40. Ansible
-­‐
Provisioning
Provisioning
based
on:
• Core
Ansible
modules
• Oracle
response
files
• Playbooks
• Roles
40

41. Ansible
-­‐
Provisioning
Example
RAC
Amack
12c
github.com/oravirt/ansible-­‐oracle
41

42. 42

43. Ansible
-­‐
Check
patch
---
- hosts: all
vars:
patch_number:
- 19121551
- 19121552
oracle_home: "/u01/app/oracle/product/11.2.0/dbhome_1"
tasks:
- name: Check
stat: path={{ oracle_home }}/inventory/oneoffs/{{ item }}
with_items: patch_number
register: patch_status
failed_when: not patch_status.stat.exists
ignore_errors: True
43

44. Ansible
-­‐
Patching
• Using
opatch
and
scp/wget
to
copy
files
• Can
be
run
from
any
client
with
Ansible
• Check
example
on
Frits
Hoogland
blog
fritshoogland.wordpress.com
44

45. Ansible
Real
world
examples
45

46. Ansible
-­‐
Patching
---
- hosts: all
vars:
- name: check u01 free disk space
- name: check tmp free disk space
- name: create directory for installation files
- name: copy opatch and psu
- name: install opatch in database home
- name: unzip psu patch
- name: patch conflict detection
- name: apply psu
- name: clean up install directory
46

47. Ansible
–
Weblogic
SSO
---
- hosts: formal
tasks:
- name: check out config
- name: test connection
- name: Copy krb.conf to etc
- name: Copy keytab
- name: copy krb5Login.conf.j2 template to WLS
- name: run kinit
- name: add SSO conf to setCustom
- name: configure AD in Weblogic
- name: configure Weblogic SSO provider
- name: reorder providers
47

48. Ansible
–
Host
prep
for
Weblogic
---
- hosts: all
tasks:
- name: check out config
- name: Copy oem.repo to etc
- name: Add oem ssh key
- name: delete public repo
- name: upgrade all packages
- name: disable existing repo if any
- name: install Java - copy rpm
- name: install java - yum
- name: generating hosts file
- name: add NFS software share
- name: add user
- name: add samba /shared
48

49. Cloning
/
Data
Provisioning
• Typically
most
complex
/
<me
consuming
task
• Some
system
are
too
big
for
Data
Pump
and
this
generally
slow
process
• Need
mul<
team
coopera<on
for
big
system
(storage
/
OS
/
DBA)
49

50. Cloning
/
Data
Provisioning
If
s<ll
using
Data
Pump
50
Automate
it
use
scripts
or
bemer
PL/SQL
API
and
develop
one
click
refresh
procedure

51. Cloning
/
Data
Provisioning
• Storage
snapshot
–
OS
/
Storage
/
DB
admin
• Oracle
SnapClone
–
Self
service
• Delphix
–
Self
service
51
But
this
is
topic
for
other
presenta<on

52. The
Goal
:
Theory
of
Constraints
Improvement
not
made
at
the
constraint
is
an
illusion
factory
floor
op<miza<on

53. DevOps
:
• Goals
Clarify
• Metrics
Define
• Constraints
Iden<fy
• Priori<es
Set
• Itera<ons
Fast
• Con<nuous
Integra<on
• Cloud
• Agile
• Kanban
• Kata
“IT
is
the
factory
floor
of
this
century”

54. Thank
you!
Marcin
Przepiorowski
marcin@delphix.com
oracleprof.blogspot.ie
@pioro