How Ansible help to manage DBA Task and what are the best practices we followed are shared in this presentation.

1. Ansible is Our Wishbone
Python, Yaml, Jinja2
Presenter
Manosh Malai
Senior Devops & NoSQL Consultant
www.mydbops.com info@mydbops.com
1

2. Manosh Malai
Senior Devops and NoSQL Consultant @ Mydbops
mmalai@mydbops.com
@ManoshMalai
ME
2

3. About Mydbops
● Founded in 2015, HQ in Bangalore India with 150+ customer base across the globe.
● Mydbops is on Database Consulting with core specialization on MySQL and MongoDB Administration and
Support.
● We have expert team with 20+ certified DBA’s providing full time support and currently managing 300+
servers on cloud.
● Mydbops was created with a motto of developing a Devops model for Database administration offering
24*7 expert remote DBA support.
● We help organisations to architect and scale systems in MySQL/Mongo by implementing the advanced
technologies in industry which are completely open source.
● We are leading solution provider in the market for all sort of cloud based deployments and management.
3

4. Our Support Ecosystem
Percona Server
MySQL
MariaDB MariaDB Galera
Cluster
Galera Cluster for
MySQL
MongoDBProxySQL
Amazon RDS for
MySQL/Aurora
Percona XtraDB
Cluster
MariaDB MaxScale
MySQL Router
4

5. In This Presentation
How we spend our days as a DBA ?
What Next...
What Finally we got
How Ansible Helped us
5

6. Agenda
● How we spend our days as a DBA ?
● Findings our boring Tasks.
● Discovering our Wishbone.
● How we design our Wishbone ?
● What we got finally ?
● Where are we focusing now ?
6

7. Analysis of Boring Task
7

8. Our Days
How we really spend our days as a DBA
8

9. Finding Our Boring Task
9

10. Time = Money
10

11. 9.3% + 9.3% + 17.4% + 5.8% = 42%Repetitive Task
11

12. Details of 42% Task
● Install new version of DB server and its tools and any other tools that access the DB’s.
● Creating or Pushing optimized and standard config to server.
● Implements and enforce security for all of the DB server.
● Deploying Cluster, Load Balancer and Provisioning new node in a cluster(PXC, Maxscale and ProxySQL).
● Perform ongoing tuning of the database instances.
● Deploying Monitoring Infra and Monitoring Add-ons into a new or existing DB Infra(PMM,
NagiOS,MySQL-Utilities, Percona Toolkit, Sysstat and etc...).
12

13. 42% Repetitive Task and its Other Problem
Repeating same task on number of device is not only the problem, Other things too.
● Need to work with different Cloud platform(AWS, Google Cloud, Azure and etc).
● Need to work with different SAAS(Aurora, RDS).
● Need to work with different packaging system(apt, yum, npm, pip and etc).
● Each OS distribution have different configuration path for the same packages(sysstat, MySQL and etc).
● Susceptible to human error*
*80% of all downtime due to people/process error
13

14. Discovering our Wishbone
14

15. Ansible is a radically simple IT Automation Engine
Our Wishbone
15

16. BASIC FEATURE
SIMPLE
Human-readable YAML Language
No special coding skill required
Task Execute in Order
AGENTLESS
OpenSSH
Secure
Scalable
Efficient
POWERFUL
Deployment
Orchestration
Provisioning
16

17. VS
Asking Myself?
● Which method is most likely to end up in source control?
● Which method can be run multiple times safely with confidence?
● Which method can easily be run against multiple servers?
● Which method actually verifies (tests) your server for correctness?
● Which method can target certain servers easily (web, db, etc)?
● Which method supports easily templating your configuration files?
● Which method will grow to easily support your whole stack?
17

18. Ansible Architecture
18

19. Ansible Core Concept
● Inventories
● Ad-hoc Commands
● Playbooks
● Playbook >>Tasks(Handlers, Template, Variable)
● Playbook >>Roles
19

20. Inventories
● Ansible is used for managing multiple servers in the Infrastructure. The collection of hosts is known as
“Ansible Inventory”.
● Groups and Hosts are stored in inventory file
● An inventory file is expressed in a simple INI format
● Default location: /etc/ansible/hosts
● ansible or ansible-playbook -i <custom inventories path>
20

21. Inventories
21

22. Example Inventories
[all:vars]
ansible_user=vagrant
ansible_ssh_pass=vagrant
ansible_become=yes
[common]
v11 ansible_host=192.168.33.11
v12 ansible_host=192.168.33.12
[Common:vars]
percona_repo_url:
"https://www.percona.com/redir/downloads/percona-release/redhat/percona-release-0.1-4.noarch.rpm"
percona_deb_url: "https://repo.percona.com/apt/percona-release_0.1-4.{{ ansible_distribution_release
}}_all.deb"
All Groups: Variables
Group Hosts
Hosts
Groups Vars
22

23. Ad-hoc
● We don't have to create an elaborate set of tasks just to perform simple operations on your nodes.
● Ad-hoc commands allow you to execute simple tasks at the command line against one or all of your hosts.
Syntax:
ansible <host-pattern> options
23

24. Example Ad-hoc
ansible -i /etc/ansible/hosts common -m ping -f 10
ansible -i /etc/ansible/hosts common -m yum -a “name=vim state=present” -f 10
ansible all -m command -a "docker info" -u mydbops --become-user root --ask-pass --ask-become-pass
Module
Module Argument
User Becoming Sudo
Asking user password
Asking SUDO password
24

25. Playbook
● Playbooks we specify what and how our server will be configured and provisioned.
● Consist of variables, task, handlers, files, templates and roles.
● Are expressed in YAML format.
- name: Oracle Mysql Server Installation-5.7.
hosts: mysql
vars:
mysql_version: 5.7
mysql_version_repo: mysql{{ mysql_version|string |replace(".", "")}}-community
roles:
- common
- linux-tuning
- mysql
25

26. Task(Handlers, Template and Variable)
Task:
Tasks are nothing but modules called on hosts. That run on remote machines defined in inventory file.
- name: Installing Percona Server For RedHat Family.
yum:
name={{ item }}
state=present
with_items: "{{ mysql_percona_packages }}"
when: ansible_os_family == "RedHat" and 'percona' in group_names
- name: Installing Percona Server For Debian Family.
apt:
name={{ item }}
state=present"
with_items: "{{ mysql_percona_packages }}"
environment:
DEBIAN_FRONTEND: noninteractive
when: ansible_os_family == "Debian" and 'percona' in group_names
26

27. Handlers
Handlers:
● They will only run if a ‘notify’ directive is declared.
● It is useful in scenarios where a configuration file is changed and time it requires a service to be restarted.
● Handler will make sure that the service is restarted.
- name: restart mysql
service:
name={{ mysql_daemon }}
state=started
enabled={{ mysql_enabled_on_startup }}
register: mysql_service_configuration
- name: Copy my.cnf global MySQL configuration.
template:
src: mysql_conf.j2
dest: "{{ mysql_config_file }}"
owner: root
group: root
force: "{{ overwrite_global_mycnf }}"
mode: 0644
notify: restart mysql
27

28. Template
Template:
Templates is a file which contains all your configuration parameters, but the dynamic values are given as
variables.
[mysqld]
port = {{ mysql_port }}
bind-address = {{ mysql_bind_address }}
datadir = {{ mysql_datadir }}
socket = {{ mysql_socket }}
pid-file = {{ mysql_pid_file }}
{% if mysql_skip_name_resolve %}
skip-name-resolve
{% endif %}
{% if mysql_sql_mode %}
sql_mode = {{ mysql_sql_mode }}
{% endif %}
28

29. CONT...
# Memory settings.
key_buffer_size = {{ mysql_key_buffer_size }}
table_open_cache = {{ mysql_table_open_cache }}
sort_buffer_size = {{ mysql_sort_buffer_size }}
read_buffer_size = {{ mysql_read_buffer_size }}
thread_cache_size = {{ mysql_thread_cache_size }}
max_connections = {{ mysql_max_connections }}
tmp_table_size = {{ mysql_tmp_table_size }}
# InnoDB settings.
{% if mysql_supports_innodb_large_prefix %}
innodb_large_prefix = {{ mysql_innodb_large_prefix }}
innodb_file_format = {{ mysql_innodb_file_format }}
{% endif %}
innodb_file_per_table = {{ mysql_innodb_file_per_table }}
innodb_buffer_pool_size = {{ mysql_innodb_buffer_pool_size }}
innodb_log_file_size = {{ mysql_innodb_log_file_size }}
innodb_log_buffer_size = {{ mysql_innodb_log_buffer_size }}
29

30. Variable
Variable:
Variables as a way to deal with things that differ between one system and the next.
# InnoDB settings.
mysql_innodb_file_per_table: "1"
# Set .._buffer_pool_size up to 70% of RAM but beware of setting too high.
mysql_innodb_buffer_pool_size: "{{ (ansible_memtotal_mb * 0.7) | int }}M"
mysql_innodb_log_file_size: "1G"
mysql_innodb_log_buffer_size: "10M"
mysql_innodb_flush_log_at_trx_commit: "1"
mysql_innodb_lock_wait_timeout: "50"
30

31. Variable Precedence
In 2.x, we have made the order of precedence more specific (Low -> High Priority)
1. role defaults
2. inventory INI or script group vars
3. inventory group_vars/all
4. playbook group_vars/all
5. inventory group_vars/*
6. playbook group_vars/*
7. inventory INI or script host vars
8. inventory host_vars/*
9. playbook host_vars/*
10. host facts
11. play vars
12. play vars_prompt
13. play vars_files
14. role vars (defined in role/vars/main.yml)
15. block vars (only for tasks in block)
16. task vars (only for the task)
17. role (and include_role) params
18. include params
19. include_vars
20. set_facts / registered vars
21. extra vars (always win precedence)
31

32. Roles
● Ansible roles are consists of many playbooks, which is similar to modules in puppet and cook books in chef.
We term the same in ansible as roles.
● Roles are a way to group multiple tasks together into one container to do the automation in very effective
manner with clean directory structures.
● Roles are set of tasks and additional files for a certain role which allow you to break up the configurations.
● It can be easily reuse the codes by anyone if the role is suitable to someone.
● It can be easily modify and will reduce the syntax errors.
32

33. Role Directory Structure
33

34. Directory Structure
tasks - contains the main list of tasks to be executed by the role.
handlers - contains handlers, which may be used by this role or even anywhere outside this role.
defaults - default variables for the role.
vars - other variables for the role. Vars has the higher priority than defaults.
files - contains files required to transfer or deployed to the target machines via this role.
templates - contains templates which can be deployed via this role.
meta - defines some data / information about this role (author, dependency, versions, examples, etc,.)
34

35. How we designed Our Wishbone
35

36. Execution Process
36

37. Best Practice we follow for Our Wishbone
Inventory:
● Define meaningful inventory hostname, it doesn't want to resolvable.
● Inventory hostname help to view the result of play very meaningful.
● Dynamic Inventory using SSH config.
● Group all role related host into Role related groups(So we could run certain playbook against certain group).
37

38. Task & Playbook
Task & Playbook
● Use Human readable name for the Tasks and Plays.
● Keep plays and playbooks focused. don't create monolithic playbooks.
● Try to stick with ansible module try to avoid command or shell.
● last but not the least command module is safer than shell module(Because command module dont evaluate |
symbol, stringing two commands together and expanding shell variables or file globs and etc..) .
38

39. Templates
Templates
● Jinja2 is robust, try to use particularly.
● Don't overplay it, In some ways it comes down to documentation, a mixing of languages (YAML, Python,
Jinja2), and variables.
● Jinja2 can be a pain, but ultimately a very powerful tool.
● Should be:
○ Variable substitution
○ Conditional
○ Simple control structure and iteration
○ Advance filters(eg: math, ip address and etc)
39

40. What we got finally
● As we saw above, We are in heterogeneous environment. Using our Wishbone we are managing all
Obstacles in our way.
● Single Click Infra Provisioning and Managing.
● Help to keep all environment config in same way.
● Ensure no change unless things change.
● Improve Person/Team Performance and liability
● Stop everything do by hand.
● Get More sleep.
40

41. What’s Next...
● Consistency in Ansible Role development like PXC Cluster upgradation, Mysql 8.0 and etc.
● In DB Management if we config the parameter properly, we only need to focus on quieres tuning. So we are
currently developing Automate Slow query tuner.
● We are seriously planning to develop auto healing database management system.
41

42. 42

43. Happy Customers
43

44. Thank you !!!
Manosh Malai
mmalai@mydbops.com
44