Docker Concepts for Oracle/MySQL DBAs and DevOps

Docker Concepts for Oracle/MySQL DBAs and
DevOps
Zohar Elkayam
CTO, Brillix
Zohar@Brillix.co.il
www.realdbamagic.com
Twitter: @realmgic

2
• Zohar Elkayam, CTO at Brillix
• Programmer, DBA, team leader, database trainer, public speaker, and
a senior consultant for over 19 years
• Oracle ACE Since 2014
• ilOUG Board Member – Israel Oracle User Group
• Blogger – www.realdbamagic.com and www.ilDBA.co.il
Who am I?

3
About Brillix
• We offer complete, integrated end-to-end solutions based on best-of-
breed innovations in database, security and big data technologies
• We provide complete end-to-end 24x7 expert remote database
services
• We offer professional customized on-site trainings, delivered by our
top-notch world recognized instructors

5
• Founded in 1999, over 7000 members at its peak
• Divided into Technologies (Databases) and Applications groups
• Free Bi-monthly meetups for DBAs and Developers at Oracle Israel offices
• Yearly conference with guests speakers from all over the world:
• January 22-23, 2018
• Over 30 Speakers from all over the world: Ask TOM team, Oracle product mangers
and Oracle ACEs from 9 different countries!
• You can join our community: meetup group, facebook group or follow us
on twitter (@ilOUG_org)!
• Our website: www.ilOUG.org
ilOUG – Israel Oracle User Group

6
Agenda
• Docker overview – why do we even need containers?
• Installing Docker and getting started
• Images and Containers
• Docker Networks
• Docker Storage and Volumes
• Oracle and Docker
• Docker tools, GUI and Swarm

7
Our Goal Today
• Learn what Docker is and how to use it
• See how Docker containers might be helpful with Oracle
• We will not expert everything
• Did you hear about Oracle 18c?
• This is a starting point – don’t be afraid to try!

8
•‫דעתכם‬ ‫חוות‬ ‫את‬ ‫לקבל‬ ‫ונשמח‬ ‫משוב‬ ‫טופס‬ ‫יחולק‬ ‫הסמינר‬ ‫יום‬ ‫בסיום‬.
•‫חשובה‬ ‫דעתכם‬:‫יותר‬ ‫טובים‬ ‫הסמינרים‬ ‫את‬ ‫לעשות‬ ‫לנו‬ ‫המאפשרת‬ ‫זו‬ ‫היא‬!
Our REAL Agenda
10:30-10:45‫הפסקה‬
12:30-13:30‫משתתפ‬ ‫לכל‬ ‫צהריים‬ ‫ארוחת‬‫המלון‬ ‫בגן‬ ‫הכנס‬ ‫י‬
15:00-15:15‫הפנים‬ ‫קבלת‬ ‫במתחם‬ ‫מתוקה‬ ‫הפסקה‬
16:30‫הביתה‬ ‫הולכים‬

10
What is Docker?
• Docker is an open-source project that automates the deployment
of applications inside software containers, by providing an additional
layer of abstraction and automation of operating system–level
virtualization on Linux

11
What’s the Buzz Around Containers?
• The software industry has changed dramatically
• 5-10 years ago:
• Monolithic applications
• Long development cycles
• Slowly scaling up
• Now:
• Decoupled services, Microservices
• Fast, iterative improvements
• Quickly scaling out
• Google claims all of its services now run in containers

12
It’s All About CI/CD
• CI/CD: Continuous Integration and Continuous Deployment
• We want to always integrate new things into our applications and
have a fast deployment
• The challenge: development and deployment became complex:
multiple stacks, multiple targets. Keeping everything in sync and
deploying a reliable environment in short spans to production is hard
• Docker Goal: Provide a uniformed wrapper around a software
package: «Build, Ship and Run Any App, Anywhere»
• Using Docker to achieve CI/CD is the most common usage for Docker
today

13
The Analogy: Cargo Transport

14
The Analogy: Cargo Transport Solution

15
Docker Solution: Containers

16
Docker Containers
• The concept: One Application per Container
• Self contained - everything you need is in an isolated environment
• Isolation for security - containers not aware of each other
• Isolation for performance – hard limits on CPU, RAM and IO per
container

17
VMs vs. Containers
• Less overhead but It’s not a lightweight VM – it’s a Software Delivery Platform
• It's about isolation - Containers run on Linux kernel of host and are visible on host

18
Why Use Docker Containers
• It’s highly portable
• It Runs everywhere: bare metal, VM, and even the cloud (AWS, GCP,
Oracle Cloud, Microsoft Azure)
• Small, lightweight and highly scalable
• Better performance than VMs, smaller foot print
• Very cost effective

19
Why Does IT Cares About Docker?
Developers care because:
• Quickly create ready-to-run packaged applications
• A clean, safe, hygienic, portable runtime environment
• No missing/conflicting dependencies or packages
• Each app runs in an isolated container
• Automate testing, integration, packaging
• Reduce/eliminate platform compatibility issues
• Cheap/zero cost deployment, with instant replay and reset

20
Why Does IT Cares About Docker? (2)
Administrators care because:
• Configure once, run many times
• Makes app lifecycle efficient, consistent and repeatable
• Eliminate environment inconsistencies between development, test,
production
• Supports segregation of duties
• Improve speed and reliability of continuous integration and deployment
• Lightweight containers address performance, costs, deployment and
portability issues

21
Common Use Cases
• Development and Test Environment
• Micro-services
• Application Deployment without “dependency hell” of multiple
applications
• It Solves the “It Worked For Me!” issue
• Continuous Integration
• Platform-as-a-Service (PaaS)

22
Docker Inc.
• Founded in 2009
• Primary sponsor of the Docker Project
• Hires maintainers and contributors
• Provides infrastructure for the project
• Runs the Docker Hub and Store
• How does Docker Inc. make money?
• Docker Hub has free and paid services
• DTR (Docker Trusted Registry)
• Enterprise support for Engine and other products
• Training and professional services

26
Let’s Begin: Installing Docker
• Docker is easy to install
• Runs on practically every Linux distribution (Kernel must be 2.6.32+ or
3.10+)
• Runs on OS X via a virtual machine
• Microsoft Windows (10) via virtual machine
• Community Edition (CE) and Enterprise Edition are available
• EE has support, and some certifications to hardware/software
• CE has the community support and is way more common
• Latest version is 17.09 (released on September, 2017), I still use
17.06.2…

27
Docker on Windows 10
• Make sure your virtualization is on:

28
Docker on Windows
• Download the Docker Toolbox from
https://www.docker.com/products/docker-toolbox
• That will install:
• Docker Client for Windows
• Docker Toolbox management tool and ISO
• Oracle VM VirtualBox
• Git MSYS-git UNIX tools
• Click the “QuickStart Terminal” on your desktop and enable the User
Account Control permission

29
Installing Docker: Oracle Linux 7
• Unlike other distros, Oracle Linux doesn’t enable the yum repository
holding Docker by default, so we need to first enable it:
• Use yum to install Docker engine:
[root@lnx7-oracle-1 ~]# vi /etc/yum.repos.d/public-yum-ol7.repo
...
[ol7_addons]
name=Oracle Linux $releasever Add ons ($basearch)
baseurl=http://yum.oracle.com/repo/OracleLinux/OL7/addons/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
[root@lnx7-oracle-1 ~]# yum install docker-engine

30
• Create a btrfs mount point for the docker file system
• This filesystem will provide better performance and copy-on-write
support
• More on copy-on-write and btrfs - later
mkfs.btrfs -L btrfs1 /dev/sdb
echo “LABEL=btrfs1 /var/lib/docker btrfs defaults 0 1” >>/etc/fstab
mkdir -p /var/lib/docker
mount /var/lib/docker

31
• Start the Docker daemon and enable it to start automatically:
[root@lnx7-oracle-1 ~]# systemctl start docker
[root@lnx7-oracle-1 ~]# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service
to /usr/lib/systemd/system/docker.service.
[root@lnx7-oracle-1 ~]# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor
preset: disabled)
Drop-In: /etc/systemd/system/docker.service.d
└─docker-sysconfig.conf
Active: active (running) since Thu 2017-11-16 21:30:31 IST; 13min ago
Docs: https://docs.docker.com
Main PID: 881 (dockerd)
Memory: 18.6M
CGroup: /system.slice/docker.service
├─881 /usr/bin/dockerd --selinux-enabled

32
Check Docker Version
• After install, verify the docker version and docker info
[root@lnx7-oracle-1 ~]# docker version
Client:
Version: 17.06.2-ol
API version: 1.30
Go version: go1.8.3
Git commit: d02b7ab
Built: Fri Oct 6 00:02:23 2017
OS/Arch: linux/amd64
Server:
Version: 17.06.2-ol
API version: 1.30 (minimum version 1.12)
Go version: go1.8.3
Git commit: d02b7ab
Built: Fri Oct 6 00:03:48 2017
OS/Arch: linux/amd64
Experimental: false

33
The docker Group
• Add a docker group
• Add a user to the docker group
• Restart the docker deamon
• Relogin to the use the user, without using sudo to run things from
root
$ sudo groupadd docker
$ sudo gpasswd -a $USER docker
$ sudo systemctl restart docker.service

34
Docker and Security
• The docker user is root equivalent
• It provides root-level access to the host
• You should restrict access to it like you would protect root
• If you give somebody the ability to access the Docker API, you are
giving them full access on the machine
• Therefore, the Docker control socket is (by default) owned by the
docker group, to avoid unauthorized access on multi-user machines

Getting Started:
Working With Docker

37
Docker Architecture
• The Docker daemon (or "Engine"): receives and processes incoming
Docker API requests
• The Docker client talks to the Docker daemon via the Docker API
• We'll use mostly the CLI embedded within the docker binary
• GUI tools available, we will talk about them later
• Docker Hub Registry (and Docker Store): Collection of public images
• The Docker daemon talks to it via the registry API

38
Hello World: Busybox
• Let’s run a container that echo “hello world”
• This started a very small (>1 MB) container called Busybox, and ran
the single command we provided
• Busybox is usually used for embedded system (routers, switches, etc.)
[zohar@lnx7-oracle-1 ~]$ docker run --rm busybox echo hello world
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
0ffadd58f2a6: Pull complete
Digest: sha256:bbc3a03235220b170ba48a157dd097dd1379299370e1ed99ce976df0355d24f0
Status: Downloaded newer image for busybox:latest
hello world

39
Using Shell to Busybox
• Let’s connect to that container and have a look around
• We used -it option to indicate “interactive” and “tty” (terminal)
[zohar@lnx7-oracle-1 ~]$ docker run --rm -it busybox sh
/ # ps -ef
PID USER TIME COMMAND
1 root 0:00 sh
5 root 0:00 ps -ef
/ # uname -a
Linux a451342a146a 4.1.12-103.9.2.el7uek.x86_64 #2 SMP Tue Oct 31 16:43:46 PDT
2017 x86_64 GNU/Linux
/ #

40
Which Containers are Up?
• We can check which containers are up using docker ps
• We can check which container was the last to start by using
docker ps -l

41
Detaching an Interactive Container
• We can detach the interactive container by using CTRL+P,Q
• When checking on the host what containers are up, we can see it’s
running in the background
• We can re-attach back to it by using docker attach command
• We can use the container name or ID in order to re-attach
[zohar@lnx7-oracle-1 ~]$ docker attach vibrant_borg
/ #

42
Running Ubuntu in Container
• Let’s try a more useful container: Ubuntu Linux
• Here we can actually install things on it (it’s connected to the
internet!)
[zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu
root@502f2202f0bc:/#
root@502f2202f0bc:/# apt-get update
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [52.0
kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
[...]

43
Working in the Container
• Let’s have some fun: let’s install figlet
• This only install the program on our current container, it does not
change our image or our host system
root@502f2202f0bc:/# apt-get install figlet
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
figlet
[…]
root@502f2202f0bc:/# figlet hello
_ _ _
| |__ ___| | | ___
| '_ / _ | |/ _
| | | | __/ | | (_) |
|_| |_|___|_|_|___/

44
Running Containers in the Background
• We can start containers and run them in the background using the
run -d flag – but what happens to their console output?
• Console output is being logged – we can attach to the full output by
using the docker logs command
[zohar@lnx7-oracle-1 ~]$ docker run -d jpetazzo/clock
70e4be1a0c7ece5831632d169f21974ef9945234c6f48223a24ac92f8e73517f
[zohar@lnx7-oracle-1 ~]$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
70e4be1a0c7e jpetazzo/clock "/bin/sh -c 'while..." 12 seconds ago
Up 10 seconds tender_murdock
[zohar@lnx7-oracle-1 ~]$ docker logs 70e
Fri Nov 17 11:23:50 UTC 2017
Fri Nov 17 11:23:51 UTC 2017
Fri Nov 17 11:23:52 UTC 2017
Fri Nov 17 11:23:53 UTC 2017

45
View Tail of the Log
• To view only the tail of the log, we use docker logs --tail
command (with or without –f)
[zohar@lnx7-oracle-1 ~]$ docker logs 70e --tail 3
Fri Nov 17 11:27:30 UTC 2017
Fri Nov 17 11:27:31 UTC 2017
Fri Nov 17 11:27:32 UTC 2017

46
Stopping the Container
• Stopping our container gracefully:
• We can also use the kill command which will not wait for the
container to stop gracefully for 10 seconds
• When using interactive mode, CTRL+C will send SIGINIT and close the
container
[zohar@lnx7-oracle-1 ~]$ docker ps
STATUS PORTS NAMES
502f2202f0bc ubuntu "/bin/bash" 5 minutes ago
Up 5 minutes tender_keller
[zohar@lnx7-oracle-1 ~]$ docker stop 502
502
[zohar@lnx7-oracle-1 ~]$

47
One-Off Containers
• Using the --rm flag when running the container, will cause the
container to be removed completely when it is taken down
• This is useful for “one-off” containers that will do an action and then
disappear (like the “hello world” we did before)
• We can also use the docker rm command to delete unused
containers

48
Showing Stopped Containers
• Stopped containers still exist – they are not automatically removed
• We can list all containers (running and stopped) by using
docker ps –a
[zohar@lnx7-oracle-1 ~]$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS
70e4be1a0c7e jpetazzo/clock "/bin/sh -c 'while..." 16 minutes ago Exited (137) 1 second ago
502f2202f0bc ubuntu "/bin/bash" 2 hours ago Exited (0) 16 minutes ago
262ebcab83c8 ubuntu "/bin/bash" 2 hours ago Exited (0) 2 hours ago

49
Restarting and Attaching to Our Container
• If we restart our container, the figlet command will be available (but
we still can’t duplicate our container)
• If we start another container (using run), the figlet will not be
available because it’s not in our base image
[zohar@lnx7-oracle-1 ~]$ docker start 502
502
[zohar@lnx7-oracle-1 ~]$ docker attach 502
root@502f2202f0bc:/# figlet hello again
_ _ _ _
| |__ ___| | | ___ __ _ __ _ __ _(_)_ __
| '_ / _ | |/ _ / _` |/ _` |/ _` | | '_
| | | | __/ | | (_) | | (_| | (_| | (_| | | | | |
|_| |_|___|_|_|___/ __,_|__, |__,_|_|_| |_|
|___/

50
docker exec Command
• In some cases, attaching to a container will not allow us to interact
with it (we’ll talk about CMD and ENTRYPOINT later)
• In those cases, we will want to execute a command in the container
to interact with it:
• We can run whatever command we want, but bash is one of the
more common…
[zohar@lnx7-oracle-1 ~]$ docker exec -ti plugpdb bash

51
Summary for the Basic Commands
• docker ps [-a] [-l] [-q]
• docker run [-it|-d] [--rm]
• docker attach
• docker start
• docker stop
• docker kill
• docker rm (docker rmi)

Under the Hood:
Images and Layers

53
Images and Layers
• An image is a collection of files + some meta data
(technically: those files form the root filesystem of a container)
• Images are made of layers, conceptually stacked on top of each other
• Each layer can add, change, and remove files
• Images can share layers to optimize disk
usage, transfer times, and memory use
• Layer default location: /var/lib/docker
[Source: https://docs.docker.com/terms/layer/]

54
Differences Between Containers and Images
• An image is a read-only filesystem - we can’t change them after they
are created
• A container is an encapsulated set of processes running in a read-
write copy of that filesystem
• Images are like templates/stencils we create Containers from
• To optimize container boot time, copy-on-write is used instead of
regular copy
• The command docker run starts a container from a given image

55
Copy-On-Write
• Copy-on-write, sometimes referred to as implicit sharing or
shadowing is a resource-management technique to efficiently
implement a "duplicate" or "copy" operation on modifiable resources
• If a resource is duplicated but not modified, it is not necessary to
create a new resource
• Modifications must still create a copy, but the copy operation is
deferred to the first write
• By sharing resources in this way, it is possible to significantly reduce
the resource consumption of unmodified copies, while adding a small
overhead to resource-modifying operations

56
UnionFS and BTRFS
• AUFS is a union filesystem. The aufs storage driver was previously the
default storage driver used for managing images and layers on Docker
• Btrfs is a next generation copy-on-write filesystem that supports
many advanced storage technologies that make it a good fit for
Docker
• Btrfs is included in the mainline Linux kernel
• Docker’s btrfs storage driver leverages many btrfs features for image
and container management: block-level operations, thin provisioning,
copy-on-write snapshots, and ease of administration

57
Where Do We Get a Base Image?
• Build our own from “Scratch”
• Docker Hub and Docker Hub Store
• Some images might require approve in the store before pulling it
• Get recipes off GitHub or BitBucket
• Remember: Images are read-only, we can’t change them after they
are created but we can evolve them

58
Wait! Images Are Read-Only?!
• Question: If images are read-only, how do we change them?
• Answer: We don’t!
• We create a new container from that image
• We make changes to that container
• When we are satisfied with those changes, we evolve them into a new layer
• A new image is created by stacking the new layer on top of the old image

59
The Chicken and Egg Problem

60
Creating the First Image
• There is a special empty image called scratch which means the
first image is built from “scratch”
• The docker import command loads a tarball into Docker
• The imported tarball becomes a standalone image
• That new image has a single layer
• We probably don’t need this – we will use more “common” base
images

61
The Docker Hub Repository
• Docker Hub is a public repository of pre-built images
• We can upload (push) images, or download (pull) other
people/organizations images to base our containers/layers on
• If we don’t want to use the public Hub, we can create our own local
repositories (and namespaces)

62
Downloading an Image
• Explicitly: we can download an image without initiating a container
using the docker pull command
• Download the exact image only
• Image kept on the Docker host ready to use
• Implicitly: when executing docker run command, it will download
all the images needed that are not locally stored
• Download the entire dependency layers
• Only being done once, on the first run of that image

63
Pulling Image Example
• Using the run command, we tried to start a container from the
Ubuntu image. Some layers did not exist so they were automatically
pulled and downloaded:
[zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
ae79f2514705: Downloading [===================> ]
18.19MB/47.62MB
c59d01a7e4ca: Download complete
41ba73a9054d: Download complete
f1bbfd495cc1: Download complete
0c346f7223e2: Download complete

64
Listing The Local Docker Images
• List all the local images by using the docker images command
[zohar@lnx7-oracle-1 ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
figlet latest 564ca4364896 5 days ago 162MB
mysql latest 5709795eeffa 12 days ago 408MB
ubuntu latest dd6f76d9cc90 13 days ago 122MB
busybox latest 6ad733544a63 13 days ago 1.13MB
jpetazzo/namer master f1557ed35146 2 years ago 723MB
jpetazzo/clock latest 12068b93616f 2 years ago 2.43MB

65
But How Do We Find Images?
• Finding an image in the repository is easy :
• Find it online using hub.docker.com or store.docker.com
• Use the shell interface:
$ docker search mysql

66
Finding an Image: Shell
• Name is the name and namespace of the image we are searching for
• Root-like (i.e. mysql or mariadb)
• User/organizaion (i.e. zabbix or realmgic)
• Self hosted (e.g. registry.example.com:5000/my-private-
image)
• Description is an optional description for your images

67
Namespaces
• Root: official images. They are put there by Docker Inc., but they are
generally authored and maintained by third parties
• Small, "swiss-army-knife" images like busybox
• Distro images to be used as bases for your builds, like ubuntu, fedora
• Ready-to-use components and services, like redis, postgresql, mysql
• User: The user namespace holds images for Docker Hub users and
organizations
• Self-Hosted: Images which are not hosted on Docker Hub, but on
third party registries. They contain the hostname (or IP address), and
optionally the port, of the registry server.

68
Finding an Image (cont.)
• Stars indicate how poplar an image is
• Official indicate it is part of the root namespace
• Automated images are built automatically by the Docker Hub using
submitted Dockerfiles in GitHub - this means that their build recipe is
always available

69
A Word of Warning!
• Images on the Hub are not being verified and might be harmful
• Use trusted images from the store to make sure legitimate source
• You can also use images with a known Dockerfile – for example,
Automated Docker Images where you can verify their build process

70
Image Tags
• Images can have tags to define image versions or variants
• docker pull ubuntu will refer to ubuntu:latest
• The :latest tag is generally updated often
• We only specify tags when:
• We want to ensure that the same version will be used everywhere
• Recording a procedure into a script
• Going to production
• Ensure repeatability later

71
Saving a New Image
• docker commit
• Saves all the changes made to a container into a new layer
• Creates a new image (effectively a copy of the container)
• Be careful, it saves everything you did to the container!
• docker build
• Performs a repeatable build sequence in a Dockerfile (recipe)
• This is the preferred method!

72
Pushing an Image to the Hub
• Once the image is ready, pushing it to the hub is simple - using the
docker push command
• Make sure your image is tagged before uploading
[zohar@lnx7-oracle-1 ~]$ export DOCKER_ID_USER=“realmgic“
[zohar@lnx7-oracle-1 ~]$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't
have a Docker ID, head over to https://hub.docker.com to create one.
Username (realmgic):
Password:
Login Succeeded
[zohar@lnx7-oracle-1 ~]$ docker push $DOCKER_ID_USER/my_image

73
Building a New Image Example
• Let’s use the example from earlier and build an image that has the
figlet tool in it
• Our base image will be the Ubuntu Image
• The base package might not be updated, so let’s update its Linux
packages and install figlet
root@502f2202f0bc:/#
root@502f2202f0bc:/# apt-get update && apt-get install figlet
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
[...]

74
Check Container Differences
• We can check what changed between this container and the base
image using the docker diff command
• Since Docker uses copy-on-write, these are the only files kept for this
container on disk. Multiple containers share the base image files (due
to UnionFS/btrfs for example)
[zohar@lnx7-oracle-1 ~]$ docker diff 502
C /root
A /root/.bash_history
C /etc
C /etc/alternatives
A /etc/alternatives/figlet
A /etc/alternatives/figlet.6.gz
[...]

75
Commit The Changes
• Create a new image from the newly built container using the
docker commit command
• The new image will look like this since we didn’t tag it with a name:
• Tag the new image:
[zohar@lnx7-oracle-1 ~]$ docker commit 502
sha256:8cc5680434bb94b272e59624cd44667eaab191a55a08c38a90c472f9f9b45b3f
<none> <none> 8cc5680434bb About a minute ago 162MB
[zohar@lnx7-oracle-1 ~]$ docker tag 8cc5680434bb figlet
figlet latest 8cc5680434bb 5 minutes ago 162MB

77
Automated Docker Images: Dockerfile
• Dockerfile is a build recipe to customize the base (r/o) image and
make it into a new image
Base Docker
Image
Instruction
• Run commands
• Add files/directories
• Define ENV variables
• Define processes to run
• Set Entrypoint
New Docker
Image

78
Automated Docker Images on Hub
• Docker Hub can automatically build images based on Dockerfiles
placed on Github
• These Images can be built automatically by the Hub or we can
download the recipe and build it ourselves
• Advantages:
• Images built this way are built exactly as specified
• The Dockerfile is available to anyone with access to your Docker Hub
repository
• Your repository is kept up-to-date with code changes automatically

79
Creating a Dockerfile
• Our new Dockerfile must be in an empty directory
• In the Dockerfile, we specify the base Image we start from using the
FROM command
• The RUN commands are never interactive, so we’ll need to sort it out
when we build the script
[zohar@lnx7-oracle-1 ~]$ mkdir figlet
[zohar@lnx7-oracle-1 ~]$ cd figlet/
[zohar@lnx7-oracle-1 figlet]$ vi Dockerfile
FROM ubuntu
RUN apt-get update -y
RUN apt-get install –y figlet

80
Build The Image
• Create a new image from the Dockerfile we createdusing the
docker build command
[zohar@lnx7-oracle-1 figlet]$ docker build -t figlet_auto .
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM ubuntu
---> dd6f76d9cc90
Step 2/3 : RUN apt-get update -y
---> Running in 174cec778baa
[...]
---> 035ad14c7312
Removing intermediate container 174cec778baa
Step 3/3 : RUN apt-get install figlet -y
---> Running in e9d2c2870c5d
[...]
---> e6e0b459c15c
Removing intermediate container e9d2c2870c5d
Successfully built e6e0b459c15c
Successfully tagged figlet_auto:latest

81
Reading the Output: Step 1
• Our base image is Ubuntu – so we start a new container for it
(dd6f76d9cc90) and make it our base image
• The -t indicates the tag for the name of our new image
[zohar@lnx7-oracle-1 figlet]$ docker build -t figlet_auto .
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM ubuntu
---> dd6f76d9cc90

82
• We start a new container from our base image (174cec778baa) and
modify it using our RUN command
• At the end of this step, we’ll have a new base image: 035ad14c7312
• The intermediate base image will be removed
Step 2/3 : RUN apt-get update -y
---> Running in 174cec778baa
[...]
---> 035ad14c7312
Removing intermediate container 174cec778baa

83
• We repeat the process for step 3
• The final container becomes our base image (e6e0b459c15c) and it
is tagged to the name we chose (figlet_auto:latest)
Step 3/3 : RUN apt-get install figlet -y
---> Running in e9d2c2870c5d
[...]
---> e6e0b459c15c
Removing intermediate container e9d2c2870c5d
Successfully built e6e0b459c15c
Successfully tagged figlet_auto:latest

84
Image History
• The history command lists all the layers composing an image
• For each layer, it shows its creation time, size, and creation command
• When using Dockerfile, each line indicates a single command
[zohar@lnx7-oracle-1 figlet]$ docker history figlet_auto
IMAGE CREATED CREATED BY SIZE
COMMENT
4cbdf48e2076 About a minute ago /bin/sh -c apt-get install figlet -y 1.02MB
c12b1ebe20b8 2 minutes ago /bin/sh -c apt-get update -y 39.2MB
dd6f76d9cc90 13 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 13 days ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B
<missing> 13 days ago /bin/sh -c sed -i 's/^#s*(deb.*universe... 2.76kB
<missing> 13 days ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 13 days ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B
<missing> 13 days ago /bin/sh -c #(nop) ADD file:5b334adf9d9a225... 122MB

85
Image History (cont.)
• When NOT using Dockerfile, some lines will not explain what really
happened:
[zohar@lnx7-oracle-1 figlet]$ docker history figlet
IMAGE CREATED CREATED BY SIZE
COMMENT
8cc5680434bb 43 minutes ago /bin/bash 40.3MB
dd6f76d9cc90 13 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 13 days ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B
<missing> 13 days ago /bin/sh -c sed -i 's/^#s*(deb.*universe... 2.76kB
<missing> 13 days ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 13 days ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B
<missing> 13 days ago /bin/sh -c #(nop) ADD file:5b334adf9d9a225... 122MB

86
The Caching System
• Building the same image again will be instantaneous
• After each step docker takes a snapshot of the resulting image
• Before executing the step, Docker checks if it has already built that
sequence (based on the text in the Dockerfile)
• Example:
• Changing the apt-get install figlet and add a space, it will run again
• The apt-get update will not run again even though the apt mirrors
updated (same command)
• We can force a rebuild with docker build --no-cache

87
CMD or ENTRYPOINT?
• Both ENTRYPOINT and CMD give you a way to identify which
command should be run when a container is started from your image
• For example, most Linux container will use /bin/bash or /bin/sh
as their default CMD to allow shell access
• Running a container without a CMD or ENTRYPOINT will result with an
error (either built in or via docker run command)
• Each image can have only one CMD and one ENTRYPOINT. Specifying
more than one in the Dockerfile will override the previous command

88
Using CMD
• Example: let’s say we want to change or figlet image to display
“hello” message.
• We need to modify the Dockerfile and build it:
• When running a new image:
FROM ubuntu
RUN apt-get install figlet -y
CMD figlet hello
[zohar@lnx7-oracle
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto
_ _ _
| |__ ___| | | ___
| '_ / _ | |/ _
| | | | __/ | | (_) |
|_| |_|___|_|_|___/

89
Override CMD
• If we want to override the CMD, we just send in a new command in
the run command
• The bash canceled our figlet command – but what happens if we
don’t want it to override? What if we want to send in a parameter?
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto bash
root@388ac4ed1d28:/#

90
Using ENTRYPOINT
• We want to be able to specify a different message on the command
line (parameter)
• When running a new image:
FROM ubuntu
ENTRYPOINT [“figlet”, “-f”, “banner”]
[zohar@lnx7-oracle
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto banner
##### ## # # # # ###### #####
# # # # ## # ## # # # #
##### # # # # # # # # ##### # #
# # ###### # # # # # # # #####
# # # # # ## # ## # # #
##### # # # # # # ###### # #

91
Override ENTRYPOINT
• If we want to override the ENTRYPOINT, we can’t just send in a new
command in the run command because it will be printed…
• We need to override it with the --entrypoint flag:
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto bash
##### ## #### # #
# # # # # # #
##### # # #### ######
# # ###### # # #
# # # # # # # #
##### # # #### # #
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti --entrypoint bash figlet_auto
root@5dd5da9a0825:/#

92
Using CMD and ENTRYPOINT Together
• In our example, figlet needs and extra parameter to work (the
message). How can we specify a default value for it?
• We can combine using CMD and ENTRYPOINT
• ENTRYPOINT will specify the command to run
• CMD will specify the default parameter
FROM ubuntu
ENTRYPOINT ["figlet", "-f", "banner"]
CMD ["CMD"]

93
Using ENTRYPOINT and CMD Example
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto
##### # # ######
# # ## ## # #
# # # # # # #
# # # # # #
# # # # #
# # # # # #
##### # # ######
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto test
##### ###### #### #####
# # # #
# ##### #### #
# # # #
# # # # #
# ###### #### #

94
Copying Files From Host to Image
• We can copy files from our host to the image when we build it
• We can see the file on our image’s container
• Changing the file will invalidate the cached layer – so rebuilding the
image with a new file will generate a new image
FROM ubuntu
COPY zohar.txt /
ENTRYPOINT ["figlet", "-f", "banner"]
CMD ["CMD"]
[zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti --entrypoint bash figlet_auto
root@b65dd0df7597:/# cat /zohar.txt
Testing a copy file...

95
The ADD Command
• Since the COPY command does not support copying file from a URL,
we can use the ADD command for that
• ADD also allows us to copy tar files and automatically open them
• It is best practice to use COPY unless you must use ADD

96
The WORKDIR Instruction
• The WORKDIR instruction sets the working directory for subsequent
instructions
• It also affects CMD and ENTRYPOINT, since it sets the working
directory used when starting the container
• You can specify WORKDIR again to change the working directory for
further operations
WORKDIR /src

97
The ENV Instruction
• The ENV instruction specifies environment variables that should be
set in any container launched from the image
• This is the same as exporting in the OS level
• ENV variables can be overridden by the -e flag in the docker run
command
ENV ORACLE_SID orclcdb
docker run -p 1521:1521 -e ORACLE_SID=orcl store/oracle/database-enterprise:12.2.0.1

98
The USER Instruction
• The USER instruction sets the user name or UID to use when running
the image
• It can be used multiple times to change back to root or to another
user

100
Docker Network Basics
• Containers cannot have public IPv4 addresses
• They do have private addresses
• Services have to be exposed port by port
• Ports have to be mapped to different ports on the host to avoid
conflicts
• Containers can see the network by default – it uses our default bridge
network
• Docker --link only worked on a single host – does not work in
swarm/cluster and considered deprecated

101
Example: NGINX Server
• Starting a NGINX (web) server
• The -P flag (publish all) indicates every exposed port to be mapped
automatically to a port in the host server.
• In this case, port 80 will be mapped to port 9000
• We can also ask the container for his mapped ports:
[zohar@lnx7-oracle-1 ~]$ docker run -d -P nginx
[zohar@lnx7-oracle-1 ~]$ docker ps -l
PORTS NAMES
47733cd2d419 nginx "nginx -g 'daemon ..." About a minute ago Up About a minute
0.0.0.0:9000->80/tcp gallant_poincare
[zohar@lnx7-oracle-1 ~]$ docker port 477
80/tcp -> 0.0.0.0:9000

102
Example: NGINX Server (cont.)
• When we try to reach the container port, we use the localhost port
for it:
[zohar@lnx7-oracle-1 ~]$ curl -l localhost:9000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

103
Manual Binding of Ports
• Setting the ports manually can be done during the run command
• The convention is host-port:container-port
• Trying to start another container with the same port will result with
an error
[zohar@lnx7-oracle-1 ~]$ docker run --rm -d -p 80:80 nginx
[zohar@lnx7-oracle-1 ~]$ docker run --rm -d -p 80:80 nginx
6abc743b8dbaa46b0af1d6f4d72f6313bd41baa9c74524649ac7fe489e387709
docker: Error response from daemon: driver failed programming external connectivity
on endpoint friendly_bohr
(dde052b96b19300988e0c4ea389fe5d0a91eab3e91bafd0dfd9d1d454869dc27): Bind for
0.0.0.0:80 failed: port is already allocated.

104
Exposing Ports in the Image using Dockerfile
• The EXPOSE instruction tells Docker what ports are to be published in
this image
• The Dockerfile doesn’t actually map the port to a port on the host –
we still need to make it public in our run command
EXPOSE 1521

105
Finding Container Private IP
• We can use docker inspect command to find the private IP of a
container
• The docker inspect is an advanced command, that can retrieve tons of
information about our containers
• We can ping our container
[zohar@lnx7-oracle-1 ~]$ docker inspect --format '{{ .NetworkSettings.IPAddress }}'
keen_neumann
172.17.0.3
[zohar@lnx7-oracle-1 ~]$ ping 172.17.0.3
PING 172.17.0.3 (172.17.0.3) 56(84) bytes of data.
64 bytes from 172.17.0.3: icmp_seq=1 ttl=64 time=0.109 ms
64 bytes from 172.17.0.3: icmp_seq=2 ttl=64 time=0.148 ms

106
Docker Networks
• Conceptually, a Docker network is a virtual switch
• It can be local (to a single Engine) or global (across multiple hosts)
• Each network has a dedicated subnet
• Containers with explicit names are discoverable via DNS
• A network is managed by drivers – some of them are built in and
some can be added by plugins
[zohar@lnx7-oracle-1 ~]$ docker network ls
NETWORK ID NAME DRIVER SCOPE
03d9803daff0 bridge bridge local
3eddac377534 host host local
e733f2a48ed2 none null local

107
The Default Bridge Driver
• By default, the container gets a virtual eth0 interface
• It is connected to the Docker bridge network
• Addresses are allocated on a private, internal subnet
• Outbound traffic goes through an iptables MASQUERADE rule
• Inbound traffic goes through an iptables DNAT rule
• The container can have its own routes, iptables rules, etc.

108
The Host Driver
• The host network is internal network between the host and the
container
• A container with a host driver sees (and can access) the network
interfaces of the host
• Network traffic doesn't have to go through NAT, bridge, or veth
• Performance = native!

109
The null Driver
• It only gets the lo loopback interface. No eth0
• It can't send or receive network traffic
• Useful for isolated/untrusted workloads

110
Creating a Network
• Create a network
• Use network for a container
[zohar@lnx7-oracle-1 ~]$ docker network create OracleWeek
72635c667b4c3e54e350925339cef73cf480528cef6368d6f59954d2358c210d
[zohar@lnx7-oracle-1 ~]$ docker network ls
72635c667b4c OracleWeek bridge local
03d9803daff0 bridge bridge local
3eddac377534 host host local
e733f2a48ed2 none null local
[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont1 --net OracleWeek busybox
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
144: eth0@if145: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.18.0.2/16 scope global eth0

111
Containers DNS
• All containers with specific name in the same network share the
same dynamic DNS (placed at 127.0.0.11):
[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont2 --net OracleWeek busybox
/ # ping cont1
PING cont1 (172.18.0.2): 56 data bytes
64 bytes from 172.18.0.2: seq=0 ttl=64 time=0.176 ms
/ # nslookup cont1
Server: 127.0.0.11
Address 1: 127.0.0.11
Name: cont1
Address 1: 172.18.0.2 cont1.OracleWeek
/ # nslookup cont2
Server: 127.0.0.11
Address 1: 127.0.0.11
Name: cont2
Address 1: 172.18.0.3 05efb3eb1863

112
Connecting Multiple Networks
• Networks are independent from each other but a container can be
connected to multiple networks
• We can attach a network to a running container
• Example:
• Create a new network and a new container in that network
[zohar@lnx7-oracle-1 ~]$ docker network create ilOUG
41244791088e6970dca741b54fa54a8e0f0a08eeb1b184a78220a9cf03d2528a
[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont3 --net ilOUG busybox
/ # ping cont1
ping: bad address 'cont1‘

113
Connecting Multiple Networks (cont.)
[zohar@lnx7-oracle-1 docker network connect ilOUG cont1
[zohar@lnx7-oracle-1 ~]$ docker attach cont1
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
/ # ping cont2
/ # ping cont3

114
Multi-host Networking: Overlay Driver
• When using cluster, overlay network will interact between nodes

115
Overlay Driver
• The overlay network driver can support a clustered (swarm mode)
environment
• When you create a service that uses the overlay network, the
manager node automatically extends the overlay network to nodes
that run service tasks
• Only swarm services can connect to overlay networks, not standalone
containers
• The overlay network without swarm requires a valid key-value
store service (Consul, Etcd, and ZooKeeper (Distributed store)) and
might be deprecated in the future

Docker Storage
Volumes, Bind Mounts and tmpfs

117
Docker “External” Storage
• Our containers are independent – they can’t see the host filesystem ,
but sometimes we want to share files/directories with the host or
between containers
• The solution: map a volume to the container so it will be available to
both the host and all the using containers

118
Named Volume
• Created and managed by Docker
• We can create a volume explicitly using the docker volume
create command, or Docker can create a volume during container
or service creation
• When creating a volume, it is stored within a directory on the Docker
host (empty)
• When you mount the volume into a container, this directory is what
is mounted into the container
• A given volume can be mounted into multiple containers
simultaneously

119
Named Volumes Example
• We can create volumes, list them and inspect for actual location:
[zohar@lnx7-oracle-1 ~]$ docker volume create test
Test
[zohar@lnx7-oracle-1 ~]$ docker volume ls
DRIVER VOLUME NAME
local fc7d0d5897f2090f51a3ee7589ba360841344dc34c605626337debff4dfa9860
local test
[zohar@lnx7-oracle-1 ~]$ docker inspect test
[
{
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/test/_data",
"Name": "test",
"Options": {},
"Scope": "local"
}
]

120
Volumes Functionality
• Volumes act as passthroughs to the host filesystem
• Volumes are easier to back up or migrate than bind mounts
• Volumes can be more safely shared among multiple containers
• When you docker commit, the content of volumes is not brought
into the resulting image
• Volume drivers allow you to store volumes on remote hosts or cloud
providers, to encrypt the contents of volumes, or to add other
functionality

121
Bind Mounts
• Bind mounts have limited functionality compared to volumes
• When we use a bind mount, a file or directory on the host machine is
mounted into a container
• The file or directory is referenced by its full path on the host machine
• The file or directory does not need to exist on the Docker host
already, if it does not exist – it will be automatically created (when
using –v)
• We use bind mounts when we want to mount a non-empty directory
from the host to the container

122
Binding the Mount Volumes
• Volumes can be mounted in two ways:
• on the docker run command
• Within a Dockerfile, with a VOLUME instruction
docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox
VOLUME /zohar

123
Mounting a Volume Manually
• We can map volumes on the docker run command
• The –v flag mounts a directory from your host into your Docker
container. The flag structure is:
• If [host-path] or [container-path] doesn't exist it is created
• You can control the write status of the volume with the ro and rw options
• If you don't specify rw or ro, it will be rw by default
docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox
[host-path]:[container-path]:[rw|ro|z|Z]

124
Volumes and SElinux
• The permission on the volume will be the same as on the host
• That might be a problem if the files are owned by a user the
container doesn’t know
• We can “help” Docker and SElinux play nice together by specifying
the “z” (r/o) or “Z” (r/w) flag:
[zohar@lnx7-oracle-1 ~]$ docker run -it -v $(pwd):/zohar --rm --name cont1 --net
ilOUG busybox
/ # ls -l /zohar/
ls: can't open '/zohar/': Permission denied
/ # [zohar@lnx7-oracle-1 docker run -it -v $(pwd):/zohar:Z --rm --name cont1 --net
ilOUG busybox
/ # ls /zohar/
Desktop Documents Downloads Music Pictures Public Templates Videos
figlet myimage namer oradata

125
tmpfs Mounts
• A tmpfs mount is not persisted on disk, either on the Docker host or
within a container
• It can be used by a container during the lifetime of the container, to
store non-persistent state or sensitive information
[zohar@lnx7-oracle-1 ~]$ docker run -it --rm --tmpfs /zohar/tmpfs busybox
/ # df -h /zohar/tmpfs/
Filesystem Size Used Available Use% Mounted on
tmpfs 1.8G 0 1.8G 0% /zohar/tmpfs

126
Orphaned Volumes
• What happens when you remove containers with volumes?
• They stay on disk – even if no one will ever use them again
• Use the –v flag to remove the container with the volumes attached
• We can prune the unused volumes using the docker volume
prune command
• Ultimately, you are the one responsible for logging, monitoring, and
backup of your volumes!

Oracle RDBMS, MySQL
and Docker

128
Oracle RDBMS Official Image
• Oracle provides their own Oracle RDBMS official image with database
(12.2.0.1 EE) included (free):
• https://store.docker.com/images/oracle-database-enterprise-edition
• In order to pull this image, we need a Docker repository account
• We also need to accept the conditions on the store itself
• Official Image from Oracle Store:
• https://container-registry.oracle.com/
docker login
docker pull store/oracle/database-enterprise:12.2.0.1

129
Oracle Official Image Limitations
• Supports a single instance database,
• No multiple homes
• No Grid Infrastructure/ASM
• No multiple instances on the same container
• No AMM
• Dataguard is not supported
• Database options and patching are not supported

130
Oracle Official Slim Image
• Since the official image is quite big (~4GB) there is also a slim version:
• This image does not support the following features - Analytics, Oracle R,
Oracle Label Security, Oracle Text, Oracle Application Express and Oracle
DataVault.
• The slim version is around 1.4GB
• Oracle support does not require you to use the provided Docker files
– we can build our own images!
docker pull store/oracle/database-enterprise:12.2.0.1-slim

131
Building Our Own Images
• Oracle official Docker team github account with different Dockerfiles
and samples: https://github.com/oracle/docker-images
• On the repository:
• Oracle 11.2.0.2 (XE) Dockerfile
• Oracle 12.1.0.2 Dockerfile and scripts (SE2, EE)
• Oracle 12.2.0.1 Dockerfile and scripts (SE2, EE)
• Lots of other Oracle stuff: Oracle Java, Weblogic, GoldenGate and NoSQL…
• The build recipes build our own images, so we can customize it
• Recipes require ZIP files of the software for deployment on the image

132
How It’s Going to Work
• The official image based on Oracle Linux 7
• The image contains an instance running inside the container
• The ORACLE_HOME and datafiles are kept on a volume (host
machine)
• The image also runs the listener on port 1521 and exposes this port
• The instance runs inside the container is a root container (CDB) in our
multitenant database. It also has a single plugged database
• Persistence:
• There is a volume plug we can use to map our local pluggable database files
to the database and then we can plug our database to the instance

133
Creating the Container
• Creating the container (might take some time):
[zohar@lnx7-oracle-1 ~]$ docker volume create OracleDBData
[zohar@lnx7-oracle-1 ~]$ docker run --name plugpdb -p 1521:1521 -p 5500:5500 -e
ORACLE_SID=MYCDB -v OracleDBData:/ORCL /home/zohar/oradata:/opt/oracle/oradata
store/oracle/database-enterprise:12.2.0.1
Setup Oracle Database
Oracle Database 12.2.0.1 Setup
Sat Nov 18 16:09:47 UTC 2017
Check parameters ......
log file is : /home/oracle/setup/log/paramChk.log
paramChk.sh is done at 0 sec
untar DB bits ......
log file is : /home/oracle/setup/log/untarDB.log
untarDB.sh is done at 166 sec
config DB ......
log file is : /home/oracle/setup/log/configDB.log
[…]

134
Inside the Container
• The entire database home and database file are stored in the same
location: /ORCL, divided into u01-u04 for different usages
• We can mount these areas into our own disks or volume
• In our example, the entire /ORCL is mounted into one volume:
OracleDBData
• We can mount other mount points or volumes to keep our pluggable
databases

135
Connecting to the Database
• Connecting to the database, via the container:
• The initial password for SYS is “Oradoc_db1”
• We can also connect using bash command and work like any other
“server”
docker exec -it plugdb bash -c "source /home/oracle/.bashrc; sqlplus /nolog"

136
Building Our Own Image
• Download the appropriate directory from and unzip it on the server
• https://github.com/oracle/docker-
images/tree/master/OracleDatabase/dockerfiles
• Download the Oracle Software from the Oracle site and place them in
the same directory
• Modify whatever you need – scripts, Dockerfile, keep some of the
options manually removed, etc.
• Build the image
docker build --force-rm=true --no-cache=true -t
oracle/database:12.2.0.1-ee -f Dockerfile.ee .

137
Start the Container
• Running the container is just like before – we can pass some env
variables but most of them already has default values
• This database doesn’t have to use containers (like the official image):
ORACLE_CONTAINERDB=false
docker run --name orcl1
-p 1522:1521 -p 5502:5500
-e ORACLE_SID=orclcdb
-e ORACLE_PWD=manager1
-e ORACLE_CHARACTERSET=WE8MSWIN1252
-e ORACLE_CONTAINERDB=false
oracle/database:12.2.0.1-ee

138
MySQL
• MySQL database is ideal for Docker Containers – its containers exist
since forever
• There are actually two MySQL main images:
• mysql (also available on the store)
• mysql/mysql-server (automated, connected to mysql GitHub)
• Both images are tagged with multiple version: 5.5, 5.6, 5.7 (latest)
and 8.0 (preview). They also run sub version (e.g. 5.6.38, 5.7.20, etc.)

139
Starting the MySQL Container
• Starting the container
• Connecting to the database using root:
docker run --name my-container-name -e MYSQL_ROOT_PASSWORD=my-secret-pw -d
mysql/mysql-server:tag
docker exec -it my-container-name mysql -uroot -p

140
Where to Store Data
• There are several ways to store data used by applications that run in
Docker containers:
• Let Docker manage the storage of your database data by writing the database
files to disk on the host system using its own internal volume management.
This is the default and is easy and fairly transparent to the user
• Create a data directory on the host system (outside the container) and mount
this to a directory visible from inside the container. This places the database
files in a known location on the host system, and makes it easy for tools and
applications on the host system to access the files

141
Connecting to Existing Database
• If you start your mysql container instance with a data directory that
already contains a database (specifically, a mysql subdirectory), it will
use that as its database
• Note: the $MYSQL_ROOT_PASSWORD variable should be omitted from the
run command line; it will in any case be ignored, and the pre-existing
database will not be changed in any way
• If there is no database initialized when the container starts, then a
default database will be created.

142
Creating Database Dumps
• Most of the normal tools will work, although their usage might be a
little convoluted in some cases to ensure they have access to the
mysqld server. A simple way to ensure this is to use docker exec and
run the tool from the same container, similar to the following
$ docker exec some-mysql sh -c 'exec mysqldump --all-databases -uroot
-p"$MYSQL_ROOT_PASSWORD"' > /some/path/on/your/host/all-databases.sql

143
Overriding Options and MySQL.cnf
• We can override options by using the --option flag
• To provide a different mysql.cnf, use the –v flag:
• Remember SELinux? We need to allow the file for the container:
docker run --name my-container-name -d mysql/mysql-server --option1=value --
option2=value
docker run --name my-container-name -d mysql/mysql-server --character-set-
server=utf8 --collation-server=utf8_general_ci
docker exec -it my-container-name cat /etc/my.cnf > /my/custom/config-file
docker run --name my-new-container-name -v /my/custom/config-file:/etc/my.cnf -e
MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:tag
chcon -Rt svirt_sandbox_file_t /my/custom/config-file

145
Docker Compose Tool
• Dockerfiles are great to build a single container
• But when you want to start a complex stack made of multiple
containers, you need a different tool. This tool is Docker Compose
• Docker Compose is optional (needs to be installed separately)
• The general idea of Compose is to enable a very simple, powerful
onboarding workflow:
• Clone your code
• Run docker-compose up
• Your app is up and running!

146
Compose overview
• You describe a set (or stack) of containers in a YAML file called
docker-compose.yml
• You run docker-compose up
• Compose automatically pulls images, builds containers, and starts
them
• Compose can set up networks, volumes, and other Docker options
• Compose can run the containers in the background, or in the
foreground
• When containers are running in the foreground, their aggregated
output is shown

147
Installing Compose
• We can download and install Docker Compose right off its github
project: https://github.com/docker/compose/releases
• Make sure to use the latest version (current version: 1.17.1):
curl -L https://github.com/docker/compose/releases/download/1.17.1/docker-compose-
`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

148
docker-compose.yml
• Example for a Docker compose yaml file:
• This file will start two containers: one based on the local Dockerfile
(www) and one based on the redis image. It will also mount a volume
and map a port.
version: '2'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/code
redis:
image: redis

149
docker-compose.yml for Oracle DBMS
• Another example for a Docker compose yaml file:
• In this example, we will start a database server using the 12.2.0.1-ee
image, map some ports. Our Oracle SID is also predefined.
version: '2'
services:
database:
image: oracle/database:12.2.0.1-ee
volumes:
- /home/oracle/oradata:/opt/oracle/oradata # persistent oracle database data.
ports:
- 1521:1521
- 8080:8080
- 5500:5500
environment:
-ORACLE_SID=DEVOPSUATCDB Example by Gerald Venzl

150
Docker Compose Usages
• Starting a Docker Compose environment:
• A network and two containers were created
• Check what is up (in our work directory only):
[zohar@lnx7-oracle-1 wordpress]$ docker-compose up -d
Creating network "wordpress_default" with the default driver
Creating wordpress_db_1 ...
Creating wordpress_db_1 ... done
Creating wordpress_wordpress_1 ...
Creating wordpress_wordpress_1 ... done
[zohar@lnx7-oracle-1 wordpress]$ docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------
wordpress_db_1 docker-entrypoint.sh mysqld Up 3306/tcp
wordpress_wordpress_1 docker-entrypoint.sh apach ... Up 0.0.0.0:8000->80/tcp

151
Docker and Docker Compose
• We can still use our original docker commands
[zohar@lnx7-oracle-1 wordpress]$ docker ps
STATUS PORTS NAMES
ce8847b28d67 wordpress:latest "docker-entrypoint..." 2 minutes ago
Up 2 minutes 0.0.0.0:8000->80/tcp wordpress_wordpress_1
daff10f9d8b9 mysql:5.7 "docker-entrypoint..." 2 minutes ago
Up 2 minutes 3306/tcp wordpress_db_1
[zohar@lnx7-oracle-1 wordpress]$ docker network ls
[...]
4f395cc553ad wordpress_default bridge local

152
Stopping the Service
• We can stop the entire service by using docker-compose stop
• We can remove the service by using the docker-compose down
command. This will delete all containers as well (but not the volumes)
[zohar@lnx7-oracle-1 wordpress]$ docker-compose stop
Stopping wordpress_wordpress_1 ... done
Stopping wordpress_db_1 ... done
[zohar@lnx7-oracle-1 wordpress]$ docker-compose down
Stopping wordpress_wordpress_1 ... done
Stopping wordpress_db_1 ... done
Removing wordpress_wordpress_1 ... done
Removing wordpress_db_1 ... done
Removing network wordpress_default

154
Docker Containers After Restart
• When the host shut down, all the containers will automatically stop
• When the host comes back up, all containers will be DOWN
• In order to automatically start the container after boot, we can use the --restart option
on the run command
$ docker run -dit --restart unless-stopped redis
Flag Description
no Do not automatically restart the container. (the default)
on-failure Restart the container if it exits due to an error, which manifests as a non-zero exit code.
unless-stopped
Restart the container unless it is explicitly stopped or Docker itself is stopped or
restarted.
always Always restart the container if it stops.

155
Limiting Container Performance
• We can limit the performance on a container:
• CPU: number of cpu (which cpus to use), number of cycles, and different
quotas
• Memory: amount of memory, amount of virtual memory
docker run --rm --cpuset-cpus=0 busybox echo hello world
docker run --rm --cpuset-cpus=0 --cpu-period=50000 busybox echo hello world

156
HEALTHCHECK
• HEALTHCHECK instruction in DockerFile
• Tells Docker how to test a container to check that it is still working
properly (not just “up”) – i.e. the database/listener are up
• New status added to container lists
• Adds “(healthy)” to Status column in a “docker ps response”
[zohar@lnx7-oracle-1 ~]$ docker ps
d2926bc4b490 portainer/portainer "/portainer" 39 minutes ago Up 39 minutes
6a204203d9e7 store/oracle/database-ente... "/bin/sh -c '/bin/..." 3 hours ago Up 3 hours (healthy)

157
Docker Secrets
• Stored encrypted in swarm mode only
• Exposed only to nodes that run services that need them
• Presented in container via RAM only tmpfs files
• never persisted to disk in encrypted format
• when container stops, secret is no longer present
• All communications between swam nodes via TLS, so secret never in
the clear on the wire either
• Different secret values per environment using tags

158
Docker Security
• Use trusted images / with a known Dockerfile
• Think (twice) about pulling images from public repos / Docker hub
• Docker images can be signed
• Docker can use TLS (client to daemon)
• Do NOT disable SELinux or set enforce off (avoid setenforce 0)
• Do not keep secrets in images
• Remember: The host can always access container

GUI, Orchestration and
Clusters
Swarm, GUI Interfaces

160
Docker GUI
• Kitematic – the default (now obsolete) interface from Docker
themselves
• Portainer - very simple, very robust GUI interface for Docker
• Shipyard – Built on Docker Swarm, Shipyard gives you the ability to
manage Docker resources including containers, images, private
registries and more
• Rancher – high end solution that interfaces nicely with cloud
solutions
• There are others – but I suggest going with Portainer (for beginners)
or Rancher or Shipyard (more advanced)

161
Portainer
• Portainer is a simple but robust GUI interface for Docker
• It runs from within a Docker container
• It’s easier to see what is going on:
• Containers
• Volumes
• Networks
• Logs
• Etc.
docker run -d --privileged -p 9000:9000 -v
/var/run/docker.sock:/var/run/docker.sock -v /opt/portainer:/data
portainer/portainer

166
Kubernetes: Container Orchestration
• Kubernetes is an open source container orchestration tool developed
by Google to allow a wider usage of Docker with more control and
better scaling and availability
• It is one of the most commonly used container orchestration tool and
has a very large community
• Kubernetes allows you to completely control your container
deployment across the cluster with custom storage and network
solutions and well defined deployment rules
• Kubernetes allows for easy integration with public (AWS, GCE, Azure)
and private (VMWare) cloud solutions

167
Kubernetes Concepts
• While it provides a useful, scalable, and powerful tool, Kubernetes also gets a lot
of clout because it’s an intricate and sometimes overly-complicated system
• Installing Kubernetes is a slightly more complex endeavor since you will need
additional tools to provide network (i.e. flannel) and store the cluster data (i.e.
etcd).
• Basic Concepts:
• Pod: The basic deployment unit, One or more containers (usually one) that share a
common function.
• ReplicaSet: Controls the replication and availability of a pod.
• Deployment: Controls the pod’s deployment by creating and managing ReplicaSets.
Allows for rolling updates.
• Service: Advertises a set of pods to other pods or to the outside world. Allows easy
discovery with the DNS add-on.
• StatefulSet: Manages the deployment and scaling of a set of Pods, and provides
guarantees about the ordering and uniqueness of these Pods.

168
Kubernetes
Source: wikipedia

169
Docker Swarm
• Native Docker cluster  same API as a single engine
• Fast provisioning, about 500 msec
• Scheduling Algo: spread, binpack, rand
• Features are optional, you can continue use
Kubernetes etc.
• There is NO insecure mode

171
Docker Swarm
• Since Docker 1.12
• Swarm is merged with Docker engine:
• Load balancer included
• Service discovery
• Cluster scheduler
• Swarm has many features like Google's Kubernetes
• Easier to get started
• Better performance

172
Swarm Services
• To deploy an application image when Docker Engine is in swarm
mode, you create a service
• Frequently a service will be the image for a microservice within the
context of some larger application
• Examples of services might include an HTTP server, a database, or
any other type of executable program that you wish to run in a
distributed environment

173
Setup Swarm and Machine
• Create Swarm ID
• Create Machine with Swarm master
• Create Machine with Swarm agent01 / 02
• Set Docker env for Swarm master
• Deploy service to the swarm
• If needed, scale the service to run more container on more nodes

174
Learn More
More information:
• https://docs.docker.com/
• Reference, information, how-to
• https://training.docker.com/
• Training, self paced and trainer
• Blog: https://blog.docker.com/
• Videos: https://www.youtube.com/user/dockerrun

175
Oracle Support
• Support for Docker Running on Oracle Linux (Doc ID 1921163.1)
• https://support.oracle.com/epmos/faces/DocumentDisplay?id=1921163.1
• Oracle Support Document 2216342.1 (Oracle Support for Database
Running on Docker):
• https://support.oracle.com/epmos/faces/DocumentDisplay?id=2216342.1
• Gerald Venzl (will be here in ilOUG Tech Days 2018 (January 2018),
speaking about Docker!)
• https://geraldonit.com/2017/08/21/creating-an-oracle-database-docker-
image/

Did You Hear About
Oracle 18c?

177
• Oracle is changing the way they number products
• Products will be release more often
• Version names will be based on last two digits of the year and a
subversion
• There is no more Release 1-Release 2 for major versions, we can’t tell
when we jumped to the next major version and when for the next minor
• Examples:
• 12.2.0.1 will be followed by 18.1 (and not 12.2.0.2)
• SQL Developer 4.2.1 was followed by 17.3
• Read more at the updated MOS Note 742060.1 – Release Schedule of
Current Database Releases
Wait! 18c?! What happened to 13c?

178
Summary
• We talked about Docker and Docker containers use cases
• We learned how to use Docker (slides will be available soon)
• We saw how Docker works with Oracle and MySQL
• We overviewed swarm mode (not for databases)
• We talked about Docker in the cloud environment

Q&A
Any Questions? Now will be the time!

Zohar Elkayam
twitter: @realmgic
Zohar@Brillix.co.il
www.ilDBA.co.il
www.realdbamagic.com

Docker Concepts for Oracle/MySQL DBAs and DevOps

Docker Concepts for Oracle/MySQL DBAs and DevOps

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Docker Concepts for Oracle/MySQL DBAs and DevOps

Ähnlich wie Docker Concepts for Oracle/MySQL DBAs and DevOps (20)

Mehr von Zohar Elkayam

Mehr von Zohar Elkayam (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Docker Concepts for Oracle/MySQL DBAs and DevOps