batou - multi(component|host|environment|.*) deployment

BATOU
multi-(component¦host¦environment¦.*)
deployment
Wednesday, 3.July 13

AUTOMATING
DEPLOYMENTS IS
HARD

HOW DOES
CONVERGENCE HELP?

HOW DOES THIS
WORK WITH BATOU?

SOME PERSPECTIVE

IT'S NOT THAT BAD.

service
deployment
Fabric,
Capistrano, ...
system
conﬁguration
Puppet, Chef, ...
provisioning
kickstart, Razor,
imaging ...

FTP
bash
mkzopeinstance
zc.buildout
fabric

CONVERGENCE

"Everything that follows is a
result of what you see here."
(Dr. Alfred Lanning; I, Robot)

SIMPLE
os.mkdir('foo')
with open('foo/bar', 'w') as myfile:
myfile.write('asdf')
os.chmod('foo/bar', 0755)

•unexpected system state
•can't resume
•unnecessary updates
os.mkdir('foo')
with open('foo/bar', 'w') as myfile:
myfile.write('asdf')
os.chmod('foo/bar', 0755)
SIMPLISTIC

CORRECT(?)
if not os.path.isdir('foo'): os.unlink('foo')
if not os.path.exists('foo'): os.mkdir('foo')
try:
os.lstat('foo/bar')
except OSError: pass
else:
if os.path.isdir('foo/bar'):
shutil.rmtree('foo/bar')
else:
os.unlink('foo/bar')
if (os.path.exists('foo/bar') and
open('foo/bar', 'r').read() != 'asdf'):
open('foo/bar', 'w').write('asdf'):
current = os.stat('foo/bar').st_mode
if stat.S_IMODE(current) != 0755:
os.chmod('foo', 0755)

SIMPLE
File('foo/bar',
content='asdf',
mode=0755,
leading=True)

class File(Component):
namevar = 'path'
def configure(self):
self += Presence(
self.path, leading=self.leading)
self += Mode(self.path, self.mode)
self += Content(self.path, self.content)
File('foo/bar',
content='asdf',
mode=0755,
leading=True)

namevar = 'path'
self += Presence(
File('foo/bar',
content='asdf',
mode=0755,
leading=True)
compute target state
(no touching!)

namevar = 'path'
self += Presence(
File('foo/bar',
content='asdf',
mode=0755,
leading=True)
(no touching!)
composition operator

namevar = 'path'
self += Presence(
File('foo/bar',
content='asdf',
mode=0755,
leading=True)
(no touching!)
composition operator
order
matters

class Presence(Component):
namevar = 'path'
leading = False
if self.leading:
self += Directory(
os.path.dirname(self.path),
leading=self.leading)
def verify(self):
if not os.path.isfile(self.path):
raise batou.UpdateNeeded()
def update(self):
ensure_path_nonexistent(self.path)
with open(self.path, 'w'):
pass

namevar = 'path'
leading = False
if self.leading:
self += Directory(
def verify(self):
def update(self):
pass
run "anywhere"

namevar = 'path'
leading = False
if self.leading:
self += Directory(
def verify(self):
def update(self):
pass
run on target
run "anywhere"

namevar = 'path'
leading = False
if self.leading:
self += Directory(
def verify(self):
def update(self):
pass
run on target
run "anywhere"
after all sub-
components

namevar = 'path'
leading = False
if self.leading:
self += Directory(
def verify(self):
def update(self):
pass
run on target
only if needed
run "anywhere"
after all sub-
components

namevar = 'path'
leading = False
if self.leading:
self += Directory(
def verify(self):
def update(self):
pass
run on target
only if needed
run "anywhere"
after all sub-
components
keep
delegating!

class Directory(Component):
namevar = 'path'
leading = False
def verify(self):
if not os.path.isdir(self.path):
def update(self):
if self.leading:
os.makedirs(self.path)
else:
os.mkdir(self.path)

namevar = 'path'
leading = False
def verify(self):
def update(self):
if self.leading:
else:
os.mkdir(self.path)
could be done with
recursive
composition

namevar = 'path'
leading = False
def verify(self):
def update(self):
if self.leading:
else:
os.mkdir(self.path)
could be done with
recursive
composition
refactor with
sub-components if too
complex

namevar = 'path'
leading = False
def verify(self):
def update(self):
if self.leading:
else:
os.mkdir(self.path)
all methods
optional: no
configure()
could be done with
recursive
composition
refactor with
complex

namevar = 'path'
leading = False
def verify(self):
def update(self):
if self.leading:
else:
os.mkdir(self.path)
all methods
optional: no
configure()
could be done with
recursive
composition
pattern: just wipe
out what's wrong
refactor with
complex

CONVERGENCE
resume where
needed
handle many
system states
transparently
avoid
unnecessary
updates

COMPONENTS
composition of
simple components
no magic bullet, just a
lot easier to factor
your code
conﬁgure - verify - update

SINGLE-
COMMAND

REPEATABLE
RELIABLE

EXPRESSIVENESS
READABILITY

PLATFORM
INDEPENDENCE

DOMAIN
AGNOSTIC

NO ADDITIONAL
RUNTIME
DEPENDENCIES

CONTINUITY

MINIMAL
DOWNTIMES

PRACTICAL USAGE

REQUIREMENTS
Python 2.7
SSH
virtualenv
Mercurial

ENVIRONMENTS
[environment]
service_user = myservice
host_domain = flyingcircus.io
branch = production
[hosts]
multikarl00 = nginx, haproxy
multikarl01 = postgres, redis, memcached, crontab
multikarl12 = supervisor, logrotate, doctotext, myapp
multikarl13 = supervisor, logrotate, doctotext, myapp

LOCAL
$ bin/batou-local dev localhost
Updating Hello > File(hello) > Presence(hello)
Updating Hello > File(hello) > Content(hello)
$ bin/batou-local dev localhost
$

REMOTE
$ bin/batou-remote prod
test02.gocept.net: connecting
test01.gocept.net: connecting
test01.gocept.net: bootstrapping
test02.gocept.net: bootstrapping
OK
OK
Deploying test01.gocept.net/hello
OK
Deploying test02.gocept.net/hello
OK

OVERRIDES
class Hello(Component):
hostname = "foo"
[environment]
...
[component:hello]
hostname = bar

SECRETS
db_password = none
secrets/production.cfg
[hello]
db_password = reallysecretstuff

SECRETS
db_password = none
secrets/production.cfg
[hello]
db_password = reallysecretstuff
SciFi
but
close

PROVIDE/REQUIRE
class MyApp(Component):
self.provide('appserver',
self.host.fqdn)
class HAProxy(Component):
self.backends =
self.require('appserver')

PLATFORMS
...
@platform('flyingcircus.io', HAProxy)
class SystemWideHAProxy(Component):
self += File('/etc/haproxy',
ensure='symlink',
link_to=self.parent.haproxy_cfg.path)

VFS MAPPING
./
...
./work
./work/_/etc/haproxy.cfg
self += File('/etc/haproxy')
[environment]
...
[vfs]
sandbox = Developer

FEATURES
class MyApp(Component):
features = ['instance', 'jobrunner']
if 'instance' in self.features:
...
[hosts]
hosta = myapp:instance
hostb = myapp:jobrunner
hostc = myapp:instance, myapp:jobrunner
hostd = myapp

CONVERGENCE
COMPOSITION
DETAILS

QUESTIONS?

batou.readthedocs.org
pypi.python.org/pypi/batou
bitbucket.org/gocept/batou

batou - multi(component|host|environment|.*) deployment

Recommended

Recommended

More Related Content

Similar to batou - multi(component|host|environment|.*) deployment

Similar to batou - multi(component|host|environment|.*) deployment (20)

Recently uploaded

Recently uploaded (20)

batou - multi(component|host|environment|.*) deployment