In this talk we will talk about the design decisions behind YADT and how it helps reduce the complexity of your datacenter infrastructrure while at the same time enabling you to fully automate almost all of it. YADT - an Augmented Deployment Tool is a next generation data center deployment and system management tool. It consists of two central components: Firstly, a central management of dependencies between services, systems and software packages, and secondly a deployment based on packages for both software and configuration management. YADT is developed at and used by ImmobilienScout24 as an Open Source project. The software makes it easy to deploy your software, configuration and operating system updates. It also provides a service-oriented abstraction layer which makes dependencies explicit and comprehensible. YADT will update packages and restart the services in the correct order within your data center. It understands the traditional dependencies between packages as well as dependencies between services, even over the network.

1. Data Center Automation
with YADT
Berlin | 23.05.2013 | Schlomo Schapiro
Systems Architect, Open Source Evangelist
License: http://creativecommons.org/licenses/by-nc-nd/3.0/

2. Slide 2 | Data Center Automation with YADT | Schlomo Schapiro
www.ImmobilienScout24.de
>2 billion PI per month
2 data center with ~1400 VM
total of ~600 employees
 ~30 crossfunctional IT teams
 ~160 in IT
15 years in business
part of Deutsche Telekom

3. Slide 3 | Data Center Automation with YADT | Schlomo Schapiro

4. Slide 4 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT

5. Slide 5 | Data Center Automation with YADT | Schlomo Schapiro
The Big Picture – Continuous Delivery Platform
Business
Decision
To Go
Live
All Humans
are on the
Same Side
Build
Config
Deploy
Test
Systems-
Management
Automation
Scale Out
Data Centers

6. Slide 6 | Data Center Automation with YADT | Schlomo Schapiro
1st task: Reduce Systemic Complexity
File Management
Systems Management

7. Slide 7 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM

8. Slide 8 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
Libraries
Applications
Configuration
Operating System
100 %

9. Slide 9 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
PROD
DEV
Infrastructure
Build
Application
Build
Configuration
Build
QA
YUM
Repos

10. Slide 10 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
Interface
PROD
DEV
Infrastructure
Build
Application
Build
Configuration
Build
QA
YUM
Repos

11. Slide 11 | Data Center Automation with YADT | Schlomo Schapiro
PROD
DEV
Infrastructure
Build
Application
Build
QA
YUM
Repos
Configuration Build – Pre-built Config Content
Configuration
Build

12. Slide 12 | Data Center Automation with YADT | Schlomo Schapiro
Configuration Build – Pre-built Config Content
SVN
commit
config file
build
config RPM
RPMs
with
static
content
YUM
Repo

13. Slide 13 | Data Center Automation with YADT | Schlomo Schapiro
Templating
Variables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commit
config file
build
config RPM
RPMs
with
static
content
YUM
Repo

14. Slide 14 | Data Center Automation with YADT | Schlomo Schapiro
we use
1 RPM
per host
Templating
Variables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commit
config file
build
config RPM
RPMs
with
static
content
YUM
Repo

15. Slide 15 | Data Center Automation with YADT | Schlomo Schapiro
Shared
responsability
Ops, Dev, QA ...
we use
1 RPM
per host
Templating
Variables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commit
config file
build
config RPM
RPMs
with
static
content
YUM
Repo

16. Slide 16 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Continuous Release Delivery
PROD
DEV
Infrastructure
Build
QA
YUM
Repos
Configuration
Build
Application
Build

17. Slide 17 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Package Promotion Pipeline
YUM
Repo
Tested
Repo
YUM
Repo
Released
Repo
YUM
Repo
Comitted
Repo
DEV Pre-PROD PROD

18. Slide 18 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Package Promotion Pipeline
YUM
Repo
Tested
Repo
YUM
Repo
Released
Repo
YUM
Repo
Comitted
Repo
DEV Pre-PROD PROD
CI
Deploy
&
Test
Auto-
promote
release if
tests OK
Deploy
&
Test
Business
decision
to Go Live
with
tested
versions
Deploy
&
Test

19. Slide 19 | Data Center Automation with YADT | Schlomo Schapiro
File Management through many YUM repositories
PROD
DEV
Infrastructure
Build
Application
Build
Configuration
Build
QA
YUM
Repo
YUM
RepoYUM
RepoYUM
Repo

20. Slide 20 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT

21. Slide 21 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Unified Dependency Tree
Web Server
HTTPD
Tomcat
App Server
Load Balancer

22. Slide 22 | Data Center Automation with YADT | Schlomo Schapiro
HTTPD
Tomcat
web-
app
httpd
config
tomcat
YADT – Unified Dependency Tree
init
script
init
script

23. Slide 23 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
Web Server
HTTPD
Tomcat
web-
app
httpd
config
tomcat
App Server
Java App
app
config

24. Slide 24 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcat
web-
app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcat
web-
app
httpd
config
tomcat
App Server
Java App
app
config

25. Slide 25 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcat
web-
app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcat
web-
app
httpd
config
tomcat
App Server
Java App
app
config
1st
chunk

26. Slide 26 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcat
web-
app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcat
web-
app
httpd
config
tomcat
App Server
Java App
app
config
1st
chunk
2nd
chunk

27. Slide 27 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT
Reduce Systemic Complexity
Keep It Simple

28. Slide 28 | Data Center Automation with YADT | Schlomo Schapiro
Automation is our Continuous Delivery Platform
Business
Decision
To Go
Live
All Humans
are on the
Same Side
Build
Config
Deploy
Test
Systems-
Management
Automation
Scale Out
Data Centers

29. Slide 29 | Data Center Automation with YADT | Schlomo Schapiro
DevOps
We build it together
as one team.
We understand
it completely.
And it is fully automated.

30. Slide 30 | Data Center Automation with YADT | Schlomo Schapiro
www.yadt-project.org
Stand 7.1C145

31. Slide 31 | Data Center Automation with YADT | Schlomo Schapiro
go.schapiro.org/slides

32. Slide 32 | Data Center Automation with YADT | Schlomo Schapiro
Kontakt:
Immobilien Scout GmbH
Andreasstraße 10
10243 Berlin
Fon: +49 30 243 01-1229
Email: schlomo.schapiro@immobilienscout24.de
URL: www.immobilienscout24.de
Thank you very much!
Please contact me for further
questions and discussions.
All images are either public domain, used in appropriate
context or taken from openclipart.org