1. Building Applications
for Operability
Reinhardt Quelle
Operations Architect
rquelle@cisco.com
rquelle@*
V16e
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)
1

2. scr
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

3. Great stuff, but not that’s
not what my team does…
© 2011 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)
3

4. We host and manage SaaS
Applications
Cisco Social
Jabber
Millions of Meetings
for
10s of Millions of
Users
totaling
Billions of Minutes
each month
7x24x365
WebEx
© 2011-2012 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)
4

5. DevOps
Damon Edwards (DTO)
John Willis (now at Dell) suggest
“CAMS” as useful mnemonic.
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

6. DevOps > Tools
•Automation
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

7. DevOps > Tools
•Automation
•Metrics
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

8. DevOps > Tools
•Automation
•Metrics
•Sharing
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

9. DevOps > Tools
•Culture
•Automation
•Metrics
•Sharing
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

10. DevOps > Tools
•Culture / Collaboration
•Automation
•Metrics
•Sharing
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

11. Collaboration
Requires:
• shared goals
• communication
• empathy
•
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

12. Yay, We’re going to do
DevOps!
http://www.realclearworld.com/blog/2009/01/
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

13. When you start, its
more like this:
http://www.news.com.au/world/are-barack-obama-and-vladamir-putin-holding-highlevel-talks-or-are-they-on-an-awkward-date/storyfndir2ev-1226665803294
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

14. At a more personal level:
remember that first date?
http://www.hercampus.com/love/dating-hooking/9-first-date-disasters-how-deal-them
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

15. Where do we begin?
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

16. Perspective helps (a lot)
http://www.navigonlp.com/blog/stop-look-look-again/boat-land-perspective-2/
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

17. I come from an Operations
background
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

18. Green = Good
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

19. Illustrating the Ops
perspective to Engineering
http://cdn.twentytwowords.com/wp-content/uploads/Rhinos-Perspective-e1369734532511-634x447.jpg
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

20. A quick illustration:
A real, albeit deceased, product
(RIP)
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

21. Developer’s View
Compute Node
resolv.conf
ntp.conf
smb.conf
...
App
© 2013 Cisco and/or its affiliates. All rights reserved.
config A
config B
Reinhardt Quelle (rquelle@cisco.com)

22. Operations’ View
Compute Node
/etc/passwd
inetd
config
/etc/sysconfig/*
/etc/logrotate/*
sshd
config
rsyslog
config
LVM
...
/etc/snmp/*
/etc/*
....
resolv.conf
DR:BD
ntp.conf
...
smb.conf
...
App
© 2013 Cisco and/or its affiliates. All rights reserved.
config A
config B
Reinhardt Quelle (rquelle@cisco.com)

23. Servers run in "clusters"
Compute Node
Compute Node
Compute Node
inetd
config
inetd
config
config
sshd
config
rsyslog
config
LVM
/etc/snmp/*
config
LVM
...
...
ntp.conf
...
DR:BD
...
© 2013 Cisco and/or its affiliates. All rights reserved.
config
...
App
start/stop
orchestration
config
ntp.conf
....
sshd
config
rsyslog
config
...
ntp.conf
...
....
DR:BD
...
ntp.conf
App
config
config
...
/etc/snmp/*
/etc/*
....
resolv.conf
DR:BD
...
ntp.conf
smb.conf
...
App
config B
Reinhardt Quelle (rquelle@cisco.com)
sshd
rsyslog
smb.conf
config A
start/stop
orchestration
/etc/sysconfig/*
/etc/logrotate/*
LVM
/etc/snmp/*
...
config B
config
resolv.conf
smb.conf
config A
/etc/passwd
inetd
/etc/*
resolv.conf
DR:BD
/etc/sysconfig/*
/etc/logrotate/*
LVM
/etc/snmp/*
...
config B
start/stop
orchestration
config
LVM
...
App
sshd
smb.conf
config A
/etc/passwd
inetd
/etc/logrotate/*
resolv.conf
smb.conf
/etc/sysconfig/*
/etc/*
....
....
resolv.conf
DR:BD
config
rsyslog
/etc/snmp/*
Compute Node
/etc/passwd
inetd
/etc/*
/etc/*
rsyslog
/etc/sysconfig/*
/etc/logrotate/*
/etc/logrotate/*
sshd
Compute Node
/etc/passwd
/etc/passwd
/etc/sysconfig/*
start/stop
orchestration
...
config A
App
config B
start/stop
orchestration
config A
config B

24. Which means they may have to
share data:
Compute Node
Compute Node
Compute Node
inetd
sshd
/etc/sysconfig/*
config
config
inetd
config
sshd
config
LVM
rsyslog
...
...
ntp.conf
...
DR:BD
...
config
ntp.conf
/etc/sysconfig/*
/etc/snmp/*
config A
App
start/stop
orchestration
inetd
sshd
/etc/sysconfig/*
config
config
rsyslog
config
...
...
ntp.conf
rsyslog
config
...
....
DR:BD
ntp.conf
...
App
...
/etc/snmp/*
/etc/*
....
resolv.conf
DR:BD
ntp.conf
...
smb.conf
...
config A
App
config B
start/stop
orchestration
config
smb.conf
...
config B
rsyslog
/etc/logrotate/*
resolv.conf
smb.conf
config A
config
LVM
/etc/snmp/*
resolv.conf
DR:BD
sshd
/etc/sysconfig/*
config
/etc/*
LVM
....
/etc/passwd
inetd
/etc/logrotate/*
/etc/*
...
config B
/etc/passwd
/etc/logrotate/*
smb.conf
...
start/stop
orchestration
sshd
resolv.conf
smb.conf
App
config
LVM
....
....
resolv.conf
DR:BD
/etc/snmp/*
config
LVM
config
inetd
/etc/*
/etc/*
rsyslog
/etc/sysconfig/*
Compute Node
/etc/passwd
/etc/logrotate/*
/etc/logrotate/*
/etc/snmp/*
Compute Node
/etc/passwd
/etc/passwd
...
config A
App
config B
start/stop
orchestration
config A
config B
start/stop
orchestration
spare
SAN
LUN
© 2013 Cisco and/or its affiliates. All rights reserved.
LUN
LUN
Reinhardt Quelle (rquelle@cisco.com)
LUN

25. Which means they may have to
share data:
Compute Node
Compute Node
Compute Node
inetd
sshd
/etc/sysconfig/*
config
config
inetd
config
sshd
config
LVM
rsyslog
...
...
ntp.conf
...
DR:BD
...
config
ntp.conf
/etc/sysconfig/*
/etc/snmp/*
config A
App
start/stop
orchestration
inetd
sshd
/etc/sysconfig/*
config
config
rsyslog
config
...
...
ntp.conf
rsyslog
config
...
....
DR:BD
ntp.conf
...
App
...
/etc/snmp/*
/etc/*
....
resolv.conf
DR:BD
ntp.conf
...
smb.conf
...
config A
App
config B
start/stop
orchestration
config
smb.conf
...
config B
rsyslog
/etc/logrotate/*
resolv.conf
smb.conf
config A
config
LVM
/etc/snmp/*
resolv.conf
DR:BD
sshd
/etc/sysconfig/*
config
/etc/*
LVM
....
/etc/passwd
inetd
/etc/logrotate/*
/etc/*
...
config B
/etc/passwd
/etc/logrotate/*
smb.conf
...
start/stop
orchestration
sshd
resolv.conf
smb.conf
App
config
LVM
....
....
resolv.conf
DR:BD
/etc/snmp/*
config
LVM
config
inetd
/etc/*
/etc/*
rsyslog
/etc/sysconfig/*
Compute Node
/etc/passwd
/etc/logrotate/*
/etc/logrotate/*
/etc/snmp/*
Compute Node
/etc/passwd
/etc/passwd
...
config A
App
config B
start/stop
orchestration
config A
config B
start/stop
orchestration
spare
SAN
LUN
LUN
LUN
LUN
Note: there are *much* better ways to share data!
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

26. We have several front-end
servers
Webmail
RPC
Front-End Server
ADM
Front-End Server
Front-End Server
/etc/passwd
inetd
/etc/passwd
/etc/sysconfig/*
config
inetd
/etc/logrotate/*
sshd
config
rsyslog
config
LVM
...
sshd
config
rsyslog
config
LVM
...
config A
Service X
config B
Postpath Unified Node (PUN)
/etc/passwd
/etc/sysconfig/*
rsyslog
config
LVM
...
....
sshd
config
rsyslog
config
...
ntp.conf
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
config
rsyslog
config
...
ntp.conf
...
sshd
ntp.conf
...
rsyslog
config
...
/etc/snmp/*
....
config
rsyslog
/etc/sysconfig/*
config
sshd
config
...
/etc/logrotate/*
/etc/snmp/*
/etc/*
....
resolv.conf
DR:BD
ntp.conf
...
smb.conf
smb.conf
...
...
resolv.conf
DR:BD
ntp.conf
...
smb.conf
...
...
config A
PPSD
config B
cluster
manager
/etc/passwd
inetd
/etc/*
smb.conf
PPSD
config A
config B
LVM
config
resolv.conf
DR:BD
Service Y
/etc/logrotate/*
/etc/*
....
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
LVM
/etc/snmp/*
config A
/etc/passwd
inetd
/etc/logrotate/*
resolv.conf
DR:BD
Service X
config B
/etc/*
....
smb.conf
/etc/passwd
inetd
LVM
/etc/snmp/*
resolv.conf
...
ntp.conf
...
Service Y
config B
/etc/logrotate/*
/etc/*
DR:BD
....
resolv.conf
config A
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
LVM
/etc/snmp/*
config A
/etc/passwd
inetd
/etc/logrotate/*
config
...
...
Postpath Unified Node (PUN)
sshd
DR:BD
smb.conf
Service Y
config
...
/etc/snmp/*
/etc/*
ntp.conf
...
/etc/sysconfig/*
config
resolv.conf
ntp.conf
smb.conf
inetd
config
LVM
....
resolv.conf
config B
sshd
rsyslog
/etc/snmp/*
/etc/*
....
config A
config
/etc/logrotate/*
...
DR:BD
/etc/snmp/*
/etc/*
Service X
inetd
/etc/logrotate/*
...
DR:BD
/etc/passwd
/etc/sysconfig/*
config
config A
PPSD
config B
cluster
manager
config A
PPSD
config B
cluster
manager
smb.conf
...
config A
PPSD
config B
cluster
manager
config A
config B
cluster
manager
spare
SAN
LUN
© 2013 Cisco and/or its affiliates. All rights reserved.
LUN
LUN
Reinhardt Quelle (rquelle@cisco.com)
LUN

27. ...which also have to be
redundant.
Webmail
RPC
Front-End Server
ADM
Front-End Server
Front-End Server
/etc/passwd
inetd
sshd
/etc/passwd
/etc/sysconfig/*
config
config
inetd
sshd
config
/etc/logrotate/*
config
config
resolv.conf
/etc/sysconfig/*
...
inetd
...
sshd ntp.conf
config
smb.conf
rsyslog
Service X
Service Y
DR:BD
config B
config
resolv.conf
...
inetd
...
sshd ntp.conf
config
smb.conf
/etc/*
rsyslog
config B
...
ntp.conf
Service X
config A
/etc/snmp/*
config B
Postpath Unified Node (PUN)
Service Y
sshd
config
rsyslog
config
LVM
...
Service X
config B
sshd
config
rsyslog
config
LVM
/etc/sysconfig/*
config
ntp.conf
...
ntp.conf
...
sshd
config
/etc/snmp/*
config A
config B
/etc/*
config
...
Service Y
DR:BD
....
...
config A
resolv.conf
config B
...
ntp.conf
smb.conf
....
DR:BD
ntp.conf
...
rsyslog
...
config B
config
rsyslog
ntp.conf
...
config
...
config B
...
/etc/logrotate/*
....
/etc/snmp/*
/etc/*
....
resolv.conf
ntp.conf
...
resolv.conf
DR:BD
ntp.conf
...
smb.conf
smb.conf
...
config A
PPSD
config B
cluster
manager
config
LVM
/etc/snmp/*
...
PPSD
config
rsyslog
/etc/sysconfig/*
config
sshd
/etc/*
DR:BD
...
config A
PPSD
config B
cluster
manager
config A
config B
cluster
manager
spare
SAN
LUN
© 2013 Cisco and/or its affiliates. All rights reserved.
LUN
LUN
Reinhardt Quelle (rquelle@cisco.com)
LUN
config A
config B
/etc/passwd
inetd
/etc/logrotate/*
smb.conf
config A
cluster
manager
sshd
resolv.conf
DR:BD
Service Y
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/*
....
config A
/etc/passwd
inetd
LVM
/etc/snmp/*
...
PPSD
...
Service X
config B
Postpath Unified Node (PUN)
config
smb.conf
...
Service Y
config A
/etc/logrotate/*
resolv.conf
config A
cluster
manager
config B
/etc/*
smb.conf
PPSD
config A
/etc/sysconfig/*
config
LVM
/etc/snmp/*
resolv.conf
DR:BD
Service X
/etc/passwd
inetd
/etc/logrotate/*
/etc/*
....
/etc/logrotate/*
smb.conf
rsyslog
config B
...
Postpath Unified Node (PUN)
/etc/passwd
inetd
/etc/logrotate/*
/etc/snmp/*
/etc/passwd
/etc/sysconfig/*
...
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
....
config
resolv.conf
smb.conf
config A
/etc/passwd
inetd
sshd ntp.conf
config
LVM
...
Service X
inetd
...
....
resolv.conf
smb.conf
config A
...
/etc/*
...
config A
Service Y
DR:BD
config B
LVM
/etc/logrotate/*
config
...
LVM
resolv.conf
config
DR:BD
/etc/sysconfig/*
LVM
....
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
DR:BD
/etc/snmp/*
...
config A
/etc/sysconfig/*
config
/etc/logrotate/*
....
/etc/logrotate/*
config
...
LVM
config A
config
/etc/passwd
LVM
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
....
DR:BD
inetd
/etc/logrotate/*
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
/etc/sysconfig/*
config

28. And that means load balancers
to direct traffic
Webmail
RPC
Front-End Server
ADM
Front-End Server
Front-End Server
/etc/passwd
inetd
/etc/passwd
/etc/sysconfig/*
config
inetd
/etc/logrotate/*
sshd
config
sshd
/etc/snmp/*
config
config
resolv.conf
/etc/sysconfig/*
LVM
...
inetd
...
sshd ntp.conf
config
smb.conf
config A
Service Y
DR:BD
config B
sshd
/etc/snmp/*
config
config
resolv.conf
...
inetd
...
sshd ntp.conf
config
smb.conf
/etc/*
rsyslog
resolv.conf
config B
...
ntp.conf
Service X
config A
/etc/snmp/*
config B
Load
Balancer
Postpath Unified Node (PUN)
Service Y
config A
Service X
config B
config
rsyslog
config
LVM
...
/etc/sysconfig/*
config
sshd
config
rsyslog
config
...
resolv.conf
ntp.conf
ntp.conf
...
config B
Service X
....
sshd
config
rsyslog
ntp.conf
...
Service X
config B
....
resolv.conf
config B
...
ntp.conf
...
sshd
rsyslog
ntp.conf
...
config
...
config
rsyslog
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
...
/etc/logrotate/*
/etc/snmp/*
/etc/*
....
/etc/*
....
resolv.conf
DR:BD
ntp.conf
...
smb.conf
smb.conf
...
resolv.conf
DR:BD
ntp.conf
...
smb.conf
...
...
config A
PPSD
config B
config A
PPSD
config B
cluster
manager
config A
PPSD
config B
cluster
manager
smb.conf
...
config A
PPSD
config B
cluster
manager
config A
config B
cluster
manager
spare
SAN
LUN
© 2013 Cisco and/or its affiliates. All rights reserved.
LUN
LUN
Reinhardt Quelle (rquelle@cisco.com)
LUN
config A
config B
/etc/passwd
inetd
LVM
config
resolv.conf
DR:BD
config B
/etc/logrotate/*
/etc/*
....
Service Y
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
LVM
/etc/snmp/*
config A
/etc/passwd
inetd
...
cluster
manager
Service Y
DR:BD
...
config A
Postpath Unified Node (PUN)
config
resolv.conf
DR:BD
Service Y
smb.conf
PPSD
config A
config B
/etc/*
...
config A
smb.conf
/etc/logrotate/*
/etc/*
resolv.conf
DR:BD
config A
/etc/sysconfig/*
config
LVM
/etc/snmp/*
/etc/*
....
/etc/snmp/*
config
...
LVM
/etc/passwd
inetd
/etc/logrotate/*
LVM
/etc/snmp/*
/etc/logrotate/*
smb.conf
config B
...
Postpath Unified Node (PUN)
/etc/passwd
inetd
/etc/logrotate/*
sshd
/etc/passwd
/etc/sysconfig/*
...
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
....
smb.conf
/etc/passwd
inetd
sshd ntp.conf
config
config
resolv.conf
rsyslog
...
config A
inetd
...
....
smb.conf
Service X
...
/etc/*
...
config A
Service Y
DR:BD
config B
LVM
/etc/logrotate/*
config
...
LVM
...
config A
/etc/snmp/*
config
DR:BD
/etc/sysconfig/*
LVM
....
config
Front-End Server
/etc/*
/etc/passwd
DR:BD
/etc/snmp/*
/etc/sysconfig/*
rsyslog
....
/etc/logrotate/*
config
...
LVM
Service X
config
/etc/passwd
DR:BD
config
/etc/logrotate/*
Front-End Server
/etc/*
rsyslog
....
rsyslog
inetd
/etc/logrotate/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
/etc/sysconfig/*
config

29. There are multiple clusters
Webmail
RPC
Front-End Server
ADM
Front-End Server
Front-End Server
/etc/passwd
inetd
/etc/passwd
/etc/sysconfig/*
config
sshd
config
inetd
config
config
LVM
config A
Service X
/etc/logrotate/*
sshd ntp.conf
config
...
config A
Service X
Postpath Unified Node (PUN)
config A
Postpath Unified Node (PUN)
Postpath Unified Node (PUN)
sshd
config
rsyslog
/etc/*
....
resolv.conf
config B
...
ntp.conf
Postpath Unified Node (PUN)
config
ntp.conf
PPSD
config B
DR:BD
cluster
manager
DR:BD
...
/etc/snmp/*
/etc/*
....
resolv.conf
ntp.conf
...
ntp.conf
smb.conf
...
smb.conf
...
config A
PPSD
...
....
smb.conf
config B
/etc/logrotate/*
config
config
LVM
/etc/*
resolv.conf
ntp.conf
...
DR:BD
...
/etc/sysconfig/*
config
sshd
rsyslog
/etc/snmp/*
....
config A
/etc/passwd
inetd
...
/etc/*
smb.conf
...
config
config
LVM
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
rsyslog
/etc/snmp/*
resolv.conf
ntp.conf
...
DR:BD
smb.conf
config B
/etc/logrotate/*
...
....
config A
Service Y
/etc/passwd
inetd
config
LVM
/etc/*
resolv.conf
config A
...
config A
Service X
config B
/etc/sysconfig/*
config
sshd
rsyslog
/etc/snmp/*
....
cluster
manager
/etc/snmp/*
...
config A
Service Y
DR:BD
config B
config A
/etc/logrotate/*
...
/etc/*
resolv.conf
PPSD
inetd
config
LVM
/etc/snmp/*
...
/etc/logrotate/*
config
...
smb.conf
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
...
DR:BD
config A
Service X
ntp.conf
Service Y
config B
/etc/passwd
inetd
/etc/logrotate/*
config
config
LVM
/etc/passwd
/etc/sysconfig/*
smb.conf
resolv.conf
config B
...
config A
Service X
config B
/etc/sysconfig/*
config
sshd
....
config
resolv.conf
LVM
...
/etc/passwd
inetd
rsyslog
sshd ntp.conf
config
smb.conf
Service Y
config B
...
rsyslog
...
config A
DR:BD
....
...
config A
Service Y
DR:BD
config B
ntp.conf
inetd
/etc/*
config
...
LVM
resolv.conf
...
/etc/snmp/*
smb.conf
rsyslog
config B
...
/etc/snmp/*
config
/etc/logrotate/*
smb.conf
Service X
config
Front-End Server
/etc/*
LVM
/etc/sysconfig/*
....
...
config A
Service Y
DR:BD
config B
/etc/sysconfig/*
config
sshd
rsyslog
config
resolv.conf
inetd
/etc/passwd
/etc/*
config
...
LVM
...
DR:BD
/etc/snmp/*
smb.conf
rsyslog
LVM
....
/etc/logrotate/*
sshd ntp.conf
config
...
DR:BD
/etc/sysconfig/*
config
/etc/passwd
config
resolv.conf
inetd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
....
...
inetd
/etc/logrotate/*
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/logrotate/*
PPSD
config B
cluster
manager
...
config A
config A
PPSD
config B
cluster
manager
config B
cluster
manager
spare
SAN
LUN
LUN
LUN
Webmail
RPC
Front-End Server
ADM
Front-End Server
Front-End Server
/etc/passwd
inetd
/etc/passwd
/etc/sysconfig/*
config
inetd
config
config
DR:BD
rsyslog
...
....
inetd
DR:BD
...
/etc/snmp/*
Service Y
DR:BD
ntp.conf
config A
Service X
resolv.conf
ntp.conf
config A
Postpath Unified Node (PUN)
config A
Service Y
config
inetd
Postpath Unified Node (PUN)
config
config
inetd
....
ntp.conf
Postpath Unified Node (PUN)
config
config
config
...
inetd
/etc/sysconfig/*
config
sshd
config
rsyslog
config
/etc/logrotate/*
/etc/logrotate/*
/etc/snmp/*
/etc/snmp/*
/etc/*
/etc/*
....
...
LVM
...
....
...
DR:BD
LVM
resolv.conf
ntp.conf
/etc/passwd
/etc/sysconfig/*
config
sshd
....
LVM
DR:BD
resolv.conf
config B
Postpath Unified Node (PUN)
/etc/*
...
...
inetd
rsyslog
/etc/snmp/*
....
LVM
DR:BD
config A
Service Y
config B
/etc/passwd
config
/etc/*
....
ntp.conf
config A
Service X
/etc/logrotate/*
/etc/snmp/*
/etc/*
resolv.conf
...
DR:BD
...
resolv.conf
ntp.conf
resolv.conf
ntp.conf
smb.conf
smb.conf
smb.conf
...
...
ntp.conf
smb.conf
...
PPSD
/etc/*
resolv.conf
...
config A
config B
/etc/sysconfig/*
config
sshd
rsyslog
/etc/logrotate/*
/etc/snmp/*
...
/etc/snmp/*
...
config A
config B
...
smb.conf
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
/etc/logrotate/*
...
Service Y
config B
/etc/passwd
config
LVM
config A
Service X
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
DR:BD
/etc/logrotate/*
config
...
Service Y
DR:BD
config B
...
/etc/passwd
inetd
rsyslog
config A
Service X
smb.conf
...
config B
/etc/passwd
/etc/sysconfig/*
smb.conf
LVM
...
config A
smb.conf
Service X
....
config
resolv.conf
sshd ntp.conf
config
rsyslog
....
config B
...
Service Y
DR:BD
config B
...
inetd
/etc/*
config
...
LVM
resolv.conf
DR:BD
/etc/snmp/*
smb.conf
...
config A
...
/etc/logrotate/*
sshd ntp.conf
config
rsyslog
....
config B
...
/etc/snmp/*
config
LVM
/etc/passwd
/etc/sysconfig/*
/etc/*
config
...
LVM
config A
/etc/logrotate/*
config
rsyslog
....
config
resolv.conf
inetd
/etc/logrotate/*
sshd ntp.conf
config
config B
...
/etc/sysconfig/*
smb.conf
Service X
LVM
Front-End Server
/etc/*
config
/etc/passwd
config
resolv.conf
rsyslog
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
config
...
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
Front-End Server
/etc/*
LVM
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
rsyslog
Load
Balancer
LUN
...
config A
PPSD
config B
cluster
manager
config A
config A
PPSD
config B
cluster
manager
PPSD
config B
cluster
manager
smb.conf
...
config A
config A
PPSD
config B
cluster
manager
config B
cluster
manager
spare
SAN
LUN
LUN
LUN
Webmail
RPC
Front-End Server
ADM
Front-End Server
Front-End Server
/etc/passwd
inetd
/etc/passwd
/etc/sysconfig/*
config
inetd
config
config
LVM
...
DR:BD
...
inetd
rsyslog
config
resolv.conf
smb.conf
Service X
Service Y
DR:BD
config B
config
LVM
...
DR:BD
/etc/snmp/*
...
...
config A
config B
...
inetd
config
resolv.conf
sshd ntp.conf
config
smb.conf
rsyslog
LVM
Service X
ntp.conf
...
config A
Service Y
DR:BD
config B
config A
Postpath Unified Node (PUN)
config B
...
config
config
Service Y
config A
Service X
config B
/etc/snmp/*
/etc/sysconfig/*
config
sshd
config
config
/etc/snmp/*
Service X
ntp.conf
config
...
DR:BD
...
/etc/snmp/*
config B
/etc/*
config
...
Service Y
DR:BD
....
...
config A
resolv.conf
config B
...
ntp.conf
...
config A
Service X
config B
/etc/snmp/*
config
config
/etc/snmp/*
/etc/sysconfig/*
config
sshd
config
config
/etc/logrotate/*
/etc/snmp/*
/etc/*
/etc/*
....
LVM
...
DR:BD
...
....
LVM
...
DR:BD
...
resolv.conf
ntp.conf
smb.conf
smb.conf
...
...
resolv.conf
ntp.conf
smb.conf
...
...
config A
PPSD
config B
config A
PPSD
config B
cluster
manager
config A
PPSD
config B
cluster
manager
smb.conf
...
config A
PPSD
config B
cluster
manager
config A
config B
cluster
manager
spare
SAN
LUN
© 2013 Cisco and/or its affiliates. All rights reserved.
LUN
LUN
config A
config B
/etc/passwd
inetd
rsyslog
/etc/logrotate/*
....
resolv.conf
ntp.conf
Service Y
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
/etc/*
...
...
config A
config B
/etc/passwd
inetd
rsyslog
/etc/logrotate/*
LVM
DR:BD
resolv.conf
ntp.conf
Service Y
Postpath Unified Node (PUN)
config
....
LVM
cluster
manager
config A
smb.conf
smb.conf
PPSD
/etc/logrotate/*
smb.conf
LVM
resolv.conf
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/*
....
resolv.conf
ntp.conf
inetd
rsyslog
/etc/logrotate/*
/etc/*
...
...
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
rsyslog
/etc/logrotate/*
LVM
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
...
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
DR:BD
....
inetd
rsyslog
....
smb.conf
/etc/passwd
inetd
rsyslog
...
/etc/*
...
config B
DR:BD
/etc/snmp/*
smb.conf
Service X
...
/etc/logrotate/*
config
...
....
config A
/etc/snmp/*
config
LVM
/etc/passwd
/etc/sysconfig/*
/etc/*
resolv.conf
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
....
/etc/logrotate/*
config
...
LVM
config
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/passwd
/etc/sysconfig/*
sshd ntp.conf
config
config A
config
Front-End Server
/etc/*
....
rsyslog
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
LUN
LUN
Reinhardt Quelle (rquelle@cisco.com)

30. ... and geo-redundant data
centers.
GSS
Webmail
RPC
Front-End Server
ADM
Front-End Server
inetd
config
sshd ntp.conf
config
/etc/snmp/*
config A
sshd ntp.conf
config
rsyslog
....
resolv.conf
config B
...
config A
Service X
resolv.conf
config B
...
config B
Postpath Unified Node (PUN)
config B
Postpath Unified Node (PUN)
inetd
Postpath Unified Node (PUN)
config
config
Postpath Unified Node (PUN)
config
config
LVM
config
LVM
config
config
config B
Postpath Unified Node (PUN)
inetd
config
config
config
LVM
config
LVM
/etc/logrotate/*
config
LUN
LUN
Webmail
config
LVM
ADM
inetd
smb.conf
...
...
config A
config
/etc/sysconfig/*
inetd
sshd ntp.conf
config
Service Y
DR:BD
/etc/sysconfig/*
inetd
sshd ntp.conf
config
/etc/snmp/*
resolv.conf
config A
Service X
Service X
config B
inetd
Postpath Unified Node (PUN)
inetd
Postpath Unified Node (PUN)
config
LVM
config
inetd
config
Service Y
DR:BD
config B
resolv.conf
config A
Service X
config B
inetd
Service X
config B
inetd
config
/etc/*
....
...
config A
resolv.conf
config B
...
ntp.conf
smb.conf
LVM
config
inetd
inetd
/etc/sysconfig/*
config
/etc/logrotate/*
config
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
config
/etc/snmp/*
/etc/*
rsyslog
....
LVM
...
config
/etc/*
rsyslog
....
LVM
...
config
....
...
resolv.conf
ntp.conf
...
DR:BD
resolv.conf
ntp.conf
...
smb.conf
DR:BD
ntp.conf
...
smb.conf
...
smb.conf
...
config A
PPSD
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
DR:BD
...
config A
config B
cluster
manager
config B
Postpath Unified Node (PUN)
resolv.conf
ntp.conf
smb.conf
PPSD
inetd
/etc/logrotate/*
config
LVM
resolv.conf
...
...
config A
Service Y
config B
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
...
DR:BD
smb.conf
config A
config B
...
config A
Service X
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
....
resolv.conf
ntp.conf
/etc/snmp/*
config
...
Service Y
DR:BD
config B
config A
/etc/passwd
/etc/snmp/*
/etc/*
...
...
PPSD
cluster
manager
Service Y
config B
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
DR:BD
...
config A
Service X
ntp.conf
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
config
rsyslog
....
config A
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/*
ntp.conf
/etc/passwd
/etc/logrotate/*
smb.conf
LVM
resolv.conf
config B
...
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
...
Postpath Unified Node (PUN)
smb.conf
config B
inetd
rsyslog
....
smb.conf
config A
Service Y
/etc/snmp/*
...
...
/etc/*
...
config A
Service Y
DR:BD
config B
ntp.conf
....
/etc/snmp/*
config
...
LVM
...
config A
config
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
....
/etc/passwd
config A
PPSD
cluster
manager
sshd ntp.conf
config
/etc/*
config B
...
config A
Service X
...
...
DR:BD
...
inetd
...
resolv.conf
ntp.conf
smb.conf
/etc/logrotate/*
config
Front-End Server
/etc/*
/etc/passwd
/etc/sysconfig/*
...
...
smb.conf
/etc/snmp/*
config
LVM
resolv.conf
config A
config B
cluster
manager
/etc/snmp/*
config
...
LVM
config A
Service X
/etc/logrotate/*
config
rsyslog
....
...
...
PPSD
Load
Balancer
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
/etc/sysconfig/*
config
sshd
/etc/snmp/*
/etc/*
LVM
DR:BD
...
config B
sshd ntp.conf
config
Postpath Unified Node (PUN)
/etc/logrotate/*
config
rsyslog
....
ntp.conf
smb.conf
config A
PPSD
cluster
manager
inetd
rsyslog
inetd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/*
...
...
DR:BD
...
config A
config B
cluster
manager
config B
Postpath Unified Node (PUN)
resolv.conf
ntp.conf
smb.conf
PPSD
inetd
/etc/snmp/*
config
LVM
resolv.conf
...
...
config A
Service Y
config B
/etc/passwd
/etc/logrotate/*
config
rsyslog
....
...
DR:BD
smb.conf
config A
config B
config A
Service X
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
....
resolv.conf
ntp.conf
ntp.conf
...
config A
/etc/sysconfig/*
config
sshd
/etc/snmp/*
/etc/*
...
...
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
PPSD
resolv.conf
config B
...
smb.conf
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
LVM
DR:BD
Service Y
config B
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
config A
Service X
config B
/etc/passwd
cluster
manager
...
config A
Service Y
DR:BD
config B
....
...
config A
Service Y
config A
Service X
ntp.conf
/etc/*
smb.conf
...
config A
/etc/snmp/*
config
...
LVM
resolv.conf
config B
...
smb.conf
Front-End Server
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
....
...
config A
Service Y
DR:BD
config B
ntp.conf
sshd ntp.conf
config
/etc/*
config
...
LVM
...
config A
config B
...
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
ADM
/etc/passwd
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
....
RPC
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
config
resolv.conf
...
...
/etc/*
config
...
LVM
config A
config B
rsyslog
....
/etc/snmp/*
smb.conf
Postpath Unified Node (PUN)
PPSD
config B
cluster
manager
...
config A
config A
PPSD
config B
cluster
manager
config B
cluster
manager
spare
spare
SAN
SAN
LUN
LUN
LUN
Webmail
LUN
RPC
Front-End Server
/etc/sysconfig/*
smb.conf
rsyslog
Service Y
DR:BD
inetd
sshd ntp.conf
config
...
DR:BD
/etc/sysconfig/*
smb.conf
/etc/*
rsyslog
LVM
...
config A
resolv.conf
config B
...
ntp.conf
Service X
config A
inetd
DR:BD
...
/etc/snmp/*
smb.conf
ntp.conf
Service X
config A
config B
...
Service Y
DR:BD
/etc/sysconfig/*
sshd ntp.conf
config
...
DR:BD
smb.conf
rsyslog
resolv.conf
ntp.conf
Service X
config A
Service Y
DR:BD
config B
...
/etc/sysconfig/*
smb.conf
/etc/*
rsyslog
LVM
...
config A
resolv.conf
ntp.conf
Service X
config A
config A
Postpath Unified Node (PUN)
config
resolv.conf
ntp.conf
smb.conf
smb.conf
...
....
Service Y
config A
Service X
config B
inetd
/etc/sysconfig/*
config
sshd
config
/etc/snmp/*
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
inetd
config
....
config
ntp.conf
...
/etc/snmp/*
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
inetd
/etc/sysconfig/*
config
sshd
config
config
....
ntp.conf
...
/etc/snmp/*
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
/etc/*
....
...
config A
resolv.conf
config B
...
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
config
/etc/snmp/*
config
....
ntp.conf
...
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
SAN
config
....
LUN
LUN
/etc/logrotate/*
/etc/snmp/*
ntp.conf
...
/etc/*
rsyslog
config
....
...
resolv.conf
DR:BD
ntp.conf
...
smb.conf
smb.conf
...
PPSD
...
config A
PPSD
config B
cluster
manager
config A
config B
cluster
manager
spare
LUN
Reinhardt Quelle (rquelle@cisco.com)
config
LVM
...
...
config A
config B
SAN
LUN
/etc/sysconfig/*
config
sshd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
inetd
/etc/*
rsyslog
LVM
...
...
LUN
LUN
LUN
config A
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
...
Service X
spare
LUN
/etc/snmp/*
config
...
Service Y
DR:BD
smb.conf
config A
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
PPSD
cluster
manager
Service Y
/etc/passwd
/etc/logrotate/*
LVM
...
...
config A
config B
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
Service X
/etc/*
rsyslog
LVM
...
...
config A
config B
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
config A
Service X
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/logrotate/*
LVM
...
...
Service Y
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
resolv.conf
DR:BD
smb.conf
config A
config B
config A
config B
/etc/passwd
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
Service X
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
config A
config B
Service Y
/etc/passwd
/etc/logrotate/*
resolv.conf
...
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
LVM
...
config A
config B
...
/etc/*
config
PPSD
cluster
manager
Service X
smb.conf
...
/etc/logrotate/*
/etc/snmp/*
/etc/logrotate/*
smb.conf
LVM
...
config A
...
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
rsyslog
....
smb.conf
/etc/passwd
inetd
/etc/snmp/*
inetd
/etc/*
...
config B
...
....
/etc/snmp/*
config B
...
Service Y
DR:BD
config B
...
/etc/logrotate/*
config
...
....
config B
...
/etc/sysconfig/*
config
LVM
DR:BD
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
...
DR:BD
/etc/snmp/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
smb.conf
Service X
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
/etc/logrotate/*
config
...
LVM
...
config A
config B
...
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
/etc/passwd
config
resolv.conf
....
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
inetd
/etc/*
config
...
LVM
resolv.conf
Front-End Server
/etc/passwd
inetd
/etc/snmp/*
config
/etc/logrotate/*
sshd ntp.conf
config
rsyslog
....
...
config A
config
Front-End Server
/etc/*
/etc/sysconfig/*
LVM
/etc/passwd
config
resolv.conf
/etc/*
ADM
Front-End Server
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
rsyslog
....
/etc/snmp/*
config B
...
Service Y
DR:BD
config B
...
/etc/logrotate/*
config
...
....
inetd
/etc/snmp/*
config
LVM
/etc/passwd
config
resolv.conf
LUN
RPC
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
....
/etc/snmp/*
config
...
LVM
config A
config B
...
/etc/logrotate/*
LUN
Front-End Server
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
...
DR:BD
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
...
LUN
Webmail
Front-End Server
/etc/passwd
inetd
/etc/logrotate/*
config
config
LVM
LUN
ADM
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
© 2013 Cisco and/or its affiliates. All rights reserved.
config B
LUN
Front-End Server
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
config
LVM
DR:BD
/etc/logrotate/*
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
config
resolv.conf
...
...
rsyslog
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
....
LVM
config A
PPSD
config B
cluster
manager
LUN
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
DR:BD
ntp.conf
...
DR:BD
smb.conf
PPSD
cluster
manager
LUN
Webmail
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
sshd
DR:BD
Service X
LUN
Front-End Server
/etc/passwd
inetd
rsyslog
rsyslog
....
...
resolv.conf
ntp.conf
...
...
config B
cluster
manager
spare
LUN
RPC
Front-End Server
Service X
/etc/snmp/*
/etc/*
rsyslog
....
...
DR:BD
smb.conf
config A
PPSD
sshd
/etc/snmp/*
config
resolv.conf
ntp.conf
...
DR:BD
...
/etc/sysconfig/*
config
/etc/*
rsyslog
....
...
resolv.conf
ntp.conf
smb.conf
config A
config B
inetd
/etc/logrotate/*
sshd
/etc/*
rsyslog
....
resolv.conf
PPSD
cluster
manager
config A
/etc/passwd
/etc/sysconfig/*
config
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
/etc/snmp/*
/etc/*
...
...
...
config A
config B
...
Postpath Unified Node (PUN)
SAN
LUN
sshd
ntp.conf
Service Y
config B
spare
inetd
....
resolv.conf
smb.conf
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
DR:BD
smb.conf
SAN
Load
Balancer
/etc/*
...
config A
config B
...
config A
Service X
config B
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
resolv.conf
ntp.conf
/etc/snmp/*
config
...
Service Y
DR:BD
config B
config A
/etc/passwd
/etc/sysconfig/*
config
rsyslog
....
...
...
PPSD
cluster
manager
config A
Service X
ntp.conf
Service Y
config B
/etc/passwd
inetd
/etc/snmp/*
config
LVM
DR:BD
...
config B
/etc/logrotate/*
smb.conf
resolv.conf
...
/etc/*
rsyslog
....
ntp.conf
smb.conf
config A
PPSD
cluster
manager
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
LVM
config B
...
config A
Service X
/etc/logrotate/*
config
/etc/*
...
...
DR:BD
...
....
inetd
rsyslog
....
...
config A
smb.conf
Postpath Unified Node (PUN)
resolv.conf
ntp.conf
smb.conf
config A
config B
cluster
manager
config B
/etc/snmp/*
...
...
/etc/*
config
...
Service Y
DR:BD
config B
config A
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
LVM
resolv.conf
PPSD
config B
cluster
manager
inetd
/etc/logrotate/*
config
rsyslog
....
...
...
...
config A
PPSD
sshd
/etc/snmp/*
/etc/*
LVM
DR:BD
smb.conf
...
config A
config B
cluster
manager
config A
Service X
ntp.conf
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
ntp.conf
...
DR:BD
smb.conf
PPSD
rsyslog
....
...
resolv.conf
ntp.conf
...
DR:BD
...
resolv.conf
Service Y
Postpath Unified Node (PUN)
/etc/passwd
inetd
/etc/snmp/*
...
config B
config
config
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
LVM
config B
...
config A
Service X
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/*
rsyslog
....
...
resolv.conf
ntp.conf
smb.conf
config B
/etc/logrotate/*
sshd ntp.conf
config
smb.conf
config A
Service Y
/etc/passwd
inetd
/etc/snmp/*
/etc/*
rsyslog
....
resolv.conf
config A
config B
config B
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
...
...
PPSD
Service Y
DR:BD
config B
/etc/sysconfig/*
config
Front-End Server
/etc/*
/etc/passwd
/etc/sysconfig/*
inetd
....
...
config A
rsyslog
....
config
resolv.conf
...
...
/etc/*
config
...
...
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
DR:BD
inetd
config A
Service X
ntp.conf
config A
Service X
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
rsyslog
cluster
manager
resolv.conf
config B
...
/etc/snmp/*
smb.conf
rsyslog
sshd
/etc/snmp/*
config
LVM
DR:BD
/etc/logrotate/*
smb.conf
config A
Service Y
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service X
/etc/passwd
inetd
Service Y
DR:BD
config B
sshd ntp.conf
config
LVM
...
config A
Service Y
config A
Service X
ntp.conf
inetd
....
...
config A
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
smb.conf
...
config A
/etc/snmp/*
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
LVM
smb.conf
Service X
sshd ntp.conf
config
....
...
config A
Service Y
DR:BD
config B
ntp.conf
inetd
Front-End Server
/etc/passwd
inetd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
...
config A
/etc/snmp/*
ADM
Front-End Server
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
/etc/*
config
...
Service Y
DR:BD
config B
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
RPC
/etc/passwd
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
LVM
Service X
config
/etc/passwd
/etc/sysconfig/*
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
...
...
rsyslog
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
sshd
DR:BD
Webmail
Front-End Server
/etc/passwd
inetd
rsyslog

31. Which means data must be
replicated - somehow.
GSS
Webmail
RPC
Front-End Server
ADM
Front-End Server
inetd
config
sshd ntp.conf
config
/etc/snmp/*
rsyslog
Service X
rsyslog
config A
Service X
sshd ntp.conf
config
rsyslog
config B
...
/etc/snmp/*
config A
Service X
inetd
sshd ntp.conf
config
rsyslog
config B
...
/etc/snmp/*
config A
Service X
/etc/logrotate/*
sshd ntp.conf
config
rsyslog
config B
...
config A
Service X
Service Y
DR:BD
config B
ntp.conf
config A
config B
...
config
config A
config A
Service X
config B
Postpath Unified Node (PUN)
inetd
Postpath Unified Node (PUN)
/etc/logrotate/*
sshd
/etc/snmp/*
config
config
rsyslog
/etc/logrotate/*
rsyslog
DR:BD
Postpath Unified Node (PUN)
config
config
/etc/logrotate/*
config
LUN
LUN
Webmail
config
....
LVM
...
resolv.conf
resolv.conf
ntp.conf
...
ADM
smb.conf
...
PPSD
...
config A
inetd
config
/etc/sysconfig/*
inetd
sshd ntp.conf
config
smb.conf
config A
Service Y
DR:BD
/etc/sysconfig/*
inetd
sshd ntp.conf
config
smb.conf
rsyslog
LVM
resolv.conf
config B
...
config A
Service X
resolv.conf
config B
Postpath Unified Node (PUN)
Postpath Unified Node (PUN)
inetd
Postpath Unified Node (PUN)
/etc/logrotate/*
sshd
config
config
LVM
config
LVM
Postpath Unified Node (PUN)
config
DR:BD
config
LVM
config A
Service X
config
resolv.conf
Postpath Unified Node (PUN)
inetd
....
resolv.conf
ntp.conf
smb.conf
config
config
config
config
LVM
/etc/logrotate/*
config
/etc/snmp/*
/etc/*
rsyslog
....
LVM
...
DR:BD
config
....
...
resolv.conf
ntp.conf
...
smb.conf
DR:BD
ntp.conf
...
smb.conf
...
smb.conf
...
config A
PPSD
sshd
/etc/snmp/*
config
resolv.conf
ntp.conf
...
DR:BD
...
/etc/sysconfig/*
config
/etc/*
LVM
resolv.conf
ntp.conf
smb.conf
config A
config B
cluster
manager
inetd
/etc/logrotate/*
config
rsyslog
....
...
resolv.conf
PPSD
sshd
/etc/snmp/*
/etc/*
rsyslog
....
...
...
...
config A
/etc/passwd
/etc/sysconfig/*
config
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
LVM
DR:BD
smb.conf
config A
config B
...
Postpath Unified Node (PUN)
Service Y
config B
PPSD
config B
cluster
manager
...
config A
config A
PPSD
config B
cluster
manager
config B
cluster
manager
spare
spare
SAN
SAN
LUN
LUN
LUN
Webmail
LUN
RPC
Front-End Server
/etc/sysconfig/*
smb.conf
rsyslog
Service Y
DR:BD
inetd
sshd ntp.conf
config
...
DR:BD
/etc/sysconfig/*
smb.conf
/etc/*
rsyslog
LVM
...
config A
resolv.conf
config B
...
ntp.conf
Service X
config A
inetd
DR:BD
...
/etc/snmp/*
smb.conf
ntp.conf
Service X
config A
config B
...
Service Y
DR:BD
/etc/sysconfig/*
sshd ntp.conf
config
...
DR:BD
smb.conf
rsyslog
resolv.conf
ntp.conf
Service X
config A
Service Y
DR:BD
config B
...
/etc/sysconfig/*
smb.conf
/etc/*
rsyslog
LVM
...
config A
resolv.conf
ntp.conf
Service X
config A
config A
Postpath Unified Node (PUN)
config
resolv.conf
ntp.conf
smb.conf
smb.conf
...
....
Service Y
config A
Service X
config B
inetd
/etc/sysconfig/*
config
sshd
config
/etc/snmp/*
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
inetd
config
....
config
ntp.conf
...
/etc/snmp/*
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
inetd
/etc/sysconfig/*
config
sshd
config
config
....
ntp.conf
...
/etc/snmp/*
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
/etc/*
....
...
config A
resolv.conf
config B
...
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
config
/etc/snmp/*
config
....
ntp.conf
...
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
SAN
config
....
LUN
LUN
/etc/logrotate/*
/etc/snmp/*
ntp.conf
...
/etc/*
rsyslog
config
....
...
resolv.conf
DR:BD
ntp.conf
...
smb.conf
smb.conf
...
PPSD
...
config A
PPSD
config B
cluster
manager
config A
config B
cluster
manager
spare
LUN
Reinhardt Quelle (rquelle@cisco.com)
config
LVM
...
...
config A
config B
SAN
LUN
/etc/sysconfig/*
config
sshd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
inetd
/etc/*
rsyslog
LVM
...
...
LUN
LUN
LUN
config A
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
...
Service X
spare
LUN
/etc/snmp/*
config
...
Service Y
DR:BD
smb.conf
config A
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
PPSD
cluster
manager
Service Y
/etc/passwd
/etc/logrotate/*
LVM
...
...
config A
config B
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
Service X
/etc/*
rsyslog
LVM
...
...
config A
config B
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
config A
Service X
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/logrotate/*
LVM
...
...
Service Y
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
resolv.conf
DR:BD
smb.conf
config A
config B
config A
config B
/etc/passwd
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
Service X
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
config A
config B
Service Y
/etc/passwd
/etc/logrotate/*
resolv.conf
...
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
LVM
...
config A
config B
...
/etc/*
config
PPSD
cluster
manager
Service X
smb.conf
...
/etc/logrotate/*
/etc/snmp/*
/etc/logrotate/*
smb.conf
LVM
...
config A
...
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
rsyslog
....
smb.conf
/etc/passwd
inetd
/etc/snmp/*
inetd
/etc/*
...
config B
...
....
/etc/snmp/*
config B
...
Service Y
DR:BD
config B
...
/etc/logrotate/*
config
...
....
config B
...
/etc/sysconfig/*
config
LVM
DR:BD
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
...
DR:BD
/etc/snmp/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
smb.conf
Service X
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
/etc/logrotate/*
config
...
LVM
...
config A
config B
...
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
/etc/passwd
config
resolv.conf
....
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
inetd
/etc/*
config
...
LVM
resolv.conf
Front-End Server
/etc/passwd
inetd
/etc/snmp/*
config
/etc/logrotate/*
sshd ntp.conf
config
rsyslog
....
...
config A
config
Front-End Server
/etc/*
/etc/sysconfig/*
LVM
/etc/passwd
config
resolv.conf
/etc/*
ADM
Front-End Server
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
rsyslog
....
/etc/snmp/*
config B
...
Service Y
DR:BD
config B
...
/etc/logrotate/*
config
...
....
inetd
/etc/snmp/*
config
LVM
/etc/passwd
config
resolv.conf
LUN
RPC
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
....
/etc/snmp/*
config
...
LVM
config A
config B
...
/etc/logrotate/*
LUN
Front-End Server
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
...
DR:BD
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
...
LUN
Webmail
Front-End Server
/etc/passwd
inetd
/etc/logrotate/*
config
config
LVM
LUN
ADM
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
© 2013 Cisco and/or its affiliates. All rights reserved.
/etc/*
...
config A
config B
...
config A
Service X
config B
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
rsyslog
....
resolv.conf
ntp.conf
/etc/snmp/*
config
...
Service Y
DR:BD
config B
config A
/etc/passwd
/etc/sysconfig/*
config
/etc/*
...
...
PPSD
cluster
manager
config A
Service X
ntp.conf
Service Y
config B
/etc/passwd
inetd
/etc/snmp/*
config
LVM
DR:BD
...
config A
Service X
config B
/etc/logrotate/*
sshd
rsyslog
....
ntp.conf
/etc/passwd
/etc/logrotate/*
smb.conf
LVM
config B
...
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
...
Postpath Unified Node (PUN)
smb.conf
config A
....
inetd
rsyslog
....
smb.conf
config A
Service Y
/etc/snmp/*
...
...
/etc/*
...
config A
Service Y
DR:BD
config B
ntp.conf
/etc/sysconfig/*
config
config B
cluster
manager
resolv.conf
/etc/passwd
inetd
resolv.conf
...
PPSD
config B
config
LVM
/etc/snmp/*
config
...
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
....
config B
...
config A
Service X
/etc/snmp/*
config
...
sshd ntp.conf
config
...
...
DR:BD
smb.conf
config A
config B
cluster
manager
inetd
smb.conf
/etc/*
rsyslog
....
ntp.conf
...
PPSD
Service Y
DR:BD
config B
/etc/logrotate/*
sshd
/etc/snmp/*
config
...
DR:BD
...
config B
config A
Service X
/etc/logrotate/*
/etc/sysconfig/*
config
/etc/passwd
config
resolv.conf
...
...
/etc/*
...
config A
Front-End Server
/etc/*
rsyslog
....
/etc/snmp/*
config
...
Postpath Unified Node (PUN)
resolv.conf
ntp.conf
smb.conf
config A
PPSD
cluster
manager
Load
Balancer
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
LVM
DR:BD
/etc/logrotate/*
smb.conf
/etc/sysconfig/*
config
/etc/*
LVM
resolv.conf
...
config A
config B
cluster
manager
sshd ntp.conf
config
LVM
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
/etc/passwd
/etc/sysconfig/*
inetd
rsyslog
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
...
...
/etc/passwd
inetd
/etc/logrotate/*
config
rsyslog
....
...
...
DR:BD
smb.conf
PPSD
sshd
/etc/snmp/*
config
LVM
ntp.conf
...
...
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
config A
Service Y
config B
/etc/passwd
inetd
/etc/*
rsyslog
....
...
resolv.conf
ntp.conf
smb.conf
config A
config B
config A
Service X
config B
/etc/logrotate/*
sshd
/etc/snmp/*
config
resolv.conf
PPSD
...
config A
/etc/sysconfig/*
config
/etc/*
rsyslog
....
...
...
DR:BD
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
rsyslog
cluster
manager
ntp.conf
smb.conf
/etc/passwd
/etc/sysconfig/*
config
Service Y
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service X
config B
/etc/passwd
inetd
resolv.conf
config B
...
...
config A
Service Y
Service Y
DR:BD
config B
....
...
config A
smb.conf
...
Service X
config A
Service X
ntp.conf
/etc/*
config
...
LVM
config B
...
/etc/snmp/*
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
smb.conf
config A
sshd ntp.conf
config
....
...
config A
Service Y
DR:BD
config B
ntp.conf
inetd
Front-End Server
/etc/passwd
inetd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
....
...
config A
/etc/snmp/*
ADM
Front-End Server
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
LUN
RPC
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
config
resolv.conf
...
...
/etc/*
config
...
LVM
config B
rsyslog
....
/etc/snmp/*
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
config
LVM
DR:BD
/etc/logrotate/*
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
config
resolv.conf
...
...
rsyslog
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
....
LVM
config B
cluster
manager
LUN
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
DR:BD
config A
PPSD
config B
cluster
manager
LUN
Webmail
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
sshd
DR:BD
Service X
LUN
Front-End Server
/etc/passwd
inetd
rsyslog
rsyslog
ntp.conf
...
DR:BD
smb.conf
...
config B
cluster
manager
spare
LUN
RPC
Front-End Server
Service X
/etc/snmp/*
/etc/*
rsyslog
...
DR:BD
smb.conf
config A
PPSD
sshd
/etc/snmp/*
config
....
LVM
ntp.conf
...
DR:BD
...
/etc/sysconfig/*
config
/etc/*
rsyslog
....
...
resolv.conf
ntp.conf
smb.conf
config A
config B
cluster
manager
inetd
/etc/logrotate/*
config
LVM
resolv.conf
PPSD
sshd
/etc/snmp/*
/etc/*
rsyslog
....
...
...
DR:BD
...
config A
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
ntp.conf
smb.conf
config B
cluster
manager
inetd
config B
Postpath Unified Node (PUN)
/etc/passwd
/etc/sysconfig/*
config
/etc/*
rsyslog
....
resolv.conf
config A
PPSD
config B
cluster
manager
...
SAN
LUN
sshd
ntp.conf
Service Y
config B
spare
inetd
....
resolv.conf
config A
Service X
config B
/etc/passwd
inetd
/etc/logrotate/*
config
/etc/*
...
...
config A
PPSD
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
LVM
DR:BD
...
config A
config B
cluster
manager
SAN
Load
Balancer
/etc/*
...
config A
config B
...
smb.conf
config A
Service Y
config B
/etc/passwd
inetd
/etc/logrotate/*
rsyslog
....
smb.conf
...
PPSD
config
/etc/*
ntp.conf
...
DR:BD
smb.conf
...
config B
cluster
manager
sshd
/etc/snmp/*
...
resolv.conf
ntp.conf
...
smb.conf
config A
PPSD
config
LVM
...
resolv.conf
ntp.conf
...
DR:BD
...
config
rsyslog
....
LVM
...
resolv.conf
ntp.conf
smb.conf
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
config
....
LVM
resolv.conf
config A
config B
cluster
manager
sshd
/etc/snmp/*
/etc/*
config
....
...
...
PPSD
config
/etc/*
config
LVM
DR:BD
...
sshd
/etc/snmp/*
/etc/*
rsyslog
....
ntp.conf
smb.conf
inetd
config A
Service X
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service Y
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
config A
Service X
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
resolv.conf
config A
config B
inetd
config A
Service Y
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service X
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service Y
config B
/etc/passwd
inetd
...
...
PPSD
/etc/snmp/*
config
...
Service Y
DR:BD
config B
...
config
LVM
DR:BD
Service X
smb.conf
...
/etc/logrotate/*
sshd
rsyslog
/etc/logrotate/*
smb.conf
config A
ntp.conf
smb.conf
...
Postpath Unified Node (PUN)
cluster
manager
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
LVM
smb.conf
/etc/sysconfig/*
config
inetd
rsyslog
...
/etc/passwd
inetd
....
...
...
....
resolv.conf
smb.conf
Service Y
Postpath Unified Node (PUN)
/etc/snmp/*
config
/etc/*
...
config A
...
config B
config
DR:BD
/etc/snmp/*
config
...
LVM
resolv.conf
LVM
/etc/logrotate/*
smb.conf
smb.conf
Service X
/etc/sysconfig/*
config
sshd
Front-End Server
/etc/*
inetd
....
...
config A
Service Y
DR:BD
config B
ntp.conf
/etc/passwd
inetd
rsyslog
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
ntp.conf
Front-End Server
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
ADM
/etc/passwd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
config B
...
/etc/snmp/*
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
sshd ntp.conf
config
/etc/*
config
...
LVM
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
RPC
/etc/passwd
inetd
/etc/logrotate/*
config
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
Front-End Server
/etc/*
config
/etc/passwd
/etc/sysconfig/*
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
...
...
config A
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
sshd
DR:BD
Webmail
Front-End Server
/etc/passwd
inetd
rsyslog

32. But wait! There’s more!
GSS
Webmail
RPC
Front-End Server
ADM
Front-End Server
inetd
config
sshd ntp.conf
config
/etc/snmp/*
rsyslog
Service X
rsyslog
config A
Service X
sshd ntp.conf
config
rsyslog
config B
...
/etc/snmp/*
config A
Service X
inetd
sshd ntp.conf
config
rsyslog
config B
...
/etc/snmp/*
config A
Service X
/etc/logrotate/*
sshd ntp.conf
config
rsyslog
config B
...
config A
Service X
Service Y
DR:BD
config B
ntp.conf
Security Appliances
config A
config B
...
config
config A
config A
Service X
config B
Postpath Unified Node (PUN)
inetd
Postpath Unified Node (PUN)
/etc/logrotate/*
sshd
/etc/snmp/*
config
config
rsyslog
/etc/logrotate/*
rsyslog
DR:BD
Postpath Unified Node (PUN)
config
config
/etc/logrotate/*
config
LUN
LUN
Webmail
config
....
LVM
...
resolv.conf
resolv.conf
ntp.conf
...
ADM
smb.conf
...
PPSD
...
config A
inetd
config
/etc/sysconfig/*
inetd
sshd ntp.conf
config
smb.conf
config A
Service Y
DR:BD
/etc/sysconfig/*
inetd
sshd ntp.conf
config
smb.conf
rsyslog
LVM
resolv.conf
config B
...
config A
Service X
resolv.conf
config B
Postpath Unified Node (PUN)
Postpath Unified Node (PUN)
inetd
Postpath Unified Node (PUN)
/etc/logrotate/*
sshd
config
config
LVM
config
LVM
Postpath Unified Node (PUN)
config
DR:BD
config
LVM
config A
Service X
config
resolv.conf
Postpath Unified Node (PUN)
inetd
config
config
config
config
LVM
/etc/logrotate/*
config
/etc/snmp/*
/etc/*
rsyslog
....
LVM
...
DR:BD
config
....
...
resolv.conf
ntp.conf
...
smb.conf
DR:BD
ntp.conf
...
smb.conf
...
smb.conf
...
config A
PPSD
sshd
/etc/snmp/*
config
resolv.conf
ntp.conf
...
DR:BD
...
/etc/sysconfig/*
config
/etc/*
LVM
resolv.conf
ntp.conf
smb.conf
config A
config B
cluster
manager
inetd
/etc/logrotate/*
config
rsyslog
....
...
resolv.conf
PPSD
sshd
/etc/snmp/*
/etc/*
rsyslog
....
...
...
...
config A
/etc/passwd
/etc/sysconfig/*
config
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
LVM
DR:BD
smb.conf
config A
config B
...
Postpath Unified Node (PUN)
PPSD
config B
cluster
manager
...
config A
config A
PPSD
config B
cluster
manager
config B
cluster
manager
SAN
Identity Servers
LUN
LUN
LUN
Webmail
LUN
RPC
Front-End Server
/etc/sysconfig/*
smb.conf
rsyslog
Service Y
DR:BD
inetd
sshd ntp.conf
config
...
DR:BD
/etc/sysconfig/*
smb.conf
/etc/*
rsyslog
LVM
...
config A
resolv.conf
config B
...
ntp.conf
Service X
config A
inetd
DR:BD
...
/etc/snmp/*
smb.conf
ntp.conf
Service X
config A
config B
...
Service Y
DR:BD
/etc/sysconfig/*
sshd ntp.conf
config
...
DR:BD
smb.conf
rsyslog
resolv.conf
ntp.conf
Service X
config A
Service Y
DR:BD
config B
...
/etc/sysconfig/*
smb.conf
/etc/*
rsyslog
LVM
...
config A
resolv.conf
ntp.conf
Service X
config A
config A
Postpath Unified Node (PUN)
config
resolv.conf
ntp.conf
smb.conf
smb.conf
...
....
Service Y
config A
Service X
config B
inetd
/etc/sysconfig/*
config
sshd
config
/etc/snmp/*
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
inetd
config
....
config
ntp.conf
...
/etc/snmp/*
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
inetd
/etc/sysconfig/*
config
sshd
config
config
....
ntp.conf
...
/etc/snmp/*
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
/etc/*
....
...
config A
resolv.conf
config B
...
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
config
/etc/snmp/*
config
....
ntp.conf
...
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
SAN
config
....
LUN
LUN
/etc/logrotate/*
/etc/snmp/*
ntp.conf
...
/etc/*
rsyslog
config
....
...
resolv.conf
DR:BD
ntp.conf
...
smb.conf
smb.conf
...
PPSD
...
config A
PPSD
config B
cluster
manager
config A
config B
cluster
manager
spare
LUN
Reinhardt Quelle (rquelle@cisco.com)
config
LVM
...
...
config A
config B
SAN
LUN
/etc/sysconfig/*
config
sshd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
inetd
/etc/*
rsyslog
LVM
...
...
LUN
LUN
LUN
config A
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
...
Service X
spare
LUN
/etc/snmp/*
config
...
Service Y
DR:BD
smb.conf
config A
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
PPSD
cluster
manager
Service Y
/etc/passwd
/etc/logrotate/*
LVM
...
...
config A
config B
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
Service X
/etc/*
rsyslog
LVM
...
...
config A
config B
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
config A
Service X
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/logrotate/*
LVM
...
...
Service Y
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
resolv.conf
DR:BD
smb.conf
config A
config B
config A
config B
/etc/passwd
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
Service X
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
config A
config B
Service Y
/etc/passwd
/etc/logrotate/*
resolv.conf
...
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
LVM
...
config A
config B
...
/etc/*
config
PPSD
cluster
manager
Service X
smb.conf
...
/etc/logrotate/*
/etc/snmp/*
/etc/logrotate/*
smb.conf
LVM
...
config A
...
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
rsyslog
....
smb.conf
/etc/passwd
inetd
/etc/snmp/*
inetd
/etc/*
...
config B
...
....
/etc/snmp/*
config B
...
Service Y
DR:BD
config B
...
/etc/logrotate/*
config
...
....
config B
...
/etc/sysconfig/*
config
LVM
DR:BD
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
...
DR:BD
/etc/snmp/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
smb.conf
Service X
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
/etc/logrotate/*
config
...
LVM
...
config A
config B
...
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
/etc/passwd
config
resolv.conf
....
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
inetd
/etc/*
config
...
LVM
resolv.conf
Front-End Server
/etc/passwd
inetd
/etc/snmp/*
config
/etc/logrotate/*
sshd ntp.conf
config
rsyslog
....
...
config A
config
Front-End Server
/etc/*
/etc/sysconfig/*
LVM
/etc/passwd
config
resolv.conf
/etc/*
ADM
Front-End Server
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
rsyslog
....
/etc/snmp/*
config B
...
Service Y
DR:BD
config B
...
/etc/logrotate/*
config
...
....
inetd
/etc/snmp/*
config
LVM
/etc/passwd
config
resolv.conf
LUN
RPC
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
....
/etc/snmp/*
config
...
LVM
config A
config B
...
/etc/logrotate/*
LUN
Front-End Server
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
...
DR:BD
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
...
LUN
Webmail
Front-End Server
/etc/passwd
inetd
/etc/logrotate/*
config
config
LVM
LUN
ADM
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
© 2013 Cisco and/or its affiliates. All rights reserved.
ntp.conf
smb.conf
spare
SAN
LVM
....
resolv.conf
Service Y
config B
spare
DR:BD
/etc/*
...
config A
config B
...
config A
Service X
config B
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
rsyslog
....
resolv.conf
ntp.conf
/etc/snmp/*
config
...
Service Y
DR:BD
config B
config A
/etc/passwd
/etc/sysconfig/*
config
/etc/*
...
...
PPSD
cluster
manager
config A
Service X
ntp.conf
Service Y
config B
/etc/passwd
inetd
/etc/snmp/*
config
LVM
DR:BD
...
config A
Service X
config B
/etc/logrotate/*
sshd
rsyslog
....
ntp.conf
/etc/passwd
/etc/logrotate/*
smb.conf
LVM
config B
...
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
...
Postpath Unified Node (PUN)
smb.conf
config A
....
inetd
rsyslog
....
smb.conf
config A
Service Y
/etc/snmp/*
...
...
/etc/*
...
config A
Service Y
DR:BD
config B
ntp.conf
/etc/sysconfig/*
config
config B
cluster
manager
resolv.conf
/etc/passwd
inetd
resolv.conf
...
PPSD
config B
config
LVM
/etc/snmp/*
config
...
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
....
config B
...
config A
Service X
/etc/snmp/*
config
...
sshd ntp.conf
config
...
...
DR:BD
smb.conf
config A
config B
cluster
manager
inetd
smb.conf
/etc/*
rsyslog
....
ntp.conf
...
PPSD
Service Y
DR:BD
config B
/etc/logrotate/*
sshd
/etc/snmp/*
config
...
DR:BD
...
config B
config A
Service X
/etc/logrotate/*
/etc/sysconfig/*
config
/etc/passwd
config
resolv.conf
...
...
/etc/*
...
config A
Front-End Server
/etc/*
rsyslog
....
/etc/snmp/*
config
...
Postpath Unified Node (PUN)
resolv.conf
ntp.conf
smb.conf
config A
PPSD
cluster
manager
Load
Balancer
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
LVM
DR:BD
/etc/logrotate/*
smb.conf
/etc/sysconfig/*
config
/etc/*
LVM
resolv.conf
...
config A
config B
cluster
manager
sshd ntp.conf
config
LVM
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
/etc/passwd
/etc/sysconfig/*
inetd
rsyslog
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
...
...
/etc/passwd
inetd
/etc/logrotate/*
config
rsyslog
....
...
...
DR:BD
smb.conf
PPSD
sshd
/etc/snmp/*
config
LVM
ntp.conf
...
...
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
config A
Service Y
config B
/etc/passwd
inetd
/etc/*
rsyslog
....
...
resolv.conf
ntp.conf
smb.conf
config A
config B
config A
Service X
config B
/etc/logrotate/*
sshd
/etc/snmp/*
config
resolv.conf
PPSD
...
config A
/etc/sysconfig/*
config
/etc/*
rsyslog
....
...
...
DR:BD
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
rsyslog
cluster
manager
ntp.conf
smb.conf
/etc/passwd
/etc/sysconfig/*
config
Service Y
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service X
config B
/etc/passwd
inetd
resolv.conf
config B
...
...
config A
Service Y
Service Y
DR:BD
config B
....
...
config A
smb.conf
...
Service X
config A
Service X
ntp.conf
/etc/*
config
...
LVM
config B
...
/etc/snmp/*
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
smb.conf
config A
sshd ntp.conf
config
....
...
config A
Service Y
DR:BD
config B
ntp.conf
inetd
Front-End Server
/etc/passwd
inetd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
....
...
config A
/etc/snmp/*
ADM
Front-End Server
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
LUN
RPC
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
config
resolv.conf
...
...
/etc/*
config
...
LVM
config B
rsyslog
....
/etc/snmp/*
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
config
LVM
DR:BD
/etc/logrotate/*
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
config
resolv.conf
...
...
rsyslog
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
....
rsyslog
config B
cluster
manager
LUN
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
Billing Mgt Servers
config A
PPSD
config B
cluster
manager
LUN
Webmail
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
sshd
DR:BD
Service X
LUN
Front-End Server
/etc/passwd
inetd
rsyslog
Service X
ntp.conf
...
DR:BD
smb.conf
...
config B
cluster
manager
spare
LUN
RPC
Front-End Server
sshd
/etc/snmp/*
/etc/*
rsyslog
...
DR:BD
smb.conf
config A
PPSD
sshd
/etc/snmp/*
config
....
LVM
ntp.conf
...
DR:BD
...
/etc/sysconfig/*
config
/etc/*
rsyslog
....
...
resolv.conf
ntp.conf
smb.conf
config A
config B
cluster
manager
inetd
/etc/logrotate/*
config
LVM
resolv.conf
PPSD
sshd
/etc/snmp/*
/etc/*
rsyslog
....
...
...
DR:BD
...
config A
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
ntp.conf
smb.conf
config B
cluster
manager
inetd
config B
Postpath Unified Node (PUN)
/etc/passwd
/etc/sysconfig/*
config
/etc/*
rsyslog
....
resolv.conf
config A
PPSD
config B
cluster
manager
...
SAN
LUN
inetd
ntp.conf
Service Y
config B
spare
Load
Balancer
....
resolv.conf
config A
Service X
config B
/etc/passwd
inetd
/etc/logrotate/*
config
/etc/*
...
...
config A
PPSD
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
LVM
DR:BD
...
config A
config B
cluster
manager
SAN
Monitoring
Servers
/etc/*
...
config A
config B
...
smb.conf
config A
Service Y
config B
/etc/passwd
inetd
/etc/logrotate/*
rsyslog
....
smb.conf
...
PPSD
config
/etc/*
ntp.conf
...
DR:BD
smb.conf
...
config B
cluster
manager
sshd
/etc/snmp/*
...
resolv.conf
ntp.conf
...
smb.conf
config A
PPSD
config
LVM
...
resolv.conf
ntp.conf
...
DR:BD
...
config
rsyslog
....
LVM
...
resolv.conf
ntp.conf
smb.conf
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
config
....
LVM
resolv.conf
config A
config B
cluster
manager
sshd
/etc/snmp/*
/etc/*
config
....
...
...
PPSD
config
/etc/*
config
LVM
DR:BD
...
sshd
/etc/snmp/*
/etc/*
rsyslog
....
ntp.conf
smb.conf
inetd
config A
Service X
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service Y
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
config A
Service X
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
resolv.conf
config A
config B
inetd
config A
Service Y
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service X
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service Y
config B
/etc/passwd
inetd
...
...
PPSD
/etc/snmp/*
config
...
Service Y
DR:BD
config B
...
config
LVM
DR:BD
Service X
smb.conf
...
/etc/logrotate/*
sshd
rsyslog
/etc/logrotate/*
smb.conf
config A
ntp.conf
smb.conf
...
Postpath Unified Node (PUN)
cluster
manager
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
LVM
smb.conf
/etc/sysconfig/*
config
inetd
rsyslog
...
/etc/passwd
inetd
....
...
...
....
resolv.conf
smb.conf
Service Y
Postpath Unified Node (PUN)
/etc/snmp/*
config
/etc/*
...
config A
...
config B
config
DR:BD
/etc/snmp/*
config
...
LVM
resolv.conf
LVM
/etc/logrotate/*
smb.conf
smb.conf
Service X
/etc/sysconfig/*
config
sshd
Front-End Server
/etc/*
inetd
....
...
config A
Service Y
DR:BD
config B
ntp.conf
/etc/passwd
inetd
rsyslog
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
ntp.conf
Front-End Server
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
ADM
/etc/passwd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
config B
...
/etc/snmp/*
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
sshd ntp.conf
config
/etc/*
config
...
LVM
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
RPC
/etc/passwd
inetd
/etc/logrotate/*
config
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
Front-End Server
/etc/*
config
/etc/passwd
/etc/sysconfig/*
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
...
...
config A
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
sshd
DR:BD
Webmail
Front-End Server
/etc/passwd
inetd
rsyslog

33. But wait! There’s more!
GSS
Webmail
RPC
Front-End Server
ADM
Front-End Server
inetd
config
sshd ntp.conf
config
/etc/snmp/*
rsyslog
Service X
rsyslog
config A
Service X
sshd ntp.conf
config
rsyslog
config B
...
/etc/snmp/*
config A
Service X
inetd
sshd ntp.conf
config
rsyslog
config B
...
/etc/snmp/*
config A
Service X
/etc/logrotate/*
sshd ntp.conf
config
rsyslog
config B
...
config A
Service X
Service Y
DR:BD
config B
ntp.conf
Security Appliances
config A
config B
...
config A
Postpath Unified Node (PUN)
inetd
Postpath Unified Node (PUN)
/etc/logrotate/*
sshd
config
rsyslog
sshd
config
rsyslog
/etc/snmp/*
smb.conf
smb.conf
...
smb.conf
...
config A
PPSD
config B
PPSD
config B
smb.conf
PPSD
config B
cluster
manager
config
/etc/*
....
LVM
...
DR:BD
...
resolv.conf
resolv.conf
ntp.conf
LUN
LUN
smb.conf
...
smb.conf
...
PPSD
config B
cluster
manager
...
config A
ADM
inetd
config
/etc/sysconfig/*
inetd
sshd ntp.conf
config
smb.conf
config A
Service Y
DR:BD
/etc/sysconfig/*
inetd
sshd ntp.conf
config
smb.conf
rsyslog
LVM
resolv.conf
config B
...
config A
Service X
resolv.conf
config B
config
config
LVM
config
DR:BD
config
LVM
config A
Service X
resolv.conf
config
inetd
config
LVM
config
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
config
LVM
....
...
resolv.conf
ntp.conf
DR:BD
ntp.conf
...
smb.conf
smb.conf
...
PPSD
...
config A
config A
PPSD
config B
cluster
manager
config B
cluster
manager
SAN
Identity Servers
LUN
LUN
LUN
Webmail
LUN
RPC
Front-End Server
Front-End Server
Front-End Server
...
/etc/sysconfig/*
smb.conf
LVM
config A
Service Y
DR:BD
config B
...
...
/etc/snmp/*
/etc/sysconfig/*
smb.conf
rsyslog
LVM
...
config A
resolv.conf
config B
...
ntp.conf
Service X
config A
...
inetd
Front-End Server
/etc/snmp/*
resolv.conf
ntp.conf
Service X
config A
config B
Service Y
DR:BD
Front-End Server
inetd
...
/etc/sysconfig/*
smb.conf
resolv.conf
ntp.conf
Service X
config A
Service Y
DR:BD
config B
...
...
/etc/snmp/*
/etc/sysconfig/*
smb.conf
rsyslog
LVM
resolv.conf
ntp.conf
Service X
config A
config A
Postpath Unified Node (PUN)
resolv.conf
ntp.conf
smb.conf
smb.conf
...
...
config
Service Y
config A
Service X
config B
inetd
/etc/sysconfig/*
config
sshd
config
....
/etc/snmp/*
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
inetd
config
....
config
ntp.conf
...
/etc/snmp/*
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
inetd
/etc/sysconfig/*
config
sshd
config
config
....
ntp.conf
...
/etc/snmp/*
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
/etc/*
....
...
config A
resolv.conf
config B
...
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
config
/etc/snmp/*
config
....
ntp.conf
...
inetd
/etc/sysconfig/*
config
sshd
config
config
....
/etc/snmp/*
ntp.conf
...
SAN
config
....
LUN
LUN
/etc/logrotate/*
/etc/snmp/*
ntp.conf
...
/etc/*
rsyslog
config
....
...
resolv.conf
DR:BD
ntp.conf
...
smb.conf
smb.conf
...
PPSD
...
config A
PPSD
config B
cluster
manager
config A
config B
cluster
manager
spare
LUN
Reinhardt Quelle (rquelle@cisco.com)
config
LVM
...
...
config A
config B
SAN
LUN
/etc/sysconfig/*
config
sshd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
inetd
/etc/*
rsyslog
LVM
...
...
LUN
LUN
LUN
config A
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
...
Service X
spare
LUN
/etc/snmp/*
config
...
Service Y
DR:BD
smb.conf
config A
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
PPSD
cluster
manager
Service Y
/etc/passwd
/etc/logrotate/*
LVM
...
...
config A
config B
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
Service X
/etc/*
rsyslog
LVM
...
...
config A
config B
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/*
rsyslog
LVM
...
PPSD
cluster
manager
config B
/etc/passwd
/etc/logrotate/*
resolv.conf
DR:BD
...
config A
config B
config A
Service X
Postpath Unified Node (PUN)
/etc/*
rsyslog
smb.conf
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/logrotate/*
LVM
...
...
config A
config B
Service Y
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
resolv.conf
DR:BD
smb.conf
PPSD
cluster
manager
config A
config B
/etc/passwd
/etc/*
rsyslog
LVM
...
...
Service X
/etc/logrotate/*
resolv.conf
DR:BD
smb.conf
config A
config B
config A
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
resolv.conf
...
Service Y
/etc/passwd
/etc/logrotate/*
LVM
...
config A
config B
Postpath Unified Node (PUN)
/etc/passwd
/etc/*
config
PPSD
cluster
manager
config A
config B
...
/etc/logrotate/*
/etc/snmp/*
Service X
smb.conf
...
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
/etc/logrotate/*
smb.conf
LVM
...
config A
smb.conf
/etc/passwd
inetd
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
rsyslog
....
...
config B
/etc/snmp/*
inetd
/etc/*
smb.conf
Service X
...
....
/etc/snmp/*
config B
...
Service Y
DR:BD
config B
...
/etc/logrotate/*
config
...
....
...
config A
config
LVM
DR:BD
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
/etc/*
config B
...
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
/etc/logrotate/*
config
...
LVM
...
config A
config B
...
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
DR:BD
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
rsyslog
....
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
...
/etc/*
config
...
LVM
...
config A
config
config
LVM
/etc/logrotate/*
smb.conf
rsyslog
....
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
/etc/sysconfig/*
DR:BD
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
/etc/*
ADM
/etc/logrotate/*
sshd
rsyslog
....
/etc/snmp/*
config B
...
Service Y
DR:BD
config B
...
/etc/logrotate/*
config
...
....
inetd
/etc/snmp/*
config
LVM
DR:BD
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
/etc/*
Identity Servers
LUN
RPC
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
/etc/logrotate/*
config
...
LUN
Front-End Server
config
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
DR:BD
/etc/passwd
config
resolv.conf
sshd ntp.conf
config
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
inetd
rsyslog
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
...
LUN
Webmail
/etc/passwd
inetd
/etc/logrotate/*
config
config
LVM
DR:BD
LUN
ADM
/etc/passwd
/etc/sysconfig/*
config
Front-End Server
/etc/*
rsyslog
© 2013 Cisco and/or its affiliates. All rights reserved.
config
resolv.conf
...
...
config B
cluster
manager
spare
SAN
LVM
/etc/snmp/*
/etc/*
rsyslog
....
...
DR:BD
smb.conf
config A
PPSD
/etc/sysconfig/*
config
/etc/*
LVM
ntp.conf
...
DR:BD
...
config A
config B
cluster
manager
config B
/etc/passwd
inetd
/etc/logrotate/*
config
rsyslog
....
...
resolv.conf
ntp.conf
smb.conf
PPSD
sshd
/etc/snmp/*
config
LVM
resolv.conf
...
...
config A
Service Y
config B
Postpath Unified Node (PUN)
/etc/*
rsyslog
....
...
DR:BD
smb.conf
config A
config B
Monitoring
Servers
ntp.conf
config A
Service X
/etc/sysconfig/*
config
spare
DR:BD
....
resolv.conf
/etc/passwd
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
/etc/*
rsyslog
....
resolv.conf
ntp.conf
/etc/*
...
config A
config B
...
...
config A
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
sshd
/etc/*
...
...
PPSD
cluster
manager
/etc/snmp/*
config
...
Service Y
DR:BD
config B
smb.conf
/etc/passwd
/etc/sysconfig/*
config
Service Y
config B
Postpath Unified Node (PUN)
/etc/passwd
/etc/snmp/*
config
LVM
DR:BD
...
config A
Service X
ntp.conf
...
config A
Service X
config B
inetd
/etc/logrotate/*
sshd
rsyslog
....
ntp.conf
smb.conf
config A
config B
/etc/passwd
/etc/logrotate/*
smb.conf
LVM
config B
...
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
smb.conf
config A
Service Y
....
inetd
rsyslog
....
...
config B
/etc/snmp/*
...
...
/etc/*
...
config A
Service Y
DR:BD
config B
ntp.conf
Postpath Unified Node (PUN)
resolv.conf
...
PPSD
cluster
manager
resolv.conf
/etc/sysconfig/*
config
config
LVM
/etc/snmp/*
config
...
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
/etc/passwd
/etc/snmp/*
config
...
sshd ntp.conf
config
....
config B
...
config A
Service X
inetd
...
DR:BD
smb.conf
config A
config B
cluster
manager
inetd
smb.conf
/etc/*
rsyslog
....
ntp.conf
...
PPSD
Service Y
DR:BD
config B
/etc/logrotate/*
sshd
/etc/snmp/*
config
...
DR:BD
...
config B
cluster
manager
config A
Service X
/etc/logrotate/*
/etc/sysconfig/*
config
/etc/passwd
config
resolv.conf
...
...
/etc/*
...
config A
Front-End Server
/etc/*
rsyslog
....
/etc/snmp/*
config
...
Postpath Unified Node (PUN)
resolv.conf
ntp.conf
smb.conf
config A
PPSD
config B
Load
Balancer
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
LVM
DR:BD
/etc/logrotate/*
smb.conf
/etc/sysconfig/*
config
/etc/*
LVM
resolv.conf
...
config A
config B
cluster
manager
sshd ntp.conf
config
LVM
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
/etc/passwd
/etc/sysconfig/*
inetd
rsyslog
/etc/passwd
/etc/sysconfig/*
config
sshd
rsyslog
....
...
...
/etc/passwd
inetd
/etc/logrotate/*
config
rsyslog
....
...
...
DR:BD
smb.conf
PPSD
sshd
/etc/snmp/*
config
LVM
ntp.conf
...
...
config A
Service Y
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/*
rsyslog
....
...
resolv.conf
ntp.conf
smb.conf
config A
Service X
/etc/passwd
inetd
/etc/logrotate/*
config
/etc/*
rsyslog
....
resolv.conf
config A
config B
...
config A
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
sshd
/etc/snmp/*
/etc/*
...
...
PPSD
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
LVM
DR:BD
ntp.conf
smb.conf
/etc/passwd
/etc/sysconfig/*
config
Service Y
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
/etc/logrotate/*
sshd
rsyslog
config A
Service X
config B
Postpath Unified Node (PUN)
cluster
manager
resolv.conf
config B
...
...
config A
Service Y
/etc/passwd
/etc/sysconfig/*
config
Service Y
DR:BD
config B
....
...
config A
smb.conf
...
config A
Service X
inetd
config A
Service X
ntp.conf
/etc/*
config
...
LVM
config B
...
/etc/snmp/*
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
rsyslog
smb.conf
Postpath Unified Node (PUN)
sshd ntp.conf
config
....
...
config A
Service Y
DR:BD
config B
ntp.conf
inetd
Front-End Server
/etc/passwd
inetd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
....
...
config A
/etc/snmp/*
ADM
Front-End Server
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
LUN
RPC
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
config
resolv.conf
...
...
/etc/*
config
...
LVM
config B
rsyslog
....
/etc/snmp/*
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
config
LVM
DR:BD
/etc/logrotate/*
/etc/passwd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
/etc/passwd
config
resolv.conf
...
...
rsyslog
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
....
rsyslog
config B
cluster
manager
LUN
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
LUN
Webmail
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
sshd
DR:BD
Service X
LUN
Front-End Server
/etc/passwd
inetd
rsyslog
Billing Mgt Servers
config A
PPSD
config B
cluster
manager
spare
LUN
RPC
Front-End Server
Service X
ntp.conf
SAN
Webmail
sshd
/etc/snmp/*
....
...
...
config A
PPSD
config
config
/etc/*
LVM
DR:BD
smb.conf
config A
config B
cluster
manager
sshd
rsyslog
/etc/snmp/*
....
ntp.conf
Security Appliances
/etc/sysconfig/*
config
/etc/logrotate/*
config
/etc/*
resolv.conf
...
config A
PPSD
config B
inetd
/etc/logrotate/*
sshd
rsyslog
/etc/snmp/*
...
...
smb.conf
...
config A
PPSD
cluster
manager
config
config
LVM
DR:BD
ntp.conf
config A
/etc/passwd
/etc/sysconfig/*
config
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
/etc/logrotate/*
sshd
rsyslog
/etc/snmp/*
...
...
resolv.conf
ntp.conf
...
config A
config B
cluster
manager
Postpath Unified Node (PUN)
....
LVM
DR:BD
resolv.conf
...
/etc/sysconfig/*
config
/etc/*
...
...
smb.conf
...
config A
PPSD
cluster
manager
config
config
....
LVM
DR:BD
ntp.conf
smb.conf
...
config A
config B
cluster
manager
LUN
inetd
ntp.conf
Service Y
config B
spare
Load
Balancer
....
resolv.conf
config A
Service X
config B
/etc/passwd
inetd
/etc/logrotate/*
sshd
rsyslog
/etc/snmp/*
/etc/*
...
...
resolv.conf
ntp.conf
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
config
....
LVM
DR:BD
resolv.conf
SAN
Monitoring
Servers
/etc/*
...
config A
config B
...
smb.conf
config A
Service Y
config B
/etc/passwd
inetd
config
/etc/*
...
...
sshd
rsyslog
/etc/snmp/*
....
LVM
DR:BD
ntp.conf
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
config
config
/etc/*
...
...
resolv.conf
ntp.conf
sshd
rsyslog
/etc/snmp/*
....
LVM
DR:BD
resolv.conf
cluster
manager
config
config
/etc/*
...
...
sshd
rsyslog
/etc/snmp/*
....
LVM
DR:BD
/etc/logrotate/*
config
/etc/*
....
ntp.conf
inetd
config A
Service X
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service Y
config B
Postpath Unified Node (PUN)
/etc/passwd
inetd
config A
Service X
config B
Postpath Unified Node (PUN)
/etc/sysconfig/*
config
/etc/logrotate/*
config
/etc/*
resolv.conf
inetd
config A
Service Y
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service X
config B
/etc/passwd
/etc/sysconfig/*
config
config A
Service Y
config B
/etc/passwd
inetd
/etc/snmp/*
PPSD
config A
Service X
config B
...
...
/etc/snmp/*
config
...
Service Y
DR:BD
config B
...
Postpath Unified Node (PUN)
config
Service X
smb.conf
...
config
LVM
DR:BD
/etc/logrotate/*
smb.conf
config A
ntp.conf
smb.conf
...
/etc/logrotate/*
sshd
rsyslog
/etc/passwd
/etc/sysconfig/*
config
resolv.conf
sshd ntp.conf
config
LVM
smb.conf
/etc/sysconfig/*
config
inetd
rsyslog
...
/etc/passwd
inetd
....
...
...
....
resolv.conf
smb.conf
Service Y
Postpath Unified Node (PUN)
/etc/snmp/*
config
/etc/*
...
config A
...
config B
config
DR:BD
/etc/snmp/*
config
...
LVM
resolv.conf
LVM
/etc/logrotate/*
smb.conf
smb.conf
Service X
/etc/sysconfig/*
config
sshd
Front-End Server
/etc/*
inetd
....
...
config A
Service Y
DR:BD
config B
ntp.conf
/etc/passwd
inetd
rsyslog
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
ntp.conf
Front-End Server
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
ADM
/etc/passwd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
ntp.conf
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
/etc/*
config
...
LVM
resolv.conf
config B
...
/etc/snmp/*
inetd
/etc/logrotate/*
config
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
....
...
config A
Service Y
DR:BD
config B
sshd ntp.conf
config
/etc/*
config
...
LVM
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
config
/etc/passwd
/etc/sysconfig/*
...
...
RPC
/etc/passwd
inetd
/etc/logrotate/*
config
rsyslog
....
config
resolv.conf
LVM
DR:BD
/etc/logrotate/*
smb.conf
Front-End Server
/etc/*
config
/etc/passwd
/etc/sysconfig/*
inetd
/etc/sysconfig/*
config
sshd
/etc/snmp/*
Front-End Server
/etc/*
rsyslog
....
config
resolv.conf
...
...
config A
inetd
/etc/logrotate/*
sshd
/etc/snmp/*
config
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
/etc/logrotate/*
config
Front-End Server
/etc/*
LVM
Front-End Server
/etc/passwd
/etc/sysconfig/*
config
sshd
DR:BD
Webmail
Front-End Server
/etc/passwd
inetd
rsyslog
Billing Mgt Servers

34. “Turtles all the way down”
Even individual systems are not like your developer’s
workstation:
physical hostname
service hostname
hostname
eth0 physical
/etc/passwd
inetd
config
sshd
config
NIC
/etc/sysconfig/*
/etc/logrotate/*
rsyslog
LVM
eth0
inetd
/etc/snmp/*
/
/etc/*
config
config
...
sshd
config
rsyslog
config
LVM
hd
ntp.conf
...
NIC
/etc/sysconfig/*
...
/etc/snmp/*
/
/etc/*
....
eth1 mgt
eth2 iSCSI, NFS
resolv.conf
smb.conf
DR:BD
ntp.conf
...
...
smb.conf
...
Service Y
eth0:1 cluster
/etc/logrotate/*
....
resolv.conf
DR:BD
/etc/passwd
/
config A
config B
Service Y
config A
config B
/var..
SAN
inst1
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)
inst2

35. Deployment Environment is
Complex
• Many nodes in cluster = many “points” to
monitor, including systems and services
• Dependencies on external services (DNS, Active
Directory, SAN)
• Dependencies within application stack
• Sequencing of operations critical
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

36. So, lets just use PaaS!
$ cf deploy appname
Container
App
A
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

37. Uh,Yeah.
HTTPS
HTTP
Internet
LB/
SSL
CF
Router
UAM
REST Server
Ubuntu 12.04
Ubuntu 12.04
NATS
HM
Cloud Controller
Ubuntu 12.04
C/C: NATS/
HTTP
HTTP/S
Data/Msg Bus
Container
Container
App
A
App
B
Container
Container
App
A'
DEA
App
B'
DEA 1
DEA 1
Ubuntu DEA 1
12.04
Ubuntu 12.04
Ubuntu 12.04
© 2013 Cisco and/or its affiliates. All rights reserved.
VHOST
A
VHOST
B
RabbitMQ
RabbitMQ
CentOS 6.4
CentOS 6.4
Reinhardt Quelle (rquelle@cisco.com)
Keysp.
A
Keysp.
B
Cassandra
Cassandra
CentOS 6.4
Cassandra
CentOS 6.4
Cassandra
CentOS 6.4
CentOS 6.4

38. I’m not going to discuss
application architecture in detail
• Designing for Always-on Services:
• We don’t ever stop serving
customers (intentionally, anyway)
• Avoiding the Fallacies of
Distributed Computing:
Emphasized by load and geography
• Embracing Failure:
At scale, something is ALWAYS
breaking
• Designing for the real world
Circuit breakers, exponential fallback, dynamic configuration
• The need for loosely coupled
services:
Multi-service synchronized updates
are extraordinarily costly
Read Web Operations (Alspaw, et al), Release It! (Nygaard)
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

39. Running Bits in Production
1
Where do we want the bits to run?
Deployment Targets
2
How do they get there?
Packaging/Distribution
3
How are they supposed to
behave?
Configuration
4
How do we start and stop them?
Command/Control (aka
Orchestration)
5
Are we sure they are running (or
ready to run)?
Health Checks and
Verification
6
When will we run into a wall?
Metrics
7
Ow! A wall!
Alerts
8
How did we end up at that wall?
Logs
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

40. Note: “Monitoring” is multifaceted
Where do we want the bits to run?
Deployment Targets
2
How do they get there?
Packaging/Distribution
3
How are they supposed to
behave?
Configuration
4
How do we start and stop them?
Command/Control (aka
Orchestration)
5
Are we sure they are running (or
ready to run)?
Health Checks and
Verification
6
When will we run into a wall?
Metrics
7
Was that a wall we just hit?
Alerts
8
What just happened there?
Logs
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)
Monitoring
1

41. 1
Where does a package
run?
Service ≠ Server
• Multiple servers may run same service
• A single server may run multiple services even several of the same kind (esp. 32bit)
© 2013 Cisco and/or its affiliates. All rights reserved.
Reinhardt Quelle (rquelle@cisco.com)

42. One Service, Multiple
Hosts
LB
Front-End Server
Front-End Server
/etc/passwd
inetd
config
/etc/sysconfig/*
/etc/passwd
inetd
config
/etc/logrotate/*
sshd
config
rsyslog
config
LVM
...
/etc/logrotate/*
sshd
config
rsyslog
config
LVM
/etc/snmp/*
...
/etc/*
....
...
ntp.conf
/etc/snmp/*
/etc/*
resolv.conf
DR:BD
/etc/sysconfig/*
....
resolv.conf
DR:BD
...
ntp.conf
smb.conf
...
Service X
© 2013 Cisco and/or its affiliates. All rights reserved.
config A
config B
smb.conf
...
Service Y
config A
config B
Reinhardt Quelle (rquelle@cisco.com)
Service X
config A
config B
Service Y
config A
config B