1. OpenStack Architecture for the Enterprise
Keith Tobin
Cloud Architect

2. Speaker Profile
Keith Tobin
• Dell Cloud Architect
• Located in the Dell Cloud Center of
Excellence in Dublin, Ireland
• Over two decades of software,
platform and cloud development
• Organizer for OpenStack Ireland
• Loves all things OpenStack and
Cloud
2
OpenStack Architecture for the Enterprise
Services

3. Forward & Backward Leaning Applications
Backward leaning
applications are like dogs
Forward leaning
applications are like cows
• They get names like ‘pepper’.
• We care about them, when they get
sick the nurse them back to health.
3
• They get numbers like ‘726’
• When the get sick the are replaced.
OpenStack Architecture for the Enterprise
Services

4. Important Question
• What type of workload will my cloud being running?
a) Forward leaning.
b) Backward leaning.
c) Mixed, both forward and backward.
4
OpenStack Architecture for the Enterprise
Services

5. Not all OpenStack deployments are the same
• Openstack is like a kit
car. It is a set of
building blocks that
can be assembles in
top a cloud.
5
OpenStack Architecture for the Enterprise
Services

6. Design Goals
• Design an OpenStack architecture to run both enterprise and cloud
applications
• Meet enterprise performance expectations for VM’s, networking and storage
• Zero downtime on the control layer
• No data loss in the event of a single node failure
• Must be durable, reliable and highly available
• Must automatically recover after a fault situation has been fixed
• Minimize management tasks
6
OpenStack Architecture for the Enterprise
Services

7. Solution Stack
OpenStack
Titanium
OpenStack
Tested
Solution
Storage
RabbitMQ Cluster For HA
CEPH on R720
RabbitMQ Cluster
Percona MySQL
Highly Available Dynamic
Load Balancing
Dynamic Load balancer
Deployment
SUSE Linux
SUSE Linux
Networking Physical Layer
R620
R720
C6100
7
OpenStack Architecture for the Enterprise
Force10
SonicWall
Servers
Security
Cloud Services
Crowbar
SecureWorks
Managed
Services

8. Deployment
With
Crowbar
8
OpenStack Architecture for the Enterprise
Services

9. Deploying OpenStack with Crowbar Is Easy
• Crowbar boots and installs from ISO on a management node (server)
• Provides user with web UI for configuration and operations management
• Discovers all potential targets (servers) automatically
• Performs hardware configuration
– BIOS configuration
– Performs updates to BIOS and firmware
• Installs OS on targets
• Deploys OpenStack from a set of configuration modules called barclamps
9
OpenStack Architecture for the Enterprise
Services

10. Crowbar Deployment
OpenStack
Database
Controller
Barclamp
Node 01
Identity
Controller
DHCP
Barclamp
Node 02
Image
Controller
Barclamp
Node 03
Block storage
Barclamp
Crowbar
PXE
Node 01
Networking
Barclamp
Nova
Compute
Chef Server
Compute
Node 02
Barclamp
Compute
Horizon
Node 03
Barclamp
10
OpenStack Architecture for the Enterprise
Services

11. Crowbar 2 Status
11
OpenStack Architecture for the Enterprise
Services

12. Network
Infrastructure
12
OpenStack Architecture for the Enterprise
Services

13. Redundant Network Infrastructure
Existing network fabric
F10-Z9000
Force 10 - Z9000
Network
Switch
Force 10 - S4810
Compute
Node
13
Compute
Node
OpenStack Architecture for the Enterprise
Force 10 - S4810
Force 10 - S4810
Compute
Node
Compute
Node
Compute
Node
Force 10 - S4810
Compute
Node
Compute
Node
Services

14. Load
Balancer
14
OpenStack Architecture for the Enterprise
Services

15. Active/Passive (Fault)
(Normal)
Incoming API
Incoming API
Requests
Requests
Server 01
VIP
Server 02
VIP
KeepaliveD
Load
Balancer 01
KeepaliveD
Load
Balancer 02
(Inactive)
(Active)
(Passive)
(Active)
Keystone
etc.
15
OpenStack Architecture for the Enterprise
Keystone
etc.
Keystone
etc.
Services

16. Single HAProxy Bottleneck
Server 01
iVIP
eVIP
Server 02
Inactive LB
(Active LB)
HAProxy
MySQL
MySQL
RabbitMQ
RabbitMQ
Horizon
Horizon
Glance-API
Glance-API
Glanceregistry
Novascheduler
Glanceregistry
Novascheduler
Nova-API
Nova-API
Keystone-API
Keystone-API
Cinder-API
Cinder-API
Neutron-API
Neutron-API
Neutron L2
Neutron L2
Neutron L3
Neutron L3
Neutron
DHCP
16
HAProxy
Neutron
DHCP
OpenStack Architecture for the Enterprise
Services

17. Dynamic Load Balancing
Incoming API Traffic
Network
Network
Switch
Switch
OSPF
ECMP
Server 02
Server 01
Server 03
Quagga
Quagga
HAProxy
HAProxy
Quagga
HAProxy
Keystone
Keystone
Etc….
17
Keystone
Etc…..
Etc……
OpenStack Architecture for the Enterprise
Services

18. Neutron
18
OpenStack Architecture for the Enterprise
Services

19. Neutron (Single Networking Node)
Controller 01
Controller 02
Compute 01
Compute 03
Keystone
Keystone
Nova
Nova
Glance
Glance
MySQL
MySQL
Compute 02
Compute 04
Etc….
Etc….
Nova
Nova
Network 01
Neutron API
L3 Agent
OVS Agent
DHCP Agent
Internet
19
OpenStack Architecture for the Enterprise
Services

20. Neutron (Separate Networking Nodes)
Controller 02
Controller 03
Compute 01
Compute 02
HAProxy
HAProxy
HAProxy
L3 Agent
L3 Agent
Quagga
Quagga
Quagga
DHCP Agent
DHCP Agent
Neutron API
Neutron API
Neutron API
OVS Agent
OVS Agent
Neutron
Scheduler
Neutron
Scheduler
Neutron
Scheduler
Other
OpenStack
Services
Other
OpenStack
Services
OpenStack
Services
Compute 03
Compute 04
L3 Agent
L3 Agent
DHCP Agent
DHCP Agent
OVS Agent
OVS Agent
Controller 01
Ect….
Other
Internet
20
OpenStack Architecture for the Enterprise
Services

21. RabbitMQ
21
OpenStack Architecture for the Enterprise
Services

22. RabbitMQ Cluster (With Mirrored Queues)
Physical View
Logical View
Controller 02
HAProxy
HAProxy
HAProxy
Quagga
Quagga
Quagga
RabbitMQ
RabbitMQ
RabbitMQ
Other
OpenStack
Services
Other
OpenStack
Services
End Point
Controller 03
Other
Ect….
OpenStack
Services
Controller 01
Dynamic Load
Balancer
RabbitMQ 01
22
OpenStack Architecture for the Enterprise
RabbitMQ 02
RabbitMQ 03
Services

23. MySQL
Database
23
OpenStack Architecture for the Enterprise
Services

24. MySQL Single Server (Normal Operation)
Application
DB Server
Data
24
Openstack Architecture for the Enterprise
Services

25. MySQL Single Server (Failed)
Application
DB Server 1
(Active)
Data
25
Openstack Architecture for the Enterprise
Services

26. MySQL Single Server (Failed)
• Disadvantage
• Single point of failure
Application
DB Server 1
(Active)
Data
26
Openstack Architecture for the Enterprise
Services

27. MySQL
Active/Passive
Failover
27
Openstack Architecture for the Enterprise
Services

28. MySQL Active/Passive Failover (Failed Node)
Active/Passive Failover (Normal Operation)
Disadvantages
• Inactive Server
• Limits scale
• Reduces the potential
workload processing
ability of the solution by
half
• Async replication has
potential for data
inconsistencies across
server nodes in fault
situations
28
Openstack Architecture for the Enterprise
Application
DB Server 1
DB Server 2
DB Server 2
(Active)
(Failed)
(Passive)
(Active)
Async Replication
Data
Data
Services

29. MySQL With
DRDB
29
Openstack Architecture for the Enterprise
Services

30. MySQL With DRDB (Normal Operation)
Disadvantages
• Inactive server
• Limits scale
• Reduces the potential
workload processing
ability of the solution
by half
Application
DB Server 1
DB Server 2
(Failed)
Server Not Available !
Event
Data
Data
Event
Confirmed
30
Openstack Architecture for the Enterprise
Services

31. MySQL MultiMaster
31
Openstack Architecture for the Enterprise
Services

32. MySQL Multi-Master Cluster
Application
01
Application
02
Load balancer
DB
Server 1
DB
Server 2
Data
Data
DB
Server 3
Data
32
Openstack Architecture for the Enterprise
Services

33. MySQL Multi-Master Cluster (Failed Node)
Application
01
Application
02
Load balancer
DB
Server 1
DB
Server 2
Data
Data
DB
Server 3
Data
33
Openstack Architecture for the Enterprise
Services

34. MySQL Async Replication
34
Openstack Architecture for the Enterprise
Services

35. MySQL Async Replication
• No guarantee that data
will be consistent across
all nodes at any point in
time
Openstack Architecture for the Enterprise
Server N
Data
35
Server 1
Data
Services

36. MySQL Single Thread replication
Processor
C1
Openstack Architecture for the Enterprise
C3
C4
C5
36
C2
C6
C7
C8
Services

37. MySQL Multi-Master Cluster
Application
01
Disadvantages
• At any point in time there
is no guarantee that data is
in a consistent state within
the cluster
• One thread allocated for
replication tasks
Application
02
Load balancer
DB
Server 1
DB
Server 2
Data
Data
DB
Server 3
Data
37
Openstack Architecture for the Enterprise
Services

38. Percona
MySQL
Cluster
38
Openstack Architecture for the Enterprise
Services

39. Percona MySQL Cluster
Capability
Synchronous Replication
Data Consistency
Parallel Applying on All Nodes
Atomic Node Provisioning
Multi-Master
39
Openstack Architecture for the Enterprise
Services

40. Synchronous Replication
Application
01
Capability
Server 1
Server 2
Data Consistency
Event
Data
Data
Event
Confirmed
40
Openstack Architecture for the Enterprise
Synchronous
Replication
Parallel Applying on All
Nodes
Atomic Node
Provisioning
Multi-Master
Services

41. Data Consistency
Server 1
Server 2
Server 3
Capability
Data
Data
Data
Synchronous
Replication
Data Consistency
Parallel Applying on All
Nodes
At any point in time data is always
consistent across all nodes in the cluster
41
Openstack Architecture for the Enterprise
Atomic Node
Provisioning
Multi-Master
Services

42. Parallel Applying on All Nodes
Processor
C1
C2
C3
C4
C5
C6
C7
C8
Capability
Synchronous
Replication
Data Consistency
Parallel Applying on All
Nodes
Atomic Node
Provisioning
Multi-Master
42
Openstack Architecture for the Enterprise
Services

43. Atomic Node Provisioning
DB
Server 1
New Node
DB
Server 4
Join Cluster
Data
Join Cluster
Data
Capability
Cluster
DB
Server 2
DB
Server 3
Data
Data
Synchronous
Replication
Data Consistency
Parallel Applying on All
Nodes
Atomic Node
Provisioning
Multi-Master
43
Openstack Architecture for the Enterprise
Services

44. Multi-Master Cluster
Application
01
Application
02
Dynamic Load balancer
Capability
DB
Server 1
DB
Server 2
Data
Synchronous
Replication
Data
Data Consistency
DB
Server 3
Data
44
Openstack Architecture for the Enterprise
Parallel Applying on All
Nodes
Atomic Node
Provisioning
Multi-Master
Services

45. Block Storage
45
OpenStack Architecture for the Enterprise
Services

46. Block Storage
CEPH on R720
46
OpenStack Architecture for the Enterprise
Services

47. Logical Architecture
47
OpenStack Architecture for the Enterprise
Services

48. Thank you….
Keith Tobin
Cloud Architect
48
OpenStack Architecture for the Enterprise
Services