Deploy in minutes your web infrastructure on OpenStack with Saltstack and Salt Cloud extension!

1. CloudParty 2014
Application and Infrastructure deployment
on Cloud Providers
by /
- www.corley.it
Walter Dal Mut @walterdalmut
Corley S.r.l.

2. The Problem
Your web application never scale

3. It is really scalable?

4. Our daily goal?

5. A simple scalable web app

6. Why so many layers?
Manage every layer by it-self
Optimize every layer by it-self
Scale every layer by it-self
Monitor every layer by it-self
Securize every layer by it-self
...

7. The Cloud
Add more resources when we need in seconds
Remove resources when we don't need them anymore
Reduce time to market
Turn a fixed cost into a variable costs
pay only for what you use

8. Deploy & Orchestation
Automate your infrastructure and deployment

9. Distributed Application

11. Split out your Infrastructure
Networks
Subnetworks

12. Security Groups
We will assign a different security-group for every group of VMs. In that
way we can apply our security policy in a simple and powerful way.

13. Security Map
Database layer (MySQL)
PORT 3306 <- from web layer
Cache layer (Memcached)
PORT 11211 <- from web layer
Web layer (Apache2)
PORT 80 <- from proxy layer
Proxy layer (Nginx)
PORT 80 <- from everywhere

14. Security Groups

15. We will use Salt-Cloud
It means that we also need to allow SSH connections from the "master"
Every "minion" has also another security-group "salt-minion" that allows
SSH connections from the "salt-master" instance

16. Salt TOP.SLS
base:
'*':
‐ base
dev:
...
prod:
'proxy.milan.enter.*.prod':
‐ nginx
'web.milan.enter.*.prod':
‐ webserver
‐ webapp
'cache.milan.enter.*.prod':
‐ memcached
'rdb.milan.enter.*':
‐ mysql
‐ mysql.master
'srdb.milan.enter.*':
‐ mysql

17. What about "salt-cloud"
Salt cloud is made to integrate Salt into cloud providers in a clean way
so that minions on public cloud systems can be quickly and easily
modeled and provisioned.
http://salt-cloud.readthedocs.org/en/latest/

18. salt-cloud for OpenStack
We need a Provider definition
enter‐openstack‐config:
minion:
master: 111.111.111.111
identity_url: https://api‐legacy.entercloudsuite.com:5000/v2.0/tokens
compute_name: nova
protocol: ipv4
compute_region: ItalyMilano1
user: name@user.tld
password: YourPassword
tenant: name@user.tld
provider: openstack

19. salt-cloud for OpenStack
We need VMs profiles
rdb:
provider: enter‐openstack‐config
size: e1standard.x4
image: GNU/Linux Ubuntu Server 12.04 LTS Precise Pangolin x64
ssh_username: ubuntu
ssh_key_file: /root/private‐key.pem
ssh_key_name: 'private‐key‐name'
ssh_interface: public_ips
security_groups: salt‐minion,mysql
networks:
‐ fixed:
‐ xxxxxxxx‐xxxx‐xxxx‐xxxx‐xxxxxxxxxxxx
web:
...

20. Automatic IP management
Proxies need Web instaces IPs and so on...

21. Enable Peer communication
allow Salt minions to pass commands to each other
peer:
.*:
‐ .*
We will use this features to share IP addresses
You can use "grains" or "mines" instead

22. Let's go

23. Database layer
Create a new resource with salt-cloud
salt‐cloud ‐p PROFILE VM‐NAME
salt‐cloud ‐p rdb rdb.milan.enter.1.prod

24. Deploy the Master RDB
salt 'rdb.*' state.highstate

25. Create a group of slaves
Can we paralelize all VM creation?
salt‐cloud ‐Pp PROFILE VM‐NAME VM‐NAME ...
"-P" option means "parallel"
salt‐cloud ‐Pp srdb
srdb.milan.enter.1.prod
srdb.milan.enter.2.prod
srdb.milan.enter.3.prod

26. Deploy Slave RDB
salt 'srdb.*' state.highstate

27. Prepare all databases
Now we have a Master instance and 3 slaves
We have to prepare Read-Replicas
CHANGE MASTER TO
MASTER_HOST='xxx.xxx.xxx.xxx',
MASTER_USER='repl‐user',
MASTER_PASSWORD='repl‐pass',
MASTER_LOG_FILE='mysql‐bin‐xxxxx',
MASTER_LOG_POS=xxx

28. Execute MySQL commands
salt 'rdb.milan.enter.1.prod' mysql.query mysql 'show master status'
salt 'srdb.*' mysql.query mysql '
CHANGE MASTER TO
MASTER_HOST="xxx.xxx.xxx.xxx",
MASTER_USER="repl‐user",
MASTER_PASSWORD="repl‐pass",
MASTER_LOG_FILE="mysql‐bin‐xxxxx",
MASTER_LOG_POS=xxx
'

29. Now we have our databases

30. Now the cache layer

31. Add cache resources
salt‐cloud ‐Pp cache
cache.milan.enter.1.prod
cache.milan.enter.2.prod
cache.milan.enter.3.prod
cache.milan.enter.4.prod
salt 'cache.*' state.highstate

32. Memcached will help us with
caching and session
management

33. When we distribute the load
across a group of VMs all
information should be available
to the group otherwise we have
connectivity problems
Cache warm up, disconnected users, and more...

34. Distribute the load

35. The Web Tier
All Web VMs need to know DB and Cache nodes addresses
Master DB address
Slaves DB addresses
Session VMs addresses
Cache VMs addresses

36. Distribute the load

37. Memcached Session handler
# php.ini
session.save_handler = memcached
session.save_path = "192.168.0.5, 192.168.0.6, 192.168.0.7, 192.168.0.8"
{% set memcached_servers = [] %}
{% for server,ip in salt['publish.publish']('cache.*', 'network.interfaces').items() %}
{% set m = memcached_servers.append(ip.eth0.inet[0].address) %}
{% endfor %}
session.save_handler = memcached
session.save_path = "{{ memcached_servers|join(", ") }}

38. Database
The problem: no default multiple connections
MySQLi($host, $username, $password);
//PDO, ...
How to handle multiple connections? Write-Read and Read only?
Master/Slave Async Replication

39. MySQL_ND
Replace libmysql driver
Default connector in PHP 5.4
MySQL_ND Master/Slave (plugin)

40. MySQLND_MS Configuration
{
"myapp": {
"master": {
"master_0": {
"host": "localhost",
}
},
"slave": {
"slave_0": {
"host": "192.168.2.27",
}
}
}
}

41. Configuration in Salt
{
"cwitter.db": {
"master": {
{% for server,ip in salt['publish.publish']('rdb.*', 'network.interfaces').items() %}
"master_{{ server[0] }}": {
"host": "{{ ip.eth0.inet[0].address }}"
}
{% endfor %}
},
"slave": {
{% for server,ip in salt['publish.publish']('srdb.*', 'network.interfaces').items() %}
"slave_{{ server[0] }}": {
"host": "{{ ip.eth0.inet[0].address }}"
}{% if not loop.last %},{% endif %}
{% endfor %}
},
"trx_stickiness": "master"
}
}

42. Transaction Aware
By default MySQLND_MS is not transaction aware
trx_stickiness: master
BEGIN TRANSACTION
INSERT INTO ...
DELETE FROM
SELECT u1, u2, ... FROM ...
UPDATE FROM
COMMIT

43. Now the application distribute
user sessions and DB queries

44. Distribute also HTTP requests!

45. Proxy HTTP/s requests

46. Proxy configuration needs
Pubic IPs
proxy:
provider: enter‐openstack‐config
size: e1standard.x1
image: GNU/Linux Ubuntu Server 12.04 LTS Precise Pangolin x64
ssh_username: ubuntu
ssh_key_file: /root/test.pem
ssh_key_name: 'my key name'
ssh_interface: public_ips
security_groups: salt‐minion,proxy
networks:
‐ fixed:
‐ xxxxxxxx‐xxxx‐xxxx‐xxxx‐xxxxxxxxxxxx
‐ floating:
‐ yyyyyyyy‐yyyy‐yyyy‐yyyy‐yyyyyyyyyyyy

47. NGINX as a proxy
upstream app {
server 192.168.0.10:80;
server 192.168.0.11:80;
server 192.168.0.12:80;
# web server list
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}

48. NGINX proxy with Salt
upstream app {
{% for server,ip in salt['publish.publish']('web.*', 'network.interfaces').items() %}
server {{ ip.eth0.inet[0].address }}:80;
{% endfor %}
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}

49. Use DNS round-robin feature in
order to resolve proxies's IP

50. The app is ready!

51. Thanks for listening
Walter Dal Mut
Github:
Twitter:
Linkedin:
wdalmut
@walterdalmut
Walter Dal Mut