How to Use SystemD with Capistrano - the presentation of Alex Simonov prepared for the Ruby Conf LT 2017

1. Капістрано Та СістемД
Олександр Сімонов
CEO Amoniac OU
RubyConfLT 2017

2. Capistrano && SystemD
Oleksandr Simonov
CEO Amoniac OU
RubyConfLT 2017

3. cat /dev/about/me
Software Developer
System Administrator
Engineer
Founder
Team Leader
Speaker

4. Application Server
4 CPUs / 48Gb RAM

5. Application Server Inside
PostgreSQL 9.5
Ruby 2.3.3
2 x Rails 4.2 Applications
Puma Cluster 3x32
Sidekiq 3x25

6. Memory Usage After Boot
PostgreSQL - 500Mb
Puma Cluster - 120Mb
Sidekiq - 100Mb
Total: 500Mb DB + 440Mb Rails App

8. Memory Usage After One Week
PostgreSQL - 550Mb
Puma Cluster - 2Gb
Sidekiq - 1Gb
Total: 550Mb DB + 6GB Rails App

10. Memory Usage After One Month
PostgreSQL - 570Mb
Puma Cluster - 5Gb
Sidekiq - 2,5Gb
Total: 570Mb DB + 15GB Rails App

12. CPU Usage
4 CPUs
5 Nginx processes (master + 4 workers)
8 Puma processes (master + 3 workers )
2 Sidekiq processes
Max 192 PostgreSQL processes (fork on every connection)

13. Puma
PostgreSQL
Sidekiq
Nginx

14. IO Usage
PostgreSQL - multiple DB files, log files
Nginx - log files, requests cache, static files, file uploads
Rails - log files, cache files, file uploads, third party services

15. DiskPostgreSQL
Nginx
Rails
Rails

16. WHAT TO DO?!

18. JAIL
nice/ionice/renice
FreeBSD Jail
Solaris Zones
Linux Control Groups

19. Linux Control Groups
CPU Time
System memory
Disk bandwidth
Network bandwidth

20. Linux Control Groups
CPU Time - max 20% of time
System memory - 1GB max
Disk bandwidth - 5Mb/sec
Network bandwidth - 2Mb/sec

21. SystemD
system and service manager
event logger
user session manager
network interfaces manager
device manager
boot manager

26. systemd + cgroups
Process supervising
Resource management
Dependency management
Logs management

28. Make a reality

29. Make a reality
Read about SystemD

30. Make a reality
Read about SystemD
Read about CGroups

31. Make a reality
Read about SystemD
Read about CGroups
Create service file

32. Make a reality
Read about SystemD
Read about CGroups
Create service file
SSH to server and setup it

33. Make a reality
Read about SystemD
Read about CGroups
Create service file
SSH to server and setup it
SSH to all your servers and do the same

34. Make a reality
Read about SystemD
Read about CGroups
Create service file
SSH to server and setup it
SSH to all your servers and do the same
PROFIT!

36. WE NEED AUTOMATE IT!

37. capistrano-systemd-core
Github: amoniacou/capistrano-systemd-core

38. capistrano-systemd-core
Simple configuration
Only 1 line setup for each server
No need in SUDO
Support any command
CPU, Memory and IO limitations
Puma, Unicorn, Sidekiq and Resque special support in the future

39. Simple configuration
set :systemd_services, {
puma: {
cmd: 'puma -p 3000',
cpu_quota: 70,
memory_max: '1G'
},
sidekiq: {
cmd: 'sidekiq',
cpu_quota: 30,
memory_max: '512M'
}
}

40. Only 1 line setup on server
# loginctl enable-linger username

41. Supported Limitations
CPUQuota
MemoryMax
IOWeight
Slice
More in the future….

43. Questions
GitHub: @simonoff
Twitter: @ceo_simonov
FB: @ceosimonov
Website: https://amoniac.eu