This slides help to understand the Linux performance problems and how to measure the same.

1. Linux Performance Monitoring
Tools
Manosh Malai
CTO, Mydbops
3rd April 2021
Mydbops

2. • Interested in Open Source technologies
• Interested in Performance Engineering, MongoDB,
DevOps & DevOpSec Practices
• Tech Speaker/Blogger
• CTO, Mydbops IT Solution
Manosh Malai
About Me

3. Consulting
Services
Managed
Services
Focuses on MySQL, MongoDB and PostgreSQL
Mydbops Services

4. 400 + Clients In 4 Yrs. of Operations
Our Clients

5. Methodologies
Linux Performance Analysis Tool
Introduction
Agenda

6. INTRODUCTION

7. Goals Of Performance Monitoring
▪ Utilization: How much of a resource is engaged in percentage overtime
▪ Cost: The price/Performance ratio
▪ Throughput: Typicallly data movement in bits or bytes per second
▪ Rate: A transaction/request per second
▪ Latency: Runtime of a query or operation, typically measured in milliseconds

8. Methodologies

9. Anti Method
Street Light Anti-Method
Drunk Man Anti-Method
Blame Someone Else Anti-Method
Pick some familiar observability tools and look for obvious issues
Tune things at random until the problem goes away
Point to other team then sit and relax

10. Performance Methodologies
Workload Characterization
Drill-Down Analysis
USE Method
RED Method

11. Workload Characterization
Who is causing the load(PID, Process Name, UID, IP address)
Why is th load Called?(Trace in code)
How the load changing over time
What is the load(IOPS, Throughput)

12. Drill-Down Analysis
Is a process of breaking problems
Into its basic elements
To find the root cause(s)

13. USE Method
Utilization
Saturation
Error

14. LINUX PERFORMANCE ANALYSIS

15. Simple System Design
CPU
Core Core
LLC
GFX
Memory
Controller
I/O
Controller
SSD
E1000
RAM
PCIe
FSB

16. Linux Performance Observability Tools

17. CPU Observability Tools(My Fav)
▪ top, htop, atop
▪ mpstat -P ALL 1
▪ vmstat 1
▪ execsnoop
▪ runqlen 10 1
▪ runqlat 10 1
▪ runqslower
▪ cpudist (This tool also help to measure Off-CPU Analysis)

18. Memory Observability Tools(My Fav)
▪ free
▪ top or ps per process memory usage
▪ pmap -x <pid>
▪ vmstat 1
▪ sar -B 1

19. File System Observability - Linux Kernel I/O Stack

20. File System Observability - File System Cache
Page Cache VFS
Directory
Cache
Inode
Cache
EXT4 XFS ...
Block Device Interface
Disks
Page
Scanner

21. File System Observability Tools(My Fav)
▪ vfsstat (we can call this VFS Level IOSTAT)
▪ opensnoop
▪ cachestat
▪ cachetop
▪ ext4slower/xfsslower
▪ ext4dist/xfsdist

22. Disk I/O Observability Tools(My Fav)
▪ iostat -dxz 1
▪ iotop
▪ biosnoop
▪ biolatency
▪ biotop

23. Tools installation
▪ https://github.com/iovisor/bcc/blob/master/INSTALL.md
▪ apt-get install ssystat
▪ apt-get install iotop
▪ apt-get install atop

25. Reach Us : Info@mydbops.com
Thank You

26. Thanks to One Person