Basic steps for conducting common cloud performance benchmarking analysis

1. Cloud Performance
Benchmarking

2. The Problem Statement
Test Performance of a Compute Instance on the cloud
The testing focuses on disk IO, network IO, and some index of CPU. This
report outlines the testing procedure ,result data and analysis on the findings
Test Performance of a DB Service on the cloud
The testing focuses on database IOPS, QPS, Connection number and others,
This report outlines the testing procedure ,result data and analysis on findings
Application Deployment
Try out WordPress based web application is deployed on cloud. The
deployment involve 1 instance of Compute and 1 Database service. This
presentation outlines the procedure of deployment and any learnings
associated with it etc.

3. Testing Steps

4. Set up
• For the purpose of this tests Alicloud is used
• For compute : An ECS instance with 2
virtual cores and 4GB is used
• For Database an RDS service used. Both
being in the same region.

5. Steps
1
2
3
4
5
6
Launch an ECS and an RDS
instance in the same region
Whitelist RDS for accessing from
the ECS
Install httpd, PHP modules and
Wordpress on the ECS instance.
Configure Database access in
wordpress config file
Run CPU, Memory and IO test on
ECS. Monitor with top and then with
AliCloud default Monitoring
Create Database instance for
Wordpress app, create user for
database access: Using Alicloud
Console
Install Sysbench tool in ECS
instance.
Run IOPS and Connections test for
the RDS.
Test
Report

6. IOPS Test on
Compute Node

7. IOPS on Compute Node: Test Method
FIO utility command is used
# throw-away: 5 min warm-up
fio --runtime=300 --time_based --clocksource=clock_gettime --
name=randread --numjobs=8
--rw=randread --random_distribution=pareto:0.9 --bs=8k --
size=10g --filename=fio.tmp
# Sample 1min runs with varying job threads and block size
fio --runtime=60 --time_based --clocksource=clock_gettime --
name=randread --numjobs=1
--rw=randread --random_distribution=pareto:0.9 --bs=8k --
size=10g --filename=fio.tmp

8. IOPS on Compute Node : Monitor from
Cloud Console

9. IOPS on Compute Node: Results and
Analysis
• Random Writes are optimized at 4k blocks
• Random Reads work best at 8k blocks
• Writes perform better than reads in this test

10. Issues
• IOPs for the writes were observed higher than that of in cloud console monitoring of Alicloud.
This is possibly to due to write buffering ignored by fio
• Increasing number of threads does not increase IOPs. This is possibly due to what is
supported in the virtualized disk controller/driver in the hypervisor.

11. CPU Test on
Compute Node

12. CPU Performance Test on Compute Node :
Testing Steps
Tools Used sysbench and top
sysbench
--test=cpu
--cpu-max-prime=40000
--num-threads=1 run
Top is used to observe peak CPU utilization
in parallel to the sysbench run

13. CPU Performance Test on Compute Node :
Monitor from Cloud Console

14. CPU Performance on Compute Node:
Results & Analysis
# CPU
Events
Number of
Threads
Peak CPU Utilization %
During Test
Total Time
Taken in s
40000 1 50.1 99.86
40000 2 99.5 50.19
40000 3 99.8 50.12
40000 4 99.5 50.13
• No performance anomalies observed. Behaviour expected for CPU with 2 cores
• Comparing the time taken for the same test with other known CPUs the performance can be benchmarked

15. Issues
• There is no direct relation between CPU events / sec and real CPU clock speed. Only relative
estimate indicates benchmarking against other processors
• This test is not suitable for observing occasional CPU bursts / CPU hogging due to shared
nature of virtual instances. That needs real time monitoring tool for a longer period.

16. Network
Performance Test

17. Testing Method
On Server
[root@iZu1iy3602qZ ~]# iperf -s -l 128k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local xx.xx.xx.xx port 5001 connected with xx.xx.xx.xx
port 42490
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-30.0 sec 175 MBytes 48.8 Mbits/sec
On Client
# iperf -c xx.xx.xx.xx -l 128k -i 4 -t 30
------------------------------------------------------------
Client connecting to xx.xx.xx.xx, TCP port 5001
TCP window size: 45.0 KByte (default)
------------------------------------------------------------
[ 3] local 172.31.54.251 port 42489 connected with
xx.xx.xx.xx 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 4.0 sec 23.2 MBytes 48.8 Mbits/sec
[ 3] 0.0-30.2 sec 176 MBytes 49.0 Mbits/sec

18. Results & Analysis
With 1 Connections -> 0.0-30.2 sec 176 MBytes 48.9 Mbits/sec
With 4 Connections -> 0.0-30.2 sec 176 MBytes 49.0 Mbits/sec
• The bandwidth limit observed at 49 Mbps could be a throttling on the client or the server. The server was the ECS instance on Alicloud.
The client was external to Alicloud on another cloud provider.
• Within Alicloud the speed was much better - 18 Gbps

19. RDS Performance

20. Using sysbench OLTP test mode
#Preparation
sysbench --test=oltp --oltp-table-
size=200000 --mysql-host=<host> --
mysql-db=perfdatabase --mysql-
user=perfuser --mysql-
password=password1 --db-driver=mysql
prepare
# Read Test
sysbench --test=oltp --oltp-table-
size=200000 --mysql-host=<host> --
mysql-db=perfdatabase --mysql-
user=perfuser --mysql-
password=password1 --db-driver=mysql -
-max-time=60 --oltp-read-only=on --
max-requests=0 --num-threads=8 run

21. Run 1 : 1 Connection
# Read only Test
read: 55594
write: 0
other: 7942
total: 63536
transactions: 3971 (66.17 per sec.)
read/write requests: 55594 (926.38 per sec.)
other operations: 7942 (132.34 per sec.)
# Read & Write Test
read: 43120
write: 15400
other: 6160
total: 64680
transactions: 3080 (51.33 per sec.)
read/write requests: 394060 (975.24 per sec.)
other operations: 41480 (102.66 per sec.)

22. Run 2 : 4 Parallel Connections
# Read only Test
read: 232890
write: 0
other: 33270
total: 266160
transactions: 16635 (277.18 per sec.)
read/write requests: 232890 (3880.50 per sec.)
other operations: 33270 (554.36 per sec.)
# Read & Write Test
read: 154042
write: 55015
other: 22006
total: 231063
transactions: 11003 (183.34 per sec.)
read/write requests: 209057 (3483.40 per sec.)
other operations: 22006 (366.67 per sec.)

23. Run 3 : 8 Parallel Connections
# Read only Test
read: 471324
write: 0
other: 67332
total: 538656
transactions: 33666 (560.99 per sec.)
read/write requests: 471324 (7853.89 per sec.)
other operations: 67332 (1121.98 per sec.)
# Read & Write Test
read: 290360
write: 103700
other: 41480
total: 435540
transactions: 20740 (345.59 per sec.)
read/write requests: 394060 (6566.21 per sec.)
other operations: 41480 (691.18 per sec.)

24. Run 4 : 16 Parallel Connections
# Read only Test
read: 707126
write: 0
other: 101018
total: 808144
transactions: 50509 (841.57 per sec.)
read/write requests: 707126 (11782.05 per sec.)
other operations: 101018 (1683.15 per sec.)
# Read & Write Test
read: 482804
write: 172430
other: 68972
total: 724206
transactions: 34486 (574.49 per sec.)
read/write requests: 655234 (10915.38 per sec.)
other operations: 68972 (1148.99 per sec.)

25. Run 5 : 32 Parallel Connections
# Read only Test
read: 707126
write: 0
other: 101018
total: 808144
transactions: 50509 (841.57 per sec.)
read/write requests: 707126 (11782.05 per sec.)
other operations: 101018 (1683.15 per sec.)
# Read & Write Test
read: 708680
write: 253100
other: 101240
total: 1063020
transactions: 50620 (843.33 per sec.)
read/write requests: 961780 (16023.35 per sec.)
other operations: 101240 (1686.67 per sec.)

26. Maximum Performance
Eventually the following maximums were reached
Maximum Number of Queries per second (QPS): 22 k
IoPS: Could not measure
Concurrent Connections: Tested upto 600 concurrent connections. After which server threw error
FATAL: error 1040: Too many connections

27. RDS Monitoring Metrics

28. Issues
• Mysqlslap could not be run due to lack of database admin privilege : Error: mysqlslap: Cannot
drop database 'mysqlslap' ERROR : OPERATION need to be executed set by ADMIN.
• IOPs could not be reliably measured without local access to mySQL instance.
• Only query performance could be measured
• The number of parallel connections can be indirectly measured by running netstat | grep
<hostname>| wc -l when the test is going on
• Cloud console monitoring collects data every 300 seconds. For short tests running for about 1
min each, those test data are not reflected in the console monitoring graphs.

29. Default ECS Disk Usage Metrics
IOPS
Bytes / Sec

30. Sample Application
Deployment on ECS
and RDS

31. Wordpress Deployment
• Create WordPress DB in RDS & whitelist the ECS Server
• Create Wordpress DB and DB user in RDS
• Install HTTPD and PHP module on the ECS instance
• Configure Wordpress to use the RDS BD information
• Following this use apache ab tool performance test the wordpress site for concurrent
requests, max TPS and latency measures.
ECS Instance
httpd
PHP Module
Wordpress (php)
RDS Service
Wordpress DB
Connection
Whitelisting

32. Thank You