The document reports on a study that measured mobile broadband speed and quality of service on the Softbank and KDDI networks in Japan. 11 iPhones measured round-trip time, upload/download speeds, and number of hops over 2 hour periods on January 24th and 31st, 2014 from a classroom. Results showed Softbank's network offered faster average speeds but more volatility, while KDDI's network was more consistent. Significant variations over short time periods meant single speed measurements lacked meaning. The number of TCP streams in download tests did not produce significant statistical differences.

1. A test for
mobile broadband speed
measurement
Toshiya Jitsuzumi
Kyushu University
Jitsuzumi@econ.kyushu‐u.ac.jp
Feb. 12, 2014

2. Test Setting (1/2)
• Installed an original measurement application on 11 iPhones
and measured QoS on a 10 minute interval for 2 hours.
• Date and time
Jan. 24, 2014 16:00‐18:00 (Japan Time)
Jan. 31, 2014 16:00‐18:00 (Japan Time)
• Measurement
A classroom at the 5th floor of the building of
the Faculty of Economics at Kyushu University.
Each iPhone cut Wi‐Fi connection and used only
LTE.
The measurement server was located at Tokyo.

3. Test Setting (2/2)
• Our application conducted QoS measurements every 10
minutes for 2 hours.
• 12 measurement windows for each handset.
• In each measurement window
• RTT was measured 150 times at the interval of 0.2 seconds.
• Download/Upload speed was calculated based on throughputs for
10 seconds.
• Upload speed was measured once.
• Download speeds were measured in three different settings:
one on a single TCP stream, the others are on 3 and 5 TCP
streams.
• The number of hops was counted 3 times using “traceroute.”

4. Result 1: Averages
Date
Median
RTT
(second)
N
Upload
Speed
(Mbps)
Download Speed (Mbps)
Number of
hops
1 TCP stream 3 TCP streams 5 TCP streams
0.0782
2.02
5.78
5.69
5.87
14.62
13
0.1085
0.82
1.91
1.85
1.42
14.08
iPhone5
16
0.0536
3.01
8.92
8.82
9.49
15.06
SoftBank average
95
0.0582
9.84
16.50
18.64
18.81
24.84
iPhone5
69
0.0556
11.01
16.75
18.36
19.10
25.97
iPhone5S
Jan. 31
29
iPhone4S
Jan. 24
26
0.0653
6.72
15.85
19.37
18.02
21.85
17
0.0700
4.92
10.66
11.14
11.10
14.55
iPhone4S
5
0.0970
8.23
8.98
11.60
11.31
13.33
iPhone5
12
0.0588
3.54
11.36
10.95
11.02
15.06
SoftBank average
82
0.0596
15.03
17.32
17.86
18.69
20.82
iPhone5
61
0.0597
14.52
17.90
18.11
20.00
23.62
iPhone5S
21
0.0593
16.50
15.66
17.14
14.87
12.68
KDDI average
KDDI average

5. Result 2: Standard Deviations
Date
Median
RTT
(second)
N
Upload
Speed
(Mbps)
Download Speed (Mbps)
Number of
hops
1 TCP stream 3 TCP streams 5 TCP streams
0.0280
1.64
3.93
4.09
4.61
0.66
13
0.0038
0.08
0.80
0.52
0.53
0.58
iPhone5
16
0.0075
1.64
2.35
2.89
3.02
0.19
SoftBank average
95
0.0133
5.79
8.59
9.85
10.57
5.14
iPhone5
69
0.0119
5.46
7.49
7.01
7.99
3.49
iPhone5S
Jan. 31
29
iPhone4S
Jan. 24
26
0.0142
5.45
10.96
14.93
15.44
7.21
17
0.0258
7.80
8.53
8.61
8.64
1.07
iPhone4S
5
0.0242
13.63
14.49
14.81
14.93
1.29
iPhone5
12
0.0166
1.51
3.72
3.67
3.58
0.18
SoftBank average
82
0.0134
12.97
10.36
11.65
10.70
7.29
iPhone5
61
0.0136
8.99
7.94
9.98
9.12
6.32
iPhone5S
21
0.0127
20.47
15.24
15.49
13.64
1.43
KDDI average
KDDI average

6. Result 4: Coefficient of Variations
Date
Median
RTT
(second)
N
Upload
Speed
(Mbps)
Download Speed (Mbps)
Number of
hops
1 TCP stream 3 TCP streams 5 TCP streams
0.36
0.81
0.68
0.72
0.79
0.05
13
0.04
0.10
0.42
0.28
0.37
0.04
iPhone5
16
0.14
0.55
0.26
0.33
0.32
0.01
SoftBank average
95
0.23
0.59
0.52
0.53
0.56
0.21
iPhone5
69
0.21
0.50
0.45
0.38
0.42
0.13
iPhone5S
Jan. 31
29
iPhone4S
Jan. 24
26
0.22
0.81
0.69
0.77
0.86
0.33
17
0.37
1.58
0.80
0.77
0.78
0.07
iPhone4S
5
0.25
1.66
1.61
1.28
1.32
0.10
iPhone5
12
0.28
0.43
0.33
0.34
0.32
0.01
SoftBank average
82
0.22
0.86
0.60
0.65
0.57
0.35
iPhone5
61
0.23
0.62
0.44
0.55
0.46
0.27
iPhone5S
21
0.21
1.24
0.97
0.90
0.92
0.11
KDDI average
KDDI average

7. Result 5: Median RTT (second)
0.14
16:00‐, Jan. 24, 2014
0.14
16:00‐, Jan. 31, 2014
0.12
0.12
0.10
0.10
0.08
0.08
KDDI‐4S
0.06
KDDI‐5
0.06
SB‐5
KDDI‐5
0.04
SB‐5S
0.04
SB‐5S
0.02
0.02
0.00
0.00
950
1000
1050
TIME(s)
1100
1150
1950
KDDI‐4S
SB‐5
2000
2050
TIME(s)
2100
2150

8. Result 6: Upload Speed (Mbps)
20.0
16:00‐, Jan. 24, 2014
20.0
18.0
18.0
16.0
16.0
14.0
14.0
12.0
8.0
12.0
KDDI‐4S
10.0
KDDI‐5
8.0
SB‐5
6.0
SB‐5S
6.0
4.0
4.0
2.0
2.0
0.0
0.0
1950
16:00‐, Jan. 31, 2014
10.0
950
1000
1050
TIME(s)
1100
1150
KDDI‐4S
KDDI‐5
SB‐5
SB‐5S
2000
2050
TIME(s)
2100
2150

9. Result 7: Download Speed on 1 TCP stream(Mbps)
50.0
16:00‐, Jan. 24, 2014
50.0
40.0
40.0
30.0
16:00‐, Jan. 31, 2014
30.0
KDDI‐4S
KDDI‐4S
KDDI‐5
20.0
SB‐5
SB‐5S
20.0
10.0
KDDI‐5
SB‐5S
SB‐5
10.0
0.0
950
1000
1050
TIME(s)
1100
1150
0.0
1950
2000
2050
TIME(s)
2100
2150

10. Result 8: Download Speed on 3 TCP streams(Mbps)
50.0
16:00‐, Jan. 24, 2014
50.0
40.0
40.0
30.0
16:00‐, Jan. 31, 2014
30.0
KDDI‐4S
KDDI‐4S
KDDI‐5
20.0
SB‐5
SB‐5S
20.0
10.0
KDDI‐5
SB‐5S
SB‐5
10.0
0.0
950
1000
1050
TIME(s)
1100
1150
0.0
1950
2000
2050
TIME(s)
2100
2150

11. Result 8: Download Speed on 5 TCP streams(Mbps)
50.0
16:00‐, Jan. 24, 2014
50.0
40.0
40.0
30.0
16:00‐, Jan. 31, 2014
30.0
KDDI‐4S
KDDI‐4S
KDDI‐5
20.0
SB‐5
SB‐5S
20.0
10.0
KDDI‐5
SB‐5S
SB‐5
10.0
0.0
950
1000
1050
TIME(s)
1100
1150
0.0
1950
2000
2050
TIME(s)
2100
2150

12. Findings
• Softbank’s LTE network offered faster but more volatile speed than
KDDI’s LTE.
• Since the variations in the measurement outputs were significant, a
small number of speed measurements could not produce a
meaningful outcome.
• Throughputs changed quite dramatically in a short amount of time.
• Considering the size of standard deviation, the difference that comes
from the number of TCP streams in the download speed
measurement was not statistically significant, and therefore can be
ignored in a practical sense.