IPQ reduces packet loss (>90% on avergae), reduces TCP retransmssisons (>87% on avergae), and improves user QoE in this test of thin client computing over the WAN.
Remote Display Protocol Performance, Thin Client Computing, and IPQ: Test Report
1. Remote Display Protocol Performance,
Thin Client Computing, and IPQ:
Test Report
Tests conducted and report prepared by
1 February 2011
2. Remote Display Protocol Performance, Thin Client Computing, and IPQ:
Test Report
Contents
Executive Summary
Test Setup
Test Description
Test Data Collection and Evaluation Considerations
Packet Loss
TCP Retransmissions
Quality of Experience MOS (Mean Opinion Score)
Test Results
Packet Loss
TCP Retransmissions
Quality of Experience MOS (Mean Opinion Score)
Report Prepared by Page 2 of 9
3. Remote Display Protocol Performance, Thin Client Computing and IPQ:
Test Report
Executive Summary
Executive Summary
A leading vendor of thin client computing devices engaged IPeak Networks to test the performance of
one of their devices running each of three standard and widely remote display protocols over a lossy
WAN protected by IPeak Networks’ patented QoS software called IPQ.
The test results show that IPQ improved the performance of those protocols in several important ways.
IPQ consistently achieved better than 90% reduction in network packet loss and a substantial mitigation
of the damaging effects of packet loss on the performance of PCoIP, ICA and RDP. On average, IPQ
reduced 1% packet loss to less than 0.08%, 3% packet loss was reduced to less than 0.16%, and 5% was
reduced to less than 0.25%.
IPQ also enabled an average reduction in the numbers of TCP retransmissions (where applicable) of
better than 87% across all test scenarios. The reduction in TCP retransmissions supported welcome
relief from the user experience of performance lag and delays in application responsiveness.
The improvements in the quality of experience were impressive. Remote sessions suffering 3% and 5%
packet loss were essentially unusable without IPQ protection but achieved high Mean Opinion Scores for
quality of experience when IPQ was enabled.
Report Prepared by Page 3 of 9
4. Remote Display Protocol Performance, Thin Client Computing and IPQ:
Test Report
Details and Interpretation
Test Setup
A leading vendor of thin client computing devices engaged IPeak Networks to test the performance of
one of their devices running each of three standard and widely remote display protocols over a lossy
WAN protected by IPeak Networks’ patented QoS software called IPQ. The physical test system and
network setup consisted of an off-the-shelf thin client computing device running Windows XP
Embedded and connected through a Layer 2 switch to a VMWare ESX server located in the IPeak
Networks data center. IPQ protection was added to this system and network setup. At the user end,
IPQ protection was enabled by installing the IPQ 32 bit Windows client on the thin client computing
device. In the data center, an IPQ Virtual Appliance was deployed on the ESX server. In addition, a
virtual switch was configured on the ESX server to ensure that all traffic from the thin client computing
device to the Windows XP virtual machine (VM) was bridged through the IPQ Virtual Appliance, all as
shown in the diagram below.
This test network was a high-quality LAN supporting 100 Mbps links and less than 1ms of latency. To
simulate a typically lossy WAN, IPeak Networks took the following steps:
bandwidth between the IPQ Virtual Appliance and the Windows XP Desktop was limited to 5 Mbps,
30ms of latency was added in each direction, and
varying amounts of packet loss were introduced.
Report Prepared by Page 4 of 9
5. Remote Display Protocol Performance, Thin Client Computing, and IPQ:
Test Report
Details and Interpretation
Test Description
Script-based application testing was used to test PCoIP, ICA and RDP. The script ran through a suite of
four user activities that characterize a common virtual desktop computing use-case:
entering data and graphing the results in an MS Excel spreadsheet,
viewing a MS PowerPoint presentation,
playing a short video on YouTube, and
scrolling through a PDF file in Adobe Reader.
Tests were run with each of the three remote display protocols—first without and then with IPQ
protection enabled—while packet loss was introduced at 0%, 1%, 3% and 5% levels. The generally
available Linux TC network simulation tools were used to introduce the packet loss on the link from the
IPQ Virtual Appliance to the thin client computing device, the direction in which the bulk of data flows in
a virtual desktop deployment.
Test Data Collection and Evaluation Considerations
Packet Loss
Packet loss degrades network performance which in turn degrades application performance and the
user quality of experience. Packet loss data—total average network loss with and without IPQ
enabled—was collected using the logging facilities built in to the IPQ client installed on the thin client
computing device.
TCP Retransmissions
TCP retransmissions increase dramatically in the presence of network packet loss. TCP retransmissions
manifest as delays and input/response lag which degrade the user’s quality of experience of real time
network applications. Where applicable, TCP retransmission data was also captured by running
Wireshark at the Windows XP Virtual Desktop.
Mean Opinion Score (MOS)
Quality of Experience was evaluated was and a Mean Opinion Score (MOS) assigned based on the test
operators’ observations with each combination of protocol, loss rate, user activity, and IPQ status (on or
off). A total of ninety-six scores were recorded.
The IPeak Networks MOS scheme and evaluation criteria are as shown in the table below.
MOS Quality Impairment
5 Excellent Imperceptible
4 Good Perceptible but not annoying
3 Fair Very apparent and annoying
2 Poor Very Annoying
1 Bad Unusable
Report Prepared by Page 5 of 9
6. Remote Display Protocol Performance, Thin Client Computing, and IPQ:
Test Report
Details and Interpretation
Results
Packet Loss
With IPQ protection against packet loss enabled, the reduction in packet loss was significant, with all
three remote display protocols and at all levels of loss. The summarized results are shown in the
following table and graphs.
IPQ OFF IPQ ON
Total Average Total Average Improvement Percentage
Protocol Injected Loss
Network Loss Network Loss Factor Reduction
1% 1.09% 0.06% 18.2 94%
PCoIP 3% 3.18% 0.17% 18.7 95%
5% 4.91% 0.25% 19.6 95%
1% 1.27% 0.08% 15.9 94%
ICA 3% 3.53% 0.08% 44.1 98%
5% 5.58% 0.26% 21.5 95%
1% 1.14% 0.10% 11.4 91%
RDP 3% 3.66% 0.23% 15.9 94%
5% 5.97% 0.24% 24.9 96%
PCoIP
5.00%
4.00%
Actual Loss
3.00%
2.00% IPQ OFF
1.00% IPQ ON
0.00%
1% 3% 5%
Injected Loss
Report Prepared by Page 6 of 9
7. Remote Display Protocol Performance, Thin Client Computing, and IPQ:
Test Report
Details and Interpretation
RDP
6.00%
5.00%
Actual Loss
4.00%
3.00%
IPQ OFF
2.00%
1.00% IPQ ON
0.00%
1% 3% 5%
Injected Loss
ICA
6.00%
5.00%
Actual Loss
4.00%
3.00%
IPQ OFF
2.00%
IPQ ON
1.00%
0.00%
1% 3% 5%
Injected Loss
Report Prepared by Page 7 of 9
8. Remote Display Protocol Performance, Thin Client Computing, and IPQ:
Test Report
Details and Interpretation
TCP Retransmissions
With IPQ enabled, the reduction in TCP retransmissions was significant. The summarized test results are
shown in the following table and graphs.
IPQ OFF IPQ ON
Protocol Loss Level TCP Retrans
TCP Retrans TCP Retrans
Reduction
PCoIP 1% NA NA NA
3% NA NA NA
5% NA NA NA
ICA 1% >500 <150 70%
3% >1700 <150 91%
5% >3000 <250 95%
RDP 1% >500 <100 80%
3% >1900 <100 95%
5% >4000 <250 94%
ICA and RDP TCP Retransmissions
5000
TCP Retransmissions
4000
3000 RDP IPQ OFF
2000 RDP IPQ ON
1000 ICA IPQ OFF
0 "ICA IPQ ON"
0% 1% 3% 5%
Injected Loss
Note that this measurement is not applicable to PCoIP which is a UDP-based remote display protocol.
Report Prepared by Page 8 of 9
9. Remote Display Protocol Performance, Thin Client Computing, and IPQ:
Test Report
Details and Interpretation
Mean Opinion Score (MOS)
The IPeak Networks MOS scheme and evaluation criteria are as shown in the table below.
MOS Quality Impairment
5 Excellent Imperceptible
4 Good Perceptible but not annoying
3 Fair Very apparent and annoying
2 Poor Very Annoying
1 Bad Unusable
When network packet loss was present, the quality of experience was degraded very rapidly, regardless
of protocol. However, when IPQ was enabled, the quality of experience remained very consistent, even
when there was up to 5% loss on the network.
IPQ Off IPQ ON
Protocol Loss Level MOS Quality MOS Quality
PCoIP 1% 3.5 Good 4.75 Excellent
3% 2.5 Fair 4.5 Excellent
5% 2 Fair 4 Good
ICA 1% 3.5 Good 5 Excellent
3% 2.5 Fair 4.75 Excellent
5% 1.5 Poor 4.25 Good
RDP 1% 1.75 Poor 2.5 Fair
3% 1 Poor 2 Fair
5% 1 Poor 2 Fair
Report Prepared by Page 9 of 9