3 test solutions for 4 g devices in r&d or conformance environments
1. Test Solutions for 4G devices
in R&D or Conformance
environments
Roberto Monteiro
Account Manager (Rohde & Schwarz do Brasil)
I International Workshop on
Challenges and Trends for
Broadband Mobile Networks –
Beyond LTE-A
2. Standards
Different standards, study groups, and different applications
Core specification
Test specification
Work Items (WI)
Novembro 2013 | Roberto Monteiro | 2
3. Test needs
Mobile comms, Non-mobile comms, all together today
HSPA
Non-Cellular
HSPA+
1xEV-DO
0AB
WCDMA
DC
HSDPA
Cellular
TDSCDMA
Mobile
WiMAX
CDMA2000
1xRTT
EDGEevo
VAMOS
LTE-FDD
LTE-TDD
GSM
(E)GPRS
Wifi
Offloading
FM Stereo
ISDB-Tb
GPS
Bluetooth
Glonass
BT EDR
Galileo
BT LE
Novembro 2013 | Roberto Monteiro | 3
WLAN
abgn
4. WiFi OffLoading
WLAN <-> LTE
EPC
Internet
3GPP
Access
Wi-Fi
Access
Handover
seamless,
individual flows
EPC
Internet
3GPP
Access
UE
Wi-Fi
Access
UE
VoIP
Video Conf.
Novembro 2013 | Roberto Monteiro | 4
Web
FTP
5. Test needs
Mobile comms, Non-mobile comms, all together today
HSPA
Non-Cellular
HSPA+
1xEV-DO
0AB
WCDMA
DC
HSDPA
Cellular
TDSCDMA
Mobile
WiMAX
CDMA2000
1xRTT
EDGEevo
VAMOS
LTE-FDD
LTE-TDD
GSM
(E)GPRS
Wifi
Offloading
FM Stereo
ISDB-Tb
GPS
Bluetooth
Glonass
BT EDR
Galileo
BT LE
Novembro 2013 | Roberto Monteiro | 5
WLAN
abgn
6. Test needs
Mobile comms, Non-mobile comms, all together today
HSPA
1xEV-DO
0AB
Mobile
WiMAX
CDMA2000
1xRTT
EDGEevo
VAMOS
LTE-FDD
LTE-TDD
GSM
(E)GPRS
LTE-R10
2CC 2x2
MIMO
FM Stereo
ISDB-Tb
Non-Cellular
HSPA+
WCDMA
DC
HSDPA
Cellular
TDSCDMA
GPS
Bluetooth
Glonass
BT EDR
Galileo
BT LE
Wifi
Offloading
Novembro 2013 | Roberto Monteiro | 6
WLAN
abgn
9. Test needs
Mobile comms, Non-mobile comms, all together today
HSPA
1xEV-DO
0AB
Mobile
WiMAX
CDMA2000
1xRTT
EDGEevo
VAMOS
LTE-FDD
LTE-TDD
GSM
(E)GPRS
LTE-R10
2CC 2x2
MIMO
FM Stereo
ISDB-Tb
Non-Cellular
HSPA+
WCDMA
DC
HSDPA
Cellular
TDSCDMA
GPS
Bluetooth
Glonass
BT EDR
Galileo
BT LE
Wifi
Offloading
Novembro 2013 | Roberto Monteiro | 9
WLAN
abgn
10. Test needs
Mobile comms, Non-mobile comms, all together today
HSPA
1xEV-DO
0AB
LTE-FDD
LTE-TDD
GSM
(E)GPRS
LTE-R10
2CC 2x2
MIMO
FM Stereo
GPS
BT EDR
Galileo
BT LE
Wifi
Offloading
Bluetooth
Glonass
FADING
Mobile
WiMAX
CDMA2000
1xRTT
EDGEevo
VAMOS
ISDB-Tb
Non-Cellular
HSPA+
WCDMA
DC
HSDPA
Cellular
TDSCDMA
Novembro 2013 | Roberto Monteiro | 10
WLAN
abgn
11. BLER results with and without fading
Novembro 2013 | Roberto Monteiro | 11
12. Test needs
Mobile comms, Non-mobile comms, all together today
HSPA
1xEV-DO
0AB
LTE-FDD
LTE-TDD
GSM
(E)GPRS
LTE-R10
2CC 2x2
MIMO
FM Stereo
GPS
BT EDR
Galileo
BT LE
Wifi
Offloading
Bluetooth
Glonass
FADING
Mobile
WiMAX
CDMA2000
1xRTT
EDGEevo
VAMOS
ISDB-Tb
Non-Cellular
HSPA+
WCDMA
DC
HSDPA
Cellular
TDSCDMA
Novembro 2013 | Roberto Monteiro | 12
WLAN
abgn
13. Test needs
Mobile comms, Non-mobile comms, all together today
HSPA
1xEV-DO
0AB
LTE-FDD
LTE-TDD
GSM
(E)GPRS
LTE-R10
2CC 2x2
MIMO
FM Stereo
GPS
BT EDR
Galileo
BT LE
Wifi
Offloading
Bluetooth
Glonass
FADING
Mobile
WiMAX
CDMA2000
1xRTT
EDGEevo
VAMOS
ISDB-Tb
Non-Cellular
HSPA+
WCDMA
DC
HSDPA
Cellular
TDSCDMA
Novembro 2013 | Roberto Monteiro | 13
WLAN
abgn
14. Test needs
Mobile comms, Non-mobile comms, all together today
HSPA
1xEV-DO
0AB
LTE-FDD
LTE-TDD
GSM
(E)GPRS
LTE-R10
2CC 2x2
MIMO
FM Stereo
GPS
BT EDR
Galileo
BT LE
Wifi
Offloading
Bluetooth
Glonass
FADING
Mobile
WiMAX
CDMA2000
1xRTT
EDGEevo
VAMOS
ISDB-Tb
Non-Cellular
HSPA+
WCDMA
DC
HSDPA
Cellular
TDSCDMA
Novembro 2013 | Roberto Monteiro | 14
WLAN
abgn
15. End to End (E2E) Data and Call
Performance
l Crucial for user experience of data services
l Verify quality of data and voice services
l
Service availability
l At cell borders
l
During handovers
Good service ideas may fail
when the required performance
is not available
Novembro 2013 | Roberto Monteiro | 15
16. Testing UE Performance with Realistic
Radio Conditions
LTE CA BTS
3G BTS
LTE BTS
Throughput and call performance
measurement with
l
Noise (AWGN, OCNS)
l
Fading
l
During HO
Novembro 2013 | Roberto Monteiro | 16
17. Performance testing of Applications
l Web page Loading
l Video quality testing
l Audio performance testing
CMW-PQA
Novembro 2013 | Roberto Monteiro | 17
VTE Video
Analyzer
UPV Audio
Analyzer
18. Voice over LTE (VoLTE) testing
Different test setups
R&S®CMW-ATE
LTE attach procedure
IMS terminal testing
Session Initiation Protocol
(SIP)
Conformance Testing
Audio: Echo verification in
loopback mode
R&S®CMW-PQA
Functional
testing
LTE attach procedure
IMS registration
Audio: Echo verification in loopback mode
IP Traffic Logging
Impact of noise, IP traffic impairment
Automation via R&S CMWrun sequencer
tool
Audio Quality
testing
R&S®CMW500 LTE Call-box
R&S®UPV Audio Analyzer
Verify audio quality using
PESQ® and POLQA®
algorithm
Acoustic analysis:
3GPP TS 26.132
Performance
testing
Automated performance testing
impacts of noise, fading, IP traffic impairments on
audio quality
R&S®UPV Audio Analyzer
Novembro 2013 | Roberto Monteiro | 18
1
19. Image quality analysis with the R&S VTE
H.264/MPEG4AVC
1080p
R&S talking head
2 MBit/s
ı SSIM: 0.9821
ı PSNR: 38.13 dB
ı MOSV: 85.14%
50 kBit/s
ı SSIM: 0.7812
ı PSNR: 25.68 dB
ı MOSV: 22.32%
Novembro 2013 | Roberto Monteiro | 19
20. What is the picture failure point?
Error free
ı C/N > 18 dB
ı BER <1.40e-03
C/N = >18,00 dB
1,2
1
SSIM
0,8
0,6
0,4
0,2
0
0
10
20
30
40
50
60
t in s
C/ N = 17,60 dB
Picture failure point
ı C/N 17,60 dB
ı BER 3.30e-03
1,2
1
SSIM
0,8
0,6
0,4
0,2
0
0
10
20
30
40
T in s
50
60
(BER = after Viterbi)
Σtime(PSNR drop) > 120ms = picture failure point
Novembro 2013 | Roberto Monteiro | 20
21. Influence of SNR on picture failure
Novembro 2013 | Roberto Monteiro | 21
22. LARGE NUMBER OF TESTS DEMAND…
AUTOMATION
REPEATABILITY
FIDELITY
Novembro 2013 | Roberto Monteiro | 22
25. BUT SOMETIMES A CALLBOX IS NOT
ENOUGH…
PROTOCOL TESTS…
DEEP ANALYSIS
OR SIMPLY…..
Novembro 2013 | Roberto Monteiro | 25
26. Protocol tests
l R&D and verification
l Conformance Testing
Testing
l MLAPI / LLAPI
l ETSI
Medium / Lower Layer Application
Programming Interface
LLAPI:
PHY Layer
Link Layer
MAC Layer
MLAPI
RRC Layer
Layer 3
DAU
Upper layers
Novembro 2013 | Roberto Monteiro | 26
European Telecommunications Standards
Institute
l TTCN
Testing and Test Control Notation
TTCN-1 (no longer used)
TTCN-2 (still widely used today)
TTCN-3 (current version and “universal”)
27. Protocol Test with MLAPI or TTCN
LLAPI
Testcase
MLAPI
Testcase
CMW500 LTE
Protocol Stack
TTCN
Testcase
NAS
RRC
Layer 3 MLAPI
Layer 2 LLAPI
PDCP
TTCN API
RLC
MAC
PHY
ı Testcase created in C++ (MLAPI, IOT) or TTCN (Conformance)
ı Focus on UE signalling procedures
Novembro 2013 | Roberto Monteiro | 27
TTCN
Runtime
Adapter
28. CMWcards – Closing the gap
RF Tester
„CallBox“
Protocol Tester
„CMWcards
Signaling Tester“
Novembro 2013 | Roberto Monteiro | 28
29. CMWcards – Closing the gap
high
„CMWcards
Signaling Tester“
Ease of Use
RF Tester
„CallBox“
CMWcards
Protocol Tester
low
Flexibility
Novembro 2013 | Roberto Monteiro | 29
high
30. CMWcards – Key Features
l Inter-RAT capabilities
l LTE WCDMA
l LTE GSM
l WCDMA GSM
l Triple-RAT LTE / WCDMA / GSM
l LTE CDMA2000
LTE cells /
WCDMA cells
l Multi-cell support
l 2 cells on CMW500 standalone
l 4 cells on CMW500 Multibox
0
2
3
4
Single
0
1
Single
Multi
Multi
Single
Single
Multi
Multi
-
2
Single
Multi
Multi
-
-
3
Multi
Multi
-
-
-
4
Novembro 2013 | Roberto Monteiro | 30
1
Multi
-
-
-
-
31. CMW500 Use Cases
RF Tester
„CallBox“
Protocol Tester
„CMWcards
Signaling Tester“
RF parametric
tests
BLER
measurements
RMCs
Protocol
Conformance
IOT
Acceptance
R&D
LLAPI/MLAPI
scenarios
Dynamic
behaviour
PRACH
Tx/Rx
measurements
TTCN2/3 R&D
development
L1 – L3 and
above
debugging
Multicell
scenarios > 2
cells
Ack/Nack
measurements
Basic
IMS/VoLTE
UL Power
Control
Inter-RAT
scenarios
Triple-RAT
…
Transmit Power
Control
…
…
…
…
…
Novembro 2013 | Roberto Monteiro | 31
32. CMWcards Use Cases
„CMWcards
Signaling Tester“
• LTE, WCDMA (GSM and CDMA2000 planned) in one
GUI
• LTE Rel-10 carrier aggregation supported
• Basic registration and attach procedures
• Attach Reject and other failure causes
• Security and ciphering
• Intraband and interband cell
selection/reselection/handover
• Inter-RAT cell
selection/reselection/handover/redirection
• IMS and VoLTE including CS fallback
• SRVCC (planned)
• LTE and WCDMA Throughput Tests
Novembro 2013 | Roberto Monteiro | 32
33. Conformance
36.521 / 36.523
l Protocol Conformance (3GPP 36.523)
l Up to 8 cells (2G/3G/4G) at
sametime
l Or 2xCC 2x2 MIMO LTE R10 + 1x 2G + 1x 3G + 1x 4G
2x2 MIMO
l No extra Hardware
required!!!
l And you can use your
existing CallBox hardware
Novembro 2013 | Roberto Monteiro | 33
34. The road to Full Type Approval System
RF Full Type Approval System (3GPP 36.521)
TS80CON
N
CMW500
TS-PQA
+real time fading
TS8980S
TS-RRM
TS8980IB
TS8980FTA
+Multi-RAT
+self-calibration
for max. RF accuracy
+ out-of-band and
TV interferer tests
+interferers
RF tests
Protocol conformance / R&D
Performance Quality Analysis (PQA)
RF precompliance
RRM conformance
RF conformance
Network operator suppl. RF conformance
e.g. Verizon, AT&T, USCC, Sprint
Novembro 2013 | Roberto Monteiro | 34
35. AND LAST BUT NOT LEAST…
Novembro 2013 | Roberto Monteiro | 35
36. What do we do about “Beyond 4G / 5G”?
Novembro 2013 | Roberto Monteiro | 36