EuroSTAR Software Testing Conference 2009 presentation on Evolution of New Feature Verification in 3G Networks by Michael Monaghan. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

1. Evolution of new feature
verification in 3G Networks
Eurostar
Stockholm, December 2009
Michael Monaghan, LM Ericsson (Ireland) Ltd

2. Short overview of LM Ericsson
 Founded 1876, Stockholm, Sweden
 Presence in 175 countries
 > 75,000 employees
 ~24,000 patents
 > 40% of all mobile traffic passes through Ericsson networks
 Ericsson supports networks which serve > 1 billion subscribers
 Business Units
– Networks
– Multimedia
– Global Services
– Sony Ericsson
 In this presentation, we will visit the 3G Mobile Network Development area
– 3G refers to 3rd
Generation, a family of standards for wireless communication
– A glossary of terms appears at the end of this presentation

3. Feature Verification in 3G Networks
 The scenario
– Very complex systems
– Very complex features
– Very difficult to recreate real world environment
– Many levels of test from block test to complex network test
– Many tools needed
 The aim
– To verify that features behave according to requirements
– To make a quality statement about new features, so a decision can
be made about whether to release them
– To satisfy Time to Market and cost goals for the feature

4. Feature Verification in 3G Networks
Element
management
OSS-RC
RNC
RBS
RXI
EM
Operations Support System
Radio & Core
Radio Network Controller
Radio Base Station
Radio Network Integrator
Element Manager
EM
WCDMA: Wideband Code Division Multiple Access, an air interface standard

5. HSPA Evolution
Uplink
20-40 Mbps
12 Mbps
5.8 Mbps
1.4 Mbps
0.4 Mbps
2 ms TTI
16QAM
Downlink
3.6 Mbps
14 Mbps
21 Mbps 28 Mbps
42 Mbps
80-160 Mbps
15 codes
2x2 MIMO64QAM
Multi Carrier
Multi Carrier
64QAM + MIMO
Higher Modulation
Combinations
Multi Carrier
Refer to glossary for explanation of terms...

6. Major Advances in Verification of
3G Network Features
 Evolution from Function Based to Feature Based
Verification
 More generic development leads to verification savings
 Evolution of Feature Verification
 Looking to the Future
 Summary

7. Advances in RNC Feature
Verification
2004 2005 2006 2007 2008 2009 2010
Generic
Design/Test
Strategy
FT to FV
Move
Function
Test
Parallel
Testing
Future
FT: Function Test
FV: Feature Verification

8. Function Test
 RNC verification was done from a functional
viewpoint
 Several functional teams triggered the same test
case and then checked one part of the result
 A test case most often consists of
– Setting up a particular type of call (Control Plane)
– Sending data (User Plane)
– Moving the phone (Mobility)
Function
Test
2004 2005

9. Function Test
Function
Test
2004 2005
Feature1
Feature2
Feature3
Feature4
Feature5
Mobility Team Control Plane Team User Plane Team
Mobility
Mobility
Mobility
Mobility
Control Plane
Control Plane
Control Plane
Control Plane
User Plane
User Plane
User Plane

10. Function Test
 Small area of expertise for each engineer
 Little feature overview knowledge in verification
 Led to rigid test organisations and verification
projects
Function
Test
2004 2005

11. Feature Based Verification
 Verification based on Use Cases
 Verify features end to end
 Verify each Use Case in the most suitable
environment
 Many challenges had to be overcome
FT to FV
Move
2006 2007

12. Feature Verification
Feature1
Feature2
Feature3
Feature4
Feature5
Mobility Team Control Plane Team User Plane Team
Mobility
Mobility
Mobility
Mobility
Control Plane
Control Plane
Control Plane
Control Plane
User Plane
User Plane
User Plane
FT to FV
Move
2006 2007

13. Feature Based Verification
 Feature Verification has led to significant cost
savings
 Very responsive and flexible test organisation
 An enabler for a feature based release strategy
FT to FV
Move
2006 2007

14. Feature Based Verification
 Feature Verification, an evaluation:
Feature #TCs Cost per
Test Case
(hours)
Total
Cost
(hours)
#TCs Cost per
Test
Case
Total
Cost
(hours)
Feature1 168 9 1512 34 8-20 (Avg
14)
476
69%
saving!!
FT to FV
Move
2006 2007
Function Test Feature Verification

15. RAB (Radio Access Bearer)
Concept
Speech
Speech
Data
Data
Speech
plus data
Speech
plus data
Generic Design/
Test
Strategy
2005 2006

16. Generic Design/Test Strategy
 Market pressure for many radio connection combinations
– e.g. different interactive packet RAB rates and combinations
 Development began to be based on generic code units which
made it easier to introduce new RAB rates and combinations
 In P6 the number of RAB combinations exploded : (13 in P3, 18 in
P4, 39 in P5), > 70 in P6, Not possible to verify all transitions
CS Streaming
UDI (CS64)PS Streaming+Int 8/8
SP0
SP64
UDI+Int. 8/8
Int. 64/128
Int. 64/384
64/64
Int. FACH
Speech
SRB
Other int. RAB state
SRB
Int. 128/128
Int. 128/384
128/64
Int. 384/128
Int. 384/384
384/64
HS
SPHS
Generic Design/
Test
Strategy
2005 2006

17. Generic Test Strategy
 Test strategy was to verify the generic code once
and then run a greatly reduced test scope on new
RABs/RAB Combinations going forward
 Reduced scope testing based on RAB type
groups
2005 2006
Generic Design/
Test
Strategy

18. Generic Test Strategy Savings
Feature Pre-Generic
(including all
error and
exception
TCs)
Generic
Test
Principle 1
(Removal of
error and
exception
TC)
Generic
Test
Principle 2
(Rab type
groups)
Generic
Test
Principle 3
(Some
functions
moved
outside FV)
Generic Test
Principle 4
(Functional
based
changes
costs
removed)
Feature2 ~ 196 TC
~ Exe cost
1764 hours
96 TC (-100)
~ Exe cost
864 hours
88 TC (-8)
~ Exe cost
792 hours
58 TC (-30)
~ Exe cost
522 hours
43 TC (-15)
~ Exe cost
387 hours
79% saving!
2005 2006
Generic Design/
Test
Strategy

19. Generic Test Strategy Effectiveness
Feature #TCs run in RNC
Feature
Verification
Faults found
by RNC
Feature
Verification
Faults found
by IODT
(Mobile
phone
Verification)
Faults found
by RAN I&V
Slipthrough
analysis to
RAN I&V
Feature3 102 34 8
Analysis:
Same faults
would have
been found in
RNC but
testing was
done in
parallel
2
Actual TRs
on feature
2
Analysis: 1
due to tools
issues, 1 due
to intro of
fault after test
case had
been run.
 Generic test is very effective when applied correctly
2005 2006
Generic Design/
Test
Strategy

20. More Demanding Environment
 2 releases per year
 Market demand for many and more complex features
(Mobile Broadband)
 Continuous improvements in Time To Market required
 Need for continuous improvements in operational
excellence
 Need to streamline verification further:
– Introduce more parallel testing
– Minimize repetition in testing at all levels
– Maximise test effectiveness
Parallel
Testing
2007 2008

21. Parallel Testing
 RNC node test work more closely with higher layer test
organisations
 Form a RAN Level Joint Test team with participation
from network nodes (RNC, RBS) and RAN
 Make Phased Deliveries for key features
 Split scope of feature test between RAN
and nodes
Parallel
Testing
2007 2008

22. Node/Subsystem I&V Integration
 Subsystem I&V is a very useful, low cost and under-utilised test
environment
 Was Best Effort/Happy Test
 Integration of test activities means node level TCs are passed in a
Subsystem I&V environment
 Improves verification lead times at RNC node level
 Reduces repetition between Subsystem and RNC test
documentation and execution
 Need to minimise quality risk to RNC and Subsystem
Node/
Subsystem
I&V
Merge
2008 2009

23. Maximise Test Effectiveness
 Merge of RNC node feature verification with RAN feature verification
– Pass RAN level test cases in a subsystem verification environment
– Remove RNC node verification as a layer of verification
 Eurostar conference
– Ericsson verification problems are industry-wide
– We compare favourably with other companies in many respects
– However there are some methodologies we should study more
 Risk Based Test Case Selection
– Each test case ranked by likelihood and impact
– Trial was inconclusive
 Estimate number of remaining faults
– Tools exist which predict number of latent faults in a feature
– Too much emphasis now on “slip-through”
 Preventive Testing
Future
2009 2010

24. Summary
 We have come a long way
 Time pressure will continue to mount
 Borders between layers of the development
process are being broken down
Summary
2004 2010

25. Contact Details
 Michael Monaghan, LM Ericsson (Ireland) Ltd
 Michael.A.Monaghan@ericsson.com

26. Glossary
 EM - Element Manager
– A simplified management interface for RNC and RBS nodes
 FT - Function Test
– a test strategy where new features are verified by verifying the functions which realise the
features one by one
 FV - Feature Verification
– a test strategy where new features are verified as a unit
 HSPA - High Speed Packet Access
– a collection of protocols which provide high bandwidth and improved spectral efficiency in 3G
networks
 Interfaces – The communication channel and protocol between network nodes
– Iu - interface between the RNC and the core network
– Iub - interface between the RNC and the Radio Base Station
– Iur - interface between 2 RNC nodes
– Mun – interface between other management systems and OSS-RC
– Mur - operator management system interface towards RNC
– Mut - operator management system interface towards RXI
– Mub - interface from operator management system towards RBS
– Uu - interface between the RBS and the Ue

27. Glossary (contd...)
 I&V - Integration and Verification
– a generic term in Ericsson to refer to all verification activities
 MIMO - Multiple-input and multiple-output
– use of multiple antennas at both the transmitter and receiver to improve communication
performance
 Multi-Carrier
– a method of allowing a mobile device listen to two HSPA data streams from different cells
simultaneously in downlink, thus facilitating higher bandwidth
 OSS-RC – Operation Support System for Radio and Core
– a network management node
 P6 – Project or Phase 6
– Ericsson RAN product releases are numbered P2...Pn
 RAB - Radio Access Bearer
– a UMTS service providing an application using the UMTS network with the ability to send
and receive data over the UMTS network with a specific Quality of Service (QoS).
 RAN - Radio Access Network
– part of a mobile telecommunications system which implements the radio access technology,
so the mobile equipment can communicate with the backbone networks

28. Glossary (contd...)
 RBS - Radio Base Station
– a 3G network node containing radio frequency transmitter(s) and receiver(s) used to
communicate directly with the User Equipment
 RNC - Radio Network Controller
– a network node in a 3G network which controls the base stations which are connected to it.
It also carries out other functions including radio resource management and mobility
management
 RXI – An Access Network Aggregator
– an ATM switch supporting handling of various network topologies and technologies
 TC - Test Case
– For the purposes of this presentation, a test case generally involves establish a specific type
of phone call (voice and/or data)
 Ue - User Equipment
– any device used directly by an end-user to communicate. It can be a hand-held telephone, a
card in a laptop computer, or any other device. It connects to the base station
 WCDMA - Wideband Code Division Multiple Access
– an air interface standard used in 3G networks
 16/64 QAM -16/64 Quadrature amplitude modulation
– methods of coding data for transmission over the air

29. Glossary (contd...)
 2MS TTI - 2MS Transmission Time Interval
– Transmission Time Interval refers to the length of an independently decodable radio
transmission
 3G - 3rd Generation
– a family of standards for wireless communications. Main characteristics are wide range of
services, high throughput and spectral efficiency