1. An I t d ti t
A Introduction to
Wireless Small Ce Ne wo s
W e ess S Cell Networks
Mehdi Bennis and Walid Saad
University of Oulu, Centre for Wireless Communications, Finland
Electrical and Computer Engineering Department, University of
Miami,
Miami USA
http://www.cwc.oulu.fi/~bennis/ http://resume.walid-saad.com 1
bennis@ee.oulu.fi walid@miami.edu

2. Outline
• Part I: Introduction to small cell networks
– Introduction and key challenges
• Part II: Network modeling
– Baseline models and key tools (stochastic geometry)
• Part III: Interference management
– Interference in a heterogeneous, small cell environment
– Emerging techniques for interference management
• P IV: T
Part IV Toward self-organizing small cell networks
d lf i i ll ll k
– Introduction to game theory and learning
– Applications in small cells
• Part V: Conclusions and open issues 2

3. Part I
Introduction to Small Cell Networks
3

4. Outline
4/120

5. Outline
5/120

6. What happens in one
hour?
Around the globe, in one hour:
– 685 million sms messages
– 128 million Google searches
– 9 million tweets
pp
– 1.2 million mobile apps downloaded
– 2880 hours of YouTube videos uploaded
– 50,000 smart phones activated
We need innovative network designs to
handle all of this!
6

7. Technology Convergence
Technology Convergence
Computing
C i
Wireless services
Digital imaging Gaming
TV and video
7

8. Main Implications
Main Implications
• Operators dilemma
– Meet the demand and maintain low costs (i.e., revenues an issue)
• Need to decrease the expenditure per bit of data (to avoid
uglier alternatives such as limiting usage)
• Solutions that have been explored in the past few years
– Multiple antenna systems and MIMO
• Cannot provide order of magnitude gains
• Scalability and practicality issues
– Cognitive radio
• Availability of white spaces in major areas at peak hours is questionable
• MIMO and Cognitive radio will stay but must co-exist along
with better, more scalable, and smarter alternatives
• Is there any better, cost-effective solution? 8

9. Small Cell Networks – A Necessary Paradigm Shift
Facts
F
Consumer behaviour
is changing
• Operators face an unprecedented increasing demand for mobile
data traffic - More devices, higher
• 70-80% volume from indoor & hotspots already now bit
bi rates, always active
l i
- Larger variety of
• Mobile data traffic expected to grow 500-1000x by 2020 traffic types e.g. Video,
• 1000-times mobile traffic is expected in 2020 to 2023 MTC
• Sophisticated devices have entered the market
• Increased network density introduces Local Area and Small Cells
• 2011, an estimated 2.3 million femtocells were already deployed
globally, and this is expected to reach nearly 50 million by 2014
Ultimately, the only viable way of reaching “the promised land”
is making cells smaller, denser and smarter
Macrocell
9
Small Cells/Low power Nodes

10. In a nutshell….
• Heterogeneous (small cell) networks operate on licensed spectrum owned by the
mobile operator
• Fundamentally different from the macrocell in their need to be autonomous and self-
organizing and self-adaptive so as to maintain l
i i d lf d i i i low costs
• Femtocells are connected to the operator through DSL/cable/ethernet connection
• Picocells have dedicated backhauls since deployed by operators
• Relays are essentially used for coverage extension
• Heterogeneous (wired,wireless, and mix) backhauls are envisioned
@ London’s
Hotpost Lamp Post Solar panel Olympics Games
10

11. In a nutshell….
D2D Femto-BS
Characteristics
• Wired backhaul
Relay
R l Characteristics
• Resource reuse • User-deployed
• Closed/open/hybrid
• Operator‐assisted
Characteristics Major Issues access
• Wireless backhaul • Neighbor discovery Major Issues
• Open access • Offloading traffic • Femto-to-femto
•OOperator‐deployed
d l d interference and femto-to-
Major Issues macro interference
• Effective backhaul design
• Mitigating relay to macrocell
interference D2D
backhaul
b kh l
Macro-BS Femto-BS
Relay
Pico-BS
Characteristics
• Wired backhaul
• Operator‐deployed
• Open access
Macrocells: 20 40 watts
ll 20-40 Major Issues
M j I
•Offloading traffic from
macro to picocells
(large footprint) • Mitigate interference
toward macrocell users
11

12. Standardization Efforts
• Small Cell Forum (formerly Femto-Forum) is a governing
body with arguably most impact onto standardization bodies.
• Non-profit membership organization founded in 2007 to
enable and promote small cells worldwide.
• Small Cell Forum is active in two main areas:
1) standardization, regulation & interoperability;
2) marketing & promotion of small cell solutions
Next Generation Mobile Networks (NGMN) Alliance:
• Created in 2006 by group of operators
• Business requirements driven
• Often based on use‐cases of daily networking routines
• Heavily related to Self-Organizing Networks (SON) activities 12

13. Small Cell Access Policies
• Three access policies
• Closed access:
 only registered users b l
l i d belonging to a closed subscriber group (
i l d b ib (CSG) can
)
connect
 Potential interference from loud (macro UE) neighbors
• Open access:
 all users connect to the small cells (pico/metro/microcells)
 Alleviate interference but needs incentives for users to share their access
• Hybrid access:
 all users + priority to a fixed number of femto users
 Subject to cost constraints and backhaul conditions
j
• Femtocells are generally closed, open or hybrid access
• Picocells are usually open access by nature and used for offloading macrocell
traffic and achieving cell splitting gains.
13

14. Small Cells vs. WiFi
Friends or Foes?
• Recent trials using a converged
- Deployed to improve network coverage and
gateway Wi-Fi/3G architecture
improve capacity (closed access)
showed how the technologies g
- There i considerable planning effort f
Th is id bl l i ff t from th
the
could be combined and exploited
operator in deploying a femtocell network
- Prediction: there will be more small cells than
• Several companies are likely to
devices! (Qualcomm CTW 2012)
simultaneously introduce both
y
technologies for offloading.
- A cheap alternative for data offloading
- Availability f Wi-Fi
A il bili of Wi Fi networks, hi h d
k high data rates
and lower cost of ownership has made it
attractive for catering to increasing data demand  Small cells vs. Wi-Fi:
- However, seamless interworking of Wi-Fi and - Managed vs. Best effort
mobile networks are still challenging
bil t k till h ll i - Simultaneously push
both technologies for
offloading
Open Problem
How to combine and integrate 3G/4G/Wi-Fi in a cost effective manner? 14

15. The Backhaul – a new bottleneck
• The backhaul is critical for small cell base stations
• Low-cost backhaul is key!
• What is the best solution?
• Towards h
d heterogeneous small cell b kh l options
ll ll backhaul i
• Conventional point-to-point (PtP):
•  high capacity
•  coverage, spectrum OPEX, high costs
• E-band (spectrum available at 71-76 and 81GHz)
•  high capacity
•  high CAPEX and OPEX
• Fib (l
Fiber (leased or b ilt)
d built)
•  high capacity
•  recurring charges, availability and time to deploy
• Non-Line of sight (NLOS) multipoint microwave
g ( ) p
•  good coverage, low cost of ownership
•  low capacity, spectrum can be expensive
+ possibly TV White Space...
 Milimeter wave backhaul currently a strong potential
Milimeter-wave
 Proactive caching ~30-40% savings (source: Intel)
15
Sub 6 GHz Point-to-Multipoint Backhaul Links

16. Summary of Challenges
Summary of Challenges
Radio resource management and Inter-cell
interference coordination
i t f di ti
Modeling and analysis
Self-organization, self-optimization
g p
Security Self-healing
And many more..
Backhaul-aware RRM for small cell networks
Handover and mobility management
Energy Efficiency and
power savings (green small cells) Intra-RAT offloading, inter-RAT offloading
( g
(tighter coordination)
)
Cell association and
load balancing
16

17. Summary of Challenges
Summary of Challenges
• Dense and ad hoc deployment -> new network models
• How to manage interference?
– Key to successful deployment of small cells
• How can we design the small cells in a way to co-exist with the
mainstream wireless system?
– Most critically, mobility and handover
• What is the best backbone to support the small cells?
– Small cells’ performance can be degraded when the backhaul is being
used by other technologies (e.g. WiFi or home DSL)
• How can we handle dense deployments?
• What about energy efficiency?
• Ultimately,
Ultimately can we have a multi-tier wireless network that is
multi tier
built in a plug-and-play manner?
17

18. Challenges in SCNs –
Radio Resource Management and Inter-cell interference coordination
Macro-BS DL Macro-BS
UL
Macro UE

Macro UE Small cell UE Small cell UE

Small cell BS Small cell BS
Aggressor/Victim: small cell/macro Aggressor/Victim: macro/small cell
• DL interference from the small cell BS to nearby Macro UE • UL interference from nearby macro UE to small cell BS
• A Macro UE far from its MBS will be affected the most • A macro UE far from its MBS causes interference toward the
small cell
Macro UE inside / near femto coverage
18

19. Challenges in SCNs –
Radio Resource M
R di R Management and I t
t d Inter-cell i t f
ll interference coordination
di ti
Macro-BS DL Macro-BS
UL
Small cell BS  Small cell BS

Small cell UE Small cell UE
Macro UE Macro UE
Aggressor/Victim: macro/small cell Aggressor/Victim: small cell/macro
• DL interference from nearby Macro-BS to small cell UE • UL interference from small cell UE to nearby Macro-BS
• Interference from nearby Macro-BS can lower SINR of • Many active small cell UEs can cause severe interference to the
small cell UE Macro BS
Macro-BS
Small cell very close to Macro base station
19

20. Challenges in SCNs –
Radio Resource M
R di R Management and I t
t d Inter-cell i t f
ll interference coordination
di ti
Macro-BS DL Macro-BS
UL
Small cell BS Small cell BS
Small cell BS Small cell BS
Macro UE Macro UE
Aggressor/Victim: small cell/small cell Aggressor/Victim: small cell/small cell
• DL interference among nearby small cell networks • UL interference among nearby small cell networks
(co-tier) interference among small cell networks
20

21. Challenges in SCNs –
Mobility management and handover
Mobility h
M bilit enhancement for
tf
traffic offloading
Enhancement of small cell discovery is
needed for offloading to small cells
standard macrocell HO parameters are
obsolete
SON enhancements for HetNet
 How to control the mobility with SON
features needs to be studied?
 How long to wait ? What is the
threshold? etc
 disruptive to standard scheduling
LPN LPN Macro LPN
Too late HO
• UE mobility is faster than the HO parameter settings Too early HO Wrong cell HO
21
• HO triggered when the signal strength of the source cell is too low

22. Challenges in SCNs –
Self-Organizing Networks (SONs)
S lf O i i N t k (SON )
• Traditional ways of network optimization using
manual processes, staff monitoring KPIs, maps,
trial and errors ..........is unreasonable i SCNs!
i l d i bl in
• Self-organization and network automation is a
necessity not a privilege. Why?
• Femtocells (pico) are randomly (installed)
deployed by users (operators)
need fast d lf
 d f t and self-organizing capabilities
i i biliti
• Need strategies without human intervention
• Self-organization helps reduces OPEX
• Homogeneous vs. Heterogeneous deployments
g g p y
every cell behaves differently
Individual parameter for every cell
SON is crucial for enhanced/further enhanced-ICIC, mobility
management, load balancing, etc.. 22

23. Challenges in SCNs –
Energy Effi i
E Efficiency
• Green communications in HetNets requires redesign at each level. Why?
• Simply adding small cells is not energy-efficient (need smart mechanisms)
• Dynamic switch ON/OFF for small cells
• Dynamic neighboring cell expansion based on cell cooperation
Dynamic neighboring
cell expansion
ll i Dynamic cell ON/OFF
Switch OFF
Macro-BS Macro-BS
Small
Cell range expansion cell
Switch OFF for power savings Small
cell
Active Mode
Energy harvesting is also a nice trait of HetNets!
e.g., autonomous network configuration properties 23
converting ambient energy into electrical during sleep mode

24. Part II
Nework Modeling in
Small Cell Networks
24

25. Current Cellular Models
Developing analytically tractable models for cellular
systems is very difficult
• Stochastic Geometry (StoGeo) has been used
in
i cellular networks with h
ll l t k ith hexagonal b
l base
station model, i.e., macrocell base stations
(grid-based).
Wyner model was predominantly used in the 1990’s
• Too idealized; used in Information Theory (IT)
• used in Academia for tractability and analysis
With advent of heteregeneous and dense small cell
networks, random and spatial models are needed
• HHexagonal models f i l obsolete
l d l fairly b l
• Need to model HetNets to characterize
performance metrics (Operators want pointers!!)
Operators pointers
• Transmission ,
rate, coverage,
g, outage
g
probability
• Ease of simulation
25
Source: J. Andrews, keynote ICC Smallnets, 2012.

26. Current Cellular Architectures
Nuts and Bolts
• How to model and
analyze multi-tier
wireless networks?
• How to characterize
interference?
• How to derive key
metrics such as coverage
probability, spectral
efficiency etc?
26

27. Baseline Downlink Model (1-tier)
coverage
probability
Aggregate interference at tagged receiver
......First, let us look at the coverage probability in a 1-tier setting
27

28. Coverage Probability (1-tier)
Where
Incredibly simple expressions
28
Source: J. Andrews, keynote ICC Smallnets, 2012.

29. How accurate is this model?
• Fairly accurate, even for
traditional
di i l planned
l d
cellular networks.
• Industry is somewhat
reluctant to use these
models due to possible
difficulty in system level
simulations
29

30. Moving on to K-tier Hetnets
Aggregate interference at tagged receiver 30

31. K-Tier Small Cell Networks
Theorem 2 [Dhillon, Ganti, Bacelli ’11]: The coverage probability for a typical
mobile user connecting to the strongest BS, neglecting noise and assuming Rayleigh
fading:
Key assumption!
• Single tier cellular network (K=1):
Only depends on SIR target and path loss
• K-tier network with same SIR threshold for all tiers (practical?)
Interestingly,
Interestingly same as K=1 tier
K 1 tier.
Neither adding tiers nor base stations changes
the coverage/outage in the network!
- Network sum-rate increases linearly with number of BSs 31
Source: J. Andrews, keynote ICC Smallnets, 2012.

32. How accurate is the K-tier model?
Source: J. Andrews, keynote ICC Smallnets, 2012. 32

33. Summary
• How good is the Poisson assumption?
• Femtocells: deployments fairly random but distribution is known
• Macrocells: have some structure but definitely not grid-like
• Picocells: some randomness due to the deployment at hotspots
• How good is the independence assumption?
• Femtocells: fairly good since users typically don’t know the locations of operator
deployed towers
• Pi
Picocells and macrocells: questionable since b h are operator d l d
ll d ll i bl i both deployed
 Need novel tools that capture more realistic models in small cell and heterogeneous
networks
 Need models that actually incorporate space and time correlation (open problem)
33

34. Open Issues in Stochastic Geometry
• Most results assume base stations to transmit all the time;
• untrue in practical systems
• Biasing and cell association and load balancing
• Push traffic toward open access underload picocells
• Achieving cell splitting gains
• Uplink SINR model much harder
• Requires a thorough study
• Interference management, scheduling, MIMO, mobility management and load
balancing
• Most importantly, operators want pointers for their network deployments.
34

35. Part III
Interference Management
g
35

36. LTE-A: Goals
• Greater flexibility with wideband deployments
• Wider bandwidths, intra-band and inter-band carrier aggregation
• Higher peak user rates and spectral efficiency
• Higher order DL and UL MIMO
• Flexible deployment using heteregenous networks
• Coordinated macro, pico, remote radio heads, femto, relays, Wi-Fi
• Robust interference management for improved fairness
• Better coverage and user experience for cell edge users
bps  bps/Hz  bps/Hz/km2
Towards Hyper-Dense Networks
36

37. Inter-cell Interference Coordination in LTE/LTE-A
• LTE (Rel. 8-9)
• Only one component carrier (CC) is available
 Macro and femtocells use the same component
carrier
 Frequency domain ICIC is quite limited
• LTE A (Rel 10 11)
LTE-A (Rel. 10-11)
•Multiple CCs available
•Frequency domain ICIC over multiple CCs is possible
Frequency
•Time domain ICIC within 1 CC is also possible
•Much greater flexibility of interference management
Source: Ericsson 37

38. ICIC in LTE-A: Overview
• Way to get additional capacity
 cell splitting is the way to go about it
• Make cells smaller and smaller and make the network
closer to user equipments
• Flexible placement of small cells is the way to address
capacity needs
 How do we do that?
 I R l
In Release-8 LTE picocells are added where users
8 LTE, i ll dd d h
associate to strongest BS.
 Inefficient 
 Release-10 techniques with enhanced solutions are
proposed
 Cell range expansion (CRE)
 AAssociate t cells th t ” k sense”
i t to ll that ”makes ”
 Slightly weaker cell but lightly loaded e.g., Why not offload the UE to
the picocell ?
38
Source: DOCOMO

39. Inter-cell Interference Coordination
• ICIC and its extensions are study items in SON
A combination
Orthogonal Orthogonal
g thereof +
transmission, transmission, coordination
Almost Blank Carrier beamforming,
Subframe, Cell aggregation, coordinated
Range
R Cell Range
C ll R scheduling,
scheduling joint
Expansion, etc Expansion, etc transmission,
DCS, etc
Time-Domain Frequency- Spatial Domain
ICIC Domain ICIC ICIC
39

40. Inter-cell Interference Coordination -
Time Domain Increased footprint of pico
p p
When macro frees up resources
• Typically, users associate to base
stations with strongest SINR
• BUT max-SINR is not efficient in
SCNs
Pico
• Cell range expansion (CRE) ? Pico Macro
• Mandates smart resource
partitioning/ICIC solutions
• Bias operation intentionally allows Subframes reserved for picocell transmission
UEs to camp on weak (DL) pico cells
• RSRP = Reference signal g Limited footprint of p
p pico due
received power (dBm) To macro signal
• Pico (serving) cell RSRP + Bias
= Macro (interfering) cell RSRP
•Need for time domain subframe partitioning
between macro/picocells Pico
Pico
Macro
• In reserved subframes, macrocell does not
transmit any data
•Almost Blank Subframes (ABS) + duty cycle
Subframes reserved for macrocell transmission
40

41. Inter-cell Interference Coordination -
Time Domain
(Static) Time-Domain Partitioning
• (St ti ) Ti D i P titi i
50% Macro and Pico; Semi-Static
• Negotiated between macro and
picocells via backhaul (X2) Macro DL 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
• Macro cell frees up certain
p Pico DL
Pi 0 1 2 3 4 5 6 7 8 90 1 2 3 4 5 6 7 8 9
subframes (ABS) to minimize
Data
interference to a fraction of UEs No transmission
time
transmission
served by pico cells
• All picocells follow same pattern
Inefficient in high loads with non- #1
uniform Ues #1
Macro Pico
• Duty cycle: 1/10,3/10,5/10 etc
• Reserved subframes used by
multiple small cells
25% Macro and Pico; Adaptive
• Increases spatial reuse
Macro DL 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
• Adaptive Time-Domain Partitioning
Pico DL 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
• Load balancing is constantly
time
performed in the network
Possible Data
• Macro and picocells negotiate No transmission transmission
transmission
partitioning based on
spatial/temporal traffic distribution. 41

42. Inter-cell Interference Coordination
ABS
• Inter-cell interference coordination is necessary for effective femto/pico deployment
• Almost blank subframe (ABS)
• During defined subframes, the aggressor cell does not transmit its control + data
channel to protect a victim cell
• ABS pattern transmitted via X2 (dynamic) for macro/pico
• Macro/pico aggressor/victim
or via OAM (semi static) for macro/femto (=victim/aggressor) New
• Issues with the UEs who should know device
those patterns + detect weak cells
cells.
Macro Pico
• Common reference, sync and primary broadcast Legacy
should be protected device
• Co-existence of legacy and new devices in pico CRE zone
• Need for enhanced receivers for interference suppression of
residual signals transmitted by macro cells
Macro DL
FBS DL
Macro UE
Victim
Femto BS
Small Data
Aggressor No TXABS
cell UE Macro-BS transmission
Example of macro/femto ICIC through ABS 42

43. Inter-cell Interference Coordination
• Further enhanced ICIC (feICIC) for non-CA based
deployment
• Some proposals under discussion X2 X2
Macro Pico
• At the transmitter side in DL  combination of
ABS + power reduction (soft-ABS)
• At the receiver side in DL use of advanced
receiver cancellation
• Further enhanced ICIC (feICIC) for CA based
deployment
• Several cells and CCs are aggregated
• Up to 5 CCs (100 MHz bandwidth)
• Cross scheduling among CCs is
possible
• Primary CC carrying
control/data information and
rest of CCsc carrying data
and vice-versa
• Greater flexibility for Cross scheduling
g
interference management
How to distribute the primary and secondary CCs to
optimize the overall network p
p performance?? 43

44. Inter-cell Interference Coordination -
Frequency Reuse
• Several configurations exist (full hard soft fractional) frequency reuse
(full, hard, soft,
• Requires coordination through message exchange (X2)
• Relative Narrowand Transmit Power Indicator (RNTP) for DL
• High Interference Indicator (HII) for UL
• Interference Overload Indicator (OI) for UL; reactive
• Frequency partitioning in HetNet  LTE Rel. 8/9
• Static FFR
• Partition the spectrum into subbands and assign a given subband to a cell in a coordinated
manner that minimizes intercell interference
• E.g., N=1/3 FFR yields improvements in terms of SINR albeit lower spectral efficiency
• Dynamic FFR
• Assignments based on interference levels/thresholds as well as scheduling users based on
CQI from users feedbacks.
HFR
X2
X2
FFR
X2
X2
X2
SFR
X2
44
Static FFR vs. Reuse 1 Protecting cell edge users using FFR

45. Inter-cell Interference Coordination -
Frequency Reuse
• From operators’ viewpoint a co channel deployment is
operators viewpoint, co-channel
highly desirable due to limited and scarce bandwidth
• Co-channel deployment  high interference
• Assigning reuse-1 in the macrocell and femtocell networks
yields high interference
• Interference mitigation is crucialfractional frequency reuse
(FFR) is one (potential) solution
• In terms of spatial reuse, it is not better than reuse-1 but
improves cell edge conditions in the outer region
• Sniffing carried out by femtocells
Dual stripe
Source: SAGEM
 Interference mitigation scheme FFR in the macro increases with higher antenna configurations 45
 FFR at macro is beneficial to both macro and femto tiers

46. Inter-cell Interference Coordination - Carrier Aggregation
• C i aggregation i used i LTE A via C
Carrier ti is d in LTE-A i Component t 100 MHz
Carriers (CCs) CC1 CC2 CC3 CC4 CC5
• Macro and Pico cells can use separate carriers to
freq.
avoid strong interference
g
• Carrier aggregation (CA) allows additional flexibility
to manage interference
 Macrocells transmit at full power on anchor CC1
CC2 CC3 Macro
carrier (f1) and lower power on second carrier
(f2), etc
CC2
CC1 CC3 Pico
 Picocells use second carrier (f2) as anchor carrier freq.
 Partitioning ratio limited by number of carriers
But trend is changing aggressor
 (in some cases) Aggressor is victim and victim is aggressor
macro UE
CC1 pico
victim
victim
How/when to swap victim/aggressor roles? CC2 macro UE S pico
46
aggressor

47. Co-tier Interference Management
• I d
In dense network d l
t k deployments, f t t
t femto-to-
femto interference can be severe
• especially for cell edge users
• Assigning orthogonal resources among
g g g g
neighboring femtocells protects cell edge UEs
albeig low spectral efficiency Macro-BS FBS-1
• Need dynamic ICIC techniques which are
scalable to accommodate multiple Ues
• Key: Assign primary CCs and secondary CCs
depending on interference map, dynamic FBS-2
interference mitigation through resource FBS-3
partitioning
Macro UE
• Centralized vs. Decentralized approaches
Aggressor/Victim: small cell/small cell
 Resources are assigned by a central controller
 More efficient resource utilization than the
distributed approach  Resources are assigned autonomously by BSs
 Needs extra signaling between the BSs and the  Less complexity
controller  High signaling overhead
g g g
 Highly computational  Requires long time period to reach a stable resource allocation
 Low resource efficiency
47

48. Co-tier Interference Management (Centralized Approach)
Interfering Neighbor Di
I t f i N i hb Discovery
Feedback
- UE makes measurement
- Identifies its interfering neighbors according to a predefined SINR #3 Interference
#3
threshold
• BSs send cell IDs of the interfering neighbors to the central FBS-1 #2
controller (through the backhaul)
• The central controller maps this information into an interference
graph where each node corresponds a BS, and an edge connecting #1,3
#2
FBS-3
two nodes represents th i t f
t d t the interference b t
between two BSs
t BS FBS-2
5x5 grid case Centralized
DL controller
Focus on F2F
#1,3
Graph Coloring
- GB‐DFR attains a significant capacity improvement for
cell‐edge UEs, at the expense of a modest decrease for
cell‐centre users
- Nearly all UEs achieve an SINR exceeding 5 dB
48
“Graph-Based Dynamic Frequency Reuse in Femtocell Networks,” IEEE VTC 2011

49. Co-tier Interference Management (Distributed Approach)
Dynamic interference environment 3 CC
CCs A B C
- Number and position of neighbors change during the
Operation
- Fixed frequency planning is sub‐optimal FBS-1 A C
Dynamic assignment of resources! FBS-2 B
Multi‐user deployment
- Users in same cell experience different interference FBS-3 C
conditions
- Resource assignment should depend on UE measurements
to maximize resource utilization
 Classify resources according to their foreseen usages C
Reserved CC
A
– Allocated to cell edge UEs
– Protected region
Banned CC:
– Interfering neighbors are restricted to use the RCC B C FBS-3
allocated to the victim UE
FBS-2
– This guarantees desired SINR at cell edge UEs
Auxiliary CC: Example
– Allocated to the UEs facing less interference
– Neighbors are not restricted
– Increases resource efficiency, especially, for the
multi‐user deployments “Decentralized interference coordination via autonomous component carrier assignment ,” IEEE
GLOBECOM 2011 49

50. Co-tier Interference Management (Distributed Approach)
• 5x5 grid model, 40 MHz system bandwidth
• Tradeoff between SINR and user capacity
• Proposed approach has more flexibility in assigning
component carriers according to its traffic
• The proposed approach outperforms the static schemes,
especially for cell edge users
users.
SINR i
improvements f users at the cost of lower capacity
for h fl i
 Extensions:
 Issues with convergence and scalabilities yet to be
addressed
 Multi-antenna extension
“Decentralized interference coordination via autonomous component carrier
assignment,” in proc. IEEE GLOBECOM 2011 50

51. Part III
Toward Self-Organizing
Small Cell Networks
51

52. Self-Organizing Networks
• Manual network deployment and maintenance
is simply not scalable in a cost-effective
manner for large femtocell deployments
– Trends toward Automatic configuration and
network adaptation
• SON is key for
– Automatic resource allocation at all levels
(frequency, space, time, etc.)
• Not just a buzzword 
– It will eventually make its way to practice
Large Small
picocell picocell
footprint footprint
with fewer with more
users users 52

53. Toward Self‐Organization: Tools
Game Theory & Learning The intelligence Physics
and protocol
The dynamics foundations The physics
foundations foundations
Evolutionary Biology Random Matrix Theory
The economic The large
and legal system
foundations foundations
Micro‐economics Free Probability
Future Communication
The statistical
The traffic Networks inference
foundations
f d ti foundations
Network
Queuing Theory
Information theory
The security The uncertainty
foundations foundations
The feedback The coding
Wireless foundations foundations Discrete Mathematics
Cryptography
Control Theory
We focus on game-theoretic/learning aspects
53

54. Introduction
• What is Game Theory?
– The formal study of conflict or cooperation
h f l d f fli i
– How to make a decision in an adversarial environment
– Modeling mutual interaction among agents or players that are rational
decision makers
– Widely used in Economics
• Components of a “game”
game
– Rational Players with conflicting interests or mutual benefit
– Strategies or Actions
– Solution or Outcome
• Two types
– Non-cooperative game theory
No coope at ve ga e t eo y
– Cooperative game theory
• Close cousins: Reinforcement learning 54

55. Heard of it before?
Heard of it before?
• In Movies
• Childhood games
– Rock, Paper, Scissors:
which one to choose?
– Matching pennies:
how to d id
h t decide on heads or tails?
h d t il ?
• You have witnessed at
least
l t one game-theoretic
th ti
decision in your life 
55

56. Non‐cooperative game theory
• Rational players having conflicting interests
– E.g. scheduling in wireless networks
• Often…
– Each player is selfish and wishes to maximize his payoff or
‘utility’
• The term ‘utility’ refers to the benefit that a player can
obtain in a game
• Solution using an equilibrium concept (e.g., Nash), i.e., a
state in which no player has a benefit in changing its strategy
• Misconception: non-cooperative is NOT always competition
– It implies that decisions are made independently without
p p y
communication, these decisions could be on cooperation!
56

57. Nash Equilibrium
• Definition: A Nash equilibrium is a strategy profile s*
with the property that no p y i can do better by
p p y player y
choosing a strategy different from s*, given that every
other player j ≠ i .
• In other words, for each player i with payoff function ui
, we have:
• Nash is robust to unilateral deviations
– No player has an incentive to change its strategy
given a fixed strategy vector by its opponents
57

58. Example: Prisoner s dilemma
Example: Prisoner’s dilemma
• Two suspects in a major crime held for interrogation in separate
cells
– If they both stay quiet, each will be convicted with a minor offence and
will spend 1 year in prison
– If one and only one of them finks, he will be freed and used as a witness
d l f h fi k h ill b f d d d i
against the other who will spend 4 years in prison
– If both of them fink, each will spend 3 years in prison
• Components of the Prisoner’s dilemma
– Rational Players: the prisoners
– Strategies: Confess (C) or Not confess (NC)
– Solution: What is the Nash equilibrium of the game?
• Representation in Strategic Form
58

59. Prisoner’s Dilemma
Pareto optimal Nash Equilibrium
P2 Not Confess Confess
P1 Not Confess -1,-1 -4,0
P1 Confess 0, 4
0 -4 -3 -3
3, 3
• P1 chooses NC P2’s best response is C
NC, P2 s
• P1 chooses C, P2’s best response is C
• F P2 C i a d i
For P2, is dominant strategy
t t t
59

60. Design Consideration
• Existence and Uniqueness
-Convexity/concavity of payoff
function
- Best response is standard function
(positivity, monotonicity,
scalability)
-Potential game
Utility of
p y
player 2
given
strategies Nash equilibrium? Pareto optimality
of players
1 and 2
Utility of player 1 given strategies of
players 1 and 2
60

61. Non‐cooperative Games
• Pure vs. mixed strategies
– Existence result for Nash in mixed strategies (1950)
ste ce esu t o Nas ed st ateg es ( 950)
• Complete vs. incomplete information
• Zero-sum vs. Non zero-sum
Zero sum zero sum
• Non zero-sum are games between multiple players
– Two player games are a special case
• Matrix game vs. continuous kernel games
• Static vs. Dynamic
vs
– Evolutionary games
– Differential games
– …..
61

62. More on NC games
• Refinements on Nash
– To capture wireless characteristics or other stability
notions
• Stackelberg game
– Important in small cell networks due to hierarchy
• Correlated equilibrium
– Useful for coordinated strategies
• Special games
– Potential/Supermodular games (existence of Nash)
• Bayesian games, Wardrop equilibrium
y g , p q
• …..
62

63. Cooperative Game Theory
• Non-cooperative games describe situations where the
p aye s
players do not coo d ate their strategies
ot coordinate t e st ateg es
• Players have mutual benefit to cooperate
• Namely two types
– Nash Bargaining problems and Bargaining theory
– Coalitional game
• Bargaining theory
g g y
• For both
– A li ti
Applications in wireless networks are numerous
i i l t k
63

64. Bargaining Example
Bargaining theory
I can give you 100$ if
and the Nash
and only if you agree
d l
bargaining solution!
on how to share it
CanMight be a unsatistifactory
be deemed
Given each Man’s !
better scheme ! wealth!!!
Rich Man Poor Man
64

65. Coalitional Games
Coalitional Games
• Definition of a coalitional game (N,v)
– A set of players N, a coalition S is a group of cooperating players
– Worth (utility) of a coalition v
• In general, v(S) is a real number that represents the gain resulting
from a coalition S in the game (N,v)
– User payoff xi : the portion of v(S) received by a player i in coalition S
• Characteristic form
– vd
depends only on the i t
d l th internal structure of the coalition
l t t f th liti
• Partition form
– v depends only on the whole partition currently in place
• Graph form
– The value of a coalition depends on a graph structure that connects the
coalition members
65

66. CF vs. PF
CF vs. PF
In Characteristic form: the
value depends only on
internal structure of the
i t l t t f th
coalition
66

67. Cooperative Games
Cooperative Games
- Players’ interactions are governed by a communication graph structure.
Players
-Key network structure that forms depends on gains and costs from cooperation.
- The question “How to stabilize the grand coalition or form a network
structure The grand coalition of communication graph?”
-Key question “How to formthe all users is an optimal structure. (topology) and
- taking into account an appropriate coalitional structure
how to study question “How combine concepts from coalitions, and non-
Solutions are complex, to stabilize the grand coalition?”
-Key its p p
y properties?”
- More complexl than Class I, with ti formal solution concepts.
cooperative games fi d solution concepts exist.
-SSeveral well-defined l no
ll d t it
67/124

68. Learning in Games
• For a general N-player game, finding the set of NEs is
not possible in polynomial time!
• Unless the game has a certain structure
• We talk about learning the equilibrium/solution
• Some existing algorithms
– Fictitious play (based on empirical probabilities)
– Iterative algorithms (can converge for certain classes of
i l ih ( f i l f
games)
– Best response algorithms
• Popular in some games (continuous kernel games for example)
– Useful Reference
• D. Fundenberg and D. Levine, The theory of learning in games, the
MIT press, 1998.
68

69. Learning Algorithms
• Distributed Implementation/Algorithm
– Which information can be collected or exchanged
– How to obtain knowledge and state of system
– How to optimize action/strategy
Observe - Q-learning, fuzzy Q-learning
-Evolutionary based learning
Analyze and
Adapt Cognitive cycle
g y learning
- Non-regret learning
- Best response dynamics
- Gradient update
Optimize
• Di ib d Implementation/Algorithm
Distributed I l i /Al i h
– Convergence? Speed? Efficiency?
– O erhead and complexity
Overhead comple it
(communication/computation/storage)
69

70. Examples:
Access Control in Small Cell Networks (Nash game)
User Association in Small Cell Networks (Matching
game)
Cooperative interference management
(Coalitional game)
70

71. To Open or To Close?
To Open or To Close?
Base Station
OpenClosed access FAP
access for one
71

72. To Open or To Close?
To Open or To Close?
• Tradeoff between allocating resources and
absorbing MUEs/reducing interference
• Optimizing this tradeoff depends on the locations of
the MUEs, the number of interferers, etc.
h h b fi f
• The choices of the FAPs are interdependent
– If an FAP absorbs a certain MUE, it may no longer be
beneficial for another FAP to open its access
• S O
So, Open or Cl d?
Closed?
– Neither: Be strategic and adapt the access policy
–NNoncooperative game!
ti !
72

73. Formally…the Femto Problem
Formally…the Femto Problem
• Consider the uplink of an OFDMA system with
– M underlaid FAPs, 1 FUE per FAP, and N MUEs
– Assuming no femtocell-to-femtocell interference
– An MUE connects to one FAP
– For simplicity, we use subbands instead of subcarriers, i.e., each FAP
has a certain contiguous band that it can flexibly allocate
• N
Noncooperative game
i
– Players: FAPs
– Strategies: close or open access (allocate subbands)
– Objective: Maximize the rate of home FUE (under a constraint)
Fraction of subband Coupling of actions in SINR
allocated by FAP m to MUE n (next slide) 73

74. Formally…
• Zoom in on the SINR:
- Coupling of all FAPs actions
- Only MUEs not absorbed by others
are a source of interference
- Discontinuity in the utility function
• Game solution: Nash equilibrium
• Does it exist?
– Oh not again  74

75. Existence of Nash equilibrium
Existence of Nash equilibrium
• Common approaches for finding a Nash equilibrium mostly
deal with nicely behaved functions (e.g., in power control,
(e g control
resource allocation games, etc.)
– Discontinuity due to open vs. closed choice
• P. J. Reny (1999) showed that for a game with discontinuous
utilities, if
– The utilities are quasiconcave
h ili i i
– The game is better-reply secure, i.e.,
Non-equilibrium vector
q
Strategy of an
arbitrary FAP m
• O game satisfies both properties => Pure strategy Nash
Our i fi b h i P N h
exists 75

76. Simulation results (1)
A mixture of
closed
l d
and open access
emerges at
equilibrium
76

77. Simulation results (2)
Improved
performance
For the worst-case
FAP (equilibrium
is a more
fair h
f i scheme
than all-open)
77

78. Access point assignment in
small cell networks
ll ll k
 A macro-cellular wireless
network
 A number of small cell base
stations
 Different cell sizes
 A number of wireless users
seeking uplink transmission
 How to assign users to access
points?
 More challenging than
traditional cellular
networks
78

79. Access point assignment
Access point assignment
• The problem is well studied in classical cellular networks but..
– ..most approaches focus on the users point of view only in the presence
of one type of pre-fixed base stations
– Do not account for different cell sizes and offloading
• New challenges when dealing with small cell base stations
• Three decision makers with different often conflicting objectives:
– Small cells who want to ensure good QoS, Improve macro-cell coverage
via offloading (cell range expansion)
– Users that want to optimize their own QoS
– Macro-cells seeking to ensure connectivity
• C we address th problem using a f h small cell-oriented
Can dd the bl i fresh ll ll i t d
approach?
79

80. Access point assignment as a matching
game
1- Student B 1- Student A
2- Student
2 St d t A 2- Student B
How to match students (workers) to colleges (employers)?
How to assign wireless users to access points (SCBS and macro) ?
1- U Miami 1- FIU
2- FIU 2- U Miami
Student A Student B 80

81. Simulation results
Performance
advantage
increasing
with the
users density
81

82. Notes and Future Extensions
• Adapts to slow mobility by periodic re-runs as well as to
quota changes and users l i or returning
h d leaving i
• Can we design a college admissions game that can handle
fast dynamics, i.e., handovers?
– Combine with dynamic games
• How to accommodate traffic and advanced schedulers?
– Use concepts from polling systems and queueing theory
• Ideally, we can build a matching game that enable us to
design heterogeneous networks where assignment is made
based on preferences and service types!
– Explore new dimensions in network design and resource allocation
– Diff
Different classes of matching games to exploit
l f hi l i
82

83. Cooperative Interference Management
• We consider the downlink problem
• Femto access points can form a coalition to share the
spectrum resource (i.e., subchannels), reducing the co-tier
interference
Coalition S1 Macro
m1 users
m2 Coalition S2
Macro base
station
f1
Femto
access
point
f2 f4
f3
``Cooperative Interference Alignment in Femtocell Networks,'‘ IEEE Trans. on Mobile Computing, to
appear, 2012 83

84. Cooperative Interference Management
• Coalition formation game model
– Players: Femto access points
– Strategy: Form coalitions
– V l of any coalition
Value f li i
Transmission rate
Interference from Interference from
femto access points macrocell
not in the same coalition
84

85. Cooperative Interference Management
• Not all femto access points can form coalition, since they may
not be able to exchange coalition formation information
among each other
• Cooperation entails COSTS
• We model it via power for information exchange (more
elaborate models needed)
Coalition S1 Macro
m1 users
m2 Coalition S2
Macro base
station
f1
Femto
access
point
f2
f3 f4
85

86. Cooperative Interference Management
Chance of cooperation is small
(information cannot be exchanged Many femto access points
among f t access points)
femto i t) can f
form coalition
liti
Too congested
Solution is
co-opetition...
86

87. Learning how to self-organize in a
dense small cell network?
”Decentralized Cross-Tier Interference Mitigation in Cognitive Femtocell Networks," IEEE International
Conference on Communications (ICC), Kyoto, Japan, June 2011.
87

88. Toward Evolved SON
Femtocell networks aim at increasing spatial reuse of spectral resources, offloading,
boosting capacity, improving indoor coverage 
• BUT inter-cell/co-channel interference   Need for autonomous ICIC, self-
organizing/self-configuring/self-X interference management solutions to cope with
network densification
• Many existing solutions such as power control, fractional frequency reuse
(FFR), soft frequency reuse (SFR), semi-centralized approaches …
We examine a fully decentralized self-organizing learning algorithm based on local
information, robust, and without information exchange
•Femtocells do not know the actions taken by other femtocells in the network
•Focus is on the downlink
•Closed subscription group (CSG)
•No cross-tier nor co-tier cooperation + No carrier aggregation (no leeway !!)
88

89. Solution (in a nutshell)
Due to their fully-decentralized nature, femtocells need to:
- Estimate their long-term utility based on a feedback (from their UEs)
long term
- Choose the most appropriate frequency band and power level based on the accumulated
knowledge over time (key!)
- A (natural) exploration vs. exploitation trade-off emerges;
i. should femtocells exploit their accumulated knowledge OR
ii. explore new strategies?
- Some reinforcement learning procedures (QL and its variants) implement (i)-(ii) but
(i) (ii)
sequentially
- Inefficient
- Model-based learning.
Proposed solution is a joint utility estimation + transmission optimization where
the goal is to mitigate interference from femtocells towards the macrocell network
+ maximize spatial reuse
• (i)-(ii) are two learning processes carried out simultaneously!
• Every f
E femtocell i d
ll independently optimizes i own metric and there i no
d l i i its i d h is
coupling between femtocell’s strategies (correlation-free);
• for correlation/coordination  other tools are required
89

90. ..”Behavioral” Rule..
FBS
- History Ultimately,
- Cumulated Play a given maximize the
rewards action
... long-term
performance
Should i explore?
Should i exploit?
90

91. Basic Model
SINR of MUE
SINR of FUE
f
Maximize the long-term transmission rate of every
femtocell (selfish approach)
91

92. Game Model
• The cross-tier interference management problem is
modeled as a strategic N.C game
• The players are the femto BSs
• The set of actions/strategies of player/FBS k is the
power allocation vector
• The utility/objective function of femtocell k
• Rate, power, delay, €€€ or a combination thereof
Here transmission rates are considered
• At each time t FBS k chooses its action from the finite
t,
set of actions following a probability distribution:
92

93. Information Aspects
• Femtocells are unable to observe current and all previous actions
• Each femtocell knows only its own set of actions.
• Each femtocell observes (a possibly noisy) feedback from its UE
( p y y)
• Balance between maximizing their long-term performance AND
exploring new strategies-----------okay but HOW?
• A reasonable behavioral rule would be choosing actions yielding
high payoffs more likely than actions yielding low payoffs, but in any
case, always letting a non-null probability of playing any of the
actions
• This behavioral rule can be modeled by the following probability
distribution:
(x)
Maximize the long-term
performance utility +
Entropy/Perturbation
perturbation
93

94. Proposed SON Algorithm
• At every time t, every FBS k jointly estimates its long-term utility function and
updates its transmission probability over all carriers:
Utility
estimation
Strategy
optimization
Learning
parameters
Other SON variants can be derived in a similar way
Both procedures are
p
done simultaneously!
!!!
This algorithm converges to
Players learn their utility faster than the
the so-called epsilon-close Nash Optimal strategy
94

95. Numerical Results
First scenario Parameters
2 MUEs, 2 RBs, K=8
FBSs
Macro BS TX power
Femto BS TX power
•The larger the temperature parameter is,
the more SON explores, and the
algorithm uses more often its best
transmission configuration and
converges closer to the BNE.
•In contrast, the smaller it is, femtocells
are more tempted to uniformly play all
their actions
 Convergence of SON 1 learning algorithms with respect to the Best NE.
 The temperature parameter has a considerable impact on the performance
Altruism vs. Selfishness 95
Myopic vs. Foresighted

96. Numerical Results
Second scenario SON1 SON-RL
• 6 MUEs, 6 RBs, K=60 FBSs
2.2 SON2:
SON1(+imitation)
2.1
Average Spectral Efficiency (bps/Hz)
SON1
2 SON2 SON3:
Best response
y
SON3
1.9
- no history
1.8
- myopic (maximize
performance at every
S
1.7 time instant)
1.6
1.5
15
0 1 2 3 4 5 6 7 8 9 10
Convergence Time x 1000
Average femtocell spectral efficiency vs. time for SON and best response learning algorithm
SON1 outperforms SON2 and SON3
Being foresighted yields better performance in the long term
96

97. Now, let us add some implicit
coordination among small cells
”Learning Coarse correlated equilibria in small cell networks," IEEE International Conference on
Communications (ICC), Ottawa, Canada, June 2012.
97

98. The Cross‐Tier Game
The cross-tier interference management problem is modeled as a
normal-form game
At each time instant, every small cell chooses an action from its
finite set of action following a probability distribution:
© Centre for Wireless Communications, University of Oulu

99. (Classical) Regret-based learning procedure
e.g.,
Player k would have obtained a higher performance
y g p
By ALWAYS playing action
© Centre for Wireless Communications, University of Oulu

100. Regret‐based Learning
Given a vector of regrets up to time t,
Every small cell k is inclined towards taking actions yielding
highest regret, i.e.,
g g , ,
..From
..From perfect world to reality...
reality...
In classical RM, each small cell knows the explicit expression of its utility function
and it observes the actions taken by all the other small cells  full information
Impractical and non scalable in HetNets
© Centre for Wireless Communications, University of Oulu

101. Regret‐based Learning
• Remarkably, one can design variants of the classical regret
matching procedure which requires no knowledge about other
p y
players’ actions, and yet yields closer performance. How?
, y y p
• (again) trade-off between exploration and exploitation,
whereby small cells choose actions that yield higher regrets
more often than those with lower regrets,
– But always leaving a non-zero probability of playing any of
the actions (perturbation is key!)
© Centre for Wireless Communications, University of Oulu

102. Exploration vs. Exploitation
The temperature parameter represents the interest of small
cells to choose other actions than those maximizing the regret,
in order to improve the estimation of the vector of regrets.
p g
The solution that maximizes the behavioral rule is:
Boltzmann
distribution
Always positive!!
Decision function mapping past/history + cumulative regrets into future
© Centre for Wireless Communications, University of Oulu

103. Numerical Results
0.8
bps/Hz]
0.7
ocell spectrall efficiency [b
0.6
0.5
0.4
2X increase
0.3
Average femto
reuse 1
0.2
reuse 3
0.1 SON-RL; [Bennis ICC'11]
regret-based
A
0
0.2 0.4 0.6 0.8 1
femtocell density in %
Average femtocell spectral efficiency versus the density of
femtocells for SON learning algorithms.
© Centre for Wireless Communications, University of Oulu

104. Take Home Message
• Can small cells self-organize in a decentralized manner?
Yes!
- no information exchange
- solely based on a mere feedback
- Robust to channel variations and imperfect feedbacks
- N synchronization i required unlike some other l
No h i i is i d lik h learning algorithms!
i l ih !
•Numerous tradeoffs are at stake when studying self-organization
•Open Issues:
Open
•How to speed up convergence?
•Introduce QoS-based equilibria?
•Optimality i not always what operators want!!
O i li is l h !!
104

105. Part VI
Release 12 and Beyond
Open Issues
105

106. Release 12 and beyond
• Facilitate “seamless” mobility between macro and pico layers
• Reduced handover overhead, increased mobility robustness, less loading to the core network
• Increased user throughput with carrier aggregation or by selecting the best cell for uplink and
downlink
• Wide-area assisted Local area access f2
f1/booster
Macro-BS FUE
LTE multiflow / inter site CA Small cell BS
Soft-Cell concepts
Non-fiber based connection
TDD Traffic Adaptive DL/UL Configuration
• Depends on traffic load and distribution DL is dominant
i d i t
• Interference mitigation is required for alignment Macro-BS
Of UL/DL
• Flexible TDD design DL UL UL UL UL is dominant DL DL DL UL
106

107. Release 12 and beyond
CA btw LB & ULB
 Utilization of various frequency resources
LTE or WiFi
 Aggregation of FDD and TDD carriers Licensed
Band
Unlicensed
Band
 Aggregation of unlicensed band (LTE or
WiFi)
FDD Hetero-
f1 f2 f3 f4
UL DL UL DL
CA f5 TDD f6
UL UL
DL DL
CA btw FDD & TDD
Source: LG Electronics
• Intra-RAT Cooperation
• CoMP based on X2 interface F14
F13
• More dynamic eICIC
F12 M2
• Maximized energy saving F1
F5
F3 F4
F11 P3
 Carrier based ICIC for HeNB F10 F2
 Macro/Pico-Femto, Femto-Femto F9
 Multi carrier
Multi-carrier supportable HeNB
M1 F8
F7
F6
107
Source: LG Electronics