1. HA and SIP Load
Balancing
Design documentation
Revised: Sep, 2012

2. Important!
● HA here doesn't imply replication.
● HA quality units
○ one nine, two nines, three nines and so on
○ can claim them without any replication
■ it's not even cheating
● It is sufficient to partition your calls onto a set of machines
○ The SIP protocol doesn't mind address changes
○ Can go into production against any production-ready
phone

3. Deployment Scenarios Overview
● Pure IP load-balancing
○ No SIP-based affinity, only IP-based affinity
○ Might violate some SIP rules
○ Not recommended
● Standalone SIP-based load-balancing
○ Provides SIP-based affinity
○ Provides SIP protocol compliance
● Distributed load balancing
○ IP load balancer in front
○ Multiple SIP load balancers at the back end
○ Eliminates the Single Point of Failure problem
○ Scales better when SIP LB capacity is exceeded
● Cooperative load balancing with HTTP (integrated, mod_jk
and mod_cluster)

4. Deployment Scenarios Overview
● Pure IP load-balancing
○ No SIP-based affinity, only IP-based affinity
NOT ENTIRELY BAD
○ Might violate some SIP rules
○ Not recommended
● Standalone SIP-based load-balancing
ALMOST USELESS
○ Provides SIP-based affinity
○ Provides SIP protocol compliance
● Distributed load balancing
○ IP load balancer in front
○ Multiple SIP load balancers at the back end
○ Eliminates the Single Point of Failure problem
○ Scales better when SIP LB capacity is exceeded
● Cooperative load balancing with HTTP (integrated, mod_jk
and mod_cluster)

5. Deployment Scenarios Overview
● New this year!
● DNS load balancing
○ No SIP message affinity
○ Affinity is temporary per UA by TTL
○ No built-in heartbeats with the SIP servers - must do it
on your own, with a module or by the UA
○ You better try to keep all your IPs up
■ Use IP takeover for fast recovery

6. Two kinds of DNS load balancing
● Dynamic record DNS
○ Round-Robin DNS
● DNS SRV
○ Built-in load balancing with statistical weights
○ Requires support from the SIP phones (very common)
_service._proto.name TTL class SRV priority weight port target
For instance:
_sip._tcp.example.com. 86400 IN SRV 0 5 5060 sipserver.example.com.
_sip._tcp.example.com. 86400 IN SRV 0 5 5060 sipserver.example.com.

7. Deployment Scenarios Overview
● New this year!
● Dedicated IP load balancing by SIP headers
○ Equivalent to DNS SRV without the need of phone
support

8. Pure IP load balancing
● Low priority
● Not very useful - just to clarify theoretically
○ Routes based on IP/UDP/TCP (Layer 3/4) fields - IP
address, source or destination port, etc
○ Can not make routing decisions based on SIP messages
(BigIP F5, etc are exception)
○ SIP AS-initiated requests and responses hard to route
○ IP LB is not a SIP entity.

9. Pure IP load balancing (cont'ed)
Retransmissions
● UDP
○ Retransmissions will get sprayed randomly
○ Forks and race conditions
● TCP
○ Works fine
■ Unless the TCP connection fails and the new
connection ends up on new node

10. Via headers in IP load balancing
● Via headers should contain the original node address, not
the address of the IP load balancer. Otherwise the SIP
phones will follow the SIP spec and route responses
through the IP load balancer, creating additional traffic.
● Via headers are per-transaction. Mid-transaction fail-over is
not supported although we are able to recover from it with
retransmissions.
● Before JBCP 1.2.6 and MSS 1.3 the Via headers were with
IP balancer addresses.
● Via headers must be IP balancer headers only if the
load balancer is capable of Via branch affinity

11. Standalone Mobicents SIP Load
Balancer
● The SIP LB is a stateless SIP proxy
● Responses and subsequent requests will follow the same
path - UDP follows Vias and TCP/TLS follows the
established connections as required per SIP spec.
● SIP LB can make routing decisions based on SIP headers
or content. It parses the SIP messages.
● The Standalone SIP LB is a Single Point of Failure

12. Standalone Load Balancer (proxy)
Note: In case of SIP AS or SIP LB failure in in step 4 the
response will be lost.

13. Distributed Load Balancer
● IP Load Balancer in front of the SIP LB
● The SIP LB will advertise the IP LB address instead of its
own
● Support for multiple SIP LBs. The IP LB will distribute the
load among the SIP LBs
● Support for bidirectional load-balancing

14. Distributed Load Balancer
● The SIP LBs may maintain shared state when it is needed
for certain load-balancing algorithms
● Certain algorithms don't need shared state (like consistent
hash)

15. Bidirectional Distributed SIP LB
● The SIP Application Servers not only receive client
requests, but can initiate SIP requests and transactions on
their own

16. Solving the case for both directions in
the Distributed LB
When requests come from SIP phones(clients) it is clear we
should use an IP load balancer in front of the external ports of
the SIP LBs. However, when the Application Servers are
initiating requests there are two options:
1. The Application Server is always aware which SIP LBs are
alive, so if one dies the AS will pick another one on it's own.
No IP load balancer is needed. This method works in terms
of heartbeat resolution.
2. Put an IP load balancer between the Application Servers
and the internal ports of the SIP load balancers as shown in
the next slide.

17. Distributed Load Balancer (2 IP LBs)
BIDIRECTIONAL

18. Bidirectional Distributed SIP LB
Deployment Scenarios
● The SIP LB can be configured with separate ports for
inbound and outbound messages (simply specify the
internalPort property in the SIP LB configuration file)
● Two IP LBs - use separate IP LBs for request that come
from clients and requests that come from servers
● One IP LB - use the same IP LB for both types of requests.
The problem with this one is that direction analysis must be
done using Via header. If the SIP AS is a non-Record-
Routing proxy application then non-initial requests initiated
by the callee will bypass the SIP AS and there will be no SIP
AS Via header to give a hint that the request comes from
the callee and should no go to the SIP AS.

19. Distributed Load Balancer (2 IP LBs)
BIDIRECTIONAL

20. Distributed Load Balancer (or 1 IP LB)
BIDIRECTIONAL

21. Converged and
Cooperative Load
Balancing
SIP, HTTP and other protocols

22. Integrated HTTP forwarding
● The Mobicents Load Balancer supports HTTP forwarding
● SIP and HTTP can use common consistent hash affinity key
to group and fail-over together SIP and HTTP sessions
● Example
○ SIP URI sip:app1@host.com
○ HTTP URL http://host.com/page?appsession=app1
○ app1 is sip user in the URI and appsession parameter
○ app1 will be hashed against the AS nodes for both and
will cause the SIP and the HTTP request to always stick
to the same node
● When there is no key, the balancer behaves like mod_jk
and analyses jvmRoute component to selet node
● Alternatively, you can use mod_jk and mod_cluster

23. Distributed Load Balancer (3 IP LBs)
BIDIRECTIONAL

24. HTTP and SIP consistent hashing

25. Session affinity
with mod_jk

26. mod_jk support
mod_jk and mod_cluster can be manipulated by changing the
jsessionid cookie to reroute requests to a node of choice!
Additionally mod_cluster can be controlled by the MCCP
protcol

27. mod_jk hints

28. Rolling Upgrades from the LB
● Each node is bootstrapped with a version system property
○ Each node is started with -Dversion=1
● The version is advertised in the SIP LB heartbeat
● The SIP LB has awareness of the groups with particular
version and can detect conditions that jeopardize the
opertaions
○ More than two versions
○ Node count dangerously low
○ Stalled upgrade with idle nodes

29. Cluster groups
Divide the cluster into subgroups that failover only internally
subclusterMap=( 192.168.1.1, 192.168.1.2 ) (
10.10.10.10, 20.20.20.20, 30.30.30.30)

30. SIP LB Internal
Architecture
What is needed for the SIP LB to
support the deployment scenarios?

31. Quick SIP LB functional spec
● Dumb SIP parsing - as dumb as possible with JSIP
stateless.
● Pluggable routing decision algorithms. Pass the message to
the algorithm and it will return the node where to send the
message.
● Shared store - use JBoss Cache 3.2.1
● No need to translate between TCP and UDP. The SIP AS
will be able to handle both anyway.
● Support separate SIP ports for inbound and outbound
messages (add an internalPort property in the config file)
● Support for single SIP port (delete the internalPort
property)
● RMI and JGroups heartbeats (right now RMI)
● Protocol to allow AS or other entity to give instructions to the
SIP LB.(like the mod_cluster protocol)

32. Example
Algorithms
Sample pluggable algorithms for the
SIP load balancer

33. #1 - Call-ID affinity workflow

34. #2 - Consistent-hash on Call-ID

35. #3- Persistent Consistent Hash

36. Balancer Algorithm Interface
public interface BalancerAlgorithm {
SIPNode processExternalRequest(Request request);
SIPNode processHttpRequest(HttpRequest request);
void processInternalRequest(Request request);
void processExternalResponse(Response response);
void processInternalResponse(Response response);
void nodeRemoved(SIPNode node);
void nodeAdded(SIPNode node);
Properties getProperties();
void setProperties(Properties properties);
BalancerContext getBalancerContext();
void jvmRouteSwitchover(String fromJvmRoute, String toJvmRoute);
void init();
void stop();
void assignToNode(String id, SIPNode node);
}
● Click here to see the full interface with
documentatione/browse/trunk/tool
● Click here to see one example algorithm Call-ID affinity with
association map

37. The SIP LB configuration file
# The binding address of the load balancer
host=127.0.0.1
# The RMI port used for heartbeat signals
rmiRegistryPort=2000
# The SIP port used where client should connect
externalPort=5060
# The SIP port from where servers will receive messages
# Delete if you want to use only one port for both inbound and outbound)
internalPort=5065
# The HTTP port for HTTP forwarding.
# If you like to have integrated HTTP load balancer, this is the entry point
httpPort=8080
#Specify UDP or TCP (for now both must be the same)
internalTransport=UDP
externalTransport=UDP
# If you are using IP load balancer, put the IP address and port here
externalIpLoadBalancerAddress=127.0.0.1
externalIpLoadBalancerPort=111
# Requests initited from the App Servers can route to this address (if you are using 2 IP load balancers for bidirectional SIP LB)
internalIpLoadBalancerAddress=127.0.0.1
internalIpLoadBalancerPort=111
# Designate extra IP addresses as serer nodes
#extraServerNodes=222.221.21.12:21,45.6.6.7:9003,33.5.6.7,33.9.9.2

38. ...the SIP LB configuration file
# Call-ID affinity algortihm settings. This algorithm is the default. No need to uncomment it.
#algorithmClass=org.mobicents.tools.sip.balancer.CallIDAffinityBalancerAlgorithm
# This property specifies how much time to keep an association before being evitcted.
# It is needed to avoid memory leaks on dead calls. The time is in seconds.
#callIdAffinityMaxTimeInCache=500
# Uncomment to enable the consistent hash based on Call-ID algorithm.
#algorithmClass=org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm
# This property is not required, it defaults to Call-ID if not set, cna be "from.user" or "to.user" when you want the SIP URI
username
#sipHeaderAffinityKey=Call-ID
#specify the GET HTTP parameter to be used as hash key
#httpAffinityKey=appsession
# Uncomment to enable the persistent consistent hash based on Call-ID algorithm.
#algorithmClass=org.mobicents.tools.sip.balancer.PersistentConsistentHashBalancerAlgorithm
# This property is not required, it defaults to Call-ID if not set
#sipHeaderAffinityKey=Call-ID
#specify the GET HTTP parameter to be used as hash key
#httpAffinityKey=appsession
#This is the JBoss Cache 3.1 configuration file (with jgroups), if not specified it will use default
#persistentConsistentHashCacheConfiguration=/home/config.xml
# Call-ID affinity algortihm settings. This algorithm is the default. No need to uncomment it.
#algorithmClass=org.mobicents.tools.sip.balancer.CallIDAffinityBalancerAlgorithm
# This property specifies how much time to keep an association before being evitcted.
# It is needed to avoid memory leaks on dead calls. The time is in seconds.
#callIdAffinityMaxTimeInCache=500

39. ...the SIP LB configuration file
# Uncomment to enable the consistent hash based on Call-ID algorithm.
#algorithmClass=org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm
# This property is not required, it defaults to Call-ID if not set, cna be "from.user" or "to.user" when you want the SIP URI
username
#sipHeaderAffinityKey=Call-ID
# and specify the GET HTTP parameter to be used as hash key
#httpAffinityKey=appsession
# Uncomment to enable the persistent consistent hash based on Call-ID algorithm.
#algorithmClass=org.mobicents.tools.sip.balancer.PersistentConsistentHashBalancerAlgorithm
# This property is not required, it defaults to Call-ID if not set
#sipHeaderAffinityKey=Call-ID
# and specify the GET HTTP parameter to be used as hash key
#httpAffinityKey=appsession
#This is the JBoss Cache 3.1 configuration file (with jgroups), if not specified it will use default
#persistentConsistentHashCacheConfiguration=/home/config.xml
#NEW PROPERTIES IN MSS 1.2
#If a node doesnt check in within that time, it is considered dead
nodeTimeout=5100
#The consistency of the above condition is checked every heartbeatInterval milliseconds
heartbeatInterval=5000
#JSIP stack configuration.....

40. Example Configurations
How to configure common scenarios?
Note: All distributed configurations must use
a consistent hash routing algorithm.
Example with Call-ID header hashing:
# Uncomment to enable the consistent hash based on Call-ID algorithm.
algorithmClass=org.mobicents.tools.sip.balancer.HeaderConsistentHashBalancerAlgorithm
# This property is not required, it defaults to Call-ID if not set
sipHeaderAffinityKey=Call-ID
#specify the GET HTTP parameter to be used as hash key
httpAffinityKey=appsession

41. Two SIP LBs with client pools
Request coming from phone.
If a load balancer fails then a
phone pool will experience
outage.

43. Two SIP LBs with Internal and
External IP LBs in the same
network with sample
configurations.
Request coming from
phone.

45. Two SIP LBs with Internal and
External IP LBs in the same
network with sample
configurations.
Request coming from
Application Server.

47. Two SIP LBs with Internal and
External IP LBs in the same
network with sample
configurations.
Full picture

49. IP load balancer problems

51. SIP LB Topics 2011

52. Large scale tests
● Simulate the IP load balancing and rewriting the addresses
● Run many nodes, many load balancers
● Execute the test as part of a larger scenario
● Simulate application server
○ Heartbeat
○ Handling and initiating requests
● TLS

53. Cluster groups
Divide the cluster into subgroups that failover only internally
subclusterMap=( 192.168.1.1, 192.168.1.2 ) (
10.10.10.10, 20.20.20.20, 30.30.30.30)

54. Worst-case testing
Load balancing without affinity

55. Rolling Upgrades from the LB
● Each node is bootstrapped with a version system property
● The version is advertised in the SIP LB heartbeat
● The SIP LB has awareness of the groups with particular
version and can detect conditions that jeopardize the
opertaions
○ More than two versions
○ Node count dangerously low
○ Stalled upgrade with idle nodes

56. Performance testing
10K requests/s

57. Moving the load balancing on the
server side
● Eliminates the worst case
● Some requests will go deeper in the pipeline costing more
● Can't change the Route headers
● Delay the load balancing decision as much as possible

58. NIO in the SIP LB
BIO is limited at 2500-10000 concurrent sockets on servers