We describe the features of Communication Groups i TIPC, i.e., the ability to instantiate multiple lightweight brokerless messages buses per user as part of the TIPC messaging service.

1. TIPC
COMMUNICATION
GROUPS
By Jon Maloy

2. - One socket per member
- “Type” part of a TIPC address is group identity, - determined by user
- “Instance” part of a TIPC address is member identity, determined by user
- TIPC Binding table is registry and distribution channel for member identities and events
- Along with data, member sockets may optionally deliver join/leave events for other members in group
- Membership events are just empty messages with two source addresses: <group:member> and <node:port>
- Closed groups: members can only exchange messages with other members in same group
join(group:member) TIPC
Distributed
Binding Table recvmsg(OOB,
group:member,
node:port);
leave() TIPC
Distributed
Binding Table recvmsg(OOB|EOR,
group:member,
node:port);
TIPC
Distributed
Binding Table recvmsg(OOB|EOR,
group:member,
node:port);
TIPC COMMUNICATION GROUPS
Membership

3. TIPC COMMUNICATION GROUPS
Message Distribution Modes Within a Group
Unicast
28
60
34
7
28
60
34
7
Anycast
Multicast Broadcast
28
60
34
7
28
60
34
7
sendto(SOCKID,node:port);
recvmsg(group:member,node:port); recvmsg(group:member,node:port);
recvmsg(group:member,node:port); recvmsg(group:member,node:port);
send();
sendto(MBRID,group:member);
sendto(MCAST,group=NN:member=28);
All received messages come with two source addresses, <group:member> and <node:port>

4.  User instantiates its own “brokerless message bus”
 Same addressing properties as traditional TIPC
 But different traffic properties
 Four different message distribution modes
 Sequence order and delivery guaranteed, even between distribution modes
 Leveraging L2 broadcast/UDP multicast when deemed favorable
 End-to-end flow control
 Messages never rejected because of destination buffer overflow
 Same mechanism covers all message types/modes
 Point-to-multipoint, - “sliding window” algorithm
 Multipoint-to-point, - “coordinated sliding window”
 Available from Linux 4.14
TIPC COMMUNICATION GROUPS
Basic Features

5. GROUP UNICAST
28
60
34
7
recvmsg(group=NN:member=60,node:port);
sendto(addrtype=SOCKID,node:port);
 Unicast messages are sent using the ephemeral socket address <node:port>
 The socket address is known from the received membership event, or from any received
message from that socket
Member Id: <group=NN:member=13>

6. GROUP ANYCAST
28
60
34
7
recvmsg(group=NN:member=60,node:port);
sendto(addrtype=MBRID,group=NN:member=28);
 Anycast messages are sent using a logical member address <group, member> (aka <type, instance>)
 If there is more than one member bound to the identity, destination is selected by round-robin among those
members
 Optionally, the sending member may be made eligible as recipient if it matches (loopback option)
 If the selected member is congested (too low advertised window) the algorithm continues until it finds a non-
congested destination
 If no non-congested destination is found, it blocks/returns EAGAIN

7. GROUP MULTICAST
28
60
34
7
recvmsg(group=NN:member=60,node:port);
sendto(addrtype=MCAST,group=NN:member=28); recvmsg(group=NN:member=60,node:port);
 Multicast messages are sent using a logical member address <group, member> (aka <type, instance>)
 If there is more than one member bound to the identity, all of them will receive a copy of the message
 Optionally, the sending member may be made eligible for receiving a copy if it matches
(loopback option)

8. GROUP BROADCAST
28
60
34
7
recvmsg(group:member,node:port);
send();
recvmsg(group:member,node:port);
 Broadcast messages are sent using the send() primitive, since no destination address is needed
 All members of the group, irrespective of member identity, receive a copy of the message
 Optionally, the sending member may be made eligible for receiving a copy (loopback option)

9. GROUP BROADCAST AND MULTICAST
Carried as link layer broadcast or “replicast”
Dedicated “Broadcast Link” using UDP multicast
or L2 broadcast
› Consumes CPU cycles on non-destination nodes
Drop
Drop
Drop
Drop
Accept
Drop
Drop
Accept
Drop
Drop
Drop
Accept
Drop
Drop
Drop
Replicate and transmit message on discrete links to
identified nodes only
› When multicast/broadcast media support is missing
› When number of real destination nodes << group size
Accept
Accept
Accept
A group broadcast/multicast sender is able to switch algorithm
on the fly without compromising sequential delivery

10. ADVANTAGES OVER TRADITIONAL MODELS
 Provides “connectionless” communication with flow
control
 Loss-free datagram messaging, as opposed to UDP and traditional
connectionless TIPC
 Loss-free multicast messaging, as opposed to traditional UDP
multicast groups
 Simple programming model
 Single socket for sending, receiving and membership events
 Just do a single join() call, then wait for membership events
or messages
 Sequence/cardinality guarantee comes without any user effort
 Memory and resource efficient
 A single socket in Linux occupies ~4 kB
 Standard connection-oriented TIPC or TCP would need N x
4kB per member in an N-member group
 No timers
 Reserves (N x min_window) + (M x max_window), where
M << N
 Standard TIPC or TCP would reserve (N x max_window)
 Bandwidth efficient
 Leverages L2 broadcast or UDP multicast when possible
 A corresponding group using standard TIPC or TCP would have
to replicate all messages
 Scales to hundreds of members without choking
 Leverages all other TIPC advantages
 Service addressing, - no need to know the whereabouts of
other members
 Immediate reception of membership events without active
monitoring

11. MORE INFORMATION
TIPC Overview
https://www.slideshare.net/JonMaloy/intro-to-the-tipc-messaging-service
TIPC Overlapping Ring Neighbor Monitoring
https://www.youtube.com/watch?v=ni-iNJ-njPo
TIPC project page
http://tipc.sourceforge.net
TIPC protocol specification
http://tipc.sourceforge.net/doc/draft-spec-tipc-10.html
TIPC programmer’s guide
http://tipc.sourceforge.net/doc/tipc_2.0_prog_guide.html
Communication Group Demo/Test program
http://sourceforge.net/p/tipc/tipcutils/ci/master/tree/bus_test