MLAG provides invisible Layer 2 redundancy across switches by making them appear as a single logical switch. It establishes dual-connected ports across switches and synchronizes MAC address tables and BPDUs to eliminate duplicate packets and prevent spanning tree loops. MLAG configuration involves bonding dual-connected ports with a common CLAG ID and running the CLAGD protocol over a peer link to synchronize state.
2. ®
Agenda
u What is MLAG?
u How does MLAG work?
u How to set up an MLAG
u Tools for MLAG analysis and debugging
3. ®
MLAG Introduction
You need to set up a rack of servers for a new
application
u Add some extra servers for redundancy
u Uplink to redundant core switches
u Redundant Internet connections
u Backup power with batteries and generators
u Over-provisioned cooling
You receive a midnight call that everything is
down
4. ®
MLAG Introduction
MLAG – A LAG across more than one node
u Multi-homing for redundancy
u Active-active to utilize all links which otherwise
may get blocked by Spanning Tree
u No modification of LAG partner
5. ®
MLAG Terminology
S1 S2
H1 H2 H3 H4 H5
Secondary Role
ISL – Inter-Switch Link
Dually Connected
Primary Role
Singly Connected
8. ®
MLAG and LACP
u Both ends must run LACP
u Normally, when connected
to two different systems,
only one link is used
• Common system ID is used on
each switch
u Identification of which ports
on each system are dual-
connected pairs
S1 S2
H1 H2 H3 H4 H5
9. ®
Eliminating Duplicate Packets
u BUM1 packets are flooded and result in:
§ Duplicate packets at dual-connected hosts
§ A dual-connected host receives packets which
it transmitted
1 BUM packets are: Broadcast, Unknown unicast, and Multicast
11. ®
Eliminating Duplicate Packets
S1 sends the packet out all interfaces in the bridge, except the
interface on which the packet arrived
S1 S2
H1 H2 H3 H4 H5
12. ®
Eliminating Duplicate Packets
S2 sends the packet out all interfaces in the bridge, except the
interface on which the packet arrived
S1 S2
H1 H2 H3 H4 H5
13. ®
Eliminating Duplicate Packets
u Dual-connected hosts receive duplicate
copies of the packet
u Dual-connected hosts which send BUM
packets receive the packet they sent
u To fix this: Packets received on the ISL are
not forwarded to dual-connected ports
15. ®
MAC Address Learning
u To act as a single logical switch, both switches
must synchronize their MAC address tables
§ Addresses learned on dual-connected ports are
added to the corresponding port on the other
switch
§ Addresses learned on singly-connected ports are
added to the ISL on the other switch
§ Address learning is disabled on the ISL
17. ®
MAC Address Learning
S1 sends the packet out all interfaces in the bridge, except the
interface on which the packet arrived
S1 S2
H1 H2 H3 H4 H5
H2
18. ®
MAC Address Learning
S2 would ordinarily learn H2 on the ISL and forward the
packet out all singly-connected ports
S1 S2
H1 H2 H3 H4 H5
H2
H2
19. ®
MAC Address Learning
But, learning is disabled on the ISL. Instead, S1 sends a MAC
sync message to S2 which adds H2 to the dual connected port
S1 S2
H1 H2 H3 H4 H5
H2 H2
20. ®
MAC Address Learning
For singly-connected hosts, the MAC sync message causes the
address to be added to the ISL
S1 S2
H1 H2 H3 H4 H5
H1
H1
21. ®
MAC Address Learning
Final MAC address tables may look like this. Red: Address
originally learned on switch. Blue: Address added by MAC sync
S1 S2
H1 H2 H3 H4 H5
H2 H5H1 H4H3
H5
H2
H1
H4H3
22. ®
Switch-Switch MLAG
u Just like a host can be connected to two
switches, a pair of MLAG'd switches can be
connected to another pair of MLAG'd
switches
§ Used to create larger redundant L2 networks
§ Each pair of MLAG'd switches views the other
switches as a single logical switch
25. ®
Spanning Tree
u One switch is set as the primary,
the other is secondary
u Both switches use the same
bridge ID, dual connected ports
have the same port ID
u Only primary sends BPDUs on
dual-connected ports
u BPDUs received on dual
connected ports are sent to the
peer unmodified
u BPDUs received on the root port
are sent to the peer unmodified
u Source MACs of BPDUs received
on peer link are checked
u Peer link never blocks
S1 S2
M1
R1
26. ®
Split Brain
u If one switch sees that
the ISL is down it
cannot distinguish
between the link going
down (split brain) and
the peer switch going
down (solo)
u A backup link is used to
make this distinction
S1 S2
H1 H2 H3 H4 H5
S1 S2
H1 H2 H3 H4 H5
??? Which One ???
27. ®
Split Brain
u When the ISL goes
down, the backup
link can determine
if the peer switch is
still alive
S1 S2
H1 H2 H3 H4 H5
28. ®
Configuring MLAG
In /etc/network/interfaces put all dual-connected ports
in an 802.3ad bond and assign them a clag-id
auto bond1
iface bond1 inet static
bond-slaves swp48
bond-mode 802.3ad
bond-miimon 100
bond-use-carrier 1
bond-lacp-rate 1
bond-min-links 1
bond-xmit_hash_policy layer3+4
clag-id 1
auto bond11
iface bond11 inet static
bond-slaves swp4
bond-mode 802.3ad
bond-miimon 100
bond-use-carrier 1
bond-lacp-rate 1
bond-min-links 1
bond-xmit_hash_policy layer3+4
clag-id 1
Switch S1 Switch S2
29. ®
Configuring MLAG
In /etc/network/interfaces assign clagd
parameters on a VLAN sub-interface of the ISL link
auto peer6.4000
iface peer6.4000 inet static
address 169.254.0.1
netmask 255.255.255.0
clagd-peer-ip 169.254.0.2
clagd-sys-mac 44:38:39:ff:bb:01
clagd-backup-ip 192.168.1.101
auto peer16.4000
iface peer16.4000 inet static
address 169.254.0.2
netmask 255.255.255.0
clagd-peer-ip 169.254.0.1
clagd-sys-mac 44:38:39:ff:bb:01
clagd-backup-ip 192.168.1.100
Switch S1 Switch S2
30. ®
MLAG Tools
clagctl can be used to get the current state of
the MLAG
# clagctl
The peer is alive
Peer Priority, ID, and Role: 32768 00:02:00:00:00:17 primary
Our Priority, ID, and Role: 32768 70:72:cf:e9:f0:76 secondary
Peer Interface and IP: peer6.4000 169.254.0.2
Backup IP: 192.168.1.101 (active)
System MAC: 44:38:39:ff:bb:01
Dual Attached Ports
Our Interface Peer Interface CLAG Id
---------------- ---------------- -------
bond4 bond14 4
bond5 bond15 5
bond1 bond11 1
bond2 bond12 2
bond3 bond13 3
$ clagctl
The peer is alive
Our Priority, ID, and Role: 32768 00:02:00:00:00:17 primary
Peer Priority, ID, and Role: 32768 70:72:cf:e9:f0:76 secondary
Peer Interface and IP: peer16.4000 169.254.0.1
Backup IP: 192.168.1.100 (active)
System MAC: 44:38:39:ff:bb:01
Dual Attached Ports
Our Interface Peer Interface CLAG Id
---------------- ---------------- -------
bond14 bond4 4
bond15 bond5 5
bond12 bond2 2
bond13 bond3 3
bond11 bond1 1
Switch S1 Switch S2
31. ®
MLAG Tools
/var/log/syslog contains MLAG status changes
# grep clagd /var/log/syslog
May 19 16:25:31 act-5712-08 clagd[7253]: Beginning execution of clagd version 1.1.0
May 19 16:25:31 act-5712-08 clagd[7253]: Invoked with: /usr/sbin/clagd --daemon 169.254.0.2
peer6.4000 44:38:39:ff:bb:01
May 19 16:25:31 act-5712-08 clagd[7258]: Role is now secondary
May 19 16:25:32 act-5712-08 clagd[7258]: Initial config loaded
May 19 16:25:33 act-5712-08 clagd[7258]: The peer switch is active.
May 19 16:25:33 act-5712-08 clagd[7258]: Initial data sync from peer done.
May 19 16:25:33 act-5712-08 clagd[7258]: Initial handshake done.
May 19 16:25:33 act-5712-08 clagd[7258]: Initial data sync to peer done.
May 19 16:25:37 act-5712-08 clagd[7258]: bond2 is now dual connected.
May 19 16:25:37 act-5712-08 clagd[7258]: bond3 is now dual connected.
May 19 16:25:37 act-5712-08 clagd[7258]: bond1 is now dual connected.
May 19 16:25:37 act-5712-08 clagd[7258]: bond5 is now dual connected.
May 19 16:25:37 act-5712-08 clagd[7258]: bond4 is now dual connected.