WARNING: Native VLANs on Cisco switches, and interference with bonded (802.3ad) and trunked (802.1q) ports.

Please heed this warning, getting the knowledge to write this note involved 80 hours of blood, sweat, and tears.

When working with bonded + trunked interfaces between servers and switches, remember:

1. Do not mismatch the native vlans between the two sides of a link configured with teaming/bonding (802.3ad) and VLAN trunking (802.1q).
2. Do not tag the native VLAN, this will interfere with the transport of the LACP frames used by teaming/bonding (802.3ad), causing the links to fail.

It is important that the native VLANs be identical on both sides of the link.

Linux systems do not appear to have the ability to alter their native VLAN. Futhermore, they actively discourage the use of VLAN 1 for host attachment as the untagged/native VLAN is where the control packets for LACP and other such protocols travel. Cisco IOS can change the native/untagged vlan with the instruction. Do not use this! Keep the native VLAN as VLAN1.

It is important that the native VLAN not be tagged on both sides of the link.

On cisco devices, watch for the use of the global config option , which will tag the LACP control frames, making them "invisible" to the server on the other side of the link that's expecting these frames to be untagged even on a trunk port.

Note that if this instruction is present in a switch, it cannot simply just be removed. Other trunked devices may be expecting this behaviour, and hence require either server or switchport reconfiguration to function as expected.

Symptoms of a problem are:

• LACP + VLAN trunk link coming up for 120 seconds or 2 minutes, then failing
• Failure occurs because the keepalive packets aren't being "seen" by both sides due to a native vlan mismatch
• troubleshooting is very difficult, as neither the Linux or IOS debugging is useful at the most verbose level
• Cisco Switch: "sh lacp nei" shows the partner ports (on the server), as state 0xD instead of the normal state 0x3D.

Cisco IOS Configuration

Cisco CatOS based devices require alternate configuration.

We require:

1. A port-channel interface to represent the bonded/teamed interfaces
2. Physical interfaces to join the port-channel
3. configuration parameters for the physical interfaces (trunking, speed, duplex) must match the parameters of the port-channel interface
4. The configurations below add trunking information

In this example, we are trunking VLANs 1,50-89 on the bonded interface.
interface Port-channel1
description LACP Channel Bundle for virt-host-1
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,50-89 <<< controls which VLANs this trunk can access, "all" for all
switchport trunk native vlan 1
switchport mode trunk
no ip address
no shutdown
speed 1000
duplex full
spanning-tree portfast trunk
exit

Add physical interfaces to the bonded interface (max 8), note that the "switchport" instructions must match.
interface GigabitEthernet1/1
description virt-host-1 port 1 (eth0)
switchport
switchport trunk encapsulation dot1q <<< sets trunk type, dot1q vs Cisco ISL
switchport trunk allowed vlan 1,50-89 <<< select vlans to expose on trunk
switchport trunk native vlan 1 <<< forces the native VLAN (see warning)
switchport mode trunk <<< puts the interface into trunk (802.1q) mode
no ip address
no shutdown
speed 1000
duplex full
spanning-tree portfast trunk <<< only used if this does not create an L2 loop
channel-protocol lacp <<< selects lacp as the bonding protocol (802.3ad)
channel-group 1 mode active <<< channel-group # must match port-channel #
!
interface GigabitEthernet1/2
description virt-host-1 port 2 (eth1)
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,50-89
switchport trunk native vlan 1
switchport mode trunk
no ip address
no shutdown
speed 1000
duplex full
spanning-tree portfast trunk
channel-protocol lacp
channel-group 1 mode active
!