Newbie Questions - eval Cumulus and HP altoline 6920


I am looking at replacing my core 10G network. I have looked at some arista switches and now I am evaluating these HP's with cumulus. Hoping to use this thread to ask questions along the way. I am setting up a MLAG, so I can LACP across the switches. My plan is to have 2 x 2 switches in 2 DC. I will have 2 x 10G connections that I was going to LACP . But I realise that might not be the best way to connect the switches. Is there another way. I do want to try and stay with open standards though. Definitely thing MLAG in the DC. where I can LACP to my mikrotek routers I found this article on how to vlan https://support.cumulusnetworks.com/hc/en-us/articles/204909397 with this The only reasons to use the traditional mode are: Familiarity with traditional Linux syntax. VXLAN support: The traditional mode currently (as of CL 2.5.5) supports VXLAN. The VLAN-Aware mode will support VXLANs in a Cumulus Linux future release. PVSTP+ interoperability: The traditional mode currently runs an instance of spanning tree per bridge. The VLAN-Aware STP mode is compatible with other types of spanning tree but only runs single instance MST. To achieve Per-VLAN STP/RSTP the traditional bridge mode must be used. I have just moved every thing to RSTP (Dell) and RPVST (?) on the cisco. should I be using traditional or the newer vlan aware... Currently don't have 200 vlans, but there might be a time in the future where I could have that many - not the near future.

21 replies

Userlevel 3
Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus Linux supports MLAG. You can read the documentation about how to configure it here.
It sounds like you should give the VLAN-aware bridging a try. Support for VxLAN on VLAN-aware bridges is coming soon (days, not weeks). As you say, VLAN-aware bridging is the more modern bridging code which mimics what other vendors in the industry do.
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

Thanks, I had found that. So you saying best to go with the newish vlan aware setup.
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

Something i wanted to ask i have this # # MLAG auto peerlink iface peerlink bond-slaves swp49 swp52 mtu 9216 auto peerlink.4094 iface peerlink.4094 address 169.254.255.1 netmask 255.255.255.0 clagd-priority 4096 clagd-peer-ip 169.254.255.2 clagd-backup-ip 10.172.250.52 clagd-sys-mac 44:38:39:ff:00:01 swp49,sw52 are 40G ports. Will this transfer all my vlans across it or do I have to add more config ?
Userlevel 4
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

The bridge configuration (not shown) is what actually controls which VLANS are allowed on to each of the ports in the bridge, something like this: auto bridge iface bridge bridge-vlan-aware yes bridge-ports peerlink uplink downlink bridge-stp on mstpctl-treeprio 4096 bridge-vids 1-1999 bridge-pvid 1
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

argh i think i missed that in the config https://docs.cumulusnetworks.com/display/DOCS/Multi-Chassis+Link+Aggregation+-+MLAG?_ga=1.64815791.184085857.1467590207 the bridge that adds the peerlink and the downlinks together. that explains why my L2 was screwed up as well !
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

just to add to this. I think I should assume that nothing routes/switches unless its bonded/bridged together.. This i think I am missing !
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

Also why in the example in the bridge definition is vlan 4094 not added to the bridge-vids ? is it because there is an explicit interface for 4094 that it gets added ?
Userlevel 4
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

The vlan-aware bridge in use here is called a vlan filtering bridge. The bridge filters which vlans can participate on the ports which comprise the bridge. The "bridge-vids" line specifies those vlans. The "bridge-vids" settings are just the default that are provided to all bridge ports, a port which is a member of a bridge can override them and that is what is happening here; only the peerlink port is participating in vlan 4094 hence it does not need to be specified on the main bridge (doing so would allow ALL bridge-ports to participate in 4094 which would be undesirable).
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

Hmm Okay new interesting problem hpsw1 & hpsw2 in mlag # # MLAG auto peerlink iface peerlink bond-slaves swp49 swp52 mtu 9216 auto peerlink.4094 iface peerlink.4094 address 169.254.255.1 netmask 255.255.255.0 clagd-priority 4096 clagd-peer-ip 169.254.255.2 clagd-backup-ip 10.172.250.52 clagd-sys-mac 44:38:39:ff:00:01 auto lacp-ybosw3 iface lacp-ybosw3 bond-slaves swp1 swp2 mtu 9216 clag-id 1 # LACP to ybortr2 # swp17 - sfp1 # swp17 - sfp2 auto lacp-ybortr2 iface lacp-ybortr2 bond-slaves swp17 mtu 9216 clag-id 2 mstpctl-portadminedge yes mstpctl-bpduguard yes # bridge that carries everything ?? auto br-trunk iface br-trunk mtu 9216 bridge-vlan-aware yes bridge-vids 4-5 8-11 17-19 24 80 95 99 201-205 207-209 212-222,250 bridge-pvid 1 bridge-ports lacp-ybosw3 lacp-ybortr2 swp33 peerlink bridge-stp on mstpctl-treeprio 32768 #mstpctl-treeprio 8192 and similiar on hpsw2 Now my stp root is on the dell - on the end of lacp-ybosw3 and I want to keep it that way Now I have loop dell switch -> hpsw1 and hpsw2 via the lacp link swp1&swp2 on both hpsw's and now the peerlink ... its going to block on the peerlink so traffic bound for hpsw2 from hpsw1 will have to go out the lacp to the dell switch and then back into hpsw2 on the lacp.... that seems like an issue ?
Userlevel 2
Scott Emery wrote:

Alex,

Glad you're trying out Cumulus Linux. To answer some of your questions/comments:

Cumulus ...

That would be a problem if you weren't using MLAG. MLAG allows for both devices to act as the same device. Meaning they are the same STP device as well. To the rest of the network they only see BPDUs that look like they are from the same system.

Also, since lacp-vbosw3 and lacp-ybortr2 show as lacp bonds from a single host to the other side, there is no loop!

Check out the bottom of the output from the "mstpctl showportdetail br-trunk lacp-ybortr2" for more clag/mlag specific output. In there you will see the system mac that you set that both devices own.
More strange issues hpsw1 config # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). source /etc/network/interfaces.d/*.intf # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 #iface eth0 inet dhcp iface eth0 address 10.172.250.51/24 gateway 10.172.250.1 mtu 9216 # # MLAG auto peerlink iface peerlink bond-slaves swp49 swp52 mtu 9216 auto peerlink.4094 iface peerlink.4094 address 169.254.255.1 netmask 255.255.255.0 clagd-priority 4096 clagd-peer-ip 169.254.255.2 clagd-backup-ip 10.172.250.52 clagd-sys-mac 44:38:39:ff:00:01 # LACP to ybosw3 # swp1 - sw1 -> ybosw3 A te1/0/18 # swp1 - sw2 -> ybosw3 B te1/0/19 # swp2 - sw1 -> ybosw3 A te2/0/18 # swp2 - sw2 -> ybosw3 B te2/0/19 auto lacp-ybosw3 iface lacp-ybosw3 bond-slaves swp1 swp2 mtu 9216 clag-id 1 # LACP to ybortr2 # swp17 - sfp1 # swp17 - sfp2 auto lacp-ybortr2 iface lacp-ybortr2 bond-slaves swp17 mtu 9216 clag-id 2 mstpctl-portadminedge yes mstpctl-bpduguard yes # bridge that carries everything ?? auto br-trunk iface br-trunk mtu 9216 bridge-vlan-aware yes bridge-vids 4-5 8-11 17-19 24 80 95 99 201-205 207-209 212-222,250 bridge-pvid 1 bridge-ports lacp-ybosw3 lacp-ybortr2 swp33 peerlink bridge-stp on mstpctl-treeprio 32768 #mstpctl-treeprio 8192 # # vlans auto br-trunk.213 iface br-trunk.213 address 10.172.213.21 netmask 255.255.255.0 # # esx host # swp33 - sw1 - vmnic2 te1/0/20 # swp33 - sw1 - vmnic7 te2/0/20 auto swp33 iface swp33 mtu 9216 mstpctl-portadminedge yes mstpctl-bpduguard yes hpsw2 config # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). source /etc/network/interfaces.d/*.intf # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 #iface eth0 inet dhcp iface eth0 address 10.172.250.52/24 gateway 10.172.250.1 mtu 9216 # # MLAG auto peerlink iface peerlink bond-slaves swp49 swp52 mtu 9216 auto peerlink.4094 iface peerlink.4094 address 169.254.255.2 netmask 255.255.255.0 clagd-priority 8192 clagd-peer-ip 169.254.255.1 clagd-backup-ip 10.172.250.51 clagd-sys-mac 44:38:39:ff:00:01 # LACP to ybosw3 # swp1 - sw1 -> ybosw3 A te1/0/18 # swp1 - sw2 -> ybosw3 B te1/0/19 # swp2 - sw1 -> ybosw3 A te2/0/18 # swp2 - sw2 -> ybosw3 B te2/0/19 auto lacp-ybosw3 iface lacp-ybosw3 bond-slaves swp1 swp2 mtu 9216 clag-id 1 # LACP to ybortr2 # swp17 - sfp1 # swp17 - sfp2 auto lacp-ybortr2 iface lacp-ybortr2 bond-slaves swp17 mtu 9216 clag-id 2 mstpctl-portadminedge yes mstpctl-bpduguard yes # bridge that carries everything ?? auto br-trunk iface br-trunk mtu 9216 bridge-vlan-aware yes bridge-vids 4-5 8-11 17-19 24 80 95 99 201-205 207-209 212-222,250 bridge-pvid 1 bridge-ports lacp-ybosw3 lacp-ybortr2 swp33 peerlink bridge-stp on mstpctl-treeprio 32768 #mstpctl-treeprio 8192 # # vlans auto br-trunk.213 iface br-trunk.213 address 10.172.213.22 netmask 255.255.255.0 # # esx host # swp33 - sw1 - vmnic2 te1/0/20 # swp33 - sw1 - vmnic7 te2/0/20 auto swp33 iface swp33 mtu 9216 mstpctl-portadminedge yes mstpctl-bpduguard yes I added br-trunk.213 live on the system did a ifreload -a on hpsw1 and then hpsw2 no strange thing is I kept getting port up and down for te1/0/19 and te2/0/19, which equates to swp1 and 2 on hpsw2 from my dell switch Aug 26 15:52:33 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90540 %% Link Down: Te2/0/19 Aug 26 15:52:33 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90541 %% Link on Te2/0/19 is failed Aug 26 15:52:48 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90542 %% Link Up: Te2/0/19 Aug 26 15:52:48 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90543 %% Link Up: Te1/0/19 Aug 26 15:53:03 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90544 %% Link Down: Te1/0/19 Aug 26 15:53:03 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90545 %% Link on Te1/0/19 is failed Aug 26 15:53:03 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90546 %% Link Down: Te2/0/19 Aug 26 15:53:03 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90547 %% Link on Te2/0/19 is failed Aug 26 15:53:05 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90548 %% Link Up: Te2/0/19 Aug 26 15:53:06 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90549 %% Link Up: Te1/0/19 Aug 26 15:53:21 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90550 %% Link Down: Te1/0/19 Aug 26 15:53:21 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90551 %% Link on Te1/0/19 is failed Aug 26 15:53:21 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90552 %% Link Down: Te2/0/19 Aug 26 15:53:21 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90553 %% Link on Te2/0/19 is failed Aug 26 15:53:24 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90554 %% Link Up: Te2/0/19 Aug 26 15:53:24 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90555 %% Link Up: Te1/0/19 Aug 26 15:53:39 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90556 %% Link Down: Te1/0/19 Aug 26 15:53:39 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90557 %% Link on Te1/0/19 is failed Aug 26 15:53:39 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90558 %% Link Down: Te2/0/19 Aug 26 15:53:39 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90559 %% Link on Te2/0/19 is failed Aug 26 15:53:42 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90560 %% Link Up: Te1/0/19 Aug 26 15:53:42 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90561 %% Link Up: Te2/0/19 Aug 26 15:53:57 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90562 %% Link Down: Te1/0/19 Aug 26 15:53:57 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90563 %% Link on Te1/0/19 is failed Aug 26 15:53:57 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90564 %% Link Down: Te2/0/19 Aug 26 15:53:57 10.172.250.8-1 TRAPMGR[245885632]: traputil.c(638) 90565 %% Link on Te2/0/19 is failed dmes from hpsw2 5935.777287] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 5935.790213] lacp-ybosw3: Setting ad_actor_system to 44:38:39:ff:00:01 [ 5935.796225] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 5937.878399] lacp-ybortr2: link status definitely up for interface swp17, 10000 Mbps full duplex [ 5937.878410] lacp-ybortr2: now running without any active interface! [ 6058.545990] lacp-ybortr2: Setting ad_actor_system to 48:0f:cf:b0:c1:f9 [ 6058.553777] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 6058.565426] lacp-ybosw3: Setting ad_actor_system to 48:0f:cf:b0:c1:ea [ 6058.572372] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 6059.053809] lacp-ybortr2: link status definitely down for interface swp17, disabling it [ 6059.053823] lacp-ybortr2: now running without any active interface! [ 6060.569439] lacp-ybortr2: Setting ad_actor_system to 44:38:39:ff:00:01 [ 6060.577176] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 6060.590643] lacp-ybosw3: Setting ad_actor_system to 44:38:39:ff:00:01 [ 6060.598783] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 6062.678635] lacp-ybortr2: link status definitely up for interface swp17, 10000 Mbps full duplex [ 6062.678646] lacp-ybortr2: now running without any active interface! [ 6195.410952] lacp-ybortr2: Setting ad_actor_system to 48:0f:cf:b0:c1:f9 [ 6195.416745] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 6195.426380] lacp-ybosw3: Setting ad_actor_system to 48:0f:cf:b0:c1:ea [ 6195.433544] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 6195.816492] lacp-ybortr2: link status definitely down for interface swp17, disabling it [ 6195.816506] lacp-ybortr2: now running without any active interface! [ 6209.480728] lacp-ybortr2: Setting ad_actor_system to 44:38:39:ff:00:01 [ 6209.487505] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 6209.495495] lacp-ybosw3: Setting ad_actor_system to 44:38:39:ff:00:01 [ 6209.503114] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 6211.487937] lacp-ybortr2: link status definitely up for interface swp17, 10000 Mbps full duplex [ 6211.487949] lacp-ybortr2: now running without any active interface! [ 6332.244189] lacp-ybortr2: Setting ad_actor_system to 48:0f:cf:b0:c1:f9 [ 6332.250522] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 6332.260719] lacp-ybosw3: Setting ad_actor_system to 48:0f:cf:b0:c1:ea [ 6332.266990] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 6332.950511] lacp-ybortr2: link status definitely down for interface swp17, disabling it [ 6332.950525] lacp-ybortr2: now running without any active interface! [ 6346.322332] lacp-ybortr2: Setting ad_actor_system to 44:38:39:ff:00:01 [ 6346.329789] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 6346.339705] lacp-ybosw3: Setting ad_actor_system to 44:38:39:ff:00:01 [ 6346.345410] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 6348.230639] lacp-ybortr2: link status definitely up for interface swp17, 10000 Mbps full duplex [ 6348.230651] lacp-ybortr2: now running without any active interface! this is not going to be any good if I can't add interfaces on the fly.. the up and down causes lots of problems.
I have tested with this watch -d -n 1 mstpctl showportdetail br-trunk ; clagctl I noticed hpsw2 thinks clag peer is down !!! Every 1.0s: mstpctl showportdetail br-trunk ; clagctl Fri Aug 26 16:09:50 2016 br-trunk:lacp-ybortr2 CIST info enabled yes role Designated port id 8.001 state forwarding external port cost 2000 admin external cost 0 internal port cost 2000 admin internal cost 0 designated root 2.000.D0:67:E5:AF:12:68 dsgn external cost 1000 dsgn regional root 8.000.44:38:39:FF:00:01 dsgn internal cost 0 designated bridge 8.000.44:38:39:FF:00:01 designated port 8.001 admin edge port no auto edge port yes oper edge port no topology change ack no point-to-point yes admin point-to-point auto restricted role no restricted TCN no port hello time 2 disputed no bpdu guard port no bpdu guard error no network port no BA inconsistent no Num TX BPDU 1340 Num TX TCN 5 Num RX BPDU 0 Num RX TCN 0 Num Transition FWD 1 Num Transition BLK 0 bpdufilter port no clag ISL no clag ISL Oper UP no clag role primary clag dual conn mac 00:00:00:00:00:00 clag remote portID F.FFF clag system mac 44:38:39:FF:00:01 br-trunk:lacp-ybosw3 CIST info enabled yes role Root port id 8.002 state forwarding external port cost 1000 admin external cost 0 internal port cost 1000 admin internal cost 0 designated root 2.000.D0:67:E5:AF:12:68 dsgn external cost 0 dsgn regional root 2.000.D0:67:E5:AF:12:68 dsgn internal cost 0 designated bridge 2.000.D0:67:E5:AF:12:68 designated port 6.097 admin edge port no auto edge port yes oper edge port no topology change ack no point-to-point yes admin point-to-point auto restricted role no restricted TCN no port hello time 2 disputed no bpdu guard port no bpdu guard error no network port no BA inconsistent no Num TX BPDU 22 Num TX TCN 21 Num RX BPDU 3283 Num RX TCN 2 Num Transition FWD 1 Num Transition BLK 0 bpdufilter port no clag ISL no clag ISL Oper UP no clag role primary clag dual conn mac 00:00:00:00:00:00 clag remote portID F.FFF clag system mac 44:38:39:FF:00:01 br-trunk:peerlink CIST info enabled no role Disabled port id 8.003 state discarding external port cost 1382 admin external cost 0 internal port cost 1382 admin internal cost 0 designated root 2.000.D0:67:E5:AF:12:68 dsgn external cost 1000 dsgn regional root 8.000.44:38:39:FF:00:01 dsgn internal cost 0 designated bridge 8.000.44:38:39:FF:00:01 designated port 8.003 admin edge port no auto edge port yes oper edge port no topology change ack no point-to-point yes admin point-to-point auto restricted role no restricted TCN no port hello time 2 disputed no bpdu guard port no bpdu guard error no network port no BA inconsistent no Num TX BPDU 3285 Num TX TCN 4 Num RX BPDU 64 Num RX TCN 3 Num Transition FWD 2 Num Transition BLK 2 bpdufilter port no clag ISL yes clag ISL Oper UP no clag role primary clag dual conn mac 00:00:00:00:00:00 clag remote portID F.FFF clag system mac 44:38:39:FF:00:01 br-trunk:swp33 CIST info enabled yes role Designated port id 8.004 state forwarding external port cost 2000 admin external cost 0 internal port cost 2000 admin internal cost 0 designated root 2.000.D0:67:E5:AF:12:68 dsgn external cost 1000 dsgn regional root 8.000.44:38:39:FF:00:01 dsgn internal cost 0 designated bridge 8.000.44:38:39:FF:00:01 designated port 8.004 admin edge port no auto edge port yes oper edge port no topology change ack no point-to-point yes admin point-to-point auto restricted role no restricted TCN no port hello time 2 disputed no bpdu guard port no bpdu guard error no network port no BA inconsistent no Num TX BPDU 3289 Num TX TCN 5 Num RX BPDU 0 Num RX TCN 0 Num Transition FWD 2 Num Transition BLK 1 bpdufilter port no clag ISL no clag ISL Oper UP no clag role primary clag dual conn mac 00:00:00:00:00:00 clag remote portID F.FFF clag system mac 44:38:39:FF:00:01 The peer is not alive Our Priority, ID, and Role: 4096 48:0f:cf:b0:71:9b primary Peer Interface and IP: peerlink.4094 169.254.255.2 Backup IP: 10.172.250.52 (active) System MAC: 44:38:39:ff:00:01 CLAG Interfaces Our Interface Peer Interface CLAG Id Conflicts Proto-Down Reason ---------------- ---------------- ------- -------------------- ----------------- lacp-ybortr2 - 2 - - lacp-ybosw3 - 1 - -
just did a dmesg -w got these, just sitting there [ 632.812359] lacp-ybortr2: Setting ad_actor_system to 48:0f:cf:b0:c1:f9 [ 632.818198] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 632.833084] lacp-ybosw3: Setting ad_actor_system to 48:0f:cf:b0:c1:ea [ 632.839179] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 633.438827] lacp-ybosw3: link status definitely down for interface swp1, disabling it [ 633.517914] lacp-ybortr2: link status definitely down for interface swp17, disabling it [ 633.517926] lacp-ybortr2: now running without any active interface! [ 633.538849] lacp-ybosw3: link status definitely down for interface swp2, disabling it [ 633.538858] lacp-ybosw3: now running without any active interface! [ 646.903283] lacp-ybortr2: Setting ad_actor_system to 44:38:39:ff:00:01 [ 646.909196] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 646.917001] lacp-ybosw3: Setting ad_actor_system to 44:38:39:ff:00:01 [ 646.923239] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 648.809997] lacp-ybortr2: link status definitely up for interface swp17, 10000 Mbps full duplex [ 648.810010] lacp-ybortr2: now running without any active interface! [ 649.024138] lacp-ybosw3: link status definitely up for interface swp2, 10000 Mbps full duplex [ 649.024150] lacp-ybosw3: now running without any active interface! [ 649.024157] lacp-ybosw3: link status definitely up for interface swp1, 10000 Mbps full duplex [ 769.722174] lacp-ybortr2: Setting ad_actor_system to 48:0f:cf:b0:c1:f9 [ 769.730590] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 769.740152] lacp-ybosw3: Setting ad_actor_system to 48:0f:cf:b0:c1:ea [ 769.747192] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 770.347568] lacp-ybosw3: link status definitely down for interface swp2, disabling it [ 770.347581] lacp-ybosw3: first active interface up! [ 770.347606] lacp-ybosw3: link status definitely down for interface swp1, disabling it [ 770.630788] lacp-ybortr2: link status definitely down for interface swp17, disabling it [ 770.630802] lacp-ybortr2: now running without any active interface! [ 783.858023] lacp-ybortr2: Setting ad_actor_system to 44:38:39:ff:00:01 [ 783.865246] 8021q: adding VLAN 0 to HW filter on device lacp-ybortr2 [ 783.873108] lacp-ybosw3: Setting ad_actor_system to 44:38:39:ff:00:01 [ 783.879080] 8021q: adding VLAN 0 to HW filter on device lacp-ybosw3 [ 785.865865] lacp-ybortr2: link status definitely up for interface swp17, 10000 Mbps full duplex [ 785.865876] lacp-ybortr2: now running without any active interface! [ 785.879872] lacp-ybosw3: link status definitely up for interface swp2, 10000 Mbps full duplex [ 785.879883] lacp-ybosw3: now running without any active interface! [ 785.879891] lacp-ybosw3: link status definitely up for interface swp1, 10000 Mbps full duplex
Seems like it might be a cable issue. Swapped the cables between hpsw1 and dell with hpsw2 and dell .. no issues. swap back and issues ! sigh makes testing very hard.
Userlevel 4
Alex Samad YB wrote:

Seems like it might be a cable issue. Swapped the cables between hpsw1 and dell with hpsw2 and...

If you have physical hardware, you probably also have a support contract. I would strongly recommend opening a support ticket in the future as it will get you IMMEDIATE support for these kinds of activities--> https://support.cumulusnetworks.com/hc/en-us/requests/new
Alex Samad YB wrote:

Seems like it might be a cable issue. Swapped the cables between hpsw1 and dell with hpsw2 and...

Thanks, its demo gear, evaulating the setup / switch. Seemed to have fixed it by swapping cables. The faulty ones on HPSW2 are not connected to HPSW1 and working fine .... thats strange !
Alex Samad YB wrote:

Seems like it might be a cable issue. Swapped the cables between hpsw1 and dell with hpsw2 and...

I have spoken too soon. My setup Hpsw1 … swp1 -> dell sw A 18 Hpsw1 … swp1 -> dell sw b 18 Hpsw2 … swp2 -> dell sw A 19 Hpsw2 … swp2 -> dell sw b 19 The cables that gave me issues on HPSW2 I swapped with HPSW1, the issue went away , infact all weekend and monday no issue - but I didn't make any change. Now I tried to add vrrp and did a ifreload -a if up -a and its ports are flapping seems to be every 10-20sec. note hpsw2 is the MLAG partner/second...
Another question around VRRP # # vlans auto br-trunk.212 iface br-trunk.212 address 10.172.212.21 netmask 255.255.255.0 # vrrp setup address-virtual 00:00:5E:00:01:ee 10.172.212.1/24 but I don't see 10.172.212.1 turn up I found the VRRP doc's under L2 documentation. But its very basic and it doesn't seem to be working. some other docs suggested I needed to install the vrrpd package - which I can't find.
Userlevel 3
I think you're confusing VRR and VRRP. VRR is sort of a play on the acronym, because it is like VRRP but without the protocol (the "P"). So there's no protocol running in VRR, which in my opinion makes it much more reliable. Instead, the two MLAG'd switches will both reply to ARPs for the virtual IP address, 10.172.212.1 in your example. Since they both reply with the exact same information, the ARP requester will not get confused. He will either discard the second reply, or use it and overwrite the first reply. Either way he's got the correct information. If one switch goes down, the other is still there and will still reply to the ARPs.

When you say it doesn't seem to be working, what are you actually seeing? Are the servers configured to use the virtual IP address as the default gateway? Can you ping that address from the servers? What's your route table look like (ip route show)? Do you have routing properly configured?
Hi

I had an old doco and I was looking for the vrrpd deamon, which doesn't seem to be there in the 3.0.1 version.

I have ti working. but I would like to be able to do more config. authentication and setting vrid EDIT - KNow that i look at this more. I see. So VRR is cumulus and VRRP is open standard. Can Cumulus do VRRP ?
More reading done. Know I know the difference between VRR and VRRP. The debian package for vrrpd doesn't do AH authentication. Mean no interop with my routers if( !strncmp(str, "ah/0x", 5 ) ){ vif->auth_type = VRRP_AUTH_AH; return -1; /* WORK: not yet implemented */

Reply