Cumulus VX ping does not work over bonded interface


I have downloaded the 2 switches setup of Cumulus VX. It is a VirtualBox image. I use VirtualBox 5.0.8 on Macbook.

The nic2 is named "swp1" in both switches. The nic3 is named "swp2". When I bond them into "bond0" and set up IP address at the bond0 interface, I can not ping each other. If I shutdown the bonding interface and move the ip address into either "swp1" or "swp2", I can ping each other.

If I continue the lab to configure ospf over the "bond0" interface, the ospf neighbor comes up and the routing table is correct.

Has anybody have this problem before ?

5 replies

Userlevel 2
I think the issue is that the interface needs to be set to promiscuous mode inside of virtualbox.
Userlevel 3
In VirtualBox set nicpromisc1 and nicpromisc2 to allow-vms. Or cover all of your bases and set this for all nics from 2 on up.
Well, both interfaces are in Promiscuous Mode "allow all". I can ping over the physical interface, but not the bonded interface. So I think this shall not be related to any physical interface, right ?
Userlevel 3
It's difficult to help more without more information. Some things to look at are:
  1. The output of lldpctl on both switches. You can use this to verify the physical connectivity and that each switch is connected as you expect, which is sometimes difficult to do in a virtual environment.
  2. If everything looks proper in the lldpctl output, take a look at /proc/net/bonding/ on each switch. This shows the status of the bond from the bonding driver's perspective. Sometimes there are problems with the LACP status which prevent the bond from coming up.
  3. 'ip link show dev " is another tool you can use to determine if the bond is up or down.
  4. And finally, you can post your /etc/network/interfaces files. Perhaps there's an issue with the way you've configured the bonds that someone else can determine. If you're not using /etc/network/interfaces to set up the bonds, make sure that you are setting all of the necessary bond parameters with whatever mechanism you are using.
I use GNS3 to connect virtualbox images, apart from all the technicalities of integrating virtualbox with gns3, I also like the way I can drag and drop the images and visualize the topology.

Is it possible for you to test it with GNS3?

The ping worked just fine over the bonded interface.
Please check the config below

cumulus@leaf-1$ sudo ifquery -a
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1

auto bond0
iface bond0
address 12.0.0.1/24
bond-slaves swp1 swp2
bond-mode 802.3ad
bond-miimon 100
bond-use-carrier 1
bond-lacp-rate 1
bond-min-links 1
bond-xmit-hash-policy layer3+4

cumulus@leaf-2$ sudo ifquery -a
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1

auto swp2
iface swp2

auto bond0
iface bond0
address 12.0.0.2/24
bond-slaves swp1 swp2
bond-mode 802.3ad
bond-miimon 100
bond-use-carrier 1
bond-lacp-rate 1
bond-min-links 1
bond-xmit-hash-policy layer3+4

cumulus@leaf-2$

As soon as I configured the 2nd node, the log showed that the bond interface had come up

LOG

cumulus@leaf-1$ [ 236.863162] ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready

cumulus@leaf-2$ sudo ifup -a
[ 230.243520] 8021q: adding VLAN 0 to HW filter on device eth0
[ 230.409559] bonding: bond0 is being created...
[ 230.417683] bonding: bond0: setting mode to 802.3ad (4).
[ 230.419810] bonding: bond0: Setting MII monitoring interval to 100.
[ 230.433610] bonding: bond0: Setting LACP rate to fast (1).
[ 230.439149] bonding: bond0: setting xmit hash policy to layer3+4 (1).
[ 230.442528] bonding: bond0: Setting min links value to 1
[ 230.454482] e1000: swp2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[ 230.460239] 8021q: adding VLAN 0 to HW filter on device swp2
[ 230.466902] bonding: bond0: enslaving swp2 as a backup interface with an up link.
[ 230.481656] e1000: swp1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[ 230.487055] 8021q: adding VLAN 0 to HW filter on device swp1
[ 230.493999] bonding: bond0: enslaving swp1 as a backup interface with an up link.
[ 230.529882] ADDRCONF(NETDEV_UP): bond0: link is not ready
[ 230.531605] 8021q: adding VLAN 0 to HW filter on device bond0
cumulus@leaf-2$
cumulus@leaf-2$
cumulus@leaf-2$
cumulus@leaf-2$
cumulus@leaf-2$ [ 232.733550] ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready

cumulus@leaf-2$ ping 12.0.0.1 -c5
PING 12.0.0.1 (12.0.0.1) 56(84) bytes of data.
64 bytes from 12.0.0.1: icmp_req=1 ttl=64 time=3.78 ms
64 bytes from 12.0.0.1: icmp_req=2 ttl=64 time=1.46 ms
64 bytes from 12.0.0.1: icmp_req=3 ttl=64 time=1.02 ms
64 bytes from 12.0.0.1: icmp_req=4 ttl=64 time=1.47 ms
64 bytes from 12.0.0.1: icmp_req=5 ttl=64 time=1.84 ms

--- 12.0.0.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4012ms
rtt min/avg/max/mdev = 1.023/1.920/3.788/0.970 ms
cumulus@leaf-2$

Reply