Cumulus VX - 2 Spine 2 Leaf demo - ping not working


Userlevel 1
Running Cumulus VX 3.3.2 on a Ubuntu 14.04 server install.
Virtualbox is installed. Worked through the demos in the Intro to Cumulus VX Lab guide and wanted to try something bigger.
Followed the instructions at https://docs.cumulusnetworks.com/pages/viewpage.action?pageId=5126706
ping from leaf 1 to other VM's isn't successful.
Happy to provide additional info if required, just not sure what.
thank you
Rich

17 replies

Userlevel 3
Hi Richard, if you could post your configs (/etc/network/interfaces) for the VMs that would be awesome. Thanks!
Userlevel 1
Sure Pete, here you go along with Quagga.conf files

Vx-leaf1 $ cat /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback
address 10.2.1.1/32

# The primary network interface
auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1
address 10.2.1.1/32

auto swp2
iface swp2
address 10.2.1.1/32

auto swp3
iface swp3
address 10.4.1.1/24

Vx-leaf1 $ cat /etc/quagga/Quagga.conf
log file /var/log/quagga/quagga.log
log timestamp precision 6

service integrated-vtysh-config

interface swp1
ip ospf network point-to-point

interface swp2
ip ospf network point-to-point

router-id 10.2.1.1

router ospf
ospf router-id 10.2.1.1
network 10.2.1.1/32 area 0.0.0.0
network 10.4.1.0/24 area 0.0.0.0

***********************
Vx-leaf2 $ cat /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback
address 10.2.1.2/32

# The primary network interface
auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1
address 10.2.1.2/32
auto swp2
iface swp2
address 10.2.1.2/32
auto swp3
iface swp3
address 10.4.2.1/24

Vx-leaf2 $ cat /etc/quagga/Quagga.conf
log file /var/log/quagga/quagga.log
log timestamp precision 6

service integrated-vtysh-config

interface swp1
ip ospf network point-to-point

interface swp2
ip ospf network point-to-point

router-id 10.2.1.2

router ospf
ospf router-id 10.2.1.2
network 10.2.1.2/32 area 0.0.0.0
network 10.4.2.0/24 area 0.0.0.0

****************************************************************

Vx-spine1 $ cat /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback
address 10.2.1.3/32

# The primary network interface
auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1
address 10.2.1.3/32

auto swp2
iface swp2
address 10.2.1.3/32

auto swp3
iface swp3

Vx-spine1 $ cat /etc/quagga/Quagga.conf

log file /var/log/quagga/quagga.log
log timestamp precision 6

service integrated-vtysh-config

interface swp1
ip ospf network point-to-point

interface swp2
ip ospf network point-to-point

router-id 10.2.1.3

router ospf
ospf router-id 10.2.1.3
network 10.2.1.3/32 area 0.0.0.0

***************************************************************

Vx-spine2 $ cat /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback
address 10.2.1.4/32

# The primary network interface
auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1
address 10.2.1.4/32

auto swp2
iface swp2
address 10.2.1.4/32

auto swp3
iface swp3

Vx-spine2 $ cat /etc/quagga/Quagga.conf

log file /var/log/quagga/quagga.log
log timestamp precision 6

service integrated-vtysh-config

interface swp1
ip ospf network point-to-point

interface swp2
ip ospf network point-to-point

router-id 10.2.1.4

router ospf
ospf router-id 10.2.1.4
network 10.2.1.4/32 area 0.0.0.0

Userlevel 1
Anyone have any suggestions?
thanks
Userlevel 1
Richard,

Are your OSPF peers establishing?
  • vtysh -c "show ip ospf nei"
Are the LLDP neighbors properly detecting?
  • net show lldp
  • lldpctl
The configuration looks good so far, we may want to pivot our attention to make sure that the virtual L1 connections are set up correctly.

- Rama
Userlevel 1
Interesting, when i try to issue the vtysh -c ....above i get:

Exiting: failed to connect to any daemons

Userlevel 1
Try using:
sudo vtysh

You need root access to run the command.
Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

Ok that got me a big further now: sudo vtysh -c "show ip ospf nei"
[quote]


Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

The LLDP connections look a bit interesting. I would recommend teh following things:
1. Set the hostname on each of your switches so we can better validate LLDP.
net add hostname  
2. It looks like swp1 and swp2 are connected to the same host? Validate the connectivity.

3. Your OSPF peering is stuck in exstart. This generally means that the MTU isn't matching between your interfaces.
Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

Its looking like a basic problem with the setup - I was following the instructions on the how to set up this demo. I downloaded cumulus-linux-3.3.2-vx-amd64.ova & cloned 3x to make leaf2, spine1, spine2. Is there some setup in the VM's network configuration I am missing?
Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

It looks like the switches aren't set up correctly in virtualbox. When you configured the fix VX and then cloned it, it looks like it also copied the network settings from the original VX VM, resulting in crossed links.

Go and doublecheck your networking configuration by going to:
Settings > Networking

Make sure that each adapter is mapped to the correct network name. Using the linke you provided:
https://docs.cumulusnetworks.com/pages/viewpage.action?pageId=5126706
Adapter2 of spine1 and Adapter2 of leaf1 should be named the same network (maybe spine01-leaf01).

After you name the switches their logical names (ie. leaf1, leaf2, spine1, spine2), send in the output of lldpctl again and a screenshot of the Adapter2 network setting on leaf1 from virtualbox and we can isolate why the connection isn't working.
Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

lldpctl output:

-------------------------------------------------------------------------------

LLDP neighbors:

-------------------------------------------------------------------------------

Interface: swp1, via: LLDP, RID: 2, Time: 0 day, 00:25:09

Chassis:

ChassisID: mac 08:00:27:f1:b6:d8

SysName: vx-spine1

SysDescr: Cumulus Linux version 3.3.2 running on innotek GmbH VirtualBox

TTL: 120

MgmtIP: 10.2.1.3

MgmtIP: fe80::a00:27ff:fef1:b6d8

Capability: Bridge, off

Capability: Router, on

Port:

PortID: ifname swp1

PortDescr: swp1

PMD autoneg: supported: no, enabled: no

MAU oper type: 1000BaseTFD - Four-pair Category 5 UTP, full duplex mode

-------------------------------------------------------------------------------

Interface: swp2, via: LLDP, RID: 1, Time: 0 day, 00:25:24

Chassis:

ChassisID: mac 08:00:27:71:c3:9d

SysName: vx-spine2

SysDescr: Cumulus Linux version 3.3.2 running on innotek GmbH VirtualBox

TTL: 120

MgmtIP: 10.2.1.4

MgmtIP: fe80::a00:27ff:fe71:c39d

Capability: Bridge, off

Capability: Router, on

Port:

PortID: ifname swp1

PortDescr: swp1

PMD autoneg: supported: no, enabled: no

MAU oper type: 1000BaseTFD - Four-pair Category 5 UTP, full duplex mode

-------------------------------------------------------------------------------



The invalid settings is just about display and doesn't affect.
Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

Okay, so we know that leaf01 swp1 is assigned to the virtual network called l1-s1 and that that leaf01 swp1 connects to swp1 of spine1. We also see that swp2 connects to swp1 of spine2.

Can we double check to make sure there are no incorrectly mapped interfaces?

If you fixed all the names, get the output of net show lldp from all the VMs and upload them here. Using the lldp information we can verify the connectivity is all correct. We need the LLDP of all 4 hosts.

- Rama
Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

Hi Rama,
Good eyes, there were a couple of typos in the names I missed.
Leaf1: net show lldp

LocalPort Speed Mode RemotePort RemoteHost Summary

----------- ------- ------------ ------------ ------------ ---------------

swp1 1G Interface/L3 swp1 vx-spine1 IP: 10.2.1.1/32

swp2 1G Interface/L3 swp1 vx-spine2 IP: 10.2.1.1/32

Leaf 2: net show lldp

LocalPort Speed Mode RemotePort RemoteHost Summary

----------- ------- ------------ ------------ ------------ ---------------

swp1 1G Interface/L3 swp2 vx-spine1 IP: 10.2.1.2/32

swp2 1G Interface/L3 swp2 vx-spine2 IP: 10.2.1.2/32

Spine 1: net show lldp

LocalPort Speed Mode RemotePort RemoteHost Summary

----------- ------- ------------ ------------ ------------ ---------------

swp1 1G Interface/L3 swp1 vx-leaf1 IP: 10.2.1.3/32

swp2 1G Interface/L3 swp1 vx-leaf2 IP: 10.2.1.3/32

Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

Spine 2: net show lldp

LocalPort Speed Mode RemotePort RemoteHost Summary

----------- ------- ------------ ------------ ------------ ---------------

swp1 1G Interface/L3 swp2 vx-leaf1 IP: 10.2.1.4/32

swp2 1G Interface/L3 swp2 vx-leaf2 IP: 10.2.1.4/32

Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

All these connections look good now it seems. The IPs under Summary are interesting, but it could just be the way i'm interpreting the data.

Is OSPF up and operational now?
Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

I have made some progress and can ping 10.2.1.3 (spine 1) and 10.2.1.4 (spine 2) but not 10.2.1.2 (leaf 2) this is curious as 'net show lldp' reveals each spine can see each of the leafs and each leaf can see both spines. Perhaps this isn't a valid test
Which brings me to the documentation - the old version of the 2 spine 2 leaf documentation includes a lot more information that is missing from the latest version.
Perhaps someone can take a look into this.
thank you very much for your help Rama

https://docs.cumulusnetworks.com/display/VX30/Creating+a+Two-Spine%2C+Two-Leaf+Topology - old version

https://docs.cumulusnetworks.com/display/VX/Create+a+Two-Leaf%2C+Two-Spine+Topology - latest version

Userlevel 1
Rama Darbha wrote:

Try using:
sudo vtysh

You need root access to run the command.

Richard, don't hesitate to reach out if you need further assistance.

You can always hit me up realtime on our public slack channel.
To sign up: slack.cumulusnetworks.com
To login: cumulusnetworks.slack.com

Good luck!

Reply