Unnumbered OSPF/BGP Configuration on VMware ESXi with Cumulus VX


Userlevel 1
By default, when bringing up the Cumulus VX virtual machine, all network ports/switch port interfaces point to a single virtual network called "VM Network." Consequently, it doesn’t allow point-to-point connections between the interfaces; thus, an unnumbered OSPF/BGP routing setup will not work out of the box. Quagga doesn’t identify the connections and chokes the VM, causing out-of-memory issues and kernel panics.

In order to enable point-to-point interfaces in ESXi, you need to create port groups where each set of connecting ports has a separate virtual network and VLAN ID to isolate them with other ports and interfaces. To enable this feature, follow these steps:
1. In vSphere Client, select the ESXi server and click the Configuration tab. Select Networking, which contains the network settings and then click on Properties tab (circled in red).

2. In the Properties window, click Add to create a new port group. Make sure you keep the default, Virtual Machine, selected in the next window. A wizard appears, where you can create a port group by assigning it a unique name and VLAN ID.


3. In order to create a two-leaf/two-spine topology with an unnumbered OSPF configuration, you need to create four port groups, as follows:
  • Network Label: L1S1 with VLAN ID 2001
  • Network Label: L1S2 with VLAN ID 2002
  • Network Label: L2S1 with VLAN ID 2003
  • Network Label: L2S2 with VLAN ID 2004


4. Edit the settings for the CumulusVX-Leaf1 VM (choose VM > Edit settings) and assign network labels to the adapters as follows:
  • Network adapter 2: L1S1
  • Network adapter 3: L1S2


5. Similarly, add port groups to the other VMs CumulusVX-Leaf2, CumulusVX-Spine1 and CumulusVX-Spine2 to ensure proper point-to-point connectivity:
  • For CumulusVX-Leaf2: [list]
  • Network adapter 2: L2S1
  • Network adapter 3: L2S2
For CumulusVX-Spine1:
  • Network adapter 2: L1S1
  • Network adapter 3: L2S1
For CumulusVX-Spine2:
  • Network adapter 2: L1S2
  • Network adapter 3: L2S2
[/list] 6. Now, you should be able to configure the interfaces as point-to-point and Quagga for unnumbered OSPF/BGP.

Caution: Type 2 hypervisors by VMware , i.e. VMware Workstation and Fusion don’t have the necessary advanced network configuration capabilities to create port groups, which are needed to setup point-to-point connection. So , unnumbered OSPF/BGP will not work properly with those hypervisors and can cause out-of-memory issues and kernel panic.

2 replies

It looks like the screenshots are no longer working. Can they be fixed? Thanks.
Thanks for the heads up and walk through

Reply