VXLan / EVPN enabled hypervisors with quagga 1.0.0+cl3u11 in a leaf spine architecture


Hey there,

we're currently planning to redesign our datacenter network with cumulus enabled switches in a leaf-spine architecture.

At the moment we're having a test-setup with cumulus vx vms (2 spines, 2 leafs) and two guest-vms that should communicate through a custom vxlan.

For this setup I grabbed the quagga deamon in version 1.0.0+cl3u11 and deployed it to the guest-vms.

I configured an unnumbered eBGP setup and as far as I can see the BGP sessions are properly established for ipv4, ipv6 and evpn.

I created two vxlan interfaces, one on each guest, and assigned them to a bridge br0. I assigned an ip address on both systems and expected to be able to ping each other host.

Unfortunately the pings are not successful. I see the requests arrive on the other host and also the replies going out but they do not reach the first host.

I created a github repository with configs and cl-support files. You can find it here:

https://github.com/hensoko/cumulus-vxlan-evpn

Could you give me some markers what am I missing here?

Regards

Hendrik

3 replies

Userlevel 4
So you're attempting to run EVPN on the host in this scenario it looks like. Recommend running the version of Quagga that ships with 3.2.1 as there are no significant kernel modifications required there.

wget -O /root/quagga_1.0.0+cl3eau8_amd64.deb http://repo3.cumulusnetworks.com/repo/pool/cumulus/q/quagga/quagga_1.0.0+cl3eau8_amd64.debwget -O /root/quagga-doc_1.0.0+cl3eau8_all.deb http://repo3.cumulusnetworks.com/repo/pool/cumulus/q/quagga/quagga-doc_1.0.0+cl3eau8_amd64.deb
wget -O /root/quagga-dbg_1.0.0+cl3eau8_amd64.deb http://repo3.cumulusnetworks.com/repo/pool/cumulus/q/quagga/quagga-dbg_1.0.0+cl3eau8_amd64.deb


Because the Cumulus Quagga implementation is written for a TOR environment, to make it work on a host TODAY you need to make the host look like a TOR. To do that you can add a vETH which acts like a host-facing port might on a TOR. This behavior will be changing in 3.4 where you'll be able to add an IP address directly to the vxlan interface, but that software will require kernel support and other knobs to do arp suppression etc. 3.2.1 has no kernel dependencies.
Hey

thanks for your support. It worked as you mentioned when using hypervisors to terminate the vxlan and bridge it to guest-interfaces.

I wonder which version of quagga I should use. You provided some links of the cumulus debian repository to get the version cl3eau8 but I already installed cl3u11. Is cl3eau8 a special "early access" version with additional features that cannot yet be bound in the cl3u11?

Is there a release-candidate available for quagga in Cumulus 3.4?

Regards

Hendrik
Userlevel 4
Hendrik Sokolowski wrote:

Hey

thanks for your support. It worked as you mentioned when using hypervisors to terminate the ...

There isn't a release candidate available for Cumulus 3.4 yet, as we're aiming to shift to FRR for 3.4. Recommend using 3.2.1's EVPN implementation because 3.3 and 3.4 will require kernel changes to be able to support the arp suppression and MAC address functionality.

Reply