evpn type 5 routes


Userlevel 1


I have setup bgp evpn peering between leaf1 and leaf4 in Cumulus VX , trying to get evpn type 5 routes exchanged for vrf RED ( followed the instructions as per "Announcing EVPN Type-5 Routes" from https://docs.cumulusnetworks.com/display/DOCS/Ethernet+Virtual+Private+Network+-+EVPN ).

I couldn't get this to work, not sure if i have configured incorrectly.

Any help is appreciated.

is there an option to upload the configs over here ?

10 replies

Userlevel 1
I got it to work finally.  But i see icmp redirects when pinging from  server1 [ 10.10.10.10 ] to server4 [ 10.10.100.10 ]

cumulus@server1:~$ ping 10.10.100.10
PING 10.10.100.10 (10.10.100.10) 56(84) bytes of data.
From 10.10.10.1: icmp_seq=1 Redirect Host(New nexthop: 10.0.0.14)
64 bytes from 10.10.100.10: icmp_seq=1 ttl=62 time=7.36 ms
From 10.10.10.1: icmp_seq=2 Redirect Host(New nexthop: 10.0.0.14)
64 bytes from 10.10.100.10: icmp_seq=2 ttl=62 time=5.94 ms
From 10.10.10.1: icmp_seq=3 Redirect Host(New nexthop: 10.0.0.14)
64 bytes from 10.10.100.10: icmp_seq=3 ttl=62 time=3.89 ms

cumulus@leaf1:~$ net show route vrf RED
show ip route vrf RED
======================
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel,
> - selected route, * - FIB route

VRF RED:
K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:56:20
C>* 10.10.10.0/24 is directly connected, vlan10, 00:56:20
B>* 10.10.100.0/24 [20/0] via 10.0.0.14, vlan10 onlink, 00:43:41

10.0.0.14 is the loopback on leaf4, is it normal to see icmp redirects ?
seems like a confused kernel routing, would like to see what is cumulus's answer for this, in hardware based platform, cpu might not see this pkt at all, this could be cx specific issue?
Hi Vikram,

This is Diane, TME with Cumulus.

We support EVPN Type 5 routes with the L3VNI with the symmetric IRB model today (CL 3.5). According to your diagram and the output you provided, it looks like you are trying to run Type 5 routes over a L2VNI that should be already transporting the type 2 routes (which is not supported) - but I have not seen all your configs. I would need to see those to recreate and determine what is causing the icmp redirects.

In this setup, you may be better off keeping the asymmetric model - and do your VXLAN Routing using Type 2 EVPN routes. All that is required for this model is to configure the SVIs on the ToR. You will need to configure both VNIs on both ToRs as the asymmetric model routes only on the ingress ToR and thus traffic always travels on the destination VNI.

More information on symmetric vs asymmetric can be found here.

If you wish to use the symmetric model, we can do that also. It will also use Type 2 routes over the L3VNI for routing to a local host - you just won't need all the VNIs configured except the local ones on the local rack and the L3VNI and associated vlan. Type 5 routes are generally used for external routing only

Please let us know if you have any further questions.

Best Regards and have a great day!

Diane
Userlevel 1
Thanks Diane for the explanation. I was kind of lost when it came to the L3VNI and might have misconfigured. I was trying to simulate communication between 2 subnets which are in different DC’s but belong to the same tenant vrf using evpn. Will refer to the cldemo-evpn-symmetric you posted on github.
Hi Vikram,

Thanks for the reply and explanation.

You can send the subnet as a Type 5 route with the "advertise-subnet" command - but this is mostly used to announce silent hosts when arp suppression is on. After the /32 is learned (i.e. the destination host speaks) the Type 2 will be learned and the route will be a /32. Since the routing table follows the longest match rule, the Type 2 route is used with the L3VNI. CL3.5 does not yet support filtering the Type 2 routes with the L3VNI- this is coming in an upcoming release.

Userlevel 1
Hi Diane,

I re-did the configs in citc based on the configs from cldemo-evpn-symmetric (github) and i no longer see the icmp redirects. It was a L3VNI misconfiguration on my part in the first attempt 😞.

Thanks,
Vikram 🙂

Vikram A wrote:

Hi Diane,

I re-did the configs in citc based on the configs from cldemo-evpn-symmetric (github)...

Hi Vikram,

Would you happen to have the configs for this? I'm struggling to get this working in my own setup.
Userlevel 1
Vikram A wrote:

Hi Diane,

I re-did the configs in citc based on the configs from cldemo-evpn-symmetric (github)...

Hi Steve,
Here is the link to nclu configs for Cumulus in the Cloud Topology https://drive.google.com/drive/folders/1SiglA6Ar7LMg0vbcxJ257v0CdoywYv4u?usp=sharing

it's got the Diagram as well ( EVPN Symmetric model is what i have tried )

Thanks,
Vikram
Vikram A wrote:

Hi Diane,

I re-did the configs in citc based on the configs from cldemo-evpn-symmetric (github)...

Awesome thanks so much
Userlevel 1
pls run the below commands as these do not show up in "net show configuration commands" command and only show up in frr.conf

leaf01 & leaf04
================

net add vrf RED vni 310
net add vrf BLUE vni 320
net add vrf GREEN vni 330

leaf02 & leaf03
===============

net add vrf RED vni 310
net add vrf BLUE vni 320

Config files ( output from "net show configuration files" ) for each of the devices are available in the below link

https://drive.google.com/drive/folders/1M73mMdUwqKhTUinpeesYQelolzCF_koT?usp=sharing

Reply