Route Leaking from vrf to global and vice versa


I followed the instructions ( https://docs.cumulusnetworks.com/display/DOCS/Virtual+Routing+and+Forwarding+-+VRF ) to configure route-leaking in cumulus cloud , but couldn't get it to working.

I have also added the routes in vrf and global table using the below commands
net add routing route 100.124.0.0/25 vlan2 vrf cust1
net add routing route 100.124.1.0/24 vlan3

here is the output from

cumulus@leaf01:mgmt-vrf:~$ net show route | grep 100.124.[0-1].0
O 100.124.0.0/25 [110/10] is directly connected, vlan2, 01:28:15
C>* 100.124.0.0/25 is directly connected, vlan2
K * 100.124.1.0/24 is directly connected, unknown inactive
S 100.124.1.0/24 [1/0] is directly connected, unknown inactive

cumulus@leaf01:mgmt-vrf:~$ net show route vrf cust1 | grep 100.124.[0-1].0
S>* 0.0.0.0/0 [1/0] via 100.124.100.254, vlan101
K * 100.124.0.0/25 is directly connected, unknown inactive
S 100.124.0.0/25 [1/0] is directly connected, unknown inactive
C>* 100.124.1.0/24 is directly connected, vlan3
C>* 100.124.100.0/24 is directly connected, vlan101

i get a Unreachable when pinging from server01[100.124.1.11] to vlan2[100.124.0.1] ,
cumulus@server01:~$ ping 100.124.0.1
PING 100.124.0.1 (100.124.0.1) 56(84) bytes of data.
From 100.124.1.1 icmp_seq=1 Destination Host Unreachable

is vrf route leaking not supported in Cumulus VX ?

8 replies

Userlevel 1
Hi Vikram,

Route leaking is supported today, but not in FRR, however this is on the roadmap.

By adding the routes using the NCLU net add routing route command, the routes were added to FRR and not directly to the kernel. For route leaking to work, we need to add the routes to the kernel using the ip route add command, as is done in the documentation:
ip route add vrf blue 5.5.5.0/24 dev swp2  
Please trying removing the routes from FRR using the net del routing route command and re-add them to the kernel directly with ip route add.
Hello Nick,

I made the changes, but the route shows up without a ">" at the beginning

In the Global Routing Table
=====================

cumulus@leaf01:mgmt-vrf:~$ net show route | grep 100.124.[0-1].0
O 100.124.0.0/25 [110/10] is directly connected, vlan2, 00:31:04
C>* 100.124.0.0/25 is directly connected, vlan2
K * 100.124.1.0/24 is directly connected, unknown inactive

In the VRF Routing Table
===================

cumulus@leaf01:mgmt-vrf:~$ net show route vrf cust1 | grep 100.124.[0-1].0
S>* 0.0.0.0/0 [1/0] via 100.124.100.254, vlan101
K * 100.124.0.0/24 is directly connected, unknown inactive
C>* 100.124.1.0/24 is directly connected, vlan3
C>* 100.124.100.0/24 is directly connected, vlan101

i still can't get the pings to work , is this normal ?

btw, cumulus vx is running 3.4.1

Thanks,
Vikram

Userlevel 1
Hi Vikram,

You'll need to check the kernel routing table in each VRF using the command ip route show. Please share the exact ip route commands you used to add the routes to the kernel and the output of the following two commands:
ip route show
ip route show vrf cust1

Hi Nick,

server01 ip is 100.124.1.11/24 gw 100.124.1.1

Here are the commands i used to add the routes

##############Commands to add the routes#########

sudo ip route add 100.124.1.0/24 dev vlan3

sudo ip route add vrf cust1 100.124.0.0/25 dev vlan2

##############Show commands output#########

cumulus@leaf01:mgmt-vrf:~$ ip route show
10.1.0.1 via 169.254.0.1 dev swp51 proto bgp metric 20 onlink
10.1.0.2 via 169.254.0.1 dev swp52 proto bgp metric 20 onlink
10.1.1.2 proto bgp metric 20
nexthop via 169.254.0.1 dev swp51 weight 1 onlink
nexthop via 169.254.0.1 dev swp52 weight 1 onlink
10.1.1.11 proto bgp metric 20
nexthop via 169.254.0.1 dev swp51 weight 1 onlink
nexthop via 169.254.0.1 dev swp52 weight 1 onlink
10.1.1.12 via 169.254.0.1 dev swp2 proto bgp metric 20 onlink
10.1.2.1 proto bgp metric 20
nexthop via 169.254.0.1 dev swp52 weight 1 onlink
nexthop via 169.254.0.1 dev swp51 weight 1 onlink
10.1.2.2 proto bgp metric 20
nexthop via 169.254.0.1 dev swp52 weight 1 onlink
nexthop via 169.254.0.1 dev swp51 weight 1 onlink
10.1.2.11 proto bgp metric 20
nexthop via 169.254.0.1 dev swp51 weight 1 onlink
nexthop via 169.254.0.1 dev swp52 weight 1 onlink
10.1.2.12 proto bgp metric 20
nexthop via 169.254.0.1 dev swp52 weight 1 onlink
nexthop via 169.254.0.1 dev swp51 weight 1 onlink
10.1.3.11 via 169.254.0.1 dev swp44 proto bgp metric 20 onlink
100.124.0.0/25 dev vlan2 proto kernel scope link src 100.124.0.1
100.124.1.0/24 dev vlan3 scope link

cumulus@leaf01:mgmt-vrf:~$ ip route show vrf cust1
unreachable default metric 8192
100.124.0.0/25 dev vlan2 scope link
100.124.1.0/24 dev vlan3 proto kernel scope link src 100.124.1.1
100.124.100.0/24 dev vlan101 proto kernel scope link src 100.124.100.1

#########Here is the full config############

interface lo
address 10.1.1.1/32
address fd00:1:0:1::1/128

interface lo0

interface eth0
vrf mgmt
address dhcp

interface swp1
alias server01
bridge-access 3
ipv6 nd ra-interval 10
mtu 9216
no ipv6 nd suppress-ra

interface swp2
ipv6 nd ra-interval 10
no ipv6 nd suppress-ra

interface swp44
ipv6 nd ra-interval 10
no ipv6 nd suppress-ra

interface swp51
ipv6 nd ra-interval 10
no ipv6 nd suppress-ra

interface swp52
ipv6 nd ra-interval 10
no ipv6 nd suppress-ra

interface bridge
bridge-ports swp1
bridge-vids 2-3 101
bridge-vlan-aware yes

interface cust1
vrf-table 1002

interface mgmt
address 127.0.0.1/8
vrf-table auto

interface vlan2
address 100.124.0.1/25
alias Management Vlan
vlan-id 2
vlan-raw-device bridge

interface vlan3
address 100.124.1.1/24
alias Transit Network
vlan-id 3
vlan-raw-device bridge
vrf cust1

interface vlan101
address 100.124.100.1/24
alias cust1 lan
vlan-id 101
vlan-raw-device bridge
vrf cust1

hostname leaf01

frr version 3.1+cl3u1

frr defaults datacenter

username cumulus nopassword

service integrated-vtysh-config

log syslog informational

router bgp 65101
bgp router-id 10.1.1.1
neighbor swp1 interface remote-as external
neighbor swp2 interface remote-as external
neighbor swp44 interface remote-as external
neighbor swp51 interface remote-as external
neighbor swp52 interface remote-as external

address-family ipv4 unicast
network 10.1.1.1/32

address-family ipv6 unicast
network fd00:1:0:1::1/128
neighbor swp1 activate
neighbor swp2 activate
neighbor swp44 activate
neighbor swp51 activate
neighbor swp52 activate

router ospf
network 100.124.0.0/25 area 0
network 100.124.1.0/24 area 0

line vty

dot1x
mab-activation-delay 30
eap-reauth-period 0

radius
accounting-port 1813
authentication-port 1812

time

zone
Etc/UTC

ntp

servers
0.cumulusnetworks.pool.ntp.org iburst
1.cumulusnetworks.pool.ntp.org iburst
2.cumulusnetworks.pool.ntp.org iburst
3.cumulusnetworks.pool.ntp.org iburst

source
eth0

dns

nameserver
8.8.8.8 # vrf mgmt

snmp-server
listening-address localhost

pls let me know incase you need any additional information
Userlevel 1
Hi Vikram,

Can you confirm if you are attempting to ping between two hosts, or from a single host to each SVI on the Cumulus switch?

In your provided configuration, I only see one member port for the bridge. I've applied your configurations to a simple 3 switch topology in VX, with two switches emulating a host in each VLAN. In this scenario, I am able to ping between the switches emulating the hosts successfully. If you are only using a single host to ping both SVI addresses, can you add another host to your test and confirm that ping works between the hosts.

In my setup, I am able to ping the SVI address for the VLAN associated with the host where the ping command is executed. However, due to how the static route is configured to allow leaking to work, pinging the address of the SVI for the other VRF from the given host does not work. I don't believe this should be an issue in practice though, as it doesn't seem likely that a host would use this address to communicate with the switch directly.
Hi Nick,

i have added another server02[100.124.0.11/25] on swp2 to vlan 2 and ran the ping from 100.124.1.11 -> 100.124.0.11 and vice versa . i am still getting "Destination net Unreachable" from the gateway's.

leaf01 is where both the servers are connected.

BTW , this is the demo instance i am using [redacted]

Thanks,
Vikram
Userlevel 1
Hi Vikram,

I logged into the CITC instance to check the configuration. When I logged in, the static routes were missing on leaf01 in each VRF. I configured the following:
cumulus@leaf01:mgmt-vrf:~$ sudo ip route add 100.124.0.0/25 dev vlan2 vrf cust1
cumulus@leaf01:mgmt-vrf:~$ sudo ip route add 100.124.1.0/24 dev vlan3
And pings between the hosts are now working.

Thanks Nick

Reply