Not learning Mac Address of ASA


Cumulus does not learn mac of interface connected to ASA.
ASA does learn mac of cumulus interface.

Cisco ASA 5505 (8.2) e0/1 connected to swp1

ASA:

interface Ethernet0/1 switchport access vlan 2

R1(config)# sh interface vlan 2
Interface Vlan2 "private", is up, line protocol is up
Hardware is EtherSVI, BW 100 Mbps, DLY 100 usec
MAC address 0021.55bc.288c, MTU 1500
IP address 10.1.1.1, subnet mask 255.255.255.0

R1(config)# capture c1 interface private circular-buffer real-time
Warning: using this option with a slow console connection may
result in an excessive amount of non-displayed packets
due to performance limitations.

Use ctrl-c to terminate real-time capture

1: 16:45:56.329572 802.1Q vlan#2 P0 arp who-has 10.1.1.1 tell 10.1.1.2
2: 16:45:56.330152 802.1Q vlan#2 P0 arp reply 10.1.1.1 is-at 0:21:55:bc:28:8c

R1(config)# sh arp
private 10.1.1.2 2c60.0cc0.7de4 63

Quanta LY9:

root@S1:~# ifquery -a
iface lo inet loopback
address 172.10.1.1
netmask 255.255.255.255

auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1
address 10.1.1.2/24

auto vtep1000
iface vtep1000
vxlan-id 1000
vxlan-local-tunnelip 172.10.1.1

auto br100
iface br100
bridge-ports swp1 vtep1000

root@S1:~# arp -an
? (10.1.1.1) at on swp1

root@S1:~# brctl showmacs br100
port name mac addr vlan is local? ageing timer
swp1 00:21:55:bc:28:8c 0 no 102.32
vtep1000 2a:58:44:ff:55:8f 0 yes 0.00
swp1 2c:60:0c:c0:7d:e4 0 yes 0.00

root@S1:~# ifconfig swp1
swp1 Link encap:Ethernet HWaddr 2c:60:0c:c0:7d:e4
inet addr:10.1.1.2 Bcast:10.1.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:4108 errors:0 dropped:297 overruns:0 frame:0
TX packets:6082 errors:0 dropped:3811 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:274278 (267.8 KiB) TX bytes:545728 (532.9 KiB)

root@S1:~# ip link show swp1
3: swp1: mtu 9000 qdisc pfifo_fast master br100 state UP mode DEFAULT qlen 500
link/ether 2c:60:0c:c0:7d:e4 brd ff:ff:ff:ff:ff:ff

root@S1:~# ip link show br100
61: br100: mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 2a:58:44:ff:55:8f brd ff:ff:ff:ff:ff:ff

root@S1:~# brctl show
bridge name bridge id STP enabled interfaces
br100 8000.2a5844ff558f no swp1 vtep1000

root@S1:~# uname -a
Linux S1 3.2.65-1+deb7u2+cl2.5+2 #3.2.65-1+deb7u2+cl2.5+2 SMP Mon Jun 1 18:26:59 PDT 2015 x86_64 GNU/Linux

15 replies

what you are asking is why cl is not learning ARP of ASA.

the fact that cl is responding to arp query from ASA, but ASA is not responding to cl, I would debug more on ASA side.

is this firewall configiured to respond to arp?

I'm not sure how it is a problem with the ASA.
From below the ASA responds to the ARP request with the correct mac.

CL never sees the arp reply.
I have flushed arptables and ebtables just to ensure nothing is being dropped as well.

18:03:01.229418 ARP, Request who-has 10.1.1.1 tell 10.1.1.2, length 28
18:03:02.229405 ARP, Request who-has 10.1.1.1 tell 10.1.1.2, length 28
18:03:03.229417 ARP, Request who-has 10.1.1.1 tell 10.1.1.2, length 28
18:03:04.229427 ARP, Request who-has 10.1.1.1 tell 10.1.1.2, length 28

1: 17:30:17.463599 802.1Q vlan#2 P0 arp who-has 10.1.1.1 tell 10.1.1.2
2: 17:30:17.463751 802.1Q vlan#2 P0 arp reply 10.1.1.1 is-at 0:21:55:bc:28:8c
3: 17:30:18.463629 802.1Q vlan#2 P0 arp who-has 10.1.1.1 tell 10.1.1.2
4: 17:30:18.463782 802.1Q vlan#2 P0 arp reply 10.1.1.1 is-at 0:21:55:bc:28:8c

Hi Nate,

Looks like the cumulus config has ip address 10.1.1.2 on swp1, indicating it is a L3 interface...

auto swp1
iface swp1
address 10.1.1.2/24

But swp1 is also included in br100 bridge interface...

auto br100
iface br100
bridge-ports swp1 vtep1000

Suspect the arp reply is received by cumulus and passed to bridge, not the L3 ip address.

Would try removing the ip address from swp1 and adding it to br100 instead.
ex.

auto swp1
iface swp1

auto br100
iface br100
address 10.1.1.2/24
bridge-ports swp1 vtep1000

Paul Schmidt wrote:

Hi Nate,

Looks like the cumulus config has ip address 10.1.1.2 on swp1, indicating it is a L3 in...

ideally ifquery should spot this error.
hmm, in that case,

is swp1 suppose to be in bridge?
( it is a routed port, seems like a uplink for vxlan)
Thanks so much for your time and assistance.
I am new to cumulus and trying to setup vxlan in a lab environment.
Here is a diagram of what I have setup. From the documentation it appears that the two cumulus switches need to have a layer 3 relationship via the loopback addresses for vxlan. I guess I am confused as how this needs to be setup.



I personally haen't got chance to setup the cumulus vxlan yet,
but concept wise, I think loopback is the tunnel end point of vxlan, it is the vtep ip addr.

Hi Nate,

Not clear at this point if ultimate goal is to set up controller-less VXLAN or Lightweight Network Virtualization (LNV).

Controller-less VXLAN requires pre-configuring remote MAC address, and therefore does not scale well for most practical implementations.

https://docs.cumulusnetworks.com/display/DOCS/Static+MAC+Bindings+with+VXLAN

LNV does not require an external controller. LNV runs the VXLAN service and registration daemons on Cumulus Linux itself.

https://docs.cumulusnetworks.com/display/DOCS/Lightweight+Network+Virtualization+-+LNV

Presently, it appears you may be initially attempting to configure controller-less VXLAN. In that case, here is relevant config of simple example that does not use loopback address.

SWITCH1:

auto swp1
iface swp1
address 10.1.2.19/24

auto swp2
iface swp2

auto vtep1000
iface vtep1000
vxlan-id 1000
vxlan-local-tunnelip 10.1.2.19

auto br100
iface br100
bridge-ports swp2 vtep1000
post-up bridge fdb add 70:72:cf:9d:4e:4d dev vtep1000 dst 10.1.2.20 vni 1000

SWITCH2:

auto swp1iface swp1
address 10.1.2.20/24

auto swp2
iface swp2

auto vtep1000
iface vtep1000
vxlan-id 1000
vxlan-local-tunnelip 10.1.2.20

auto br100
iface br100
bridge-ports swp2 vtep1000
post-up bridge fdb add 70:72:cf:9d:48:1d dev vtep1000 dst 10.1.2.19 vni 1000

notes:
1. This example (controller-less) VXLAN uses static mac addresses. It assumes end system with mac address 70:72:cf:9d:4e:4d is connected to SWITCH2 and end system with mac address 70:72:cf:9d:48:1d is connected to SWITCH1. Therefore, packets rx'd on swp2 of SWITCH1 with a destination mac of 70:72:cf:9d:48:1d will be tunneled to SWITCH2 via swp1, and vice versa.
2. This example does not use the loopback address. In this case, the local tunnel IP is the same as the interface ip on swp1.It is possible to use the loopback interface instead. In that case, it
would also be necessary to have ip routing between the loopback addresses.
3. Additional mac addresses can be added by adding additional post-up bridge... commands.
4. This will not usually scale. Therefore, recommend looking into LNV as alternative if possible.
5. Also, not sure which switch platform is in use. Please be aware that trident II chipset is required for VXLAN.

Hope this helps.




static config is not as bad as it look, as long as cl tell T2 to flood, questions I had would be,
~~~
will this static config support multiple remote vtep(s)?
how is broadcast associate with n x vteps?
will link local get flooded? ( does cl terminate any BPDU locally ? )

Hi Eric,

Only one vtep per bridge is supported, however there can be multiple bridges.
Believe broadcasts are not forwarded in this scenario (ie. Static arps may required)
Believe that BPDU is not flooded in this scenario.

Paul,

Thanks so much. LNV is actually what I was looking for. Going to try and set this up instead.

I am trying to setup NLV with just two leaf's. Layer 3 communication between loopbacks is established however the remote peers are not showing up.
Does this look right ?

root@S2:~# vxrdctl vxlans
VNI Local Addr Svc Node
=== ========== ========
100 172.20.1.1 0.0.0.0

root@S2:~# vxrdctl peers
VNI Peer Addrs
=== ==========
100 172.20.1.1

root@S1:~# vxrdctl vxlans
VNI Local Addr Svc Node
=== ========== ========
100 172.10.1.1 0.0.0.0
root@S1:~# vxrdctl peers
VNI Peer Addrs
=== ==========
100 172.10.1.1

auto lo
iface lo inet loopback
address 172.10.1.1
netmask 255.255.255.255
vxrd-src-ip 172.10.1.1

auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1
address 192.168.10.2/30
link-speed 100
link-duplex full

auto swp2
iface swp2

auto vni-100
iface vni-100
vxlan-id 100
vxlan-local-tunnelip 172.10.1.1

auto br-100
iface br-100
bridge-ports swp2 vni-100

root@S2:~# ifquery -a

auto lo
iface lo inet loopback
address 172.20.1.1/32
vxrd-src-ip 172.20.1.1

auto eth0
iface eth0 inet dhcp

auto swp1
iface swp1
link-speed 100
link-duplex full
address 192.168.20.2/30

auto swp2
iface swp2
link-speed 100
link-duplex full

auto vni-100
iface vni-100
vxlan-id 100
vxlan-local-tunnelip 172.20.1.1

auto br-100
iface br-100
bridge-ports swp2 vni-100

Ni Nate,

A service node daemon (vxsnd) is required for LNV. The service node daemon can run on any switch running Cumulus Linux as long as the switch is not also running VXLAN VETP. Therefore, a third switch is needed. This service node appears to missing based on the provided configuration.
For example:
auto lo
iface lo inet loopback
address 172.10.1.1
netmask 255.255.255.255

vxrd-src-ip x.x.x.x <--- need ip address of service node daemon

vxrd-src-ip 172.10.1.1 vxrd-src-ip 10.2.1.1
It is also important to be aware the service node daemon (vxrd) and service node daemons(vxsnd) may need to be installed if they are not installed already. (packages are vxfld-vxrd and vxfld-vxsnd respectively). In addition, registration node daemon must also be enabled on each leaf switch acting as a VTEP. (/etc/default/vxrd) Service node daemon on service node switch (ie. spine) must also be enabled (/etc/vxsnd.conf).
Probably best to follow detailed explanation and full example directly from technical documentation here:

https://docs.cumulusnetworks.com/display/DOCS/Lightweight+Network+Virtualization+-+LNV
https://docs.cumulusnetworks.com/display/DOCS/LNV+Full+Example

If you don't have a third switch available for experimenting, consider trying out an LNV config in the cumulus workbench.

https://cumulusnetworks.com/get-started/test-drive-open-networking-in-our-remote-lab/

The workbench contains a four switch topology that can be used to configure the exact LNV full example from our technical documentation.

Hopes this helps.

Paul,

I added a third switch and set it up as the service node. It is working as expected 🙂

Thanks for the help !
Are you running BGP between ASA and SPINE's/Leafs?

Reply