Solved

Spine won't route traffic out to the internet via a PFSense.


On the spine (config)

cumulus@stevelab-s-spine1:~$ net show configuration commands
net del all
net add time zone Etc/UTC
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add interface swp2-3 ospf network point-to-point
net add routing defaults datacenter
net add routing service integrated-vtysh-config
net add routing log syslog informational
net add routing route 0.0.0.0/0 10.1.1.9
net add routing line vty exec-timeout 0 0
net add ospf router-id 10.2.1.3
net add ospf network 10.1.1.8/30 area 0.0.0.0
net add ospf network 10.2.1.3/32 area 0.0.0.0
net add ospf network 10.20.1.0/24 area 0.0.0.0
net add ospf network 172.16.200.0/24 area 0.0.0.0
net add ospf default-information originate
net add dns nameserver ipv4 192.168.1.133
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp4,swp5,swp6,swp7
net add bridge bridge vids 200
net add bridge bridge vlan-aware
net add interface eth0 ip address 192.168.1.103/24
net add interface eth0 ip gateway 192.168.1.1
net add interface swp1 alias Internet Uplink PF Sense
net add interface swp1 ip address 10.1.1.10/29
net add interface swp2 alias L3 from Leaf1 to Spine1
net add interface swp2-3 ip address 10.2.1.3/32
net add interface swp3 alias L3 from Leaf2 to Spine1
net add interface swp4 alias L2 from Leaf1 to Spine1
net add interface swp4-5 bridge vids 200
net add interface swp4-5 mtu 9000
net add interface swp4-5 vlan-protocol 802.1ad
net add interface swp5 alias L2 from Leaf2 to Spine1
net add interface swp6 alias TestVM on VLAN 200
net add interface swp6-7 bridge pvid 200
net add interface swp7 alias JumpHost to Spine Connection
net add loopback lo ip address 10.2.1.3/32
net add vlan 200 alias TEP-Overlay-Network
net add vlan 200 ip address 172.16.200.1/24
net add vlan 200 vlan-id 200
net add vlan 200 vlan-raw-device bridge
net add hostname stevelab-s-spine1
net add dot1x radius accounting-port 1813
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 30
net add dot1x eap-reauth-period 0
net commit

This port connects the spine to the PFSense.

swp1 Link encap:Ethernet HWaddr 00:50:56:96:07:50
inet addr:10.1.1.10 Bcast:0.0.0.0 Mask:255.255.255.248
inet6 addr: fe80::250:56ff:fe96:750/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:142293 errors:0 dropped:0 overruns:0 frame:0
TX packets:1452332 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:16633281 (15.8 MiB) TX bytes:152336988 (145.2 MiB)

It only works when I ping from the spine and specify the port that’s connected to the pfsense.

cumulus@stevelab-s-spine1:~$ ping 8.8.8.8 -I swp1
PING 8.8.8.8 (8.8.8.8) from 10.1.1.10 swp1: 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=50 time=1.55 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=50 time=1.54 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=50 time=1.53 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=50 time=1.77 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 1.539/1.604/1.779/0.108 ms
cumulus@stevelab-s-spine1:~$


[2.4.4-RELEASE][admin@stevelab-pfgateway.stevelab.local]/root: ifconfig
em0: flags=8843 metric 0 mtu 1500
options=9b
ether 00:50:56:96:a9:23
hwaddr 00:50:56:96:a9:23
inet6 fe80::250:56ff:fe96:a923%em0 prefixlen 64 scopeid 0x1
inet 10.29.13.156 netmask 0xffffffe0 broadcast 10.29.13.159
nd6 options=21
media: Ethernet autoselect (1000baseT )
status: active
em1: flags=8843 metric 0 mtu 1500
options=9b
ether 00:50:56:96:c5:cb
hwaddr 00:50:56:96:c5:cb
inet6 fe80::250:56ff:fe96:c5cb%em1 prefixlen 64 scopeid 0x2
inet 10.1.1.9 netmask 0xfffffff8 broadcast 10.1.1.15
nd6 options=21
media: Ethernet autoselect (1000baseT )
status: active

Looking good.. tcpdump on the pf appliance

[2.4.4-RELEASE][admin@stevelab-pfgateway.stevelab.local]/root: tcpdump -i em1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em1, link-type EN10MB (Ethernet), capture size 262144 bytes
16:28:40.969545 IP 10.1.1.10 > 8.8.8.8: ICMP echo request, id 15023, seq 1, length 64
16:28:40.970827 IP 8.8.8.8 > 10.1.1.10: ICMP echo reply, id 15023, seq 1, length 64
16:28:41.971242 IP 10.1.1.10 > 8.8.8.8: ICMP echo request, id 15023, seq 2, length 64
16:28:41.972558 IP 8.8.8.8 > 10.1.1.10: ICMP echo reply, id 15023, seq 2, length 64
16:28:42.973128 IP 10.1.1.10 > 8.8.8.8: ICMP echo request, id 15023, seq 3, length 64
16:28:42.974357 IP 8.8.8.8 > 10.1.1.10: ICMP echo reply, id 15023, seq 3, length 64
16:28:45.514847 IP 10.1.1.10 > 8.8.8.8: ICMP echo request, id 15032, seq 1, length 64

If I ping from the leaf or workloads it fails to get routed…

cumulus@stevelab-s-leaf1:~$ ping 10.1.1.9 -I swp1 ß interface on the pfsense
PING 10.1.1.9 (10.1.1.9) from 10.2.1.1 swp1: 56(84) bytes of data.
^C
--- 10.1.1.9 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2000ms

Fails but I can see the requests on the spine but nothing on the pf

cumulus@stevelab-s-spine1:~$ sudo tcpdump -i swp1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on swp1, link-type EN10MB (Ethernet), capture size 262144 bytes
23:43:05.590996 IP 10.2.1.1 > 10.1.1.9: ICMP echo request, id 30223, seq 1, length 64
23:43:06.591088 IP 10.2.1.1 > 10.1.1.9: ICMP echo request, id 30223, seq 2, length 64
23:43:07.591060 IP 10.2.1.1 > 10.1.1.9: ICMP echo request, id 30223, seq 3, length 64

The .10 is the interface on the spine… works…

cumulus@stevelab-s-leaf1:~$ ping 10.1.1.10 -I swp1
PING 10.1.1.10 (10.1.1.10) from 10.2.1.1 swp1: 56(84) bytes of data.
64 bytes from 10.1.1.10: icmp_seq=1 ttl=64 time=0.295 ms
64 bytes from 10.1.1.10: icmp_seq=2 ttl=64 time=0.283 ms

And to google

cumulus@stevelab-s-leaf1:~$ ping 8.8.8.8 -I swp1
PING 8.8.8.8 (8.8.8.8) from 10.2.1.1 swp1: 56(84) bytes of data.
From 10.2.1.3 icmp_seq=1 Destination Host Unreachable
From 10.2.1.3 icmp_seq=2 Destination Host Unreachable
From 10.2.1.3 icmp_seq=3 Destination Host Unreachable
From 10.2.1.3 icmp_seq=4 Destination Host Unreachable
From 10.2.1.3 icmp_seq=5 Destination Host Unreachable

cumulus@stevelab-s-leaf1:~$ ping 10.1.1.9 -I swp1
PING 10.1.1.9 (10.1.1.9) from 10.2.1.1 swp1: 56(84) bytes of data.

Now I only see the request… no reply.

cumulus@stevelab-s-spine1:~$ sudo tcpdump -i swp1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on swp1, link-type EN10MB (Ethernet), capture size 262144 bytes
23:43:05.590996 IP 10.2.1.1 > 10.1.1.9: ICMP echo request, id 30223, seq 1, length 64
23:43:06.591088 IP 10.2.1.1 > 10.1.1.9: ICMP echo request, id 30223, seq 2, length 64
23:43:07.591060 IP 10.2.1.1 > 10.1.1.9: ICMP echo request, id 30223, seq 3, length 64
23:50:41.754374 IP 10.2.1.1 > 10.1.1.9: ICMP echo request, id 30323, seq 1, length 64

Here’s another test shows spine can get out… default route is set… but still can’t work out why it won’t forward packets from the leafs.

cumulus@stevelab-s-spine1:~$ sudo traceroute -i swp1 8.8.8.8
[sudo] password for cumulus:
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 10.1.1.9 (10.1.1.9) 0.302 ms 0.228 ms 0.222 ms
... removed all the hops...
12 * * *
13 dns.google (8.8.8.8) 1.485 ms 1.519 ms 1.494 ms

cumulus@stevelab-s-spine1:~$ net show route

S>* 0.0.0.0/0 [1/0] via 10.1.1.9, swp1, 00:54:34
O 10.1.1.8/29 [110/100] is directly connected, swp1, 02w2d05h
C>* 10.1.1.8/29 is directly connected, swp1, 02w2d05h
O>* 10.2.1.1/32 [110/100] via 10.2.1.1, swp2 onlink, 02w2d06h
O>* 10.2.1.2/32 [110/100] via 10.2.1.2, swp3 onlink, 02w2d05h
C * 10.2.1.3/32 is directly connected, swp3, 02w2d05h
O 10.2.1.3/32 [110/0] is directly connected, lo, 03w0d05h
C * 10.2.1.3/32 is directly connected, swp2, 03w0d05h
C>* 10.2.1.3/32 is directly connected, lo, 03w0d05h
O 172.16.200.0/24 [110/10] is directly connected, vlan200, 03w0d05h
C>* 172.16.200.0/24 is directly connected, vlan200, 03w0d05h
O>* 172.16.201.0/29 [110/110] via 10.2.1.1, swp2 onlink, 02w2d06h
O>* 172.16.202.0/29 [110/110] via 10.2.1.2, swp3 onlink, 02w2d05h
O>* 172.16.203.0/29 [110/110] via 10.2.1.1, swp2 onlink, 02w2d06h
O>* 172.16.204.0/29 [110/110] via 10.2.1.2, swp3 onlink, 02w2d05h
C>* 192.168.1.0/24 is directly connected, eth0, 02w2d05h


cumulus@stevelab-s-leaf1:~$ sudo traceroute -i swp1 8.8.8.8
[sudo] password for cumulus:
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 10.2.1.3 (10.2.1.3) 1.328 ms 1.263 ms 1.244 ms
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *

cumulus@stevelab-s-leaf1:~$ net show route

O>* 0.0.0.0/0 [110/1] via 10.2.1.3, swp1 onlink, 00:55:00
O>* 10.1.1.8/29 [110/200] via 10.2.1.3, swp1 onlink, 02w2d05h
C * 10.2.1.1/32 is directly connected, swp1, 03w1d06h
O 10.2.1.1/32 [110/0] is directly connected, lo, 03w1d06h
C>* 10.2.1.1/32 is directly connected, lo, 03w1d06h
O>* 10.2.1.2/32 [110/200] via 10.2.1.3, swp1 onlink, 02w2d05h
O>* 10.2.1.3/32 [110/100] via 10.2.1.3, swp1 onlink, 02w2d06h
O>* 172.16.200.0/24 [110/110] via 10.2.1.3, swp1 onlink, 02w2d06h
O 172.16.201.0/29 [110/10] is directly connected, vlan201, 03w1d06h
C>* 172.16.201.0/29 is directly connected, vlan201, 03w1d06h
O>* 172.16.202.0/29 [110/210] via 10.2.1.3, swp1 onlink, 02w2d05h
O 172.16.203.0/29 [110/10] is directly connected, vlan203, 03w1d06h
C>* 172.16.203.0/29 is directly connected, vlan203, 03w1d06h
O>* 172.16.204.0/29 [110/210] via 10.2.1.3, swp1 onlink, 02w2d05h
C>* 192.168.1.0/24 is directly connected, eth0, 03w1d06h
icon

Best answer by Attilla 22 June 2019, 15:38

Hi Steve,

Seems you are not using the mgmt vrf and you are using OSPF unnumbered. I don't think that pinging with swp1 as src (with the /32) will work.

Since you have a default route on the mgmt interface, try to use the mgmt vrf (net add vrf mgmt) and then the ping again.

To get the file from a CL box, I would suggest scp. This come by default on linux/Mac hosts. It has been a long time since I worked with windows, but does "winscp" still exist?
View original

18 replies

Can you post the routing table of the pfsense box?
Userlevel 2
Hi Steve, can you give us the routing table of the PF sense box as well? I'd like to see if it has route back to the network.
Here's the original output.

netstat -r
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 10.29.13.129 UGS em0
10.1.1.8/29 link#2 U em1
stevelab-pfgateway link#2 UHS lo0
10.29.13.128/27 link#1 U em0
10.29.13.156 link#1 UHS lo0
localhost link#4 UH lo0

I've just added the 172.16.0.0/16 (test networks I use) and 10.2.1.2/32 10.2.1.1/32 that I use for leaf1 and 2... still won't ping from the leaf.

route add -net -net 10.2.1.1/32 10.1.1.10
route add -net -net 10.2.1.2/32 10.1.1.10
route add -net -net 172.16.0.0/16 10.1.1.10

netstat -r
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 10.29.13.129 UGS em0
10.1.1.8/29 link#2 U em1
stevelab-pfgateway link#2 UHS lo0
10.2.1.1/32 10.1.1.10 UGS em1
10.2.1.2/32 10.1.1.10 UGS em1
10.29.13.128/27 link#1 U em0
10.29.13.156 link#1 UHS lo0
localhost link#4 UH lo0
172.16.0.0/16 10.1.1.10 UGS em1

Recall earlier I said that If did a capture on the PFSense when pinging from the leafs, I don't see the icmp requests in the capture.
I appreciate the replies. I'm only a few weeks into using the cumulus switches and this is making me pull my hair out... I'm willing to try just about anything. I've even considered using BGP only but part of me wants to get to the bottom of this...
Hey Steve. Thanks for pointing that out. I think I read the initial post a little too quickly.

Looking at the routing table for the spine, it shows the OSPF route for 10.2.1.1/32 points out swp2. You mentioned that swp1 is the iface that connects the leaf and the spine so I suspect something there might be awry.

You can run the tcpdump on swp2 on the spine to validate that the echo replies are being sent, but I would investigate why the route is being learned out a different interface than expected.
Userlevel 2
Hi Steve,

Trey and I actually posted the same thing separately from each other, because we had the same thought. I do see the tcpdumps, but not on the pfsense side that doesn't show the echo requests or am I mis reading it?

Either way, since you didn't have the routes back, you wouldn't see replies. Could you do a tcpdump on the em1 interface now with the new routes configured while pinging from the leafs?

My guess at this point would be that traffic is being dropped by a default firewall rule. If the tcpdump shows the echo requests coming in, we would know that routing is working as it should be.
You might also try running the tcpdump with the flag '-i any', as that will validate whether you see the request forwarded to begin with. It should appear twice per seq number, once on ingress and once on egress.
To clarify:
on Spine1 the interfaces are:
net add interface swp1 alias Internet Uplink PF Sense
net add interface swp2 alias L3 from Leaf1 to Spine1
net add interface swp3 alias L3 from Leaf2 to Spine1
net add interface swp4 alias L2 from Leaf1 to Spine1
net add interface swp5 alias L2 from Leaf2 to Spine1

in Leaf1 (same on Leaf2) the interfaces are:
(there are more but they go to the servers)
net add interface swp1 alias L3 from Leaf2 to Spine1
net add interface swp2 alias L2 from Leaf2 to Spine1

cumulus@stevelab-s-leaf1:~$ ping 8.8.8.8 -I swp1
PING 8.8.8.8 (8.8.8.8) from 10.2.1.1 swp1: 56(84) bytes of data.
From 10.2.1.3 icmp_seq=1 Destination Host Unreachable
From 10.2.1.3 icmp_seq=2 Destination Host Unreachable
From 10.2.1.3 icmp_seq=3 Destination Host Unreachable

at the same time on the pfsense.

[2.4.4-RELEASE][admin@stevelab-pfgateway.stevelab.local]/root: tcpdump -i em1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em1, link-type EN10MB (Ethernet), capture size 262144 bytes

Nothing is being captured...
While that same ping on Leaf1 is happening...

cumulus@stevelab-s-spine1:~$ sudo tcpdump -i swp2 icmp
[sudo] password for cumulus:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on swp2, link-type EN10MB (Ethernet), capture size 262144 bytes
02:26:18.350076 IP 10.2.1.1 > dns.google: ICMP echo request, id 4531, seq 158, length 64
02:26:19.349802 IP 10.2.1.1 > dns.google: ICMP echo request, id 4531, seq 159, length 64
02:26:20.349824 IP 10.2.1.1 > dns.google: ICMP echo request, id 4531, seq 160, length 64
c02:26:21.349858 IP 10.2.1.1 > dns.google: ICMP echo request, id 4531, seq 161, length 64
02:26:22.349844 IP 10.2.1.1 > dns.google: ICMP echo request, id 4531, seq 162, length 64
02:26:23.349850 IP 10.2.1.1 > dns.google: ICMP echo request, id 4531,

and still at the same time...

cumulus@stevelab-s-spine1:~$ sudo tcpdump -i swp1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on swp1, link-type EN10MB (Ethernet), capture size 262144 bytes

Nothing...

It seems to me the spine just gives up and doesn't forward the packet to the default route...
and by all means, if you think the way I'm doing things is wonky feel free to tell me.

All suggestions welcome. just trying to learn the right way of doing all of this.
Userlevel 2
Hi Steve,

You're not doing anything wonky. The fact that you see the echo requests on spine though is. While you can do a tcpdump, this traffic should be forwarded in hardware to the pfsense and not even visible when doing the tcpdump. For some reason it is sent to the cpu which shouldn't. So you shouldn't see anything when tcpdumping on either interfaces on the spine.

While we're trying to help you, please remember that you're always free to open a support case.

There are a few things i'd like to see:

  • The full configs ("net show configuration") of both the leaf and spine? (without the "commands" it makes it easier to read for me)
  • Could you write the tcpdump of the spine to a pcap file? (-w ) and share this?

I first thought that there is something strange with the /32 being on the swp interface of the leaf:

C * 10.2.1.1/32 is directly connected, swp1, 03w1d06h"

But that is because you are using OSPF unnumbered.
Here's the Leaf1 Config:

cumulus@stevelab-s-leaf1:~$ net show con com
net del all
net add time zone Etc/UTC
net add time ntp server 0.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 1.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 2.cumulusnetworks.pool.ntp.org iburst
net add time ntp server 3.cumulusnetworks.pool.ntp.org iburst
net add time ntp source eth0
net add snmp-server listening-address localhost
net add bgp autonomous-system 65000
net add interface swp1 ospf network point-to-point
net add routing defaults datacenter
net add routing service integrated-vtysh-config
net add routing log syslog informational
net add routing line vty exec-timeout 0 0
net add bgp router-id 10.2.1.1
net add bgp bestpath as-path multipath-relax
net add bgp neighbor 172.16.201.1 remote-as external
net add bgp neighbor 172.16.201.1 bfd 3 1000 1000
net add bgp neighbor 172.16.201.3 remote-as external
net add bgp neighbor 172.16.201.3 bfd 3 1000 1000
net add bgp neighbor 172.16.203.1 remote-as external
net add bgp neighbor 172.16.203.1 bfd 3 1000 1000
net add bgp neighbor 172.16.203.3 remote-as external
net add bgp neighbor 172.16.203.3 bfd 3 1000 1000
net add bgp ipv4 unicast redistribute ospf
net add ospf router-id 10.2.1.1
net add ospf redistribute bgp
net add ospf network 10.2.1.1/32 area 0.0.0.0
net add ospf network 172.16.201.0/29 area 0.0.0.0
net add ospf network 172.16.203.0/29 area 0.0.0.0
net add dns nameserver ipv4 192.168.1.133
net add ptp global slave-only no
net add ptp global priority1 255
net add ptp global priority2 255
net add ptp global domain-number 0
net add ptp global logging-level 5
net add ptp global path-trace-enabled no
net add ptp global use-syslog yes
net add ptp global verbose no
net add ptp global summary-interval 0
net add ptp global time-stamping
net add bridge bridge ports swp2,swp3,swp4,swp5,swp6,swp7,swp8
net add bridge bridge vids 100,200-204
net add bridge bridge vlan-aware
net add interface eth0 ip address 192.168.1.101/24
net add interface eth0 ip gateway 192.168.1.1
net add interface swp1 alias L3 from Leaf2 to Spine1
net add interface swp1 ip address 10.2.1.1/32
net add interface swp2 alias L2 from Leaf2 to Spine1
net add interface swp2 bridge vids 100,200
net add interface swp3 alias Leaf1 to ESXi1
net add interface swp3-8 bridge vids 100,200-204
net add interface swp3-8 mtu 9000
net add interface swp3-8 vlan-protocol 802.1ad
net add interface swp4 alias Leaf1 to ESXi2
net add interface swp5,7 alias Leaf1 to Edge1
net add interface swp6,8 alias Leaf1 to Edge2
net add loopback lo ip address 10.2.1.1/32
net add vlan 201 alias T0 Uplink-1 Network
net add vlan 201 ip address 172.16.201.2/29
net add vlan 201 vlan-id 201
net add vlan 201 vlan-raw-device bridge
net add vlan 203 alias To Uplink-3 Network
net add vlan 203 ip address 172.16.203.2/29
net add vlan 203 vlan-id 203
net add vlan 203 vlan-raw-device bridge
net add hostname stevelab-s-leaf1
net add dot1x radius accounting-port 1813
net add dot1x radius authentication-port 1812
net add dot1x mab-activation-delay 30
net add dot1x eap-reauth-period 0
net commit
How do I get the pcap file off of the cumulus appliance? No SCP installed.
Userlevel 2
Hi Steve,

Seems you are not using the mgmt vrf and you are using OSPF unnumbered. I don't think that pinging with swp1 as src (with the /32) will work.

Since you have a default route on the mgmt interface, try to use the mgmt vrf (net add vrf mgmt) and then the ping again.

To get the file from a CL box, I would suggest scp. This come by default on linux/Mac hosts. It has been a long time since I worked with windows, but does "winscp" still exist?
net add interface eth0 ip address 192.168.1.101/24
net add interface eth0 ip gateway 192.168.1.1
net add interface eth0 vrf mgmt

added the vrf mgmt. ping still fails however now I don't have to specify the interface to source.

cumulus@stevelab-s-spine1:~$ sudo tcpdump -i any icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
14:22:26.935529 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 1, length 64
14:22:27.935054 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 2, length 64
14:22:28.935049 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 3, length 64
14:22:29.935068 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 4, length 64
14:22:30.935083 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 5, length 64
14:22:31.935127 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 6, length 64
14:22:32.935080 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 7, length 64
14:22:33.935091 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 8, length 64
14:22:34.935115 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 9, length 64
14:22:35.935133 IP 10.2.1.1 > dns.google: ICMP echo request, id 11705, seq 10, length 64
14:22:36.955316 IP 10.2.1.3 > 10.2.1.1: ICMP host dns.google unreachable, length 92
14:22:36.955350 IP 10.2.1.3 > 10.2.1.1: ICMP host dns.google unreachable, length 92
14:22:36.955368 IP 10.2.1.3 > 10.2.1.1: ICMP host dns.google unreachable, length 92
14:22:36.955374 IP 10.2.1.3 > 10.2.1.1: ICMP host dns.google unreachable, length 92
14:22:36.955382 IP 10.2.1.3 > 10.2.1.1: ICMP host dns.google unreachable, length 92
14:22:36.955389 IP 10.2.1.3 > 10.2.1.1: ICMP host dns.google unreachable, length 92
14:22:53.949301 IP 10.2.1.1 > dns.google: ICMP echo request, id 11726, seq 1, length 64
14:22:54.949341 IP 10.2.1.1 > dns.google: ICMP echo request, id 11726, seq 2, length 64
14:23:03.969336 IP 10.2.1.3 > 10.2.1.1: ICMP host dns.google unreachable, length 92

Working on getting the pcap.
Userlevel 2
Use one of the SVIs as src interface: ping 8.8.8.8 -I vlan201 if you want to specify one. Unless specified, an interface is in the default VRF.
I added (net add interface eth0 vrf mgmt) to the spine and it started working!


Attilla, Thank You!
Userlevel 2
Great to hear Steve. Everything clear why this didn't work in the first place?

Reply