BGP peer reset with CumulusVX but not with Quagga on Ubuntu


I have a working Quagga config on an Ubuntu machine but when I copy the exact same config to a CumulusVX VM one of my peers is getting resets and another one doestn come up at all.
IP's are the same.
The peers that have issues are Cisco cloud services router vm's.
Exabgp peers have no problems.

The bgpd version is 0.99.22.4.
CumuluxVX bgpd version is 0.99.23.1

This is my config:

!! Zebra configuration saved from vty
! 2015/09/29 17:01:54
!
password wwww
enable password xxxx
!
router bgp 7000
bgp router-id 5.5.5.1
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 1.1.1.1 remote-as 7000
neighbor 1.1.1.1 timers 2 6
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 route-server-client
neighbor 1.1.1.1 route-map RSCLIENT-IMPORT import
neighbor 1.1.1.1 route-map RSCLIENT-EXPORT export
neighbor 1.1.1.2 remote-as 7000
neighbor 1.1.1.2 timers 2 6
neighbor 1.1.1.2 activate
neighbor 1.1.1.2 route-server-client
neighbor 1.1.1.3 remote-as 7000
neighbor 1.1.1.3 timers 2 6
neighbor 1.1.1.3 activate
neighbor 1.1.1.3 route-server-client
neighbor 1.1.1.3 route-map RSCLIENT-IMPORT import
neighbor 1.1.1.3 route-map RSCLIENT-EXPORT export
neighbor 4.4.4.253 remote-as 7000
neighbor 4.4.4.253 timers 2 6
neighbor 4.4.4.253 activate
neighbor 4.4.4.253 route-server-client
!
ip prefix-list VIPS seq 5 permit 8.8.8.0/24 le 32
ip prefix-list VIPS seq 10 deny any
!
route-map RSCLIENT-EXPORT permit 10
match ip address prefix-list VIPS
!
route-map RSCLIENT-IMPORT deny 10
!
route-map R-IMPORT permit 10
!
route-map R-EXPORT deny 10
!
line vty
!

The peers having problems are 1.1.1.2 and 4.4.4.253.
1.1.1.2 is not contacted and 4.4.4.253 gets getting resets

Any ideas?

I thought about evaluating Cumulus for the job as BGP route reflector but at the moment I am not convinced...

12 replies

Userlevel 2
Hey Thomas.

Are all the peers directly connected?

What do you see in /var/log/quagga/quagga.log?

Can you enable "debug bgp updates" on both the route server and a failing peer. This should write the debug to /var/log/quagga/quagga.log

-Pete
Hi, the Cisco routers are not directly attached. There is at least one hop in between. I have also corrected my config -> route reflector client in stead of route server client (since I use iBGP). It seems like for iBGP this version of quagga does not do multihop. The logging does not reveal anything. The peer connections are never attempted:

this forum is very therapeutic, the peer connections are alive, now see why there is no route propagation 🙂
Userlevel 2
Hey Thomas, glad things are at least moving in the right direction.

Multihop iBGP is supported, but I'd wonder do we have routes to the peer IP?

Can you share the BGP config and "show ip route" (from Quagga) output from both the Cisco and Cumulus sides?
Hi, the peering is ok! But for some reason the Cumulus route reflector wont announce routes to the Cisco's. This is my current config: router bgp 7000 bgp router-id 5.5.5.2 bgp log-neighbor-changes no bgp default ipv4-unicast neighbor 1.1.1.1 remote-as 7000 neighbor 1.1.1.1 activate neighbor 1.1.1.1 route-reflector-client neighbor 1.1.1.3 remote-as 7000 neighbor 1.1.1.3 activate neighbor 1.1.1.3 route-reflector-client neighbor 11.11.11.2 remote-as 7000 neighbor 11.11.11.2 timers 2 6 neighbor 11.11.11.2 activate neighbor 11.11.11.2 route-reflector-client neighbor 11.11.11.3 remote-as 7000 neighbor 11.11.11.3 timers 2 6 neighbor 11.11.11.3 activate neighbor 11.11.11.3 route-reflector-client The exabgp routes are announced to the Cumulus:l bgpd# show ip bgp BGP table version is 0, local router ID is 5.5.5.2 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path i8.8.8.1/32 1.1.1.1 150 0 i i 1.1.1.3 25 0 i i8.8.8.2/32 1.1.1.3 200 0 i i8.8.8.3/32 1.1.1.1 14 0 i i8.8.8.4/32 1.1.1.3 120 0 i But they are not announced to the Cisco's (11.11.11.2 and 3) while the peerings are ok: bgpd# show ip bgp summary BGP router identifier 5.5.5.2, local AS number 7000 BGP table version 0 RIB entries 7, using 840 bytes of memory Peers 4, using 67 KiB of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 1.1.1.1 4 7000 165 161 0 0 0 02:39:25 2 1.1.1.3 4 7000 166 161 0 0 0 02:39:25 3 11.11.11.2 4 7000 4203 4306 0 0 0 00:14:16 0 11.11.11.3 4 7000 177 161 0 0 0 02:39:22 0 bgpd# show ip bgp neighbors 11.11.11.2 advertised-routes bgpd# show ip bgp neighbors 11.11.11.3 advertised-routes I do notice a difference between Quagga on Ubuntu and Cumulus: what are update-groups, can they have something to do with it?
Userlevel 2
The update-groups would have to do with outbound BGP policies like route filters. It's a way BGP will make the building of route advertisements more efficient, but I don't think that's the problem in this case.

If the host bgpd is your cumulus node, it looks like we don't consider any of the routes "best" which means we won't advertise them on. What do you see in "show ip bgp 8.8.8.2"?
Interesting: there is indeed no best path Cumulus output: bgpd# show ip bgp 8.8.8.2 BGP routing table entry for 8.8.8.2/32 Paths: (1 available, no best path) Not advertised to any peer Local, (Received from a RR-client) 1.1.1.3 (inaccessible) from 1.1.1.3 (1.1.1.3) Origin IGP, localpref 200, invalid, internal Last update: Wed Sep 30 16:27:51 2015 Quagga output: RR# show ip bgp 8.8.8.2 BGP routing table entry for 8.8.8.2/32 Paths: (1 available, best #1, table Default-IP-Routing-Table) Advertised to non peer-group peers: 1.1.1.1 11.11.11.2 11.11.11.3 Local, (Received from a RR-client) 1.1.1.3 from 1.1.1.3 (1.1.1.3) Origin IGP, localpref 200, valid, internal, best Last update: Wed Sep 30 17:33:37 2015 Exabgp announces the routes with local-preference values.
I found the problem: the cumulus VM did not have an explicit route to 1.1.1.0/24, it only had a default route (so the ip's were reachable), the quagga had the specific route. Is this a requirement for BGP?
Userlevel 2
Great catch Thomas. I would expect following a default should be valid. Let me ask around internally and let you know.
Userlevel 2
Thomas today I learned about "ip nht resolve-via default". You configure this in Quagga global config mode. This will allow a route to be considered valid if the next hop is via a default.

# conf t
(config)# ip nht resolve-via-default
aha perfect thx!

Reply