Routing on the Host user guide


Userlevel 5
Here is the Routing on the Host user guide.

https://docs.cumulusnetworks.com/display/ROH/Routing+on+the+Host

In a typical data center, connections between servers and the leaf or top of rack switches are often done at layer 2. In order to build more resilient data centers, many Cumulus Networks customers are leveraging the Linux ecosystem to run routing protocols directly to their servers, running layer 3 protocols like OSPF (Open Shortest Path First) or BGP (Border Gateway Protocol) directly on the hosts. This is often referred to as Routing on the Host. In Cumulus Linux 3.0, Routing on the Host works on server hosts in a number of different environments:
  • Ubuntu 12.04, 14.04 and 16.04
  • Red Hat Enterprise Linux 7
  • Docker containers
Routing on the Host provides you with:
  • Simplified, modern data center design
  • Subnet freedom and mobility
  • Enhanced redundancy
  • Stateless services with Anycast

33 replies

Userlevel 3
We should not link to this article any more, as it's been replaced by the user guide, which is the canonical source moving forward: https://docs.cumulusnetworks.com/display/ROH/
Userlevel 3
And when you're ready to get started, download Cumulus Quagga.
Userlevel 1
Routing at the host level has always been a question of mine and my research a year or two ago didn't get me too far. This is a great option to see now! In some situations companies are using MLAG and FHR since they want fault tolerance and VLAN Gateways at this tier. With this we can start to push all gateways down to the host level which is ideal. Couple questions though... what about VLAN's? Do we still build our VM Environment off of VLAN's (portgroups) and use an overlay network to connect different PODs? Or do we start keeping networks local to a POD and start using anycast technology?

I and so many others are use to the L2 all the way down or more recently L3 down and then L2 to the hosts (POD). Maybe you can offer a couple differnet designs here. Typically a company will be using ESXi and you would trunk down to it. In this case we are doing a point to point L3 connection to the host. What would be your suggestion? I am going to lab this out soon with VX.

Thanks,
D
Userlevel 1
This is something I have been looking forward to. Should we expect overlay networks still for shared VLANs or is this an attempt to move away from VLANs entirely?
Userlevel 2
dlambert wrote:

This is something I have been looking forward to. Should we expect overlay networks still for sha...

A move away from VLANs entirely would certainly be nice, not sure that everyone is ready to go there but ROH opens that door. Combined with overlay networks, this should provide a path to new and interesting network designs.
Userlevel 1
In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Could you, please, clarify - what's the mechanism of putting these addresses into routing table?
Sergei Hanus wrote:

In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Coul...

As part of creating VM's you create network interfaces that allow the VM to communicate to the outside world. Quagga when run sees the network interfaces and the routing protocols can be configured to automatically use those interfaces. This is normal behavior on a switch to.
Userlevel 1
Sergei Hanus wrote:

In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Coul...

But this is much different from redistributing VM IP address - you propose to redistribute whole subnet on interface, as I understand?
I was thinking about scenario, when there's same IP subnet for VMs on two sites, but L3 between these two sites. In this case your offer is not going to work.
But, if routing on host is somehow equivalent to "redistribute neighbor" feature on cumulus switches - then yes, this would be perfect match for what I want to achive.
Userlevel 5
Sergei Hanus wrote:

In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Coul...

There is a lot of flexibility in what can be advertised, and how. In the case of Docker you may want to redistribute entire bridge subnets while disabling the NAT component; or you may want to add the newly created /32 Docker IPs to a loopback and simply redistribute the loopback in so as new docker containers are provisioned they have an additional IP added to the loopback. Or in the case of traditional VMs you may want to build a script to inspect the arp entries on that bridge using the "arp -n" command and advertise those into BGP /w a /32 network statement or into a separate kernel routing table that can be redistributed into Quagga. There are truly a ton of different options -- I think our goal is to enable people to explore new ways of deploying applications with routing that can be tailored to your precise needs.
Userlevel 1
Sergei Hanus wrote:

In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Coul...

Eric, I totally agree with flexibility point.
As you said, for Docker there are options out-of-the-box (like adding /32 to loopback).
As for traditional vms - there's working solution from Cumulus (redistribute neighbor), which just needs to be "blessed", like you did for Quagga, in order to be ported to hypervisor - and we get out-of-the-box solution for traditional vms as well. That what I meant to point in my post.

Sergei.
Looking at the user guide, it appears RoH within a hypervisor (VMware, for example) is done via a router within the hypervisor, and not Quagga @ guest level. Am I correctly interpreting that? See: Subnet Freedom and Mobility diagram.
Userlevel 3
Ryan wrote:

Looking at the user guide, it appears RoH within a hypervisor (VMware, for example) is done via a...

Hi Ryan, that image is wrong, sorry about the confusion. RoH is not within a hypervisor; Quagga is indeed installed and run on the servers. I'll update that image now.
Userlevel 1
Ryan wrote:

Looking at the user guide, it appears RoH within a hypervisor (VMware, for example) is done via a...

Is Cumulus Quagga supported on an ESXi Server? I understand that it is not ran within the hypervisor.
Userlevel 3
Ryan wrote:

Looking at the user guide, it appears RoH within a hypervisor (VMware, for example) is done via a...

Not at the moment, no. Have you looked into redistribute neighbor?

https://docs.cumulusnetworks.com/display/DOCS/Redistribute+Neighbor
Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a host seems a nice solution for me. Unfortunately I'm bound to use Centos 6.x on my servers in the data centre for another year at least. And from what I've seen docker is available only for RHEL//Centos 7.
Thank you in advance.

Cheers,
Gabriel
Userlevel 5
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

You don't need to deploy "Routing on the Host" inside of a container; you can deploy it directly on the bare-metal host via an RPM install. That RPM is built for Centos7 but you might want to try it on 6.8. The RPM is on this page --> https://cumulusnetworks.com/routing-on-the-host/ under the "Download" section.
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

OK. I'll try to install the Centos7 RPM on 6.8 and see if it works.
Userlevel 5
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Gabriel,

Here is the source for it as well.

https://github.com/CumulusNetworks/quagga

If the CentOS 7 packages do not work.
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

I tried the Centos7 RPM on 6.8, but installation is not working because of glibc 2.14 dependency and most important because centos7 is using systemd and not initd like centos 6.8.
Userlevel 5
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

You will need to install from source. The link I posted above should help you get it setup.
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Thank you for help. I'll keep you posted about that.
Userlevel 5
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Gabriel,

I am reaching out to the team to have someone reach-out to give you some assistance.

Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

OK. Thanks a lot.
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Gabriel - Your compile errors are centered around the HAVE_POLL define which don't show up in your configure line as cut-n-pasted above. Additionally our quagga.spec that we provide with the source does not have snmp enabled. This sure looks like a comple that got broken and started over without cleaning up properly. I would guess that we need to completely clean out the build system and start over. Additionally the problem with the Quagga.spec file that we have provided though is that it assumes systemd. Let me build up a centos 6.8 and see what I can get for you
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Hi Donald,

Thanks a lot for your help.
First I didn't cut-n-pasted everything because it was to much, but I can provide you the entire output if necessary.
Trying to debug my rpm building problem I observed myself that the spec file and the package is for systemd (centos/rhel 7). And it was the next question...
Then I'll wait for a 6.8 version.
Thanks again.

Best regards,
Gabriel

Reply