Routing on the Host user guide


Userlevel 5
Here is the Routing on the Host user guide.

https://docs.cumulusnetworks.com/display/ROH/Routing+on+the+Host

In a typical data center, connections between servers and the leaf or top of rack switches are often done at layer 2. In order to build more resilient data centers, many Cumulus Networks customers are leveraging the Linux ecosystem to run routing protocols directly to their servers, running layer 3 protocols like OSPF (Open Shortest Path First) or BGP (Border Gateway Protocol) directly on the hosts. This is often referred to as Routing on the Host. In Cumulus Linux 3.0, Routing on the Host works on server hosts in a number of different environments:
  • Ubuntu 12.04, 14.04 and 16.04
  • Red Hat Enterprise Linux 7
  • Docker containers
Routing on the Host provides you with:
  • Simplified, modern data center design
  • Subnet freedom and mobility
  • Enhanced redundancy
  • Stateless services with Anycast

31 replies

Userlevel 3
And when you're ready to get started, download Cumulus Quagga.
Userlevel 1
This is something I have been looking forward to. Should we expect overlay networks still for shared VLANs or is this an attempt to move away from VLANs entirely?
Userlevel 2
dlambert wrote:

This is something I have been looking forward to. Should we expect overlay networks still for sha...

A move away from VLANs entirely would certainly be nice, not sure that everyone is ready to go there but ROH opens that door. Combined with overlay networks, this should provide a path to new and interesting network designs.
Userlevel 1
In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Could you, please, clarify - what's the mechanism of putting these addresses into routing table?
Sergei Hanus wrote:

In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Coul...

As part of creating VM's you create network interfaces that allow the VM to communicate to the outside world. Quagga when run sees the network interfaces and the routing protocols can be configured to automatically use those interfaces. This is normal behavior on a switch to.
Userlevel 1
Sergei Hanus wrote:

In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Coul...

But this is much different from redistributing VM IP address - you propose to redistribute whole subnet on interface, as I understand?
I was thinking about scenario, when there's same IP subnet for VMs on two sites, but L3 between these two sites. In this case your offer is not going to work.
But, if routing on host is somehow equivalent to "redistribute neighbor" feature on cumulus switches - then yes, this would be perfect match for what I want to achive.
Userlevel 5
Sergei Hanus wrote:

In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Coul...

There is a lot of flexibility in what can be advertised, and how. In the case of Docker you may want to redistribute entire bridge subnets while disabling the NAT component; or you may want to add the newly created /32 Docker IPs to a loopback and simply redistribute the loopback in so as new docker containers are provisioned they have an additional IP added to the loopback. Or in the case of traditional VMs you may want to build a script to inspect the arp entries on that bridge using the "arp -n" command and advertise those into BGP /w a /32 network statement or into a separate kernel routing table that can be redistributed into Quagga. There are truly a ton of different options -- I think our goal is to enable people to explore new ways of deploying applications with routing that can be tailored to your precise needs.
Userlevel 1
Sergei Hanus wrote:

In the user guide I see, that VM IP addresses are being redistributed into routing protocol.
Coul...

Eric, I totally agree with flexibility point.
As you said, for Docker there are options out-of-the-box (like adding /32 to loopback).
As for traditional vms - there's working solution from Cumulus (redistribute neighbor), which just needs to be "blessed", like you did for Quagga, in order to be ported to hypervisor - and we get out-of-the-box solution for traditional vms as well. That what I meant to point in my post.

Sergei.
Looking at the user guide, it appears RoH within a hypervisor (VMware, for example) is done via a router within the hypervisor, and not Quagga @ guest level. Am I correctly interpreting that? See: Subnet Freedom and Mobility diagram.
Userlevel 3
Ryan wrote:

Looking at the user guide, it appears RoH within a hypervisor (VMware, for example) is done via a...

Hi Ryan, that image is wrong, sorry about the confusion. RoH is not within a hypervisor; Quagga is indeed installed and run on the servers. I'll update that image now.
Userlevel 1
Ryan wrote:

Looking at the user guide, it appears RoH within a hypervisor (VMware, for example) is done via a...

Is Cumulus Quagga supported on an ESXi Server? I understand that it is not ran within the hypervisor.

Userlevel 3
Ryan wrote:

Looking at the user guide, it appears RoH within a hypervisor (VMware, for example) is done via a...

Not at the moment, no. Have you looked into redistribute neighbor?

https://docs.cumulusnetworks.com/display/DOCS/Redistribute+Neighbor
Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a host seems a nice solution for me. Unfortunately I'm bound to use Centos 6.x on my servers in the data centre for another year at least. And from what I've seen docker is available only for RHEL//Centos 7.
Thank you in advance.

Cheers,
Gabriel
Userlevel 5
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

You don't need to deploy "Routing on the Host" inside of a container; you can deploy it directly on the bare-metal host via an RPM install. That RPM is built for Centos7 but you might want to try it on 6.8. The RPM is on this page --> https://cumulusnetworks.com/routing-on-the-host/ under the "Download" section.
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

OK. I'll try to install the Centos7 RPM on 6.8 and see if it works.

Userlevel 5
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Gabriel,

Here is the source for it as well.

https://github.com/CumulusNetworks/quagga

If the CentOS 7 packages do not work.

Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

I tried the Centos7 RPM on 6.8, but installation is not working because of glibc 2.14 dependency and most important because centos7 is using systemd and not initd like centos 6.8.
Userlevel 5
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

You will need to install from source. The link I posted above should help you get it setup.
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Thank you for help. I'll keep you posted about that.
Userlevel 5
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Gabriel,

I am reaching out to the team to have someone reach-out to give you some assistance.

Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

OK. Thanks a lot.
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Gabriel - Your compile errors are centered around the HAVE_POLL define which don't show up in your configure line as cut-n-pasted above. Additionally our quagga.spec that we provide with the source does not have snmp enabled. This sure looks like a comple that got broken and started over without cleaning up properly. I would guess that we need to completely clean out the build system and start over. Additionally the problem with the Quagga.spec file that we have provided though is that it assumes systemd. Let me build up a centos 6.8 and see what I can get for you
Gabriel Stoicea wrote:

Hi,

Is it any chance to use "routing on a host"/Cumulus Quagga on RHEL/Centos 6.8?
Routing on a...

Hi Donald,

Thanks a lot for your help.
First I didn't cut-n-pasted everything because it was to much, but I can provide you the entire output if necessary.
Trying to debug my rpm building problem I observed myself that the spec file and the package is for systemd (centos/rhel 7). And it was the next question...
Then I'll wait for a 6.8 version.
Thanks again.

Best regards,
Gabriel
how does this integrate with KVM using openvswitch, would like to find out how to inject the routes when using bridges on vlans.
Userlevel 5
Mr Roger wrote:

how does this integrate with KVM using openvswitch, would like to find out how to inject the rout...

I have not looked at that particular integration however it should be possible. If you can add a hook to openvswitch to add the route via a CLI command that would work; the other option that comes to mind is to write a little daemon that looks at configured bridges for some predefined interval and adds/removes them from the routing fabric.

Reply