Campus Switch Stacking

  • 19 September 2016
  • 9 replies

Userlevel 1
We're looking into using Cumulus Linux in our campus environment, and I was wondering what would be the best configuration solution for multiple switches daisy-chained to each other?

9 replies

Here are some solutions for that:

If it's layer 2 then you can use these:

Layer 3 information:

Userlevel 2
"B", no stacking as you'd know it with the 3850. Typically interconnect boxes with trunk ports and link aggregation.

If you're end goal is to simply create a single configuration point, centralized automation would give you a similar abstract for controlling large numbers of ports and devices without having to be concerned about stacking devices.
Speaking from a pure, traditional, physical topology point of view I would think the ideal topology here would be to implement a tree instead of a chain in order to minimize the dependencies that will be affected by a random member dropping off (failing or rebooting.)

Now if you want to do something COOL though, consider an entire layer 3 to the edge closet, possibly using Cumulus' BGP/OSPF unnumbered, then a chain, or even a fully connect graph, makes more sense. Why? Well at layer 2 the loop mitigation protocol is spanning tree which simply disables/blocks all but a single path from each point of the network back to the defined network root (the switch in the network that wins the election and is likely at or near your core.) But at layer 3, routing protocols would mitigate any loops or redundant paths and routing protocols can utilize those extra paths using ECMP (equal cost multi-path routing.) This means that a routing protocol won't block or disable those duplicate paths but could potential utilize them for passing more traffic.


Core <-> A - B - C - D - E


Core <-> A
/ \
/ \

Fully Connected Graph:

Core <-> A - B
/ \ \ / /
C - D -E

I hope this helps!

Jeff H
SE, southeast US
Thanks for replying B! I see now that you are specifically asking about stacking more so than chaining? Well the short answer to your question is no, Cumulus does not support stacking.

The longer answer is to look at why and how Cumulus provides a better solution to the problems that the inventors were trying to solve with the release of stacking. (Warning! History and sales content to follow) Stacking solves 3 distinct problems: 1) ease of management through the use of a single IP address or management plane, 2) sharing of the control plane to allow a closet to survive a switch failure and 3) multi-chassis uplink redundancy.

Let's tackle these 3 problems 1 by 1:

1) ease of management: There is an undoubtably elegant beauty to interfacing with a single IP address to manage an entire edge closet's worth of switches. But wouldn't it be even cooler to manage an entire closet, floor, building, or campus in the same concept. Stacking keeps this concept of single mgmt IP to a physical location, but Cumulus, through the use of automation and orchestration allows for unlimited "stacking" by closets, floors, buildings, or even use cases like wireless switches, VOIP switches, or Dell switches. Automation and orchestration take the concept of stacking and moves it into the 21st century by allowing for dynamic stacking at the admin's discretion. I see stacking here like the janitor's large key ring hanging on the hip. With the inventions from Nest, Kevo, and the like these key rings are a dying breed as we just keep our "keys" on our phone!

2) Switch failure survival of the control plane: This is another solution for a problem of a by gone era. "Back in the day" when switches were the latest and greatest, the CPU costs for a switch were expensive. And so companies looking to save costs would buy one or two expensive "stack switch controllers" and then multiple less expensive "stack member switches." Those member switches wouldn't have the beef to handle entire ARP or CAM tables but they could pass traffic to and from the switch controllers that could. Nowadays the x86 and ARM CPU's are less expensive so this is no longer a concern.

3) Uplink redundancy: This problem is a direct result of that ancient technology spanning tree. Spanning tree, as I referenced above, disables all paths except the single path that it computes is best for a path. This means that two paths, redundant paths, from the same stack or chain of switches guarantees that there will be at least one link blocked down. This isn't exactly a problem but it isn't efficient. What is a problem in this scenario is the time it takes that blocked link to come up after failure, and even worse, the time it takes the original link to go back down when the original link is operational once more. This solution also completely invalidates the concept of aggregating those 2 links together to achieve greater uplink output. Cumulus' solution here is utilizing layer 3 to the edge and then achieving ECMP as discussed above.


So I hope you agree from the above solution that while Cumulus doesn't support stacking it supports a series of superior solutions to the same problems that stacking tried to solve way back when.


Jeff H
Userlevel 1
Hey Jeff, thank for the reply! I will keep that in mind, but maybe I should have phrased it a different way. What about stacking the switches to look like one switch? Right now we have a couple of Cisco 3850 that we stack to seem as one. If we were to replace those switches, could we configure Cumulus Linux to give us the same result using white boxes?
Userlevel 1
Thanks Kevin!
Thanks for replying B! I see now that you are specifically asking about stacking more so than ch...Oops. While I was writing my lengthy response it looks like Kevin and Srivats got something out faster! Well better multiple correct answers than no answers at all!


Jeff H
Userlevel 1
Here are some solutions for that:

If it's layer 2 then you can use these:

Thanks Srivats!
Userlevel 1
Thanks for replying B! I see now that you are specifically asking about stacking more so than ch...As you say, better to have more than none at all! I really do appreciate the feedback and the quick responses by you and your team!!!