Reference Network Deployment Designs


Over in a Hardware topic, I posted:

The components of a starter kit are a good starting place, but it would be even more useful to build a picture of the components and costs of several deployment scenarios with current products. I'm in Chicago, so if we based our models on a local build, the network scale is on the large end, but we would want to plan in a way that can grow from seed deployments on a neighborhood scale. The design exercise might start from one or two hubs with backhaul connections and a growing fiber and wireless point-to-point, and be designed to grow to offer service to the whole region. So from about 10,000 client networks to millions.

and I would like to continue to discuss the basic elements of this design exercise. If there is a better place for this, or it would be better re-focussed a bit, I'm open to any of that.

I'm offering to be of service in this exercise by representing someone with enough knowledge to evaluate designs and proposals, but not an expert in networking. My actual background is in software development and operations, and I have a good foundation in hardware as well. Little work experience with hardware or networking, but I had the opportunity to take a VLSI design course as an undergraduate back in the '80s. My project had about 400-500 CMOS gate, and if I recall correctly it was to be fabricated in a 5 micron process.

So, to set up the scenario, I am recruiting a team to lead the establishment of a last-mile network in Chicago. We plan on building on existing infrastructure as possible and taking advantage of being in a competitive market for backhaul. We want to keep costs as low as possible because we want to offer free connections to those who can't afford it. Our revenue model will be more like public media than a typical network operator. Assume we will be able to raise $10-20/month/household connected.

First, is this reasonable? It is hard to assess the market costs of the network service components without actually getting service quotes and such. My understanding that the market was at about $10/symmetric Mb/sec/month a while back. Is there any reference that tracks this number in different markets? For Chicago, I expect to be able to find rates that are similar to most competitive markets, but the long term plan would offer services to establish similar networks in other locations. In that case, we would either need to find competitive local backhaul or transit services to a POP where they are.

I don't expect the network hardware to be a big monthly budget issue and the capital costs spread across a reasonable period and they disappear into the bandwidth costs. The Cumulus starter kit at about $50k with longer bits of fiber, dark fiber where it is already installed, or point-to-point radio links and such where that makes more sense would be enough to connect a number of neighborhoods.

There seem to be many solutions to bridge between our neighborhood backhaul and the customers. FTTB (business or block) plus wireless and local wire/fiber is what we would prefer, but wireless mesh could be used for more rapid build-outs. Ideally that would be RMP based appliances, and if they don't exist that's maybe another project to prototype nodes as educational projects for embedded hardware design.

3 replies

Userlevel 4
Hey Gerry,

This is a unique use-case that we could be able to handle given the exact requirements. For metro ethernet there is a lot of requirements that certain customers require that we just don't have yet. For example QinQ is something we don't support. We are more geared towards data centers then FTTH or FTTB.

That being said we will have a CSE reach out to you to discuss. We want to make Open Networking successful wherever we can. Look for an email in your inbox. Its not that we don't want to discuss in the Open Community, its just that we will end up writing a novella here trying to figure out what you are up to 🙂

For reference architectures these are all based around the data center but you can find them here on the main page cumulusnetworks.com and hover solutions, then Validated Design Guides (see screen shot)



I guess I'm going to have to do some more research. I understand about having the initial discussion of scope in email, I will keep an eye out for that mail.

Is it worth providing some general discussion of QinQ here? Just generally, as in why it would be needed for this situation and whether there are long terms plans to support it in Cumulus offerings, or are there other network OS's that are open and cover this area? I'm guessing that QinQ may be necessary for certain kinds of private (VPN-ish) network service offerings. We might be able to either not offer those services, or offer them later as the open network tools support it better. We could open this up as a separate topic if this makes any sense.
Userlevel 4

Is it worth providing some general discussion of QinQ here? Just generally, as in why it would be needed for this situation and whether there are long terms plans to support it in Cumulus offerings, or are there other network OS's that are open and cover this area?

There are other Open NOS (Network OS) out there but I don't think any support QinQ at this time. Most of our allies/competitors/etc are geared towards data center for various reasons. Feel free to Google around (I don't want to dissuade you from Open Networking at all).

We are really open about what we do and don't support but sharing a road map on the community is a bit of a stretch right now. Thats why we kind of want to take that conversation privately.

Also again there could be numerous other features out there we don't support that you would think are supported. If a customer comes to a company and wants to purchase 1000 switches for a feature that won't take that long to implement I bet things could change (make sense?). The hardware supports QinQ, we just don't have much demand to take the time to program the ASIC to accept Linux commands for this feature.

I'm guessing that QinQ may be necessary for certain kinds of private (VPN-ish) network service offerings.

Lets say you had a customer that is in two buildings and you want to have them on the same network. This company has 3 VLANs for themselves (VLAN 10 for PCs, VLAN 20 for Voice, VLAN 30 for printers). You could throw them into VLAN 10 for your upper network and all 3 VLANs would be sub-VLANs for that VLAN. This makes it very easy to scale out metro-Ethernet without something like MPLS/VRF which can accomplish something similar with virtual routing and forwarding (segmentation with VPNs).

In this case think of VPNs of private networks, not encrypted... (although they can be).

We might be able to either not offer those services, or offer them later as the open network tools support it better. We could open this up as a separate topic if this makes any sense.

Yeah when the CSE reaches out hopefully you can have this conversation. If customers are willing to work with the limitations we can try to make them successful.

Reply