BradReese.Com Cisco Hardware Quotes Call Toll Free 877-549-2680 or 828-277-7272

Home About Repair Power Supplies Refurbished Blog Quick Links Site Map Contact Us

M H Raza speaks out
Learn more about M H Raza...


Power Supplies

VoIP Gateways

Cisco Repair

Refurbished Cisco

Cisco CPQRGs

New Cisco

New HP ProCurve

Cisco Tools

Competitive Lab Tests

Tech Forums

How-to Tutorials

CCIE Gossip


View archive of M H Raza speaks out


The runway is running out on the Theory of Aggregation

If server virtualization saved the industry billions of dollars in the past decade, surely packet/network/cable virtualization is the new frontier that will save the industry billions in the next decade.

Cheshire, Connecticut:   Mon, 2/23/15 - 2:02pm    View comments

Data center networks have been designed with core switches (e.g. Nexus 7xxx) at the center of the network, aggregation switches (e.g. Nexus 5xxx) connected to the core and then TOR switches connected to aggregation switches, and in turn servers connected to the TOR switches. This Hub & Spoke model has been prevalent for quite some time, and it is changing rapidly, with Spine/Leaf type configurations as alternatives. Although these new architectures are better, they are still fixed architectures. Can we imagine a world where we do not have to rely on such fixed architectures, where we can allow network administrators to reconfigure physical infrastructure as well as packet forwarding plane dynamically via software? In such a world we would allow any port to connect to any other port by making the change in the physical infrastructure via software. Can you imagine what would happen if you could cohesively control the change in the physical connectivity and the packet flow rules and decisions from the same programmatic software interface?

In our office in Cheshire, Connecticut we have a framed picture of a lighthouse in a stormy ocean, with the following words under it: "The voyage of discovery is not in seeking new landscapes, but in having new eyes" (Marcel Proust). Hence I want to stand back a little bit and look at how we are evolving our data center architecture, so that we can use the good from past designs, but are able to think differently when we need to. The end goal for all of us is to provide unrestricted growth to data centers at a controlled cost and with as much flexibility as possible. In doing so if we can simplify the network, it would be a bonus. I would like to ask the reader to look at data center networking from a different eye for the next few paragraphs, and for this short journey to break away from the "Theory of Aggregation", as I call it.

The Theory of Aggregation can be seen in the development of switches from incumbent and new players alike. Switches with 48 x 1Gbps ports, plus 4 x 10Gbps aggregation ports; or 48 x 10Gbps ports, plus 4 x 40Gbps aggregation ports. This type of aggregation is designed to support a multi-tier Hub/Spoke model and then live with the restrictions that come with this structure. During its evolution cycle the Theory of Aggregation saw us uplink 1Gbps ports into 10Gbps ports, a 10:1 ratio; then we evolved to aggregating 10Gbps into 40Gbps, a 4:1 ratio; now we evolve to aggregate 40Gbps into 100Gbps, a 2.5:1 ratio. As this ratio shrinks, the aggregation pipe seems to be getting more constricted indicating that the Theory of Aggregation has a rather shortening runway. At the same time the reason that we even have these switches in the data center is to allow connectivity between thousands of servers. The server processor and number of cores also continue to evolve (more VMS, etc.) which leads to fill rates in the pipes leaving the server to increase, further shortening the runway to aggregation. We need to relook at how we build networks and we expect data centers to shift towards a more flexible architecture that can provide the benefits of a purpose-built network; an architecture that can be reconfigured dynamically as the network grows and its needs change.

Fiber MountainThe Glass Core architecture from Fiber Mountain says that there are thousands of fiber strands available to be configured as the network grows or its needs change.


Cable once and use software to define how a particular strand will be used, and reconfigure it as many times as you desire. You can use a strand in a 10Gbps connection today and connect it from one point to destination X, and then tomorrow you can reuse the same strand for a 40Gbps connection and deliver it to the same destination X, or to a different destination Y, with zero packet processing hops in between, with latency of less than 10 nanoseconds. All of this controlled via software, no human hands need to touch the network; no 40Gbps cable required instead of the 10Gbps cable; all work performed from the comfort of your desktop. Add to this the luxury of programmatically optimizing your data transport plane and packet flow rules as the network requires.

If server virtualization saved the industry billions of dollars in the past decade, surely packet/network/cable virtualization is the new frontier that will save the industry billions in the next decade.

Let us talk a little bit about servers and see how the Theory of Aggregation has influenced this most important component of the data center. The network exists to serve these glorious machines that grow in performance and capability every day. Everything other than the server is plumbing. The cables, the switches, the routers, all plumbing, installed to serve the needs of the host computer and its applications.

Servers have a need to send traffic to thousands of destinations, however they have been told that they are to send only to one master, the switch, which will then determine how to reach those thousands of destinations, they have been told that you have a few interfaces to the switch and as long as you can get it out of one of these interfaces, the switch knows what to do, you should only worry about aggregating packets to go out of the few interfaces.

Now let us imagine that a central software system, such as Fiber Mountain's Alpine Orchestration System (AOS) is aware of the entire network and can help that server have a better view of the landscape. And also imagine that the interfaces at the server have many fibers in them (24-fiber MTP, or 64-fiber MXC). With that many FIBER STRANDS THAT ARE A RECONFIGURABLE RESOURCE CONTROLLED BY SOFTWARE, the server can send traffic down many paths, and break away from the Theory of Aggregation. You can programmatically designate as many connections as are required, directly to storage, or a different servers or server clusters, or use the connections to create a server cluster, all without any intermediary switches. Can packet forwarding decisions be made in the server? How about thinking of switching itself as a NFV, in the hands of the server for whom we built the network. Yes, it looks like the power is in software and yes compute virtualization and VMware can have a field day with this type of network control and power. Add the innovations and work in silicon photonics into the equation and the future of networking gets even more exciting.

The enhancements to servers will take some time and it depends on several vendors and how they work together, but with the benefits being what they are, we will expect to see it sooner than most people anticipate and much sooner than some incumbent vendors would hope.

Other than server enhancements, technology mentioned in this write-up is working today and Fiber Mountain, a young startup company, continues to evolve their technology every day.

Related story:

Should data centers replace core and aggregation switches with hundreds of intelligent fiber cables?

What's your take?

Favorite Blog Story Picks
  1. The Cloud shakes up the WAN
  2. Former Cisco data center guru illustrates birth of first open hardware modular switching platform
  3. Cisco CEO and Red Hat CEO appear to contradict each other
  4. Did Goldman Sachs become fed up with the outlandish hubris of Cisco's CEO?
  5. Cisco vulgarly attacked after East Carolina University (ECU) campus recruitment
  6. Wall Street ranks Cisco #3 among companies cutting most jobs in 2014
  7. Mr. Showmanship bolts from Cisco's Office of the Chairman and CEO
  8. Cisco's new "Our People Deal" a cleverly sneaky layoff scheme?
  9. Three Meraki founders quit Cisco
  10. Cisco's most famed and credible technical evangelist, Jimmy Ray Purser, bolts
  11. Cisco techie ingests LSD to solve tough technical problems
  12. Rumor Padmasree Warrior fired by Cisco
  13. Rumor Cisco's internal network computer systems were hacked yesterday
  14. Cisco appears to have quietly demoted Carl Wiese, Senior Vice President of Global Collaboration
  15. Former Cisco M&A managers discuss Linksys disaster
  16. CCIEs are the most obvious hiring need for Cisco channel partners
  17. Cisco cloud security guru Raja Patel joining Intel Security
  18. Layoff Notice given to Cisco Partner Talent Network (CPTN)
  19. Former Cisco executive Don Pyle and 5 family members missing after tragic mansion fire
  20. View the archive of Bloggers speak out on BradReese.Com
comments powered by Disqus

Supplement Cisco SMARTnet Contracts


©2015 Alliance Networking LLC - Home - About - Repair - Power Supplies - Refurbished - Blog - Quick Links - Site Map - Contact Us