Clos Fabrics AKA Spine & Leaf Architecture

Let's start the series with discussion of CLOS Fabrics AKA Spine & Leaf Architectures.

Now CLOS design is not fundamentally new, but most of the Network Engineers were not talking about it till recent times (Well...this is true to an extent). So as Network Engineer should you really care ?

Well you should start by asking why CLOS in first place ?

The major problem that CLOS fabric solves is about solving scalability issues. While scalability is a matter of context, it's not necessary that everyone needs or to be precise going too far about it.

Also CLOS fabric also doesn't define your Layer2 - Layer 3 boundaries itself. So you are pretty much dependent upon what works best for you from vendor implementation perspective while keeping your overall goal in mind. Now in theory Layer 3 Fabrics scale much better than Layer 2 Fabric. Here are some questions/Things you figure out about CLOS if you decide to go for it :

- What is the scale that you got to deal with ?
- What are technical and business requirements ?
- Your DC traffic is mostly east-west or north-south ?
- How you can minimize the state of the Core (Spine) to minimum ?
- How flooding works in your fabric ?
- How multicast is handled in fabric ?
- Where to define Layer2-Layer 3 boundary ?
- Your network is going to multi vendor now/In future ?
- How you gonna manage and monitor such large network ?
- How you gonna introduce security & Services such as Load Balancer ?
- How you gonna connect to external world ? (Border Spine Vs. Border Leaf) 
- Define you convergence requirements 
- You gonna need single or multi stage CLOS ?
- Your over subscription ratio ? (Usually 3:1 is good for most part)
- Understand your failure domains and impact they may have
- Do you need Spine to Spine or Leaf to Leaf connections to mitigate some of     
   failure scenario ?
- If you are going with Layer 3 fabric, is it going to be good idea to use 
   summarization ?
- EBGP vs IBGP (Also RR placement) in Layer 3 fabric ?


Even as an example, Cisco's famous buzzword these days ACI (Application Centric Infrastructure ) also uses Spine & Leaf design. It uses BGP EVPN (Some secret souce but soon EVPN will be there too) control plane and on top of which it uses VXLAN as Data Plane. So between Spine & Leaf (Single Stage) it uses Layer 3 fabric. The entire fabric is managed with a centralized command and control system called Cisco APIC Controller. With ACI you can go as far as 6 Spines at the moment and all services (e.g. load balancer), firewalls, external connectivity gets terminated on Leaf switches. For server redundancy (Bare Metal Or Virtual ) it uses our old friend Virtual Port Channel (vPC) but this time doesn't require directly connected interfaces among leaf switches for peer link and peer keep alive link functions. 

Cisco ACI is kind of build around another buzz word that you hear more often these days called SDN (Software Defined Networks). Now whether it fits into true SDN definition or not needs another discussion :).

In the mean while below is the list of URLs which you may find very handy to get started with CLOS:

http://packetpushers.net/podcast/podcasts/datanauts-011-understanding-leaf-spine-networks/

https://code.facebook.com/posts/360346274145943/introducing-data-center-fabric-the-next-generation-facebook-data-center-network

http://www.networkworld.com/article/2226122/cisco-subnet/clos-networks--what-s-old-is-new-again.html

http://searchdatacenter.techtarget.com/feature/The-case-for-a-leaf-spine-data-center-topology

http://searchdatacenter.techtarget.com/answer/Whats-the-best-data-center-network-topology

http://searchdatacenter.techtarget.com/feature/Data-center-network-design-moves-from-tree-to-leaf

http://www.excitingip.com/4490/distributed-coreleaf-spine-network-architecture-an-intro/

http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=4448982&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel5%2F90%2F4359146%2F04448982.pdf%3Farnumber%3D4448982

http://etherealmind.com/which-network-topology/

http://packetpushers.net/network-topologies/

http://blog.ipspace.net/2014/04/security-in-leaf-and-spine-fabrics.html

http://www.thenetworkingdom.net/bgp-clos-networks/

http://thenetworksurgeon.com/cisco-spine-and-leaf-architecture-discussion-nexus-5500-vs-6001/

http://blog.ipspace.net/2012/04/full-mesh-is-worst-possible-fabric.html

http://conferences.sigcomm.org/sigcomm/2015/pdf/papers/p183.pdf

http://conferences.sigcomm.org/co-next/2013/program/p49.pdf

https://www.ietf.org/id/draft-ietf-rtgwg-bgp-routing-large-dc-09.txt

https://www.nanog.org/meetings/nanog55/presentations/.../Lapukhov.pdf

http://content.blubrry.com/packetpushers/Show_164-Cool_or_Hot-Lapukhov_Nkposong_s_BGP_SDN.mp3

http://www.juniper.net/us/en/local/pdf/whitepapers/2000565-en.pdf

https://cumulusnetworks.com/blog/routed-vmotion-why/

https://www.youtube.com/watch?v=nPKLe0M5yJU

https://www.youtube.com/watch?v=O8wU1qNlsyI

https://www.youtube.com/watch?v=fQ2l_v_53qQ

HTH...
Deepak Arora
Evil CCIE

How To Be Network Ninja In 21st Century

I have been asked so many times by Network Engineers right from starter level to Expert level people about how network industry is changing at rapid pace in last few years and questions like if certification and in particular CCIE holds any value any longer. I also spoke with couple of friends that I truly admire and are working in US , Europe & Australia to get feedback on how Network industry is evolving there.

Now to start with, following technologies are definitely picking up in some form or shape :

- SDX like (SDN - Software Defined Networks)
- CLOS/ Spine & Leaf Designs
- NFV (Network Function Virtualization)
- Virtualization ( In Areas like Network, Compute & Storage)
- Automation ( Chef, Puppet etc...)
- Scripting & Programming ( Python, Bash, Java etc...)
- Cloud Computing
- Network Visibility 
- Overlay Networks/Tunneling Technologies (VXLAN, NvGRE etc...)
- Network Modeling Methods
- API (Application Program Interface like REST )
- Understanding on Unix & Linux
- Big Data
- Active Active Data Centres
- Machine Learning
- Segment Routing 
- Containers ( Docker...)
- Deep Understanding of Applications Structures/Component & Life Cycle 

And last but not least deep understanding of protocols like TCP, HTTP etc...

But again the impact these may have on current network industry (What they call traditional networking now) may vary by large margin depending upon:

- Which part of the world you are living in
- How IT Industry is driven there and Network Industry in particular
- Which company you work for
- What are your personal/political views
- Company's IT Strategy & Road Maps
- Your current skill set & fears about these new technologies
- Is there any simulation tool to get familiar with these technologies
- What is the maturity level of given technology (RFC ?, New Model ? etc...)
- How you want to manage these solution (Afraid of Open Source ? Multi vendor   blame game ? etc...)
- Do you really have a good business case 

Now there are of course other factors including budget/cost, ROI, How to get your operational staff ready etc...

But hope you get the idea. In the coming series of posts I would express my personal opinions around all these but those articles are going to be semi technical rather being completely technical since I am more of a Pre-Sales guy now.

Regards,
Deepak Arora
Evil CCIE