Cisco NEXUS 2000 Series Fabric Extender (FEX)
In today’s post, we are going to
dive into new Concept – Fabric Extender. If you are familiar with Traditional
Modular Switches AKA Distributed Switching Architecture such as Catalyst
6500/6500 E series or 4500/4500 X series than Fabric Extender is going to be a
piece of cake for you. In Distributed Switching Architecture we basically have
following components:
Ø
Supervisor Engine –
A supervisor engine is basically Mind of switch responsible for management &
control plane management.
Ø
Line Card/ Modules -
The End Hosts or other devices gets connected on Line Cards/Modules, the basic function
of these is to take care of Data Plane/ Forwarding Plane.
Ø
Back Plane/ X-Bar
Fabric - The Supervisor Engine/ Engines talk to line cards & Modules using
High speed backplane circuitry.
Now in older Distributed
Switching platforms all these components reside in a single Chassis. What Cisco
did to take this concept a step further is came up with something known as
Fabric Extender. A fabric extender basically comes as Cisco Nexus 2000 Series.
A fabric extender works as a line card of its parent switch which could be
either Nexus 7k or Nexus 5k.
The communication between Parent
Switch and Fabric Extender happens through something called Fabric Ports. Let’s
see how our new Design with Parent Switch and Fabric Extender Looks like:
So as you can see, the Fabric
Extender (FEX) is now the device (Remote Line Cards) to which end hosts gets
attached. The FEX further gets connected to Parent Switch which is essentially
acting as supervisor engine. It’s the Supervisor Engine where entire
configuration is saved which means FEX don’t save any of the configurations
locally. Also it’s the parent switch where all forwarding decisions are made. Which
means if HOST A on FEX 1 wants to talk to another HOST B connected on same FEX
1, the traffic has to go to parent Switch and sent back towards the FEX as
shown in diagram below:
Which is obviously not an idle
scenario though but this is the way FEX has been designed. But on the other
hand introduction of FEX offers us couple of benefits as well:
Ø
Reduce Cable Runs
Ø
Reduces Management
Point ( Since Parent Switch Controls all FEXs)
Ø
Reduces IOS Management
& Standardization Load (The FEX always Runs same NX-OS as Parent SW, FEX
gets shipped with No NX-OS Image. When we connect FEX to Parent switch and
provision it, it downloads the NX-OS from parent switch and uses it)
Ø
Enables Parent
Switch (7K/5K) to become a high density access layer switch
Ø
STP Free Access
Layer ( The fabric links between Parent Switch & FEX runs no STP, on the
flip side FEX Host Interfaces(HIF) cannot be used to connect any further
switches or devices running Spanning-Tree. BPDU Guard feature is by default
enabled on HIF and cannot be disabled. Which means as soon as you plug any
device to the FEX running STP, the port will get error disabled)
Ø
QOS & Security
Management from Parent Switch
Ø
All Troubleshooting
From Parent Switch
Ø Less number of OOB connection
Since in most of the real world
designs you would want to Pair your FEX with Nexus 5K, so we will be discussing
basic FEX configuration from Nexus 5K perspective:
The official NX-OS version in
CCIE DC lab for Nexus 5548 is 5.1(3) & FEX models available in lab will be
2232 & 2224.
Let’s review the topology and
relative configuration now:
Step 1 > Enable FEX feature
set on Nexus 5k :
5k(config)# feature fex
Step 2 > Create a Port
Channel :
5k(config)# interface port-channel 101
Step 3 > Configure Port
Channel to Act as Fabric Link:
5k(config-if)# switchport mode
fex-fabric
Step 4 > Assign Associate ID
to Remote FEX (From range 101 – 199)
5k(config-if)# fex associate 101
Step 5 > Map Fabric Interfaces
to Port Channel:
5k(config)# interface
e1/1 - 4
5k(config-if-range)# channel-group 101
Once this configuration will be
done, we should be ready use FEX. The associate ID gets prepended to FEX
interface for identification. For example if FEX port were like : E1/1, E1/2….,
After association ID 101 assigned to FEX, on parent switch the ports will now
appear as: E101/1/1, E101/1/2. Now any configuration you apply under these
ports will be pushed to FEX itself.
Couple of verification commands:
Ø
sh fex
Ø
sh fex detail
Ø
sh interface status
fex = sh interface status (in regular IOS)
Couple of limitations with FEX :
> FEX Doesn't support Private VLANs
> Any port available on FEX cannot be SPAN destination port
> FEX Model like 2148 Can't be associated with Nexus 7K
> So far only FEX model 2232PP supports FCOE on FEX ports
> On Nexus 7k, the non-default VDC gets access to FEX feature only if
feature is installed and enabled under default VDC
> In Nexus 7-K all the uplinks and host ports of a Fabric Extender
belong to a single VDC. The ports cannot be allocated or split
among multiple VDCs.
> Nexus 7k F1 modules are though Layer 2 cards, but still doesn't support
FEX
> You can configure the Fabric Extender host interfaces as edge
ports only.The interface is placed in an error disabled state if a
downstream switch is detected.
> The Cisco Nexus 2148 Fabric Extender does not support frames
with the dot1p vlan 0 tag
And here is most interesting one. Just take a look at Pic below.
Actually you are looking at Back (rear) side of Nexus 2000 FEX,
and default Air-flow is from front to back.
HTH...
Deepak Arora
Evil CCIE
