Book Review - Building Data Centers with VXLAN BGP EVPN: A Cisco NX-OS Perspective

 

If you are completely new to EVPN, this book might help you understand the basics. But interestingly enough the book is dedicated to VxLAN BGP EVPN spread across roughly 450 pages and still miss to cover following topics in details:

- BGP underlay & Overlay designs details and tradeoffs beside best practices

- No details of EVPN route types, their encoding, flooding scope & optimization

- No details of design choices & tradeoffs 

- Lot of repetition 

- Very condensed packing of contents

- EVPN MLAG/Multi-homing details are skipped

- EVPN Policies stitching details are missing between different BGP AFIs & potential issues in ref. to policy control and information loss

- Multi-pod and Multi-fabric designs are are briefly touched  with no details but just plain theory

- EVPN over DCIs (L2 vs. L3) are very briefly touched with no details at all

- VxLAN & EVPN OAM are completely skipped

- No complete end to end configuration examples and scenarios

- Dual Stack & IPv6 only DCs are completely skipped

- Open Stack integrations are skipped

- How to deal between Physical and Virtual VTEPs is not covered

- Controller vs. Controller less designs are skipped

- Programmable fabric scalability details are not covered 

- Platforms specific details and limitations are not addressed 

- EVPN control plane security options are left for you to explore

HTH...

A Tech Artist 🎨

OTV Multicast-Mode Configuration Example Using OTV On Stick Design - OTV Series Part 3

 

Configuration Template Used In Demo:

DC1-N7K-1
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
vlan 200
 name OTV-EXTEND-VLAN
!
int lo0
 ip add 11.11.11.11/32
 exit
!
int e1/7
 ip add 1.1.1.1 255.255.255.0
 no sh
 exit
!
int e1/5
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
feature interface
!
feature pim
!
int vlan 200
 ip add 200.0.0.1/24
 no sh
 exit
!
int e1/1,e1/7
 ip igmp ver 2
 ip pim sp
 exit
!
ip pim rp-add 30.0.0.1
!


DC2-N7K-1
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
vlan 200
 name OTV-EXTEND-VLAN
!
int lo0
 ip add 22.22.22.21/32
 exit
!
int e1/14
 ip add 2.2.2.1 255.255.255.0
 no sh
 exit
!
int e1/12
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
feature interface
!
feature pim
!
int vlan 200
 ip add 200.0.0.2/24
 no sh
 exit
!
int e1/10,e1/14
 ip igmp ver 2
 ip pim sp
 exit
!
ip pim rp-add 30.0.0.1
!



DC1-N7K-2
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
vlan 200
 name OTV-EXTEND-VLAN
!
int lo0
 ip add 111.111.111.111/32
 exit
!
feature otv
!
int e1/11
 ip add 1.1.1.2 255.255.255.0
 ip igmp ver 3
 no sh
 exit
!
int e1/9
 sw
 sw mode tr
 sw tr all vlan 100,200
 no sh
 exit
!
otv site-vlan 100
 exit
!
otv site-iden 0x001
!
int overlay 1
 otv join-interface e1/11
 otv control-group 235.0.0.1
 otv data-group 232.0.0.0/4
 otv extend-vlan 200
 no sh
exit
!


DC2-N7K-1
+++++++++
!
vlan 100
 name OTV-SITE-VLAN
vlan 200
 name OTV-EXTEND-VLAN
!
int lo0
 ip add 222.222.222.222/32
 exit
!
feature otv
!
int e1/20
 ip add 2.2.2.2 255.255.255.0
 no sh
 ip igmp ver 3
 no sh
 exit
!
int e1/18
 sw
 sw mode tr
 sw tr all vlan 100,200
 no sh
 exit
!
otv site-vlan 100
 exit
!
otv site-iden 0x002
!
int overlay 1
 otv join-interface e1/20
 otv control-group 235.0.0.1
 otv data-group 232.0.0.0/4
 otv extend-vlan 200
 no sh
exit
!

Further Readings:

http://routing-bits.com/2011/06/16/cisco-otv-part-i/

http://routing-bits.com/2011/06/28/cisco-otv-part-ii/

http://routing-bits.com/2011/07/06/cisco-otv-part-iii/

http://blog.ine.com/2010/02/15/what-is-overlay-transport-virtualization/

http://blog.ine.com/2012/08/17/otv-decoded-a-fancy-gre-tunnel/

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DCI/5.0/OTVmulticast.pdf

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DCI/whitepaper/DCI3_OTV_Intro_WP.pdf

http://www.netcraftsmen.net/user-group/c-mug-archive/904-overlay-transport-virtualization.html

http://www.netcraftsmen.net/blogs/entry/spanning-layer-2-between-data-centers.html

http://www.netcraftsmen.net/blogs/entry/understanding-layer-2-over-layer-3-part-1.html

http://www.netcraftsmen.net/blogs/entry/understanding-layer-2-over-layer-3-part-2.html

http://www.netcraftsmen.net/blogs/entry/exploiting-layer-2-over-layer-3.html

http://www.netcraftsmen.net/blogs/entry/otv-optimal-routing.html

http://www.netcraftsmen.net/blogs/entry/cisco-overlay-transport-virtualization-otv.html

http://www.netcraftsmen.net/blogs/entry/data-center-l2-interconnect-and-failover.html

http://www.netcraftsmen.net/blogs/entry/designing-an-otv-deployment.html

http://www.netcraftsmen.net/blogs/entry/otv-and-dci-require-stp-defenses.html

http://packetpushers.net/pq-show-24-cisco-otv-deep-dive-part-1/

http://packetpushers.net/pq-show-25-cisco-otv-deep-dive-part-2/

http://packetpushers.net/show-155-integrating-otv-fabricpath-lisp-sponsored/

http://packetpushers.net/cisco-otv-101-legacy-multicast-mode-mac-learning-process-walk/

http://packetpushers.net/anycast-hsrp-design-considerations/

HTH...
Deepak Arora
Evil CCIE

OTV Unicast-Mode Configuration Example Using OTV On Stick Design - OTV Series Part 2

Note: DC1-N7K-2 Is Playing Role Of OTV Unicast Adj. Server

Configuration Template Used In Demo:

Getting Infrastructure Ready
############################

DC1-N7K-1
+++++++++

!
en
!
conf t
!
feature interface-vlan
!
vlan 100
 name OTV-SITE-VLAN
 exit
!
vlan 200
 name OTV-EXTEND-VLAN
 exit
!
int vlan 200
 ip add 200.0.0.1/24
 no sh
 exit
!
int e1/5
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
int e1/7
 ip add 1.1.1.1/24
 < Enable Routing Of Your Choice >
 no sh
 exit
!
int e1/1
 ip add 10.1.13.3/24
 < Enable Routing Of Your Choice >
 no sh
 exit
!

DC2-N7K-1
+++++++++

!
en
!
conf t
!
feature interface-vlan
!
vlan 100
 name OTV-SITE-VLAN
 exit
!
vlan 200
 name OTV-EXTEND-VLAN
 exit
!
int vlan 200
 ip add 200.0.0.2/24
 no sh
 exit
!
int e1/12
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
int e1/14
 ip add 2.2.2.1/24
 < Enable Routing Of Your Choice >
 no sh
 exit
!
int e1/10
 ip add 10.1.23.3/24
 < Enable Routing Of Your Choice >
 no sh
 exit
!


DC1-N7K-2
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
 exit
!
vlan 200
 name OTV-EXTEND-VLAN
 exit
!
int e1/9
 des "OTV-INTERNAL-INTERFACE"
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
int e1/11
 des "OTV-JOIN-INTERFACE"
 ip add 1.1.1.2/24
 < Enable Routing Of Your Choice >
 no sh
 exit
!


DC2-N7K-2
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
 exit
!
vlan 200
 name OTV-EXTEND-VLAN
 exit
!
int e1/18
 des "OTV-INTERNAL-INTERFACE"
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
int e1/11
 des "OTV-JOIN-INTERFACE"
 ip add 2.2.2.2/24
 < Enable Routing Of Your Choice >
 no sh
 exit
!

****************************************

OTV Configurations
##################


DC1-N7K-2
+++++++++

!
en
!
conf t
!
feature otv
!
otv site-vlan 100
 exit
!
otv site-identifier 0x01
!
int overlay 1
 otv join-interface e1/11
 otv adjacency-server unicast-only
 otv extend-vlan 200
 no sh
 exit
!


DC2-N7K-2
+++++++++

!
en
!
conf t
!
feature otv
!
otv site-vlan 100
 exit
!
otv site-identifier 0x02
!
int overlay 1
 otv join-interface e1/20
 otv use-adjacency-server 1.1.1.2 unicast-only
 otv extend-vlan 200
 no sh
 exit
!

Further Readings:

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DCI/whitepaper/DCI3_OTV_Intro_WP.pdf

HTH...
Deepak Arora
Evil CCIE

OTV Unicast-Mode Configuration Example Using OTV In Path Design - OTV Series Part 1

Note:- 100.100.100.100 is Redundant Unicast Adjacency Server In This Configuration Example & DC1-N7K-1 Is Primary Unicast Adj. Server In This Configuration Example.

Configuration Template Used In Demo:

DC1-N7K-2
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
 exit
!
vlan 200
 name OTV-EXTEND-VLAN
 exit
!
int e1/9
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
feature interface-vlan
!
int vlan 200
 ip add 200.0.0.1/24
 no sh
 exit
!
end
!


DC2-N7K-2
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
 exit
!
vlan 200
 name OTV-EXTEND-VLAN
 exit
!
int e1/18
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
feature interface-vlan
!
int vlan 200
 ip add 200.0.0.2/24
 no sh
 exit
!
end
!


DC1-N7K-1
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
 exit
!
vlan 200
 name OTV-EXTEND-VLAN
 exit
!
system jumbo 9216
!
feature otv
!
int e1/5
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
otv site-identifier 0x01
!
otv site-vlan 100
 exit
!
interface overlay 1
 otv join-interface e1/1
 otv adjacency-server unicast-only
 otv use-adjacency-server 10.1.13.3 100.100.100.100 unicast-only
 otv extend-vlan 200
 no sh
 exit
!


DC2-N7K-1
+++++++++

!
en
!
conf t
!
vlan 100
 name OTV-SITE-VLAN
 exit
!
vlan 200
 name OTV-EXTEND-VLAN
 exit
!
system jumbo 9216
!
feature otv
!
int e1/12
 switchport
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 no sh
 exit
!
otv site-identifier 0x02
!
otv site-vlan 100
 exit
!
interface overlay 1
 otv join-interface e1/10
 otv use-adjacency-server 10.1.13.3 100.100.100.100 unicast-only
 otv extend-vlan 200
 no sh
 exit
!

Further Readings:

http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/wan_otv/configuration/xe-3s/wan-otv-xe-3s-book/wan-otv-adj-server.html

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-644634.html

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DCI/5.0/OTVunicast.pdf

https://supportforums.cisco.com/document/64881/troubleshooting-arp-issues-across-otv

https://supportforums.cisco.com/document/65531/otv-silent-host-connectivity-problem

HTH...
Deepak Arora
Evil CCIE

Nexus 5K FEX, vPC & EvPC - All In One

Note: During the Video Demo the topology shown was little incorrect. Refer topology posted above during video demo.

N5K-1
+++++

!
en
!
conf t
!
vlan 100,200
 exit
!
feature vpc
!
feature lacp
!
feature fex
!
vpc domain 100
 role priority 1
 peer-keepalive dest 10.1.1.52 sou 10.1.1.51 vrf management
 exit
!
int e1/14-15
 channel-group 200 mode active
 no sh
 exit
!
interface port-channel 200
 switchport mode trunk
 vpc peer-link
 no sh
 exit
!
int e1/17
 channel-group 100
 no sh
 exit
!
int port-channel 100
 switchport mode fex
 fex associate 100
 vpc 100
 no sh
 exit
!
int e1/16
 channel-group 199
 no sh
 exit
!
int port-channel 199
 switchport mode fex
 fex associate 199
 vpc 199
 no sh
 exit
!
int e100/1/1,e109/1/1
 no sh
 channel-group 300
 exit
!
int port-channel 300
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 spanning-tree port type edge trunk
 no sh
 exit
!
end
!
===========================================================

N5K-2
+++++

!
en
!
conf t
!
vlan 100,200
 exit
!
feature vpc
!
feature lacp
!
feature fex
!
vpc domain 100
 role priority 1
 peer-keepalive dest 10.1.1.51 sou 10.1.1.52 vrf management
 exit
!
int e1/14-15
 channel-group 200 mode active
 no sh
 exit
!
interface port-channel 200
 switchport mode trunk
 vpc peer-link
 no sh
 exit
!
int e1/16
 channel-group 100
 no sh
 exit
!
int port-channel 100
 switchport mode fex
 fex associate 100
 vpc 100
 no sh
 exit
!
int e1/17
 channel-group 199
 no sh
 exit
!
int port-channel 199
 switchport mode fex
 fex associate 199
 vpc 199
 no sh
 exit
!
int e100/1/1,e109/1/1
 no sh
 channel-group 300
 exit
!
int port-channel 300
 switchport mode trunk
 switchport trunk allowed vlan 100,200
 spanning-tree port type edge trunk
 no sh
 exit
!
end
!

Further Readings:
==============

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5500/sw/interfaces/7x/b_5500_Interfaces_Config_Guide_Release_7x/b_5500_Interfaces_Config_Guide_Release_7x_chapter_0101.html

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5500/sw/layer2/7x/b_5500_Layer2_Config_7x/b_5500_Layer2_Config_7x_chapter_01111.html

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/mkt_ops_guides/513_n1_1/n5k_enhanced_vpc.html

http://www.cisco.com/c/en/us/support/docs/switches/nexus-5000-series-switches/116271-problem-vpc-00.html

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5500/sw/troubleshooting/guide/n5500_trouble/n5500_ts_vpc.html

HTH...
Deepak Arora
Evil CCIE

Cisco Exam Prices - Increased Again

Cisco has increased the prices again for certification exam including famous mammoth CCIE. The Lab Fee has been increased by 100$ and now is 1600$ where as written fee has been increased by 50$ and now is 400$.

 

HTH...
Deepak Arora
Evil CCIE

How To Bring Old "wr" Command Into Cisco Nexus To Save Configs - Quick Tip

Well I always use "wr" command short for "write" onto Cisco devices to save configurations. I am just too much use to with this syntax that I even don't remember when I used other command " copy run start" last time. :)

On the flip side Cisco Nexus Series platforms don't have "wr" command.

Well the way you can solve this problem is by creating an ALIAS for "copy run start" command and it should be helpful for Operations/Support Teams as well.

In Global configuration mode run the following command:

"cli alias name wr copy running-config startup-config" 

HTH....
Deepak Arora
Evil CCIE

How CCIE Data Center Candidate's Browser Should Look Like

http://www.cisco.com/c/en/us/products/servers-unified-computing/product-listing.html

http://www.cisco.com/c/en/us/products/servers-unified-computing/index.html

http://www.cisco.com/c/en/us/products/switches/data-center-switches/index.html

http://www.cisco.com/c/en/us/products/switches/nexus-7000-series-switches/index.html

http://www.cisco.com/c/en/us/products/switches/nexus-5000-series-switches/index.html

http://www.cisco.com/c/en/us/products/switches/nexus-2000-series-fabric-extenders/index.html

http://www.cisco.com/c/en/us/products/storage-networking/index.html

http://www.cisco.com/c/en/us/products/application-networking-services/data-center-application-services/index.html

www.cisco.com/cisco/web/psa/configure.html

http://www.cisco.com/c/en/us/solutions/enterprise/validated-design-program/index.html

https://www.ciscolive.com/online/connect/publicDashboard.ww

https://www.ciscolive.com/online/connect/agenda.ww?zid=HomeSlideshow-Agenda2

https://learningnetwork.cisco.com/docs/DOC-17214

https://learningnetwork.cisco.com/community/learning_center/ccie_datacenter_tech_seminars


HTH...
Deepak Arora
Evil CCIE

Time To Get Rid Of 7 IP Networking Myths - Based On Recent Interviews I Did

In last 6 months I got opportunity to work closely with HR teams from different regions within company and must have interviewed well over 200 People which gave me good opportunity to talk to so many candidates ranging from CCNA to CCIE & even Multiple CCIEs.

During this whole process I came across couple of Myths related to IP Networking which I would want to share so that together we can spread more awareness among other Engineers & Clients.

Myth 1 : vPC eliminates STP into the Network.

Well vPC tweaks STP behavior quite a bit for it’s operation but STP is still running in backend as fail safe. Even more vPC Primary switch send regular STP BPDUs by default over vPC member ports. Well the only solution to get rid of STP completely today from Cisco’s perspective is implementing Fabric Path.

Myth 2 : EIGRP is a better protocol for Cisco only network since it has better convergence features such as Feasible Successor if compare to OSPF. Same thought people have when comparing ISIS with OSPF.

Well convergence in itself is domain now from IP Networking perspective or perhaps always was. It’s just people these days talk more about it as requirements have grown tight in last 5 years because of Next Generation Network Requirements and more sensitive applications.

Most routing protocols don’t spend too much time for re-calculation once the primary path gets failed. The most contributing factor in convergence is detection of failure rather. A simple example would be to have two EIGRP neighbors connected through multiple links including a Ethernet connection through L2 Switch which is currently primary path. If link between L2 switch & Router 2 fails, Router 1 will notice the failure only after dead timer expire and than will run DUAL for recalculation depending upon it has Feasible successor or not. So in this case it’s not going to be any different than OSPF or IS-IS.

Myth 3 : OTV is the best solution for VMotion over Data Center Interconnect (DCI) or any Layer 2 extension requirements.

Well It depends. The part of equation here is VMotion requirements itself. For Hot VMotion the end to end delay requirements is less than or equal to 5 msec which in new ESXi version got relaxed to 10 msec I guess. So if the delay is higher and you have Hot VMotion requirements, OTV won’t do any good as DCI. Also choosing DCI Technology Between OTV and others like MPLS Martini Style, VPLS etc has lot to do with Scale related to number of MAC Addresses to be learned & Exchanged.

Myth 4 : One major benefit of using MPLS is that Router Switches packets faster since it uses LIB

Well it use be true in history but ever since the introduction of Hardware based forwarding in platforms and platforms having great amount of CPU and Memory, it’s not true any longer.

Myth 5 : Nexus 2000 is a Switch

Well it can’t perform local switching between 2 ports which might be even in Same VLAN. All the switching is still performed by Parent Switch (7k or 5k) based on VN Tags.

Myth 6 – Once the EIGRP SIA Timer Expires, The Neighbors are bounced

Well Cisco solved this problem I guess almost 10 years back with implementation of SIA Query & Replies. It’s just of those things which are not documented in most Cisco Press Books J covering EIGRP

Myth 7 – Traceroute is part of ICMP Suite

Well traceroute is an application but not a protocol or part of any protocol suite in itself. Traceroute has different implementations using ICMP, TCP & UDP by different vendors.

HTH...

Deepak Arora

Evil CCIE

VRF Routing Context - Lets Make Virtualization Life Easier

VRF aware pings and test are frustrating many times specially during troubleshooting where you try to ping or trace a destination or check routing table and later you notice you forgot to run vrf aware command. To solve this problem Cisco has introduced a cool feature known as VRF Routing Context into IOS and NX-OS which makes life lot easier specially during troubleshooting. Here is a quick example and topology details:

 

 

 

HTH....

Deepak Arora

Evil CCIE

 

Cisco Catalyst 6500/6800 Instant Access - Answer To Cisco Nexus 2000 AKA FEX For Enterprise ?

As Cisco Launched its Next Gen Catalyst 6000 Series as Catalyst 6800 based on One of it's most successful Enterprise switch product Cat 6500's DNA at recent Cisco Live Event At Orlando, There was an advantage that competing Nexus Series had even as solution to Enterprise market was Nexus 2000 AKA FEX, which was offering lots of benefits.

While the roadmap/investment protection of Nexus Family is yet to be clarified by Cisco as they keep coming with newer versions and flavors, the Cat 6500 always seen as very mature platform for Enterprise solutions.

Even to compete further with Nexus family, Cisco Catalyst team came up with a latest solution recently as INSTANT ACCESS (IA).

The benefits Cisco IA offers are some what similar to Nexus 2000/FEX. Though there are few hard requirements like :

> SUP 2T

> VSS CODE

> Switches To be configured in VSS Mode even if it's a standalone chassis

> 6904 10/40Gig line cards to be operated in 10Gig mode only

Further Readings:

http://www.cisco.com/en/US/prod/collateral/switches/ps10902/ps715/ps13198/white_paper_c11-728265.html

http://www.cisco.com/en/US/prod/collateral/switches/ps10902/ps715/ps13198/qa_c67-728684_ns1240_Networking_Solutions_Q_and_A.html

http://www.cisco.com/en/US/netsol/ns1240/index.html#~Overview

HTH...
Deepak Arora
Evil CCIE

FCoE - (Storage Series Part-10) Final

Fibre Channel Over Ethernet
+++++++++++++++++++++++++++

- Unified Fabric = Unified Wire = Converged Ethernet = Data Center Ethernet = Data Center Bridging

- In unified fabric  - Ethernet & Fibre channel both run on common infrastructure

- FCoE Initialization Protocol (FIP) is control plane protocol for FCoE running between Initiator & FCOE Switch/FCoE   Forwarder(FCF)

- ENode in FCoE = N port in FC

- Virtual Fibre Channel (VFC) interface is logical interface mapped with physical interfaces. FIP Runs between ENode &   VFC using P2P Adjacency

- Virtual Port Types in FCOE

 > VN Port = N Port in FC
 > VF Port = F Port in FC
 > VE Port = E Port in FC < Used in Multi Hop FCOE
 > VTE Port = TE Port in FC

- FCoE replaces layer 1 & layer 2 tranport for FC but all upper layer FC services remains same like Zoning, Domain IDs,   FSPF, FLOGI, FCNS etc

- FIP is control place of FCoE and FCoE itself is the actual data plane

# FIP
++++++

- FIP has new Ether Type as 0x8914

- FIP is used to discover FCFs and perform FLOGI

# FCOE
+++++++

- New Ethertype 0x8906

- Max length of 2240 bytes which implies jumbo frames are required

www.t11.org/fcoe


# FCoE Addressing
+++++++++++++++++

- Fibre Channel uses 3 bytes FCIDs

- Ethernet uses 6 byte MAC Addresses

- FCoE ENode gets a Fabric Provided MAC Address (FPMA) for FCoE

- During FIP, ENode is assigned a 3 byte FCID

- FLOGI now is part of FIP

- FCF is configured with a 3 byte FCoE MAC Address Prefix (FC-MAP)

- ENode appends FC-MAP to FCID

- 3 byte FC-MAP + 3 byte FCID = FPMA

# Configuring FCOE
+++++++++++++++++++

- feature fcoe < Feature FCOE can only be enabled from Admin VDC in Nexus 7k
- feature lldp

- Create a VSAN

vsan database
 vsan 1010
 vsan 1010 interface fc1/10

- Associate VSAN to a VLAN

vlan 1010
 fcoe vlan 1010

- Configure a VFC
- Associate Physical Ethernet to a VFC

int vfc117
 bind interface e1/17
 switchport trunk allowed vsan 1010

- Assign VFC to the VSAN

vsan database
 vsan 1010 interface vfc117

- Configure physical ethernet as a trunk

int e1/17
 shut
 switchport mode trunk
 spanning-tree port type edge trunk
 switchport trunk native vlan 10
 switchport trunk allowed vlan 10 ,1010

- Activate Interfaces

int vfc117
 no shut

int e1/17
 no shut



sh int vfc117 < Look for trunking & VSAN should be up

sh fcoe database

Nexus 5k doesn't support AL (Arbitrated Loop) topology

sh system internal dcbx info interface e1/17

> Look for DCX Protocol where CEE means Gen2 or Gen2+ CNA and anything else like CIN means Gen1 CNA

FCOE traffic is marked with COS-3 by default

SAN gets 50% of BW incase of congestion and rest 50% is reserved for LAN

sh int fcoe

Further Readings:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/mkt_ops_guides/513_n1_1/n5k_ops_fcoe.html


HTH...
Deepak Arora
Evil CCIE

Enhanced Zoning - (Storage Series Part-6)

FC Aliases
++++++++++

- FC Aliases gives user friendly names to the WWNs, FCIDs etc and are analogous to DNS in IP

- Syntex - fcalias name

- Alias can be advertised through Zoneset distribution. Syntex - zoneset distribute vsan 1

- Example:

!
 fcalias name ABC vsan 30
 member pwwn
 exit
!

- Verification: sh fcalias

- Creating zone and Zoneset using ALIAS:

!
 zone name ABC vsan 30
 member fcalias DISK1
 member fcalias DISK2
 member fcalias DISK3
 member fcalias SERVER
 exit
!

!
 zoneset name XYZ vsan 30
 member ABC
!

!
zoneset activate name XYZ vsan 30
!

+++++++++++++++++++++++++++++++++++

Basic Vs Enhanced Zoning
########################

- By default the full zoneset is local and the active zoneset is fabric-wide

- Order of operations errors can corrupt the Active Zoneset

- Enchanced zoning prevents this by Locking the fabric which ensures people don't 
   accidentally overwrite each other

- Enhanced Zoning works on per VSAN basis

- The lock on zone is released by "committing" the zoneset

- But if Admin forgets to commit, someone else can release lock as well with 
   "clear zone lock vsan "

- Enabling Enhanced Zoning

> zone mode enhanced vsan < Per VSAN Basis

> system default zone mode enhanced < For Entire System


- Configuring Enhanced Zoning doesn't change anything for regular FCALIAS and
   they still remain local by default

- To solve this problem solution is "Device Aliases"

- Device Aliases serves the same purpose as FC Aliases by binding PWWNs to a user 
   friendly name but the difference 

 is that the binding is advertised to the fabric and doesn't remain local only

- Configured as device-alias database

- Changing the zoning mode from normal to enhanced is not disruptive

- Device aliases are advertised through CFS so should not be used in multi vendor environment


+ Configured as "device-alias database"

+ device-alias commit


E.g.>

device-alias database
 device-alias name ABC_Server pwwn
 device-alias name XYZ_Disk1 pwwn
 device-alias name XYZ_Disk2 pwwn
 device-alias name XYZ_Disk3 pwwn
 device-alias commit

show device-alias database


zone name ABC vsan
 member device-alias ABC_Server
 member device-alias XYZ_Disk1
 member device-alias XYZ_Disk2

zoneset name XYZ vsan
 member ABC
 exit

zoneset activate name XYZ vsan

sh zoneset pending

zone commit vsan

sh zone active

"sh port-resource module " shows port grouping on MDS 
  (shared vs dedicaed mode), in 5k all ports are in dedicated mode by default.

HTH...
Deepak Arora
Evil CCIE

Catalyst 6800 - A new Cisco Baby to replace Cat 6500

As the Cisco Live 2013 at Orlando is keep going on, Cisco just announced a New Baby into Catalyst family as Cat 6800. Based on Cat 6500 DNA with better performance (Perhaps matching Nexus) and backward compatibility with Cat 6500 series line cards the new Baby is meant to replace current Cat 6500 and give an option to people which probably don't want to move on to Nexus yet.

More Information
++++++++++++


http://www.youtube.com/watch?v=NHYWAE8FIzk 

http://www.youtube.com/watch?v=K-igQ-UsUz8

www.cisco.com/go/6800


HTH...
Deepak Arora
Evil CCIE

http://www.youtube.com/watch?v=K-igQ-UsUz8