Catalyst 6800 - A new Cisco Baby to replace Cat 6500

As the Cisco Live 2013 at Orlando is keep going on, Cisco just announced a New Baby into Catalyst family as Cat 6800. Based on Cat 6500 DNA with better performance (Perhaps matching Nexus) and backward compatibility with Cat 6500 series line cards the new Baby is meant to replace current Cat 6500 and give an option to people which probably don't want to move on to Nexus yet.

More Information
++++++++++++


http://www.youtube.com/watch?v=NHYWAE8FIzk 

http://www.youtube.com/watch?v=K-igQ-UsUz8

www.cisco.com/go/6800


HTH...
Deepak Arora
Evil CCIE

http://www.youtube.com/watch?v=K-igQ-UsUz8

Zoning - (Storage Series Part-5)

Zoning
######

- Controls which initiators can talk to which targets

- Zoing in required, not optional

- Default zone policy is to deny everything in Data Plane

- Default zone policy can be changed per VSAN or for entire system but not recommended

> zone default-zone permit vsan 1

> system default zone


# Soft Zoning
=============

- Initiator registers with FCNS to get zoning

- Zoning enforced in the control plane but not data plane

- Initiator could manually mount the wrong target if Target FCID is known in prior


# Hard Zoning
=============

- Initiator registers with FCNS to get zoning

- Zoning enforced in the control plane & data plane

- Initiator cannot manually mount the wrong target if Target FCID is known in prior

Hard Zoning is default in NX-OS and SAN-OS


# Zone Vs Zoneset
=================

- Zone is used to create a mapping for access based upon WWPNs, FCIDs, Aliases, 
  Interface, Domain-ID etc

- Zones are then grouped together in a Zoneset

> Zoneset is overall ACL and Zone is an Access Control Entry

- Zones are applied to VSAN and then activated

# If zone is just configured and not activated - it's called a Zone Set. But when activated
   it's called Active Zone  Set. So this is how Zone set Vs Active zone set could be different

# Any changes made into Zoning doesn't go into affect unless Zoneset is re-activated

- Only one zoneset per VSAN can be "Active" in the fabric at a time

- By default only Active Zone set is advertised into the fabric not the full zone set which 
   can cause Isolation   between Switches because of having different copies of zone sets.

- sh zone status vsan 1

- sh zone

- sh zone active

- sh zoneset

- sh zoneset active

- clear zone database vsan 1

- zoneset distribute full vsan 1 ( Global mode)

- zoneset distribute vsan 1 (Exec Mode)


# Zone Config
==============


zone name ABC vsan 30 < Create Zone
 attribute read-only < Only Supported on MDS
 member pwnn
 member pwnn
 exit

zoneset name XYZ vsan 30 < Create Zoneset
 member ABC
 exit

zoneset activate name XYZ vsan 30 < Activate ZoneSet

HTH...
Deepak Arora
Evil CCIE

San Port Channels & Order Of Operation - (Storage Series Part-4)

Sample SAN Port Channel Config
==============================

interface san-port-channel 1 >> In Nexus # in MDS >> interface port-channel
 channel mode active
 switchport mode E
 swithcport trunk allowed vsan 1
 switchport trunk allowed vsan add 10
 switchport speed 4000


!
int x/x
 channel-group 1 > Static port channel
 channel-group 1 foce > To add any link later into functional San Port Channel
 channel-mode active > To enable port channel protocol



Verification
============


sh port-channel summary

sh san-port-channel summary

sh port-channel internal info interface po1

sh interface san-port-channel 1 trunk vsan


=============================================================================


This is order of operations for E/TE:

1. Go to the MDS physical interfaces, shut them down
2. Need to ensure the individual ports are dedicated - 'switchport rate mode dedicated' 
    otherwise will error out on port-channel configuration. Can also configure 
    'switchport mode e'
3. Add ports to port-channel - 'channel-group X'
4. Go to the port-channel, apply the configuration ('switchport mode e' and 
    'switchport rate mode dedicated'.
5. If required, restrict the port-channel with the only the necessary VSANs
6. Add the port-channel to the VSAN database (optional - again depends on requirements)
7. No shut the port-channel and then physical ports

* Make use of 'show interface fc x/y' and verify config - look at: Admin port mode, 
    trunk mode, port mode, port vsan, VSANs allowed (if trunking).
* Ensure that these settings match on both sides (either MDS to N5K or UCS)

For F/TF port-channel, steps that are slightly different from above:

1. Enable MDS features (such as npiv / fport-channel-trunk - if making a TF port-channel)
2. For F ports, can leave in either shared or dedicated mode. Additionally, configure the 
    physical interfaces as 'switchport mode f', otherwise will error out as generally 
    they default to FX ports
3. Instead of 'switchport mode e' on the port-channel, obviously make it an F port

If link fails to come up and using NPV-NPiV, useful to make use of 'show npv status', 
 will also identify if you've forgotten to enable NPiV upstream, etc.

HTH...
Deepak Arora
Evil CCIE

Fiber Channel Fabric Services - (Storage Series Part-3)

Principle Switch (PS) Election
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

# Election starts when E port between two or more switches comes up

# Assign Domain IDs automatically but can be assigned manually as well 
   with - ( fcdomain domain 0x51 vsan )

- Preferred > Switch asks principal switch if it can use static domain id, if PS 
   says "Yes" than fine, If "NO" it uses Domain ID given by PS

- Static > Same theory as above but if PS says NO, the switch gets isolated from the FC network

Election based on
++++++++++++

- Lowest priority ( fcdomain priority 1 vsan 1)
- Lowest WWN ( sh wwn switch)

* sh fcdomain (vsan)
* sh fcdomain domain-list (vsan)


FC Domain Services Restart
+++++++++++++++++++

- Graceful - fcdomain restart vsan 1

- Forced - fcdomain restart disruptive vsan 1 < Hidden command

Everytime we make changes into FC Services, the services must be restarted

==============================================================

# Note: One copy of FC services runs on per VSAN basis which means One
  domain id can be used for multiple VSANs but it doesn't affect the design. 
  It's more like using one OSPF router id for multiple ospf processes. But each 
  process populates a separate Datbase which is completely unrelated from each other.

==============================================================

Fabric Shortest Path First (FSPF)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- Used to build an SPT through the fabric

- Domain ID is the Node ID in the SPT calculation

- FSPF runs automatically on per VSAN basis

- FSPF perameters can be manually modified in global config mode with 
   "fspf config vsan 1" or on interface level like
   " fspf [cost|dead-interval|hello-interval|passive|retransmit-interval]


Verification
++++++++++++

- sh fspf [database|interface] [vsan]

- sh fcroute unicast vsan
 
==============================================================

Fabric Login (FLOGI)
^^^^^^^^^^^^^^^^^^^^

- All initiators and targets must FLOGI before sending any data into the fabric

- Verification - sh flogi database vsan

- No configuration required

- No FLOGI indicates a basic link-level negotiation problem

- The FLOGI database is local to switch which means only directly connected devices 
    will show up, It's FCNS job to   progpogate this information to entire fabric


==============================================================
 
Fiber Channel Name Services
^^^^^^^^^^^^^^^^^^^^^^^^^^^

- FCNS (AKA Directory Services) keeps a mapping of FCIDs to WWPNs

- Analogous to IP ARP Cache in ethernet

- End devices register with the FCNS after FLOGI

- sh fcns database

- If Node did FLOGI but is not in everyone's FCNS, it indicates the Fabric is broken

> E.g. VSAN is isolated, EISL allowed list is wrong etc

Initializing Nexus 5K For Storage - (Storage Series Part-2)

As I continue my journey with storage networking, I compiled some more notes today covering Nexus 5K initialization from Fiber Channel Implementation Perspective. Hope you guys will enjoy !!!

 ######################################################################

 * To verify SAN Port type and topology on N5k => sh int brief

+ In output look under " Oper Mode "

Sample SAN Port Channel Config
==============================

interface san-port-channel 1
 channel mode active
 switchport mode E
 swithcport trunk allowed vsan 1
 switchport trunk allowed vsan add 10
 switchport speed 4000


* Changing Port Type to FC in 5k
================================

slot 1 or 2
 port 25 - 32 type fc

+ Requires Reload

# " feature fcoe " needs to be turned on separately in order to enable basic FC protocol
     which also further enables actual FCOE. Though NPV for FCOE requires separate license.

+ If port type has been changed to FC but feature fcoe is not enabled, the FC ports 
    will not show up in "sh int brief" output. Also we will not be able to go into FC interface 
    configuration mode.

+ The SFP for storage connecivity is different from SFP for Ethernet connection from 5K standpoint

* FC Auto Negotiation
=====================

> FC Auto Negotiation is enabled by default

> Auto Negotiation Determines:

+ Port Type [ switchport mode ]

+ Trunk Mode [ switchport trunk mode ]

+ Speed [ switchport speed <1000> ]

- From real world standpoint, Port Types etc should be configured manually

* MDS Auto-Negotiation
======================

- MDS port types are by default FX which means it can figure out if it needs to be F or FL 
    port mode during auto-negotiation

- MDS usually can't negotiate E ports so ISL must always be configured manually.

- In MDS E port requires dedicated rate-mode [switchport rate-mode dedicated ]

+ In FC network it's not only SFP type that will control the link speed but also Optical 
    cable should be of same speed in order to match it

+ In FC network there is something known as B2B or Buffer To Buffer credit which is
   analogous to Flow Control is Ethernet Networks.B2B works per link basis and on end 
   to end basis between initiator & target


* FC Trunk Protocol
===================

- Trunk protocol (TP) in FC is analogous to DTP in ethernet and is enabled by default

- Verification - sh trunk protocol

- To disable TP - switchport trunk mode on (Default - auto)

But keep in mind there is no command called "switchport mode te" to statically configure TE port

Official Cisco CCIE Count - By March 2013

2012

 

 

 

2013

Fiber Channel Notes - Welcome to the SAN World (Storage Series Part-1)

After couple of months break now I am back on my CCIE DC preparation track. Since I was pretty much done with Nexus at Initial level and also did couple of Nexus Installations at work along with OTV deployment I thought to lets start with storage now. Since I had no experience with storage at any level earlier so I thought to start with baby steps as of now and catch up the pace later.

When it comes to storage there is no better choice other to start learning one of Storage Core Protocol known as Fibre Channel AKA FC.

So I did some initial FC study yesterday and compiled some notes for reference. Hope you guys will find it helpful to begin with storage.

 ######################################################################

Fibre Channel (FC)
==================

> FC is a Protocol Stack primarily used to send SCSI commands over the SAN. But we can also run IP/ARP over FC

# FC Suppported topologies

1. P2P (FC-P2P)
2. Arbitrated Loop (FC-AL) similar to Token Ring
3. Switched Fabric (FC-SW) similar to Ethernet

# FC Port Types

1. Node Port AKA N Port - End host (Target or initiator) in P2P or Switched Fabric
2. Node Loop Port or NL Port - Similar to N port but in Arbitrated Loop Topology e.g. > JBOD Connectivity
3. Fabric Port or F Port - Switch port connecting to Node Port
4. Fabric Loop or FL Port - Switch port connecting to Node Loop Port
5. Expansion Port or E Port aka ISL = Link between two switches
6. Trunk Expension Port or TE Port = Link between two switches to carry VSANs similar to DOT1Q trunk in LAN


# Fibre Channel Addressing

1. Physical Address = World Wide Names (WWNs) is equal to MAC in Ethernet = 8 Bytes
2. Logical Address = FC Identifier (FC ID) is equal to IP address = 3 Bytes

* WWN is subdivided into two parts

1. World Wide Node Name (WWNN)

- Switch, Server or Disk's Physical Address

2. World Wide Port Name (WWPN)

- Switch, Server or Disk's Port's Physical Address

E.g. > A HBA with multiple interfaces. HBA will have one WWNN and Each HBA port will have it's own WWPN

Note - WWN is not used in Data Plane

* FCID is subdivided into three fields

1. Domain ID

- Each switch gets a separate domain id

2. Area ID

- Group of ports on switch have an area id

3. Port ID

- End station connected to switch gets a Port ID

> Domain ID is automatically assigned by Principal Switch but can also be manually assigned. Principal switch is analogous to STP root bridge in Ethernet world.


# Fabric Shortest Path First (FSPF) is protocol in SAN used to route traffic between 
  Switches based on FCID's Domain ID

FSPF support ECMP and used Dijkstra Algorithm like OSPF & ISIS and by default enabled on SAN Switch as Service

# FC supports SAN Port Channel using Port Channel Protocol (PCP) roghly analogous to 802.3ad

# In Fibre Channel network all end hosts must first register with the control plane of fabric before sending any traffic

# Fabric Registration has three parts

1. Fabric Login (FLOGI)
2. Port Login (PLOGI)
3. Process Login (PLRI)

FLOGI
++++++

> During FLOGI, the N Port tells switch's F port that it wants to register

> Once Switch gets Flogi Request, it learns the WWNN and WWPN of node

> Later switch assigns FCID to node


PLOGI
+++++

> End to End Login between N Ports

> Initiator (End Device such as server) tells target (Storage Array) that it wants to talk

> Used for applications such as End To End flow control


PLRI
+++++

> Upper Layer protocol login negotiation between N ports


# Fiber Channel Name Server (FCNS) is analogous to ARP cache and helps in FLOGI.


> FCNS is used to resolve WWN to FCID


sh fcns database


sh flogi database



ZONING
======

> Zoning controls which initiators talks to which targets and alalogous to ACL in IP World

- Associates WWNs, FCIDs, aliases etc to control who can talk to whom



Virtual SANs (VSANs)
====================

> VSANs are roughly analogous to VLANs in Ethernet world

> VSANs are used to create small SAN separations within large SAN. Also can be used to isolate the management and failure domain of the network

> Creating a VSAN creates a separate/new set of Fabric Services like FLOGI, FCNS, ZONING, Aliases etc

> Normally we don't create more than two VSANs

> With VSANs, E Port now becomes TE port


HTH...
Deepak Arora
Evil CCIE