An Engineer by Heart !!!
A Dreamer, A Pioneer, A Blogger.
A Network Engineer Trying to overtake the world with his network engineering skills :)
Opinions expressed here are solely my own and do not express the views or opinions of my Present or Past employer.
R8#sh ip int b | e una|do Interface IP-Address OK? Method Status Protocol FastEthernet0/0 89.0.0.8 YES NVRAM up up Loopback0 8.8.8.8 YES NVRAM up up
R8#sh run | s r o router ospf 1 log-adjacency-changes network 0.0.0.0 255.255.255.255 area 0
R9#sh ip int b | e una|do Interface IP-Address OK? Method Status Protocol Serial1/0 89.0.0.9 YES NVRAM up up Loopback0 9.9.9.9 YES NVRAM up up
R9#sh run | s r o router ospf 1 log-adjacency-changes network 0.0.0.0 255.255.255.255 area 0
R9#sh run int s1/0 Building configuration...
Current configuration : 132 bytes ! interface Serial1/0 ip address 89.0.0.9 255.255.255.0 encapsulation ppp ip ospf network broadcast serial restart-delay 0 end
R2#sh ip int b | e una|do Interface IP-Address OK? Method Status Protocol FastEthernet0/0 12.0.0.2 YES NVRAM up up FastEthernet0/1 23.0.0.2 YES NVRAM up up Loopback0 2.2.2.2 YES NVRAM up up
R2#sh run | s r e router eigrp 100 network 0.0.0.0 no auto-summary
R2#sh ip ro e 1.0.0.0/32 is subnetted, 1 subnets D 1.1.1.1 [90/409600] via 12.0.0.1, 00:10:30, FastEthernet0/0 3.0.0.0/32 is subnetted, 1 subnets D 3.3.3.3 [90/409600] via 23.0.0.3, 00:10:10, FastEthernet0/1
R2#sh mpls forwarding Tag switching is not operational. CEF or tag switching has not been enabled. No TFIB currently allocated.
R1#sh run | s pseudo pseudowire-class L2TPv3 encapsulation l2tpv3 interworking ip ip local interface Loopback0
R1#sh run int f2/1 Building configuration...
Current configuration : 124 bytes ! interface FastEthernet2/1 no ip address speed auto duplex auto no keepalive xconnect 3.3.3.3 13 pw-class L2TPv3 end
R3#sh run | s pseudo pseudowire-class L2TPv3 encapsulation l2tpv3 interworking ip ip local interface Loopback0
R3#sh run int s1/1 Building configuration...
Current configuration : 122 bytes ! interface Serial1/1 no ip address encapsulation ppp serial restart-delay 0 xconnect 1.1.1.1 13 pw-class L2TPv3 end
R8#sh ip ospf nei
Neighbor ID Pri State Dead Time Address Interface 9.9.9.9 1 FULL/DR 00:00:37 89.0.0.9 FastEthernet0/0
R8#sh ip ro os 9.0.0.0/32 is subnetted, 1 subnets O 9.9.9.9 [110/11] via 89.0.0.9, 00:13:00, FastEthernet0/0
R8#ping 9.9.9.9 so lo0
Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 9.9.9.9, timeout is 2 seconds: Packet sent with a source address of 8.8.8.8 .!!!! Success rate is 80 percent (4/5), round-trip min/avg/max = 88/120/180 ms
R1#sh l2tp tunnel all
L2TP Tunnel Information Total tunnels 1 sessions 1
Tunnel id 1100418086 is up, remote id is 2518749472, 1 active sessions Remotely initiated tunnel Tunnel state is established, time since change 00:14:48 Tunnel transport is IP (115) Remote tunnel name is R3 Internet Address 3.3.3.3, port 0 Local tunnel name is R1 Internet Address 1.1.1.1, port 0 L2TP class for tunnel is l2tp_default_class Counters, taking last clear into account: 111 packets sent, 107 received 9008 bytes sent, 8752 received Last clearing of counters never Counters, ignoring last clear: 111 packets sent, 107 received 9008 bytes sent, 8752 received Control Ns 9, Nr 13 Local RWS 1024 (default), Remote RWS 1024 Control channel Congestion Control is disabled Tunnel PMTU checking disabled Retransmission time 1, max 1 seconds Unsent queuesize 0, max 0 Resend queuesize 0, max 1 Total resends 0, ZLB ACKs sent 11 Total out-of-order dropped pkts 0 Total out-of-order reorder pkts 0 Total peer authentication failures 0 Current no session pak queue check 0 of 5 Retransmit time distribution: 0 0 0 0 0 0 0 0 0 Control message authentication is disabled
R1#sh l2tun session all
L2TP Session Information Total tunnels 1 sessions 1
Session id 3278375310 is up, logical session id 65537, tunnel id 1100418086 Remote session id is 1558878854, remote tunnel id 2518749472 Remotely initiated session Unique ID is 1 Session Layer 2 circuit, type is Ethernet, name is FastEthernet2/1 Session vcid is 13 Interworking type is IP Circuit state is UP Local circuit state is UP Remote circuit state is UP Call serial number is 1168000001 Remote tunnel name is R3 Internet address is 3.3.3.3 Local tunnel name is R1 Internet address is 1.1.1.1 IP protocol 115 Session is L2TP signaled Session state is established, time since change 00:15:43 117 Packets sent, 113 received 9488 Bytes sent, 9232 received Last clearing of counters never Counters, ignoring last clear: 117 Packets sent, 113 received 9488 Bytes sent, 9232 received Receive packets dropped: out-of-order: 0 other: 0 total: 0 Send packets dropped: exceeded session MTU: 0 other: 0 total: 0 DF bit off, ToS reflect disabled, ToS value 0, TTL value 255 Sending UDP checksums are disabled Received UDP checksums are verified No session cookie information available FS cached header information: encap size = 24 bytes 45000014 00000000 ff73b36f 01010101 03030303 5cea9a86 Sequencing is off Conditional debugging is disabled SSM switch id is 4096, SSM segment id is 8193
Node Port Virtualization & Node Port ID Virtualization (NPV & NPIV) +++++++++++++++++++++++++++++++++++++++++++++++++ - FCID is a 3 byte field with Domain ID as first byte - Fibre Channel forwarding is based on FCID - Domain ID is used to identify the Switch in the Fabric's SPT - It implies that hard limit of switches per fabric is 256 - Some IDs are reserved so only 239 are usable but Qualified limit by OSMs (Original Storage Manufacturer) is approx 50 - NPV fixes the Domain ID problem by removing the need for a switch to participate in Fabric Services > I.e. no FSPF, FCNS, Zoning etc - Switches running NPV appears to the rest of the fabric as an end host I.e. a Node Port (N_Port) - Upstream facing link on the NPV switch is called NP_Port AKA Proxy Node Port FC Switch/NPV Core Swith (F_Port)----------(NP_Port) NPV Switch (F_Port)----------(N_Port) Initiator - Switch upstream of NPV switch is the NPV core switch - NPV Core switch runs NPIV - NPIV allows multiple FLOGIs and FCID assignments on its F Port facing downstream ====================================================== NPV/NPIV Configuration +++++++++++++++++++++++ - Enable NPV on NPV Switch (Downstream Switch) > feature fcoe > feature npv # After enabling feature NPV,Switch would require reload. Also most of older config including Data Plane will be erased # On 5500 UP, reallocate ports as FC after first reload which would again require second reload - Now configure NP Ports on NPV Switch > switchport mode np - Configure F Ports on NPV Switch (Facing Initiator) / NPIV Switch (Facing NPV Switch) > switchport mode f - ENable NPIV on the Core Swith > feature npiv # sh npv flogi-table ( To check Initiators flogi on NPV Switch ) # sh npv external-interface-usage (To check F port to NP Port Mapping on NPV SW to verify static pinning distribution) - Zoning to be configured on NPIV Switch HTH... Deepak Arora Evil CCIE
Fiber Channel Over IP (FCIP) ++++++++++++++++++++++++++++ - SCSI over FCP over TCP over IP - Same protocol stack as fiber channel - Initiator and Targets are still Native FC (or FCOE) - Used for FC SAN Extension like SAN replication over DCI (Data Center Interconnect), for example running FCIP over OTV will be - SCSI over FCP over TCP over IP over Ethernet over MPLS over GRE over IP over Ethernet =============================================================== MDS FCIP Gateway Configuration +++++++++++++++++++++++++++++++ - Configure normal FC to initiators & targets - Configure IP Connectivity between MDSes - Configure FCIP Tunnel - FCIP tunnel now counts as a TE port _ Normal FC Switching design now applies - FCIP only supported on MDS ================================================================ FCIP Configuration Example ++++++++++++++++++++++++++ ! feature fcip ! fcip profile 10 ip add 1.1.1.1 exit ! inteface fcip 12 use-profile 10 peer-info ipadd 1.1.1.2 no shut exit ! sh fcip summary sh int fcip 12 brief sh int fcip12 trunk vsan sh fcip profile HTH... Deepak Arora Evil CCIE
+ iSCSI ( Internet Small Computer System Interface ) #################################################### - Completely separate protocol stack from Fibre channel - Typically used in small to mid range SANs - No dedicated SAN switches required which implies no SAN switching knowledge required - 1/10 GigE iSCSI hardware offload cards available - End host/Storage Array just runs IP - Transport supports IP and can be of any IP Transport type like Ethernet, Frame Relay or Token Ring - MDS is an iSCSI to FC Gateway, so MDS is a translational bridge for Fiber Channel & iSCSI ===================================== + iSCSI Gateway Operation ######################## - FC Targets FLOGI to FC Fabric - iSCSI initiators send Discovery to MDS using Ethernet for example - MDS applies zoning/Access Lists - iSCSI initiator things FC target is iSCSI target - FC Target thinks iSCSI initiator is a FC initiator ===================================== + MDS iSCSI Gateway Configuration Steps ####################################### - Configure FC to Targets - Configure IP to Initiators - Enable iSCSI - Configure ZONING/ Access Control - Point Server at MDS's IP Address ===================================== + Access Control in iSCSI ########################## - Access Control can be enforced as Zoning based upon : > pwwn, fcid, alias > Initiator's IP Address > Initiator's iSCSI qualified name (IQN),IQNs are generated automatically by initiator but can be configured manually. > iSCI based virtual target to present LUN based on IQN or IP Address/Subnet ===================================== sh int fc1/20 trunk vsan feature iscsi iscsi enable module 1 int iscsi 1/1 no sh inc iscsi 1/2 no sh iscsi import target fc sh iscsi global sh iscsi initiator HTH... Deepak Arora Evil CCIE
- FC Aliases gives user friendly names to the WWNs, FCIDs etc and are analogous to DNS in IP
- Syntex - fcalias name
- Alias can be advertised through Zoneset distribution. Syntex - zoneset distribute vsan 1
- Example:
! fcalias name ABC vsan 30 member pwwn exit !
- Verification: sh fcalias
- Creating zone and Zoneset using ALIAS:
! zone name ABC vsan 30 member fcalias DISK1 member fcalias DISK2 member fcalias DISK3 member fcalias SERVER exit !
! zoneset name XYZ vsan 30 member ABC !
! zoneset activate name XYZ vsan 30 !
+++++++++++++++++++++++++++++++++++
Basic Vs Enhanced Zoning ########################
- By default the full zoneset is local and the active zoneset is fabric-wide
- Order of operations errors can corrupt the Active Zoneset
- Enchanced zoning prevents this by Locking the fabric which ensures people don't accidentally overwrite each other
- Enhanced Zoning works on per VSAN basis
- The lock on zone is released by "committing" the zoneset
- But if Admin forgets to commit, someone else can release lock as well with "clear zone lock vsan "
- Enabling Enhanced Zoning
> zone mode enhanced vsan < Per VSAN Basis
> system default zone mode enhanced < For Entire System
- Configuring Enhanced Zoning doesn't change anything for regular FCALIAS and they still remain local by default
- To solve this problem solution is "Device Aliases"
- Device Aliases serves the same purpose as FC Aliases by binding PWWNs to a user friendly name but the difference
is that the binding is advertised to the fabric and doesn't remain local only
- Configured as device-alias database
- Changing the zoning mode from normal to enhanced is not disruptive
- Device aliases are advertised through CFS so should not be used in multi vendor environment
+ Configured as "device-alias database"
+ device-alias commit
E.g.>
device-alias database device-alias name ABC_Server pwwn device-alias name XYZ_Disk1 pwwn device-alias name XYZ_Disk2 pwwn device-alias name XYZ_Disk3 pwwn device-alias commit
show device-alias database
zone name ABC vsan member device-alias ABC_Server member device-alias XYZ_Disk1 member device-alias XYZ_Disk2
zoneset name XYZ vsan member ABC exit
zoneset activate name XYZ vsan
sh zoneset pending
zone commit vsan
sh zone active
"sh port-resource module " shows port grouping on MDS (shared vs dedicaed mode), in 5k all ports are in dedicated mode by default.
