Address Resolution Protocol - ARP

Often I have seen CCNA Candidates struggling with understanding ARP & Proxy ARP. So I always recommend such guys two things :

1. Read through the following document:

http://www.cisco.com/en/US/docs/ios/ipaddr/configuration/guide/iad_cfg_arp_ps6350_TSD_Products_Configuration_Guide_Chapter.html

2. Lab up ARP and Inverse ARP on real equipments and verify the operation using debugs.


HTH...
Deepak Arora
Evil CCIE

Network Design Challenge

PHYSICAL TOPOLOGY

       LOGICAL TOPOLOGY

Recently one of the customer I am working for came up with an interesting requirement.

Current customer setup look alike something similar to topology (logical) given above. Customer has two locations per say connected through back to back T1 Circuits. In current setup R1-R3 is the primary path, and if a failure occurs in that case traffic should fall back to backup path using R2-R4. Customer is using One HSRP group on each side for auto failover purpose and currently static routing is in place.
 
So considering all this, following is how over all traffic flow looks like at the moment from A to C and from B to C perspective:
 

Hmmm... sounds like a pretty flat and easy network... isn't it ? :-)


Now here is the new requirement: As you guys can see we are wasting our bandwidth here on the backup circuit between R2-R4. And Bandwidth in our real life means $$$$$$ :-)

So here is new desired flow: When Host A talks to Host C, It should use Path through R1-R3. At the same time if Host B talks to Host C, It should use Path through R2-R4. At the same time both links should be redundant for each other for all traffic flow. So based on this requirement , the traffic flow needs to look like following:

As you can see , we need to account 8 potential FAILURE DOMAINS to ensure our fail-over scenarios works just fine. 

Also requirement is to ensure we don't have any asymmetric routing into the network. And of course NO dynamic routing protocol should be introduced.

So how you gonna do this ? :-) , since from very high level perspective it looks very easy. But it ain't :-)

            
Current Configuration
++++++++++++++                                                                                                                        
SW-A
=====
!
en
!
conf t
!
ho SW-A
!
no ip do lo
!
no cdp run
!
ip routing
!
int f1/1
 sw mo acc
 sw acc vl 10
 exit
!
int vlan 10
 ip add 10.0.0.1 255.255.255.0
 no sh
 exit
!
ip route 20.0.0.0 255.255.255.0 12.0.0.1
ip route 30.0.0.0 255.255.255.0 12.0.0.1
!
int f1/0
 no switchport
 ip add 12.0.0.2 255.255.255.0
 no sh
 exit
!
end

!


SW-B
=====
!
en
!
conf t
!
ho SW-B
!
no ip do lo
!
no cdp run
!
ip routing
!
int f1/0
 no switchport
 ip add 34.0.0.2 255.255.255.0
 no sh
 exit
!
int f1/1
 sw mode access
 sw acc vl 20
 exit
!
int f1/2
 sw mode access
 sw acc vl 30
 exit
!
int vlan 20
 ip add 20.0.0.1 255.255.255.0
 no sh
 exit
!
int vlan 30
 ip add 30.0.0.1 255.255.255.0
 no sh
 exit
!
ip route 0.0.0.0 0.0.0.0 34.0.0.1
!
end
!

R1
====
!
en
!
conf t
!
ho R1
!
no ip do lo
!
int f0/0
 ip add 12.0.0.111 255.255.255.0
 standby 1 ip 12.0.0.1
 standby 1 pri 105
 standby 1 pree
 standby 1 track s1/0
 exit
!
int s1/0
 ip add 13.0.0.1 255.255.255.0
 encap ppp
 no sh
 exit
!
ip route 20.0.0.0 255.255.255.0 13.0.0.3
ip route 30.0.0.0 255.255.255.0 13.0.0.3
ip route 0.0.0.0 0.0.0.0 13.0.0.3
!
ip route 10.0.0.0 255.255.255.0 12.0.0.2
!
end
!


R2
===
!
en
!
conf t
!
ho R2
!
no ip do lo
!
no cdp run
!
int f0/0
 ip add 12.0.0.112 255.255.255.0
 standby 1 ip 12.0.0.1
 standby 1 pree
 no sh
 exit
!
int s1/0
 ip add 24.0.0.2 255.255.255.0
 encap ppp
 no sh
 exit
!
ip route 20.0.0.0 255.255.255.0 24.0.0.4
ip route 30.0.0.0 255.255.255.0 24.0.0.4
ip route 0.0.0.0 0.0.0.0 24.0.0.4
!
ip route 10.0.0.0 255.255.255.0 12.0.0.2
!
end
!

R3
===
!
en
!
conf t
!
ho R3
!
no ip do lo
!
no cdp run
!
int f0/0
 ip add 34.0.0.3 255.255.255.0
 standby 1 ip 34.0.0.1
 standby 1 pri 105
 standby 1 pree
 standby 1 track s1/0
 no sh
 exit
!
int s1/0
 ip add 13.0.0.3 255.255.255.0
 encap ppp
 no sh
 exit
!
ip route 10.0.0.0 255.255.255.0 13.0.0.1
ip route 0.0.0.0 0.0.0.0 13.0.0.1
!
ip route 20.0.0.0 255.255.255.0 34.0.0.2
ip route 30.0.0.0 255.255.255.0 34.0.0.2
!
end
!
 

R4
===
!
en
!
conf t
!
ho R4
!
no ip do lo
!
no cdp run
!
int f0/0
 ip add 34.0.0.4 255.255.255.0
 standby 1 ip 34.0.0.1
 standby 1 pree
 no sh
 exit
!
int s1/0
 encap ppp
 ip add 24.0.0.4 255.255.255.0
 no sh
 exit
!
ip route 10.0.0.0 255.255.255.0 24.0.0.2
ip route 0.0.0.0 0.0.0.0 24.0.0.2
!
ip route 20.0.0.0 255.255.255.0 34.0.0.2
ip route 30.0.0.0 255.255.255.0 34.0.0.2
!
end
!

So let's see who among you is a Real Network Warrior  :-)

Final Config

R1
====

!
en
!
conf t
!
ho R1
!
no ip do lo
!
ip sla 2
 icmp-echo 34.0.0.3 source-interface f0/0
 freq 5
 timeout 5000
 exit
!
ip sla schedule 2 life forever start-time now
!
track 2 rtr 2 reachability
!
int f0/0
 ip add 12.0.0.111 255.255.255.0
 standby 1 ip 12.0.0.1
 standby 1 pri 105
 standby 1 pree
 standby 1 track s1/0
 standby 1 track 2
 standby 2 ip 12.0.0.11
 standby 2 pree
 no sh
 exit
!
int s1/0
 ip add 13.0.0.1 255.255.255.0
 encap ppp
 no sh
 exit
!
ip route 20.0.0.0 255.255.255.0 13.0.0.3
ip route 0.0.0.0 0.0.0.0 13.0.0.3
!
ip route 10.0.0.0 255.255.255.0 12.0.0.2
!
end
!





R2
===

!
en
!
conf t
!
ho R2
!
no ip do lo
!
no cdp run
!
ip sla 1
 icmp-echo 34.0.0.4 source-interface s1/0
 time 500
 freq 3
 exit
!
ip sla schedule 1 life forever start-time now
!
track 2 rtr 1 reachability
!
int f0/0
 ip add 12.0.0.112 255.255.255.0
 standby 1 ip 12.0.0.1
 standby 1 pree
 standby 2 ip 12.0.0.11
 standby 2 pri 105
 standby 2 pree
 standby 2 track 2
 no sh
 exit
!
int s1/0
 ip add 24.0.0.2 255.255.255.0
 encap ppp
 no sh
 exit
!
ip route 30.0.0.0 255.255.255.0 24.0.0.4
ip route 0.0.0.0 0.0.0.0 24.0.0.4
!
ip route 10.0.0.0 255.255.255.0 12.0.0.2
!
end
!


SW-A
=====

!
en
!
conf t
!
ho SW-A
!
no ip do lo
!
no cdp run
!
ip routing
!
int f1/1
 sw mo acc
 sw acc vl 10
 exit
!
int vlan 10
 ip add 10.0.0.1 255.255.255.0
 no sh
 exit
!
ip route 20.0.0.0 255.255.255.0 12.0.0.1
ip route 30.0.0.0 255.255.255.0 12.0.0.11
!
int f1/0
 no switchport
 ip add 12.0.0.2 255.255.255.0
 no sh
 exit
!
end
!



R3
===


!
en
!
conf t
!
ho R3
!
no ip do lo
!
no cdp run
!
int f0/0
 ip add 34.0.0.3 255.255.255.0
 standby 1 ip 34.0.0.1
 standby 1 pri 105
 standby 1 pree
 standby 1 track s1/0
 no sh
 exit
!
int s1/0
 ip add 13.0.0.3 255.255.255.0
 encap ppp
 no sh
 exit
!
ip route 10.0.0.0 255.255.255.0 13.0.0.1
ip route 0.0.0.0 0.0.0.0 13.0.0.1
!
ip route 20.0.0.0 255.255.255.0 34.0.0.2
ip route 30.0.0.0 255.255.255.0 34.0.0.2
!
ip route 24.0.0.0 255.255.255.0 34.0.0.4
!
access-l 100 permit ip 30.0.0.0 0.255.255.255 10.0.0.0 0.255.255.255
!
ip sla 1
 icmp-echo 24.0.0.4 source-ip 34.0.0.3
 timeout 5000
 frequency 3
!
ip sla schedule 1 life forever start-time now
!
ip sla 3
 icmp-echo 12.0.0.112 source-interface FastEthernet0/0
 frequency 5
!
ip sla schedule 3 life forever start-time now
!
track 1 rtr 1 reachability
!
track 3 rtr 3 reachability
!
track 2 list boolean and
 object 1
 object 3
!
route-map PBR per 10
 match ip address 100
 set ip next-hop verify-availability 34.0.0.4 1 track 2
route-map PBR per 20
 exit
!
int f0/0
 ip policy route-map PBR
 no ip route-cache cef
 exit
!
ip sla 5
 icmp-echo 12.0.0.111 source-interface Serial1/0
 frequency 5
 timeoute 5000
!
ip sla schedule 5 life forever start-time now
!
track 5 rtr 5 reachability
!
int f0/0
 standby 1 track 5
 exit
!
end
!



R4
===

!
en
!
conf t
!
ho R4
!
no ip do lo
!
no cdp run
!
int f0/0
 ip add 34.0.0.4 255.255.255.0
 standby 1 ip 34.0.0.1
 standby 1 pree
 no sh
 exit
!
int s1/0
 encap ppp
 ip add 24.0.0.4 255.255.255.0
 no sh
 exit
!
ip route 10.0.0.0 255.255.255.0 24.0.0.2
ip route 0.0.0.0 0.0.0.0 24.0.0.2
!
ip route 20.0.0.0 255.255.255.0 34.0.0.2
ip route 30.0.0.0 255.255.255.0 34.0.0.2
!
end
!





SW-B
=====


!
en
!
conf t
!
ho SW-B
!
no ip do lo
!
no cdp run
!
ip routing
!
int f1/0
 no switchport
 ip add 34.0.0.2 255.255.255.0
 no sh
 exit
!
int f1/1
 sw mode access
 sw acc vl 20
 exit
!
int f1/2
 sw mode access
 sw acc vl 30
 exit
!
int vlan 20
 ip add 20.0.0.1 255.255.255.0
 no sh
 exit
!
int vlan 30
 ip add 30.0.0.1 255.255.255.0
 no sh
 exit
!
ip route 0.0.0.0 0.0.0.0 34.0.0.1
!
end
!

HTH...
DEEPAK ARORA
Evil CCIE

Cisco Fabric Extender AKA FEX - Nexus 2000 Series

Cisco NEXUS 2000 Series Fabric Extender (FEX)

In today’s post, we are going to dive into new Concept – Fabric Extender. If you are familiar with Traditional Modular Switches AKA Distributed Switching Architecture such as Catalyst 6500/6500 E series or 4500/4500 X series than Fabric Extender is going to be a piece of cake for you. In Distributed Switching Architecture we basically have following components:

Ø  Supervisor Engine – A supervisor engine is basically Mind of switch responsible for management & control plane management.

Ø  Line Card/ Modules - The End Hosts or other devices gets connected on Line Cards/Modules, the basic function of these is to take care of Data Plane/ Forwarding Plane.

Ø  Back Plane/ X-Bar Fabric - The Supervisor Engine/ Engines talk to line cards & Modules using High speed backplane circuitry.

Now in older Distributed Switching platforms all these components reside in a single Chassis. What Cisco did to take this concept a step further is came up with something known as Fabric Extender. A fabric extender basically comes as Cisco Nexus 2000 Series. A fabric extender works as a line card of its parent switch which could be either Nexus 7k or Nexus 5k.

The communication between Parent Switch and Fabric Extender happens through something called Fabric Ports. Let’s see how our new Design with Parent Switch and Fabric Extender Looks like:

So as you can see, the Fabric Extender (FEX) is now the device (Remote Line Cards) to which end hosts gets attached. The FEX further gets connected to Parent Switch which is essentially acting as supervisor engine. It’s the Supervisor Engine where entire configuration is saved which means FEX don’t save any of the configurations locally. Also it’s the parent switch where all forwarding decisions are made. Which means if HOST A on FEX 1 wants to talk to another HOST B connected on same FEX 1, the traffic has to go to parent Switch and sent back towards the FEX as shown in diagram below:

Which is obviously not an idle scenario though but this is the way FEX has been designed. But on the other hand introduction of FEX offers us couple of benefits as well:

   Ø  Reduce Cable Runs

   Ø  Reduces Management Point ( Since Parent Switch Controls all FEXs)

   Ø  Reduces IOS Management & Standardization Load (The FEX always Runs same NX-OS as Parent SW, FEX gets shipped with No NX-OS Image. When we connect FEX to Parent switch and provision it, it downloads the NX-OS from parent switch and uses it)

   Ø  Enables Parent Switch (7K/5K) to become a high density access layer switch

   Ø  STP Free Access Layer ( The fabric links between Parent Switch & FEX runs no STP, on the flip side FEX Host Interfaces(HIF) cannot be used to connect any further switches or devices running Spanning-Tree. BPDU Guard feature is by default enabled on HIF and cannot be disabled. Which means as soon as you plug any device to the FEX running STP, the port will get error disabled)

   Ø  QOS & Security Management from Parent Switch

   Ø  All Troubleshooting From Parent Switch
  Ø Less number of OOB connection

    Since in most of the real world designs you would want to Pair your FEX with Nexus 5K, so we will be discussing basic FEX configuration from Nexus 5K perspective:

   The official NX-OS version in CCIE DC lab for Nexus 5548 is 5.1(3) & FEX models available in lab will be 2232 & 2224.

   Let’s review the topology and relative configuration now:

  Step 1 > Enable FEX feature set on Nexus 5k :

                       5k(config)# feature fex

  Step 2 > Create a Port Channel :

                       5k(config)# interface port-channel 101

  Step 3 > Configure Port Channel to Act as Fabric Link:

                       5k(config-if)# switchport mode fex-fabric

  Step 4 > Assign Associate ID to Remote FEX (From range 101 – 199)

                       5k(config-if)# fex associate 101

  Step 5 > Map Fabric Interfaces to Port Channel:

                       5k(config)# interface e1/1 - 4

                       5k(config-if-range)# channel-group 101

    Once this configuration will be done, we should be ready use FEX. The associate ID gets prepended to FEX interface for identification. For example if FEX port were like : E1/1, E1/2…., After association ID 101 assigned to FEX, on parent switch the ports will now appear as: E101/1/1, E101/1/2. Now any configuration you apply under these ports will be pushed to FEX itself.

   Couple of verification commands:

   Ø  sh fex

   Ø  sh fex detail

   Ø  sh interface status fex  = sh interface status (in regular IOS)

       Couple of limitations with FEX :
  
   > FEX Doesn't support Private VLANs
   > Any port available on FEX cannot be SPAN destination port
   > FEX Model like 2148 Can't be associated with Nexus 7K
   > So far only FEX model 2232PP supports FCOE on FEX ports 
   > On Nexus 7k, the non-default VDC gets access to FEX feature only if  
          feature is installed and enabled under default VDC
   >  In Nexus 7-K all the uplinks and host ports of a Fabric Extender 
            belong to a single VDC. The ports cannot be allocated or split 
            among multiple VDCs.
   > Nexus 7k F1 modules are though Layer 2 cards, but still doesn't support 
           FEX
   > You can configure the Fabric Extender host interfaces as edge 
          ports only.The interface is placed in an error disabled state if a   
          downstream switch is detected.
   > The Cisco Nexus 2148 Fabric Extender does not support frames 
            with the dot1p vlan 0 tag


   
  And here is most interesting one. Just take a look at Pic below.

   
  Actually you are looking at Back (rear) side of Nexus 2000 FEX, 
   and default Air-flow is from front to back.

Further Readings:


  7k Fex Command Ref:

    http://www.cisco.com/en/US/docs/switches/datacenter/sw/6_x/nx-os/fex/command/reference/n7k_fex_cmd.html

 5k Fex Command Ref:

   http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/command/reference/fex/n5k-fex-cr.html


   2k NX-OS Software Rel 6.1 Configuration Guide:

  http://www.cisco.com/en/US/docs/switches/datacenter/nexus2000/sw/configuration/guide/rel_6_1/b_Cisco_Nexus_2000_Series_NX-OS_Fabric_Extender_Software_Configuration_Guide_Release_6.x.html

  
   2k Cisco Nexus 2000 NX-OS Software Rel 5.2 Configuration Guide:

  http://www.cisco.com/en/US/docs/switches/datacenter/nexus2000/sw/configuration/guide/rel_521/b_Configuring_the_Cisco_Nexus_2000_Series_Fabric_Extender_rel_5_2.html


    Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide, Release 5.2:

  http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/layer2/521_n1_1/b_Nexus_5000_Layer2_Config_521N11_chapter_010000.html

Nexus 2k Airflow:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps10110/product_bulletin_c25-680197.html

HTH...
Deepak Arora
Evil CCIE

Nexus Configuration Roll Back - Using Checkpoint Feature

While I have just started preparing for my CCIE DC. I must say I am in love with this track. Although majority of the track is focused on Cisco Specific Products - NEXUS Series, ACE, UCS etc, still there is lot of technology to learn working behind the scenes. 

So I have started my studies with Nexus. While there are couple of similarities between NX-OS and our old  buddy regular IOS. There are many major differences as well.

So today we are going to talk about a relatively small but important Nexus OS feature which should be very helpful to you in production setup. 

The feature is called CHECKPOINT. Essentially the feature is somewhat similar to our regular IOS feaure known as "Configure Replace". But NX-OS has taken the feature a step further in order to allow us more flexiability.

Basically CHECKPOINT feature allows us to save a snapshot of configuration at any given time (Though Per VDC we can have at max 10 checkpoints). At any given time if we feel there are too many mistakes we have made. Using Checkpoint we can revert to last snapshot of configuration very quickly. This is certainly a nice feature from Cisco or are we just following Microsoft now ;-)

So let's hop on to Nexus CLI and perform some basic testing in order to completely understand the feature:

N7K11-pod3# sh ver | i kick|system|Nexus
Cisco Nexus Operating System (NX-OS) Software
  kickstart: version 5.2(4)
  system:    version 5.2(4)
  kickstart image file is: bootflash:///n7000-s1-kickstart.5.2.4.bin
  kickstart compile time:  12/25/2020 12:00:00 [03/08/2012 03:58:13]
  system image file is:    bootflash:///n7000-s1-dk9.5.2.4.bin
  system compile time:     1/28/2012 23:00:00 [03/08/2012 04:59:07]
  cisco Nexus7000 C7010 (10 Slot) Chassis ("Supervisor module-1X")

Step 1. Create a Checkpoint

N7K11-pod3# checkpoint

user-checkpoint-1 created Successfully

Done


N7K11-pod3# sh checkpoint
--------------------------------------------------------------------------------
Name: user-checkpoint-1



N7K11-pod3# sh checkpoint summary
User Checkpoint Summary
--------------------------------------------------------------------------------
1) user-checkpoint-1:
Created by admin
Created at Tue, 11:04:46 02 Oct 2012
Size is 5,102 bytes
Description: None


Step 2. Check the difference between Checkpoint Snapshot Config & Running Config

N7K11-pod3# sh diff rollback-patch checkpoint user-checkpoint-1 running-config
Collecting Running-Config
#Generating Rollback Patch
Rollback Patch is Empty

As you guys can see , currently rollback patch is empty. Which mean Checkpoint config is same as current running-configuration. Let's make some changes now in current running configuration in order to make difference between two.

N7K11-pod3(config)# username deepak password 0 cisco role priv-15
N7K11-pod3(config)# end


N7K11-pod3# sh diff rollback-patch checkpoint user-checkpoint-1 running-config
Collecting Running-Config
#Generating Rollback Patch

!!
!
username deepak password 5 $1$dVPagLUf$RFkUQOcY2F4gb8YisS8vn1 role priv-15
snmp-server user deepak priv-15 auth md5 0x3d98e4cead6825e6669d98dc60209308 priv 0x3d98e4
cead6825e6669d98dc60209308 localizedkey

To Delete old checkpoint configuration snapshot, we can run following command:

N7K11-pod3# clear checkpoint database
...............Done


N7K11-pod3# sh checkpoint summary
N7K11-pod3#

Let's create a fresh checkpoint again:

N7K11-pod3# checkpoint

user-checkpoint-2 created Successfully

Done



N7K11-pod3# sh checkpoint summary
User Checkpoint Summary
--------------------------------------------------------------------------------
1) user-checkpoint-2:
Created by admin
Created at Tue, 11:11:12 02 Oct 2012
Size is 5,307 bytes
Description: None


N7K11-pod3# sh user-account
user:admin
        this user account has no expiry date
        roles:vdc-admin
user:deepak
        this user account has no expiry date
        roles:priv-15
Let's delete the user and try to get back using Checkpoint Snapshot:

N7K11-pod3# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
N7K11-pod3(config)# no username deepak password 0 cisco role priv-15
N7K11-pod3(config)# end


N7K11-pod3# sh user-account
user:admin
        this user account has no expiry date
        roles:vdc-admin


N7K11-pod3# rollback running-config checkpoint user-checkpoint-2
Note: Applying config parallelly may fail Rollback verification
Collecting Running-Config
#Generating Rollback Patch
Executing Rollback Patch
Generating Running-config for verification
Generating Patch for verification
Rollback Patch is Empty

Rollback completed successfully.


N7K11-pod3# sh user-account
user:admin
        this user account has no expiry date
        roles:vdc-admin
user:deepak
        this user account has no expiry date
        roles:priv-15



N7K11-pod3# sh rollback log verify
Operation            : Rollback to Checkpoint
Checkpoint name      : user-checkpoint-2
Rollback done By     : admin
Rollback mode        : atomic
Verbose              : disabled
Start Time           : Tue, 11:16:48 02 Oct 2012

Further Readings:

http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/system_management/configuration/guide/sm_7rollback.html

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/system_management/502_n1_1/Cisco_n5k_system_mgmt_cg_rel_502_n1_1_chapter11.html


HTH...
Deepak Arora
Evil CCIE