A Network Artist : CCDA

Showing posts with label CCDA. Show all posts

Friday, June 23, 2023

BGPv4 Mindmap

HTH...

A Tech Artist 🎨

Sunday, December 20, 2020

BGP Nuts - The Crazy Metric Story

Recently ran into an interesting BGP behavior. In order to convey the issue, I have oversimplified the design.

R2 (AS 100) has a network host 22.22.22.22/32 which is trying to reach R7 (AS 200) attached network host 77.77.77.77/32.

As you can see, between AS 100 & AS 200 we have multiple exit points for the purpose of redundancy. However the interesting twist here is that those BGP next hops are learned from different IGPs.

54.0.0.5 <- Learned from EIGRP

63.0.0.6 <- Learned from OSPF

Given the scenario, what you think about which BGP route and next hop would make into the routing table ?

Well...most likely the answer would be BGP next hop learned from EIGRP because (Assuming... 1. " no auto-summary " is now default for EIGRP (resulting into same Prefix Length for route) , 2. EIGRP AD is 90 compare to OSPF Intra Area route which has AD 110. So based on theory how Router Process this information, the EIGRP learned Next hop (54.0.0.5) should be the one picked up by BGP Best Path Selection Algorithm and should be picked up as the best route over OSPF learned Next Hop (63.0.0.6)

All good ? ... Let's look at the BGP RIB to find it out and How IGP next hops are processed based on standard BGP Next Hop Processing approach that we know from standard BGP theory. Also let's run a quick traceroute towards 77.77.77.77 sourcing from 22.22.22.22 on R2

Now sure if you expected that...right ? :)

From BGP RIB standpoint we can clearly see that BGP route that's making into routing table & FIB essentially is the one that has OSPF learned BGP next hop.

Here is what our OSPF database looks like

Let's do a basic failover test by shutting down interface g6/0 on R3

So things works fine and as expected here, so let's "un-shut" the interface in order to dig deeper into the behavior ( a little unexpected one)

So why this behavior ?...Well this is where BGP plays a little dumb. It actually ends up comparing METRICs across two IGPs since it doesn't have view of EIGRP Topology Table and OSPF Link State Database. Now if you go back and check IGP metrics for next hops learned from EIGRP vs. OSPF, The metric for EIGRP route is pretty high (in numeric ) as 3072 vs. OSPF route as 2 (Refer to first CLI screenshot in the post)

Now what if we make the OSPF metric look bigger than EIGRP by manually increasing OSPF cost to see if that would allow R2 to install EIGRP learned route as best route

And Yes it works as expected this time too :)

Now for fun let's make the metrics look same by again changing the OSPF cost manually

Well that works too as expected. Now interesting question here is - Since IGP metrics learned from OSPF and EIGRP look same, can we turn ON " i-BGP Multipath " and achieve load balancing.

Let's try that

Well that works well too and as expected.

One of the problem incase you haven't noticed yet with this design is - Assume the path through OSPF has many more hops added before you reach to the OSPF learned BGP ASN exit point. Since OSPF cost metrics are much lower value compare to EIGRP, You will end up taking sub-optimal path from R2's standpoint in this case which may not be desired behavior. Alternatively You can replace EIGRP with IS-IS and you will by default still end up following OSPF learned BGP next since in most IOS version 10 is the default IS-IS metric for each interface hop. Obviously in practice you should try to run Single IGP across AS 100 here to avoid such issues though M&A scearios are always interesting and challenging. :)

Try this with " DMZ Link-BW " for Unequal Cost Multi-Path and I am pretty sure it will be fun. BGP AIGP NLRI is another interesting bit if you care depending upon the design.

And of course, the above mentioned logic doesn't apply to following design where we have single exit point being reachable through both IGPs

To learn some more around BGP anomalies and somewhat un-predictable behavior:

BGP Wedgies

BGP Anomalies

And if You want to master BGP from Design Standpoint, I would highly recommend " BGP Zero to Hero Design Masterclass " from my friend Orhan Ergun

HTH...

A Network Artist

Thursday, April 23, 2020

Do You Really Need VxLAN in a Modern Data Center ?

It started as a thread on Linkedin yesterday when someone posted a sort of glimpse of his free Cisco ACI training on linkedin. Later a Network Architect joined the conversation thread by asking the Original poster "Would you prefer ACI or BGP EVPN VxLAN based standard fabric ?". Of course the Architect suggested he would prefer BGP EVPN VxLAN fabric (That's mouthful man :) )

As usual out of my curiosity (without being a DCN expert - Yes I only worked on 3 ACI projects and Zero Cisco Nexus 9K BGP EVPN VxLAN fabric (VTS) so far), I asked this self proclaimed Network Architect another question...very basic in my view:

In modern Data Center context, Where in VxLAN actually plays an important role. Or is it just another snowflake in Networking realm in order to keep your App guys happy since you are not good at Saying " NO " and push them to build better Apps those works well on L3 vs. Sure guys, I can help you with L2 stretches for your App Clusters and for those totally broken old legacy Apps which only works on L2 best.

The point being:

Most modern App works well on L3 in conjunction with DNS
From Cloud Native Apps perspective, L2 shouldn't happen to begin with
vMotion works well on L3 since 6.x release
Inter DC Cluster using L2 are well known bad design practices (Some people tried it with even firewalls and failed miserably)
At least in my personal view, Most Enterprises still run MLAG or vPC thingy for server redundancy compare to perfect world (Original Hyperscaler ideas) where you either run Routing with Servers and VxLAN boundary starts from Hypervisor layer to avoid broken MLAG and STP implementations
Let's agree on L3 based transport is much better compare to L2
Perhaps we have seen enough DC meltdowns in past due to broken L2 and broadcast loops by now ?
Did you ever notice that most Hyperscalers DCs don't run VxLAN in their DC fabrics (Pick your favorite one) - https://engineering.linkedin.com/blog/2016/03/project-altair--the-evolution-of-linkedins-data-center-network
Perhaps there are good reasons someone like AWS and AZURE don't let you run VxLAN between your On Prem DC and Virtual DC they host (VMware on AWS is a corner case and rather a joint GTM thingy)
How many of your Use Case requirements are persistent vs. temp in nature ( Migration and Maintenance were always temp requirements in my view at least)
What value Bridging over L3 really has today to begin with
Shouldn't we rather focus on solving real problems instead of pretending
Don't forget you can't have unlimited VxLANs in real world like most vendor slides tell you otherwise
Focus on your failure domains/blast radius and create a strategy to mitigate that - Which doesn't seems easy in VxLAN environment and more so in Controller based Clos Fabric (Single Plainer) running VxLAN
And since Overlays such as VxLAN seems to be the new norm in Enterprise DCs, Make sure you are familiar with some of well known Overlay Problems besides challenges such as Visibility, Security and so forth
Make sure you fully understand the VxLAN convergence process with EVPN added while understanding your dependencies on Physical Topology, Underlay Convergence and other nuances
Make sure you understand Leaky Abstractions & Grey Failures very well in context of VxLAN deployment
Understand some myths about Clos Fabric that your favorite vendor otherwise won't tell you - Part 1, Part 2, Part 3 & Fact Check

And of course I am totally ignoring:

Level of complexity VxLAN and EVPN brings into Design...You either got to read through few books to become an SME or your must be Genius
NSX over ACI never took off ... sort of
Most VxLAN implementations are still vendor specific and ACI runs modified VxLAN header
VxLAN & EVPN across multi vendors in Single environment... Good luck (Though things have improved a bit over time)
MPLS & SR in DCN Fabric...Perhaps needs a separate discussion
I am not aware of any strong one to one mapping in context of Overlay and Underlay correlation for simplified Day 2 and operations
VxLAN vs Geneve (What NSX-T uses now) vs STT vs NvGRE (Assuming you love MS) vs Dove....Needs separate discussion
This is not a scalability discussion but more around to find real world use cases for VxLAN
MS Fabric today (Not Originally) does run VxLAN since 2015. Spoke to few MS friends, seems like Bing was Broken from APP designs perspective so they had to do it in order to Keep Business and App guys happy....BTW who runs Bing anyway :)
If you are running HW VTEP with MLAG for servers. Perhaps you got to relook into your design and define what purpose VxLAN really serves
We still got plenty of other HW and SW in DC context that don't understand VxLAN and more importantly have no HW acceleration
Role of VxLAN offload on NIC, DPDK, SR-IOV and other stuff....perhaps needs a separate discussion
HW and SW VTEP Integration and how broken those are...let's keep it for another day
IPv6 DC/IPv6 Only DC and VxLAN v6... Perhaps need another thread
Control Plane, Data Plane & Policy Plane Security in Context of VxLAN - Requires a separate discussion
VxLAN Control Plane, Data Plane & Policy Plane in Multi DC, Multi Cloud and Active-Active DC - Needs separate discussion
Control Plane & Data Plane Optimization in VxLAN Context - Broadcast Suppression, Conversational MAC address Learning, BUM traffic (Watch out for those details in given implementation

So question remains - What value really VxLAN brings in the context of modern DC. (Please don't tell me scale, let's keep it as a topic of debate for another day) ?

Saturday, April 18, 2015

OSPFv3 Gets Family

It's been a while since I touched Cisco CLI during Non Office Hrs. There are lots of things keeping me busy enough these days like I have been doing a Proxy relocation to DC project which means tons of stuff to deal with as DC is a fairly complex structure with lots of moving pieces like Load Balancer ( SLB, GSLB, LLB), Firewalls and above that this particular DC runs all Juniper Devices for Routing , Switching & Security stuff.

On the flip side I am thinking to seriously start working on CCDE from Mid May 2015 onwards with first lab attempt probably in Feb or May 2016. Though initially CCIE Wireless was on Radar for this and coming year but as Cisco has just announced new blueprint , the idea went on to backseat for a while now. Voice could have been interesting also but similarly CCNA and CCNP Collab tracks are going to replace existing NA and NP Voice track soon and moreover I haven't been doing much around Voice at work as well so should consider it some time next year.

Now back on to idea of today's post about OSPF Address Family Structure. Well it's been a while since Network Industry was talking about one of advantage of IS-IS over OSPF which is IS-IS supports IPv6 natively. For IS-IS it was just a matter of adding new TLV (Type Length Value) to existing IS-IS packet ( TLV 232 & 236 ? I guess). Where as with OSPF it was a different story. They actually had to write a new routing protocol right from scratch now known as OSPFv3. Which introduced another problem from network design and management standpoint that if we just want to run OSPF as choice of routing protocol for both IPv4 & IPv6 on network as administrator, we got to run two separate routing protocol instances as OSPFv2 for IPv4 and OSPFv3 for IPv6. Which means two separate routing topologies.

Now with OSPFv3 Address family (AF) structure support we get rid of this problem of running two separate routing instances somewhat. Why "somewhat" and not "completely" ?

Well while this AF structure solves the multiple instances requirement problem, it has it's own limitations like:

> Specific IOS version like 15.x required, which may or may not be supported across all existing routers & switches serving as Distribution block

> The AFv4 requires IPv6 Link Local address on all IPv4 interfaces in order to function, otherwise it doesn't allow you to bind OSPFv3 AFv4 instance on interface. The Link local addresses are used as source address of packet over interfaces connecting OSPF enabled devices

> The device with capability of AFv4 still couldn't talk to traditional OSPFv2 Device since there is a Address Family bit carried inside new Hello packets which traditional OSPFv2 device don't understand

Which means the deployment is more likely to be successful only if it's a Green Field deployment.

Here is a quick lab with quick results :)

Further Readings:

http://www.ciscopress.com/articles/article.asp?p=2294214&seqNum=4

http://www.cisco.com/c/en/us/products/collateral/ios-nx-os-software/open-shortest-path-first-ospf/whitepaper_c11-668030.html

http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_ospf/configuration/xe-3s/iro-xe-3s-book/ip6-route-ospfv3-add-fam-xe.html#GUID-10C6CAB4-C220-47A5-9DEA-F57319BCFFDA

HTH...

Deepak Arora

Evil CCIE

Wednesday, December 31, 2014

OSPF Path Selection With Default Information Originate - How You Gonna Fix It ? - Part 2

As OSPF recent design issue discussed here can possibly turn into obnoxious CCIEv5 R&S troubleshooting scenario, I got one email from another friend who also ran into interesting OSPF design issue recently around OSPF default information originate too.

Here is a quick topology of his network design:

At high level the design looks pretty simple. R1 & R2 are peering with WAN cloud using E-BGP. Between R1-R2-R3-R4-R5 we are running OSPF Area 0. To ensure R3-R4-R5 have reachability to external prefixes (Since it's always good to avoid BGP to IGP Redistribution), R1 & R2 are injecting default route into the network using "default-information originate always metric-type 1" option.

Also R1 & R2 are connected to each other over a link running OSPF.

Now let's talk about problem...which is the fun part :)

If R1's WAN connection goes down, R4 can't reach any external prefixes using default route any longer. Whereas at the same time both R3 & R5 don't lose their connectivity to same external prefixes.

Now ideally once R1 find it's WAN interface down, it should get default route from R2 and should be able to route R4's traffic in that direction.

Let me show you same behaviour using CLI :

Before R1's WAN Link goes down

After R1's WAN Link goes down

No ICMP Packets Received Any Longer on R6

R1 is not installing default route from R2

See if you can figure out " WHY " behind this and offer different ways we can fix this design issue. Feel Free to post you solution, recommendation and findings under comments section. I'll update the post around next weekend with my findings and solutions to the design.

HTH...

Deepak Arora

Evil CCIE

Update

In SPF Debug

Saturday, November 15, 2014

OSPF Path Selection With Default Information Originate - How You Gonna Fix It ? - Part 1

I recently got a call from friend , he was stuck with an interesting OSPF Path Selection issue. Obviously the things didn't work the way he expected it to be :)

Now initially he was been given a Consulting task that involved Design & Configuration changes to meet a new network requirement.

Here is how the network initially looked like :

The customer had active-active WAN connections where these connections were also serving both WAN as well as Internet connectivity. Where as in LAN side they were using OSPF as shown in Topology above.

Now the new requirements given were:

> Brought Up New Internet Connection that will act as Primary Path to reach internet.

> In case the New Internet connection goes down, the old WAN cloud should still work as backup path to reach internet

> From R4's perspective, the path towards R2 should be primary path to reach internet (Through New Internet GW) while path through R3 should be backup path. I am not sure why this was a Hard Requirement but I guess the path from R4 to R3 was through RF link as shown below:

> Introduction of Static Routes, PBR, NAT for whatever reasons were not allowed to be used to solve this problem

Now my friend tried to solve this problem in an interesting way. He Introduced RIPv2 as new IGP between R5-R3-R2 to ensure he doesn't have to change anything into existing IGP since there was a large IGP domain connected to this setup.

Using RIPv2 he injected default route into RIB of R2 and R3. He had to change distance of RIPv2 originated default route so that it can get priority over BGP injected default route. Than he tried Injecting Default Routes into OSPF domain towards R4 using "Default Information Originate " option. Since the control over path selection was required here, During default route injection he used Metric Type as 1 with Higher metric given while injection route at R3 as shown below:

Obviously the solution didn't work as expected and here is why:

As you can on R4 we still have two routes instead of one in Routing Table.

Can you figure out what's the problem here and how we can fix it in easiest way ?

Or You Have any other better solution to fix it while meeting the given restrictions ?

I'll post my solution to the problem after 2 weeks.

To be on same page here is the Addressing I Used in while labbing this up:

Here is relevant configuration:

Good Luck !!! :)

HTH...

Deepak Arora

Evil CCIE

Update For Ref: See comments section for further information.

HTH...

Deepak Arora

Evil CCIE

Monday, November 3, 2014

Does EIGRP Feasible Successor Always Work As Successor Fails ?

A recent discussion with friends brought this idea in my mind to write about this exiting subject. While I won't say all but many among them thought that it's always good to have EIGRP Feasible successors into Network while designing an EIGRP based network (Which of course is true ). And if EIGRP successor Route Ever fails, the EIGRP feasible successor will be installed quickly as it's a second best route in EIGRP Topology table based on feasibility condition, which will help minimizing convergence time.

And this is where most people start assuming that it's always going to be the case. Which of course is not true :)

Let's test this quickly based on following topology:

From R1's perspective it has three different paths to reach the destination - 5.5.5.5

Now let's review R1's routing table to first find which path is preferred.

As we can see, the middle path has been chosen as best path based on Dual Algorithm. Now let's next review R1's EIGRP topology table to figure out if we have feasible successor chosen at all and if So than which path.

As we can see, the path through R4 has been chosen as second best path (Feasible Successor). Since there is no other entry in Topology table showing path through R2, which means it's neither successor nor feasible successor and has failed feasibility condition.

Let's verify Feasible Distance (FD) and Reported Distance (RD) for path through R2 by shutting down the path through R3 and R4.

Now here is an interesting scenario:

> Path through R3 is the best path (Lowest Metric)

> Path through R4 is meeting feasibility condition (making it feasible successor). But overall cost to destination is worst if we compare all three paths metric.

> Path through R2 is actually second best path based on total metric but got out of the equation as it failed feasibility condition

Though we can see that topology doesn't include any potential link which can lead traffic back to original source while forwarding traffic towards destination 5.5.5.5 , but EIGRP fails to recognize this fact.

This is where we find this true that eventually EIGRP is an Advance Distance Vector protocol as it tries to avoid any possible looping with help of Dual Algorithm but is not always successful to find it's goal. But not as good as a Link State Routing protocol which would have the complete picture of the topology.

Now in this scenario what you think would happen if Successor Route fails ?

If we go by theory discussed earlier in the post, EIGRP feasible successor should take over. Right ?

But that would mean sub-optimal routing.

But don't worry. EIGRP is intelligent enough still and it finds Optimal Path here based on overall cost to destination and avoiding sub-optimal path.

To my surprise during this test two commands didn't work the way I expected. Which I must figure out sometime :)

Further Readings:

http://www.ciscopress.com/articles/article.asp?p=1763921&seqNum=5

http://books.google.co.in/books?id=XkM6vxsVJEsC&pg=PA78&lpg=PA78&dq=eigrp+convergence+with+feasible+successor&source=bl&ots=mvVF2sg2_K&sig=1jz3SMiK6TiRNnSdj1LKTIj8AHk&hl=en&sa=X&ei=EIxWVMriI8ekuQS-gIL4AQ&ved=0CD8Q6AEwBQ#v=onepage&q=eigrp%20convergence%20with%20feasible%20successor&f=false

http://www.cisco.com/c/en/us/support/docs/ip/enhanced-interior-gateway-routing-protocol-eigrp/16406-eigrp-toc.html#feasibleandreported

http://packetlife.net/blog/2010/aug/9/eigrp-feasible-successor-routes/

http://rovingengineer.wordpress.com/2010/07/28/eigrp-feasible-successor-routes/

HTH...
Deepak Arora
Evil CCIE