Monday, March 19, 2012

Multi-Homed ISP using BGP - One Primary, the Other Backup

If you are using two different ISPs and want to make one the primary (all traffic goes here for inbound and outbound) and the other a backup, the simplest way to achieve this is to use the 'advertise-map' feature in BGP. Remember that you cannot control which ISP the inbound traffic will pick when destined for your IP block when you have 2 ISPs. You will need to prevent the advertisement of your block to the backup ISP until the primary one goes down.

This is what we recently did to get this going at a client. We got tripped up initially with the route and I explain that below.

Our topology is simple. Routers WR1 (primary) and WR2 (backup) each are connected to ISP1 (primary) and ISP2 (backup) respectively with BGP enabled. They are cross connected to each other for the iBGP session on the 172.16.25.0/24 network. Our goal is to have WR2 advertise our IP block n.x.47.0/24 into ISP2 whenever ISP1 fails.

Steps

1. On WR1,  advertise a route learned from ISP1 to WR2 (prefix list ROUTES-TO-WR2). Pick the route from the routing table. We were receiving partial routes from ISP1 so this was easy. You could also use the default route in this method even if both ISPs advertise the default route. We picked 99.0.0.0/12.

router bgp

  neighbor 172.16.25.2 prefix-list ROUTES-TO-WR2 out

ip prefix-list ROUTES-TO-WR2 seq 10 permit 99.0.0.0/12



2. On WR2, use the advertise-map and non-exist-map to look for this route in the route table. Turns out it also looks in the BGP Table too! Our router map 'NON-EXIST' looks for two conditions - the route 99.0.0.0/12 AND that it was advertised from AT&T (AS=7018). Note that you must look for 7018 as the first AS number in the AS path which has an expression ^7018_

router bgp
  neighbor y.z.32.133 advertise-map ADVERTISE non-exist-map NON-EXIST

access-list 47 permit n.x.47.0 0.0.0.255

ip as-path access-list 3 permit ^7018_

route-map NON-EXIST permit 10

 match ip address prefix-list ATT-Route
 match as-path 3

route-map ADVERTISE permit 10

 match ip address 47

3. Here is the condition when ISP1 is working normally

WR1#sh ip bgp nei
BGP neighbor is y.z.32.133,  remote AS 14265, external link
 Description: TelePacific

Route map for outgoing advertisements is outgoing

  Condition-map NON-EXIST, Advertise-map ADVERTISE, status: Withdraw

4. ISP1 goes down, so we lose the route 99.0.0.0/12 (as well as all other routes being advertised by ISP1)


WR1#sh ip bgp nei

BGP neighbor is y.z.32.133,  remote AS 14265, external link
 Description: TelePacific

Route map for outgoing advertisements is outgoing

  Condition-map NON-EXIST, Advertise-map ADVERTISE, status: Advertise

5. Prior to us adding the 'match as-path 3' in the route map NON EXIST, the status would not change to Advertise.  We found that WR2 had the route 99.0.0.0/12 in its BGP table even though it got removed from the routing table after ISP1 went down!

WR2#sh ip bgp 99.0.0.0/12
BGP routing table entry for 99.0.0.0/12, version 20658895
Paths: (1 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  14265 3549 7018, (received-only)
    y.z.32.133 from y.z.32.133 (y.z.224.176)
      Origin IGP, localpref 100, valid, external
      Community: 934871440

After adding 'match as-path 3' it worked! As you can tell from below that it receives the route with the first (and only) AS # as 7018 from it's iBGP neighbor 172.16.25.1 whereas from y.z.32.133 it has a string of AS#s 14265 3549 7018.

WR2#sh ip bgp 99.0.0.0/12
BGP routing table entry for 99.0.0.0/12, version 20658895
Paths: (2 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer

  7018, (received & used)
    172.16.25.1 from 172.16.25.1 (172.16.25.1)
      Origin IGP, metric 0, localpref 150, valid, internal, best

  14265 3549 7018, (received-only)
    y.z.32.133 from y.z.32.133 (y.z.224.176)
      Origin IGP, localpref 100, valid, external
      Community: 934871440

Configs

WR1

router bgp
 no synchronization
 no bgp fast-external-fallover
 bgp log-neighbor-changes
 bgp bestpath as-path ignore
 network n.x.47.0 mask 255.255.255.0
 neighbor n.m.117.249 remote-as 7018
 neighbor n.m.117.249 description AT&T Ethernet Peer
 neighbor n.m.117.249 version 4
 neighbor n.m.117.249 soft-reconfiguration inbound
 neighbor n.m.117.249 route-map incoming in
 neighbor n.m.117.249 route-map outgoing out
 neighbor 172.16.25.2 remote-as
 neighbor 172.16.25.2 description iBGP peer connection to WR2
 neighbor 172.16.25.2 update-source Loopback0
 neighbor 172.16.25.2 version 4
 neighbor 172.16.25.2 next-hop-self
 neighbor 172.16.25.2 soft-reconfiguration inbound
 neighbor 172.16.25.2 prefix-list ROUTES-TO-WR2 out
 no auto-summary
!

ip prefix-list ROUTES-TO-WR2 seq 10 permit 99.0.0.0/12

WR2


router bgp
 no synchronization
 no bgp fast-external-fallover
 bgp log-neighbor-changes
 bgp bestpath as-path ignore
 network n.x.47.0 mask 255.255.255.0
 neighbor y.z.32.133 remote-as 14265
 neighbor y.z.32.133 description TelePacific
 neighbor y.z.32.133 version 4
 neighbor y.z.32.133 soft-reconfiguration inbound
 neighbor y.z.32.133 route-map incoming in
 neighbor y.z.32.133 route-map outgoing out
 neighbor y.z.32.133 advertise-map ADVERTISE non-exist-map NON-EXIST
 neighbor 172.16.25.1 remote-as
 neighbor 172.16.25.1 description iBGP peer connection to WR1
 neighbor 172.16.25.1 update-source Loopback0
 neighbor 172.16.25.1 version 4
 neighbor 172.16.25.1 next-hop-self
 neighbor 172.16.25.1 soft-reconfiguration inbound
 neighbor 172.16.25.1 prefix-list DEF-ROUTE out
 no auto-summary

access-list 47 permit n.x.47.0 0.0.0.255 

ip as-path access-list 3 permit ^7018_

route-map NON-EXIST permit 10

 match ip address prefix-list ATT-Route
 match as-path 3

route-map ADVERTISE permit 10

 match ip address 47