Low Latency Queueing QoS on an MPLS Router with a Rate Limited Circuit

In a WAN architecture where routers have MPLS circuits with full bandwidth available, deploying LLQ on the WAN interface is simple as given in the example below. Here we classify the voice traffic with the ACL 140 and mark its precedence to 5 before leaving the serial interface. In addition, during times of congestion on Serial1/0, up to 1Mbps will be reserved for the voice traffic. The priority keyword ensures that this bandwidth is reserved and also ensures that the voice queue is processed before any other queue (up to 1Mbps). All other traffic will fall in a default class which follows the WFQ mechanism for QoS on the interface.

class-map match-all voice
  match access-group 110
policy-map privport-policy
  class voice
  set ip precedence 5
  priority 1000

interface Serial1/0
  ip address 10.22.13.1 255.255.255.252
  service-policy output privport-policy

access-list 110 remark Voice
access-list 110 permit ip host 10.10.1.15 host 10.20.1.15

This example is fairly straight forward and takes care of very simple applications that require voice and data transported over private WANs - whether these are private leased lines or MPLS circuits. Unfortunately, this method cannot be used when the circuit is rate limited. For example you have purchased a DS3 circuit (45Mbps) but are contracted to use just 6Mbps of that circuit. Common networking practice recommends policing the egress traffic at 6Mbps using CAR (Committed Access Rate), which rate limits the traffic to 6Mbps. Fair enough. The next logical step is to then apply LLQ policy (as depicted above) to the interface. Wrong! Recall that the rate limiting mechanism, although has some controls, is impartial as to what traffic it drops whenever the traffic exceeds the configured rate. This is a problem for any real time traffic, whether voice or video and the consequence is fairly obvious.

So how do we overcome this dilemma? The solution - use the Child Service Policy.

In our example we have a full DS3 that must be limited to 6Mbps. This implies that the total outbound traffic cannot exceed 6Mbps. In addition, prioritize voice for 1Mbps and in the absence of voice, data should be able to use available bandwidth up to 6mbps. The configuration below solves this problem.

class-map match-all voicesignal
  match access-group 120
class-map match-all voice
  match ip rtp 16384 16383

policy-map child-policy
  class voice
  priority 1000
  set precedence 5
class voicesignal
  bandwidth 100

policy-map parent-policy
class class-default
shape average 6000000
service-policy child-policy

interface Serial1/0
  ip address 10.71.28.138 255.255.255.252
  dsu bandwidth 44210
  service-policy output parent-policy

access-list 120 permit tcp any any eq 2000
access-list 120 permit tcp any eq 2000 any
access-list 120 permit udp any any eq 2000
access-list 120 permit udp any eq 2000 any

we first classified the voice packets into two parts - voice packets (uses rtp protocol) and the voice signaling packets (SCCP - Skinny Call Control Protocol). Yeah, you guessed it, we are using Cisco Call Manager! These packets are then mapped into a policy called 'child-policy' which gives priority to voice and reserves up to 1Mbps for it during times of congistion. At the same time, 100Kbps of bandwidth is reserved for the signaling packets so that other traffic cannot 'bump' these packets when congestion occurs.

The 'parent-policy' is shaped at 6Mbps and calls on the child-policy in a nested fashion. Think of the parent-policy as an interface running at 6Mbps and you are applying an LLQ policy to it just like in our first example above. The voice and data (default class) gets nicely queued up based on the policy before it hits the actual interface where the 'parent-policy' is applied.

VLAN Filters Not Working On Cisco Catalyst 6000 with MSFC

What I thought was going to be a simple 5 minute procedure ended up taking 3 hours! My goal was very simple - add some security amongst a set of VLANs on a Cisco 6000 (CatOS 7-6-17) switch based on the following requirements:


-VLAN 30 cannot initiate connections to VLAN 50 and VLAN 60 but must get to the Internet
-VLAN 50, is the Admin VLAN and can get to everything
-VLAN 60 cannot initiate connections to VLAN 50 and VLAN 30 but must get to the Internet


Piece of cake, right? I quickly define the filters on the MSFC, deciding, as it should be to place the filters closest to the initiating traffic on VLANs 30 and 60 and have no filters on VLAN 50. The configuration turned out to be like this:


interface Vlan30
description Corp VLAN
ip address 10.30.0.2 255.255.0.0
ip access-group 130 in
!
interface Vlan500
description Admin VLAN
ip address 10.50.0.2 255.255.0.0

!
interface Vlan600
description Test VLAN
ip address 10.60.0.2 255.255.0.0
ip access-group 160 in

ip route 0.0.0.0 0.0.0.0 10.30.0.11

access-list 130 permit tcp any any established
access-list 130 deny ip 10.30.0.0 0.0.255.255 10.50.0.0 0.0.255.255
access-list 130 deny ip 10.30.0.0 0.0.255.255 10.60.0.0 0.0.255.255
access-list 130 permit ip any any

access-list 160 permit tcp any any established
access-list 160 deny ip any 10.50.0.0 0.0.255.255
access-list 160 deny ip any 10.30.0.0 0.0.255.255
access-list 160 permit ip any any


We begin testing this and yay, the traffic from VLAN 30 to 50 is blocked and....then... wait a minute now it's not blocked. I check the hit counter on ACL 130 and there were a couple denies from 10.30.0.0/16 to 10.60.0.0/16 but nothing more. What's going on here? We test from VLAN 30 to VLAN 50 and then from VLAN 60 to both VLAN 30 and VLAN 50 - the same result. The first packet observe the filter rules but thereafter access to each VLAN from any other VLAN is wide open. Grrrr!


At this point I know that my filters are good - there's no rocket science to setting these so I figured that it must be beyond just the filter that was causing this. The clue that led me to the solution was the result of a trace route from a source device (10.30.0.20) to 10.60.0.20. The icmp packet got there in just one hop! Now I expected the VLAN interface hop first before getting to the destination. This wreaked of a cached entry somewhere. But, how did it bypass the Layer 3 routing functionality or...did it? And then it hit me! My theories of Multilayer Switching (MLS) came flooding back and I remembered how the switch consults the router for the initial routing of the packet (Router on a Stick) and then caches the Layer 3 entry in the switch itself for subsequent routing of the paket (Layer3 switching). Of course!


Remember, I had not worked on a Cisco 6000 switch for years so I had to do some digging and found that MLS is enabled by default on this switch (with an MSFC of course). To disable it, merely add the 'no ip mls' command to each VLAN interface. So for VLAN30 it looked like this:


interface Vlan30
description Corp VLAN
ip address 10.30.0.2 255.255.0.0
ip access-group 130 in
no ip mls


I did this on all three layer 3 interfaces and the filters are now doing the work it was supposed to. You can clear the MLS cache by issuing clear mls on the switch itself.


Switch Details


WS-C6009 Software, Version NmpSW: 7.6(17)
System Bootstrap Version: 5.3(1)
System Boot Image File is 'bootflash:cat6000-supk8.7-6-17.bin'


Modules:


WS-X6K-SUP1A-2GE
WS-F6K-PFC
WS-F6K-MSFC2


MSFC:


ROM: System Bootstrap, Version 12.1(3r)E2, RELEASE SOFTWARE (fc1)
BOOTLDR: MSFC2 Software (C6MSFC2-BOOT-M), Version 12.1(8b)E9, EARLY DEPLOYMENT RELEASE SOFTWARE (fc3)


cisco Cat6k-MSFC2 (R7000) processor with 458752K/65536K bytes of memory.
Processor board ID SAD04450FV5