We’re now going to investigate Route Reflectors and how and when they can be used. Route Reflectors are for use within iBGP only, they are never used for eBGP.
A Route Reflector (RR) is:
An iBGP router, that is specifically allowed to re-advertise iBGP-learned routes to other iBGP peers (within the AS), and is designed to remove the iBGP full-mesh requirement (amongst other things).
If you remember, iBGP has the rule: “Do not re-advertise routes learned from another iBGP peer.” If you use Route Reflectors RRs, these are the exception to this rule. Using Route Reflectors can help your network scale up to 10+ routers without reaching a significant overhead. For example, the iBGP full-mesh requirements means that the number of sessions required increases very rapidly, and thus the complexity (potential blast radius), memory and CPU usage to maintain these states.
| Routers | Sessions |
|---|---|
| 5 | 10 |
| 10 | 45 |
| 50 | 1,225 |
| 100 | 4,950 |
So what about loops? How does the Route Reflector avoid introducing routes where iBGP has not other mechanism other than the rule: “Do not re-advertise routes learned from another iBGP peer.” (a.k.a the Split Horizon iBGP Rule) Essentially it uses the Router ID of the original iBGP source, if a router sees its own ID it drops the route to ensure it doesn’t add loop.
Topology
To show how the Route Reflectors work, we’re adding another Router into the topology, this Router, Router E, is going to be an addition to the AS Blue (65001), with a single connection to Router A.
However, due to the limitations of Router E and the link between them, Router E will not be able to form a full-mesh (via the Loopback IPs of all the routers) like the other Routers can, it will only have a single BGP Neighbour peer as Router A.
The implication of this is that some routes from Router G and Router B, namely their LAN networks will be unable to be learned by Router E, because of the iBGP (Split Horizon) rule: “Do not re-advertise routes learned from another iBGP peer.”
This is where the Route Reflector mechanism can come in to safely distribute the routes learned within the AS Blue (65001) even to a Router such as Router E which is effectively a Router on a stick.
Prepare Topology
Before we can get on with the configuration, we need to perform some basic configuration steps to get Router E ready and connected to the network.
Router E
create vlan "RE-RA"
configure vlan "RE-RA" ipaddress 192.168.100.34/30
enable ipforwarding vlan "RE-RA"
configure vlan "RE-RA" add port 1 untagged
create vlan "RouterE-LAN1" tag 1007
configure vlan "RouterE-LAN1" ipaddress 10.1.3.1/24
enable ipforwarding "RouterE-LAN1"
enable loopback-mode vlan "RouterE-LAN1"
create vlan "RouterE-LP"
configure vlan RouterE-LP tag 1115
enable loopback-mode vlan RouterE-LP
configure vlan RouterE-LP ipaddress 1.1.1.5 255.255.255.255
enable ipforwarding vlan RouterE-LP
configure ospf routerid 1.1.1.5
enable ospf
configure ospf add vlan RE-RA area 0.0.0.0 link-type point-to-point
configure ospf add vlan RouterE-LP area 0.0.0.0 passive
configure bgp AS-number 65001
configure bgp routerid 1.1.1.5
enable bgp
create bgp neighbor 1.1.1.1 remote-AS-number 65001
configure bgp neighbor 1.1.1.1 source-interface ipaddress 1.1.1.5
enable bgp neighbor 1.1.1.1
configure bgp add network 10.1.3.0/24Router A
create vlan "RA-RE"
configure vlan "RA-RE" ipaddress 192.168.100.33/30
enable ipforwarding vlan "RA-RE"
configure vlan "RA-RE" add port 1 untagged
configure ospf add vlan RA-RE area 0.0.0.0 link-type point-to-point
create bgp neighbor 1.1.1.5 remote-AS-number 65001
configure bgp neighbor 1.1.1.5 source-interface ipaddress 1.1.1.1
enable bgp neighbor 1.1.1.5We are setting up the OSPF adjacency additionally to ensure that Router E when it learns a route via iBGP it has the necessary internal underlay routing information available to ensure the routes become “feasible” to BGP.
Before going further ensure that Router E and Router A can see each other via OSPF and BGP, and Router E is showing the routes learned via OSPF, you’ll notice however it’s not seeing some of the AS Blue (65001) LAN networks, namely Router G’s and Router B’s, this is because these routes are being learned via iBGP, and thus not re-advertised.
Route Reflector
The Route Reflector is just a way to ensure that all the member routers of the AS that exchange routes via iBGP can get the updates even if there is no “full-mesh”.
The Route Reflector (RR) is the centralised hub (Router) which collects and then redistributes routing information within the AS.
There is then a Route Reflector Client (RRC) which is one of the iBGP Routers within the AS that must peer directly with the Route Reflector (RR) and will receive (some or) all of its (iBGP) routes from the Route Reflector.
Finally, there is a Non-Client iBGP Peer which is a router within the AS that has not been configured as either an RR or an RRC.
And this is the key thing, each router does not need to be “fully-meshed” will all the other routers within the AS, which is one of the requirements of iBGP, it instead should just be peered to the Route Reflector (RR) Router, however you may have a need for on odd occasions it to be peered with other Routers directly, but will still get the full list of routes (via iBGP) as long as it can peer directly with the Route Reflector.
Key Points
Key Point 1, when we say peered directly, this doesn’t necessarily mean directly, i.e. we don’t mean it must be physically connected, adjacent at Layer 2 or one hop away, we mean there must be a direct BGP Neighbour peering/session between the RR and the RRC, and this peering can flow across an intermediate router (if required). What you can’t have is Router A BGP peering with Router B, which BGP peers with Router C and have Router C be a RR and Router A be a RRC of Router C; that is not a direct BGP peering in BGP terms; therefore only Router B would be the RRC, and thus get the routes, Router A in this case would not!
Key Point 2, although you can have client BGP Routers (RRC) that peer directly to other client BGP Routers (RRC), it is not recommended, it makes it difficult to diagnose problems or understand behaviour in different failure scenarios. Essentially a Route Reflector Client (RRC) Router should only peer with the Route Reflector (RR) Router and not directly to other Route Reflector Client (RRC) Routers; it doesn’t need to (typically), all the routes an RRC would to operate need would be delivered to it by the Route Reflector.
Key Point 3, a single Route Reflector (RR) is a single point of failure within an AS, even if you have redundant paths within your network, if your Route Reflector becomes unavailable your network will partially or fully fail. Therefore, you must have redundant Router Reflectors (RR) Routers within your AS, and therefore each Route Reflector Client (RRC) must peer directly with these two Routers (acting as Route Reflectors RR); additionally these Route Reflector (RR) Routers must be configured with the same Cluster ID, so they act as if they were one.
Key Point 4, where you have a Non-Client iBGP Peer that would need to be peered directly to all the other iBGP Routers within the AS (for it to essentially be a full-mesh), and to get all the routes, but you must consider the Route Reflector Advertising Rules below when considering this design.
Route Reflector Advertising Rules
- A route learned from a Non-Client iBGP Peer is advertised to all RR Clients, but not to other Non-Client iBGP Peers. Note that if it did advertise to Non-Client iBGP Peers it would break the iBGP Split Horizon rule.
- A route learned from an RR Client (RRC) is advertised to both RR Clients (RRC) and Non-Client iBGP Peers. Note that here, even the RR Client (RRC) that originated the route will also get the route (from the Route Reflector), but, it will discard it because it seems that route is from itself.
- A route learned from an eBGP Peer (Neighbour) is advertised to both RR Clients and Non-Client iBGP Peers.
Additional Concepts
- Route Reflector and Route Reflector Clients are manually configured, so you need to choose which are which.
- Multiple Route Reflectors can be configured for redundancy, and potentially load balancing, in a production environment this is very much what you’d need to do.
- The Originator ID and Cluster List are examined to prevent routing loops.
Example 1 – Single Route Reflector (RR)
We’ll start with a simple single Route Reflector configuration. In our example, as mentioned before, we’ve got ourselves a problem. Router E is connected only via a single physical link to Router A. In our example we’re pretending that Router E does not have the capability to create a full-mesh, firstly it has only a single physical link, although as we found in earlier articles, this doesn’t matter, because the actual peering should be made based on something like a Loopback IP Address (e.g. the RouterID) and then carry these Loopback IP Addresses around with an IGP (like OSPF or EIGRP).
We’re not attempting to give an example of a production network here, this is just a lab to explore Route Reflectors and see how they work.
So, back to it.
If we didn’t configure a Router Reflector (RR) on Router A, Router E would not learn all the routes it needed to. It would only learn the routes directly from Router A, it would not learn for example the Router G LAN (10.1.2.0/24) or Router B LAN (10.1.1.0/24), because these would be learned by Router A by iBGP and as we know, the split horizon rule would kick in: “Do not re-advertise routes learned from another iBGP peer”; thus meaning Router E would not learn these routes!
Verify Broken State
So let’s use Route Reflectors and see what can happen. But before that if we example the Router E routes we see the following, which confirms the problem. We are only seeing the Router A LAN (10.1.0.0/24), but not Router G and Router B’s LANs (10.1.1.0/24) and (10.1.2.0/24). But we are seeing the routes learned via eBGP, because remember the iBGP split horizon rule, and in this case the 10.2.0.0/24, 10.2.1.0/24 and 10.2.2.0/24 routes were learned via eBGP, not iBGP, thus not subject to the rule.
Additionally, if we look at Router G’s routes we see another problem, and that is that Router E’s LAN (10.1.3.0/24) is not showing up in the route table either, so although traffic going via Router A (or Router A’s LAN) can reach Router E’s LAN (10.1.3.0/24) none of the other routers within the AS Blue (Router G and Router B) can, because….you guessed it the iBGP split horizon rule.
So let’s get this broken configuration working.
Router A
Adding a route reflector client is pretty easy, you configure the Route Reflector Client on the Route Reflector router, in this case Router A. You don’t need to configure anything on Route Reflector Client (i.e. Router E in this example), as long as you have the iBGP neighbour peer configured, you’re all set.
configure bgp neighbor 1.1.1.5 route-reflector-clientNow on Router A, notice the “r” next to the Router E BGP Neighbour peer: 1.1.1.5, that means it is considered a Route Reflector client.
Verify
And when we check on Router E, after the Route Reflector is configured with the Route Reflector Client, we see all the routes from AS Blue as expected.
But, what about the other way round? Is Router G for example seeing Router E’s LAN (10.1.3.0/24). The answer is yes it is (an explanation of why will follow in a moment).
So how and why is this route from Router E actually appearing? Well, remember the Route Reflector advertising rules mentions earlier, its rule 2: “A route learned from an RR Client (RRC) is advertised to both RR Clients (RRC) and Non-Client iBGP Peers.” So this is why, Router A (Route Reflector) learned the route from Router E (a Route Reflector Client), and therefore has advertised it on to Router G (and B), because even though these two routers are Non-Client iBGP Peers (thus not Route Reflector clients), they get the routes.
But remember, the rules of iBGP still apply for Router G and Router B, because they are fully meshed with Router A, the Route Reflector doesn’t change anything for a Non-Client iBGP Peer.
Example 2 – Two Route Reflector (RR) – Redundant Pair
We’ll go a bit deeper now. So what we’re going to do is make Router G a Route Reflector Client too, so we can see a redundant pair working.
Due to the limitations of our lab, we’re going to have a slightly weird, and useless from Router E’s point of view, because if Router A was to go down, sure Router B would take over (to provide service for Router G), but Router E would be left isolated. Anyway, we’re going to set up Router A and Router B as Route Reflectors, configured as a redundant pair.
But there is a key point with this:
- A Route Reflector Client to be a client of a Route Reflector must peer with it (the RR) as a BGP Neighbour.
- And that therefore means….that if you have a redundant pair of Route Reflectors, each Route Reflector Client must be peered directly with both of the Route Reflector routers.
What that means for our example is that oddly Router E will need to be configured with a BGP Neighbour (peering) to Router B (and vice versa).
Additionally, this is a bit of an odd configuration because with only 3 Routers (well, 4 if you could Router E), we don’t need Route Reflectors because we have a full mesh configured, but if you say had 10 routers, configuring the full-mesh would need 45 BGP Neighbour peering, which be a lot of configuration, and potentially a lot of links!
Router E
Add a BGP Neighbour peering to Router B.
create bgp neighbor 1.1.1.2 remote-AS-number 65001
configure bgp neighbor 1.1.1.2 source-interface ipaddress 1.1.1.5
enable bgp neighbor 1.1.1.2Router B
Add a BGP Neighbour peering to Router E.
create bgp neighbor 1.1.1.5 remote-AS-number 65001
configure bgp neighbor 1.1.1.5 source-interface ipaddress 1.1.1.2
enable bgp neighbor 1.1.1.5However, before we can go any further, we must ensure that the two Route Reflectors will work as a redundant pair. To do this we specify the BGP Cluster ID.
Router A
Configure Router E and Router G as Route Reflector clients, and configure the BGP Cluster ID so that the two Route Reflectors (Router A and B) won’t step on each other’s toes!
disable bgp
configure bgp cluster-id 1
enable bgp
configure bgp neighbor 1.1.1.5 route-reflector-client
configure bgp neighbor 1.1.1.7 route-reflector-clientRouter B
Configure Router E and Router G as Route Reflector clients, and configure the BGP Cluster ID so that the two Route Reflectors (Router A and B) won’t step on each other’s toes!
disable bgp
configure bgp cluster-id 1
enable bgp
configure bgp neighbor 1.1.1.5 route-reflector-client
configure bgp neighbor 1.1.1.7 route-reflector-clientVerify
Now we can see that Router E has two BGP Neighbour peerings.
Break Router A – Simulate Failure of Route Reflector
We’ll break things by disabling BGP on Router A, which will simulate the router going down, but working around the limitations of our lab environment i.e. Router E doesn’t have a direct physical link to Router B, so we can’t just reboot Router A; to simulate a failure.
We issue a “disable bgp” on Router A, then if we check the BGP Neighbours (peers) and BGP routes on Router E, we see the following:
Notice on the BGP routes, we can see 10.1.1.0/24 and 10.1.2.0/24 appearing, this is good, these are the LANs of Router B and Router G respectively, this shows that Router E is still learning the routes.
The network route we are most interested in is 10.1.2.0/24 (Router G’s LAN), we want to confirm, if this is route has been learnt via Route Reflector, it should have been because remembering if the iBGP split horizon rule, Router B wouldn’t have advertised this route to Router E via iBGP, because that would break the iBGP Split Horizon rule, so the fact the route appears on Router E means the Route Reflector is still working, but let’s confirm that with the following:
Ah ha! Router E is learning via Route Reflector, you can see the “RR Cluster ID: 0.0.0.1” listed against the 10.1.2.0/24 route, that means it has learned this via the Route Reflector.
If the Route Reflector was not working, Router E would not have learned this route, because of the iBGP Split Horizon rule.
Conclusion
We’ve explored the use of Route Reflectors in this article, a mechanism that can be used within growing and large networks to work around the need for a “full-mesh” of iBGP routers, which at scale can become unmanageable and sometimes unfeasible to suitably configure. Care must be taken however to ensure that the Route Reflector does not become a single point of failure, but this can be achieved with having multiple Router Reflector routers configured (and use the same Cluster ID) so that they can act as a redundant configuration, with no single point of failure.
A quick note on peer groups, you can configure a set of neighbour parameters and then apply them to multiple neighbours. You do not need to configure the common parameters individually on each neighbour.