ISP LAB

ISP LAB Topology 

 

Below is the basic topology which we will use to build our simple ISP MPLS network, services and applications:
This simple lab is to show you the basic contents of an ISP network, protocols, applications used and services that could be offered by a provider.
**You may need to bookmark this page or save the topology - if you are interested - for future usages to accomplish the below mentioned tasks**
1- Implementing the L3 protocol ISIS as the network's IGP.
2- Building the MPLS cloud using Label Distribution Protocol LDP and assure that Edge and Core routers are exchanging labels correctly.
3- Establishing MP-BGP peering between Edge routers PE01, PE02 , PE03 and the Route Reflector P1 (Each will establish just one BGP session) for exchanging VPNv4 prefixes/Customer routes between Provider Edges PEs.
4- Implementing routing protocols between the Provider Edge & Customer Edge PE-CE routing and advertise the routes to the other side. (like shown in topology)
5- Establishing L2VPN connections between CE3 & CE4 using Any Transport over MPLS ATOM.
6- Implementing Traffic Engineering tunnels MPLS-TE:
a- Tunnel 12 (PE01-P1-P2-P3-PE02) & the required bandwidth 20 MB.
b- Tunnel 13 (PE01-p2-p1-p3-PE03), the required bandwidth 10 MB and static routes are disallowed to accomplish task 7b.
Before diving in accomplishing any task we will show a brief/revision on the used technologies and applications then dive smoothly in implementation and the configuration part.
Follow us the next few days and enjoy the moment...

ISP LAB (Implementing IGP)

 

In this post we are going to accomplish the first task of our simple ISP LAB which is Implementing IGP "Interior Gateway Protocol", as we mentioned before and shown in the topology we are going to use ISIS as the main L3 routing protocol "IGP".
It is known that IGPs are used to exchange internal network routes/prefixes, so it is the same for any ISP but to be accurate in the ISP core we only care about Loopback addresses, so every single router will advertise it's own Loopback address to all network partners to be used in reaching any IP or VPN networks beyond it. after exchanging the Loopbacks - and connected interfaces too but we don't care about them - every router will assign labels for own advertised prefixes and inform it's neighbors with these labels then the neighbor will assign random labels for these prefixes and inform it's neighbors till reaching the Edge of the MPLS "Multi-Protocol Label Switching" network, these labels are assigned and distributed using LDP "Label Distribution Protocol".
The usage of labels will be illustrated in the upcoming parts, but our task implementing the IGP which will be frankly used by the LDP in the next tasks in labeling.
Actually inside the ISP core we have multiple choices of IGPs to use like the well known OSPF "Open Shortest Path First" and the powerful Integrated ISIS "Intermediate System - Intermediate System". Choosing IGP depends on the current network and the needs which is illustrated before in ISIS vs OSPF. In our network we are going to implement ISIS which is easy and straight forward but actually if you didn't deal with it before, this time it may appears scary and it's addressing scheme as a monster which is totally wrong idea and you could check the history of ISIS and the addressing scheme in Integrated IS-IS [Part 1], trust me you will find it so easy.
Ok lets start and finish our task... [refer to the topology: ISP LAB Topology - Part 1]
Our ISP network consists of P1, P2, P3, PE01, PE02 and PE03. "P" means Provider (core) and "PE" means Provider Edge which is connected to "CEs" Customer Edges.
First of all we are going to configure the Loopbacks and directly connected interfaces as mentioned in the Topology. Loopbacks should be configured as host routes /32 for reasons shown in MPLS PHP misbehavior.

P1 interfaces configuration

interface Loopback0  ip address 10.10.10.10 255.255.255.255  ip router isis abc ! interface FastEthernet1/0  ip address 10.1.10.10 255.255.255.0  ip router isis abc ! interface FastEthernet1/1  ip address 10.10.30.10 255.255.255.0  ip router isis abc ! interface FastEthernet2/0  ip address 10.10.20.10 255.255.255.0  ip router isis abc !

Directly connected interfaces are configured as the below example: 10.X.Y.ME, X is the small end, Y is the big end and ME so that if P1 is connected to P3 it would be like that: 10.10(X).30(Y).10(ME)
Also we configured these interfaces to start sending IIH "ISIS Hellos" out to discover neighbors you can read more about discovering neighbors in Integrated ISIS [Part2]. but we didn't configure the ISIS instance so the "ip router isis abc" is not functional yet.
I am not going to show other routers configurations as it will be the same way and once you're interested to reach this part of the topic so you're aware enough how to figure out the rest.
Ok let's configure the ISIS itself.

P1 ISIS configuration

router isis abc  net 02.0010.0010.0010.0010.00  is-type level-2-only

in this part of configuration we mentioned that we are configuring the ISIS instance abc so we should mention the same instance under used interfaces like shown before and this instance could be anything like a number or nothing at all, we also configured the net address showing that we are using the Area "02" and the System ID "0010.0010.0010.0010" again you would know more about these NSAP addresses in Integrated IS-IS [Part 1]  we also mentioned that we only maintain Level-2 neighborships and databases which is also illustrated in the mentioned article.
According to the configuration part we could mention the level type under each interface by the command "isis circuit-type level-2-only" but making it as default is much better to save time and avoid human errors.
Also as long as we don't need the Loopback interfaces trying to send out - to nothing it is virtual - IIHs so it is better to make it passive:

all routers

R(config)#interface loopback 0 R(config-if)#no ip router isis abc R(config-if)#exit R(config)#router isis abc R(config-router)#passive-interface loopback 0

Now we are finished with this task by doing the same to all ISP routers, I hope that it was interesting to you and be with us we still have more to do...

ISP LAB (MPLS Cloud)

 

As mentioned in ISP LAB Topology - Part1 before getting into the configuration part we will give a brief about the technologies/protocols used in our task. In our first task ISP LAB (Implementing IGP) – Part 2 we configured the ISP routers to advertise their loopback addresses using ISIS (Layer3 routing protocol used as IGP) so that all ISP routers have an identical LSDB (Link State Data Base) and can reach each others loopback addresses. lets validate our reachability inside the ISP network on the core router P1 for example:

P1 ISIS neghbors

P1#show isis neighbors System Id      Type Interface   IP Address      State Holdtime Circuit Id PE01           L2   Fa1/0       10.1.10.1       UP    24       P1.02 P2             L2   Fa2/0       10.10.20.20     UP    21       P1.04 P3             L2   Fa1/1       10.10.30.30     UP    23       P1.03

P1 is maintaining level-2 neighborships with PE01, P2 and P3

P1 routing table

P1#show ip route | i L2        i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 i L2     1.1.1.1 [115/20] via 10.1.10.1, 03:08:56, FastEthernet1/0 i L2     2.2.2.2 [115/30] via 10.10.30.30, 03:08:56, FastEthernet1/1 i L2     3.3.3.3 [115/30] via 10.10.30.30, 03:08:46, FastEthernet1/1 i L2     10.1.20.0/24 [115/20] via 10.10.20.20, 03:08:56, FastEthernet2/0 i L2     10.2.30.0/24 [115/20] via 10.10.30.30, 03:08:56, FastEthernet1/1 i L2     10.3.30.0/24 [115/20] via 10.10.30.30, 03:08:56, FastEthernet1/1 i L2     10.20.30.0/24 [115/20] via 10.10.30.30, 03:08:56, FastEthernet1/1 i L2     20.20.20.20 [115/20] via 10.10.20.20, 03:08:56, FastEthernet2/0 i L2     30.30.30.30 [115/20] via 10.10.30.30, 03:08:56, FastEthernet1/1

The routing table of P1 (showing only ISIS level-2 routes), so lets pick 2 routes and try to reach

Sending ICMP (Ping) packets to PE02 and P2

P1#ping 2.2.2.2Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 28/54/112 ms P1#ping 20.20.20.20Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 20.20.20.20, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 8/26/56 ms

Now we are sure that P1 - and all routers respectively - has routes to every router in the network and can reach it.
The previous validation is for being sure that everything is OK before building the MPLS network which rely on the IGP (ISIS in our case) to build LSPs (Label Switched Paths).
MPLS flows are connection oriented & only defines the forwarding mechanism that rely on other two separate protocols (IGP & LDP) to establish the LSPs. For more details it is recommended to read MPLS Summary & MPLS Study Notes [Part1].
We finished the first step by implementing the IGP and we need to implement the signalling protocol LDP (Label Distribution Protocol) to close the circuit.
LDP function is that for each router to assigns local labels for it's own and connected routes/prefixes such as Loopback addresses, then distribute these labels to the LDP neighbors, and the neighbors do the same operation for both received labels and local routes/prefixes till reaching the edge of the network, and now the LSPs are build correctly.
First of all we need to instruct the router to use LDP instead of TDP (Tag Distribution Protocol - Cisco only):

P1 LDP configuration

P1#conf t Enter configuration commands, one per line.  End with CNTL/Z. P1(config)#mpls label protocol ? ldp  Use LDP (default) tdp  Use TDP P1(config)#mpls label protocol ldp

It is just a simple piece of  configuration, now we have two ways for enabling MPLS on all routers:
1st: Configuring MPLS for each interface connected to an ISP router.
2nd: Configuring MPLS automatic configuration on all enabled IGP interfaces.
First way is just doing "mpls ip" command under all ISP connected interfaces, but obviously there is a scalability problem and human errors could appear if we forgot to enable an interface and this interface is chosen as the best path by the IGP so we will have a broken LSP which will drop all packets the going through.
Second one is the most recommended because you will just do it once per router and also you will have no broken LSPs because all IGP enabled interfaces are also MPLS enabled, for more details also it is recommended to use MPLS LDP-IGP Synchronization to avoid such problems that could happen due to slow convergence or whatever.
Lets take a look at the two ways of configuration:

1st way (one command per interface)

P1#conf t Enter configuration commands, one per line.  End with CNTL/Z. P1(config)#int fa 1/0 P1(config-if)#mpls ip

2nd way (one command per router)

P1#conf t Enter configuration commands, one per line.  End with CNTL/Z. P1(config)#router isis abc P1(config-router)#mpls ldp autoconfig

I believe that everyone of you will prefer to chose the second way PLUS synchronization

MPLS LDP SYNC

P1#conf t Enter configuration commands, one per line.  End with CNTL/Z. P1(config)#router isis abc P1(config-router)#mpls ldp sync

Lets check if everything is OK with our configuration:

LDP Neighbors

P1#show mpls ldp neighbor Peer LDP Ident: 20.20.20.20:0; Local LDP Ident 10.10.10.10:0 TCP connection: 20.20.20.20.45861 - 10.10.10.10.646 State: Oper; Msgs sent/rcvd: 236/234; Downstream Up time: 03:11:52 LDP discovery sources: FastEthernet2/0, Src IP addr: 10.10.20.20 Addresses bound to peer LDP Ident: 10.1.20.20      10.20.30.20     10.10.20.20     20.20.20.20 Peer LDP Ident: 30.30.30.30:0; Local LDP Ident 10.10.10.10:0 TCP connection: 30.30.30.30.54795 - 10.10.10.10.646 State: Oper; Msgs sent/rcvd: 236/234; Downstream Up time: 03:11:50 LDP discovery sources: FastEthernet1/1, Src IP addr: 10.10.30.30 Addresses bound to peer LDP Ident: 10.10.30.30     10.20.30.30     10.2.30.30      10.3.30.30 30.30.30.30 Peer LDP Ident: 1.1.1.1:0; Local LDP Ident 10.10.10.10:0 TCP connection: 1.1.1.1.646 - 10.10.10.10.56795 State: Oper; Msgs sent/rcvd: 234/235; Downstream Up time: 03:11:49 LDP discovery sources: FastEthernet1/0, Src IP addr: 10.1.10.1 Addresses bound to peer LDP Ident: 10.1.10.1       10.1.20.1       1.1.1.1

Generally the output of "show mpls ldp neighbor" on Cisco IOS devices shows many information about the LDP neighbors, UP Time, TCP port (646 for LDP and 711 for TDP), interface and addresses bounded to the peer/neighbor.
Lets see what this output shows us: The LDP neighbors of P1 which are P2 (20.20.20.20), P3(30.30.30.30) and PE01(1.1.1.1) and my local LDP ID (10.10.10.10) is used for the connectivity, also the output shows that all neighbors are using TCP 646 to establish the LDP connection which is TRUE.

MPLS Forwarding table LFIB(Label Forwarding Information Base)

PE01#show mpls forwarding-table Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop Label      Label      or Tunnel Id     Switched      interface 16         Pop Label  10.10.10.10/32   0             Fa1/0      10.1.10.10 17         Pop Label  20.20.20.20/32   0             Fa1/1      10.1.20.20 18         22         30.30.30.30/32   0             Fa1/0      10.1.10.10            16         30.30.30.30/32   0             Fa1/1      10.1.20.20 19         Pop Label  10.10.20.0/24    0             Fa1/0      10.1.10.10            Pop Label  10.10.20.0/24    0             Fa1/1      10.1.20.20 20         Pop Label  10.10.30.0/24    0             Fa1/0      10.1.10.10 21         Pop Label  10.20.30.0/24    0             Fa1/1      10.1.20.20 22         25         10.2.30.0/24     0             Fa1/0      10.1.10.10            19         10.2.30.0/24     0             Fa1/1      10.1.20.20 23         26         10.3.30.0/24     0             Fa1/0      10.1.10.10            20         10.3.30.0/24     0             Fa1/1      10.1.20.20 24         28         2.2.2.2/32       0             Fa1/0      10.1.10.10            24         2.2.2.2/32       0             Fa1/1      10.1.20.20 25         27         3.3.3.3/32       0             Fa1/0      10.1.10.10            23         3.3.3.3/32       0             Fa1/1      10.1.20.20

The output shows the Local Label that is advertised by the local router to its neighbors, the Outgoing Label which is received by the Next Hop and the local router uses to reach the destination by binding it to the packet and forward the labeled packet to the Outgoing interface.
If the Outgoing Label shows "Pop Label" it means that the local router is the PHP (Penultimate Hop Popping) router and the destination is local or directly connected to the Next Hop.

Traceroute from PE01 to PE02

PE01#traceroute 2.2.2.2Type escape sequence to abort. Tracing the route to 2.2.2.2  1 10.1.20.20 [MPLS: Label 24 Exp 0] 80 msec     10.1.10.10 [MPLS: Label 28 Exp 0] 68 msec     10.1.20.20 [MPLS: Label 24 Exp 0] 64 msec   2 10.10.30.30 [MPLS: Label 17 Exp 0] 64 msec     10.20.30.30 [MPLS: Label 17 Exp 0] 36 msec     10.10.30.30 [MPLS: Label 17 Exp 0] 72 msec   3 10.2.30.2 56 msec *  84 msec

The output of the Traceroute from PE01 to PE02 shows that we are using labels along the path till the Penultimate hop is popping up the top most label of the stack - only one label per stack in our case - before sending it to the last hop/destination.
Now we have a ready network for implementing applications and services for the customers, and this network is IP/MPLS based, so keep your eyes opened for a very interesting part about MP-BGP in the next task.
Enjoy,

ISP LAB (MP-BGP)

 

It has been long time since the last part of this interesting - my own opinion- series, so before diving through today's task I recommend everyone to check our LAB Topology & Tasks thus everyone should have known what we are doing here and our strategy to accomplish the mentioned tasks.
In the previous tasks we already implemented IS-IS to be the used IGP and then configured MPLS LDP as the labeling protocol, this LDP is used to assign label IGP prefixes.
Note: In some vendor implementations like Juniper's LDP only assigns labels for IGP /32 prefixes.
So now we need another mechanism or a protocol to assign labels for the customer/vpn routes. Sure the engineering community gave a try to the routing protocols like IS-IS/OSPF to take this responsibility but the problem is they have to do extensions to all routing protocols or force ISPs to use just one which didn't make sense at all, so as usual they came with the magic solution of extending the BGP to carry customers routes and assign labels to the routes and that does make sense because BGP is already used in ISPs and much flexible to be extended using AFI/SAFI concept.
The new extension was named MP-BGP which is clearly for MultiProtocol-Border Gateway Protocol, this protocol is mainly works between all PEs which requires a full mesh of iBGP sessions, or between a BGP Route Reflector and all PEs which requires only one iBGP session for each, this one is so recommended and I have never seen an ISP does full mesh of iBGP sessions, if you know some ISP do that please call 911
MP-BGP is responsible to carry customers routes from a PE and offer them to all other PEs if interested interested*, so now let's see how to accomplish this easy task using Cisco routers of 7200VXR series which obviously run the traditional Cisco IOS.
*Interested: Once a customer is attached to a certain PE, this PE will import his related routes/prefixes from other PEs using BGP extended communities "will be discussed later"
In our topology we have three PE routers "PE01, PE02 & PE03" as we mentioned we can do full mesh of MP-BGP sessions which is not recommended so we picked up P1 to be the Route Reflector collecting all the VPNv4 prefixes and redistribute them again to all clients "PEs".
Note: In real operational network you could pick up a PE to play the Route Reflector role or you could have a separate router to play this role, it has no standard.
First of all we will configure P1 to be ready for his new role, by mentioning his clients under the BGP process.

router bgp 123  no synchronization  bgp log-neighbor-changes  neighbor 1.1.1.1 remote-as 123  neighbor 1.1.1.1 update-source Loopback0  neighbor 2.2.2.2 remote-as 123  neighbor 2.2.2.2 update-source Loopback0  neighbor 3.3.3.3 remote-as 123  neighbor 3.3.3.3 update-source Loopback0  no auto-summary  !  address-family vpnv4   neighbor 1.1.1.1 activate   neighbor 1.1.1.1 send-community extended   neighbor 1.1.1.1 route-reflector-client   neighbor 2.2.2.2 activate   neighbor 2.2.2.2 send-community extended   neighbor 2.2.2.2 route-reflector-client   neighbor 3.3.3.3 activate   neighbor 3.3.3.3 send-community extended   neighbor 3.3.3.3 route-reflector-client  exit-address-family

Lets clarify what we've done in this configuration exactly. Firstly we defined the iBGP peers "PEs" and then activated all these peers under the VPNv4 address-family which means that we extended our BGP process to be MP-BGP and can exchange VPNv4 prefixes normally, then we defined these peers to be clients and that obviously means that P1 now became officially a Route Reflector.
There is a small part we didn't mention which is "send-community extended", this is enabled by default when activating a neighbor within the VPN address-family which means that we can exchange extended communities like "Route-Targets" with that peer and we will discuss this in details later.
Now we have just one thing to do, is to do the same configuration "excluding client part" on the PE routers.

router bgp 123  no synchronization  bgp log-neighbor-changes  neighbor 10.10.10.10 remote-as 123  no auto-summary  !  address-family vpnv4   neighbor 10.10.10.10 activate   neighbor 10.10.10.10 send-community extended  exit-address-family

This part does not need any clarification I assume, and once this configuration was done, a notification msg of a BGP peering 10.10.10.10 became up, and after checking we found that an undesirable ipv4 BGP session became up and that because BGP expects an IPv4 BGP peer by default.
*Sep 20 00:26:21.803: %BGP-5-ADJCHANGE: neighbor 10.10.10.10 Up
To solve this issue we only need to tell the routers "Never expect an IPv4 BGP session unless mentioned" by doing the following command on all PE routers and the Route Reflector under the BGP process, please read BGP default IPv4-unicast:
Router(config-router)#no bgp default ipv4-unicast
Now lets verify what we've done already:

P1#show bgp su P1#show bgp vpnv4 unicast all summary BGP router identifier 10.10.10.10, local AS number 123 BGP table version is 1, main routing table version 1 Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd 1.1.1.1         4          123       5       5        1    0    0 00:01:21        0 2.2.2.2         4          123       2       2        1    0    0 00:00:22        0 3.3.3.3         4          123       2       2        1    0    0 00:00:31        0

Now we are done, be ready for the most interesting part, VRFs & PE-CE routing...