EIGRP Troubleshooting

Posted on April 14, 2010 by Judson Bishop

2010-04-14 07:45:39
EIGRP Troubleshooting
Enhanced Interior Gateway Protocol (EIGRP)
EIGRP is a Cisco proprietary hybrid routing protocol or an advance distance vector routing protocol.

There are four components that make up EIGRP:
1. Protocol-dependent modules — Supports several routed protocols.

2. Reliable Transport Protocol (RTP) — Send packets reliably using IP protocol 88.

3. Neighbor discovery and recovery — Uses hellos to identify its neighbors quickly and learn when neighbors are down.

4. Diffusing Update Algorithm (DUAL) — Finds the best path and feasible failover paths.

EIGRP uses three tables to track state:
1. EIGRP interface table — Interfaces participating in EIGRP, troubleshoot with the command sh ip eigrp int.

2. EIGRP neighbor table — Tracks routers which have exchanged Hello packets. Hello packets are sent regularly out all interfaces participating in EIGRP to multicast address 224.0.0.10 in order to maintain neighborship. Neighbors are removed if no hello packet is received within the holdtime. Troubleshoot the neighbor table by using sh ip eigrp neigh or debug eigrp packets.

3. EIGRP topology table — After the router discovers its neighbors it is able to create the topology table and assigns successors and feasible successors. Passive is good, active is a bad state in the topology table, it is active because the router is actively attempting to find a path. DUAL acts upon the topology table to populate the routing table. Troubleshoot the topology table with sh ip eigrp top.

Other EIGRP Definitions:
— Advertised Distance — The metric from a neighbor to the destination network.

— Feasible Distance — The best metric to the destination network, including the metric to the neighbor advertising that path.

— Feasible Successor — Advertised distance is less than the feasible distance or current best path. This is called the feasibility requirement and this route will not result in a loop if installed in the routing table.

— Variance — EIGRP automatically load balance across equal-cost links. In order to turn on unequal cost load balancing use the variance command.

sh ip eigrp int
Display all interfaces designated to participate in EIGRP.
[code]
R4#sh ip eigrp int
IP-EIGRP interfaces for process 10
Xmit Queue Mean Pacing Time Multicast Pending
Interface Peers Un/Reliable SRTT Un/Reliable Flow Timer Routes
Fa0/0 1 0/0 2 0/1 50 0
Fa0/1 1 0/0 5 0/1 50 0
[/code]

sh ip eigrp neigh
She routers EIGRP neighbors.
[code]
R4#sh ip eigrp neigh
IP-EIGRP neighbors for process 10
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
1 10.1.4.10 Fa0/1 14 1w5d 5 200 0 75
0 10.1.4.6 Fa0/0 12 3w0d 2 200 0 75
[/code]

sh ip eigrp top
Displays all routes known to the EIGRP process.
[code]
R4#sh ip eigrp top
IP-EIGRP Topology Table for AS(10)/ID(10.1.4.5)
Codes: P – Passive, A – Active, U – Update, Q – Query, R – Reply,
r – reply Status, s – sia Status

P 10.2.4.12/30, 2 successors, FD is 30720
via 10.1.4.6 (30720/15360), FastEthernet0/0
via 10.1.4.10 (30720/15360), FastEthernet0/1
P 0.0.0.0/0, 1 successors, FD is 28160
via Summary (28160/0), Null0
P 10.1.4.8/30, 1 successors, FD is 28160
via Connected, FastEthernet0/1
P 10.2.1.0/24, 1 successors, FD is 28416
via 10.1.4.6 (28416/2816), FastEthernet0/0
P 10.2.2.0/24, 1 successors, FD is 28416
via 10.1.4.10 (28416/2816), FastEthernet0/1
P 10.1.4.4/30, 1 successors, FD is 28160
via Connected, FastEthernet0/0
[code]

<strong>sh ip route eigrp</strong>
Shows the routes in the IP routing table that were installed by the EIGRP process.
[code]
R4#sh ip route eigrp
10.0.0.0/8 is variably subnetted, 8 subnets, 2 masks
D 10.2.4.12/30 [90/30720] via 10.1.4.10, 1w5d, FastEthernet0/1
[90/30720] via 10.1.4.6, 1w5d, FastEthernet0/0
D 10.2.1.0/24 [90/28416] via 10.1.4.6, 3w0d, FastEthernet0/0
D 10.2.2.0/24 [90/28416] via 10.1.4.10, 1w5d, FastEthernet0/1
D* 0.0.0.0/0 is a summary, 3w0d, Null0
[/code]

debug ip routing
Display routing table updates, not EIGRP specific.
[code]
IP routing debugging is on
R4#
Apr 4 22:32:06.331: RT: NET-RED 0.0.0.0/0
R4#
Apr 4 22:33:06.327: RT: NET-RED 0.0.0.0/0
R4#
Apr 4 22:33:54.943: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
Apr 4 22:33:54.943: RT: is_up: FastEthernet0/1 0 state: 4 sub state: 1 line: 0 has_route: True
Apr 4 22:33:54.943: RT: interface FastEthernet0/1 removed from routing table
Apr 4 22:33:54.943: RT: del 10.1.4.8/30 via 0.0.0.0, connected metric [0/0]
Apr 4 22:33:54.943: RT: delete subnet route to 10.1.4.8/30
Apr 4 22:33:54.943: RT: NET-RED 10.1.4.8/30
Apr 4 22:33:54.943: RT: Pruning routes for FastEthernet0/1 (2)
Apr 4 22:33:54.947: RT: delete route to 10.2.4.12 via 10.1.4.10, FastEthernet0/1
Apr 4 22:33:54.947: RT: NET-RED 10.2.4.12/30
Apr 4 22:33:54.947: RT: delete route to 10.2.2.0 via 10.1.4.10, FastEthernet0/1
Apr 4 22:33:54.947: RT: no routes to 10.2.2.0, flushing
Apr 4 22:33:54.947: RT: NET-RED 10.2.2.0/24
R4#
Apr 4 22:33:54.951: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 10: Neighbor 10.1.4.10 (FastEthernet0/1) is down: interface down
[/code]

debug eigrp packets
Shows EIGRP neighbor interaction.
[code]
R4#debug eigrp packets
EIGRP Packets debugging is on
(UPDATE, REQUEST, QUERY, REPLY, HELLO, IPXSAP, PROBE, ACK, STUB, SIAQUERY, SIAREPLY)
R4#
Apr 4 22:35:19.701: EIGRP: Sending HELLO on FastEthernet0/0
Apr 4 22:35:19.701: AS 10, Flags 0x0, Seq 0/0 idbQ 0/0 iidbQ un/rely 0/0
Apr 4 22:35:19.961: EIGRP: Sending HELLO on FastEthernet0/1
Apr 4 22:35:19.961: AS 10, Flags 0x0, Seq 0/0 idbQ 0/0 iidbQ un/rely 0/0
Apr 4 22:35:20.665: EIGRP: Received HELLO on FastEthernet0/1 nbr 10.1.4.10
Apr 4 22:35:20.665: AS 10, Flags 0x0, Seq 0/0 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/0
[/code]

debug ip eigrp
Shows the information contained in EIGRP packets.
[code]
R4#debug ip eigrp
IP-EIGRP Route Events debugging is on
R4#
Apr 4 22:36:10.745: IP-EIGRP(Default-IP-Routing-Table:10): Processing incoming QUERY packet
Apr 4 22:36:10.745: IP-EIGRP(Default-IP-Routing-Table:10): Int 10.1.4.8/30 M 4294967295 – 0 4294967295 SM 4294967295 – 0 4294967295
Apr 4 22:36:10.769: IP-EIGRP(Default-IP-Routing-Table:10): 10.1.4.8/30 – don’t advertise out FastEthernet0/0
Apr 4 22:36:10.769: IP-EIGRP(Default-IP-Routing-Table:10): Int 10.1.4.8/30 metric 28160 – 25600 2560
R4#
Apr 4 22:36:14.933: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
Apr 4 22:36:14.937: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 10: Neighbor 10.1.4.10 (FastEthernet0/1) is down: interface down
Apr 4 22:36:14.937: IP-EIGRP(Default-IP-Routing-Table:10): route installed for 10.2.4.12 ()
Apr 4 22:36:14.953: IP-EIGRP(Default-IP-Routing-Table:10): 10.1.4.8/30 – not in IP routing table
Apr 4 22:36:14.953: IP-EIGRP(Default-IP-Routing-Table:10): Int 10.1.4.8/30 metric 4294967295 – 0 4294967295
Apr 4 22:36:14.953: IP-EIGRP(Default-IP-Routing-Table:10): 10.2.2.0/24 – not in IP routing table
Apr 4 22:36:14.953: IP-EIGRP(Default-IP-Routing-Table:10): Int 10.2.2.0/24 metric 4294967295 – 25600 4294967295
Apr 4 22:36:14.985: IP-EIGRP(Default-IP-Routing-Table:10): Processing incoming REPLY packet
R4#
Apr 4 22:36:14.985: IP-EIGRP(Default-IP-Routing-Table:10): Int 10.1.4.8/30 M 4294967295 – 0 4294967295 SM 4294967295 – 0 4294967295
Apr 4 22:36:14.985: IP-EIGRP(Default-IP-Routing-Table:10): Int 10.2.2.0/24 M 4294967295 – 0 4294967295 SM 4294967295 – 0 4294967295
[/code]

Administrative Distance
Believability of a routing protocol, the more believable the protocol the more likely it is to be added to the routing table.

Protocol

Administrative Distance
Connected Interface

0 Most believable
Static route

1
EIGRP Summary route

5
EBGP

20
Internal EIGRP

90
IGRP

100
OSPF

110
IS-IS

115
RIP

120
EGP

140
ODR

160
External EIGRP

170
Internal BGP

200
Unknown

255 Unreachable

Source: TSHOOT Book p. 151

Other resources:
EIGRP Introduction
EIGRP

Posted in Uncategorized | Leave a comment

Troubleshoot BGP

Posted on April 13, 2010 by Judson Bishop

2010-04-13 16:07:12
Troubleshoot BGP
BGP is a path-vector routing protocol. Routes are tracked in terms of the AS they pass through,
and routers avoid loops by rejecting routes that have already passed through their AS.

Definitions
Synchronization — Before iBGP can propagate a route, the route must be learned from an IGP.
Split Horizon — BGP will not advertise a route out the interface from which it was learned.
Path Selection — Routers avoid loops by rejecting routes that already include their AS. BGP attributes are used to determine the best route to a destination.

BGP Attributes

Order

Preference

Description
0. Synchronized

TRUE

Use only routes that meet the synchronization requirement
1. Weight
_______________

Highest

Administrative override, a router will use the interface with the highest weight when multiple paths exist. Not propagated to other routers.
2. Local Preference

Highest

Used internally to pick path out of AS and is propagated to its internal peers.
3. Self Originated

TRUE

Prefer paths that originated on this router.
4. AS-Path

Shortest

Minimize AS-hops, prefer the route that has the shortest AS Path.
5. Origin

 i<?

Prefer stability, where IGP is lower than EGP, and EGP is lower than incomplete.
6. Multi-Exit Discriminator (MED)

Lowest

Advertised to external neighbors in an attempt to influence path selection in your AS.
7. External

EBGP<IBGP

External path is preferred over an internal path.
8. IGP cost

Lowest

Prefer the path with the lowest IGP cost.
9. EBGP Peering

Oldest

Prefer stability.
10. RID

Lowest

Finally choose the route with lowest BGP router ID.

Sources: BSCI p. 443 and TSHOOT P.229

BGP Updates three tables

Table

Description

Troubleshoot
Neighbor table

Contains listing of neighbors, their IP address, AS Number, neighborship state.

sh ip bgp summary
sh ip bgp neighbor
debug ip bgp updates
BGP table or BGP Routing Information Base (RIB)

Contains routes learned from BGP neigbors and locally injected routes.

sh ip bgp
__________________
IP Routing table

Only the “best” routes as defined by the attributes make it into the routing table.

sh ip route bgp

BGP States
BGP cycles through five states as it runs:
Idle – Searching for neighbors
Connect – TCP three-way handshake complete with neighbor
Open Sent – BGP Open message has been sent
Open Confirm – Response received
Established – BGP neighborship is established
Remember: Established is good, anything else is bad.

Troubleshoot BGP States
– If a neighbor does not progress from idle, look for a next-hop address that cannot be reached.
– If the neighbor stays active, the neighbor is not responding as expected, so look for things that affect this reply. For instance,the peering IP address or AS number may be incorrect, the neighbor may be misconfigured, or authentication may be misconfigured.
BSCI p.406, TSHOOT p.229

TSHOOT BGP
sh ip bgp summary
Helps to debug the neighbor table.
[code]
R1#sh ip bgp summ
BGP router identifier 209.65.200.225, local AS number 65001
BGP table version is 16, main routing table version 16
2 network entries using 234 bytes of memory
3 path entries using 156 bytes of memory
3/2 BGP path/bestpath attribute entries using 372 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 786 total bytes of memory
BGP activity 7/5 prefixes, 10/7 paths, scan interval 60 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
209.65.200.226 4 65002 47706 47714 16 0 0 4w5d 2
[/code]

sh ip bgp neighbor
More neighbor table debugging.

[code]
R1#sh ip bgp neighbors
BGP neighbor is 209.65.200.226, remote AS 65002, external link
BGP version 4, remote router ID 209.65.200.242
BGP state = Established, up for 4w5d
Last read 00:00:54, last write 00:00:17, hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 2 2
Notifications: 1 0
Updates: 3 2
Keepalives: 47711 47704
Route Refresh: 0 0
Total: 47717 47708
Default minimum time between advertisement runs is 30 seconds

For address family: IPv4 Unicast
BGP table version 16, neighbor version 16/0
Output queue size : 0
Index 1, Offset 0, Mask 0x2
1 update-group member
Sent Rcvd
Prefix activity: —- —-
Prefixes Current: 1 2 (Consumes 104 bytes)
Prefixes Total: 1 2
Implicit Withdraw: 0 0
… output removed for brevity …
[/code]

sh ip bgp neigh x.x.x.x advertised-routes
This command should be in the troubleshooting list in the book but is not. It shows what advertisements you are sending to a neighbor.

[code]
R1#sh ip bgp neigh 209.65.200.226 advertised-routes
BGP table version is 16, local router ID is 209.65.200.225
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
r RIB-failure, S Stale
Origin codes: i – IGP, e – EGP, ? – incomplete

Network Next Hop Metric LocPrf Weight Path
*> 209.65.200.224/30
0.0.0.0 0 32768 i

Total number of prefixes 1
[/code]

sh ip bgp neigh x.x.x.x routes
Similarly this command is not discussed in the book. This command shows which routes are being received. Helps to debug the RIB.

[code]
R1#sh ip bgp neigh 209.65.200.226 routes
BGP table version is 16, local router ID is 209.65.200.225
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
r RIB-failure, S Stale
Origin codes: i – IGP, e – EGP, ? – incomplete

Network Next Hop Metric LocPrf Weight Path
* 209.65.200.224/30
209.65.200.226 0 0 65002 i
*> 209.65.200.240/29
209.65.200.226 0 0 65002 i

Total number of prefixes 2
[/code]

sh ip bgp
Help to debug the RIB.

[code]
R1#sh ip bgp
BGP table version is 16, local router ID is 209.65.200.225
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
r RIB-failure, S Stale
Origin codes: i – IGP, e – EGP, ? – incomplete

Network Next Hop Metric LocPrf Weight Path
* 209.65.200.224/30
209.65.200.226 0 0 65002 i
*> 0.0.0.0 0 32768 i
*> 209.65.200.240/29
209.65.200.226 0 0 65002 i
[/code]

sh ip route bgp
What routes are making it from the RIB to routing table.

[code]
R1#sh ip route bgp
209.65.200.0/24 is variably subnetted, 2 subnets, 2 masks
B 209.65.200.240/29 [20/0] via 209.65.200.226, 4w5d
[/code]

Debug Commands
debug ip bgp
debug ip bgp updates

Posted in Routing | Leave a comment

Wednesday Night Worlds

Posted on April 12, 2010 by Judson Bishop

One of my friends got onto me that my blog was all about work. He wanted to read stories about some of my cycling adventures. Rather than launch into stories about riding across America, I’ll start with some stories from my distant past.

Every city has that one ride that is the crazy all out sprint for the roses and kisses. In Auburn it is the Wednesday Night Worlds. When I was in Los Angeles it was the Pier ride, in San Antonio it was the Helotes Loop, in Omaha it was the Wednesday night Train Track ride, and in Montgomery it was the Thursday night Emerald Mountain ride.

Out in LA they meet at the piers. When I got to LA I went to Helen’s Cycles and asked where the club rides met. The guys told me, “The pro-1-2s meet at pier X, the 3s meet at Y and the 4s at Z.” So I showed up at X, hey I wanted to go to nationals that year, why would I ride with the 3s? There were maybe 25 of us and we headed down the beach bike path, at each intersection we would pick up a different group. By the time we made it to the city streets there had to be 150 riders in the peloton. It was so big that when the front half made it through the light the rest of the peloton made it too, even if by that time the light was red. I was working in the front and when it was my turn to pull I looked at the guy behind me and said, “I don’t know where I’m going.” He replied, “Don’t worry I’ll tell you where to turn.” About that time I got in behind some little red hatchback, must have been a Yugo, there were 150 riders behind me and I was drafting that Yugo at 30mph praying he didn’t put on his breaks. I stayed with that little Yugo until I pulled off the front.

The LA ride ends in a sprint somewhere over by the airport and it was crazy fast. There were about 10 of us contesting the sprint, guys would pull and when they were done would immediately get shelled, and it was my turn to pull again. So I gave it everything I had and all of a sudden it was like I had on the breaks. Guys were coming around me left and right. Somebody won the sprint and I asked, “What were we sprinting for?” One of the riders replied, “The fire hydrant.” I should have known.

That was Tuesday morning and Wednesday night I went to Burger King and got a Whopper meal. Thursday morning I showed up at the pier and the guys were nice to me this time around, because now they knew I could ride. That morning I had a burger pit in my stomach, I was wishing I would just throw up my dinner from last night so I could ride. I don’t even think I made a pull that morning and I remember that suffering to this day.

Well I had purposely made my plane reservations out of LA for late Saturday so I could do the Saturday ride. It did not disappoint. I believe it’s called the Doughnut ride. Anyway I was careful what I ate the rest of the week because I wanted to redeem myself for my Thursday morning performance. The ride got started and the engines moved to the front so I went with them. Some pro rider had shown up and launched himself off the front, and I grabbed his wheel, just the two of us swapping pulls. At some point we hit a stop light red and he started doing a track stand. Dummy me, I decided I would too, even in my hypoxic state. Of course you can see where this is headed, I fell over and he rode off and left me. Because I had no idea where I was I had to wait for the chase group in order to find my way back to the car. My pride was hurt more than my skinned knee.

I also have a good story about my first Helotes ride in San Antonio. The Tuesday night Helotes ride is for the racers. It’s a crazy, all out hammer fest. I was told about this ride from the guys in one of the local shops, but they told me it was 50 miles. I decided it had my name on it, but when I showed up there was the Pacificare RAAM team. I had no idea what I was getting myself into. They leave right at 6:00pm on the clock and it was all out from the start. I remember doing 30mph and loosing ground on the guys in the front. I got to thinking 50 miles of this is crazy, I better find a pack. So I got into the second group which was a bunch of special forces pararescue guys I had gotten to know. When we got back to Helotes it was only 30 miles and I was pissed. The guys couldn’t understand why until I told them I thought the ride was 50 miles, not 30. They all laughed at me and one guy said, “First time huh. Did they tell you about it at the shop?” It seems the shop makes that ride sound harder than it is to deter people they are not sure should be there.

Posted in Cycling, Thoughts | Leave a comment

Troubleshoot OSPF

Posted on April 12, 2010 by Judson Bishop

My notes for EIGRP felt like I was just regurgitating the Key Points from the book. So with OSPF I read the book and looked at the command output, then decided to copy my notes from the BSCI and edit them down. In this manner I get more detail than the book, it refreshes my mind and I don’t feel like I am bored redoing the commands shown in the book. Please note, these are my notes, you got it off the web from some CCNA who is working toward a CCNP, you get the idea.

Open Shortest Path First (OSPF)
OSPF is a vendor neutral link-state routing protocol designed around the shortest path first or Dijkstra algorithm. OSPF learns all possible routes in the network and never produces routing loops.

  • Classless and allows summarization.
  • Quickly converges
  • Conserves network bandwidth
  • Uses multicast
  • Sends incremental change based updates
  • Metric is cost
  • Generates routing updates only upon change

    • OSPF creates three tables
    1. Neighbor Table or Adjacency Database ñ Contains a list of recognized neighbors
    2. Topology Table or LSDB ñ Contains all routers and their attached links the area or network. All routers in an area have Identical LSDB
    3. Routing Table or Forwarding Database ñ Holds the best paths to destinations

    Basic OSPF Operation
    Exchange Hello messages to build neighbor table.
    Elect DR and BDR.
    Exchange LSA to build topology table.
    Run SPF on topology table to build routing table.

    OSPF Area Structure
    OSPF uses a two-layer hierarchy.
    1. Transit Area ñ Primary function is fast and efficient packet movement. End users are not usually found here. OSPF area 0 by definition is a transit area.
    2. Regular Area ñ Primary function is to connect users and resources. By default a regular area does not allow transit traffic. It has a number of subtypes; standard area, stub area, totally stubby area and not-so-stubby area.

    Neighorship
    OSPF sends multicast hello packets to destination address 224.0.0.5. On a broadcast link hello packets are sent every 10 seconds and every 30 seconds on a nonbroadcast link. After initial hello exchange routers become neighbors as soon as they see themselves listed in the neighborís hello packet, this guarantees two way communication. Neighbor negotiation applies to the primary address only, secondary addresses have to belong to the same area as the primary address.

    Routers must agree on the following to become neighbors:

  • Area-ID ñ If two routers share a segment, their interfaces have to belong to the same area on that segment. Should also belong to the same subnet and share the same mask.
  • Authentication ñ OSPF allows for a password in an area, routers must exchange the same password in an area to become neighbors.
  • Hello Interval — Hellos are a form of keep alive used for acknowledgment on a segment and to elect a designated router on multicast segments. Hello interval specifies the length of time, in seconds, between hello packets on an OSPF interface.
  • Dead Interval ñ Number of seconds that a routerís Hello packets have not been seen before its neighbors declare a router down.
  • Stub area flag ñ Routers must agree on the stub area flag in the hello packet.

    • Adjacency
    Routers form an adjacency after they have synchronized their topology databases. After the hello exchange, routers proceed to the database exchange process. In order to minimize the amount of information exchange on a particular segment OSPF elects one router as the designated router (DR) and another as the backup designated router (BDR) on each multi-access segment. Instead of routers having to exchange information with every other router in the segment, they only have to exchange information with the DR and BDR. The DR and BDR relay information to all the other routers.

    In mathematical terms this cuts communication from O(n*n) to O(n) where n is the number of routers on a segment. Each router tries to establish adjacency with the DR and BDR.

    Building the Adjacency

    Adjacency States:
    The following are the states of an interface in the process of becoming adjacent to another router.

    Down

    		No hellos received from any router on segment
    Attempt

    		Only valid for manually configured neighbors
    Init

    		The router has received a hello packet from its neighbor but the receiving routerís ID was not in that hello packet.
    Two-Way

    		Bi-directional communication has been established between the routers. At the end of this stage DR and BDR election is complete and the routers will decide whether to proceed with an adjacency.
    Exstart

    		Routers are trying to establish the initial sequence number for packet exchange, the sequence number insures routers maintain recent information.
    Exchange

    		Routers send entire link-state database.
    Loading

    		Routers are finalizing their information exchange.
    Full

    		Adjacency is complete, fully synchronized databases.

    DR Election
    OSPF routers on a LAN segment elects one DR and BDR, all other routers on that segment form full adjacencies with these two routers and pass LSAs only to them. The router with the highest OSPF priority on a segment will become the DR for that segment. Priority can be set from 0 ñ 255 with the default priority being 1 and a priority of 0 prevents the router from being elected.

    OSPF Multicast Addresses:
    224.0.0.5 goes to all OSPF routers.
    224.0.0.6 goes to the DR and BDR.

    Link State Advertisement (LSA)
    LSAs are reliable, delivery is acknowledged they also have a sequence number and a set lifetime, so that each router recognizes that it has the most current link state database.

    LSA Type_______

    		Name

    		Description
    Types 1 and 2 are flooded througout an area and are the basis of SPF path selection.
    Type 1

    		Router Link LSA

    		Generated by every router and sent to every area to which it is connected. Lists each neighbor, the cost to every neighbor, directly connected link state. Establishes and maintains neighbor relationships.

    Type 2

    		Network Link LSA

    		DR generated LSA that lists all the routers on the segment, the topological database.
    Types 3 and 4 are called inter-area LSAs because they are passed between areas.
    Type 3

    		Network Summary Link LSA

    		ABRs generate Type 3 LSAs to send between areas describing routes to the area’s networks. When an ABR receives type 1 LSAs it sends out type 3 LSAs to other areas advertising the networks it learned via type 1 LSAs.
    Type 4

    		 AS external ASBR summary link LSA

    		ASBRs produce this LSA to advertise their presence and describe routes to themselves.
    Type 5

    		External Link LSA

    		Generated by ASBRs to describe routes external to OSPF, are flooded throughout the the AS except into totally stubby areas and totally stubby NSSAs.
    Type 6

    		 Multicast LSA

    		Used in multicast OSPF applications.
    Type 7

    		 NSSA External LSA

    		Created by an ASBR with one link in a NSSA. Stubby areas do not allow type 5 LSAs, so a type 7 LSA is a type 5 tunneled through the NSSA.

    OSPF Network Types
    The TSHOOT book did a much better job than my notes summarizing the OSPF network types.

    Broadcast

    		Nonbroadcast

    		Point-to-Point

    		Point-to-Multipoint
    Default network type for LAN interfaces

    		Default network type for Frame Relay serial interfaces

    		Default network type for non-Frame Relay serial interfaces

    		Can be configured for any interface
    Neighbors discovered

    		Neighbors statically configured

    		Routers form adjacency

    		Neighbors automatically determined
    All routers on same subnet

    		All routers on same subnet

    		Each point-to-point link on a separate subnet

    		All routers on same subnet
    Has a designated router

    		Has a designated router

    		Does not have a dedicated router

    		Does not have a dedicated router

    TSHOOT p. 176

    sh ip route meanings

    Designator

    		Description
    O

    		OSPF intra-area router LSA and network LSA. Networks from within the router’s area, advertised by router and network LSAs.
    O IA

    		OSPS interarea or summary LSA, networks from outside the router’s area but within the OSPF AS. Advertised by summary LSAs.
    O E1

    		Type 1 External Routes, networks from outside the router’s AS, advertised by external LSAs. Calculate the cost of an external route by adding the the external cost to the internal cost of each link the packet crosses. Use this type when multiple ASBRs are advertising an external route to the same AS.

    O E2

    		Type 2 External Routes, networks from outside the router’s AS advertised by external LSAs. The calculated cost is the external cost only. Use this type if only one ASBR is advertising an external route the autonomous system.

    Commands to verify OSPF configuration:
    sh ip ospf int [br]
    From the output you can see that this command give a good overview of “who” is on the other end of an interface. When the brief option is on it gives a summation of the interfaces active in OSPF and their neighbors. This is intended to help debug the interface table.

    [code]
    R2#sh ip ospf int
    Serial0/0/0.23 is up, line protocol is up
    Internet Address 10.1.1.5/30, Area 0
    Process ID 1, Router ID 10.1.1.5, Network Type POINT_TO_POINT, Cost: 64
    Transmit Delay is 1 sec, State POINT_TO_POINT
    Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
    oob-resync timeout 40
    Hello due in 00:00:01
    Supports Link-local Signaling (LLS)
    Index 1/2, flood queue length 0
    Next 0x0(0)/0x0(0)
    Last flood scan length is 2, maximum is 2
    Last flood scan time is 0 msec, maximum is 4 msec
    Neighbor Count is 1, Adjacent neighbor count is 1
    Adjacent with neighbor 10.1.1.9
    Suppress hello for 0 neighbor(s)
    [/code]

    [code]
    R2#sh ip ospf int br
    Interface PID Area IP Address/Mask Cost State Nbrs F/C
    Se0/0/0.23 1 0 10.1.1.5/30 64 P2P 1/1
    Se0/0/0.12 1 12 10.1.1.2/30 64 P2P 1/1
    [/code]

    sh ip ospf neighbor
    Intended to help debug the neighbor table of OSPF.

    [code]
    R2#sh ip ospf neigh

    Neighbor ID Pri State Dead Time Address Interface
    10.1.1.9 0 FULL/ – 00:00:39 10.1.1.6 Serial0/0/0.23
    209.65.200.225 0 FULL/ – 00:00:31 10.1.1.1 Serial0/0/0.12
    [/code]

    sh ip ospf database
    Shows the LSA headers contained in the link state database. Router Link States come from Type 1 LSAs and Net Link States come from Type 2 LSAs.

    [code]
    R2#sh ip ospf database

    OSPF Router with ID (10.1.1.5) (Process ID 1)

    Router Link States (Area 0)

    Link ID ADV Router Age Seq# Checksum Link count
    10.1.1.5 10.1.1.5 429 0x8000066B 0x00805D 2
    10.1.1.9 10.1.1.9 1590 0x80000669 0x002AB0 2

    Summary Net Link States (Area 0)

    Link ID ADV Router Age Seq# Checksum
    10.1.1.0 10.1.1.5 429 0x80000537 0x005D4A
    10.1.1.8 10.1.1.9 1591 0x80000538 0x00F2A7

    Summary ASB Link States (Area 0)

    Link ID ADV Router Age Seq# Checksum
    10.1.4.9 10.1.1.9 1590 0x80000538 0x00CBC6
    209.65.200.225 10.1.1.5 429 0x80000537 0x00D021

    Router Link States (Area 12)

    Link ID ADV Router Age Seq# Checksum Link count
    10.1.1.5 10.1.1.5 429 0x80000667 0x00FC43 2
    209.65.200.225 209.65.200.225 1693 0x8000065E 0x00DAC1 2

    Summary Net Link States (Area 12)

    Link ID ADV Router Age Seq# Checksum
    10.1.1.4 10.1.1.5 436 0x80000539 0x003170
    10.1.1.8 10.1.1.5 436 0x80000539 0x008BD1

    Summary ASB Link States (Area 12)

    Link ID ADV Router Age Seq# Checksum
    10.1.4.9 10.1.1.5 436 0x80000539 0x0064F0

    Type-5 AS External Link States

    Link ID ADV Router Age Seq# Checksum Tag
    10.1.4.4 10.1.4.9 1287 0x800004B2 0x00CEF3 0
    10.1.4.8 10.1.4.9 1287 0x800000FC 0x001E5A 0
    10.2.1.0 10.1.4.9 1287 0x800004B2 0x001EA7 0
    10.2.2.0 10.1.4.9 1287 0x800000FC 0x008AF3 0
    10.2.4.12 10.1.4.9 1287 0x800004B8 0x00664D 0
    209.65.200.224 209.65.200.225 1693 0x80000536 0x004AB7 0
    209.65.200.240 209.65.200.225 1697 0x80000536 0x00F319 65002
    [/code]

    sh ip ospf statistics
    Shows the last time SPF algorithm was run, how often and the reason.

    [code]
    R2#sh ip ospf statistics

    OSPF Router with ID (10.1.1.5) (Process ID 1)

    Area 0: SPF algorithm executed 34 times

    Area 12: SPF algorithm executed 25 times

    Summary OSPF SPF statistic

    SPF calculation time
    Delta T Intra D-Intra Summ D-Summ Ext D-Ext Total Reason
    4w3d 0 0 0 0 0 0 0 R, SN, X
    4w3d 0 0 0 0 0 0 0 X
    4w3d 0 0 0 0 0 0 0 X
    4w3d 0 0 0 0 0 0 0 R, X
    4w3d 0 0 0 0 0 0 0 R, N, SN, SA, X
    4w3d 0 0 0 0 0 0 0 R,
    4w3d 0 0 0 0 0 0 0 R,
    4w3d 0 0 0 0 0 0 0 R, N, SN, SA, X
    4w3d 0 0 0 0 0 0 0 R,
    4w3d 0 0 0 0 0 0 0 R, X

    RIB manipulation time during SPF (in msec):
    Delta T RIB Update RIB Delete
    4w3d 0 0
    4w3d 0 0
    4w3d 0 0
    4w3d 0 0
    4w3d 0 0
    4w3d 0 0
    4w3d 0 0
    4w3d 0 0
    4w3d 0 0
    4w3d 0 0
    [/code]

    sh ip ospf border-routers
    Who is sending what type of LSA?

    [code]
    R2#sh ip ospf border-routers

    OSPF Process 1 internal Routing Table

    Codes: i – Intra-area route, I – Inter-area route

    i 10.1.1.9 [64] via 10.1.1.6, Serial0/0/0.23, ABR, Area 0, SPF 34
    i 209.65.200.225 [64] via 10.1.1.1, Serial0/0/0.12, ASBR, Area 12, SPF 25
    I 10.1.4.9 [128] via 10.1.1.6, Serial0/0/0.23, ASBR, Area 0, SPF 34
    [/code]

    sh ip route ospf
    Shows what routes are being advertised through OSPF.
    [code]
    R2#sh ip route ospf
    209.65.200.0/24 is variably subnetted, 2 subnets, 2 masks
    O E2 209.65.200.240/29 [110/1] via 10.1.1.1, 4w3d, Serial0/0/0.12
    O E2 209.65.200.224/30 [110/1] via 10.1.1.1, 4w3d, Serial0/0/0.12
    10.0.0.0/8 is variably subnetted, 8 subnets, 2 masks
    O E2 10.2.4.12/30 [110/20] via 10.1.1.6, 4w0d, Serial0/0/0.23
    O IA 10.1.1.8/30 [110/128] via 10.1.1.6, 4w3d, Serial0/0/0.23
    O E2 10.1.4.8/30 [110/20] via 10.1.1.6, 5d20h, Serial0/0/0.23
    O E2 10.2.1.0/24 [110/20] via 10.1.1.6, 3w6d, Serial0/0/0.23
    O E2 10.2.2.0/24 [110/20] via 10.1.1.6, 5d20h, Serial0/0/0.23
    O E2 10.1.4.4/30 [110/20] via 10.1.1.6, 3w6d, Serial0/0/0.23
    [/code]

    Commands to debug OSPF:
    log-adjacency-changes
    debug ip packet
    debug ip ospf events
    Watch the OSPF process in real time.
    [code]
    R2#debug ip ospf events
    OSPF events debugging is on
    R2#
    *Apr 12 19:19:50.000: OSPF: Rcv hello from 10.1.1.9 area 0 from Serial0/0/0.23 10.1.1.6
    *Apr 12 19:19:50.000: OSPF: End of hello processing
    *Apr 12 19:19:50.388: OSPF: Send hello to 224.0.0.5 area 0 on Serial0/0/0.23 from 10.1.1.5
    *Apr 12 19:19:50.528: OSPF: Send hello to 224.0.0.5 area 12 on Serial0/0/0.12 from 10.1.1.2
    [/code]

    Posted in Routing | Leave a comment

    Layer 3 Troubleshooting

    Posted on April 4, 2010 by Judson Bishop

    Cisco Express Frowarding (CEF)
    The two main components of CEF are the forwarding information base (FIB) and the adjacency table. IP routing table is maintained by a router’s control plane. CEF operates in the data plane.

  • Forwarding Information Base (FIB) — Conceptually similar to a routing table, however, it is structured in a format that is optimized for forwarding. It is even updated when changes occur in the network and there is a one-to-one correlation between FIB entries and routing table entries.

  • Adjacency Table — Maintains layer 2 next-hop addresses for all FIB entries. Adjacencies can be added manually or discovered through protocols such as ARP or OSPF.

    • sh ip cef
    Shows layer 3 forwarding information. Version is the version of the table and epoch is the number of times the table has been rebuilt.

    [code]
    R1#sh ip cef 10.1.1.1
    10.1.1.1/32, version 42, epoch 0, receive
    [/code]

    clear ip cef epoch
    Here you can see that I rebuild the CEF table and increase the epoch count.
    [code]
    R1#sh ip cef 10.1.1.1
    10.1.1.1/32, version 42, epoch 0, receive
    R1#clear ip cef epoch
    R1#sh ip cef 10.1.1.1
    10.1.1.1/32, version 42, epoch 1, receive
    [/code]

    sh ip cef 10.0.0.0 255.0.0.0 longer-prefixes
    Attached, obviously the subnet is attached to this router. Receive, the router listens on that subnet or address.
    [code]
    R1#sh ip cef 10.0.0.0 255.0.0.0 longer-prefixes
    Prefix Next Hop Interface
    10.1.1.0/30 attached Serial0/0/0.12
    10.1.1.0/32 receive
    10.1.1.1/32 receive
    10.1.1.3/32 receive
    10.1.1.4/30 10.1.1.2 Serial0/0/0.12
    10.1.1.8/30 10.1.1.2 Serial0/0/0.12
    10.1.4.4/30 10.1.1.2 Serial0/0/0.12
    10.1.4.8/30 10.1.1.2 Serial0/0/0.12
    10.2.1.0/24 10.1.1.2 Serial0/0/0.12
    10.2.2.0/24 10.1.1.2 Serial0/0/0.12
    10.2.4.12/30 10.1.1.2 Serial0/0/0.12
    [/code]

    sh adjacency
    Verifies an adjacency exists for a connected host.

    [code]
    R1#sh adjacency
    Protocol Interface Address
    IP Serial0/0/0.12 point2point(19)
    IPV6 Serial0/0/0.12 point2point(10)
    IP Serial0/1/0.15 point2point(7)
    [/code]

    Troubleshooting Layer 2 to Layer 3 Mapping
    sh ip arp — Shows the ARP cache of the router, by default stores them for four hours. IT is the MAC address to IP address mapping.
    sh frame-relay map — Displays the Frame Relay DLCIs and their next hop ip address.
    sh adjacency detail — Displays information from a router CEF adjacency table.

    sh ip arp
    [code]
    R4#sh ip arp
    Protocol Address Age (min) Hardware Addr Type Interface
    Internet 10.1.4.10 207 0018.1825.2543 ARPA FastEthernet0/1
    Internet 10.1.4.9 – 001b.d421.480b ARPA FastEthernet0/1
    Internet 10.1.4.6 41 0012.d9a5.1542 ARPA FastEthernet0/0
    Internet 10.1.4.5 – 001b.d421.480a ARPA FastEthernet0/0
    [/code]

    sh frame-relay map
    [code]
    R1#sh frame-relay map
    Serial0/0/0.12 (up): point-to-point dlci, dlci 102(0x66,0x1860), broadcast
    status defined, active
    Serial0/1/0.15 (up): point-to-point dlci, dlci 105(0x69,0x1890), broadcast
    status defined, active
    [/code]

    sh adjacency detail
    [code]
    R1#sh adjacency detail
    Protocol Interface Address
    IP Serial0/0/0.12 point2point(19)
    25 packets, 1940 bytes
    18610800
    CEF expires: 00:02:35
    refresh: 00:00:35
    Epoch: 0
    IPV6 Serial0/0/0.12 point2point(10)
    0 packets, 0 bytes
    186186DD
    IPv6 CEF never
    Epoch: 0
    IP Serial0/1/0.15 point2point(7)
    32 packets, 3955 bytes
    18910800
    CEF expires: 00:02:26
    refresh: 00:00:26
    Epoch: 0
    [/code]

    CEF Overview

    Posted in Routing | Leave a comment

    Cisco Express Forwarding and Layer 2 to Layer 3

    Posted on April 4, 2010 by Judson Bishop

    2010-04-04 14:21:36

    Cisco Express Forwarding (CEF)
    The two main components of CEF are the forwarding information base (FIB) and the adjacency table. IP routing table is maintained by a router’s control plane. CEF operates in the data plane.

  • Forwarding Information Base (FIB) — Conceptually similar to a routing table, however, it is structured in a format that is optimized for forwarding. It is even updated when changes occur in the network and there is a one-to-one correlation between FIB entries and routing table entries.

  • Adjacency Table — Maintains layer 2 next-hop addresses for all FIB entries. Adjacencies can be added manually or discovered through protocols such as ARP or OSPF.

    • sh ip cef
    Shows layer 3 forwarding information. Version is the version of the table and epoch is the number of times the table has been rebuilt.

    [code]
    R1#sh ip cef 10.1.1.1
    10.1.1.1/32, version 42, epoch 0, receive
    [/code]

    clear ip cef epoch
    Here you can see that I rebuild the CEF table and increase the epoch count.
    [code]
    R1#sh ip cef 10.1.1.1
    10.1.1.1/32, version 42, epoch 0, receive
    R1#clear ip cef epoch
    R1#sh ip cef 10.1.1.1
    10.1.1.1/32, version 42, epoch 1, receive
    [/code]

    sh ip cef 10.0.0.0 255.0.0.0 longer-prefixes
    Attached, obviously the subnet is attached to this router. Receive, the router listens on that subnet or address.
    [code]
    R1#sh ip cef 10.0.0.0 255.0.0.0 longer-prefixes
    Prefix Next Hop Interface
    10.1.1.0/30 attached Serial0/0/0.12
    10.1.1.0/32 receive
    10.1.1.1/32 receive
    10.1.1.3/32 receive
    10.1.1.4/30 10.1.1.2 Serial0/0/0.12
    10.1.1.8/30 10.1.1.2 Serial0/0/0.12
    10.1.4.4/30 10.1.1.2 Serial0/0/0.12
    10.1.4.8/30 10.1.1.2 Serial0/0/0.12
    10.2.1.0/24 10.1.1.2 Serial0/0/0.12
    10.2.2.0/24 10.1.1.2 Serial0/0/0.12
    10.2.4.12/30 10.1.1.2 Serial0/0/0.12
    [/code]

    sh adjacency
    Verifies an adjacency exists for a connected host.

    [code]
    R1#sh adjacency
    Protocol Interface Address
    IP Serial0/0/0.12 point2point(19)
    IPV6 Serial0/0/0.12 point2point(10)
    IP Serial0/1/0.15 point2point(7)
    [/code]

    Troubleshooting Layer 2 to Layer 3 Mapping
    sh ip arp — Shows the ARP cache of the router, by default stores them for four hours. IT is the MAC address to IP address mapping.
    sh frame-relay map — Displays the Frame Relay DLCIs and their next hop ip address.
    sh adjacency detail — Displays information from a router CEF adjacency table.

    sh ip arp
    [code]
    R4#sh ip arp
    Protocol Address Age (min) Hardware Addr Type Interface
    Internet 10.1.4.10 207 0018.1825.2543 ARPA FastEthernet0/1
    Internet 10.1.4.9 – 001b.d421.480b ARPA FastEthernet0/1
    Internet 10.1.4.6 41 0012.d9a5.1542 ARPA FastEthernet0/0
    Internet 10.1.4.5 – 001b.d421.480a ARPA FastEthernet0/0
    [/code]

    sh frame-relay map
    [code]
    R1#sh frame-relay map
    Serial0/0/0.12 (up): point-to-point dlci, dlci 102(0x66,0x1860), broadcast
    status defined, active
    Serial0/1/0.15 (up): point-to-point dlci, dlci 105(0x69,0x1890), broadcast
    status defined, active
    [/code]

    sh adjacency detail
    [code]
    R1#sh adjacency detail
    Protocol Interface Address
    IP Serial0/0/0.12 point2point(19)
    25 packets, 1940 bytes
    18610800
    CEF expires: 00:02:35
    refresh: 00:00:35
    Epoch: 0
    IPV6 Serial0/0/0.12 point2point(10)
    0 packets, 0 bytes
    186186DD
    IPv6 CEF never
    Epoch: 0
    IP Serial0/1/0.15 point2point(7)
    32 packets, 3955 bytes
    18910800
    CEF expires: 00:02:26
    refresh: 00:00:26
    Epoch: 0
    [/code]

    CEF Overview

    Posted in Routing | Leave a comment

    Layer 3 Switch Troubleshooting

    Posted on April 2, 2010 by Judson Bishop

    The big topics of this chapter include HSRP, VRRP, GLBP, TCAM and the port error table. I broke out my notes from the BCMSN and I’ll summarize each topic from them, then move into the troubleshooting of each protocol.

    Hot Standby Router Protocol (HSRP)
    Cisco proprietary protocol to allow multiple routers to appear as a single gateway IP address. EAch router is assigned to a common HSRP group one active router and one standby router, the others, up to 16 group members, just listen. There is no preemption without configuration.

  • Election of one active router — Based on priority that is assigned when the router is configured for the group. Default priority is 100, range is from 0 – 255, highest priority for election is 255. Highest IP address on HSRP interface breaks ties.

  • Election of one standby router — Second highest priority. Listens to ehloo messages, by default every 3 seconds with hold time of 10 seconds.

  • Virtual router — IP and MAC address pair that end devices use as a gateway. The active router processes all packets and frames to this address, the virtual router processes no physical frames.

    • Troubleshoot
    sh standby …
    sh standby delay

    sh standby
    [code]
    DSW1#sh standby
    Vlan10 – Group 10
    State is Active
    Virtual IP address is 10.2.1.254
    Active virtual MAC address is 0000.0c07.ac0a
    Local virtual MAC address is 0000.0c07.ac0a (v1 default)
    Hello time 3 sec, hold time 10 sec
    Next hello sent in 0.704 secs
    Preemption enabled
    Active router is local
    Standby router is 10.2.1.2, priority 100 (expires in 10.480 sec)
    Priority 200 (configured 200)
    Group name is "hsrp-Vl10-10" (default)
    [/code]

    sh standby br
    [code]
    DSW1#sh standby br
    P indicates configured to preempt.
    |
    Interface Grp Pri P State Active Standby Virtual IP
    Vl10 10 200 P Active local 10.2.1.2 10.2.1.254
    [/code]

    sh standby delay
    [code]
    DSW1#sh standby delay
    Interface Minimum Reload
    Vlan10 1 5
    [/code]

    Virtual Router Redundancy Protocol (VRRP)
    VRRP is on open protocol that provides redundancy for the real IP address of a router or the virtual address of a group. By default all VRRP routers are configured to preempt the current master if their priority is greater.

  • Election of active router — If a real IP address is used, the master is the router with that IP. If a virtual one is used the master becomes the one with the highest priority, all others are in backup state. VRRP group numbers are from 0 to 255, router priority is from 1 to 254 with 254 being the highest and default priority is 100.

    • Troubleshoot
    sh vrrp
    sh vrrp brief

    sh vrrp
    [code]
    R7#sh vrrp
    FastEthernet0/0 – Group 1
    "Just playing"
    State is Master
    Virtual IP address is 10.2.1.253
    Virtual MAC address is 0000.5e00.0101
    Advertisement interval is 1.000 sec
    Preemption enabled
    Priority is 200
    Master Router is 10.2.1.247 (local), priority is 200
    Master Advertisement interval is 1.000 sec
    Master Down interval is 3.218 sec
    [/code]

    sh vrrp brie
    [code]
    R8#sh vrrp br
    Interface Grp Pri Time Own Pre State Master addr Group addr
    Fa0/0 1 150 3414 Y Backup 10.2.1.247 10.2.1.253
    [/code]

    Gateway Load Balancing Protocol (GLBP)
    Cisco proprietary protocol that is more robust than other redundancy protocols. When a client sends an ARP request looking for the router address, GLBP send back a reply with the virtual MAC supported by one of the routers in the GLBP group.

  • GLBP Active Virtual Gateway (GLBP-AVG) — Group members elect the AVG which replies to all host ARP requests. It has either the highest priority value or the highest IP address in the group is the tie breaker. Priority ranges from 0 to 255 with 255 being highest, 100 is the default. AVG assigns the virtual router address and assigns the secondary roles. Up to four virtual MAC address can be used in any group.
  • GLPB Active Virtual Forwarder (GLBP-AVF) — Each gateway assumes responsibility for forwarding packets that are sent to the virtual MAC address assigned to that gateway. GLBP load balances across the AVFs in round robin by default. AVFs are expected to send hellos to every other GLBP peer. If the hold time on the AVG expires without receiving a hello packet from an AVF the virtual MAC is assigned to another router.

    • Troubleshoot:
    sh glbp [brief]

    sh glbp
    [code]
    R8(config-if)#do sh glbp
    FastEthernet0/0 – Group 1
    State is Active
    2 state changes, last state change 00:03:58
    Virtual IP address is 10.2.1.252
    Hello time 3 sec, hold time 10 sec
    Next hello sent in 1.776 secs
    Redirect time 600 sec, forwarder time-out 14400 sec
    Preemption enabled, min delay 0 sec
    Active is local
    Standby is 10.2.1.247, priority 150 (expires in 7.744 sec)
    Priority 200 (configured)
    Weighting 100 (default 100), thresholds: lower 1, upper 100
    Load balancing: round-robin
    Group members:
    0004.dd69.fd01 (10.2.1.248) local
    00d0.bbef.64c1 (10.2.1.247)
    There are 2 forwarders (1 active)
    Forwarder 1
    State is Active
    1 state change, last state change 00:03:48
    MAC address is 0007.b400.0101 (default)
    Owner ID is 0004.dd69.fd01
    Redirection enabled
    Preemption enabled, min delay 30 sec
    Active is local, weighting 100
    Forwarder 2
    State is Listen
    MAC address is 0007.b400.0102 (learnt)
    Owner ID is 00d0.bbef.64c1
    Redirection enabled, 597.584 sec remaining (maximum 600 sec)
    Time to live: 14397.584 sec (maximum 14400 sec)
    Preemption enabled, min delay 30 sec
    Active is 10.2.1.247 (primary), weighting 100 (expires in 7.584 sec)
    [/code]

    Ternary Content Addressable Memory (TCAM):
    TCAM works with CEF to make extremely fast forwarding decisions. TCAM entries are composed of Value, Mask and Result combinations.

  • Values — Always 134 bits consisting of source and destination addresses along with relevant protocol infromation.
  • Masks — Also 134 bits set to match value bits, or not match value bits that do not matter. This is so that a switch can do a bitwise and with access lists or access control entities (ACE).
  • Results — Numerical values that represent what action to take after the TCAM lookup.

    • When the TCAM cannot process the packet it is punted to the CPU for processing. Here are some reasons why a packet is punted:

  • Protocols that send multicast or broadcast packets.
  • Administrative access to a switch such as telnet or ssh.
  • Packets that use features not supported in hardware.
  • A switch’s TCAM has reached capacity. TCAM might reach capacity with large routing tables or a large number of ACLs.

    Troubleshoot TCAM:
    3550:
    show tcam
    3560:
    show platform tcam
    show platform ip unicast counts

    3550#sh tcam inacl 1 statistics
    [code]
    3550#sh tcam inacl 1 statistics
    Ingress ACL TCAM#1: Number of active labels: 3
    Ingress ACL TCAM#1: Number of masks allocated: 14, available: 194
    Ingress ACL TCAM#1: Number of entries allocated: 17, available: 1647
    [/code]

    3560#sh platform tcam utilization
    [code]
    3560#sh platform tcam utilization

    CAM Utilization for ASIC# 0 Max Used
    Masks/Values Masks/values

    Unicast mac addresses: 544/4352 25/76
    IPv4 IGMP groups + multicast routes: 144/1152 6/26
    IPv4 unicast directly-connected routes: 544/4352 25/76
    IPv4 unicast indirectly-connected routes: 176/1408 15/69
    IPv6 Multicast groups: 544/4352 25/76
    IPv6 unicast directly-connected routes: 544/4352 25/76
    IPv6 unicast indirectly-connected routes: 262/2096 11/42
    IPv4 policy based routing aces: 256/256 2/2
    IPv4 qos aces: 768/768 324/324
    IPv4 security aces: 512/512 27/27
    IPv6 policy based routing aces: 0/0 0/0
    IPv6 qos aces: 0/0 0/0
    IPv6 security aces: 204/510 5/5

    Note: Allocation of TCAM entries per feature uses
    a complex algorithm. The above information is meant
    to provide an abstract view of the current TCAM utilization
    [/code]

    Port Error Table:

    Error

    		Description

    		Cause
    Xmit-Err

    		Transmit buffer overlflow

    		Check for speed mismatch.
    Rcv-Err

    		Receive buffer overflow

    		Check for duplex mismatch.
    UnderSize

    		Has valid checksum but less than 64 bytes.

    		Host is sending invalid frames.
    Single-Col

    		A single collision occurred before the port transmitted the frame.

    		High bandwidth utilization or duplex mismatch.
    Multi-Col

    		Multiple collisions occur before the port transmitted the frame.

    		High bandwidth utilization or duplex mismatch.
    Late-Col

    		Collision detected further into frame being forwarded.

    		Cable too long or duplex mismatch.
    Excess-Col

    		Frame has experienced sixteen successive collisions, resulting in the frame being dropped.

    		High bandwidth, duplex mismatch or too many devices on a segment.
    Carri-Sen

    		Normal operation on a half-duplex link.

    		This is not a problem.
    Runts

    		Frames smaller than 64 bytes with a bad CRC.

    		Duplex mismatch or layer 1 problem.
    Giants

    		Frame size greater than 1518 bytes with a bad FCS.

    		Bad host NIC.

    Source: TSHOOT p.125

    • Posted in Routing | Leave a comment

    CCIE Checklist

    Posted on April 1, 2010 by Judson Bishop

    I picked this up from CLN. As the view as pdf option did not work, I just viewed the source and copied it in.

    CCIE Routing and Switching

    Expansion of Routing and Switching Lab v4.0 Blueprint
    Detailed Checklist of Topics to Be Covered

     

    Please be advised that this topic checklist is not an all-inclusive list of Cisco CCIE Routing and Switching lab exam subjects. Instead, we provide this outline as a supplement to the existing lab blueprint to help candidates prepare for their lab exams. Other relevant or related topics may also appear in the actual lab exam.

     

    We would like to get your feedback please comment and/or rate this document.

     


    1.0

    Implementing Layer 2 Technologies

    Configuring and Troubleshooting Layer 2 Technologies
    1.01 Frame Relay
    1.01.1 Frame Relay Multipoint Links on a Physical Interface Using Inverse ARP
    1.01.2 Frame Relay Multipoint Links on a Physical Interface Without Using Inverse ARP
    1.01.3. Frame Relay Multipoint Link on a Subinterface Using Inverse ARP
    1.01.4. Frame Relay Multipoint Link on a Subinterface Without Using Inverse ARP
    1.01.5. Frame Relay Point-to-Point Subinterfaces
    1.01.6. PVC with a Multipoint Interface on One Side and a Subinterface on the Other Side
    1.01.7. Authentication on a Frame Relay Link Using PPP
    1.2 Catalyst Configuration
    1.2.01. Trunks Using an Industry-Standard Encapsulation
    1.2.02. Trunks Using a Cisco Proprietary Encapsulation
    1.2.03. Creating, Deleting, and Editing VLANs
    1.2.04. VTP in Client/Server Mode
    1.2.05. VTP in Transparent Mode
    1.2.06. VTP Authentication
    1.2.07. VTP Pruning
    1.2.08. Controlling VLANs That Cross a Trunk
    1.2.09. Optimizing STP by STP Timers
    1.2.10. PortFast
    1.2.11. Loop Guard
    1.2.12. BPDU Guard
    1.2.13. BPDU Filters
    1.2.14. UplinkFast
    1.2.15. BackboneFast
    1.2.16. MSTP
    1.2.17. Selecting the Root Bridge for VLANs in a PVST Environment
    1.2.18. Selecting the Root Bridge for an MST Instance in an MST Environment
    1.2.19. Setting the Port Priority to Designate the Forwarding Ports
    1.2.20. EtherChannel Using an Industry-Standard Protocol
    1.2.21. EtherChannel Using a Cisco Proprietary Protocol
    1.2.22. Disabling Protocols on the EtherChannel
    1.2.23. Load-Balancing Type on the EtherChannel
    1.2.24. SNMP Management on the Switch
    1.2.25. Telnet and SSH Management on the Switch
    1.2.26. Controlling Inbound and Outbound Telnet on the Switch
    1.2.27. Regular and Smart Macros
    1.2.28. Switch Banners
    1.2.29. UDLD
    1.2.30. Switch Virtual Interfaces (SVIs) for IP Routing
    1.2.31. Router on a Stick
    1.2.32. SPAN
    1.2.33. RSPAN
    1.2.34. IP Routing on the Switch Using RIPv2, EIGRP, OSPF, and BGP
    1.2.35. IP Phones to Connect to the Catalyst Switch
    1.2.36. Dot1q Tunneling
    1.3 Other Layer 2 Technologies
    1.3.1. HDLC
    1.3.2. PPP
    1.3.3. PPP over Ethernet
    2.0

    Implementing IPv4

    Configuring and Troubleshooting IPv4
    2.1. IPv4 Addressing
    2.1.1. IPv4 Addressing
    2.1.2. IPv4 Subnetting
    2.1.3. IPv4 VLSM
    2.2. OSPFv2
    2.2.01. OSPF on a Broadcast Multicast Access Network (Ethernet)
    2.2.02. OSPF over a Frame Relay Multipoint Network by Changing Network Types
    2.2.03. OSPF over a Frame Relay Multipoint Network by Using the neighbor Command
    2.2.04. OSPF over a Frame Relay Point-to-Point Network
    2.2.05. Virtual Links
    2.2.06. Stub Areas
    2.2.07. Totally Stubby Areas
    2.2.08. NSSA Areas
    2.2.09. NSSA and Stub Areas
    2.2.10. NSSA and Totally Stubby Areas
    2.3. EIGRP
    2.3.1. Basic EIGRP
    2.3.2. Passive Interfaces
    2.3.3. EIGRP Stub on Routers and Switches
    2.3.4. EIGRP UpdateóBandwidth Control
    2.3.5. Changing the Administrative Distance of EIGRP
    2.3.6. Unequal-Cost Load Balancing for EIGRP
    2.4. Filtering, Redistribution, and Summarization
    2.4.01. Route Filtering for OSPF Within the Area Using a Distribute List with an ACL and Prefix Lists
    2.4.02. Route Filtering for OSPF Between Areas
    2.4.03. Summarization of OSPF Routes Between Areas
    2.4.04. Summarization of External Routers Within OSPF
    2.4.05. Filtering with a Distribute List Using an ACL and Prefix Lists
    2.4.06. Using Advanced ACLs and a Prefix List for Filtering Routes
    2.4.07. Summarizing Routes with EIGRP
    2.4.08. Route Summarization for RIP
    2.4.09. Redistribution Between OSPF and EIGRP
    2.4.10. Redistribution Between RIP and EIGRP
    2.4.11. Redistribution of Directly Connected Routes
    2.4.12. Redistribution of Static Routes
    2.4.13. Redistribution with Filtering Using ACLs and Prefix Lists
    2.4.14. Redistribution with Filtering Using Route Tagging
    2.5. IBGP
    2.5.1. IBGP Peering
    2.5.2. Advertising Routes in BGP
    2.5.3. Next-Hop Attribute
    2.5.4. Route Reflectors
    2.5.5. Redundancy by Neighbor Relationships Based on Loopbacks
    2.6. EBGP
    2.6.1. EBGP Peering
    2.6.2. EBGP Peering Based on Loopbacks
    2.7. BGP Advanced Features
    2.7.01. Filtering Using ACLs
    2.7.02. Filtering Using Prefix Lists
    2.7.03. Filtering Using AS Path Filters
    2.7.04. Redistributing Connected Routes into BGP
    2.7.05. Redistributing Dynamic Routing Protocols into BGP
    2.7.06. BGP Aggregation
    2.7.07. BGP Aggregation with the Summary Only Parameter
    2.7.08. BGP Aggregation with Suppress Maps
    2.7.09. BGP Aggregation with Unsuppress Maps
    2.7.10. BGP Best-Path Selection ñ Weight
    2.7.11. BGP Best-Path Selection ñ Local Preference
    2.7.12. BGP Best-Path Selection ñ MED
    2.7.13. BGP Communities ñ No-Export
    2.7.14. BGP Communities ñ No-Advertise
    2.7.15. BGP Confederation
    2.7.16. BGP Local AS
    2.7.17. Working with Private AS Numbers
    2.7.18. Route Dampening
    2.7.19. Conditional Advertising
    2.7.20. Peer Groups
    3.0

    Implementing IPv6

    Configuring and Troubleshooting IPv6
    3.1. IPv6
    3.1.1. IPv6 Addresses
    3.1.2. OSPFv3
    3.1.3. EIGRPv6
    3.1.4. IPv6 Tunneling
    3.1.5. IPv6 on a Frame Relay Network ñ Multipoint
    3.1.6. IPv6 on a Frame Relay Network ñ Point-to-Point
    3.1.7. Route Filtering with a Distribute List Using an ACL and Prefix Lists
    3.1.8. Route Redistribution Between OSPFv3 and EIGRPv6
    4.0

    Implementing MPLS

    Configuring and Troubleshooting MPLS
    4.1. MPLS Unicast Routing
    4.1.1. MPLS Unicast Routing Using LDP
    4.1.2. Controlling Label Distribution
    4.2. MPLS VPN
    4.2.1. MPLS VPN Using Static Routing Between PE-CE
    4.2.2. MPLS VPN Using EIGRP as the PE-CE Routing Protocol
    4.2.3. MPLS VPN Using OSPF as the PE-CE Routing Protocol
    4.2.4. MPLS VPN Using EBGP as the PE-CE Routing Protocol
    4.2.5. Controlling Route Propagation Using the Route Target with Import and Export Maps
    4.3. VRF-Lite
    4.3.1. VRFs at the Customer Sites Using VRF-Lite
    5.0

    Implementing IP Multicast

    Configuring and Troubleshooting IP Multicast
    5.1. PIM and Bidirectional PIM
    5.1.1. PIM Dense Mode
    5.1.2. PIM on an NMBA Network
    5.1.3. PIM Sparse Mode ñ Static Rendezvous Point
    5.1.4. PIM Sparse Mode ñ Multiple Static Rendezvous Points
    5.1.5. PIM Sparse Mode ñ Auto Rendezvous Point
    5.1.6. PIM Sparse Mode with Multiple Rendezvous Points Using the Auto Rendezvous Point
    5.1.7. Bidirectional PIM
    5.2. MSDP
    5.2.1. MSDP
    5.2.2. MSDP to an Anycast Rendezvous Point
    5.3. Multicast Tools
    5.3.1. Multicast Rate Limiting
    5.3.2. IGMP Filtering on the Switch
    5.3.3. Use of the Switch to Block Multicast Traffic
    5.3.4. Multicasting Through a GRE Tunnel
    5.3.5. Multicast Helper Address
    5.4. IPv6 Multicast
    5.4.1. IPv6 Multicast Routing Using PIM
    5.4.2. IPv6 Multicast Listener Discovery (MLD) Protocol
    6.0

    Implementing Network Security

    Configuring and Troubleshooting Network Security
    6.1. AAA and Security Server Protocols
    6.1.1. Use of a Router to Authenticate Against a AAA Server Using TACACS+
    6.1.2. Use of a Router to Authenticate Against a AAA Server Using RADIUS
    6.1.3. Local Privilege Authorization
    6.1.4. Accounting to a AAA Server Using TACACS+
    6.1.5. Accounting to a AAA Server Using RADIUS
    6.2. Access Lists
    6.2.1. Standard Access Lists
    6.2.2. Extended Access Lists
    6.2.3. Time-Based Access Lists
    6.2.4. Reflexive Access Lists
    6.3. Routing Protocol Security
    6.3.1. Routing Protocol Authentication for EIGRP
    6.3.2. Routing Protocol Authentication for OSPF ñ Area-Wide
    6.3.3. Routing Protocol Authentication for OSPF ñ Interface-Specific
    6.3.4. Routing Protocol Authentication for OSPF Virtual Links
    6.3.5. Routing Protocol Authentication for BGP
    6.4. Catalyst Security
    6.4.1. Storm Control
    6.4.2. Switch Port Security
    6.4.3. Dot1x Authentication
    6.4.4. Dot1x Authentication for VLAN Assignment
    6.4.5. VLAN Access Maps
    6.4.6. DHCP Snooping
    6.4.7. DAI
    6.4.8. IP Source Guard
    6.4.9. Private VLANs
    6.5. Cisco IOS and Zone-Based Firewalls
    6.5.1. Basic Cisco IOS Firewall
    6.5.2. DoS Protection on a Cisco IOS Firewall
    6.5.3. Basic Zone-Based Firewall
    6.5.4. Zone-Based Firewall with Deep Packet Inspection
    6.6. NAT
    6.6.1. Dynamic NAT
    6.6.2. PAT
    6.6.3. Static NAT
    6.6.4. Static PAT
    6.6.5. Policy-Based NAT
    6.7. Other Security Features
    6.7.1. Configuring the TCP Intercept Feature
    6.7.2. Configuring Blocking of Fragment Attacks
    6.7.3. Configuring Switch Security Features
    6.7.4. Configuring Antispoofing Using an ACL
    6.7.5. Configuring Antispoofing Using uRPF
    6.7.6. SSH on Routers and Switches
    6.7.7. Cisco IOS IPS
    6.7.8. Controlling Telnet and SSH Access to the Router and Switch
    7.0

    Implementing Network Services

    Configuring and Troubleshooting Network Services
    7.1. DHCP
    7.1.1. Configuring DHCP on a Cisco IOS Router
    7.1.2. Configuring DHCP on a Switch
    7.1.3. Using a Router and a Switch to Act as a DHCP Relay Agent (Helper Address)
    7.2. HSRP
    7.2.1. HSRP Between Two Routers
    7.2.2. Pre-empt for HSRP
    7.2.3. Authentication for HSRP
    7.2.4. VRRP
    7.2.5. GLBP
    7.3. IP Services
    7.3.1. Use of the Router for WCCP
    7.3.2. Use of the Router to Generate an Exception Dump Using TFTP
    7.3.3. Use of the Router to Generate an Exception Dump Using FTP
    7.3.4. Use of the Router to Generate an Exception Dump Using RCP
    7.3.5. Broadcast Forwarding for Protocols
    7.4. System Management
    7.4.1. Telnet Management on the Router and Switch
    7.4.2. SSH Management on the Router and Switch
    7.4.3. Disabling Telnet and the SSH Client on the Switch
    7.4.4. HTTP Management on the Router and Switch
    7.4.5. Controlling HTTP Management on the Router and Switch
    7.5. NTP
    7.5.1. NTP Using the NTP Master and NTP Server Commands
    7.5.2. NTP Without Using the NTP Server
    7.5.3. NTP Using NTP Broadcast Commands
    8.0

    Implementing QoS

    Configuring and Troubleshooting QoS
    8.1. Classification
    8.1.1. Marking Using DSCP
    8.1.2. Marking Using IP Precedence
    8.1.3. Marking Using CoS
    8.2. Congestion Management and Congestion Avoidance
    8.2.1. Priority Queuing
    8.2.2. Custom Queuing
    8.2.3. Weighted Fair Queuing
    8.2.4. WRED
    8.2.5. RSVP
    8.3. Policing and Shaping
    8.3.1. CAR Using Rate Limiting Under the Interface
    8.3.2. Frame Relay Traffic Shaping Using Map Classes
    8.3.3. Discard Eligible List
    8.4. Link Efficiency Mechanisms
    8.4.1. Compression
    8.4.2. Link Fragmentation and Interleaving (LFI) for Frame Relay
    8.5. Modular QoS CLI
    8.5.1. Policing
    8.5.2. Class-Based Weighted Fair Queuing (CB-WFQ)
    8.5.3. Low Latency Queuing (LLQ)
    8.5.4. Shaping Using MQC
    8.5.5. Random Early Detection Using MQC
    8.5.6. WRED Using MQC
    8.5.7. Using NBAR for QoS
    8.5.8. Discard Eligible Marking Using MQC
    8.6. Catalyst QoS
    8.6.1. WRR on the Catalyst Switch
    9.0

    Troubleshooting a Network

    Troubleshooting Network-Wide Connectivity Issues
    9.1. Troubleshooting Layer 2 Problems
    9.1.1. Troubleshooting Catalyst Switch Network Issues
    9.1.2. Troubleshooting Frame Relay Network Issues
    9.2. Troubleshooting Layer 3 Problems
    9.2.1. Troubleshooting IP Addressing Network Issues
    9.2.2. Troubleshooting Routing Protocol Network Issues
    9.2.3. Troubleshooting Routing Protocol Loop Issues
    9.3. Troubleshooting Application Problems
    9.3.1. Determining Which Aspects of the Network to Troubleshoot to Determine Network Functionality (Given a Set of Symptoms)
    9.4. Troubleshooting Network Services
    9.4.1. Troubleshooting Misconfigured NTP Setup
    9.4.2. Troubleshooting Misconfigured DHCP Setup
    9.4.3. Troubleshooting Misconfigured Telnet and SSH Setup
    9.4.4. Troubleshooting Misconfigured SNMP Setup
    9.5. Troubleshooting Security Services
    9.5.1. Troubleshooting Misconfigured ACLs
    9.5.2. Troubleshooting Misconfigured NAT
    9.5.3. Troubleshooting Misconfigured AAA Services
    10.0

    Optimizing a Network

    Configuring and Troubleshooting Optimization of a Network
    10.1. Logging In
    10.1.1. Logging into a Remote Syslog Server
    10.1.2. Logging into the Internal Buffer
    10.2. SNMP
    10.2.1. Use of a Router to Communicate to an SNMP Management Station
    10.2.2. Use of a Router to Generate SNMP Traps
    10.3. RMON
    10.3.1. Use of a Router to Generate SNMP Traps Using RMON
    10.4. Accounting
    10.4.1. IP Accounting
    10.5. SLA
    10.5.1. IP SLA
    10.6. Implementing Network Services on the Routers
    10.6.1. Use of a Router as an FTP Server
    10.6.2. Use of a Router as a TFTP Server
    10.6.3. Cisco IOS Embedded Event Manager
    10.6.4. NetFlow
    10.6.5. HTTP and HTTPS on a Router
    10.6.6. Telnet on a Router
    10.6.7. Implementing Secure Copy Protocol (SCP) on a Router

     

    Posted in CCIE | Leave a comment

    Making the Transition

    Posted on March 29, 2010 by Judson Bishop

    How do you know you have crossed over from a sysadmin to a network admin?

    When you log on to a Linux server and type “en.”

    There is a story here. I used to login to a router or switch and immediately type “ls” or “uptime.” It would drive me crazy when I would get this dreaded error:
    DSW1#ls
    Translating “ls”…domain server (255.255.255.255)

    And then I would invariably configure:
    no ip domain-lookup
    Or:
    ip ospf name-lookup
    ip name-server 192.168.100.64
    ip name-server 192.168.100.65

    Now when I rarely type “ls” when I log into a router but often type “en” when I log into a server.

    Posted in Linux, Routing, Thoughts | Leave a comment

    What I Learned Today

    Posted on March 28, 2010 by Judson Bishop

    One of my coworkers solved a problem today that I should have been able to solve. I got pulled into a project that uses a closed system with it’s own programming language that I had never seen nor programmed. My coworker was trying to figure out why his syntax was not working. So we cranked up the logging in Warn.log, Error.log and Fatal.log. We quickly figured out why it was failing, the program was looking for a file that did not exist and he was not catching that error, there is no catch statement.

    So I began trying to write an IF/ELSE statement that would check for the error condition. I was given a .pdf of the manual for the system and just started trying different functions that made sense to me. I wrote a nice little one liner in the SYSTEM command that returned TRUE or FALSE if the FILE environment variable existed.

    The problem was that we could not set the environment variable with a variable from the program. It has to be a string literal.

    From the Trace.log:
    [code]
    LET sys = EXPORT("FILE=" & $format)

    // Variable "Part01::sys" is a String
    // [ 1] = ""

    LET sys = SYSTEM("if [ -f /usr/program/fmt/$FILE ]; then echo "TRUE"; else echo "FALSE"; fi")

    // Variable "Part01::sys" is a String
    // [ 1] = "FALSE"

    IF sys = "TRUE" THEN

    // Test result is FALSE – skipping…

    END IF
    [/code]

    We tried a number of different variations on that theme, imported and exported variables to a subshell, tried different return values from the exit command. We got the system to turn circles but what we really wanted was squares.

    Finally my coworker called and said, why don’t we just invert the logic. If it falls through to the end of the IF/ELSE statements it must be in the format we want. Let me take a step back and say this was his code, he should have been able to figure it out. I believe I helped him figure out what his main problem was, which helped him find the solution. I’m not trying to minimize his work, nor am I trying to minimize my input into the solution.

    The moral of the story is that I should have taken a higher level look at the problem to understand the solution. I am quick to believe that I can code my way out of any problem, regardless whether or not I have ever seen the language. What I really needed to do was step back and take a higher level view of the logic, rather than dive into syntax and functions.

    Posted in Code | Leave a comment