Contents
Introduction
Prerequisites
Requirements
Components Used
Conventions
Exstart State
Exchange State
Neighbors Stuck in Exstart/Exchange State
The Solution
Introduction
OSPF states for adjacency formation are Down, Init, Attempt, 2-way, Exstart, Exchange, Loading and Full. There can be number of reasons why the Open Shortest Path First (OSPF) neighbors are stuck in exstart/exchange state. This document focuses on an MTU mismatch between OSPF neighbors resulting in exstart/exchange state. For more details on troubleshooting OSPF refer to Troubleshooting OSPF.
Prerequisites
Requirements
Readers of this document shoud be familar with basic OSPF operation and configuration, especially about OSPF neighbor states.
Components Used
The information in this document is based on these software and hardware versions:
-
Cisco 2503 routers
-
Cisco IOS® Software Release 12.2(24a) running on both routers
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Conventions
For more information on document conventions, refer to the Cisco Technical Tips Conventions.
Exstart State
After two OSPF neighboring routers establish bi-directional communication and complete DR/BDR election (on multi-access networks), the routers transition to the exstart state. In this state, the neighboring routers establish a master/slave relationship and determine the initial database descriptor (DBD) sequence number to use while exchanging DBD packets.
Exchange State
Once the master/slave relationship has been negotiated (the router with the highest Router-ID becomes the master), the neighboring routers transition into the exchange state. In this state, the routers exchange DBD packets, which describe their entire link-state database. The routers also send link-state request packets, which request more recent link-state advertisements (LSA) from neighbors.
Although OSPF neighbors transition through the exstart/exchange states during the normal OSPF adjacency-building process, it is not normal for OSPF neighbors to be stuck in this state. Below is the most common reason that OSPF neighbors get stuck in this state. Refer to OSPF Neighbor States to learn more about the different OSPF states.
Neighbors Stuck in Exstart/Exchange State
The problem occurs most frequently when attempting to run OSPF between a Cisco router and another vendor's router. The problem occurs when the maximum transmission unit (MTU) settings for neighboring router interfaces don't match. If the router with the higher MTU sends a packet larger that the MTU set on the neighboring router, the neighboring router ignores the packet.0 When this problem occurs, the output of the show ip ospf neighbor command displays output similar that shown below.
An actual recreation of this problem is described in this section.
Router 6 and Router 7 in the above topology are connected via Frame Relay and Router 6 has been configured with 5 static routes redistributed into OSPF. The serial interface on Router 6 has the default MTU of 1500, while the serial interface on Router 7 has an MTU of 1450. Each router configuration is shown in the table below (only necessary configuration information is shown):
Router 6 Configuration |
Router 7 Configuration |
---|---|
interface Serial2 !--- MTU sets to it's default value of 1500. no ip address no ip directed-broadcast encapsulation frame-relay no ip mroute-cache frame-relay lmi-type ansi ! interface Serial2.7 point-to-point ip address 170.170.11.6 255.255.255.0 no ip directed-broadcast frame-relay interface-dlci 101 ! router ospf 7 redistribute static subnets network 170.170.11.0 0.0.0.255 area 0 ip route 192.79.34.0 255.255.255.0 Null0 ip route 192.79.35.0 255.255.255.0 Null0 ip route 192.79.36.0 255.255.255.0 Null0 ip route 192.79.37.0 255.255.255.0 Null0 ip route 192.79.38.0 255.255.255.0 Null0 |
! interface Serial0 mtu 1450 no ip address no ip directed-broadcast encapsulation frame-relay frame-relay lmi-type ANSI ! interface Serial0.6 point-to-point ip address 170.170.11.7 255.255.255.0 no ip directed-broadcast frame-relay interface-dlci 110! ! router ospf 7 network 170.170.11.0 0.0.0.255 area 0 |
The show ip ospf neighbor command output for each router is:
router-6# show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface 170.170.11.7 1 EXCHANGE/ - 00:00:36 170.170.11.7 Serial2.7 router-6# router-7# show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface 170.170.11.6 1 EXSTART/ - 00:00:33 170.170.11.6 Serial0.6 router-7#
The problem occurs when Router 6 sends a DBD packet larger than 1450 bytes (Router 7's MTU) while in the exchange state. Use the debug ip packet and the debug ip ospf adj commands on each router to see the OSPF adjacency process as it takes place. The output from Router 6 and 7 from steps 1 to 14 is:
-
Router 6 debug output:
***ROUTER6 IS SENDING HELLOS BUT HEARS NOTHING, STATE OF NEIGHBOR IS DOWN 00:03:53: OSPF: 170.170.11.7 address 170.170.11.7 on Serial2.7 is dead 00:03:53: OSPF: 170.170.11.7 address 170.170.11.7 on Serial2.7 is dead, state DOWN
-
Router 7 debug output:
OSPF NOT ENABLED ON ROUTER7 YET
-
Router 6 debug output:
***ROUTER6 SENDING HELLOS 00:03:53: IP: s=170.170.11.6 (local), d=224.0.0.5 (Serial2.7), len 64, sending broad/multicast, proto=89 00:04:03: IP: s=170.170.11.6 (local), d=224.0.0.5 (Serial2.7), Len 64, sending broad/multicast, proto=89
-
Router 7 debug output:
OSPF NOT ENABLED ON ROUTER7 YET
-
Router 7 debug output:
***OSPF ENABLED ON ROUTER7, BEGINS SENDING HELLOS AND BUILDING A ROUTER LSA 00:17:44: IP: s=170.170.11.7 (local), d=224.0.0.5 (Serial0.6), Len 64, sending broad/multicast, proto=89 00:17:44: OSPF: Build router LSA for area 0, router ID 170.170.11.7, seq 0x80000001
-
Router 6 debug output:
***RECEIVE HELLO FROM ROUTER7 00:04:04: IP: s=170.170.11.7 (Serial2.7), d=224.0.0.5, Len 64, rcvd 0, proto=89 00:04:04: OSPF: Rcv hello from 170.170.11.7 area 0 from Serial2.7 170.170.11.7 00:04:04: OSPF: End of hello processing
-
Router 6 debug output:
***ROUTER6 SEND HELLO WITH ROUTER7 ROUTERID IN THE HELLO PACKET 00:04:13: IP: s=170.170.11.6 (local), d=224.0.0.5 (Serial2.7), Len 68, sending broad/multicast, proto=89
-
Router 7 debug output:
***ROUTER7 RECEIVES HELLO FROM ROUTER6 CHANGES STATE TO 2WAY 00:17:53: IP: s=170.170.11.6 (Serial0.6), d=224.0.0.5, Len 68, rcvd 0, proto=89 00:17:53: OSPF: Rcv hello from 170.170.11.6 area 0 from Serial0.6 170.170.11.6 00:17:53: OSPF: 2 Way Communication to 170.170.11.6 on Serial0.6, state 2WAY
-
Router 7 debug output:
***ROUTER7 SENDS INITIAL DBD PACKET WITH SEQ# 0x13FD 00:17:53: OSPF: Send DBD to 170.170.11.6 on Serial0.6 seq 0x13FD opt 0x2 flag 0x7 Len 32 00:17:53: IP: s=170.170.11.7 (local), d=224.0.0.5 (Serial0.6), Len 52, sending broad/multicast, proto=89 00:17:53: OSPF: End of hello processing
-
Router 6 debug output:
***ROUTER6 RECEIVES ROUTER7'S INITIAL DBD PACKET CHANGES STATE TO 2-WAY 00:04:13: IP: s=170.170.11.7 (Serial2.7), d=224.0.0.5, Len 52, rcvd 0, proto=89 00:04:13: OSPF: Rcv DBD from 170.170.11.7 on Serial2.7 seq 0x13FD opt 0x2 flag 0x7 Len 32 mtu 1450 state INIT 00:04:13: OSPF: 2 Way Communication to 170.170.11.7 on Serial2.7, state 2WAY
-
Router 6 debug output:
***ROUTER6 SENDS DBD PACKET TO ROUTER7 (MASTER/SLAVE NEGOTIATION - ROUTER6 IS SLAVE) 00:04:13: OSPF: Send DBD to 170.170.11.7 on Serial2.7 seq 0xE44 opt 0x2 flag 0x7 Len 32 00:04:13: IP: s=170.170.11.6 (local), d=224.0.0.5 (Serial2.7), Len 52, sending broad/multicast, proto=89 00:04:13: OSPF: NBR Negotiation Done. We are the SLAVE
-
Router 7 debug output:
***RECEIVE ROUTER6'S INITIAL DBD PACKET (MTU MISMATCH IS RECOGNIZED) 00:17:53: IP: s=170.170.11.6 (Serial0.6), d=224.0.0.5, Len 52, rcvd 0, proto=89 00:17:53: OSPF: Rcv DBD from 170.170.11.6 on Serial0.6 seq 0xE44 opt 0x2 flag 0x7 Len 32 mtu 1500 state EXSTART 00:17:53: OSPF: Nbr 170.170.11.6 has larger interface MTU
-
Router 6 debug output:
***SINCE ROUTER6 IS SLAVE SEND DBD PACKET WITH LSA HEADERS, SAME SEQ# (0x13FD) TO ACK ROUTER7'S DBD. (NOTE SIZE OF PKT) 00:04:13: OSPF: Send DBD to 170.170.11.7 on Serial2.7 seq 0x13FD opt 0x2 flag 0x2 Len 1472 00:04:13: IP: s=170.170.11.6 (local), d=224.0.0.5 (Serial2.7), Len 1492, sending broad/multicast, proto=89
-
Router 7 debug output:
***NEVER RECEIVE ACK TO ROUTER7'S INITIAL DBD, RETRANSMIT 00:17:54: IP: s=170.170.11.7 (local), d=224.0.0.5 (Serial0.6), Len 68, sending broad/multicast, proto=89 00:18:03: OSPF: Send DBD to 170.170.11.6 on Serial0.6 seq 0x13FD opt 0x2 flag 0x7 Len 32 00:18:03: OSPF: Retransmitting DBD to 170.170.11.6 on Serial0.6 [1]
At this point, Router 6 keeps trying to ACK the initial DBD packet from Router 7.
00:04:13: IP: s=170.170.11.7 (Serial2.7), d=224.0.0.5, Len 68, rcvd 0, proto=89 00:04:13: OSPF: Rcv hello from 170.170.11.7 area 0 from Serial2.7 170.170.11.7 00:04:13: OSPF: End of hello processing 00:04:18: IP: s=170.170.11.7 (Serial2.7), d=224.0.0.5, Len 52, rcvd 0, proto=89 00:04:18: OSPF: Rcv DBD from 170.170.11.7 on Serial2.7 seq 0x13FD opt 0x2 flag 0x7 Len 32 mtu 1450 state EXCHANGE 00:04:18: OSPF: Send DBD to 170.170.11.7 on Serial2.7 seq 0x13FD opt 0x2 flag 0x2 Len 1472 00:04:18: IP: s=170.170.11.6 (local), d=224.0.0.5 (Serial2.7), Len 1492, sending broad/multicast, proto=89 00:04:23: IP: s=170.170.11.6 (local), d=224.0.0.5 (Serial2.7), Len 68, sending broad/multicast, proto=89 00:04:23: IP: s=170.170.11.7 (Serial2.7), d=224.0.0.5, Len 52, rcvd 0, proto=89 00:04:23: OSPF: Rcv DBD from 170.170.11.7 on Serial2.7 seq 0x13FD opt 0x2 flag 0x7 Len 32 mtu 1450 state EXCHANGE
Router 7 never gets an ACK from Router 6 because the DBD packet from Router 7 is too large for the Router 7 MTU. Router 7 repeatedly retransmits the DBD packet.
0:17:58: IP: s=170.170.11.7 (local), d=224.0.0.5 (Serial0.6), Len 52, sending broad/multicast, proto=89 00:18:03: OSPF: Send DBD to 170.170.11.6 on Serial0.6 seq 0x13FD opt 0x2 flag 0x7 Len 32 00:18:03: OSPF: Retransmitting DBD to 170.170.11.6 on Serial0.6 [2] 00:18:03: IP: s=170.170.11.7 (local), d=224.0.0.5 (Serial0.6), Len 52, sending broad/multicast, proto=89 00:18:03: IP: s=170.170.11.6 (Serial0.6), d=224.0.0.5, Len 68, rcvd 0, proto=89 00:18:03: OSPF: Rcv hello from 170.170.11.6 area 0 from Serial0.6 170.170.11.6 00:18:03: OSPF: End of hello processing 00:18:04: IP: s=170.170.11.7 (local), d=224.0.0.5 (Serial0.6), Len 68, sending broad/multicast, proto=89 00:18:03: OSPF: Send DBD to 170.170.11.6 on Serial0.6 seq 0x13FD opt 0x2 flag 0x7 Len 32 00:18:03: OSPF: Retransmitting DBD to 170.170.11.6 on Serial0.6 [3] 00:18:08: IP: s=170.170.11.7 (local), d=224.0.0.5 (Serial0.6), Len 52, sending broad/multicast, proto=89 router-7#
Because Router 6 has a higher MTU, it continues to accept the DBD packet from Router 7, attempts to acknowledge them, and remains in the EXCHANGE state.
Because Router 7 has a lower MTU, it ignores the DBD packets along with ACK from Router 6, continues to retransmit the initial DBD packet, and remains in the EXSTART state.
Let us look at some of the steps above. In step 9 and 11, Router 7 and Router 6 send their first DBD packets with flag 0x7 set as part of Master/Slave negotiation. After Master/Slave determination, Router 7 is elected as Master because of it's higher Router-ID. The flags in steps 13 and 14 clearly shows that Router 7 is Master (Flag 0x7) and Router 6 is Slave (Flag 0x2).
In step 10, Router 6 receives the Router 7 initial DBD packet and transitions its state to 2-way.
In step 12, Router 7 receives the Router 6 initial DBD packet and recognizes an MTU mismatch. (Router 7 is able to recognize a MTU mismatch because Router 6 includes its interface MTU in the interface MTU field of the DBD packet). The Router 6 initial DBD is rejected by Router 7. Router 7 retransmits the initial DBD packet.
Step 13 shows that Router 6, as slave, adopts the Router 7 sequence number and sends its second DBD packet containing the headers of its LSAs, which increases the size of the packet. However, Router 7 never receives this DBD packet because it is larger than the Router 7 MTU.
After step 13, Router 7 continues to retransmit the initial DBD packet to Router 6, while Router 6 continues to send DBD packets that follow the master sequence number. This loop continues indefinitely, which prevents either router from transition out of the exstart/exchange state.
The Solution
Since the problem is caused by mismatched MTUs, the solution is to change either router MTU to match the neighbor MTU. Note that Cisco IOS does not support a chang of the physical MTU on a LAN interface (such as Ethernet or Token Ring). When the problem occurs between a Cisco router and another vendor router over LAN media, adjust the MTU on the other vendor router.
Note: Cisco IOS Software Release 12.0(3) introduced interface MTU mismatch detection. This detection involves OSPF advertising the interface MTU in the DBD packets, which is in accordance with the OSPF RFC 2178 , appendix G.9. When a router receives a DBD packet advertising a MTU larger than the router can receive, the router ignores the DBD packet and the neighbor state remains in exstart. This prevents the formation of an adjacency. To fix this problem, ensure that the MTU are the same on both ends of a link.
In Cisco IOS Software 12.01(3), the ip ospf mtu-ignore interface configuration command was also introduced to turn off the MTU mismatch detection; however, this is only needed in rare instances, as shown in this diagram:
The above diagram shows a Fiber Distributed Data Interface (FDDI) port on a Cisco Catalyst 5000 with a Route Switch Module (RSM) connected to a FDDI interface on Router 2. The Virtual LAN (VLAN) on the RSM is a virtual Ethernet interface with a MTU of 1500, and the FDDI interface on Router 2 has a MTU of 4500. When a packet is received on the FDDI port of the switch, it goes to the backplane and the FDDI to Ethernet conversion/fragmentation happens within the switch itself. This is a valid setup, but with MTU mismatch detection feature, OSPF adjacency is not formed inbetween the router and the RSM. Since FDDI and Ethernet MTU are different, this ip ospf mtu-ignore command is useful on the VLAN interface of the RSM so that OSPF stops detecting MTU mismatch and forms the adjacency.
It is important to note that MTU mismatch, although the most common, is not the only reason that OSPF neighbors get stuck in the exstart/exchange state. The problem is most frequently caused by the inability to successfully exchange DBD packets. However, the root cause could be any of the these:
-
MTU mismatch
-
Unicast is broken. In the exstart state, the router sends a unicast packet to the neighbor to elect master and slave. This is true unless you have a point-to-point link, in which case it sends a multicast packet. These are the possible causes:
-
Wrong virtual circuit (VC) mapping in an Asynchronous Transfer Mode (ATM) or Frame Relay environment in highly redundant network.
-
MTU problem, meaning the routers can only ping a packet of a certain length.
-
Access list is blocking the unicast packet.
-
NAT is running on the router and is translating the unicast packet.
-
-
Neighbor between PRI and BRI/dialer.
-
Both routers have the same router ID (mis-configuration).
In addition, the OSPF RFC 2328 , section 10.3, states that the exstart/exchange process is initiated for any of these events (any of which could be caused by internal software problems):
-
SequenceNumberMismatch
-
Unexpected DD Sequence number
-
"I" bit is set unexpectedly
-
Option field different from the last option field received in the DBD packet
-
-
BadLSReq
-
Neighbor sends unrecognized LSA during exchange process.
-
Neighbor requested an LSA during exchange process that cannot be found
-
When OSPF does not form neighbors, consider the factors mentioned above, such as the physical media and networking hardware, in order to troubleshoot the problem.