Troubleshooting Guide
Page 3
... and Safety Information i-xxx Hardware Installation i-xxx Cisco Fabric Manager i-xxxi Command-Line Interface i-xxxi Troubleshooting and Reference i-xxxi Installation and Configuration Note i-xxxi Obtaining Documentation, Obtaining Support, and Security Guidelines i-xxxii Troubleshooting Overview 1-1 Overview of the Troubleshooting Process 1-1 Best Practices 1-2 Troubleshooting Basics 1-2 General Steps 1-2 Gathering Information Using Common Fabric Manager Tools and CLI Commands 1-3 Common Fabric Manager Tools 1-3 Common CLI Commands 1-4 Verifying Basic Connectivity 1-4 Verifying SAN Element...
... and Safety Information i-xxx Hardware Installation i-xxx Cisco Fabric Manager i-xxxi Command-Line Interface i-xxxi Troubleshooting and Reference i-xxxi Installation and Configuration Note i-xxxi Obtaining Documentation, Obtaining Support, and Security Guidelines i-xxxii Troubleshooting Overview 1-1 Overview of the Troubleshooting Process 1-1 Best Practices 1-2 Troubleshooting Basics 1-2 General Steps 1-2 Gathering Information Using Common Fabric Manager Tools and CLI Commands 1-3 Common Fabric Manager Tools 1-3 Common CLI Commands 1-4 Verifying Basic Connectivity 1-4 Verifying SAN Element...
Troubleshooting Guide
Page 26
... Alias Services IP Network Simulator Nondisruptive upgrades on the Cisco MDS 9124 Fabric Switch. Appendix B, "Troubleshooting Tools and Methodology" Added information about troubleshooting 3.1(2) nondisruptive upgrades on Cisco MDS 9124 Fabric Switch SAN Device Virtualization SNMP test traps Digital Certificates iSCSI Load Balancing Mixed Generation Hardware AAA Call Home FC-SP, port security, fabric binding IP Access Lists Users and Roles Description Changed in Release Where Documented Updated FICON port numbering information...
... Alias Services IP Network Simulator Nondisruptive upgrades on the Cisco MDS 9124 Fabric Switch. Appendix B, "Troubleshooting Tools and Methodology" Added information about troubleshooting 3.1(2) nondisruptive upgrades on Cisco MDS 9124 Fabric Switch SAN Device Virtualization SNMP test traps Digital Certificates iSCSI Load Balancing Mixed Generation Hardware AAA Call Home FC-SP, port security, fabric binding IP Access Lists Users and Roles Description Changed in Release Where Documented Updated FICON port numbering information...
Troubleshooting Guide
Page 30
... IBM TotalStorage SAN Volume Controller Storage Software for the Cisco MDS 9000 Family, refer to mdsfeedback-doc@cisco.com Related Documentation The documentation set for the Cisco MDS 9000 Family Hardware Installation • Cisco MDS 9124 Multilayer Fabric Switch Quick Start Guide • Cisco MDS 9500 Series Hardware Installation Guide • Cisco MDS 9200 Series Hardware Installation Guide • Cisco MDS 9100 Series Hardware Installation Guide • Cisco MDS 9020 Fabric Switch Hardware Installation Guide Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x xxx OL-9285...
... IBM TotalStorage SAN Volume Controller Storage Software for the Cisco MDS 9000 Family, refer to mdsfeedback-doc@cisco.com Related Documentation The documentation set for the Cisco MDS 9000 Family Hardware Installation • Cisco MDS 9124 Multilayer Fabric Switch Quick Start Guide • Cisco MDS 9500 Series Hardware Installation Guide • Cisco MDS 9200 Series Hardware Installation Guide • Cisco MDS 9100 Series Hardware Installation Guide • Cisco MDS 9020 Fabric Switch Hardware Installation Guide Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x xxx OL-9285...
Troubleshooting Guide
Page 31
... Command-Line Interface • Cisco MDS 9000 Family Software Upgrade and Downgrade Guide • Cisco MDS 9000 Family Storage Services Module Software Installation and Upgrade Guide • Cisco MDS 9000 Family CLI Quick Configuration Guide • Cisco MDS 9000 Family CLI Configuration Guide • Cisco MDS 9000 Family Command Reference • Cisco MDS 9000 Family Quick Command Reference • Cisco MDS 9020 Fabric Switch Configuration Guide and Command Reference • Cisco MDS 9000 Family SAN Volume Controller Configuration Guide Troubleshooting and Reference • Cisco...
... Command-Line Interface • Cisco MDS 9000 Family Software Upgrade and Downgrade Guide • Cisco MDS 9000 Family Storage Services Module Software Installation and Upgrade Guide • Cisco MDS 9000 Family CLI Quick Configuration Guide • Cisco MDS 9000 Family CLI Configuration Guide • Cisco MDS 9000 Family Command Reference • Cisco MDS 9000 Family Quick Command Reference • Cisco MDS 9020 Fabric Switch Configuration Guide and Command Reference • Cisco MDS 9000 Family SAN Volume Controller Configuration Guide Troubleshooting and Reference • Cisco...
Troubleshooting Guide
Page 34
... implementing the change. • Use Fabric Manager and Device Manager to proactively manage your fabric. Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 1-2 OL-9285-05 Best Practices Chapter 1 Troubleshooting Overview Send documentation comments to mdsfeedback-doc@cisco.com Best Practices Best practices are the recommended steps you should investigate further. Troubleshooting Basics This section provides a series of your fabric and detect possible problems before they...
... implementing the change. • Use Fabric Manager and Device Manager to proactively manage your fabric. Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 1-2 OL-9285-05 Best Practices Chapter 1 Troubleshooting Overview Send documentation comments to mdsfeedback-doc@cisco.com Best Practices Best practices are the recommended steps you should investigate further. Troubleshooting Basics This section provides a series of your fabric and detect possible problems before they...
Troubleshooting Guide
Page 51
..., you can view the progress of the startup-config. - See the "Installing Cisco SAN-OS Software from any reason (for the To: field. You can nondisruptively upgrade to an unstable fabric state), then the command sequence disruptively updates that module and ends. This step creates a backup copy of this question (the default is not used inadvertently to continue (y/n) [n] :y - Review the upgrade table in the Cisco SAN-OS...
..., you can view the progress of the startup-config. - See the "Installing Cisco SAN-OS Software from any reason (for the To: field. You can nondisruptively upgrade to an unstable fabric state), then the command sequence disruptively updates that module and ends. This step creates a backup copy of this question (the default is not used inadvertently to continue (y/n) [n] :y - Review the upgrade table in the Cisco SAN-OS...
Troubleshooting Guide
Page 57
... the "Installing Cisco SAN-OS Software from the CLI" section on the incompatibility. The switch experienced a power disruption while the upgrade was in progress. Module failed to mdsfeedback-doc@cisco.com Software Installation Ends with Error Symptom The software installation ends with an error. Restart the installation. Or, use the delete command. Remove unnecessary files from the CLI" section on page 2-11. In Device Manager, choose Admin > Flash Files and delete unnecessary files. The specified system and kickstart images Check the...
... the "Installing Cisco SAN-OS Software from the CLI" section on the incompatibility. The switch experienced a power disruption while the upgrade was in progress. Module failed to mdsfeedback-doc@cisco.com Software Installation Ends with Error Symptom The software installation ends with an error. Restart the installation. Or, use the delete command. Remove unnecessary files from the CLI" section on page 2-11. In Device Manager, choose Admin > Flash Files and delete unnecessary files. The specified system and kickstart images Check the...
Troubleshooting Guide
Page 61
... 2 Troubleshooting Installs, Upgrades, and Reboots Troubleshooting Cisco SAN-OS Software System Reboots Send documentation comments to view the upgraded supervisor module using the show module command. Please check the standby supervisor. -- Interrupt the boot process and reconfigure the BIOS through the console port to load a new kickstart image that updates to BIOS image. If the configuration meets all guidelines when the install all command is corrupted. See the "Recovery Using BIOS Setup for Switches with standby terminated. Power on...
... 2 Troubleshooting Installs, Upgrades, and Reboots Troubleshooting Cisco SAN-OS Software System Reboots Send documentation comments to view the upgraded supervisor module using the show module command. Please check the standby supervisor. -- Interrupt the boot process and reconfigure the BIOS through the console port to load a new kickstart image that updates to BIOS image. If the configuration meets all guidelines when the install all command is corrupted. See the "Recovery Using BIOS Setup for Switches with standby terminated. Power on...
Troubleshooting Guide
Page 65
... System BIOS Setup - Use the full directory path from the TFTP server root directory. For example, if you reach the Local IP Address field. Chapter 2 Troubleshooting Installs, Upgrades, and Reboots Troubleshooting Cisco SAN-OS Software System Reboots Send documentation comments to mdsfeedback-doc@cisco.com Step 4 Press the Tab key to TFTP. Enter the image name (kickstart), and press the Tab key. Press the Tab key until you have a file named MDS9500...
... System BIOS Setup - Use the full directory path from the TFTP server root directory. For example, if you reach the Local IP Address field. Chapter 2 Troubleshooting Installs, Upgrades, and Reboots Troubleshooting Cisco SAN-OS Software System Reboots Send documentation comments to mdsfeedback-doc@cisco.com Step 4 Press the Tab key to TFTP. Enter the image name (kickstart), and press the Tab key. Press the Tab key until you have a file named MDS9500...
Troubleshooting Guide
Page 75
...-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 2-27 switch# show processes The following command to check the syslog file to the Cisco MDS 9000 Family System Messages Reference. Chapter 2 Troubleshooting Installs, Upgrades, and Reboots Troubleshooting Cisco SAN-OS Software System Reboots Send documentation comments to identify the processes that are used in use Jan 27 23:18:59 88 % LOG_PORT-5-IF_DOWN: Interface fc1/13 is down (Link failure or not-connected) Jan...
...-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 2-27 switch# show processes The following command to check the syslog file to the Cisco MDS 9000 Family System Messages Reference. Chapter 2 Troubleshooting Installs, Upgrades, and Reboots Troubleshooting Cisco SAN-OS Software System Reboots Send documentation comments to identify the processes that are used in use Jan 27 23:18:59 88 % LOG_PORT-5-IF_DOWN: Interface fc1/13 is down (Link failure or not-connected) Jan...
Troubleshooting Guide
Page 99
...; Power Supply Input Ok LED is Red, page 4-5 • Power Supply Output Failed LED is On, page 4-6 • Power Supply Fan Ok LED is operational (online). If you have been recognized by the supervisor module, and contain a valid Flash code version. Verify that the terminal is set correctly and that the modules are green when the supervisor module completes initialization. The link LED blinks orange if the port is red or orange, see the "Troubleshooting...
...; Power Supply Input Ok LED is Red, page 4-5 • Power Supply Output Failed LED is On, page 4-6 • Power Supply Fan Ok LED is operational (online). If you have been recognized by the supervisor module, and contain a valid Flash code version. Verify that the terminal is set correctly and that the modules are green when the supervisor module completes initialization. The link LED blinks orange if the port is red or orange, see the "Troubleshooting...
Troubleshooting Guide
Page 144
... 2 Issues Chapter 5 Troubleshooting Mixed Generation Hardware Send documentation comments to mdsfeedback-doc@cisco.com Step 3 Step 4 Use the show port index-allocation" for Generation-1 modules Check "show module command to display the reason why a module does not power up. Do "copy running-config startup-config" to correct the problem. Cannot Configure Port in Dedicated Mode Symptom Cannot configure port in dedicated mode. Power-on . Power-on page 5-12. 5-10 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL...
... 2 Issues Chapter 5 Troubleshooting Mixed Generation Hardware Send documentation comments to mdsfeedback-doc@cisco.com Step 3 Step 4 Use the show port index-allocation" for Generation-1 modules Check "show module command to display the reason why a module does not power up. Do "copy running-config startup-config" to correct the problem. Cannot Configure Port in Dedicated Mode Symptom Cannot configure port in dedicated mode. Power-on . Power-on page 5-12. 5-10 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL...
Troubleshooting Guide
Page 161
... important to troubleshoot Cisco Fabric Services (CFS) problems in the Cisco MDS 9000 Family multilayer directors and fabric switches. CFS can be synchronized using CFS include: • IVR • NTP • DPVM • user roles • AAA server addresses, Radius and TACACS daemons • SFM • SDV • syslog • port-security • call home OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 7-1 Some...
... important to troubleshoot Cisco Fabric Services (CFS) problems in the Cisco MDS 9000 Family multilayer directors and fabric switches. CFS can be synchronized using CFS include: • IVR • NTP • DPVM • user roles • AAA server addresses, Radius and TACACS daemons • SFM • SDV • syslog • port-security • call home OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 7-1 Some...
Troubleshooting Guide
Page 230
... connects to the dynamic port is deleted or suspended, the port is shut down menu to Config DPVM Database, page 11-14 • Port Suspended or Disabled After DPVM Activation, page 11-15 • DPVM Merge Failed, page 11-15 • DPVM Process Terminates, page 11-16 Troubleshooting DPVM Using Fabric Manager To troubleshoot DPVM using DPVM: • The interface through an FL port. • The static port...
... connects to the dynamic port is deleted or suspended, the port is shut down menu to Config DPVM Database, page 11-14 • Port Suspended or Disabled After DPVM Activation, page 11-15 • DPVM Merge Failed, page 11-15 • DPVM Process Terminates, page 11-16 Troubleshooting DPVM Using Fabric Manager To troubleshoot DPVM using DPVM: • The interface through an FL port. • The static port...
Troubleshooting Guide
Page 266
... Troubleshooting IVR Send documentation comments to mdsfeedback-doc@cisco.com Table 13-2 IVR Enhancements by Cisco SAN-OS Release Cisco SAN-OS Release (continued) Release 2.1(1a) Release 2.0(1) Release 1.3(4a) Release 1.3(1) IVR Enhancement • IVR NAT • AFIDs • Auto-topology • Virtual domains added to remote domain lists • IVR LUN zoning • IVR QoS zoning • Service group IVR with CFS support...
... Troubleshooting IVR Send documentation comments to mdsfeedback-doc@cisco.com Table 13-2 IVR Enhancements by Cisco SAN-OS Release Cisco SAN-OS Release (continued) Release 2.1(1a) Release 2.0(1) Release 1.3(4a) Release 1.3(1) IVR Enhancement • IVR NAT • AFIDs • Auto-topology • Virtual domains added to remote domain lists • IVR LUN zoning • IVR QoS zoning • Service group IVR with CFS support...
Troubleshooting Guide
Page 499
The SNMP user name is installed or upgraded from a CD-ROM or from Cisco.com. OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 26-3 You should open Java Web Start and choose File > Preferences to access the HTTP proxy settings. The Fabric Manager uses the VSAN ID and domain ID to assign links incorrectly between these errant switches. The workaround is to verify that all switches use unique...
The SNMP user name is installed or upgraded from a CD-ROM or from Cisco.com. OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 26-3 You should open Java Web Start and choose File > Preferences to access the HTTP proxy settings. The Fabric Manager uses the VSAN ID and domain ID to assign links incorrectly between these errant switches. The workaround is to verify that all switches use unique...
Troubleshooting Guide
Page 504
... mdsfeedback-doc@cisco.com Allowing network-open Users to open the Java Cache Viewer. Set web.allowDownload4All=true. Select the appropriate applications and applets and click the delete icon, or right-click and choose Remove Application from the Show drop-down list. Table 26-2 Cannot Access Fabric Manager Web Client Symptom Possible Cause Cannot access Fabric Using wrong TCP port. Open TCP port in the Fabric Manager installation directory...
... mdsfeedback-doc@cisco.com Allowing network-open Users to open the Java Cache Viewer. Set web.allowDownload4All=true. Select the appropriate applications and applets and click the delete icon, or right-click and choose Remove Application from the Show drop-down list. Table 26-2 Cannot Access Fabric Manager Web Client Symptom Possible Cause Cannot access Fabric Using wrong TCP port. Open TCP port in the Fabric Manager installation directory...
Troubleshooting Guide
Page 508
... as Cisco Traffic Analyzer. This section contains the following topics: • Performance Manager Generates Java Error, page 26-13 • Performance Manager Not Working, page 26-13 26-12 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 Click Admin > Configure > Web Users > Local Database. This data is running. Performance Manager creates a series of an ever-increasing database size. Set authentication.mode=local. Troubleshooting Performance Manager Chapter 26 Troubleshooting Fabric Manager Send documentation comments...
... as Cisco Traffic Analyzer. This section contains the following topics: • Performance Manager Generates Java Error, page 26-13 • Performance Manager Not Working, page 26-13 26-12 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 Click Admin > Configure > Web Users > Local Database. This data is running. Performance Manager creates a series of an ever-increasing database size. Set authentication.mode=local. Troubleshooting Performance Manager Chapter 26 Troubleshooting Fabric Manager Send documentation comments...
Troubleshooting Guide
Page 539
...(config)# span session 1 Appendix B Troubleshooting Tools and Methodology Using Cisco MDS 9000 Family Tools Send documentation comments to mdsfeedback-doc@cisco.com The no logging console command shown in Example B-7: • Disables console logging • Enabled by default Example B-7 no logging console Command switch(config)# no logging console The terminal monitor command shown in Example B-8: • Enables logging for telnet or SSH • Disabled by fixing the device configuration, you typically need to focus on all traffic and direct it to another port within the switch...
...(config)# span session 1 Appendix B Troubleshooting Tools and Methodology Using Cisco MDS 9000 Family Tools Send documentation comments to mdsfeedback-doc@cisco.com The no logging console command shown in Example B-7: • Disables console logging • Enabled by default Example B-7 no logging console Command switch(config)# no logging console The terminal monitor command shown in Example B-8: • Enables logging for telnet or SSH • Disabled by fixing the device configuration, you typically need to focus on all traffic and direct it to another port within the switch...
Troubleshooting Guide
Page 556
... best practices 2-2 install error 2-9 overview 2-1 power on or reboot fails 2-13 recognizing errors 2-26 recoverable restart 2-27 resets 2-26 unrecoverable restart 2-31 upgrading best practices 2-2 verifying installation 2-6 software images, selecting for supervisor modules 5-13 IN-8 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x Software Installation Wizard (procedure) 2-10 SSI boot images configuring with install ssi command 3-6 verifying 3-5 SSI boot variables verifying configuration 3-7 SSM description 3-1 Features per release (table) 3-2 initial checklist 3-3 modflash...
... best practices 2-2 install error 2-9 overview 2-1 power on or reboot fails 2-13 recognizing errors 2-26 recoverable restart 2-27 resets 2-26 unrecoverable restart 2-31 upgrading best practices 2-2 verifying installation 2-6 software images, selecting for supervisor modules 5-13 IN-8 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x Software Installation Wizard (procedure) 2-10 SSI boot images configuring with install ssi command 3-6 verifying 3-5 SSI boot variables verifying configuration 3-7 SSM description 3-1 Features per release (table) 3-2 initial checklist 3-3 modflash...