Home

Sun StorEdge™ Network FC Switch-8 and Switch

1.
2. Host IBA Switches IBB Host Adapter Host Adapter IBA IBB Host Adapter Host Adapter IBA IBB FIGURE 2 11 Two Hosts Connected to Three Sun StorEdge A5200 Arrays Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Sun StorEdge T3 partner pairs Switches Host Host Adapter Host Adapter Host Host Adapter Host Adapter FIGURE 2 12 Two Hosts Connected to Four Sun StorEdge T3 Array Partner Pairs Chapter 2 Configurations 37 Note You must enable Sun StorEdge Traffic Manager software for failover across multiple hosts to function The mp_support on the Sun StorEdge T3 array should be set to mpxio Sun StorEdge L180 or L700 FC Tape Library Switch 0 Sun Enterprise 420 1 2 3 4 Sun StorEdge T3 partner pairs Host Adapter 5 6 Host Adapter 4 8 la ea ee Be Sun Enterprise 3500 Switch 1 1 2 Host Adapter 3 4 5 6 s Host Adapter un
3. IBA ue IBB Host i Host Adapter ee IBA m oa IBB Host Adapter Co IBA Switch 1 IBB Switch 0 SL Zone 1 Ports 1 2 3 4 6 Ports All SL SL Zone 2 Ports 5 7 8 FC Tape library Ports All SL Switch 1 SL Zone 1 Ports 1 2 3 4 6 Ports All SL SL Zone 2 Ports 5 7 8 Ports All SL FIGURE 2 6 Single Host Connected to Multiple Sun StorEdge A5200 Arrays and a Single FC Tape Library 30 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Host Host Adapter Switches Sun StorEdge T3 array partner pairs Host Adapter FIGURE 2 7 Single Host Connected to Multiple Sun StorEdge T3 Array Partner Pairs Chapter 2 Configurations 31 Host Host Adapter Sun StorEdge T3 partner pairs
4. FIGURE 2 3 Single Host Connected to One Sun StorEdge T3 Array Partner Pair Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Single Host Connected to Multiple Storage Arrays FIGURE 2 4 shows one host connected through fiber optic cables to Sun StorEdge A3500FC controller modules You can connect controller modules in the same or separate cabinets FIGURE 2 5 shows one host connected to multiple Sun StorEdge A5200 arrays FIGURE 2 6 shows one host connected to Multiple Sun StorEdge A5200 arrays and a Single FC Tape library FIGURE 2 7 shows a single host connected to multiple Sun StorEdge T3 array partner pairs FIGURE 2 8 shows a single host connected to multiple Sun StorEdge T3 arrays and multiple Sun StorEdge A3500FC arrays FIGURE 2 9 shows a single host with cascading switches connected to a Sun StorEdge T3 array and a local storage Sun StorEdge A5200 array and Sun StorEdge T3 array Note You can attach different types of storage devices to the same switch as long as the storage devices are on different zones Each controller that is connected to a switch must have a unique loop ID Whenever you add a second controller to a switch make sure that the loop ID of the controller being connected is different from the loop ID of any other controller currently connected to the same switch Caution
5. Host Adapter Hard zone 1 Ports 1 8 Hard zone 2 Ports 9 16 Switch 0 Nameserver Zone 1 Ports 1 2 4 Ports 1 F 2 TL 4 TL Switch 0 1 2 a 4 5 6 __ 7 8 9 10 11 12 Sun StorEdge A3500 FC Array 13 14 i gt g Controller A 15 16 FC AL Port Controller A FC AL Port Ta 1 2 Ig l 4 zE 5 6 7 8 Controller A 9 10 FC AL Port 11 12 Controller A Sga 13 FC AL Port x5 15 16 Swiich4 Segmented loop zone 1 Ports 14 15 16 Ports All SL Switch 1 Nameserver zone 1 Ports 1 2 4 Ports 1 F 2 TL 4 TL Segmented loop zone 1 Ports 14 15 16 Ports All SL FIGURE 2 8 Single Host Connected to Multiple Sun StorEdge T3 Arrays and Multiple Sun StorEdge A3500FC Arrays 32 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Host Host Adapter Host Adapter Host Adapter Host Adapter Switch 0 1 Switches 0 2 and Switches 1 3 Nameserver Zone 1 1 Switch 2 3 Ports 1 F Ports 1 T Switch 0 and Switch 1 Segmented Loop Zone 1 Segmen
6. K K K K K K K K K K Performing Loop Bac qlctest Stopped successfully 116 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only The results of this test show that the HBA is most likely not the problem b switchtest on the switch GBIC Insert a loopback connector into the switch port Remember there is no LED indicator when inserting a loopback connector into a F Port If this test passes m Replace the cable If this test fails m Replace GBIC and rerun the test m If it fails on second pass replace the entire switch opt SUNWstade diags bin switchtest v o dev qlc0 sw0 f 3 switchtest called with options dev qlc0 sw0 f3 switchtest Started Start switchtest Testing device gqlc0 sw0 f3 Chassis Status for device qlc0O sw0 f3 Power OK Temp OK 32 0c Fan 1 OK Fan 2 OK Testing Device gqlc0 sw0 f3 Port 3 Pattern 0x7e7e7e7e 08 11 01 16 24 27 diag226 Central Sun COM Storage Automated Diagnostic Environment VTSID 6010 switchtest FATAL Switchtest failed with bad Pattern Ox7e7e7e7e on Device gqlc0 sw0 f3 Port 3 Probable_Cause s lt Faulty hba gbic hub switch cable disk gt lt Another user may be already running Switchtest on this port gt It failed the first pass Chapter 6 Examples of Fault Isolation 117 c Replace the GBIC and rerun the test opt SUNWstade diags bin switchtest v o dev qlc0 sw0 3
7. 74 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Counter Descriptions TABLE 5 1 Counter Name in port display Address ID errors AL Init Attempts AL Init Errors Busy frames Counter reset at CRC errors Delimiter errors Discarded frames Elapsed since counter reset In frames Invalid tx words recv Laser Faults LIP Flow Cntrl Errors Link Failures Port Display Window Counters Description Number of address identifiers S_ID D_ID found to be in error Number of times the port entered the initialization state Number of times the port entered initialization and the initialization failed Number class 2 and class 3 fabric busy F_BSY frames generated by this port in response to incoming frames This usually indicates a busy condition on the fabric or N_port that is preventing delivery of this frame Show the time and date of the last time the switch was reset Number of invalid Cyclic Redundancy Check CRC frames detected Number of delimiter errors detected Delimiters such as SOFc3 star of frame class 3 EOFn end of frame or others are improper or invalid Number of class 2 and class 3 sequences that were discarded by this port A sequence can be discarded because of detection of a missing frame based on SEQ_CNT detection of and E_D_TOV timeout receiving a reject frame receiving frames for a stopped sequence or other causes
8. Make sure that the controller module of the array is split between two switches For example connect controller A to switch 1 and controller B to switch 2 Chapter 2 Configurations 27 Host Host Adapter Switches Sun StorEdge A3500FC Array Controller A FC AL port Controller B FC AL port SCSI x 5 Drive tray x 5 Sun StorEdge A3500FC Array Host Adapter SCSI x 5 Drive tray x 5 e J J o Sun StorEdge A3500FC Array SCSI x 5 Drive tray x 5 FIGURE 2 4 Single Host Connected to Multiple Sun StorEdge A3500FC Arrays 28 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Host Switches Sun StorEdge A5200 Arrays Host Adapter Host Adapter C JC IBA IBB IBA IBB IBA IBB FIGURE 2 5 Single Host Connected to Multiple Sun StorEdge A5200 Arrays Chapter 2 Configurations 29 Sun StorEdge A5200 Arrays
9. port ulpl in t3 diag190 Central Sun COM Warning Mar 10 14 28 07 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 multipath status degraded path pci 6 4000 SUNW qlc 3 fp 0 0 fpl to target address 50020 23000003c5 1 is offline port ulpl in t3 diagl190 Central Sun COM Warning Mar 10 14 28 07 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g60020f20000003c53b699aal0005114c ssd71 multipath status degraded path pci 6 4000 SUNW qlc 3 fp 0 0 fpl to target address 50020f23000003c5 0 is offline port ulpl in t3 diag190 Central Sun COM Warning Mar 10 14 28 12 Sun StorEdge Traffic Manager diag226 Central Sun COM Initiating failover for device ssd GUID 60020f20000003c53b699aa10005114c 7volume ulvoll in t3 diag190 Central Sun COM Chapter 6 Examples of Fault Isolation 89 Warning Mar 10 14 28 13 SCSI diag226 Central Sun COM SCSI transport failed reason reset retrying command Warning Mar 10 14 28 27 Sun StorEdge Traffic Manager diag226 Central Sun COM Failover operation completed successfully for device ssd GUID 60020 20000003c53b699aal0005114c failed over from primary to secondary 7volume ulvoll in t3 diag190 Central Sun COM Source Agent diag226 central sun com Category Switch EventType StateChangeE
10. 0009697 0 bus address 1084e4 Mar 7 10 06 18 vikings genunix ID 936769 kern info ssd7 is pci 8 700000 pci 3 SUNW qlc 4 fp 0 0 ssd w50020 2300009697 0 Mar 7 10 06 18 vikings scsi ID 365881 kern info lt SUN T300 0117 cyl 34145 alt 2 hd 56 sec 128 gt Mar 7 10 06 18 vikings genunix ID 408114 kern info pci 8 700000 pci 3 SUNW qlce 4 fp 0 0 ssd w50020f2300009697 0 ssd7 online Mar 7 10 06 20 vikings scsi ID 799468 kern info ssd8 at fpl name w50020f230 0000457 1 bus address 1044e8 Or you may see a luxadm e dump_map output luxadm e dump_map devices pci 8 700000 pci 3 SUNW qlc 4 fp 0 0 devctl Pos Port_ID Hard_Addr Port WWN Node WWN Type 0 1084e4 1000e4 50020 2300009697 50020f2000009697 0x0 Disk device 1 108000 0 210100e08b2366f9 200100e 08b2366f9 0x1f Unknown Type Host Bus Adapter Chapter 6 Examples of Fault Isolation 85 In both of these outputs a Fibre Channel address appears the six digit number 1084e4 for example which potentially provides useful troubleshooting information The key to this is knowing how the switches encode their addresses For a Qlogic Ancor switch the address must first be changed from Hex to a 24 bit binary number Example 1084e4 translates into hexadecimal as 100001000010011100100 The calculator removes the first 3 bits because they were zeros The number is actually 000100001000010011100100 which is the number used in this
11. ID 779286 kern info scsi_vhci ssd g20000020373ccelc ssd65 degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 in a5k DPL4 to target diag226 Central Sun COM Sun ID 779286 kern info scsi_vhci ssd g200000203745053c ssd64 degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 in a5k DPL4 to target diag226 Central Sun COM Sun ID 779286 kern info scsi_vhci ssd g200000203719f6f4 ssd63 degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 disk_front 4 wwn fp3 in ad5k DPL4 to target diag226 Central Sun COM Sun ID 779286 kern info scsi_vhci ssd g2000002037450d71 ssd62 degraded path pci 6 2000 pci 1 SUNW gqlc 5 fp 0 0 Ep3 in a5k DPL4 to target diag226 Central Sun COM Sun ID 779286 kern info scsi_vhci ssd g200000203719f810 ssd61 degraded path pci 6 2000 pci 1 SUNW gqlc 5 fp 0 0 fp3 in a5k DPL4 to target diag226 Central Sun COM Sun ID 779286 kern info scsi_vhci ssd g200000203719f7d0 ssd60 degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 disk_front 3 wwn fp3 in a5k DPL4 to target Chapter 6 Examples of Fault Isolation 121 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g200000203719f7b0 ssd59 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 220000203719f7b0 0 is offline disk
12. cfgadm c configure c2 50020 23000003d5 cfgadm al Ap_Id Type Receptacle Occupant Condition co scsi bus connected configured unknown c0 dsk c0t0d0 disk connected configured unknown c0 dsk c0t1d0 disk connected configured unknown cl scsi bus connected configured unknown cl dsk clt6d0 CD ROM connected configured unknown c2 fc fabric connected configured unknown c2 210000e08b030926 unknown connected unconfigured unknown c2 50020f23000003d5 disk connected configured unknown c3 fc fabric connected unconfigured unknown c3 210000e08b032901 unknown connected unconfigured unknown e372 50020f23000003 5 disk connected unconfigured unknown Chapter 1 Introduction 5 Multiple controllers can be configured in the same command line to save time cfgadm c configure c2 c3 Unconfiguring devices is similar cfgadm c unconfigure c2 c3 See the Sun StorEdge SAN release 3 1 Installation and Configuration Guide or the cfgadm_fp 1M manpage for further details Sun StorEdge Traffic Manager Sun StorEdge Traffic Manager provides N way multipath access to I O devices as well as path management capabilities Sun StorEdge Traffic Manager is released as a component of Solaris The fundamental change that Sun StorEdge Traffic Manager delivers is to restructure the device tree to permit a multipath device to be represented as a single device instance in Solaris Sun StorEdge Traffic Manager has various path states to show how the system
13. kkkkxkxkxkxkxkxkxkkkkkkkkk kxxk kx k World wide Name Zone KKKKKKKKKKKKKKKKKKKKK WWN Zone total 0 KKKKKKKKKKKKKK KK NameServer Zone KKKKKKKKKKKKKKKK NameServer Zone total 0 KKKKKKKKKKKKKKK Broadcast Zone KKKKKKKKKKKKKK Broadcast Zone total 0 KKKKKKKKK Hard Zone KKKKKKKKK Zone 1 Enabled yes Port 1 Port 2 Ports 3 Port 6 Port F Port 8 continued on next page Chapter 3 Diagnostic Tools 51 continued from previous page Zone 2 Enabled yes Port 4 Port S KKKKKKK SL Zone KKKKKKK Zone 2 Enabled yes Port 4 Port o Sample sanbox API outputs sanbox version 172 20 67 155 HW 1003 PROM 30300 FLASH 30461 WWM 100000c0dd009e0e MAC O00cOdd009e0d sanbox nameserver 172 20 67 155 Port Address Type PortWWN Node WWN FC 4 Types 00 100000 N 210000e08b032a01 200000e 08b032a01 05 100500 N 210000e 08b030926 200000e 08b030926 SCSI_FCP sanbox links 172 20 67 155 Chassis 00 Remote Chassis Port FCAddr WWN Port 02 01 02 104100 100000c0dd009e48 Chassis 01 Remote Chassis Port FCAddr WWN Port 02 00 02 100100 1000000c0dd009e0e sanbox initiators 172 20 67 155 WWN 100000cOdd009e0e 210100 08b231026 200100e08b231026 4 0x01 sanbox port_status 172 20 67 155 0 bad port number 0 Error OK sanbox port_status 172 20 67 155 1 Port 1 F_Port Logged in 0x100000 210000e 08b032a01 200000e 08b032a01 End Note You can ga
14. 50020f23000003c5 disk connected configured unusable The output above shows that one of the Sun StorEdge T3 arrays c5 50020 23000003c5 is listed as unusable indicating a problem Correlate the WWN in Step 3 when you check the status of the storage array b luxadm e port luxadm e port Found path to 4 HBA ports devices pci 6 4000 SUNW qlc 2 fp0 0 devct1 CONNECTED devices pci 6 4000 SUNW qlc 3 fp 0 0 devct1 NOT CONNECTED devices pcit 6 2000 pci l SUNW qlc 4 fp 0 0 devetl CONNECTED devices pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 devetl CONNECTED The output above shows that one of the HBAs is not connected Given that there was only one Sun StorEdge T3 array controller on this path this would be expected behavior even if the HBA was not failing This output is inconclusive at this point 3 Check Array Status Telneting to the Sun StorEdge T3 array partner pair reveals Both controllers are online a Controller u2 has taken over LUN ownership m There was a failure reaching the master u1 controller Chapter 6 Examples of Fault Isolation 91 diag190 lt 1 gt sys stat Unit State Role Partner 4 ONLINE Master 2 ONLINE AlterM diag1l190 lt 4 gt fru stat CTLR STATUS STATE ulctr ready enabled u2ctr ready enabled diagl90 lt 2 gt port list port targetid addr_type ulpl 1 hard u2pl 2 hard diag190 lt 3 gt port listmap port targetid addr_type ulpl 1 hard ulpl 1 hard u2pl 2 hard
15. 806 6923 10 806 6924 10 804 7491 10 805 1018 13 806 1497 12 806 6419 10 805 7758 11 805 0315 12 For Internal Use Only Preface xi Accessing Sun Documentation Online The docs sun com web site enables you to access select Sun technical documentation on the Web You can browse the docs sun com archive or search for a specific book title or subject at http docs sun com Sun Welcomes Your Comments Sun is interested in improving its documentation and welcomes your comments and suggestions You can email your comments to Sun at docfeedback sun com Please include the part number for example 806 6923 10 of your document in the subject line of your email xii Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CHAPTER 1 Introduction Scope The scope of this document includes the switch and interconnections HBA GBIC cables on either side of the switch as shown in the following diagram POE J Switch FIGURE 1 1 Switch and Interconnections This troubleshooting guide is intended to provide basic guidelines that can be used for isolating problems for supported configurations identified in the Sun StorEdge SAN release 3 1 Installation and Configuration guide This guide also assumes you have been trained on all the components that comprise the storage and switch configuration The Storage Automated Diagnostic Environment software packag
16. FIGURE 4 2 Heartbeat LED Normal Failure Blink Patterns The heartbeat LED indicates the error with a series of blinks a three second pause and then the same series of blinks The number of blinks between the three second pause indicates the error The blinks occur at about twice the speed of the normal heartbeat 1 sec 3 sec pe a k pm m m m_m m FIGURE 4 3 Heartbeat LED Failure Blink Patterns Chapter 4 Diagnosing and Troubleshooting the Switch 65 PROM Checksum Failure One Blink The switch is not operable This checksum test of the PROM verifies the integrity of the PROM data A failure indicates the PROM data is corrupted The heartbeat LED blinks once between the three second pauses No port Logged in LEDs blink RAM Failure Two Blinks The switch is not operable This test verifies the data and address buses to the SRAM and verifies SRAM integrity A failure indicates that the data bus address bus or SRAM is failing The heartbeat LED blinks twice between the three second pauses No port Logged in LEDs blink Flash Checksum Failure Switch Management Port Ethernet Tests Good Three Blinks The switch is not operable The flash checksum test verifies the integrity of the flash data If the flash data is corrupt the POST next checks the Switch Management port to find out if it is functional The Switch Management port is the load path for loading new flash data If the Switch Management ports
17. IDIAG CMINOCAP CMI intended receiver ASIC ASIC or mainboard Replace mainboard Err 2033 failed to get CMI capture flag failure assembly cmiTest IDIAG CMISA1 An attempt to send a CMI ASIC failure Replace mainboard Err 2032 message from ASIC to ASIC assembly cmiTest failed IDIAG CMNOBUF Err 1029 centralMemoryTest IDIAG DATA Err 266E 306E portLoopbackTest krossPortTest DIAG ERRSTAT Err 2640 2647 3047 3840 3847 portLoobackTest crossPortTest spinSilk 3040 Port could not get any buffer not match payload transmitted non zero meaning an error was detected when receiving frames One of the following status errors occurred e Enc_in Encoding error inside frame e CRC err Cyclic redundancy check on frame failed TruncFrm Truncated frame e FrmTooLong Frame too long BadEOF Bad end of file e Enc_out Encoding error outside frame BadOrdSet Bad symbol on fiber optic cable e DiscC3 Discarded Class 3 frames ASIC failure mainboard GBIC module or fiber cable failure ASIC mainboard GBIC module or fiber cable failure Replace mainboard assembly Payload received by port did Replace mainboard assembly GBIC module or fiber cable Port Error Statistics counter is Replace mainboard assembly GBIC module or fiber cable Appendix C Brocade Error Messages 195 TABLE C 3 Diagnostic Erro
18. _ Back Apply Zoom Switch Statistics Outof buffers JO JOutofs Buffers JO e eae _ oe BufsAsico O introwBufsASiIC 4 0 COFPartyASICO O0 COF Party ASIC 1 O COFCRCASICO _ JO Do Feebee srs bu Enaco fene bus Ere ic of Frame bus Ens ASIC ola Frame bus Ens ASic tJ Fisme bus Ens ASIC 1 0 Frere Bus Ens ASIC 1 Frames Out Romes NARE 00101 aaea s Pots T Port gt 100700 Faceplate Display click ports to select or double click to enter FIGURE 3 3 Sansurfer GUI Main Window For more information and to download the package go to http www sun com storage san Chapter 3 Diagnostic Tools 47 Other Tools Sun Explorer Data Collector SUNWexplo and T3Extractor Both the Sun Explorer Data Collector and the T3Extractor are essential data gathering tools that are required for service calls or escalations Both are command line host based tools that collect pertinent information you need to see the complete picture of the host Visit the following websites for more information and to download these tools Explorer http eservices central knowledge products explorer T3Extractor http hes west nws products T3 tools html Capture and Sanbox API The Capture and Sanbox API utilities are switch information gathering tools Currently m Capture is supported for field engineer use only but not for customer use a T3 Extractor and Capture ar
19. ar 11 13 46 41 diag226 Central Sun COM scsi ID 243001 kern warning WARNING scsi_vhci ssd g60020 20000003c53b699aal0005114c ssd71 ar 11 13 46 41 diag226 Central Sun COM SCSI transport failed reason reset retrying command ar 11 13 46 56 diag226 Central Sun COM scsi ID 243001 kern info scsi_vhci scsi_vhci0 Mar 11 13 46 56 diag226 Central Sun COM Failover operation completed successfully for device ssd GUID 60020f20000003c53b699abc00006c4c failed over from primary to secondary From the var adm messages output note the following m Qlogic qlc 0 has experienced multiple OFFLINE ONLINE messages m target 1046e4 on pci 6 4000 SUNW qlc 2 fp 0 0 fcp0 was taken offline a ALUN with GUID of 60020f20000003c53b699abc00006c4c was failed over using Sun StorEdge Traffic Manager 1046e4 refers to a device on port 6 which is the physical port 7 on the switch with an ALPA of e4 This switch has a fabric ID of 1 and a chassis id of 1 See explanation earlier in this section for instructions on how to read these addresses Chapter 6 Examples of Fault Isolation 109 2 Determine the extent of the problem a cfgadm al output c4 c5 c5 c4 cfgadm al fc fabric connected 50020f23000003d5 disk connected fc fabric connected 50020f23000003c5 disk connected configured unknown configured unusable configured unknown configured unknown 110 a One of the Sun StorEdge T3 array LU
20. ar 11 16 30 50 diag226 Central Sun COM fp ID 517869 kern warning WARNING fp 0 N_x Port with D_ID 1046e4 PWWN 50020f23000003d5 reappeared in fabric Mar 11 16 30 50 diag226 Central Sun COM scsi ID 243001 kern info pci 6 4000 SUNW glc 2 fp 0 0 fcp0 Mar 11 16 30 50 diag226 Central Sun COM Dynamically discovered 2 LUNs for D_ID 1046e4 Mar 11 16 30 50 diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 multipath status optimal path pci 6 4000 SUNW qlc 2 fp 0 0 fp0 to target address 50020f23000003d5 1 is standby Mar 11 16 30 50 diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g60020f20000003c53b699aal0005114c ssd71 multipath status optimal path pci 6 4000 SUNW qlc 2 fp 0 0 fp0 to target address 50020f23000003d5 0 is standby The above messages indicate that the devices on the path are again recognized and Sun StorEdge Traffic Manager has placed them in standby mode e Re enable the path manually luxadm failover primary dev rdsk c6t60020F20000003C53B699ABC00006C4Cd0s2 You should see var adm messages similar to the example below indicating a successful failover Mar 11 16 33 01 diag226 Central Sun COM Failover operation completed successfully for device ssd GUID 60020f20000003c53b699abc00006c4c failed over from secondary to primary Chapter 6 Exam
21. April 2002 For Internal Use Only Warning reason Warning Mar 8 13 14 03 reason Warning Mar 8 13 14 03 reason Warning Mar 8 13 14 23 StorEdge multipath status address Warning Mar 8 13 14 23 StorEdge multipath status address Warning Mar 8 13 14 23 StorEdge multipat address Warning StorEdge multipat address Warning StorEdge multipat address Warning StorEdge multipat address ar 8 13 14 03 SCSI tran_err retrying command SCSI tran_err retrying command SCSI retrying command tran_err Sun StorEdge Traffic Manager Traffic Manager 22000020373ccelc 0 is offline Sun StorEdge Traffic Manager Traffic Manager 220000203745053c 0 is offline Sun StorEdge Traffic Manager Traffic Manager h status 220000203719f6f4 0 is offline ar 8 13 14 23 Sun StorEdge Traffic Manager Traffic Manager h status 2200002037450d71 0 is offline ar 8 3314323 Traffic Manager h status 220000203719f810 0 is offline Sun StorEdge Traffic Manager ar 8 233174523 Traffic Manager Sun StorEdge Traffic Manager h status 220000203719f7d0 0 is offline diag226 Central Sun COM diag226 Central Sun COM diag226 Central Sun COM disk_rear 0 wwn disk_rear 7 wwn disk_rear 3 wwn disk_front 1 wwn SCSI transport failed SCSI transport failed SCSI transport failed diag226 Central Sun COM Sun
22. ID multipath status optimal pa address 5080020000083cb3 0 i Warning Mar 8 13 14 23 Sun S torEdge Traffic Manager s offline torEdge Traffic Manager s offline torEdge Traffic Manager diag226 Central Sun COM Sun diag226 Central Sun COM Sun 779286 kern info scsi_vhci ses g5080020000083cb0 ses36 th pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target diag226 Central Sun COM Sun 779286 kern info scsi_vhci ses g5080020000083cb0 ses36 th pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g200000203719f7e0 ssd50 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target s offline disk_front 0 wwn in a5k DPL4 address 220000203719f7e0 0 i Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g20000020373ccb07 ssd49 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target s offline disk_rear 1 wwn in ad5k DPL4 address 22000020373ccb07 0 i Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g20000020373cc091 ssd48 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target s offline disk_front 10 wwn in a5k DPL4 address 22000020373cc091 0 i Warning Mar 8 13 14 23 Sun StorEdg
23. In this case port 3 may be experiencing a high rate of ongoing errors even when Sun StorEdge Traffic Manager is not using the path for data Chapter 6 Examples of Fault Isolation 115 5 Start testing FRUs The FRUs in the Host to Switch link are a HBA m Cable Switch GBIC m Switch SwitchPort Start from the HBA and work outward a Run a qlctest on the HBA Uncable the link and insert a loopback connector into the HBA port opt SUNWstade diags bin qlctest v o dev qlc0 run_connect Yes checksum Disa gt gqlctest called with options dev qlc0 run_connect Yes checksum Disable selftest Disable mbox Disable ilb_10 Disable ilb Disable elb Enable icnt 1000 run_all Critical qlctest Started Program Version is 4 0 1 Testing qlcO device at devices pci 6 4000 SUNW qlc 2 fpt 0 0 devctl Running external loopback test Performing Loop Back Frame Test Pattern 0x7e7e7e7e Frame Test Pattern Ox7e7e7e7e Frame Test Pattern 0x78787878 Frame Test Pattern Oxe7e7e7e7 Frame Test Pattern Oxb5b5b5b5 Frame Test Pattern Oxaa55aa55 Frame Test Pattern 0x7f7f7f7f Frame Test Pattern Ox0f0f0f0f Frame Test Pattern 0x4a4a4a4a Frame Test Pattern Ox00ff00ff Frame Test Pattern Oxlelelele Performing Loop Bac Performing Loop Bac Performing Loop Bac Performing Loop Bac Performing Loop Bac Performing Loop Bac Performing Loop Bac Performing Loop Bac Performing Loop Bac
24. Length of time that has elapsed since the last switch reset was performed Number of class 2 and class 3 frames received by this port Number of invalid transmission words detected during decoding Decoding is from the 10 bit characters and special K characters Number of times a laser fault was detected This is a switch internal error condition for factory use only Number of optical link failures detected by this port A link failure ia loss of synchronization for a period of time greater than the value of R_fT_fTOV or by loss of signal while not in the offline state A loss of signal causes the switch to attempt to re establish the link If the link is not re established by the time specified by R_T_TOV a link failure is counted A link reset is performed after a link failure Chapter 5 Switch Counter Information 75 TABLE 5 1 Counter Name in port display Link reset in Link reset out LIP AL_PD AL_PS LIP during INit LIP F7 AL_PS LIP F7F7 LIP F8 AL_PS LIP F8F7 LIP Total Received LISM Failed LOF Timeout ELS LOF Timeouts Long Frame Errors Loss of Signal OLS in OLS out OPN Returns Out Frames Protocol errors Port Display Window Counters Description Number of link reset primitives received from an attached device Number of link reset primitives sent from this port to an attached port Number of F7 AL_PS LIPs or AL_PD vendor specific resets performed Number of times
25. connects between the AC input power connector and an AC outlet See FIGURE 3 4 and FIGURE 3 5 An input fuse holder is incorporated into the AC input power connector assembly It holds two input fuses Switch Management Connector The switch management connector is a 10 100BASE T Ethernet interface that provides a connection to a management station Note A sticker on the back of the chassis contains the MAC Address The MAC Address is used for the physical address for ethernet communication 56 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Ethernet LEDs Link Status LED Green The green LINK status LED lights only when the Ethernet interface establishes an electronic link Activity LED Yellow The yellow Activity LED lights when the interface is transmitting data to the network or receiving data from the network Chapter 3 Diagnostic Tools 57 58 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CHAPTER 4 Diagnosing and Troubleshooting the Switch This section provides information for diagnosing and troubleshooting problems with the switch Power Checks and Troubleshooting helps you solve AC power and power supply problems Power On Self Test POST checks the condition of the Sun StorEdge Network FC Switch 8 and Switch 16 switch except for GBICs Cable Continuity tests for open fibers in the cable ne
26. d compilation Aucune partie de ce produit ou document ne peut tre reproduite sous aucune forme par quelque moyen que ce soit sans autorisation pr alable et crite de Sun et de ses bailleurs de licence s il y ena Le logiciel d tenu par des tiers et qui comprend la technologie relative aux polices de caract res est prot g par un copyright et licenci par des fournisseurs de Sun Des parties de ce produit pourront tre d riv es des syst mes Berkeley BSD licenci s par l Universit de Californie UNIX est une marque d pos e aux Etats Unis et dans d autres pays et licenci e exclusivement par X Open Company Ltd La notice suivante est applicable a Netscape Communicator Copyright 1995 Netscape Communications Corporation Tous droits r serv s Sun Sun Microsystems the Sun logo AnswerBook2 docs sun com Sun StorEdge network FC switch 8 et Solaris sont des marques de fabrique ou des marques d pos es ou marques de service de Sun Microsystems Inc aux Etats Unis et dans d autres pays Toutes les marques SPARC sont utilis es sous licence et sont des marques de fabrique ou des marques d pos es de SPARC International Inc aux Etats Unis et dans d autres pays Les produits portant les marques SPARC sont bas s sur une architecture d velopp e par Sun Microsystems Inc L interface d utilisation graphique OPEN LOOK et Sun a t d velopp e par Sun Microsystems Inc pour ses utilisateurs et licenci s Sun reconna
27. rhosts file For example if a normal user named nancy is logged in she would edit the file home nancy rhosts 4 If you are using a UNIX system with Solaris installed check the etc nsswitch conf file to make sure the hosts lookup table is appropriately set Appendix A Brocade Upgrades and Installations 141 v To Install Firmware using FTP As an alternative to modifying the rhosts file you can use the ftp method which is explained below Note You can invoke the command without any parameters in which case you are prompted for input including the choice of whether to use ftp or rshd If you invoke the command with three parameters rshd is used If you invoke the command with four parameters ftp is used 1 From a UNIX system telnet into the switch and download the firmware with the firmwareDownload command firmwareDownload lt Your UNIX IP address gt lt Your UNIX Login user s name gt lt v2 4 1f gt lt FTP_password gt 2 To check the syntax type help firmwareDownload Note With version 2 1 and higher commands are not case sensitive 3 Check the syntax by typing firmwaredownload and following the screen prompts See CODE EXAMPLE A 3 for an FTP example 142 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CODE EXAMPLE A 3 FTP Example oem240 admin gt firmwareDownload Server Name or IP Address host 10 32 99 2
28. switchtest called with options dev qlc0 sw0 f3 switchtest Started Start switchtest Testing device gqlc0 sw0 f3 Chassis Status for device qlc0O sw0 f3 Power OK Temp OK 32 0c Fan 1 OK Fan 2 OK Testing Device qlc0 sw0 f3 Port 3 Pattern 0x7e7e7e7e Pattern 0x78787878 Pattern 0xe7e7e7e7 Pattern Oxb5b5b5b5 Pattern Oxaa55aa55 Pattern Ox7f7f 7 7 Pattern Ox0f0f0f0E Pattern Ox4a4a4a4a Testing Device qlc0 sw0 f3 Port Pattern Ox00ff00ff Testing Device qlc0 sw0 f3 Port 3 Pattern Oxlelelele Testing Device qlc0 sw0 f3 Port Testing Device qlc0 sw0 f3 Port Testing Device qlc0 sw0 f3 Port Testing Device qlc0 sw0 f3 Port Testing Device qlc0 sw0 f3 Port Testing Device qlc0 sw0 f3 Port Testing Device qlc0 sw0 f3 Port WWWW WWW Ww Port 3 passed all tests on qlc0 sw0 f3 switchtest Stopped successfully The results above indicate that the original GBIC was the malfunctioning FRU d Recable the entire link and re enable this path 118 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only After recabling you should see messages similar to the example below in var adm messages ar 11 16 30 50 diag226 Central Sun COM qlc ID 686697 kern info NOTICE Qlogic qlc 0 Link ONLINE ar 11 16 30 50 diag226 Central Sun COM fp ID 517869 kern info NOTICE fp 0 Change in FC Topology old Private Loop new Fabric
29. u2p1 2 hard master PARTNER 2ctr alt master ulctr status online online lun ma o SD host wwn sun 50020 23000003c5 sun 50020 23000003d5 volume v0 v1 v0 v1 owner u2 u2 u2 u2 access primary failover failover primary 92 In the port list output from above correlate the cfgadm output from Step 2 c5 50020 23000003c5 to the current array This problem is affecting the master u1 controller For Internal Use Only Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 4 Check Switch Status View the capture output to check port 7 and the Nameserver information Port 7 should be a TL Port with the Sun StorEdge T3 array master controller connected to it 7 TL_Port online offline Not logged in KKKKKKKKKKKK Name Server KKKKKKKKKKKK Port Address Type PortWWN Node WWN FC 4 Types Database is empty As show above the port is offline and the Nameserver has no information about it This could also be confirmed by bringing up or having the customer bring up the SANsurfer GUI and checking the port state 5 Start Testing FRUs The following basic FRUs exist in the switch to storage link a Switch or Switch port Switch side GBIC Cable a Sun StorEdge T3 array MIA a Sun StorEdge T3 array controller a Rule out the Sun StorEdge T3 array controller for now based on the healthy status reported in Step 3 b To rule out the remaining FRUs uncable the link
30. 15 20 52 diag226 Central Sun COM Dynamically discovered 2 LUNs for D_ID 1006e8 Mar 10 15 20 52 diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 multipath status optimal path pci 6 4000 SUNW qlc 3 fp 0 0 fpl to target address 50020 23000003c5 1 is standby Mar 10 15 20 52 diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g60020f20000003c53b699aal0005114c ssd71 multipath status optimal path pci 6 4000 SUNW qlc 3 fp 0 0 fpl to target address 50020 23000003c5 0 is standby As shown above the Sun StorEdge T3 array PWWN 50020 23000003c5 has reappeared in the fabric Sun StorEdge Traffic Manager has noted that the path is now standby 7 You can manually fail the path back to its primary and continue production Chapter 6 Examples of Fault Isolation 95 96 Run the failover command luxadm failover primary dev rdsk c6t60020F20000003C53B699AA10005114Cd0s2 Check for success in var adm messages Mar 10 15 25 22 diag226 Central Sun COM Failover operation completed successfully for device ssd GUID 60020 20000003c53b699aa10005114c failed over from secondary to primary Check status of path luxadm display dev rdsk c6t60020F20000003C53B699AA10005114Cd0s2 DEVICE PROPERTIES for disk dev rdsk c6t60020F20000003C53B699AA10005114Cd0s2 Status Port A R Status Port B O K Vend
31. 2002 For Internal Use Only Storage Automated Diagnostic Environment Agent Functionality The Storage Automated Diagnostic Environment remotely monitors Sun network storage devices The Storage Automated Diagnostic Environment can monitor host message files for errors or connect directly through the in band data path or out of band management path of Sun StorEdge devices in order to obtain status information about each device being monitored ae Si Storage Automated Diagnostic Environment Maintenance Monitor Diagnose Report Utilities Help SUM 2 0 6 009 bradster central sun com Welcome to the Storage Automated Diagnostic Environment Copyright 2002 Sun Microsystems Inc All rights reserved Use is subject to license terms Third party software induding font technology is copyrighted and licensed from Sun suppliers Portions may be derived from Berkeley BSD systems licensed from U of CA Sun Sun Microsystems the Sun logo and Sun StorEdge are trademarks or registered trademarks of Sun Microsystems Inc in the U S and other countries Federal Acquisitions Commercial Software Government Users Subject to Standard License Term and Conditions Copyright 2002 Sun Microsystems Inc Tous droits reserves Distribue par des licences qui en restreignent l utilisation Le logidd d tenu par des tiers qui comprend Ia technologie relative aux police de awaderes est prot g par un copyright et licence par des fournisse
32. 22 disks m Four FC switches 2 local 2 remote Storage Automated Diagnostic Environment A high level SAN Topology is displayed in FIGURE 6 1 Sun age Automated Diagnostic Environment Maintenance Momom Diagnose Report Utilities Help Re OOE 010 bradster central sun com Devices Mpima Log Topology E aa Hosts Host crash3 Filter None a Go Clear Help Topology Help Mibradstr M aaa OS 3 Sicraa ha After having selected a topology several Sicrash3 functions are available in the graphical a8 amp TOPOs view MASTER crash3 012 crash3 026 f g 1 Clicking in the middle of an object or in a specific component of an object will display the status at the bottom of the page 2 Right clicking the background will present a zoom menu 3 Right clicking on an object will present a menu of functions available for that object 4 Right clicking on a link will present a menu of functions available for that link 5 Holding the Shift button while selecting objects allows to select multiple objects This can be used to move multiple objects at the same time XY Layout Save Graphics Turn On Layout Change vertical FIGURE 6 1 High Level Configuration using the Storage Automated Diagnostic Environment 82 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Troubleshooting Outline This section lists the broad steps on how to a
33. 22000020371 c3 22000020371 c3 22000020371 From the var adm messages output note the following GBIC 0 on IB Board B of the Sun StorEdge A5200 array named DPL4 has gone offline Sun StorEdge Traffic Manager path failures affecting all disks in DPL4 The HBA Qlogic qlc 3 has experienced Offline Online Events Port 5 of the switch 172 20 67 193 has changed state to offline 2 Determine the extent of the problem There are multiple areas that you can check Storage Automated Diagnostic Environment Tests for configuration changes and overall health cfgadm al for fabric configuration Sun StorEdge Traffic Manager status Type the cfgadm a1 command fc private connected configured unknown 9eb58 disk connected configured unknown 9f6f4 disk connected configured unknown 9f7bp0 disk connected configured unknown 9 7d0 disk connected configured unknown fc private connected configured unknown 9eb58 disk connected configured unusable 9f6f4 disk connected configured unusable 9f7b0 disk connected configured unusable 9f 7d0 disk connected configured unusable Notice from the abbreviated example above that one path to the disks c3 is unusable Sun StorEdge Traffic Manager hides the physical c3 controller but this output does reveal that a problem on one of the dual paths Chapter 6 Examples of Fault Isolation 125 b Check the multipathing status 1 Run luxadm display dev rdsk lt disk gt to confirm Sun Sto
34. 3 Host to Switch Failure 107 Example 4 Command Line Only Sun StorEdge A5200 Array Storage Side Error 120 Recommendations 131 Installing a New SAN 133 Required Software Components 133 Downloading Patches and Packages 135 v To Verify Successful Patch Downloads 135 Installing the Software 137 v To Install the Software 138 Installing Firmware on Brocade Silkworm Switches 140 v To Download Firmware from the Brocade Web Site 140 v To Install Firmware from UNIX Solaris 140 Upgrading theSAN 145 Downloading Patches and Packages 145 Verifying Upgrade Compliance 145 v To Upgrade Software 145 Volume Management 146 Sun StorEdge SAN 3 1 146 cfgadm Plug in Library Packages 147 Contents vii Software Installation 148 Introduction 151 Related Documentation 152 Supported Configurations 153 v___To generate the most recent patch list specific to the Brocade SAN 154 Current Issues with the Storage Automated Diagnostic Environment and Brocade 156 Diagnostic Tools 156 Storage Automated Diagnostic Environment and Brocade 156 brocadetest IM 157 Other Diagnostic Tools 158 Port Differences between Sun StorEdge Ports and Brocade Ports 164 Accessing the Brocade Silkworm switch 165 Power On Self Tests POST 167 Removing Power 168 General Troubleshooting Procedures 170 Troubleshooting Case Study 172 Configuration 172 Storage Automated Diagnostic Environment Topology 173 Error Message Formats 187 Front Panel Message Formats 188 Diagnostic Err
35. 4 Frame bus Errs ASIC 3 Port 1 Frame bus Errs ASIC 3 Port 2 Frame bus Errs ASIC 3 Port 3 Frame bus Errs ASIC 3 Port 4 Internal Parity ASIC 0 Port 1 Internal Parity ASIC 0 Port 2 Internal Parity ASIC 0 Port 3 Internal Parity ASIC 0 Port 4 Internal Parity ASIC 1 Port 1 Internal Parity ASIC 1 Port 2 Internal Parity ASIC 1 Port 3 Internal Parity ASIC 1 Port 4 Internal Parity ASIC 2 Port 1 Internal Parity ASIC 2 Port 2 Internal Parity ASIC 2 Port 3 Internal Parity ASIC 2 Port 4 Internal Parity ASIC 3 Port 1 Internal Parity ASIC 3 Port 2 Internal Parity ASIC 3 Port 3 Internal Parity ASIC 3 Port 4 Description Internal switch counter that tracks errors during frame outputs from the specified ASIC A non zero value may indicate an internal problem with the switch Parity error detected curing reading of the frame in the CPORT OUt FIF COF for the specified ASIC A non zero value may indicate an internal problem with the switch Errors detected in the data being sent over the frame bus between ASICs A non zero value may indicate an internal problem with the switch Parity error detected with data transfer internal to the switch A non zero value may indicate an internal problem with the switch Chapter 5 Switch Counter Information 79 TABLE 5 2 Counter Names and Descriptions Faceplate Window Counter Description Intr low Bus ASIC 0 Number of times a low buffer condition has occurred on Intr low Bus ASIC 1 the spe
36. 69 switch bus test failure 69 switch management port failure 70 blink patterns failure 65 flash checksum failure 67 PROM checksum failure 66 RAM failure 66 C cascading 18 cascading rules 22 cfgadm_fp 4 configuration high level using Network Storage Agent 2 1 82 multihost 34 single host connected to multiple storage arrays 27 single host connected to one storage array 25 switch 11 configuration examples 25 configuration guidelines 23 configurations 11 connections verifications using Sansurfer 47 connector switch management 56 connector and fuses 56 conventions typographic x D diagnostic tool Sansurfer GUI 47 Sun Explorer Data Collector SUNWexplo 48 T3Extractor 48 diagnostics POST 60 diagram LEDs and back panel controls 8 port 53 document purpose 2 scope 1 documentation accessing online xii E ethernet LEDs 57 INDEX 207 F fabric attachment of 4 division of 3 20 fabric ports configuration of 2 registration of 2 failure complete 71 hung flash control code 70 fault isolation command line only 120 examples of 81 host to switch failure 107 setup requirements 82 switch to storage error 89 switch to switch error 97 FC Tape supported configurations 14 FC Tape Libraries configuration of 19 features fabric ports 2 new to 3 0 2 figure chassis back 16 port switch 54 chassis back 8 port switch 53 FL Ports connection 2 force PROM 62 G GBIC
37. Contact customer support Contact customer support 198 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE C 3 Message Diagnostic Error Messages Continued Description Probable Cause Action TEMP 3_FAILED ILOG_CRITICAL TEMP 4_FAILED ILOG_CRITICAL Switch overheated Switch overheated Fan failure Fan failure Contact customer support Contact customer support TEMP 5_FAILED ILOG_CRITICAL IFANS 1_FAILED ILOG_WARNING Switch overheated Switch overheated Fan failure Fan failure Contact customer support Contact customer support IFANS 2_FAILED ILOG_ERROR IFANS 3_FAILED ILOG_CRITICAL Switch overheated Switch overheated Fan failure Fan failure Contact customer support Contact customer support IFANS 4_FAILED ILOG_CRITICAL IFANS 5_FAILED ILOG_CRITICAL Switch overheated Switch overheated Fan failure Fan failure Contact customer support Contact customer support IFANS 6_FAILED ILOG_CRITICAL IPOWER 1_FAILED ILOG_CRITICAL Switch overheated Switch power failure Fan failure Power supply failure Contact customer support Contact customer support IPOWER 2_FAILED ILOG_CRITICAL Switch power failure Power supply failure Contact customer support IFCIU IUBAD L S Invalid IU OS error Contact customer support
38. Environment Maintenance Monitor MME Report Utilitie Mmcoystems 20 56 001 diag245 central sun com TES Tools Diagnostics z a J Diagnostics Diagnostics Test Manager Test Mar popup Set Defaults Test Archives lt lt Use the buttons on the left to go to the appropriate Test function Test from Topology Execute test from the Site Topology Test from List Execute test from a list of physical devices Test Manager View and control test execution Test Defaults Set the default options for each available test Test Archives Review the results of archived tests You can run Storage Automated Diagnostic Environment diagnostic tests from the test list or from a topology The functional tests are designed to test the target FRU and operate on in band or out of band data paths The Storage Automated Diagnostic Environment will cause the test to be run on the appropriate Host Note You can invoke the Link Test by right clicking on the link displayed in the Test from Topology Storage Automated Diagnostic Environment s implementation of diagnostic tests verify the operation of all the user selected components Tests are selected from a graphical view of the system s topology The Storage Automated Diagnostic Environment Graph view shows the physical topology of a system or merged system Using the Topology view you can select specific subtests and test options The monitoring status of devices and links a
39. FCIU IUCOUNT L S Total number of IUs OS error Contact customer Count lt 0 support FCPH EXCHBAD L S Bad exchange OS error Contact customer support FCPH EXCHFREE L S Unable to free an exchange OS error Contact customer support MQ QWRITE L M Message queue overflow Task blocked Contact customer support MQ QREAD L M Message queue unread OS error Contact customer support IMQ MSGTYPE E M Unknown message type OS error Contact customer support Appendix C Brocade Error Messages 199 TABLE C 3 Diagnostic Error Messages Continued Message Description Probable Cause Action SEMA SEMGIVE L M Unable to give a semaphore OS error Contact customer support SEMA SEMTAKE L M Unable to take a semaphore OS error Contact customer support SEMA SEMFLUSH L M Unable to flush a semaphore OS error Contact customer support PANIC TASKSPAWN Task creation failed OS error Contact customer ILOG_PANIC support PANIC SEMCREATE Semaphore creation failed OS error Contact customer ILOG_PANIC support PANIC SEMDELETE Semaphore OS error Contact customer ILOG_PANIC support IPANIC QCREATE Message queuer failed OS error Contact customer ILOG_PANIC support IPANIC QDELETE Message queuer deletion OS error Contact customer ILOG_PANIC failed support PANIC MALLOC Memory allocation failed OS error Contact customer ILOG_PANIC support IPANIC FREE LOG_PANIC Memory free
40. SEGMENTED ILOG_WARNING FABRIC NO_ALIASID ILOG_WARNING The switch configuration information has become irrevocably corrupted The switch configuration information has grown too large to be saved or has an invalid size The switch has encountered an unrecognized version of the switch configuration Fabric segmented No free multicast alias OS error OS error OS error e Incompatible fabric parameters and switches e Conflict zones Too many multicast groups in use The system automatically resorts to the default configuration settings Contact customer support The system automatically resorts to the default configuration settings Reconfigure fabric or zones Remove some of the groups FABRIC BADILS ILOG_WARNING IFLASH BAD_MIRROR ILOG_WARNING IRPC SVC_EXIT IRPC SVC_REG Bad ISL ELS size The system s flash memory has encountered an error An RPC service daemon has terminated prematurely or unexpectedly An RPC service daemon could not establish service for a particular protocol handler The ISL ELS payload is wrong OS error OS error OS error Contact customer support The system attempts to recover from its mirrored backup Contact customer support Contact customer support Contact customer support TEMP 1_FAILED ILOG_WARNING TEMP 2_FAILED ILOG_ERROR Switch overheated Switch overheated Fan failure Fan failure
41. State devices pci 6 4000 SUNW gqlc 2 fp 0 0 50020 23000003d5 0 secondary STANDBY devices pci 6 4000 SUNW glc 3 fp 0 0 50020 23000003c5 0 primary ONLINE It would have two physical paths associated with it as well Note the Controller lines Also note the primary secondary paths as well as the STANDBY ONLINE states Chapter 1 Introduction 9 A Sun StorEdge A5200 array disk would look similar to the following example In the case of A5200s both paths are primary and ONLINE luxadm display dev rdsk c6t200000203719EB58d0s2 DEVICE PROPERTIES for disk dev rdsk c6t200000203719EB58d0s2 Status Port A O K Status Port B O K Vendor SEAGATE Product ID ST39103FCSUN9 0G WWN Node 200000203719eb58 WWN Port A 210000203719eb58 WWN Port B 220000203719eb58 Revision 034A Serial Num 154563930000 Unformatted capacity 8637 338 MBytes Read Cache Enabled Minimum prefetch 0x0 Maximum prefetch Oxffff Location In the enclosure named DPL4 Device Type Disk device Path s dev rdsk c6t200000203719EB58d0s2 devices scsi_vhci ssd g200000203719eb58 c raw Controller devices pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 Device Address 220000203719eb58 0 Class primary State ONLINE Controller devices pci 6 2000 pci 1 SUNW qlc 4 fp 0 0 Device Address 210000203719eb58 0 Class primary State ONLINE Refer to the Sun StorEdge Traffic Manager Installation and Configuration Guide for fu
42. StorEdge A5200 array 34 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Sun StorEdge A3500 FC Arrays Controller A FC AL Port Controller B FC AL Port SCSI x 5 Host Switches Drive Tray x 5 Host Adapter Controller A FC AL Port Host Adapter Controller B FC AL Port SCSI x 5 Host Adapter Drive Ti Host Adapter uae rayx5 Controller A FC AL Port Controller B FC AL Port SCSI x 5 Drive Tray x 5 FIGURE 2 10 Two Hosts Connected to a Maximum of Four Sun StorEdge A3500FC Arrays Chapter 2 Configurations 35 Note You can attach different storage types to the same switch so long as the storage devices are on different zones Each controller that is connected to a switch must have a unique loop ID Whenever you add a second controller to a switch make sure that the loop ID of the controller being connected is different from the loop ID of any other controller currently connected to the same switch h Caution Ensure that the controller modules are not connected to the same switch 36 Sun StorEdge A5200 Arrays
43. Switches 19 Fibre Channel FC Tape Libraries 19 Switch Port Types 20 Zones 20 Hard Zones 20 Name Server Zones 20 SL Zones 21 Zoning Rules 21 Cascading Rules 22 Rules for Adding and Removing Devices While the Hosts are Online 22 Guidelines for Configuration 23 Switches 23 Zones and Arrays 23 Zones and Storage 23 Configuration Examples 25 Single Host Connected to One Storage Array 25 Single Host Connected to Multiple Storage Arrays 27 Multihost 34 3 Diagnostic Tools 41 Storage Automated Diagnostic Environment 41 Storage Automated Diagnostic Environment Functions 42 iv Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 Storage Automated Diagnostic Environment Agent Functionality 43 Storage Automated Diagnostic Environment Diagnostic Functionality 44 Running Diagnostic Tests From the GUI Window 44 Sansurfer GUI 2 08 22 or above GSUNWsmer 47 Other Tools 48 Sun Explorer Data Collector GSUNWexplo and T3Extractor 48 Capture and Sanbox API 48 Sample Capture Output 49 FC Switch LEDs and Back Panel Controls 53 54 Power Switch 54 Back Panel LEDs 55 Heartbeat LED Yellow 55 Switch Logic Power Good LED Green 55 Fan Fail LED RED 55 Over Temperature LED Red 55 Logged In LED Green 56 Traffic LED Yellow 56 AC Input Power Connector and Fuses 56 Switch Management Connector 56 Ethernet LEDs 57 Link Status LED Green 57 Activity LED Yellow 57 Diagnosing and Troubleshooting the Switc
44. T3 array Acronyms amp Glossary 205 Glossary Broadcast Zone Cascade Fabric Hard Zones Name Server Private Loop Public Loop Segmented Loop Zone Zoning Acronyms amp Glossary 206 Zone type used to filter broadcast traffic away from end nodes that cannot use or interpret it A port will broadcast to all ports in the same Broadcast Zone s in which the port is defined Broadcast zoning is primarily used for doing IP over Fibre Channel or when mixing IP and SCSI traffic on the switch These zones are not yet useful or interesting in Suns current SAN implementations Connection of two or more switches together to increase the number of available ports or to allow for increased distances between components of the SAN Fibre channel network built around one or more switches It is also common to refer to something as a Fabric device or being in Fabric mode When used in this context it simply means a public device capable of logging into a Fabric and having public loop characteristics as opposed to a private loop legacy device Hard Zones allow the division of the Fabric one or more Switch chassis into as many as 16 Fabric wide zones that define the ports that can communicate with each other A particular port may be placed in only one Hard Zone no overlapping Hard Zones If Hard Zones are enabled Name Server Zones and SL Zones will not communicate across defined Hard Zone boundaries Zones allow the div
45. a Sun Field Engineer trying to isolate to a FRU 151 Related Documentation Brocade Documentation m Brocade Silkworm 2400 2800 Hardware Reference Manual m Brocade Silkworm 3800 Hardware Reference Manual m Brocade Fabric OS Hardware Reference Manual m Brocade Fabric OS Release Notes Brocade WebTools User s Guide a Brocade Zoning User s Guide a Brocade QuickLoop User s Guide Sun Documentation The Sun StorEdge switch documents are referenced for overall configuration guidelines a Sun StorEdge Network FC switch 8 and switch 16 Installation and Configuration Guide SAN 3 0 Release part number 816 0830 12 a Sun StorEdge Network FC switch 8 and switch 16 SAN 3 0 Release Release Notes part number 816 0842 12 152 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Supported Configurations The Brocade Silkworm switch configurations and the Sun StorEdge switch configurations follow the same rules for maximum number of initiators supported number of arrays per zone and other hardware specific information Refer to Chapter 2 Configurations of this guide for supported hardware configurations Brocade Silkworm switch configurations and Sun switch configurations have different minimum operating system OS and patch requirements shown in TABLE B 1 TABLE B 1 SAN Supportability Matrix Sun Multipath Driver Patches or Server HBA Host OS OS Level Ser
46. a specific component of an object will display the status at the bottom of the page 2 Right clicking the background will present a zoom menu 3 Right clicking on an object will Erocade 16 present a menu of functions available for that object 4 Right clicking on a link will present a menu of functions available for that link 5 Holding the Shift button while selecting objects allows to select multiple objects This can be used to move multiple objects at the same time J brocade 172 20 67 164 1000006069107125 monitored by diag229 J XY Layout Savef Graphics Turn On Layout Change Vertical Hw Goole eee l From the topology notice the HBA and port two of the first switch have errors Also a Sun StorEdge Traffic Manager event has occurred on the Sun StorEdge T3 arrays Note Sun StorEdge Traffic Manager events are only shown between the last switch and the storage From this Topology view concentrate on the link between the HBA and the switch port 2 Appendix B Brocade Troubleshooting 177 Step 3 Check the status of the array port ulpl ulpl u2pl u2p1 targetid addr_type lun 1 al 2 2 hard 0 hard al hard 0 hard 1 diag168 Central Sun COM etc lt 18 gt port listmap volume diagl69ulvl1 diagl69u2v1 diagl69ulvl diagl69u2v1 All LUNs have failed over to the u1 controller owner ul ul ul ul Step 4 Check the status of the switc
47. and place a loopback connector into the GBIC in port 7 c Run the Storage Automated Diagnostic Environment switchtest on this port If this test fails Replace GBIC and retest a If switchtest fails again replace the entire switch Chapter 6 Examples of Fault Isolation 93 If this test passes a Reseat or Replace the MIA a Reseat or Replace the cable In this example the results of switchtest from the command line follow First Pass with loopback connector inserted opt SUNWstade diags bin switchtest v o dev qlcl sw2 f1l e2 sw3 e2 t17 switchtest called with options dev qlcl sw2 f1l e2 sw3 e2 t1l7 switchtest Started Start switchtest Testing device qlcl sw2 fl e2 sw3 e2 t17 Chassis Status for device qlcl sw2 fl e2 sw3 e2 tl17 Power OK Temp OK 31 0c Fan 1 OK Fan 2 OK 08 10 01 15 16 38 diag226 Central Sun COM Storage Automated Diagnostic Environment VTSID 6008 switchtest FATAL Port 7 is Offline with Pattern 0x4a4a4a4a on Device qlcl sw2 f1 e2 sw3 e2 t17 Probable_Cause s lt Fibre Channel cable disconnected to HBA gt lt Bad GBIC or bad Fibre Channel cable gt Replace the GBIC and rerun the test opt SUNWstade diags bin switchtest v o dev qlcl sw2 f1l e2 sw3 e2 t17 switchtest called with options dev qlcl sw2 f1l e2 sw3 e2 t1l7 switchtest Started Start switchtest Testing device qlcl sw2 fl e2 sw3 e2 t17 Chassis Status for device qlcl sw2 fl e2 sw3 e2 t
48. errors should be addressed if the number of errors found increases when running the portStateShow command again TABLE C 1 Probable Failure Actions Failed Test ramTest portRegTest centralMemoryTest cmiTest cmemRetentionTest sramRetentionTest camTest portLoopbackTest crossPortTest spinSilk Action replace DRAM module or mainboard assembly replace mainboard assembly replace mainboard assembly replace mainboard assembly replace mainboard assembly replace mainboard assembly replace mainboard assembly replace mainboard assembly replace mainboard assembly GBIC or fiber cable replace mainboard assembly GBIC or fibre cable Error Message Numbers An error number ERR xxxx appears at the end of an error message TABLE C 2 matches each error number with the test that caused the error and the name of the error Look up the complete definition of the error name and the actions that will correct the error in TABLE C 3 Appendix C Brocade Error Messages 189 TABLE C 2 Error Message Codes Defined Error Number Test Name Error Name 0001 n a DIAG CLEAR_ERR 0004 n a DIAG POST_SKIPPED 0B15 sramRetentionTest DIAG REGERR 0B16 DIAG REGERR_UNRST OBOF DIAG BUS_TIMEOUT 1F25 cmemRetentionTest DIAG LCMRS 1F26 DIAG LCMTO 1F27 DIAG LCMEM 0110 ramTest DIAG MEMORY 0111 DIAG MEMSZ 0112 DIAG MEMNULL 0415 portRegTest DIAG REGERR 0416 DIAG REGERR_UNRST 040F DIAG BUS_TIMEOUT 1020 centralMemoryTest DIAG CMBISRTO 1021
49. failed OS error Contact customer support IPANIC INCONSISTENT Data out of sync OS error Contact customer ILOG_PANIC support PANIC INTCONTEXT Data out of sync OS error Contact customer ILOG_PANIC support PANIC ZOMTIMSET Attempt to set a zombie timer OS error Contact customer ILOG_PANIC support PANIC ZOMTIMKILL Zombie timer destroyed OS error Contact customer ILOG_PANIC support PANIC FREETIMRLSD Free timer released OS error Contact customer ILOG_PANIC support IPANIC TIMEUSECNT Timer use count exceeded OS error Contact customer ILOG_PANIC support PANIC LSDB_CKSUM Link State Database OS error Contact customer ILOG_PANIC checksum failed support SYS NOMEM No memory OS error Contact customer ILOG_CRITICAL support 200 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE C 3 Diagnostic Error Messages Continued Message Description Probable Cause Action SYS SYSCALL LOG_ERROR System call failed OS error Contact customer support SYS BADPTR LOG_ERROR Bad system pointer OS error Contact customer support SYS INTRPT Bad system interrupt OS error Contact customer ILOG_CRITICAL support SYS FLASHRD FLASH memory read error OS error Contact customer ILOG_ERROR support SYS FLASHWR FLASH memory write error OS error Contact customer ILOG_ERROR support TIMERS ENOFAIL Invalid timeout value OS error Contact customer ILOG_CRITICAL support T
50. is currently running m Primary Path Path to LUN disk through controller that it resides on m Secondary Path Path to LUN disk through alternate controller Path States ONLINE Path is available and will be used for I O STANDBY Path is available but will not be used for I O OFFLINE Path is unavailable Note Currently only the Sun StorEdge array uses the ONLINE STANDBY convention The Sun StorEdge A5200 array has an ONLINE ONLINE state 6 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Both paths in a dual connected Sun StorEdge A5200 array will be used for I O However there is currently no way to manually failover a Sun StorEdge A5200 array to use another path Sun StorEdge Traffic Manager is currently not supported for the Sun StorEdge A3500FC array Sun StorEdge SAN release 3 1 Features for Sun StorEdge Traffic Manager a Dynamic N Path multipathing with automatic discovery of new paths m Support for Sun StorEdge T3 arrays and Sun StorEdge A5200 arrays no Sun StorEdge A3500FC support a Automatic failover m Enable Disable globally or per HBA m luxadm changes to display multipathing information and manual failover failback m cfgadm 1M support a Dynamic reconfiguration Sun StorEdge Traffic Manager may be enabled disabled globally through the kernel drv scsi_vhci conf file or ona per host bus adapter port basis through the kernel drv qlc conf file C
51. m Revision checking Support for the service processor and virtualization engine components of Sun StorEdge 3900 and 6900 series offerings a Remote notification through SRS SRS NetConnect RSS HTTP NSM and SMTP Providers or email m Support for storage area networks SANs 41 Storage Automated Diagnostic Environment Functions For each device the Storage Automated Diagnostic Environment performs the following functions 1 Sends the information by way of a discovery event to the system administrator and or the Network Storage Command Center NSCC through an interface with the transport mechanisms Note The first access to a device yields a discovery event that collects all the information about that device plus other events for other preconfigured devices that may be generated by health monitors 2 Reads the proper var adm messages files finds relevant entries and reports them as events through the local email notification mechanism if configured 3 Connects to Sun StorEdge T3 and T3 array storage devices directly through in band data paths and out of band management paths 4 Reads the device s configuration and state information stores it locally in the cache compares the results of the last run and transmits the differences 5 Reads threshold information and reports errors when the frequency threshold reaches predefined levels 42 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April
52. pci 2 S UN W qle 4 fp 0 0 devctll irun comect Yesl selftest Disablel mbox Disablel checksum Disablel ilb_10 Disablel ilb Disable elb Enable xcnt 0x10000 iterations 100 userpattern 0x7 e7 e7 e7 el selectpattern critical qictest Started Program Version is 4 0 1 Testing qlc0 device at devices pci 600000 pci 2 S U NW qle 4 fp 0 0 devctl QLC Adapter Chip Revision 1 Risc Revision 3 Frame Buffer Revision 1029 Riscrom Revision 4 Driver Revision 5 a 2 1 15 Running external loopback test i Performing Loop Back Frame Test Pattern 0x7e7e7e7e Performing Loop Back Frame Test Pattern Ox7e7e7e7e i Performing Loop Back Frame Test Pattern Oxlelelele i Performing Loop Back Frame Test Pattern Oxflflfifi Performing Loop Back Frame Test Pattern Oxb5b5b5b5 Performing Loop Back Frame Test Pattern Oxdadadada Performing Loop Back Frame Test Pattern 0x78787878 Performing Loop Back Frame Test Pattern Oxe7e7e7e7 Performing Loop Back Frame Test Pattern Oxaa55aa55 Performing Loop Back Frame Test Pattern 0x7f7f7f7f Performing Loop Back Frame Test Pattern Ox0f0f0f0f Performing Loop Back Frame Test Pattern Ox00ff00ff Performing Loop Back Frame Test Pattern 0x25252525 qictest Stopped successfully FIGURE B 7 Storage Automated Diagnostic Environment glctest The qlctest test also passes The HBA is most likely not the cause of the error This leaves the cable as the mo
53. port gt lt user_pattern gt lt pattern_width gt a Only works on logged in L Ports m To test ports with Loopback connectors use crossPortTest CODE EXAMPLE B 7 spinFab Example Output diagl67 admin gt spinfab 1 0 0 spinFabO running spinFabO Completed 1 megs status passed port 0 test status 0x00000000 passed 162 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CODE EXAMPLE B 8 nsShow Example Output diagl64 admin gt nsshow Type Pid Cos PortName NodeName TTL sec NL 0312e4 375020270 F223 7002007 S0di2c SDf0220 202002 00tadi2c na FC4s FCP SUN T300 0118 Fabric Port Name 20 02 00 60 69 10 71 25 NL O03lee8 3 50 02 0f 23 00 00 3e e5 50 02 0f 20 00 00 3e e5 na FC4s FCP SUN T300 0118 Fabric Port Name 20 0e 00 60 69 10 71 25 The Local Name Server has 2 entries Note nsShow is a listing of WWNs of the devices connected to the switch Appendix B_ Brocade Troubleshooting 163 Port Differences between Sun StorEdge Ports and Brocade Ports TABLE B 5 Port Differences Sun StorEdge Port Brocade Port Function T_Port E_Port Expansion port Used for interswitch connections SL_Port L Port Loop port In Sun StorEdge switch the SL_Port is segmented loop Private Loop only TL_Port L Port Loop port This port enables private devices to translative loop communicate with fabric or public devices In the Brocade switc
54. remove a host without shutting down the SAN 22 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Guidelines for Configuration Switches For high availability applications configure two sets of switches in parallel Zones and Arrays Sun StorEdge T3 arrays support SL zones and name server zones or zones in which a host has made a point to point Fabric connection to a switch and the Sun StorEdge T3 array is attached to a TL port Do not mix different arrays in the same zone A single zone can contain only Sun StorEdge A3500FC arrays or only Sun StorEdge T3 arrays You may configure a minimum of one port per zone For example a 16 port switch can have a maximum of 16 zones Zones cannot overlap for example a particular port can only belong to one zone An exception to this rule is even though ISL ports must be in separate name server zones if the ISL ports are in the same hard zone the nameserver zones can overlap Zones and Storage You can dynamically add storage to an SL zone using luxadm procedures for the Sun StorEdge T3 arrays Segmented Loop SL zones and name server zones translated loop zones must be in different hard zones on a switch Chapter 2 Configurations 23 TABLE 2 8 Arrays Zones and Initiators Array Maximum Arrays Zone Maximum Initiators Zone Sun StorEdge 4 2 A3500FC array Sun StorEdge T3 8 8 Sun StorEdge 2 for a single array 4 for a part
55. software components from the following web sites listed in TABLE A 1 TABLE A 1 Software Download Sites Software Downloading Web Site packages and http www sun com storage san documentation patches http sunsolve sun com Refer to Appendix B of this guide package and patch supportability guidelines in a Brocade configuration To Verify Successful Patch Downloads Use one of the following three utilities to obtain the checksum value of the patch you downloaded CODE EXAMPLE A 1 Solaris usr bin sum Utility usr bin sum 108982 09 zip 2010 880 108982 09 zip usr bin sum r 108982 09 zip 63160 880 108982 09 zip Note The sum utility can also be located in the usr ucb directory Appendix A Brocade Upgrades and Installations 135 CODE EXAMPLE A 2 Example of GNU md5sum Utility md5sum 108982 09 zip 1297 cd385 f618c726489c6c7 57c900 108982 09 zip Download the GNU md5sum utility from http www sunfreeware com or from http sunsolve Sun com md5 md5 tar z 2 Compare the checksum value that is displayed to the patch checksum value given at the checksum File link http sunsolve Sun com If the values are identical the patches were properly downloaded Note The checksum file at http sunsolve Sun com is approximately 614 kBytes 136 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Installing the Software TABLE A 2 lists the r
56. t les efforts de pionniers de Xerox pour la recherche et le d veloppement du concept des interfaces d utilisation visuelle ou graphique pour l industrie de l informatique Sun d tient une licence non exclusive de Xerox sur l interface d utilisation graphique Xerox cette licence couvrant galement les licenci s de Sun qui mettent en place l interface d utilisation graphique OPEN LOOK et qui en outre se conforment aux licences crites de Sun CETTE PUBLICATION EST FOURNIE EN L ETAT ET AUCUNE GARANTIE EXPRESSE OU IMPLICITE N EST ACCORDEE Y COMPRIS DES GARANTIES CONCERNANT LA VALEUR MARCHANDE L APTITUDE DE LA PUBLICATION A REPONDRE A UNE UTILISATION PARTICULIERE OU LE FAIT QU ELLE NE SOIT PAS CONTREFAISANTE DE PRODUIT DE TIERS CE DENI DE GARANTIE NE S APPLIQUERAIT PAS DANS LA MESURE OU IL SERAIT TENU JURIDIQUEMENT NUL ET NON AVENU SS com Ca Adobe PostScript Contents Related Documentation xi Introduction 1 Scope 1 Features 2 Fabric Ports 2 Translative Loop 3 Name Server Zoning 3 Cascading T Ports 4 cfgadm_fp 4 Examples 5 Sun StorEdge Traffic Manager 6 Path States 6 Sun StorEdge SAN release 3 1 Features for Sun StorEdge Traffic Manager 7 Configurations 11 The Sun StorEdge Network FC Switch 8 or Switch 16 Switch 11 Supported Hardware 11 Supported Configurations 13 Hosts 13 Host Operating Environment Rules 13 Arrays 14 Supported Host Bus Adapters 14 FC Tape 14 Packages and Patches 15 Array Storage Rules 17
57. testing of 69 H hardware supported 11 HBA configuration of 2 HBA port disabling 8 HBAs supported configurations 14 host operating environment rules 13 hosts supported configurations 13 l indicator fan fail LED red 55 heartbeat LED yellow 55 logged in LED green 56 over temperature LED red 55 switch logic power good LED green 55 traffic LED yellow 56 information switch counter 73 interconnections 1 InterSwitchLink ISL 11 L LED heartbeat blink patterns 65 LEDS back panel 55 LEDs ethernet 57 link status 57 LUN T3 LUN with MPxIO 8 luxadm display 9 10 MPxIO 6 access to 6 disabling 7 enabling 7 features 7 path states 6 INDEX 208 Sun StorEdge SAN Release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only N name server zoning 3 N path multipathing 7 O operating environment required Solaris release 13 P packages supported 15 patches downloading using Sunsolve 13 supported 15 port disabling on dual port card 8 ports F FL 3 isolation of 20 SL 2 T cascading 4 TL 3 POST bus error 61 error descriptions 60 fatal errors 60 non fatal errors 60 port error 61 PROM checksum error 61 POST tests arbitrated loop test 61 auto route logic tests 61 checksum test 61 frame bus test 61 functional hardware tests 61 loopback data path tests 61 read write register tests 61 switch management p
58. values or abnormally high counter values may indicate a problem A LIP that occurs on one port in a zone propagates to all the ports that have devices attached to them in the same zone The LIP counter is incremented on all those ports m Normal activity may also increase counter values Counters increment on power cycles Running the Sun StorEdge PCI Dual Fibre Channel Host Adapter Board Test qlctest from the Storage Automated Diagnostic Environment increments the following counters a In frames a Out frames a Link failure m Sync losses 100ms a Invalid tx words rec a LIP total received m LIP F7F7 m LIP F8F7 a AL Init Attempts m Sync Loss a LIP during Init 73 To view any counter use the SanSurfer GUI Port Display window see FIGURE 5 1 through which you can view the counters non disruptively Web GUI File Edit View Special Help 3 port Display Zoning Refresh Delete Previous Next Port Name Port 6 values Statistics values wed Jan 10 15 17 24 MST 2001 Elapsed since counter reset Out frames usy frames ink failures rotocol errors a Tercerrors oS Type Address Device Name Type Status C e e A SP C send LIP E SE a r Eo Enable all Se ee a z Fe ge s S Ee Ie Eee eo a i a e a er Display FIGURE 5 1 SanSurfer GUI Port Display TABLE 5 1 on the following page describes the counters from the Port Display window
59. 0020F20000003C53B699ABCO0006C4Cd0s2 gt devices scsi_vhci ssd g60020 20000003c53b699abc00006c4c c raw Chapter 6 Examples of Fault Isolation 99 Use this path to query Sun StorEdge Traffic Manager luxadm display dev rdsk c6t60020F20000003C53B699ABC00006C4Cd0s2 DEVICE PROPERTIES for disk dev rdsk c6t60020F20000003C53B699ABCO00006C4Cd0s2 Status Port A O K Status Port B O K Vendor SUN Product ID T300 WWN Node 50020 20000003da5 WWN Port A 50020 23000003d5 WWN Port B 50020 23000003c5 Revision 0117 Serial Num Unsupported Unformatted capacity 119514 500 MBytes Write Cache Enabled Read Cache Enabled Minimum prefetch 0x0 Maximum prefetch 0x0 Device Type Disk device Path s dev rdsk c6t60020F20000003C53B699ABC00006C4Cd0s2 devices scsi_vhci ssd g60020f20000003c53b699abc00006c4c c raw Controller devices pci 6 4000 SUNW qlc 2 fp 0 0 Device Address 50020 23000003d5 1 Class primary State ONLINE Controller devices pci 6 4000 SUNW glc 3 fp 0 0 Device Address 50020 23000003c5 1 Class secondary State STANDBY As displayed in the output above the primary path is still online However given the nature of the errors CRC Loss of Signal do not assume there are no errors This could be an intermittent error 1 Fail the path to the secondary path 2 Test the FRUs in the primary path using the SANsurfer GUI 3 Fail the Sun StorEdge T3 array LUN to the secondary path man
60. 117 Serial Num Unsupported Unformatted capacity 119514 500 MBytes Write Cache Enabled Read Cache Enabled Minimum prefetch 0x0 Maximum prefetch 0x0 Device Type Disk device Path s dev rdsk c6t60020F20000003C53B699ABC00006C4Cd0s2 devices scsi_vhci ssd g60020f20000003c53b699abc00006c4c c raw Controller devices pci 6 4000 SUNW glc 2 fp 0 0 Device Address 50020 23000003d5 1 Class primary State OFFLINE Controller devices pci 6 4000 SUNW glc 3 fp 0 0 Device Address 50020 23000003c5 1 Class secondary State ONLINE This confirms that path devices pci 6 4000 SUNW qlc 2 fp 0 0 is OFFLINE You can determine if this disk is on the Master or AltMaster by keeping track of the Device Address that is on the primary controller in this case 50020 23000003d5 This is the WWN of the primary path controller Refer to 3 Check Array Status below for more information 3 Check Array Status Chapter 6 Examples of Fault Isolation 111 Briefly check the array status to ensure there are no major issues on the Sun StorEdge T3 array diagl190 lt l gt port list port targetid addr_type status host wwn ulpl A hard online sun 50020 23000003c5 u2p1 2 hard online sun 50020 23000003d5 From this output it is apparent that the problem is affecting the AltMaster u2p1 controller 50020 23000003d5 This correlates with the Device Address 50020 23000003d5 found in the previous step diag190 lt 2 gt port li
61. 13 44 35 diag226 Central Sun COM scsi ID 243001 kern info pci 6 4000 SUNW gqlc 2 fp 0 0 fcp0 ar 11 13 44 35 diag226 Central Sun COM Dynamically discovered 2 LUNs for D_ID 1046e4 ar 11 13 44 38 diag226 Central Sun COM qlc ID 686697 kern info NOTICE Qlogic qlc 0 Loop OFFLINE ar 11 13 44 43 diag226 Central Sun COM scsi ID 243001 kern warning WARNING scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 ar 11 13 44 43 diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Mar 11 13 45 51 diag226 Central Sun COM scsi ID 243001 kern warning WARNING scsi_vhci scsi_vhci0 Chapter 6 Examples of Fault Isolation 107 Mar 11 13 45 51 diag226 Central Sun COM scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 Command Timeout on path pcit 6 4000 SUNW qlc 2 fp 0 0 fp0 Mar 11 13 45 51 diag226 Central Sun COM scsi ID 243001 kern warning WARNING scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 Mar 11 13 45 51 diag226 Central Sun COM SCSI transport failed reason timeout retrying command Mar 11 13 45 55 diag226 Central Sun COM scsi ID 243001 kern warning WARNING scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 Mar 11 13 45 55 diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Mar 11 13 46 08 diag226 Central Sun COM fp ID 517869 kern warning WARNING fp 0 OFFLINE timeout Mar 11 13 46 35 diag226 Ce
62. 1346 02 3800 4800 4810 6800 Flash Prom E450 Flash Prom 3 22 0 106122 09 E250 Flash Prom 3 22 0 106530 06 E420R Flash Prom 3 29 0 109082 04 Note The packages and or patches listed in TABLE 2 3 may not be present in all configurations Array Storage Rules The following tables specify the supported features of the Sun StorEdge T3 array and the Sun StorEdge A3500FC array Note Sun StorEdge A5000 and Sun StorEdge A5100 arrays are not supported TABLE 2 4 Supported Features of the Sun StorEdge A5200 Array Feature Supported Cascading No Zone type SL zone only Maximum number of arrays per SL zone 3 Maximum initiators per SL zone 2 Maximum initiators per array 4 2 per loop Split loop support No Chapter 2 Configurations 17 TABLE 2 5 Supported Features of the Sun StorEdge T3 Array Feature Supported Cascading Yes Zone Type Name Server zone SL zone Maximum number of arrays per SL zone 8 Maximum initiators per LUN 2 Maximum initiators per zone 22 1 The host must be connected to the F Port on the switch Sun StorEdge T3 array must be connected to the TL port of the switch 2 This implies 2 initiators 2 hosts for simple arrays T3WG but 4 initiators 2 hosts for a partner pair T3ES Eash host has one path to each of the Sun StorEdge T3 arrays in the partner pair The Sun StorEdge A3500FC array is not supported on a host that has the Sun StorEdge Traffic Manager enabled with Sun StorEdge Traf
63. 5ktest Sun StorEdge FC Tape Test fctapetest Sun StorEdge PCI FC 100 Host Adapter Board Test ifptest Sun StorEdge PCI Dual Fibre Channel Host Adapter Board Test qlctest Sun StorEdge SBus FC 100 Host Adapter Board Test socaltest Sun StorEdge network FC switch 8 and switch 16 Switch Test switchtest Sun StorEdge T3 and T3 array Tests t30fdg t3test t3volverify Virtualization Engine Tests vediag veluntest Brocade Silkworm Test brocadetest From the Storage Automated Diagnostic Environment user interface you can select tests from the topology or from a list view When the tests execute the Storage Automated Diagnostic Environment initiates the test on the proper host You can retrieve test results by using the Test Manager selection from the GUI Running Diagnostic Tests From the GUI Window If you run the diagnostic test from the Storage Automated Diagnostic Environment main window you can easily access test configuration control and results using the buttons in the dialog boxes The test parameter options however are unique for each test and are illustrated in the individual sections with each test in this chapter 44 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only v To Access the Diagnostic Tests 1 Click the Diagnose link in the Storage Automated Diagnostic Environment main window 2 Click the Test link in the Diagnose page Ro SUN Storage Automated Diagnostic
64. 7 patchadd 111412 07 7 Install fct1 fp fcp usoc driver patch 111095 xx patchadd 111095 07 8 Install fcip driver patch patchadd 111096 03 9 Install qlc driver patch patchadd 111097 07 138 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only 10 Install Luxadm liba5k and lib_fc patch patchadd 111413 07 11 Install cfgadm plug in library patch patchadd 111846 03 12 Update MANPATH to access cfgadm_fp 1M man page For sh or ksh type MANPATH opt SUNWsan man SMANPATH export MANPATH For csh type setenv MANPATH opt SUNWsan man SMANPATH 13 Install the Storage Automated Diagnostic Environment package SUNWstade pkgadd d SUNWstade 14 Install the Storage Automated Diagnostic Environment Brocade patch patchadd 112580 01 15 Reboot the system For each of the storage devices upgrade the software firmware or configuration After the above steps you can leverage additional features provided by Brocade Silkworm 2400 8 port 2800 16 port for Sun StorEdge Traffic Manager functionality additional fabric zones additional initiators per zone host fabric connectivity cascaded switch configurations Quickloop zones Appendix A Brocade Upgrades and Installations 139 Installing Firmware on Brocade Silkworm Switches v To Download Firmware from the Brocade Web Site You can locate and downl
65. 9 User Name user root File Name usr switch firmware var tmp v2 6 x Protocol RSHD or FTP rshd ftp Password 84776 3832 130980 csum 2ef6 Loading to AMY sissies Seles Si eters aa ANS WEETANG Chas OM yeasts entcl Go esore dae yeh WET EIN Gia Si lie 3 4 Sieh totes wera wie eee eisai download complete oem240 admin gt fastboot 4 To download a firmware file from the host citadel using the account jdoe and the file home jdoe firmware type the following CODE EXAMPLE 6 4 RSHD Example swo admin gt firmwareDownload citadel jdoe home jdoe firmware 5 Reboot the switch Type fastboot Note It takes less than one minute to reboot the switch 6 Log back into the switch to verify the correct version as shown in CODE EXAMPLE A 4 Appendix A Brocade Upgrades and Installations 143 CODE EXAMPLE A 4 Verification of Firmware Version oem240 admin gt version Kernel 5 3 1 Fabric OS v2 6 x Made on Wed Sep 19 13 05 15 PDT 2001 Flash Wed Sep 19 13 07 16 PDT 2001 BootProm Thu Jun 17 15 20 39 PDT 1999 144 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Upgrading the SAN This section contains topics about how to upgrade your SAN system to the latest version Downloading Patches and Packages Download all patches and packages to the host prior to beginning the upgrade procedure Verif
66. Continuous test with watchdog timer disabled Dot on Faceplate es Notch on Switch shown in Force ae PROM position a A y Force PROM 4 Clicks FIGURE 4 1 Test Mode Switch Functions and Positions Chapter 4 Diagnosing and Troubleshooting the Switch 63 64 vy 1 To Troubleshoot Test Mode Switch Functions Use a small screwdriver to change the test mode switch positions Use the normal position as reference and count the number of clicks one click per position These clicks are not audible and are best detected by touch Isolate the switch chassis Data may be lost or corrupted if the test mode switch is used while data is being transmitted Using a small screwdriver rotate the test mode switch to the desired position Turn the power off and then back on to reset the switch chassis Observe the heartbeat LED for error codes five blinks is normal when in the Force PROM mode Correct conditions or reconfigure the switch as needed Return the test mode switch to the normal position aligning the small notch with the dot on the faceplate Turn the switch off and then back on to reset the switch chassis Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Heartbeat LED Blink Patterns Normal all pass If all POST diagnostics pass the switch goes to normal operation and the heartbeat LED blinks at a steady rate of one blink per second 1 sec
67. DIAG CMBISRF 1025 DIAG LCMRS 1026 DIAG LCMTO 1027 DIAG LCMEM 1028 DIAG LCMEMTX 1029 DIAG CMNOBUF 102A DIAG CMERRTYPE 102B DIAG CMERRPTN 102C DIAG INTNOTCLR 1030 DIAG BADINT 106F DIAG TIMEOUT 190 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE C 2 Error Message Codes Defined Continued Error Number Test Name Error Name 2030 cmiTest DIAG BADINT 2031 DIAG INTNIL 2032 DIAG CMISA1 2033 DIAG CMINOCAP 2034 DIAG CMIINVCAP 2035 DIAG CMIDATA 2036 DIAG CMICKSUM 223B camTest DIAG CAMINIT 223C DIAG CAMSID 2640 portLoopbackTest DIAG ERRSTAT ENCIN 2641 DIAG ERRSTAT CRC 2642 DIAG ERRSTAT TRUNC 2643 DIAG ERRSTAT 2LONG 2644 DIAG ERRSTAT BADEOF 2645 DIAG ERRSTAT ENCOUT 2646 DIAG ERRSTAT BADORD 2647 DIAG ERRSTAT DISCC3 264F DIAG INIT 265F DIAG PORT_DIED 266E DIAG DATA 266F DIAG TIMEOUT 2660 DIAG STATS FTX 2661 DIAG STATS FRX 2662 DIAG STATS C3FRX 2670 DIAG PORTABSENT 2671 DIAG XMIT Appendix C Brocade ErrorMessages 191 TABLE C 2 Error Message Codes Defined Continued Error Number Test Name Error Name 3040 crossPortTest DIAG ERRSTAT ENCIN 3041 DIAG ERRSTAT CTL 3042 DIAG ERRSTAT TRUNC 3043 DIAG ERRSTAT 2LONG 3044 DIAG ERRSTAT BADEOF 3045 DIAG ERRSTATENCOUT 3046 DIAG ERRSTAT BADORD 3047 DIAG ERRSTAT DISC3 304F DIAG INIT 305F DIAG PORTDIED 3060 DIAG STATS FTX 3061 DIAG STATS FRX 3062 DIAG STATS C3FRX 306E DIAG DATA 306F DIAG
68. Diagnostic Environment 4 1 VTSID 6008 switchtest FATAL Port 5 is Offline with Pattern 0x4a4a4a4a on Device qlc3 sw2 i4 d5 Probable_Cause s lt Fibre Channel cable disconnected to HBA gt lt Bad GBIC or bad Fibre Channel cable gt Based on the Storage Automated Diagnostic Environment email that pointed out the failed GBIC on IB board B you should consider the failed GBIC your first suspect If you did not have the email notice you could use Storage Automated Diagnostic Environment Tests or the Sun StorEdge Sansurfer GUI to help isolate the individual FRUs in the link a Switch switch port switch port GBIC m Cable m Sun StorEdge A5200 array GBIC To isolate the above FRUs start with what you can test using the switch a Insert the loopback connector into port 5 and rerun switchtest a If it passes you can rule out the switch port and switch GBIC a If it fails replace the GBIC and try again If the second test fails replace the switch b Since the path is uncabled move the GBIC from the Sun StorEdge A5200 array to the same switch port and test it as well a fit passes the problem is the cable Move the GBIC to its original location and replace the cable a If it fails replace the GBIC In this case the problem was a bad GBIC in IB Board B c Replace the GBIC 130 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only 6 Verify the fix In the previous example the
69. IGURE 6 3 Sansurfer GUI Port 2 Counter Status 5 Start testing FRUs The high number of errors counters shown in FIGURE 6 3 is a cause for concern However it should not be used without taking into account the interval over which the counters have been accumulating A switch that has not been reset in six months may very well have a high error counter rate 102 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Take care to observe the counters over a period of time and note any increases One technique is to reset the counters in the GUI a Click the Special gt Reset Count menu b Use the Storage Automated Diagnostic Environment switchtest or the SANsurfer Port test to generate load between the switches and watch for rising error counters The first Storage Automated Diagnostic Environment switchtest is run on the entire link starting from the local switch This test exercises the entire switch to switch link The output is shown below opt SUNWstade diags bin switchtest v o dev qlc0 sw0 3 e2 switchtest called with options dev qlc0 sw0 f3 e2 switchtest Started Start switchtest Testing device qlc0 sw0 f3 e2 Chassis Status for device glc0 sw0 f3 e2 Power OK Temp OK 31 0c Fan 1 OK Fan 2 OK Testing Device qlc0 sw0 f3 e2 Port 2 Pattern 0x7e7e7e7e 08 10 01 20 26 02 diag226 Central Sun COM Storage Automated Diagnostic Environment VTSID 6006 swit
70. IMERS MSG Invalid message OS error Contact customer ILOG_WARNING support FLANNEL PHANTOM Port s PLT limit exceeded OS error Contact customer ILOG_WARNING support IASIC MINI_BUFFER ILOG_WARNING ASIC Failure Bad mainboard Contact customer support ILSDB LSID LOG_ERROR Link State ID d out of range OS error Contact customer support ILSDB NOLOCALENTRY No database entry for local OS error Contact customer ILOG_CRITICAL link state record support ILSDB NOLSR No link state record for OS error Contact customer ILOG_WARNING domain support ILSDB MAXINCARN Local link state record OS error Contact customer ILOG_WARNING reached max incarnation support IFLOOD INVLSU Discard received LSU OS error Contact customer ILOG_WARNING support IFLOOD INVLSR Unknown LSR type OS error Contact customer ILOG_WARNING support IFLOOD LSRLEN Excessive LSU length OS error Contact customer ILOG_ERROR support IHLO INVHLO Invalid Hello received from OS error Contact customer ILOG_ERROR port support IHLO HLOTIMEOUT Incompatible Hello timeout OS error Contact customer ILOG_ERROR from port support Appendix C Brocade Error Messages 201 TABLE C 3 Diagnostic Error Messages Continued Message Description Probable Cause Action HLO DEADTIMEOUT Incompatible inactivity OS error Contact customer ILOG_ERROR timeout from port support IFSPF SCN LOG_WARNI
71. KKKKKKKKKKKKKKKKKKKKKKKKKKKKKEK Detected possible bad Power supply Power Supply 1 is absent KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK Power Supply 2 is OK Close 172 20 67 167 Appendix B Brocade Troubleshooting 157 Other Diagnostic Tools Brocade Silkworm switches also support a wide range of CLI tests that can be invoked while connected directly to the switch via a serial connection to the Silkworm 2400 by opening a telnet session or by way of the front panel of the Silkworm 2800 Some of the tests pertinent for troubleshooting are listed below Storage Automated Diagnostic Environment s brocadetest invokes crossPortTest for testing loopback ports spinFab for testing E Port connections between switches and loopPortTest for testing L Ports m supportShow m switchShow m qlShow m diagShow m crossPortTest m loopPortTest m spinFab m nsShow supportShow support Show runs nearly all commands and should be gathered when placing a service call or escalation The Explorer Data Collection utility SUNWexplo gathers the support Show output if the Brocade Silkworm switch is placed in the opt SUNWexplo etc saninput txt Input file for extended data collection Format is SWITCH SWITCH TYPE PASSWORD LOGIN Valid switch types are ancor and brocade LOGIN is required for brocade switches the default is admin 172 20 67 167 brocade password admin 172 20 67 164 brocade password admin 158 Sun StorEdge SAN re
72. LE C 3 Diagnostic Error Messages Continued Message Description Probable Cause Action IUCAST RRTIM Invalid reroute timer ID OS error Contact customer ILOG_CRITICAL support UCAST SPFCOST No minimum cost path in OS error Contact customer ILOG_WARNING candidate support UCAST RELICPDB Relic PDB to Domain OS error Contact customer ILOG_WARNING Appendix C support Brocade Error Messages 203 204 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Acronyms amp Glossary This glossary contains a definitions for terms used in this guide Acronyms AL_PA Arbitrated Loop Physical Address 8 bit value used to identify a device F_Port Ona Fibre Channel switch a port that supports an N_Port A Fibre Channel port in a point to point or Fabric connection FL_Port Ona Fibre Channel switch a port that supports Arbitrated Loop devices G_Port Ona Fibre Channel switch a port that supports either F_Port or E_Port NAS Network Attached Storage N_Port A Fibre Channel port in a point to point or Fabric connection NL_Port Node loop port a port that supports Arbitrated Loop protocol SL_Port Segmented Loop Port A port connected to a private loop device SNDR Sun StorEdge Network Data Replicator formerly Sun StorEdge Remote Dual Copy T_Port An inter switch port one that is used to attach a port on one switch to a port on another switch TL_Port A Translated Loop Port on the Sun StorEdge
73. N 3 1 The packages on your system that were previously used should be available To verify their availability use the pkginfo command pkg_name is the name of the package on which you need to obtain the information pkginfo 1 pkg_name 1 Upgrade your SUNWsan package to StorEdge SAN 3 1 Before you start check your system to see if it has been installed and if it is already up to date Use the pkginfo command to see if it has been installed pkginfo SUNWsan 146 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only The system returns system SUNWsan SAN Foundation Kit The output indicates that the SUNWsan package has already been installed If your system has the SUNWsan package installed use the showrev command to see if it is up to date showrev p grep 111847 The system returns Patch 111847 01 Obsoletes Requires Incompatibles Packages SUNWsan The showrev results line indicates that the SUNWsan package is already up to date If patch 111847 01 has not been installed install it using the pat chadd command patchadd patch location 111847 01 2 If your system does not have the SUNWsan package installed install the new SUNWsan package from your StorEdge SAN 3 1 software pkgadd d pkg location SUNWsan cfgadm Plug in Library Packages 1 Install cfgadm plug in library packages SUNWcfpl and SUNWcfpl1x pkgadd d pkg location SUNW
74. NG Illegal SCN OS error Contact customer support IFSPF NBRCHANGE Wrong neighbor ID in Hello OS error Contact customer ILOG_WARNING message from port support IFSPF INPORT LOG_ERROR Input port out of range OS error Contact customer support IFSPF VERSION FSPF version not supported OS error Contact customer ILOG_ERROR support IFSPF SECTION Wrong section ID OS error Contact customer ILOG_ERROR support IFSPF REMDOMAIN Remote Domain ID out of OS error Contact customer ILOG_ERROR range support INBFSM NGBRSTATE Wrong input to neighbor FSM OS error Contact customer ILOG_ERROR support MCAST ADDPORT Add port failed OS error Contact customer ILOG_WARNING support IMCCAST REMPORT Remove port failed OS error Contact customer ILOG_WARNING support MCAST ADDBRANCH Add branch failed OS error Contact customer ILOG_ERROR support MCAST REMBRANCH Remove branch failed OS error Contact customer ILOG_ERROR support IMCAST NOPARENT Null parent OS error Contact customer ILOG_ERROR support IMCAST NOPARENTLSR Null IsrP OS error Contact customer ILOG_ERROR support UCAST ADDPATH Add path failed OS error Contact customer ILOG_CRITICAL support IUCAST ADDPORT Add port failed OS error Contact customer ILOG_WARNING support IUCAST REMPORT Remove port failed OS error Contact customer ILOG_WARNING 202 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only support TAB
75. Ns C4 50020 23000003d5 is unusable b luxadm e port luxadm e port Found path to 4 HBA ports devices pci 6 4000 SUNW glc 2 fp 0 0 devctl devices pci 6 4000 SUNW qlc 3 fp 0 0 devctl devices pci 6 2000 pci 1 SUNW qlc 4 fp 0 0 devetl devices pci 6 2000 pci 1 SUNW glc 5 fp 0 0 devctl NOT CONNECTED CONNECTED CONNECTED CONNECTED a The device devices pci 6 4000 SUNW glc 2 f p 0 0 devct1 has lost all of its targets in this example it had only one target The Host to HBA connection seems like a good place to begin troubleshooting c Multipathing Status 1 From the command line search for a disk with GUID 60020 f20000003c53b699abc00006c4Cc 2 Display luxadm on the disk to get the Sun StorEdge Traffic Manager status Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only An example is shown below ls la dev rdsk grep i 60020 20000003c53b699abc00006c4c grep s2 lrwxrwxrwx 1 root root 67 Mar 10 17 38 c6t60020F20000003C53B699ABCO0006C4Cd0s2 gt devices scsi_vhci ssd g60020f20000003c53b699abc00006c4c c raw luxadm display dev rdsk c6t60020F20000003C53B699ABC00006C4Cd0s2 DEVICE PROPERTIES for disk dev rdsk c6t60020F20000003C53B699ABCO00006C4Cd0s2 Status Port A O K Status Port B O K Vendor SUN Product ID T300 WWN Node 50020 20000003c5 WWN Port A 50020 23000003d5 WWN Port B 50020 23000003c5 Revision 0
76. ODE EXAMPLE 1 1 kernel drv scsi_vhci conf Copyright c 2001 by Sun Microsystems Inc All rights reserved pragma ident scsi_vhci conf 12 01 03 19 SMI name scsi_vhci class root Sun StorEdge Traffic Manager Global enable disable configuration possible values are Sun StorEdge Traffic Manager disable no or Sun StorEdge Traffic Manager disable yes Sun StorEdge Traffic Manager disable no Load Balancing global configuration possible values are load balance none or load balance round robin load balance round robin Chapter 1 Introduction 7 For individual host bus adapter control first retrieve a list of Fibre Channel adapters on the host ls 1 dev fc total 12 lrwxrwxrwx 1 root root 49 May 3 15 43 fp0O gt devices pci 6 4000 SUNW qlc 2 fp 0 0 devctl lrwxrwxrwx 1 root root 49 May 3 15 43 fpl gt devices pci 6 4000 SUNW glc 3 fp 0 0 devctl lrwxrwxrwx 1 root root 55 May 3 19 43 p2 gt devices pci 6 4000 pcit 4 SUNW glc 4 fp 0 0 devctl lrwxrwxrwx 1 root root 55 May 3 15 43 fp3 gt devices pci 6 4000 pcit 4 SUNW glc 5 fp 0 0 devctl lrwxrwxrwx 1 root other 55 May 10 11 56 fp4 gt devices pci 6 2000 pci l SUNW glc 4 fp 0 0 devctl lrwxrwxrwx 1 root other 55 May 10 11 56 fp5 gt devices pci 6 2000 pci l SUNW glc 5 fp 0 0 devctl Then modify the kernel drv qlic conf file as shown in the example name qlc parent
77. RECOMMENDED ACTION 1 Verify cables GBICs and connections along Fibre Channel path 2 Check SAN Topology GUI to identify failing segment of the data path 3 Verify correct FC switch configuration 4 Verify port is enabled brocade MessageLog on 172 20 67 167 ip 172 20 67 167 PhysicalState of port 2 changed from inSync 6 to nolLight 4 status of hba devices pci 9 600000 pci 2 SUNW qlc 4 fp 0 0 devctl on diag229 central sun com changed from CONNECTED to NOT CONNECTED INFORMATION monitors changes in the output of luxadm e port Found path to 20 HBA ports devices sbus 2 0 SUNW socal d 10000 0 NOT CONNECTED FIGURE B 3 Storage Automated Diagnostic Environment Alert This Alert shows m Anerror on port two of switch 172 20 67 167 occurred m A Sun StorEdge Traffic Manager offline event occurred a The HBA is offline Appendix B Brocade Troubleshooting 175 Step 2 Determine the extent of the problem using cfgadm al output CODE EXAMPLE B 9 cfgadm al output to check OS level fabric condition c2 fc fabric connected configured unknown c2 50020f2300003d2c disk connected configured unusable E3 fc fabric connected configured unknown c3 50020f2300003ee5 disk connected configured unknown The device on c2 has disappeared In addition the luxadm output of the Sun StorEdge T3 arrays shows the following luxadm display dev rdsk c6t60020F2000003EE53AAF7A0 9000DA257d0s2 DEVICE PROPERTIES for disk dev rds
78. Recommended Action To isolate to a single FRU please run Storage Automated Diagnostic Environment tests on affected components Register Origin qlc0 sw0 f3 e2 Register Destin qlc0 sw0 f3 e2 swl e2 From this output note the following m SCSI errors exist that affect scsi_vhci ssd g60020f20000003c53b699abc00006c4c m Various link errors exist on switch 172 20 67 155 and switch 172 20 67 156 a Storage Automated Diagnostic Environment suggests five suspect components including GBICs a cable and switch ports 98 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only 2 Determine the extent of the problem a Using the SAN Topology functionality of the Storage Automated Diagnostic Environment determine between which switch pairs the link error is located b cfgadm al output cfgadm al c4 fc fabric connected configured unknown c4 50020 23000003d5 disk connected configured unknown e5 fc fabric connected configured unknown c5 50020f23000003c5 disk connected configured unknown The cfgadm output shown above appears to be clean with no hard failures noted c Sun StorEdge Traffic Manager information To find the Sun StorEdge Traffic Manager information for scsi_vhci ssd g60020f20000003c53b699abc00006c4c you first need to find a physical path ls la dev rdsk grep i 60020 20000003c53b699abc00006c4c grep s2 lrwxrwxrwx 1 root root 67 Mar 10 17 38 c6t6
79. StorEdge A5200 Array 7 8 IBA E Ieee IBB FIGURE 2 13 Two Hosts Connected to Sun StorEdge T3 Array Partner Group Each Host with Separate Non shared Storage 38 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only 1 2 3 4 5 6 Sun Enterprise E220 Host A 7 8 3 Host Adapter Host Adapter 3 6 7 8 Host Adapter E 9 10 F Host Adapter Cia 12 13 14 Sun StorEge A5200 Arrays Tape library 15 16 Switch 0 Sun Enterprise E3000 gt d Host A 1 3 4 Host Adapter _ 5 6 Host Adapter 7 8 Host Adapter 9 10 1 2 Host Adapter 11 12 3 4 13 14 15 16 S Switch 1 7 8 Nameserver zone and SL zone Switch 3 cannot be in the same zone Switches 0 1 2 3 Nameserver Zone 1 and Switch 0 1 Ports 1 F 2 T 3 4 F T Switc Ports h 1 2 3 4 2 1 7 Segmented Loop Zones 1 1 TL Switch 1 1 T Ports 5 6 7 8 All Ports SL TL Ports 9 10 All ports SL Array and Sun StorEdge A5200 Array Tape libraries
80. TIMEOUT 3070 DIAG PORTABSENT 3071 DIAG XMIT 3078 DIAG PORTWRONG 192 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE C 2 Error Message Codes Defined Continued Error Number Test Name Error Name 384F spinSilk DIAG INIT 385F DIAG PORTDIED 3840 DIAG ERRSTAT ENCIN 3841 DIAG ERRSTAT CRC 3842 DIAG ERRSTAT TRUNC 3843 DIAG ERRSTAT 2LONG 3844 DIAG ERRSTAT BADEOF 3845 DIAG ERRSTAT ENCOUT 3846 DIAG ERRSTAT BADORD 3847 DIAG ERRSTAT DISCC3 3870 DIAG PORTABSENT 3871 DIAG XMIT 3874 DIAG PORTSTOPPED Appendix C Brocade Error Messages 193 TABLE C 3 Diagnostic Error Messages Message Description Probable Cause Action IDIAG BADINT Port received an interrupt ASIC failure Replace mainboard Err 1030 2030 when not expecting one assembly centralMemoryTest cmiTest IDIAG BUS_TIMEOUT IErr OBOF 4040F portReg Test sramRetentionTest IDIAG CAMINIT Err 223B camTest DIAG CAMSID ASIC register or ASIC SRAM did not respond to an ASIC data access Port failed to initialize due to one of the following reasons e Switch not disabled e Diagnostic queue absent e Malloc failed e Chip is not present e Port is not in loopback mode e Port is not active ASIC failed SID NO ASIC failure Software operational setup error or mainboard failure ASIC failure Replace mainboard assembly Retry reboot or replace mainboard assembly Rep
81. The tape library controller is accessible to one host only FIGURE 2 14 Multi Host with Cascading Switches Connected to FC Tape Sun StorEdge T3 Chapter 2 Configurations 39 Note You must enable Sun StorEdge Traffic Manager software for failover across multiple hosts to function The mp_support on the Sun StorEdge T3 array should be set to mpxio 40 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CHAPTER 3 Diagnostic Tools This chapter provides an overview of the tools you can use to monitor diagnose troubleshoot and gather information on the Sun StorEdge SAN release 3 0 Detailed installation and configuration information can be found in the respective documentation of the tool Storage Automated Diagnostic Environment The Storage Automated Diagnostic Environment is a host based online health and diagnostic monitoring tool for storage area network SAN and direct attached storage DAS devices It can be configured to monitor on a 24 hour basis collecting information that enhances the reliability availability and serviceability RAS of the storage devices The Storage Automated Diagnostic Environment offers the following features A common web based user interface for device monitoring and diagnostics m Distributed test invocation by means of lists or topology m Topology grouping for multi level hosts and components a Alternate master support for redundancy
82. _front 6 wwn in a5k DPL4 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g2000002037450d4d ssd58 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 2200002037450d4d 0 is offline disk_rear 2 wwn in a5k DPL4 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g2000002037450d4c ssd57 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 2200002037450d4c 0 is offline disk_rear 6 wwn in a5k DPL4 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g2000002037450d6b ssd56 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 2200002037450d6b 0 is offline disk_front 9 wwn in a5k DPL4 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g200000203719eb58 ssd55 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 220000203719eb58 0 is offline disk_front 5 wwn in a5k DPL4 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 k
83. a You can configure a maximum of three StorageTek 9840FC drives per zone m A library may share a zone with up to three drives a A single zone cannot have more than one tape library Chapter 2 Configurations 19 20 Switch Port Types Port Type Supported SL Ports Private loop Sun StorEdge A5200 array Sun StorEdge A3500FC array Sun StorEdge T3 array L180 or L700 FC Tape device TL Ports Translated loop Sun StorEdge T3 array F Ports Point to Point fabric HBA T Ports Trunk ports for switch cascading ISLs configured initially in Fabric port mode Zones Zoning allows the user to divide the switch ports into zones for more efficient and secure communication among functionally grouped nodes There are several types of zones and a port may be defined in any No port can be in all zone types simultaneously Hard Zones Hard zones isolate ports through internal switch hardware no communication across hard zone boundaries is possible Hard zoning provides a way of isolating a set of ports from other ports within the fabric desirable for security and resource dedication Up to 16 hard zones are possible within a fabric A particular port may be placed in only one hard zone no overlapping hard zones If hard zones are enabled name server zones and SL zones do not communicate across defined hard zone boundaries Name Server Zones Name server zones allow the division of the fabric one or more Switch chassis into as many a
84. al sun com a Ae Jy Host san MASTER Fitter None fco Midiag229 a amp TOPOs MASTER Clear Help Topology Help After having selected a topology several functions are available in the graphical view 1 Clicking in the middle of an object or in a specific component of an object will display the status at the bottom of the page 2 Right clicking the background will present a zoom menu 172 20 67 168 _ j2 172 20 67 167 172 20 67 164 3 Right clicking on an object will present a menu of functions available for that object 4 Right clicking on a link will present a menu of functions available for that link u a 172 20 67 168 slr mi 370 3990 01 e f0 016101 monitored by diag229 5 Holding the Shift button while selecting objects allows to select ry multiple objects This can be used to XY Layout Saye Graphics Turn On Layout Change monendo eaeaeeameiiaia FIGURE B 2 Storage Automated Diagnostic Environment Topology AppendixB Brocade Troubleshooting 173 174 Step 1 Discover the Error using Storage Automated Diagnostic Environment Alerts Site Lab Broomfield Source diag229 central sun com Severity Warning Category Message DeviceId message diag229 central sun com EventType LogEvent driver MPXIO_offline EventTime 03 19 2002 13 10 03 Found 2 driver MPXIO_offline warning s in
85. annel port loopback test verifies the ability of each switch ASIC to loop data out through each fibre channel port through a loopback plug and back to the ASIC control port In order to accomplish this test you must attach a loopback plug to each GBIC as you test it Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only To Test the GBIC Place the chassis into Continuous Test Remove all GBICs from the chassis except the one you want to test The GBIC under test may be in any port The Continuous Test skips all empty ports Insert a loopback plug into the GBIC Cycle the chassis power to cause a reset After a few seconds of testing if the heartbeat LED is blinking about once per second normal the GBIC passes the test If the heartbeat LED blinks the eight blink error code the GBIC failed Repeat steps 2 through 5 to test all the GBICs one at a time When all the tests are complete place the test mode switch back in the Normal Run position small dot on the end of the shaft pointing straight up Cycle the chassis power to cause a reset Switch Bus Test Failure Nine Blinks The switch is not operable The switch bus test verifies the ability of the switch ASICs to communicate with each other via the buses that interconnect the ASICs A failure indicates an inability of an ASIC pair to communicate over one or more buses The heartbeat LED blinks nine times be
86. ated with a Fibre Channel port is an example of a non fatal error The switch can isolate the bad port while the other ports continue to operate Note In the following POST error descriptions note that some errors result in a switch that is operable but in a degraded way non fatal errors Other errors result in a switch that is not operable fatal errors If the problem is non fatal you can run in a degraded mode until the problem is fixed When POST is complete and errors are encountered the switch uses the heartbeat LED to blink an error code that describes the first fatal error encountered The LED blinks in a pattern relating to the failure pauses and then restarts the same blinking pattern The switch then reads its error log and if it has encountered non fatal errors that affect one or more ports with remaining ports operable it disables the bad ports and blinks the Logged in LED of the affected port or ports If the errors is non fatal but does not affect a single port or group of ports only the heartbeat LED blinks an error code In all cases the switch displays the POST error indications until you power it off For example 60 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only If the POST encounters a PROM checksum error the entire switch is inoperable The heartbeat LED blinks the error code for the fatal POROM checksum error The entire switch is down and no port Log
87. below The results are captured in the Webgui Webgui log file 08 10 2001 20 41 09 Switch switch 156 Port 2 Begin 60 second online test frame size 2148 bit pattern 0x7e7e7e7e Test results recorded in Webgui Weblog gui 08 10 2001 20 42 10 Switch switch 156 Port 2 online test completed 08 10 2001 20 42 10 Switch switch 156 Port 2 Test results Port 2 Test Loops 122843 Failures Detected 0 From the results of this test you can conclude that the remote switch and GBIC are also not the cause of the error This leaves the ISL cable as suspect At this point contact a Third Party fiber cable provider to investigate the 10km link For this example you can conclude the cable was the cause for the error and proceed as if it were fixed 6 Verify the Fix After repairing the faulty cable you should see messages such as the one below appear in var adm messages when the link is reconnected Mar 10 20 45 43 diag226 Central Sun COM scsi ID 243001 kern info pci 6 4000 SUNW qlc 2 fp0 0 fcp0 Mar 10 20 45 43 diag226 Central Sun COM Dynamically discovered 2 LUNs for D_ID 1046e4 The messages above indicate that this link is again seeing the Sun StorEdge T3 array LUNs across the cascaded link You should also see the ISL ports come back online in the SANsurfer GUI a You can now re enable the primary path to the affected LUN s using Sun StorEdge Traffic Manager For example luxadm failover pri
88. c wide regardless of fabric configuration but within hard zone boundaries m There is a maximum of 256 name server zones with a maximum of 16 name server zones per hard zone a The switch allows ports to be in more than one name server zone however this is not supported in this phase a When a port receives name server information it will receive information about all ports in the name server zone in which the port is assigned a All ports not defined as being part of any enabled name server zone are name server zone orphans Name server zone orphans are all listed in the Name Server Orphan Zone Chapter 1 Introduction 3 4 Cascading T Ports In the Sun StorEdge SAN release 3 1 switches are allowed to be cascaded together Cascading is allowed with either shortwave or longwave GBICs Use of shortwave GBICs allows a higher port count in a local configuration The use of longwave GBICs and long haul fiber optics allows users to reach geographically separated storage and servers perhaps for disaster recovery purposes T Ports Trunking Ports provide the switch to switch connections Any F or FL port is capable of being configured as a T Port once the physical connection is made No user intervention is required other than ensuring the port type is either F or FL before connecting the switches You can cascade a maximum of two switches with one ISL hop between the initiator and target with distances of 10 meters and 10 kilom
89. ce is E4 Knowing this information you can easily determine where this device is located in the SAN Chapter 6 Examples of Fault Isolation 87 TABLE 6 1 ASIC amp Port Values Switch Port ASIC ID Port ID 1 0 0 2 0 1 3 0 2 4 0 3 5 1 0 6 1 1 7 1 2 8 1 3 9 2 0 10 2 1 11 2 2 12 2 3 13 3 0 14 3 1 15 3 2 16 3 3 88 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Example 1 Switch to Storage Error 1 Discover the Error The first indication of a problem surfaced from a Storage Automated Diagnostic Environment email You requested the following events be forwarded to you from diag226 central sun com Source Agent diag226 central sun com Category Message EventType AlarmEvent EventTime 08 10 2001 14 30 08 Message s found in logfile var adm messages on diag226 central sun com id 80fee746 Warning Mar 10 14 27 46 SCSI diag226 Central Sun COM scsi ID 107833 kern notice ASC 0x47 scsi parity error ASCQ 0x0 FRU 0x0 Warning Mar 10 14 27 46 SCSI diag226 Central Sun COM scsi ID 107833 kern notice ASC 0x47 scsi parity error ASCQ 0x0 FRU 0x0 Warning Mar 10 14 27 47 SCSI diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Warning Mar 10 14 27 47 FABRIC diag226 Central Sun COM fp ID 517869 kern warning WARNING fp 1 N_x Port with D_ID 1006e8 PWWN 50020f23000003c5 disappeared from fabric
90. ces scsi_vhci ssd g20000020374504e2 c raw devices pci 6 2000 pci l SUNW glc 4 fpe0 0 21000020374504e2 0 primary ONLINE devices pci 6 2000 pci l SUNW qlc 5 fpe0 0 22000020374504e2 0 primary ONLINE Other resources to check if applicable a Storage Automated Diagnostic Environment agent messages m Storage Automated Diagnostic Environment diagnostic tests 132 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only APPENDIX A Brocade Upgrades and Installations Installing a New SAN This section contains topics that describe how to install a new SAN system using Brocade Silkworm switches Required Software Components The following software components might be required for your particular SAN system Refer to the documentation for each individual product to insure proper installation Software Components m Sun StorEdge Traffic Manager This is available as a patch which can be installed on Solaris 8 release 10 01 or later It should be installed with the latest revision of Sun StorEdge Network Foundation Software a Sun StorEdge Network Foundation This software is included with the Solaris upgrades for the FC switch product It includes the drivers to support switches management tools and storage devices 133 134 a VERITAS Volume Manager VERITAS Volume Manager is a separately installed software product that provides host based storage management such a d
91. cfp1l pkgadd d pkg location SUNWcfp1x pkgadd d pkg location SUNWcfp1x Appendix A Brocade Upgrades and Installations 147 Software Installation For the Brocade Silkworm switch the GUI capability is available through the switch firmware and requires a Java enabled browser on the host Refer to Related Brocade Documentation in Appendix B for instructions on how to access Brocade documentation Note Once the packages have successfully installed follow steps 6 through 13 in To Install the Software on page 138 v To Upgrade the Storage Automated Diagnostic Environment Package For all upgrades you must first install the most recent Leadville patches Refer to To Install the Software on page 138 for installation instructions before installing the SUNWst ade package and the Brocade patch For detailed installation and usage instructions for the Storage Automated Diagnostic Environment refer to the Storage Automated Diagnostic Environment User s Guide part number 816 3142 11 1 If your SAN Management host is not running the current version remove the existing package and install the latest version Remove the old package with the pkgrm command pkgrm SUNWstade Removing the initial installation does not erase the previous configuration information The cache and topology information of each device is also retained to maintain a consistent historical view of the Sun StorEdge devices 2 Reinstall th
92. chtest FATAL Switchtest failed with bad Pattern 0x7e7e7e7e on Device qlc0 sw0 f3 e2 Port 2 Field loss_of_signal_cnt Probable_Cause s lt Faulty hba gbic hub switch cable disk gt lt Another user may be already running Switchtest on this port gt Recommended_Action s lt See var adm messages for more information gt lt Select StorEdge Expert Mode or run stexpert from the cli to re test the selected device to see if the problem persists StorEdge Expert will attempt to isolate the failing FRU gt lt If the problem persists call your authorized Sun service provider gt c Isolate the local switch and GBIC from the rest of the link First uncable the link and insert a loopback connector into the local switch GBIC Then rerun the test Note A green logged in indicator does not appear when you insert a loopback connector into a T port as well as F port making it difficult to determine if the loopback is inserted correctly Even though there is no green indicator the test does run and reports back results Chapter 6 Examples of Fault Isolation 103 opt SUNWstade diags bin switchtest v o dev qlc0 sw0 f3 e2 switchtest called with options dev qlc0 sw0 f3 e2 switchtest Started switchtest Starts Testing Chassis Testing Testing Testing Testing Testing Testing Testing Testing Testing Testing Port 2 passed all tests on qlc0 sw0 f3 e2 switchtest Stopped successfully T
93. ci ssd g60020f2000003ee53aaf7a09000da257 c raw Controller devices pci 9 600000 pci 2 SUNW qlc 4 fp 0 0 Device Address 50020 2300003d2c 1 Class primary State ONLINE Controller devices pci 9 600000 pci 2 SUNW qlc 5 fp 0 0 Device Address 50020 2300003ee5 1 Class secondary State STANDBY The primary path is now restored as the ONLINE path and I O is again using this path As a final check look to the Storage Automated Diagnostic Environment topology The ports that were in error are now green and the mpx error is green as well as shown in FIGURE B 8 184 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only ance Moniter Dia Ro SUN rage Automated Diagnostic Envi microsystems 2 0 06 010 diag229 centr Devices fopology Log Host san MASTER Filter None a co Eea Topology Help After having selected a topology several functions are available in the graphical view 1 Clicking in the middle of an object or in a specific component of an object will display the status at the bottom of the page 2 Right clicking the background will present a zoom menu 3 Right clicking on an object will present a menu of functions available for that object 4 Right clicking on a link will present a menu of functions available for that link 5 Holding the Shift button while selecting objects allows to select multiple objects This can be used to move multiple object
94. cific ASIC Intr low Bus ASIC 2 Intr low Bus ASIC 3 Out of buffers Number of large frames that have been sent by this switch Out of s buffers Number of small frames that have been sent by this switch Switch resets Number of times the switch has been reset since it was manufactured Available only for switches with more than 8 ports 80 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CHAPTER 6 Examples of Fault Isolation Overview In this section several example scenarios are detailed and tips are given about where to look for errors and how to interpret various outputs and displays The scenarios will cover these situations a Switch to Storage Fault Sun StorEdge T3 array m Switch to Switch Fault T Port cascades m Switch to HBA Fault m Sun StorEdge A5200 array Example In all scenarios command line equivalents will be demonstrated where possible Also in all scenarios it is assumed that the latest patches firmware and packages are running Sun StorEdge Traffic Manager is enabled in all cases to simulate a redundant configuration however no volume management software is running except if explicitly noted 81 Setup Requirements m One Enterprise 450 Workgroup Server Solaris 8 update 6 with all relevant Sun StorEdge SAN release 3 0 patches and packages a One Sun StorEdge T3 array Partner Pair 2 LUNs per brick One Sun StorEdge A5200 array
95. e Brocade Silkworm 2400 Hardware Reference Manual for further serial port details The Front Panel access method on the 2800 switch can be used to run most commands that the switch supports However the screen is limited in size and messages are restricted to one or two lines of output Once the IP addressed is configured through the front panel further switch setup and diagnostics can be run via a telnet connection or the WebTools GUI See the Brocade Silkworm 2800 Hardware Reference Manual for more details on the front panel operation WebTools GUI The WebTools GUI is a separately licensed feature All Brocade switches that are sold by Sun Professional Services should come with the license pre installed You can access the WebTools GUI with a standard web browser Netscape or Microsoft Internet Explorer with a Java Plugin by pointing the browser to http lt ip_address_of_switch gt Note The Java Plugin that is supplied with Solaris 8 U6 is required To Verify the Web License type the following admin gt licenseshow SeRdQeQSbzT ESqsSyY Web license Zoning license Quickloop license Appendix B Brocade Troubleshooting 165 http 172 20 67 167 F FIGURE B 1 Brocade Webtools GUI See the Brocade Web Tools User s Guide for more information on WebTools usage Note The rest of this guide will assume telnet usage 166 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 Fo
96. e Traffic Manager StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g2000002037450d3a ssd47 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target s offline disk_rear 9 wwn in a5k DPL4 address 2200002037450d3a 0 i Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun diag226 Central Sun COM Sun diag226 Central Sun COM Sun diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g20000020374504e2 ssd46 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 22000020374504e2 0 i s offline disk_rear 10 wwn in a5k DPL4 Chapter 6 Examples of Fault Isolation 123 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g20000020374507de ssd45 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 22000020374507de 0 is offline disk_rear 8 wwn in a5k DPL4 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g20000020373cclac ssd44 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 22000020373cclac 0 is offline disk_rear 4 wwn in a5k DPL4 Threshold Error Mar 8 13 13 46 Received 1 Loop Off
97. e available with Explorer 3 5 1 m There is no support for the Sanbox API For more information and to download these utilities go to http diskworks ebay SW sw html 48 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Sample Capture Output Capture Version 1 0 1 IP Address 172 20 67 155 KKKKKKKKKKKKKKKKKKKK Version Information KKEKKKKKKKKKKKKKKKKKK HW 1003 PROM 30300 FLASH 30461 CHASSIS TYPE A8 CHASSIS NUMBER 0 Fabric Id 1 WWN 100000c0dd009e0e AC 00c0dd009e0d KKKKKKKKKKKKKKK Chassis Status KKKKKKKKKKKKKKK Number of Ports 8 Power OK Temp OK Temp 31 0c Fan 1 OK Fan 2 OK GBIC 1 Optical shortwave GBIC 2 Optical shortwave GBIC 3 Optical shortwave GBIC 4 Optical shortwave GBIC 5 Optical shortwave GBIC 6 Optical shortwave GBIC 7 None installed GBIC 8 Optical shortwave KKKKKKKKKKKKKKKK Time Out Values KKKKKKKKKKKKKK KK Active Timeout Values edtov 2560 mfstov 640 ratov 5000 rttov 100 continued on next page Chapter 3 Diagnostic Tools 49 continued from previous page KKKKKKKKKKKK Port Status KKKKKKKKKKKK Port Port Type Admin State Oper State 1 F_Port online online 2 T lt Port online online 3 F_Port online offline 4 SL_Port online online Devices 1 Address 0x00 0x01 5 SL_Port online online Devices 24 Address 0x00 Oxb5 Oxba lt snip 18 devices gt Oxe8 Ox
98. e is required to support the configurations in this document Additional information and resources are available at http www sun com storage san or http sunsolve sun com These websites contain information on software versions and provide necessary patches Features This release of the Sun StorEdge SAN release 3 1 has many new features This section lists the major changes and gives a brief description of each Fabric Ports All ports on the switch are capable of being configured as fabric ports This allows public devices to connect to the switch Currently Sun supports only the host bus adapters running the correct version of FCODE and Leadville driver stack to be configured as F Ports No storage can be configured on a fabric port Also the switch will handle host bus adapters being connected via FL Ports Public Loop which is currently not supported The only supported configuration is F Ports point to point for public host bus adapters It should be noted that the switch default setting is FL Port and therefore must be changed at the time of installation to the supported F Port for HBAs Fabric ports register with the name server on the switch You can still configure the host bus adapters as private loop devices By placing the switch port into SL Port mode the host bus adapter will configure itself as a private loop device able to be zoned with other private loop only devices for example Sun StorEdge A5200 arrays
99. e package using the following command pkgadd d 3 Install the Storage Automated Diagnostic Environment Brocade patch patchadd 112580 01 148 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only 4 Run ras_instal1 to enable the cron and to configure the agent as master or slave Upgrade the Master first Check your SAN Management host to find the version of the Storage Automated Diagnostic Environment installed pkginfo 1 SUNWstade Appendix A Brocade Upgrades and Installations 149 150 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only APPENDIX B Brocade Troubleshooting Introduction This appendix provides basic guidelines that you can use to isolate problems found in a Brocade Silkworm configuration It assumes you have been trained on all the components such as storage and switch that make up the configuration The scope of this appendix is to highlight the differences of troubleshooting with a Brocade Silkworm configuration to that of a configuration that contains the current Sun StorEdge Network Fibre Channel family of switches Current support is limited to diagnosing failures down to the FRU level In Sun s support model the entire Silkworm switch is considered a FRU Many of Brocade s internal diagnostics and messages while useful for depot or Root Cause Analysis situations are not ultimately relevant to
100. ecksum on any existing data and performs a data write read test on the unused areas of the NVRAM A test failure in any the these three tests causes the heartbeat LED to blink 15 times between three second pauses Hung Flash Control Code The switch is not operable If the Power Good LED is lit and the heartbeat LED and the remaining front panel LEDs blink in unison the flash control code running the processor is hung Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Complete Failure The switch is not operable If the Power Good LED is lit and the heartbeat LED does not blink at all always ON or always OFF the switch is not operable Cable Continuity Tests When there is a problem communicating over a particular link and both the switch and the connected device pass their respective tests check the continuity of the cables by running the Storage Automated Diagnostic Environment switchtest Depending on the configuration you may need to run the switchtest offline Chapter 4 Diagnosing and Troubleshooting the Switch 71 72 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CHAPTER 5 Switch Counter Information Switch Counter data should be used only as supporting data Do not use this data as the primary source in the troubleshooting process General points to keep in mind when viewing counters follow m Quickly increasing counter
101. ed gt Switch 100000c0dd00562a lt svsName undefined gt Fabric ID H Switch Statisti Out of buffers fo Outof s Buffers 100 switch resets 48 intr low Bufs ASIC intr low Bufs Asicfo COF Parity ASIC 0 COF Parity asic ifo_ COF CRC ASIC O_ 2560 COF CRC ASIC 1 103220 Frame bus Errs Ag Frame bus errs AJO Framebuserrs ago Chassis ID jo Stage Type SLzoning Admin Mode online 5000 Frame bus errs ago Frame buserrs ago Frames In Frames Out Frames Dropped Port 5 SL Po DZS ozs o s ozs Port 7 SL Port Port 8 SL Port Faceplate Display click ports to select or double click to enter FIGURE 5 2 SanSurfer Web GUI TABLE 5 2 on the following page lists the counter names and briefly describes them Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE 5 2 Counter Names and Descriptions Faceplate Window Counter COF CRC ASIC 0 COF CRC ASIC 1 COF CRC ASIC 2 COF CRC ASIC 3 COF Parity ASIC 0 COF Parity ASIC 1 COF Parity ASIC 2 COF Parity ASIC 3 Frame bus Errs ASIC 0 Port 1 Frame bus Errs ASIC 0 Port 2 Frame bus Errs ASIC 0 Port 3 Frame bus Errs ASIC 0 Port 4 Frame bus Errs ASIC 1 Port 1 Frame bus Errs ASIC 1 Port 2 Frame bus Errs ASIC 1 Port 3 Frame bus Errs ASIC 1 Port 4 Frame bus Errs ASIC 2 Port 1 Frame bus Errs ASIC 2 Port 2 Frame bus Errs ASIC 2 Port 3 Frame bus Errs ASIC 2 Port
102. ef 6 F_Port online online 7 F_Port online offline 8 F_Port online offline KKKKKKKKK Topology KKKKKKKKK Port Remote Chassis StageType PortAddr 01 00 OT 100000 02 01 IOT 100100 03 00 OT 100200 04 00 IOT 100300 05 00 OF 100400 06 00 OT 100500 07 00 OT 100600 08 00 OT 100700 KKKKKKKKKKKKKKKKKK Links Information KKKKKKKKKKKKKKKKKK Chassis 00 Remote Chassis Port FCAddr Port 02 01 02 104100 Chassis 01 Remote Chassis Port FCAddr Port 02 00 02 100100 continued on next page Status Loop Mode logged in logged in Not logged in logged in logged in Target Target logged in Not logged in Not logged in LinkAddr 000000 104100 000000 000000 000000 000000 000000 000000 WWN 100000c0dd009e48 WWN 100000c0dd009e0e 50 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only continued from previous page KKKKKKKKKKK port count KKKKKKKKKKK Port Number a Inframes 17171439 Outframes 30858503 LinkFails 5 SyncLosses 4 nvalidTxWds 534 OLS Out 42898 Port Number 2 nframes 31052936 Outframes 17368056 nvalidTxWds ch OLS Out 3 Total LIP Revd 102 LIP F7 F7 102 Port Number 3 Inframes 0 Outframes 0 InvalidTxWds 4 lt output deleted gt KKKKKKKKKKKK Name Server KKKKKKKKKKKK Port Address Type PortWWN Node WWN FC 4 Types 00 100000 N 210000e08b032a01 200000e 08b032a01 05 100500 N 210000e 08b030926 200000e 08b030926 SCSI_FCP
103. emarks or registered trademarks of SPARC International Inc in the U S and other countries Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems Inc The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems Inc for its users and licensees Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry Sun holds a non exclusive license from Xerox to the Xerox Graphical User Interface which license also covers Sun s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun s written license agreements RESTRICTED RIGHTS Use duplication or disclosure by the U S Government is subject to restrictions of FAR 52 227 14 g 2 6 87 and FAR 52 227 19 6 87 or DFAR 252 227 7015 b 6 95 and DFAR 227 7202 3 a DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Copyright 2002 Sun Microsystems Inc 4150 Network Circle Santa Clara CA 95054 Etats Unis Tous droits r serv s Ce produit ou document est prot g par un copyright et distribu avec des licences qui en restreignent l utilisation la copie la distribution et la
104. ements If there are other Fabric elements connected it identifies the master switch Address assignment once the master switch has been identified port addresses may be assigned Each switch tries to keep the same addresses that were previously used These are stored in the switch s configuration flash PROM Routing table construction after addresses are assigned the unicast routing tables are constructed 10 Enable normal port operation Note If any of the steps listed above fails replace the entire switch as a single FRU Appendix B Brocade Troubleshooting 169 General Troubleshooting Procedures This section lists the broad steps on how to approach a Brocade Silkworm switch problem in a SAN environment It lays out a methodical approach and lists various tools and resources available at each step It is expected that using Storage Automated Diagnostic Environment for monitoring will greatly reduce the time consuming process of narrowing down the problem Step 1 Discover the Error m Storage Automated Diagnostic Environment messages and alerts m var adm messages m Application specific errors Step 2 Determine the extent of problem m cfgadm al output m luxadm e port output m Storage Automated Diagnostic Environment Topology error display a Multipathing information Sun StorEdge Traffic Manager and VxDMP Note The information gathered here will determine which subsection to focus your attention Host
105. ent to find out CODE EXAMPLE 6 1 opt SUNWstade diags bin discman Abbreviated Output HBA Port 3 Path devices pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 devctl RegisterName qlc3 LGroup StorEdge QLC HostBusAdaptors PGroup StorEdge IO Board Slot 4 Slot on IO Board 0 Port on Card I NodeWWN 200000 08b031026 PortWWN 210000e08b031026 ode offline DriverName fp Instance 3 al_pa 1 DeviceType 3 Yes Qlogic q1c3 is the same as the physical path Chapter 6 Examples of Fault Isolation 127 d Type the luxadm e port command Note You should also run luxadm e port command to see the state of the HBAs luxadm e port devices pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 devetl NOT CONNECTED In this case a NOT CONNECTED message appears In this configuration the single Sun StorEdge A5200 array was the only device on the loop You cannot draw any conclusions from this output at this time 3 Check the Array Status Run luxadm display DPL4 to look for status luxadm display DPL4 Output Abbreviated ESI Interface board IB A top B bottom A O K rev 04 GBIC module 1 on left 0 on right in IB 0 O K mod 05 1 Not Installed B O K rev 04 GBIC module 1 on left 0 on right in IB 0 Not Installed 1 Not Installed The above output confirms that IB Board B and GBIC 0 are no longer displayed 128 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide Apri
106. equired software packages and patches installation sequence TABLE A 2 Software Installation Sequence Patch or Package Solaris 8 Software Solaris 8 10 01 update 6 8_ Recommended SUNWsan SUNWcfpl SUNWcfp1lx 111412 07 111095 07 111096 03 ret 11097 07 m 11413 07 an 11846 03 pany 11847 01 SUNWstade 12580 01 pi Solaris 8 Recommended and Security patch cluster Sun StorEdge SAN Foundation Kit cfgadm plug in 32 bit package cfgadm plug in 64 bit package Sun StorEdge Traffic Manager fcp fp fctl usoc drivers patch fcip driver patch qlc driver patch luxadm liba5k and libg_fc patch cfgadm plug in library patch SANWsan patch SAN Foundation Kit patch Storage Automated Diagnostic Environment package Brocade patch Appendix A Brocade Upgrades and Installations 137 v To Install the Software Note These instructions are to install the Leadville 5c patch 1 Install Solaris 8 10 01 upgrade 6 2 Install the latest Solaris 8 Recommended Security patch cluster See the README file for patch installation instructions and notes 3 Install the SUNWsan package pkgadd d SUNWsan 4 Install cfgadm plug in library packages SUNWcfp1 and SUNWcfp1x pkgadd d pkg location SUNWcfpl pkgadd d pkg location SUNWcfplx 5 Install the SAN Foundation Kit SUNWsan patch 111847 01 or higher if required 6 Install Sun StorEdge Traffic Manager patch 111412 0
107. ern info scsi_vhci ssd g200000203745060f ssd54 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 220000203745060f 0 is offline disk_front 8 wwn in a5k DPL4 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g200000203719f803 ssd53 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 220000203719f803 0 is offline disk_front 2 wwn in a5k DPL4 Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g20000020374505ca ssd52 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 22000020374505ca 0 is offline disk_rear 5 wwn in a5k DPL4 122 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Warning Mar 8 13 14 23 Sun StorEdge Traffic Manager StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g200000203719f802 ssd51 multipath status degraded path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target s offline disk_front 7 wwn in a5k DPL4 address 220000203719f802 0 i Warning Mar 8 13 14 23 Sun S StorEdge Traffic Manager ID multipath status optimal pa address 5080020000083cb4 0 i Warning Mar 8 13 14 23 Sun S StorEdge Traffic Manager
108. error messages before removing power Status and Activity Indicators Front Panel LED Port Indicators Front Panel LEDs Definition No light showing No light or signal carrier no module no cable for media interface LEDs Steady yellow Receiving light or signal but not yet online Slow yellow Disabled result of diagnostics or portDisable command Flashes every two seconds Fast yellow Error fault with port Flashes every 1 2 second Steady green Online connected with device Slow green Online but segmented loopback cable or incompatible switch flashes every two seconds Fast green Internal loopback diagnostics Flashes every 1 2 second Flickering green Online and frames flowing through port Islow 2 seconds interval 2Fast 1 2 second interval See the Brocade Silkworm Hardware Reference Manual for further details 168 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Initialization Steps At power on or reset the following steps occur 1 23 Preliminary POST diagnostics VxWorks operating system initialization Hardware initialization resets internal addresses assigned to ASICs serial port initialized front panel initialized Full POST Universal Port Configuration Link initialization receiver transmitter negotiation to bring connected ports online Fabric analysis the switch checks for ports connected to other Fabric el
109. es may be intermixed Interoperability with other vendor switches is not supported at this time 154 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE B 4 Applications Utilities Supportability Matrix Name Version Patches VERITAS Volume Manager 3 2 GA 111413 05 111904 04 111909 04 SUNWstade 2 0 112580 01 QuickLoop QuickLoop is a feature of the Brocade Silkworm switches that allows hosts with host bus adapters HBAs that are not fully Fabric aware to communicate with other devices attached to the switch In addition QuickLoop allows switches to replace hubs in a private loop environment QuickLoop is a separately licensed product Note For the Brocade Sun StorEdge SAN 3 1 phase Sun StorEdge A3500FC arrays still need to be configured as a Quickloop port Sun StorEdge T3 arrays do not need Quickloop nor do Host Bus Adapters Sun StorEdge T3 arrays will auto configure as L Ports and HBAs will auto configure as F Ports Features a A maximum of 126 devices are allowed within a single QuickLoop m Ports looplets of up to two switches can be included in a QuickLoop m Each looplet supports transfer rates of up to 100 MB per second and multiple concurrent transfers can occur in multiple looplets a Hosts that are attached to QuickLoop can communicate to all devices in the same QuickLoop m Other public hosts can communicate to all devices in QuickLoop m Individual QuickLoop po
110. eters ISL hops do not include the connections between hosts and switches or between switches and storage For further information please refer to the Sun StorEdge Network FC Switch 8 and Switch 16 Configuration Supplement Disaster Tolerant Configurations 816 3652 10 cfgadm_fp In order to have fabric devices seen by the operating system they must be configured through the cfgadm framework A fabric plug in to cfgadm is used to attach fabric public devices Private loop devices are still configured by the Leadville driver and require no user intervention Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Examples cfgadm al Ap_Id Type Receptacle Occupant Condition c0 scsi bus connected configured unknown c0 dsk c0t0d0 disk connected configured unknown c0 dsk c0t1d0 disk connected configured unknown cl scsi bus connected configured unknown cl dsk clt6d0 CD ROM connected configured unknown c2 fc fabric connected unconfigured unknown c2 210000eC08b030926 unknown connected unconfigured unknown c2 50020 23000003d5 disk connected unconfigured unknown cs fc fabric connected unconfigured unknown c3 210000e08b032901 unknown connected unconfigured unknown c3 50020f23000003c5 disk connected unconfigured unknown A Sun StorEdge T3 array partner pair is connected and showing up on c2 and c3 Currently the Sun StorEdge T3 arrays are unconfigured and not available to Solaris
111. example 000100001000010011100100 is the 24 bit binary representation of 1084e4 Qlogic encodes this number the following way Bits 0 3 Fabric ID Bits 4 9 Chassis ID Bits 10 13 ASIC Bits 14 15 Port Bits 16 23 AL_PA a The AL_PA will be zero if the device is a full fabric device otherwise it will be the AL_PA of the loop device m Qlogic switches have 2 or 4 ASICS 2 on the 8port switch 4 on the 16port switch These ASICs are numbered from 0 3 m Each ASIC has 4 ports numbered 0 3 a The numbering on the switch faceplate goes from 1 8 or 1 16 a So ASIC 0 port 0 is actually Port 1 if you were looking at the switch ASIC 3 port 3 would be Port 16 if you were looking at the switch 000100001000010011100100 is broken down as follows 0001 Fabric ID 000010 IChassis ID 0001 ASIC ID 00 Port ID 11100100 AL_PA 86 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only You can convert these back into usable numbers Binary gt Decimal except for AL_PA change it to Hex Fabric ID 1 Chassis ID 2 ASIC ID 1 Port ID 0 AL_PA E4 From this information you can conclude the following The Fabric ID of the switch is 1 The Chassis ID of the switch is 2 set in the SANsurfer GUI The port in question is port 0 of ASIC 1 Port 0 of ASIC 1 is Port 5 if you were to look at the switch faceplate Refer to TABLE 6 1 to see the ASIC Port breakdown The AL_PA of the devi
112. fic Manager supported devices or has Fabric devices connected TABLE 2 6 Supported Features of the Sun StorEdge A3500FC Array Feature Supported Cascading No Zone Type SL zone only Maximum number of arrays per SL zone 4 Maximum initiators per SL zone 2 TABLE 2 7 Supported Features of Network Attached Storage and SNDR Supported Cascading Fabric name server zone Zone Type Fabric name server zone HBA as F Port pt to pt Maximum device ports per zone 812 1 With the PCI dual Fibre Channel Network Adapter HBA only physical port 2 can be used for FCIP 2 With the CPCI Dual Fibre Channel Network Adapter HBA only physical port 1 can be used for FCIP 18 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Switches For high availability configure any two sets of switches in parallel m Sun StorEdge Network FC Switch 8 m Sun StorEdge Network FC Switch 16 Fibre Channel FC Tape Libraries Both the L180 and L700 FC Tape Libraries require StorageTek 9840 FC tape drives You must upgrade to the most recent software for each L180 or L700 FC Tape unit You can configure either the Sun StorEdge L180 or L700 Tape Library with a switch m The Sun StorEdge L180 Tape Library requires one 16 port switch configured as two 8 port zones m The Sun StorEdge L700 requires a minimum of one 8 port switch and one 16 port switch a The switch may be zoned to allow up to four hosts per zone
113. fp plugin library package SUNWcfplx cfgadm_fp plugin library package 111846 03 cfgadm_fp plugin patch 111267 02 specfs patch 110614 02 ses driver patch SUNWstade Storage Automated Diagnostic Environment package 112580 01 Brocade patch SUNWsmer SANsurfer Switch Manager 2 08 22 package Sun Switches only 111096 03 SANsurfer patch Chapter 2 Configurations 15 Unbundled Software For a list of unbundled software refer to TABLE 2 3 TABLE 2 3 Unbundled Software 16 Package Minimum Revision Minimum Patch if any RAID Manager RM6 6 22 108553 09 Solstice Disk Suite 4 2 1 108693 04 Component Manager 2 2 T3 Firmware 1 17b See Early Notifier 14838 Switch Firmware 3 04 60 111096 03 SANsurfer GUI 2 08 30 111096 03 JAVA SDK JDK 1 3 02 PCI Single FC Host 1 12 See Early Notifier 14838 Adapter Fcode x6799a PCI Dual FC Host 1 12 See Early Notifier 14838 Adapter Fcode x6727a StorageTek 9840 1 28 126 T3 Firmware 1 17b VERITAS Volume Manager 3 1 1 SL Zones Only 111118 02 3 2 Fabric VERITAS File System 3 4 Instant Image 3 0 SNDR 3 0 Alternate Pathing 2 3 1 110722 01 110432 04 Storage Automated 2 0 T112580 01 Brocade support Diagnostic Environment VERITAS Netbackup 3 4 110539 04 Solstice Backup 6 0 Sun Enterprise 3 2 28 103346 29 3x00 4x00 5x00 6x00 Flash Prom Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE 2 3 Unbundled Software Sun Fire 5 11 6 11
114. functionality of each switch ASIC including the control port interface and all functions performable with the confines of an individual ASIC A failure indicates a faulty switch ASIC The heartbeat LED blinks six times between three second pauses The switch disables the ports associated with the bad ASIC and blinks the ports Logged in LEDs An ASIC that fails this test could affect the operation of the remaining ports Chapter 4 Diagnosing and Troubleshooting the Switch 67 68 GBIC Bypass Port Loopback Test Failure Seven Blinks The switch is operable The GBIC bypass port loopback test verifies on a port by port basis the ability of each switch ASIC to loop data out through the Serdes chip on a port and back to the ASIC control port bypassing the GBIC A failure indicates either a faulty switch ASIC or an ASIC to Serdes interface problem the heartbeat LED blinks seven times between three second pauses The switch disables the failing port or ports and blinks their Logged in LEDs The ports whose Logged in LEDs are not blinking have passed the test and are all usable Fibre Channel Port Loopback Test Failure Eight Blinks Note This test runs in Continuous Test only Continuous Test is controlled by the test mode switch Use this test only under the direction of customer support which will tell you how to activate the test The switch is not operable while in continuous test In continuous test mode the switch fibre ch
115. ged in LEDs are lit because the problem does not affect a port or ports If the POST encounters a bus error the switch may operate in a degraded mode because it has multiple buses It can operate with one or more buses in operation but some normal processing functions such as in order delivery may be adversely affected The heartbeat blinks the error code for the non fatal bus error The switch may operate more slowly but no port Logged in LEDs are lit because the problem does not affect the ports If the POST encounters a port error the switch may operate with the remaining ports The heartbeat blinks an error code for the non fatal port error The switch disables the failing port or ports and blinks their Logged in LEDs If the heartbeat LED is blinking normally and you cannot access the switch by way of the SANSurfer GUI check the IP address and verify that it is set correctly Refer to the Sun Switch Management Installer s and User s Manual for instructions on how to check and set the IP address Also check the Ethernet cable The POST diagnostic program performs the following basic tests Checksum tests on the Boot firmware located in a PROM and the main switch firmware located in FLASH memory Functional hardware tests on internal switch memory Various read write register and loopback data path tests on the switch logic board Frame bus and auto route logic tests Switch management port logic tests Arbitrated loop test
116. h 172 20 67 167 Test Mgr popup Set Defaults Test Archives Host diag229 central sun com 172 20 67 164 i Product Brocade 8 WWW 1000006069201 efc Brocade 8 brocade 172 20 67 167 1000006069201efd port 2 03 13 13 10 20 port 2 in BROCADE 172 20 67 167 ip 172 20 67 167 is XY Layout Save Layout Change Vertical Ss FIGURE B 4 Storage Automated Diagnostic Environment Brocade Switch Test Appendix B Brocade Troubleshooting 179 Using the Diagnostic Tests on port 2 the test fails and displays a message recommending you insert a loopback plug into port 2 No communication can be established between the switch and the HBA as shown in FIGURE B 5 Netscape Device 1 2 172 20 67 167 Ended 2002 03 19 13 34 03 RC 3 Called with options dev 2 172 20 67 167 1 iterations 1000 passwd xxxxxxx Connect to 172 20 67 167 Opened 172 20 67 167 Logged into 172 20 67 167 i Loopback not supported on this port type port 2 sw No_Light i robable_Cause s Port not connected gt Unknown port type gt Bad GBIC gt Bad switch gt Recommended_Action s nsert a loopback plug into port and retest gt FIGURE B 5 Test Detail showing Failed Test After inserting a loopback plug and rerunning the test the test now passes as shown in FIGURE B 6 180 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Netscape tral sun com Ou
117. h this address translation is automatic In Sun StorEdge switches the private device must be configured on a TL Port IF_Port IF_Port IA fabric port that is point to point only not loop kapable and used to connect N_Ports to the switch IFL_Port IFL_Port IA fabric port that is loop capable and used to connect NL_Ports to the switch IN A G_Port Brocade has a G_Port which is a generic port This port can operate as either an E_Port or an IF_Port A port is defined as a G_Port when it is Inot yet fully connected or has not yet assumed a specific function in the fabric N A U_Port Brocade has a U_Port or Universal Port This port kan operate as an E_Port F_Port or FL_Port A port is defined as a U_Port when it is not yet fully connected or has not yet assumed a specific function in the fabric 164 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Accessing the Brocade Silkworm switch You can access Brocade Silkworm switches in multiple ways m Open a telnet session by way of a standard RJ 45 Ethernet port m Access the front panel Brocade Silkworm 2800 only a Use a serial connection Brocade Silkworm 2400 only m Use the Brocade WebTools GUI The serial connection available on the Brocade Silkworm 2400 switch is intended for initial IP address configuration only Once the IP address is configured the switch is to be accessed via telnet or the WebTools GUI See th
118. h 59 Power Checks and Troubleshooting 59 Power On Self Test POST 60 Using the Test Mode Switch 62 Contents v Front Panel Switch Modes 63 v To Troubleshoot Test Mode Switch Functions 64 Heartbeat LED Blink Patterns 65 Normal all pass 65 Failure Blink Patterns 65 PROM Checksum Failure One Blink 66 RAM Failure Two Blinks 66 Flash Checksum Failure Switch Management Port Ethernet Tests Good Three Blinks 66 Flash Checksum Failure Switch Management port Ethernet Failure Four Blinks 67 Force PROM Mode in Effect Five Blinks 67 Switch ASIC Test Failure Six Blinks 67 GBIC Bypass Port Loopback Test Failure Seven Blinks 68 Fibre Channel Port Loopback Test Failure Eight Blinks 68 Switch Bus Test Failure Nine Blinks 69 Switch Auto Route Test Failure 10 Blinks 69 Eleven and Twelve Blinks Not Used 70 Arbitrated Loop Test Failure 13 Blinks 70 Switch Management Port Failure 14 Blinks 70 NVRAM Test Failure 15 Blinks 70 Hung Flash Control Code 70 Complete Failure 71 Cable Continuity Tests 71 5 Switch Counter Information 73 Counter Descriptions 75 6 Examples of Fault Isolation 81 vi Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 Overview 81 Setup Requirements 82 Troubleshooting Outline 83 Converting a Fabric Address into Fabric ID Chassis ID ASIC Port and AL_PA 85 Example 86 Example 1 Switch to Storage Error 89 Example 2 Switch to Switch Error 97 Example
119. h using switchshow switc switc switc switc switc switc switc switc switc port port port port port port port port hN hT hI YA oO PWD ame YPES hState hMode hRole hDomain d hWwn hBeacon Zoning 0 upstream 1s SW Sw diagl67 admin gt switchshow diag167 3 4 Online Native Subordinate 1 fffc0l 10 00 00 60 69 20 OFF ON Main Online E Port No_Module No_Light No_Module No_Module No_Module Online E Port Online F Port sle fc 1020000260269 10271225 10 80 03 0 03 60269 1027 1 25 21 00 00 e0 8b 03 61 f9 access primary failover failover primary diagl64 diagl64 This switchshow output from the first switch confirms that port 2 has gone offline No other ports seem to be affected at this point 178 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Step 5 Start testing FRUs In the Switch to HBA link there are potentially four FRUs a HBA m Cable m Switch GBIC m Switch chassis Using the Storage Automated Diagnostic Environment begin by testing the switch GBIC and the switch chassis Storage Automated Diagnostic Environment RY Ky Stora ance Monitor DINOS Report Utili microsystems 2 0 06 010 diag229 central sun com MESES Tools __ Diagnostics Host san MASTER Filter None co Start Test ase from TERNE Test from List Test Manager j brocadetest switc
120. he output of this switchtest is shown below device qlc0 sw0 f3 e2 Status for device qlc0 sw0 f3 e2 Power OK Temp OK 31 0c Fan 1 OK Fan 2 OK Device qlc0 sw0 f3 e2 Port 2 Pattern 0x7e7e7e7e Device qlc0 sw0 f3 e2 Port 2 Pattern 0x78787878 Device qlc0 sw0 f3 e2 Port Device qlc0 sw0 f3 e2 Port Pattern 0xe7e7e7e7 Pattern Oxb5b5b5b5 Pattern Oxaa55aa55 Pattern 0x7f7f7f7f Pattern Ox0f0f0f0E Pattern Ox4a4a4a4a Device qlc0O sw0 f3 e2 Port Pattern Ox00ff00ff Device qlc0 sw0 f3 e2 Port 2 Pattern Oxlelelele Device qlc0 sw0 f3 e2 Port Device qlc0 sw0 f3 e2 Port Device qlc0 sw0 f3 e2 Port Device qlc0 sw0 f3 e2 Port MM NMN DNDN NY With the loopback connector in the test passed The switch and GBIC can be ruled out as causes for the error for now d Run tests on the remote switch Storage Automated Diagnostic Environment tests communicate only with remote switches via the Inband path which has been disconnected and is still considered suspect To get around this you must use the SANsurfer GUI to connect directly to the remote switch via the remote switch s IP address and use Port Test to test the port and GBIC Remember to insert a loopback connector in the remote switch s port This will require the assistance of someone at the remote site 104 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only The results of this test are shown
121. ibre Channel Network Adapter Sun StorEdge PCI Single Fibre Channel Network Adapter Sun StorEdge CPCI Dual Fibre Channel Network Adapter Gigabit Interface Converter GBIC Long Wave LW GBIC 15m fiber optic cable Sun StorEdge FC switch 8 Switch Sun StorEdge network FC switch 16 Switch Sun StorEdge L180 FC Tape Library Sun StorEdge L700 Tape Library Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Supported Configurations To support a high availability environment use these configurations to ensure switch redundancy See the example diagrams in this chapter for more information on the supported configurations Hosts a Sun Enterprise E220R E250 E420R and E450 server hosts m Sun Enterprise E10000 server host m Sun Enterprise E3000 E6000 server hosts m Sun Enterprise E3500 E6500 server hosts m Sun Enterprise F3800 server host only with the Sun StorEdge CPCI Dual Fibre Channel Adapter m Sun Enterprise F4810 server host m Sun Enterprise F6800 server host Host Operating Environment Rules a All hosts ina zone must be running Solaris 8 Release 4 01 operating environment with all appropriate patches installed You can download the patches from Sunsolve a Mixing PCI Dual Fibre Channel Network Adapter and PCI single Fibre Channel Network Adapter HBAs in the same switch zone is supported a Mixing an Sbus host with a PCI card and PCI hosts within the same zone is s
122. ilure Replace mainboard Err 0111 less than or equal to zero assembly ramTest IDIAG MEMORY Data read from RAM location CPU RAM failure Replace mainboard Err 0110 did not match previously assembly or DRAM ramTest written data into same module location IDIAG PORTABSENT Port is not present ASIC or mainboard Replace mainboard Err 2670 3070 3870 failure assembly portLoopbackTest crossPortTest spinSilk 196 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE C 3 Diagnostic Error Messages Continued Message Description Probable Cause Action IDIAG PORTDIED Port was in loopback mode ASIC GBIC module Replace mainboard IBrr 265F 305F 385F and then went inactive or fiber cable failure assembly GBIC portLoopbackTest module or fiber cable crossPortTest spinSilk IDIAG PORTSTOPPED Err 3874 spinSilk Port is no longer transmitting as indicated by the Number Of Frames Transmitted counter being stuck at N frames ASIC GBIC module or fiber cable failure Replace mainboard assembly GBIC module or fiber cable DIAG PORTWRONG Err 3078 Frame erroneously received by port M instead of the ASIC failure Replace mainboard assembly switch initialization DIAG REGERR executed Data read from ASIC register ASIC failure crossPortTest intended port N IDIAG POST SKIPPED POST is skipped Its message Informational only None required Er
123. ision of the Fabric one or more Switch chassis into as many as 256 Fabric wide zones that define which ports or devices receive Name Server information If Hard Zones are enabled Name Server Zones will not communicate across defined Hard Zone boundaries An Arbitrated Loop without a Fabric switch An Arbitrated Loop attached to a Fabric switch A set of ports that behave as one private loop A set of ports and their connected devices zone that behave as a single private loop SL Zones on the switch allow the division of the Fabric one or more Switch chassis into Fabric wide zones that define the ports that can communicate with each other Zoning allows the user to divide the Fabric ports into zones for more efficient and secure communication among functionally grouped nodes There are several types of zones and a port may be defined in any No port can be in all zone types simultaneously Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only INDEX SYMBOLS kernel drv qlc conf 8 kernel drv scsi_vhci conf 7 A AC input power connector and fuses 56 arrays configuration guidelines 23 supported configurations 14 B blink pattern arbitrated loop test failure 70 fibre channel port loopback test failure 68 flash checksum failure 66 force PROM mode 67 GBIC bypass port loopback test failure 68 NVRAM test failure 70 switch ASIC test failure 67 switch auto route test failure
124. isk labeling mirroring striping and RAID 5 m Brocade Webtools Brocade switches support Java enabled Webtools Brocade Webtools is a GUI that provides management capabilities such as maintaining zones setting port attributes and setting up cascaded switches a cfgadm plug in for Fabric on demand node creation This component is delivered with the Solaris upgrades for the FC switch product It is used to discover SAN attached devices in addition to creating and deleting nodes for these devices These features are implemented through the cfgadm framework See the cfgadm_fp 1M man page for additional information m luxadm and libg_fc liba5k These components are enhanced to support the SAN attached devices They provide persistent switch attached device configuration across reboots probe SAN attached devices that are available on a host and provide information on various devices m Storage Automated Diagnostic Environment The Storage Automated Diagnostic Environment is a separately installed software product It is a lightweight remote monitoring agent designed to track storage product reliability availability and serviceability The Storage Automated Diagnostic Environment also provides revision and patch level checking log file monitoring and diagnostic testing Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Downloading Patches and Packages You can download the required
125. itches in which ISL can be shared across name server zones that exist in the same hard zone m Server and storage may be in the same name server zone across ISLs on separate switches This enables you to have servers at a recovery site It also means you can have local and remote storage in the same zone so that storage can be mirrored at both locations Chapter 2 Configurations 21 Cascading Rules a Hub to switch connectivity is not supported a Maximum of two switches cascaded in series a Maximum cascading distance is ten kilometers A maximum of two switches can be cascaded Both 8 port to 16 port and 16 port to 8 port switch connectivity is supported Rules for Adding and Removing Devices While the Hosts are Online You can add all initial and additional storage devices while the host is online except the Sun StorEdge A3500 FC array which requires rebooting the host on the first LUN addition In high availability configurations where alternative methods to reconstruct the data exist you can remove a device or path Host volume management or multi pathing software handles this device removal For non available configurations you must ensure that no host application is configured to use the device In the case of a fabric configuration name server zone you must unconfigure the device on the host This ensures that during the boot process the host does not attempt to probe this device to create device nodes You can add or
126. k c6t60020F2000003EE53AAF7A09000DA257d0s2 Status Port A Oa Ke Status Pott B O K Vendor SUN Product ID T300 WWN Node 50020 2000003ee5 WWN Port A 50020 2300003d2c WWN Port B 50020 2300003ee5 Revision 0118 Serial Num Unsupported Unformatted capacity 488642 000 MBytes Write Cache Enabled Read Cache Enabled Minimum prefetch 0x0 Maximum prefetch 0x0 Device Type Disk device Path s dev rdsk c6t60020F2000003EE53AAF7A09000DA257d0s2 devices scsi_vhci ssd g60020f2000003ee53aaf7a09000da257 c raw Controller devices pci 9 600000 pci 2 SUNW qlc 4 fp 0 0 Device Address 50020 2300003d2c 1 Class primary State OFFLINE Controller devices pci 9 600000 pci 2 SUNW qlc 5 fp 0 0 Device Address 50020 2300003ee5 1 Class secondary State ONLINE 176 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only From this output notice that the primary path to this LUN is OFFLINE and the secondary path has taken over This matches the Sun StorEdge Traffic Manager event from the Storage Automated Diagnostic Environment You can also check the Storage Automated Diagnostic Environment topology for any errors orage Automated Diagnosti 2 0 06 010 diag229 central sun com Host san MASTER Filter None so E Topology Help After having selected a topology several functions are available in the graphical view 1 Clicking in the middle of an object or in
127. king graphic topology display and fault isolation diagnostics Note Patch 112580 01 or higher is required with Storage Automated Diagnostic Environment 2 0 for Brocade support 156 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only brocadetest 1M The Storage Automated Diagnostic Environment has incorporated the launching of various Brocade Silkworm switch diagnostic tests under one Storage Automated Diagnostic Environment test brocadetest 1M This test is launched within the Storage Automated Diagnostic Environment framework and determines the correct Brocade specific test to run depending on port type Example CLI brocadetest 1M opt SUNWstade Diags bin brocadetest v o dev 5 172 20 67 167 passwd password iterations 1000 Called with options dev 5 172 20 67 167 passwd xxxxxxx iterations 1000 Connect to 172 20 67 167 Opened 172 20 67 167 Logged into 172 20 67 167 Clear port errors send diagClearError 5 Port errors cleared port is in loopback mode Running command CrossPortTest 1000 1 Note You should only have a loopback on port 5 If you have more than one loopback installed this test may report false errors Test Passed Loopback took 34 seconds to run Fan 1 is OK speed is 8640 RPM Fan 2 is OK speed is 8760 RPM Fan 3 is OK speed is 8910 RPM Fan 4 is OK speed is 8820 RPM Fan 5 is OK speed is 8820 RPM Fan 6 is OK speed is 8820 RPM KKKK
128. l 2002 For Internal Use Only 4 Check the switch status You can check the switch status either by viewing or asking the customer to view the Sansurfer GUI or by checking the results of the Capture utility CODE EXAMPLE 6 2 Capture Utility capture 172 20 67 193 gt tmp capture out Output abbreviated KKKKKKKKKKKK Port Status KKKKKKKKKKKK Port Port Type Admin State Oper State 1 F_Port online online 2 T_Port online online 3 F_Port online offline 4 SL_Port online online Devices 1 Address 0x00 0x01 5 SL_Port online offline 6 F_Port online online 7 F_Port online offline 8 T_Port online online Status Loop Mode logged in logged in Not logged in logged in Target Not logged in logged in Not logged in logged in In the above output Port 5 Switch to Storage link is offline but Port 4 Host to Switch link is logged in Concentrate the testing to the switch to storage link from Port 5 Chapter 6 Examples of Fault Isolation 129 5 Start Testing FRUs Running switchtest from the GUI or the command line on Port 5 should fail Command line example opt SUNWstade diags bin switchtest v o dev qlc3 sw2 i4 d5 switchtest called with options dev qlc3 sw2 i4 d5 switchtest Started Start switchtest Testing device qlc3 sw2 i4 d5 Chassis Status for device qlc3 sw2 i4 d5 Power OK Temp OK 32 0c Fan 1 OK Fan 2 OK 08 08 01 14 19 49 diag226 Central Sun COM Storage Automated
129. l Prompts Shell C shell C shell superuser Bourne shell and Korn shell Bourne shell and Korn shell superuser Examples Edit your login file Use 1s a to list all files o You have mail o 3 su Password Read Chapter 6 in the User s Guide These are called class options You must be superuser to do this To delete a file type rm filename Prompt machine_name machine_name x Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Related Documentation Application Installer User s Information GUI and User Installation Configuration Information Installer User Troubleshooting Late news Software Late news Late news Late news Best Practices Late news Hub information Title Sun SANbox 8 Fibre Channel Switch Installer s User s Manual SANbox Segmented Loop Switch User s Manual Sun StorEdge SAN release 3 0 Installation and Configuration Guide SANbox 8 16 Segmented Loop Switch Management User s Manual Sun StorEdge SAN release 3 0 Troubleshooting Guide Sun StorEdge SAN release 3 0 Release Notes CDInsert Sun StorEdge A5200 Product Notes Sun StorEdge T3 Disk Tray Release Notes Sun StorEdge A3x00 A3500 FC Best Practices Guide Sun StorEdge A3500FC Release Notes Sun StorEdge FC 100 Hub Installation and Service Manual Part Number 875 1881 10 875 3059 10 816 0830 10 875 3060 10
130. l7 Power OK Temp OK 31 0c Fan 1 OK Fan 2 OK Testing Device qlcl sw2 f1 e2 sw3 e2 tl7 Port 7 Pattern 0x7e7e7e7e Testing Device qlcl sw2 f1 e2 sw3 e2 tl7 Port 7 Pattern 0x78787878 Testing Device qlcl sw2 f1 e2 sw3 e2 tl7 Port 7 Pattern 0xe7e7e7e7 Testing Device qlc1l sw2 f1 e2 sw3 e2 tl7 Port 7 Pattern Oxb5b5b5b5 Testing Device qlcl sw2 f1 e2 sw3 e2 tl7 Port 7 Pattern 0xaa55aa55 Testing Device qlcl sw2 fl e2 sw3 e2 tl7 Port 7 Pattern 0x7f7f7f7f Testing Device qlcl sw2 f1 e2 sw3 e2 tl7 Port 7 Pattern Ox0f0f0f0f Testing Device qlc1l sw2 f1 e2 sw3 e2 tl7 Port 7 Pattern 0x4a4a4a4a Testing Device qlcl sw2 f1 e2 sw3 e2 tl7 Port 7 Pattern Ox00ff00ff Testing Device qlcl sw2 fl e2 sw3 e2 tl7 Port 7 Pattern Oxlelelele Port 7 passed all tests on qlcl sw2 f1l e2 sw3 e2 t17 switchtest Stopped successfully 94 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only After replacing the GBIC the tests pass 6 Conclude that the original GBIC was bad recable the link and check status After recabling the link the following messages appear in var adm messages Mar 10 15 20 52 diag226 Central Sun COM fp ID 517869 kern warning WARNING fp 1 N_x Port with D_ID 1006e8 PWWN 50020f23000003c5 reappeared in fabric Mar 10 15 20 52 diag226 Central Sun COM scsi ID 243001 kern info pci 6 4000 SUNW glc 3 fp 0 0 fcpl Mar 10
131. lace mainboard period Err 223C translation test assembly camTest IDIAG CLEAR_ERR Port s diag error flag OK or Information only None required Err 0001 BAD is cleared IDIAG CMBISRF ASIC s Central Memory ASIC failure Replace mainboard Err 1021 SRAMs did not complete the assembly centralMemory Test BISR within the timeout IDIAG CMBISRTO Err 1020 centralMemoryTest ASIC s Central Memory SRAMs did not complete the BISR within the timeout period ASIC failure Replace mainboard assembly IDIAG CMERRPTN Err 102B centralMemoryTest IDIAG CMERRTYPE Err 102A centralMemoryTest IDIAG CMICKSUM 2036 cmiTest Err Error detected at the wrong port Port got the wrong CMEM error type CMI message received failed bad checksum test ASIC failure ASIC failure ASIC or mainboard failure Replace mainboard assembly Replace mainboard assembly Replace mainboard assembly 194 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE C 3 Diagnostic Error Messages Continued Message Description Probable Cause Action IDIAG CMIDATA CMI data received but did ASIC or mainboard Replace mainboard Err 2035 not match data transmitted failure assembly cmiTest IDIAG CMIINVCAP Unintended ASIC ASIC or mainboard Replace mainboard Err 2034 erroneously got CMI capture failure assembly cmiTest flag
132. lease 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CODE EXAMPLE B 1 supportShow Sample Output telnet gt Trying 172 20 67 167 Connected to 172 20 67 167 Escape character is Fabric OS tm Release v2 6 0 login admin Password diagl67 admin gt supportshow Kernel 5 4 Fabric OS v2 6 0 Made on Tue Jan 15 15 10 28 PST 2002 Flash Tue Jan 15 15 12 04 PST 2002 BootProm Thu Jun 17 15 20 39 PDT 1999 26 25 26 25 27 Centigrade 78 77 78 77 80 Fahrenheit Power Supply 1 is absent Appendix B Brocade Troubleshooting 159 CODE EXAMPLE B 2 switchshow Example Output diagl67 admin gt switchshow switchName diagl67 switchType 3 4 switchState Online switchMode Native switchRole Subordinate switchDomain 1 switchId fffc0l switchWwn 10 00 00 60 69 20 le fc switchBeacon OFF Zoning ON Main port 0 sw Online E Port 10 00 00 60 69 10 71 25 diagl64 upstream port 1 No_Module port 2 sw Online F Port 21 01 00 e0 8b 23 61 f 9 port 3 No_Module port 4 No_Module port 5 No_Module port 6 sw Online E Port 10 00 00 60 69 10 71 25 diagl64 port 7 sw Online F Port 21 00 00 e0 8b 03 61 f9 CODE EXAMPLE B 3 gl show Example Output diagl67 admin gt qlshow Self 10 00 00 60 69 20 le fc domain 1 State Master Scope single AL_PA bitmap 30000000 00000000 00000000 00000000 Local AL_PAs not available Local loop
133. let states Member 1 304 Online T Looplet 1 offline Looplet 3 offline Looplet 4 offline 160 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CODE EXAMPLE B 4 diagShow Example Output diagl67 admin gt diagshow nTicks 0 Max 4473924 Diagnostics Status Tue Mar 19 14 04 30 2002 port 0 1 2 3 4 5 6 p diags OK OK OK OK OK OK OK OK State UP DN UP DN DN ON UP UP pto 4086880 frTx 64382622 frRx 0 LLI_errs pt2 38616950 frTx 300398 frRx 12 LLI_errs pt6 28852033 frTx 235091315 frRx 111 LLI_errs pt7 331090679 frTx 8930476 frRx 31 LLI_errs Central Memory OK Total Diag Frames Tx 8318 Total Diag Frames Rx 8911 CODE EXAMPLE B 5 crossPortTest Example Output diagl67 admin gt crossporttest 1000 1 Running Cross Port Test The 1000 is the number of passes the 1 denotes singlePortAlso mode which allows the test to be run on a single port with a loopback connector plug inserted Appendix B Brocade Troubleshooting 161 CODE EXAMPLE B 6 loopPortTest Example Output diagl64 admin gt loopporttest 100 2 0x7e7e7e7e 4 Configuring L port 2 to Cable Loopback Port done Will use pattern 7e7e7e7 7e7e7e7 7e7e7e7 7e7e7e7 Running Loop Port Test passed Configuring Loopback L port s back to normal L port sjon ed es done Note Notes on loopPortTest a Syntax is loopporttest lt num_passes gt lt
134. line message s in 7 mins threshold is 1 in 5mins Last Message diag226 Central Sun COM qlc ID 686697 kern info NOTICE Qlogic qlc 3 Loop OFFLINE Threshold Error Mar 8 13 14 03 Received 1 Loop Online message s in 7 mins threshold is 1 in 5mins Last Message diag226 Central Sun COM qlc ID 686697 kern info NOTICE Qlogic qlc 3 Loop ONLINE Threshold Warning Mar 8 13 14 03 Received 5 SSD Warning message s on ssd52 in 24 mins threshold is 5 in 24hours Last Message diag226 Central Sun COM scsi ID 243001 kern warning WARNING scsi_vhci ssd g20000020374505ca ssd52 Source Category Switch EventType StateChangeEvent EventTime 08 08 2001 13 16 00 port 5 in SWITCH python_local ip 172 20 67 193 is now Not Available status state changed from Online to Offline Note The Message Event from above is extracted from the var adm messages file It could be that this problem would be initially discovered by watching the var adm messages file rather than Storage Automated Diagnostic Environment emails This will depend on customer configuration However the Sun StorEdge A5200 array Event and the Switch Event would not be noted in var adm messages 124 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only cfgadm al c2 c2 21000020371 c2 21000020371 c2 21000020371 c2 21000020371 c3 c3 22000020371 c3
135. logfile var adm messages on diag229 central sun com id 80c61254 Mar 19 13 09 40 WWN 50020f2300003d2c diag229 Central Sun COM mpxio ID 779286 kern info scsi_vhci ssd g60020f2000003ee53aaf7a09000da257 ssd76 multipath status degraded path pci 9 600000 pci 2 SUNW qlc 4 fpe 0 0 fp3 to target address 50020f2300003d2c 1 is offline in backup diag229 central sun com Mar 19 13 09 40 WWN 50020f2300003d2c diag229 Central Sun COM mpxio ID 779286 kern info scsi_vhci ssd g60020f2000003ee53c18b7010003a5a8 ssd77 multipath status degraded path pci 9 600000 pci 2 SUNW qlc 4 fp 0 0 fp3 to target address 50020f2300003d2c 0 is offline in backup diag229 central sun com Found 1 driver Fabric_Warning warning s in logfile var adm messages on diag229 central sun com id 80c61254 INFORMATION Fabric warning Mar 19 13 09 20 WWN 50020f2300003d2c diag229 Central Sun COM fctl ID 288011 kern warning WARNING fct1 3 N_x Port with D_ID 312e4 PWWN 50020f2300003d2c disappeared from fabric in backup diag229 central sun com continued on next page Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only continued from previous page port 2 in BROCADE 172 20 67 167 ip 172 20 67 167 is now Not Available status state changed from online to offline INFORMATION A port on the switch has logged out of the fabric and gone offline
136. mary dev rdsk c6t60020F20000003C53B699ABC00006C4Cd0s2 Chapter 6 Examples of Fault Isolation 105 b View the results of this command in var adm messages shown below Mar 10 20 50 48 diag226 Central Sun COM Failover operation completed successfully for device ssd GUID 60020f20000003c53b699abc00006c4c failed over from secondary to primary The I O is now traveling over the primary path for this LUN c Fix all paths that were failed over in Step 2 106 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Example 3 Host to Switch Failure In this example you will explore a fault that occurs between the Host and the switch where the Host is connected to the switch via a F Port Fabric point to point The current limitations with ECHO commands and Qlogic HBAs detailed in the Introduction to this section require special considerations when troubleshooting 1 Discover the Error This time the errors are in var adm messages ar 11 13 44 33 diag226 Central Sun COM scsi ID 243001 kern warning WARNING scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 ar 11 13 44 33 diag226 Central Sun COM SCSI transport failed reason tran_err retrying command ar 11 13 44 33 diag226 Central Sun COM qlc ID 686697 kern info NOTICE Qlogic qlc 0 Loop OFFLINE Mar 11 13 44 33 diag226 Central Sun COM qlc ID 686697 kern info NOTICE Qlogic qlc 0 Link ONLINE ar 11
137. n LED The Logged In LED indicates the logged in or initialization status of the connected device or loop of devices Initially immediately after the switch completes the POST successfully the switch holds all Logged In LEDS OFF no light Each remains OFF until the port and its attached devices are able to perform a loop initialization LIP successfully Following a successful LIP on a given port the switch turns the Logged In LED ON lit for that port This shows that the port is properly connected and able to communicate with its attached devices The LED for this port remains ON as long as the port is initialized If the established link is broken a fiber opens or the connected port goes out of service the Logged In LED is shut OFF If the link is replaced or the connected port comes back into service the port and its attached devices try to regain initialized status If the initialization is re established the switch turns the Logged In LED back ON and communication continues Traffic LED Yellow Each port has its own port traffic LED The traffic LED for a particular port is ON when Class 2 or 3 frames are entering or leaving the port The switch turns the LED ON for 50 milliseconds for each frame so you should be able to see it for one frame This LED does not light for frames following an arbitrated loop in bypass mode AC Input Power Connector and Fuses A standard 3 wire computer type AC power cable supplied with the switch
138. n errors that may not show up as hard errors anywhere else For example a user might accidentally change an HBA port to SL mode when it was previously F mode Step 5 Start Testing the FRUs Storage Automated Diagnostic Environment Functional Tests switchtest qlictest SANsurfer GUI port tests Sun StorEdge T3 array tests Sun StorEdge A3500FC health check The conclusion of these tests isolate the problem to a FRU to be replaced Follow the appropriate hardware manual for proper FRU replacement procedures Step 6 Verify the Fix var adm messages Storage Automated Diagnostic Environment agent status Storage Automated Diagnostic Environment functional tests Sun StorEdge Traffic Manager or VxDMP to return to normal path conditions 84 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Converting a Fabric Address into Fabric ID Chassis ID ASIC Port and AL_PA You will occasionally see messages like the following in var adm messages Mar 7 10 06 18 vikings genunix ID 936769 kern info ssd6 is pci 8 700000 pci 3 SUNW qlc 4 fp 0 0 ssd w50020 2300009697 1 Mar 7 10 06 18 vikings scsi ID 365881 kern info lt SUN T300 0117 cyl 34145 alt 2 hd 72 sec 128 gt Mar 7 10 06 18 vikings genunix ID 408114 kern info pci 8 700000 pci 3 SUNW qlc 4 fp 0 0 ssd w50020f2300009697 1 ssd6 online Mar 7 10 06 18 vikings scsi ID 799468 kern info ssd7 at fp0 name w50020f230
139. nd the severity level 1 The possible values are a 1 critical m 2 error m 3 warning m 4 informational a 5 debug Note If the same error occurs within the same module but on different ports the number of occurrences for that message is incremented in brackets up to 999 and the error number is not incremented that is this error though it may occur 999 times occupies one message in the 32 message buffer v To Display Error Messages from the Front Panel 1 From the front panel select the Status menu 2 Select Error Log 3 Scroll through the error log If no errors are encountered the panel displays No Error Diagnostic Error Message Formats If any port fails during a diagnostic test it is marked BAD in the status display To retest a port that has been marked BAD clear the port and set to OK using the diagClearError port command This command clears the port status only and does not clear the logs or change the port s condition The diagClearError port command should only be used during diagnostic procedures to reset a bad port for retest Some messages contain the following abbreviations a sb Should Be m er Bits in error 188 Sun StorEdge San release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Note If you run the port StateShow or the diagShow command prior to running a test errors may appear as a result of the normal synchronization process These
140. ner pair array T3WG or 6 Sun StorEdge T3ES 24 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Configuration Examples Single Host Connected to One Storage Array FIGURE 2 1 shows one host connected through fiber optic cables to one Sun StorEdge A3500FC controller module Each controller module has two Fibre Channel ports FIGURE 2 2 shows one host connected through fiber optic cables to one Sun StorEdge A5200 controller module Each controller module has two Fibre Channel ports FIGURE 2 3 shows one host connected through fiber optic cables to one Sun StorEdge T3 array partner pair Sun StorEdge A3500FC Arrays Host Switches Controller A Host adapter FC AL ports Controller B Host adapter P FC AL ports SCSI x 5 Fibre optic cables a3 Drive tray x 5 FIGURE 2 1 Single Host Connected to One Sun StorEdge A3500FC Controller Module Chapter 2 Configurations 25 26 Host Switches Fibre optic cables Sun StorEdge A5200 Array IBA IBB FIGURE 2 2 Single Host Connected to One Sun StorEdge A5200 Controller Module Switches Host Host Adapter Sun StorEdge T3 array partner pair Host Adapter Fibre optic cables
141. ntTime 08 10 2001 19 02 48 Message s found in logfile var adm messages on diag226 central sun com id 80fee746 Warning Mar 10 19 01 34 SCS diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Warning Mar 10 19 01 47 SCS diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Warning Mar 10 19 02 04 SCS diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Threshold Error Mar 10 19 02 04 Received 22 SSD Alert message s on ssd70 in 137 mins threshold is 20 in 24hours Last Message diag226 Central Sun COM scsi ID 243001 kern warning WARNING scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 Source Agent diag226 central sun com Category Switch EventType AlarmEvent EventTime 08 10 2001 19 02 49 Change in Port Statistics on switch switch 155 ip 172 20 67 155 Port 2 Received 222 SyncLosses in 47 mins value 234 Port 2 Received 241 LinkFails in 47 mins value 311 Chapter 6 Examples of Fault Isolation 97 Source Agent diag226 central sun com Category San EventType LinkEvent EventTime 08 10 2001 19 03 27 CRC ERROR 20 in 4 mins Origin Port 2 on switch 172 20 67 156 Destination Port 2 on switch switch 155 172 20 67 155 Probable Cause 1 GBIC on Switchl Port 2 GBIC on Switch2 Port 3 Cable 4 Port of Switchl 5 Port of Switch2
142. ntral Sun COM scsi ID 243001 kern info pci 6 4000 SUNW glc 2 fp 0 0 fcp0 Mar 11 13 46 35 diag226 Central Sun COM offlining lun 1 trace 0 target 1046e4 trace 2800004 Mar 11 13 46 35 diag226 Central Sun COM scsi ID 243001 kern info pci 6 4000 SUNW glc 2 fp 0 0 fcp0 Mar 11 13 46 35 diag226 Central Sun COM offlining lun 0 trace 0 target 1046e4 trace 2800004 Mar 11 13 46 35 diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g60020f20000003c53b699abc00006c4c ssd70 multipath status degraded path pci 6 4000 SUNW qlc 2 fp 0 0 fp0 to target address 50020 23000003d5 1 is offline Mar 11 13 46 35 diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g60020f20000003c53b699aal0005114c ssd71 multipath status degraded path pci 6 4000 SUNW qlc 2 fp 0 0 fp0 to target address 50020 23000003d5 0 is offline 108 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Mar 11 13 46 40 diag226 Central Sun COM scsi ID 243001 kern info scsi_vhci scsi_vhci0 Mar 13 46 40 diag226 Central Sun COM Initiating failover for device ssd GUID 60020 20000003c53b699abc00006e4c Mar 13 46 40 diag226 Central Sun COM scsi ID 243001 kern warning WARNING pcit6 4000 SUNW qlc 3 fped 0 fcpl ar 11 13 46 40 diag226 Central Sun COM FCP WWN 0x50020f23000003c5 reset successfully
143. oad Brocade firmware on a special website provided by Brocade The URL for the Brocade site is site is http www brocade com 1 Click the Partners link 2 Click the Brocade Partner Network link 3 Enter the Sun internal login Enter the Sun internal password 4 Under Services and Support click Firmware 5 Click v2 6 x Firmware 6 Download v2 6 0a UNIX version and the Readme txt file to your local host v To Install Firmware from UNIX Solaris Follow these steps 1 From the Brocade web site retrieve the switch firmware for example v2 6 x 2 Download the firmware into your root directory Note Since UNIX contains rshd and cat daemons you do not need to retrieve the rsh ZIP file 140 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only 3 Log into the UNIX system as root and edit the following files a Type the IP address and the switch name into the etc hosts file vi etc hosts lt IP_address gt lt switch_name gt The output is displayed as in in CODE EXAMPLE 6 3 CODE EXAMPLE 6 3 etc hosts file cat etc hosts Internet host table 10 4 50 208 iopswitch03 ebay 10 4 50 212 iopswitch08 ebay b Type the switch name into the rhosts file rhosts lt switch_name gt Note If you are logged in as a normal user and not as root the rhosts file is referred to the user s home directory
144. or SUN Product ID T300 WWN Node 50020 20000003c5 WWN Port A 50020 23000003c5 WWN Port B 50020 23000003d5 Revision 0117 Serial Num Unsupported Unformatted capacity 119514 500 MBytes Write Cache Enabled Read Cache Enabled Minimum prefetch 0x0 Maximum prefetch 0x0 Device Type Disk device Path s dev rdsk c6t60020F20000003C53B699AA10005114Cd0s2 devices scsi_vhci ssd g60020f20000003c53b699aal0005114c c raw Controller devices pci 6 4000 SUNW glc 2 fp 0 0 Device Address 50020 23000003d5 0 Class secondary State STANDBY Controller devices pci 6 4000 SUNW qlc 3 fp 0 0 Device Address 50020 23000003c5 0 Class primary State ONLINE FIGURE 6 2 Sun StorEdge Traffic Manager Failover Example As shown above the LUN is now back using the primary path Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Example 2 Switch to Switch Error In this example Storage Automated Diagnostic Environment helps narrow down the troubleshooting to a specific link subsection almost immediately You can skip a couple of steps and save time by focusing on the correct link 1 Discover the error The first indication of any problem surfaced from a Storage Automated Diagnostic Environment email You requested the following events be forwarded to you from diag226 central sun com Source Agent diag226 central sun com Category Message EventType AlarmEvent Eve
145. or Message Formats 188 viii Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 Preface The Sun StorEdge SAN release 3 1 Field Troubleshooting Guide describes how to diagnose and troubleshoot the Sun StorEdge network FC switch 8 and switch 16 hardware It provides information and pointers to additional documentation you may need for installing configuring and using the configuration The book is primarily intended for use by experienced system support engineers who already have a good understanding of the product The Appendices found in this guide explain how to diagnose and troubleshoot Brocade Silkworm switches Using UNIX Commands This document may not contain information on basic UNIX commands and procedures such as shutting down the system booting the system and configuring devices See one or more of the following for this information m Solaris Handbook for Sun Peripherals m AnswerBook2 online documentation for the Solaris operating environment a Other software documentation that you received with your system For Internal Use Only Typographic Conventions Typeface AaBbCc123 AaBbCc123 AaBbCc123 Meaning The names of commands files and directories on screen computer output What you type when contrasted with on screen computer output Book titles new words or terms words to be emphasized Command line variable replace with a real name or value Shel
146. or Sun StorEdge A3500FC arrays 2 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Translative Loop Translative Loop Ports TL Ports allow private loop devices to communicate to off loop devices such as host bus adapters connected to F Ports and private devices on other TL Ports You can achieve this connectivity by having a translation function at each TL Port which translates private frames to public frames and vice versa m Devices connected to TL Ports are registered with the name server on the switch m Devices connected to TL Ports cannot communicate with devices on SL Ports In the current phase only the Sun StorEdge T3 array is supported on TL Ports Refer to the SANbox 8 Fibre Channel Switch Installer s and User s Manual rev 5 for more information Name Server Zoning Name server zones allow the division of the fabric one or more switch chassis into as many as 256 fabric wide zones that define which ports or devices receive name server information If hard zones are enabled name server zones will not communicate across hard zone boundaries m A name server zone is only valid if enabled m Only ports that register with the name server can be in a name server zone that is F FL Ports and TL Ports m If name server zones are used on a switch in which hard zones are defined the name server zones must not overlap hard zone boundaries m Name server zones operate fabri
147. ort logic test 61 power on self test POST 60 power switch location on switches 54 preface using UNIX commands ix R related documentation AnswerBook ix Solaris Handbook for Sun Peripherals ix rotary switch 10 position 63 rules adding and removing devices 22 array storage 17 cascading 22 zoning 21 S shell prompts x software unbundled supported 16 switch configuration guidelines 23 connection of 11 diagnostics 59 test mode 62 troubleshooting 59 switch counter information 73 switch counters descriptions 75 elapsed since counter reset 75 in frames 75 laser faults 75 role in troubleshooting 73 rules when viewing 73 switch modes front panel 63 switch port types 20 switch tool Capture 48 Sanbox API 48 T test mode switch 62 tests cable continuity 71 TL ports support of T3 arrays 3 INDEX 209 tools diagnostic 41 translative loop ports 3 troubleshooting cable continuity tests 59 power checks 59 power on self test POST 59 steps to use to approach a SAN problem 83 test mode switch functions 64 U UNIX commands use of ix Ww websites for additional information 2 Z zone hard zone 3 name server 20 zones configuration guidelines 23 definition of 20 hard 20 name server orphan 3 SL 21 zoning name server 3 zoning rules 21 INDEX 210 Sun StorEdge SAN Release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only
148. pci 6 4000 unit addres 2 Sun StorEdge Traffic Manager disable yes This would disable the single port HBA denoted in this example by fp0 To disable one port on a dual port card denoted by fp2 fp5 name qlc parent pci 6 2000 pci 1 unit address 5 Sun StorEdge Traffic Manager disable yes This would disable the HBA port denoted by fp5 A T3 lun with Sun StorEdge Traffic Manager enabled would look similar to this CODE EXAMPLE 1 2 ls l dev rdskc6t60020F20000003C53ABF4A1C000915F4d0s2 gt devices scsi_vhci ssd g60020 20000003c53abf4alc000915f4 c raw 8 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only luxadm display dev rdsk c6t60020F20000003C53ABF4A1C000915F4d0s2 DEVICE PROPERTIES for disk dev rdsk c6t60020F20000003C53ABF4A1C000915F4d0s2 Status Port Status Port Vendor Product ID WWN Node WWN Port A WWN Port B Revision Serial Num Unformatted capacity Write Cache Read Cache Minimum prefetch Maximum prefetch Device Type Path s A B O K O K SUN T300 50020 20000003c5 50020 23000003c5 50020 23000003d5 0117 096510 119514 500 MBytes Enabled Enabled 0x0 0x0 Disk device dev rdsk c6t60020F20000003C53ABF4A1C000915F4d0s2 devices scsi_vhci ssd g60020f20000003c53abf4alc000915f4 c raw Controller Device Address Class State Controller Device Address Class
149. ples of Fault Isolation 119 Example 4 Command Line Only Sun StorEdge A5200 Array Storage Side Error 1 Discover the error The problem begins with an email from the Storage Automated Diagnostic Environment to the system administrator You requested the following events be forwarded to you from diag226 central sun com Source Category A5k EventType AlarmEvent EventTime 08 08 2001 13 15 57 interface_board B gbic0O status changed from O K to Not Installed on DPL4 wwn 5080020000083chb0 Source Category Message EventType AlarmEvent EventTime 08 08 2001 13 15 58 Message s found in logfile var adm messages on diag226 central sun com id 80fee746 Warning Mar 8 13 14 03 SCSI diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Warning Mar 8 13 14 03 SCSI diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Warning Mar 8 13 14 03 SCSI diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Warning Mar 8 13 14 03 SCSI diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Warning Mar 8 13 14 03 SCSI diag226 Central Sun COM SCSI transport failed reason tran_err retrying command Warning Mar 8 13 14 03 SCSI diag226 Central Sun COM SCSI transport failed reason tran_err retrying command 120 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide
150. ppears both in the test topology view and in the list view Chapter 3 Diagnostic Tools 45 46 nostic Environment om Maintenance XY Layout Save Layout Change Vertical Monitor Utilities Help pagsa Report ests Tool Topology Help After having selected a topology several functions are available in the graphical view 1 Clicking in the middle of an object or ina specific component of an object will display the status at the bottom of the page 2 Right clicking the background will present a zoom menu 3 Right clicking on an object will present a menu of functions available for that object 4 Right clicking on a link will present a menu of functions available for that link 5 Holding the Shift button while selecting objects allows to select multiple objects This can be used to move multiple objects at the same time FIGURE 3 2 Storage Automated Diagnostic Environment Topology Window Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Sansurfer GUI 2 08 22 or above SUNWsmgr Use the Sansurfer switch GUI to configure the Sun FC Network switches It is a useful monitoring tool as it gives real time performance data and error counters It also has a built in Port Test that allows connections and GBICs to be verified SANsurfer gt Previous Next em 40 ae pa
151. pproach a SAN problem It lays out a methodical approach and lists various tools and resources available at each step It is expected that using Storage Automated Diagnostic Environment for monitoring will vastly decrease the time consuming process of narrowing down the problem The full examples that follow this section give further details on each step Step 1 Discover Error m Storage Automated Diagnostic Environment messages mw var adm messages m Application specific errors Step 2 Determine the extent of the problem m cfgadm al output m luxadm e port output m Storage Automated Diagnostic Environment topology view a Multipathing information Sun StorEdge Traffic Manager VxDMP Note The information gathered here will determine on which subsection to focus attention Host to Switch Switch to Switch cascaded or Switch to Storage Step 3 Check Array Status m open a telnet session to the Sun StorEdge T3 array m display luxadm output for Sun StorEdge A5200 arrays a Raid Manager CLI utilities for Sun StorEdge A3500FC arrays m Storage Automated Diagnostic Environment messages and alerts a LED status a Explorer T3Extractor output Chapter 6 Examples of Fault Isolation 83 Step 4 Check the Switch Status Capture Explorer output SANsurfer GUI LED status online offline Port Modes SL TL F Nameserver Information Note You can use the Storage Automated Diagnostic Environment to detect user configuratio
152. problem was a bad GBIC in IB Board B After replacing that GBIC watch for var adm messages that let you know things are back online var adm messages Example Mar 8 14 34 26 diag226 Central Sun COM Sun StorEdge Traffic Manager ID 779286 kern info scsi_vhci ssd g20000020374504e2 ssd46 multipah status optimal path pci 6 2000 pci 1 SUNW qlc 5 fp 0 0 fp3 to target address 22000020374504e2 0 is online You should see messages such as those above for all affected Sun StorEdge A5200 array disks Recommendations m Check the Sansurfer GUI to ensure Port 5 is active m Check luxadm display for proper GBIC insertion m Check the individual disks for the state of Sun StorEdge Traffic Manager Chapter 6 Examples of Fault Isolation 131 Command line example Status Port A Status Port B Vendor Product ID WWN Node WWN Port A WWN Port B Revision Serial Num Unformatted capacity Read Cache Minimum prefetch Maximum prefetch Location Device Type Path s Controller Device Address Class State Controller Device Address Class State luxadm display dev rdsk c6t20000020374504E2d0s2 DEVICE PROPERTIES for disk dev rdsk c6t20000020374504E2d0s2 O K O K SEAGATE ST39103FCSUN9 0G 20000020374504e2 21000020374504e2 22000020374504e2 034A LS6763540000 8637 338 MBytes Enabled 0x0 Oxfffft In the enclosure named DPL4 Disk device dev rdsk c6t20000020374504E2d0s2 devi
153. r 0004 recommended that POST be Replace mainboard Port failed to detect an interrupt within the timeout period Err 0B15 0415 or ASIC SRAM did not match assembly portRegTest data previously written into sramRetentionTest same location IDIAG REGERR_URST Port failed to unreset ASIC failure Replace mainboard Err 0B16 0416 assembly portRegTest sramRetentionTest IDIAG STATS Port counter value did not ASIC GBIC module Replace mainboard IBrr 2660 2662 3060 match the number of frames or fiber cable failure assembly GBIC 3062 actually transmitted Possible module or fiber cable portLoopback Test counters reporting crossPortTest e FramesTx number of frames transmitted e FramesRx number of frames received e CI3FrmRx number of Class 3 frames received IDIAG TIMEOUT For portLoopbackTest and ASIC GBIC module Replace mainboard IErr 266F 306F 386F crossPortTest or fiber cable failure assembly GBIC portLoopbackTest Port failed to receive frame module or fiber cable crossPortTest within timeout period centralMemory Test For centralMemoryTest Appendix C Brocade Error Messages 197 TABLE C 3 Diagnostic Error Messages Continued Message Description Probable Cause Action IDIAG XMIT Port failed to transmit frame ASIC failure Replace mainboard IBrr 2271 2671 3071 assembly 3871 portLoopbackTest crossPortTest spinSilk camTest CONFIG CORRUPT CONFIG OVERFLOW CONFIG VERSION FABRIC
154. r Internal Use Only Power On Self Tests POST When the switch is powered up it runs a series of POST tests including m Dynamic RAM Test m Port Register Test a Central Memory Test CMI Connector Test a CAM Test a Port Loop Back Test POST behaves differently depending on boot method A power cycle power off and power on is considered a cold boot All other boots from a powered on state are considered warm boots POST execution per cold boot executes a longer version of the Memory Test POST execution per warm boot executes a shorter version of Memory Test Boot time with POST varies depending on boot method As the POST test successfully performs each test a message Passed is displayed via telnet on the front panel After the switch completes the POST the port module returns to a steady state from the flashing state shown during tests If a yellow port module light is displayed or is slowly flashing this indicates that the port is in a failed state Should the switch fail to complete POST the green power LED will be set to blink This indicates that the switch failed one of the initial stages of POST and that the CPU is not able to bring up the operating system Should this occur replace the switch Appendix B Brocade Troubleshooting 167 Removing Power Caution Error messages are stored in RAM and are lost when power is removed from the switch Capture and view the error log output and note any
155. r Messages Continued Message Description Probable Cause Action IDIAG INIT Port failed to go active in the ASIC mainboard Replace mainboard Err 264F 304F 384F loopback mode requested GBIC module or assembly GBIC portLoopbackTest fiber cable failure module or fiber cable crossPortTest spinSilk IDIAG INTNIL ASIC failed to get a CMI ASIC failure Replace mainboard Err 2031 error interrupt assembly cmiTest IDIAG INTNOTCLR The interrupt bit could not be ASIC failure Replace mainboard Err 102C cleared assembly centralMemoryTest IDIAG LCMEM Data read from the Central ASIC failure Replace mainboard Err 1027 Memory location did not assembly centralMemory Test match data previously ccmemRetentionTest written into the same location IDIAG LCMEMTX Central Memory transmit mainboard failure Replace mainboard Err 1F27 1028 path failure ASIC 1 failed to assembly centralMemoryTest read ASIC 2 via the transmit path IDIAG LCMRS Central Memory Read Short ASIC failure Replace mainboard Err 1F25 1025 M bytes requested but got assembly centralMemoryTest less than M bytes cmemRetentionTest IDIAG LCMTO Central Memory Timeout ASIC failure Replace mainboard Err 1F267 1026 Data transfer initiated did assembly centralMemory Test not complete within the cmemkRetentionTest timeout period IDIAG MEMNULL Test failed to malloc mainboard failure Replace mainboard Err 0112 assembly ramTest IDIAG MEMSZ Memory size to be tested is mainboard fa
156. rEdge Traffic Manager failures Status Port A Status Port B Vendor Product ID WWN Node WWN Port A WWN Port B Revision Serial Num Unformatted capacity Read Cache Minimum prefetch Maximum prefetch Location Device Type Path s Controller Device Address Class State Controller Device Address Class State luxadm display dev rdsk c6t2000002037450D71d0s2 DEVICE PROPERTIES for disk dev rdsk c6t2000002037450D71d0s2 OB Daka SEAGATE ST39103FCSUN9 0G 2000002037450d71 2100002037450d71 2200002037450d71 034A LS6810850000 8637 338 MBytes Enabled 0x0 Oxffff In the enclosure named DPL4 Disk device dev rdsk c6t2000002037450D71d0s2 devices scsi_vhci ssd g2000002037450d71 c raw devices pci 6 2000 pci l SUNW glc 4 fp 0 0 2100002037450d71 0 primary ONLINE devices pci 6 2000 pci 1 SUNW glc 5 fp 0 0 2200002037450d71 0 primary OFFLINE The above output shows that the path devices pcit 6 2000 pci 1 SUNW qlc 5 fp 0 0 is OFFLINE Note A Sun StorEdge A5200 array under Sun StorEdge Traffic Manager control should have two primary ONLINE paths 126 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only c Check the HBA status Is Qlogic q1c3 the same as the physical path devices pcit6 2000 pci l1 SUNW qlc 5 fp 0 0 Using either the GUI or a command line check the Sun Storage Automated Diagnostic Environm
157. range 54 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Back Panel LEDs LEDs visible through lenses in the back of the chassis indicate chassis and port status During a reset operation for about two seconds at the beginning of power on all LEDs are forced ON The following definitions are valid following the POST when the POST finds no errors Heartbeat LED Yellow The heartbeat LED indicates the status of the internal switch processor and the results of POSTs run at power on Following a normal power on the heartbeat LED blinks about once per second to indicate that the switch has passed the POSTs and the internal switch processor is running Switch Logic Power Good LED Green This LED is ON when the power supply is delivering power within normal limits to the switch logic the power switch must be ON Fan Fail LED RED This LED is normally OFF It comes ON only when the speed of a fan drops below operational level Over Temperature LED Red This LED is normally OFF The over temperature LED lights to indicate that the air temperature inside the switch has exceeded a certain limit If this LED lights inspect the following a Ambient air temperature maximum 40 C 104 F m Proper clearance 163 mm 6 5 back right side and front m Fan Operation m Power supply operation Chapter 3 Diagnostic Tools 55 Logged In LED Green Each port has its own Logged I
158. rocade Troubleshooting 171 172 Troubleshooting Case Study The following case study illustrates a practical application of the steps outlined above This is only one way to approach the problem there may be others Note Knowledge and training on all components in a SAN are prerequisites before you attempt the procedures below In this test case the I O load was generated using the dex disk exerciser to simulate customer load and the steps below allowed the I O to continue uninterrupted throughout the procedure Configuration m Sun Fire V880 a Solaris 8 10 01 with all recommended and latest Leadville driver patches m Sun StorEdge T3 array Partner Pair with FW 1 18 Brocade Silkworm 2400 and 2800 switches with v2 6 0 firmware m Storage Automated Diagnostic Environment with patch 112580 01 Note The Storage Automated Diagnostic Environment was configured to send email alerts to a system admininstrator s email address Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Storage Automated Diagnostic Environment Topology In this example the two switches were zoned in such a way to allow two separate paths for the Sun StorEdge T3 array partner pair In normal configurations two separate pairs of switches should be used to eliminate single points of failure Environment Storage Automated Diagnostic Environment microsystems 2 0 06 010 diag229 centr
159. rther details on installing and administering Sun StorEdge Traffic Manager 10 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only CHAPTER 2 Configurations This chapter contains information and instructions for configuring your Sun StorEdge network FC switch 8 or switch 16 with one or more hosts and storage The Sun StorEdge Network FC Switch 8 or Switch 16 Switch Either of the Sun StorEdge Network FC Switch 8 or Switch 16 units functions with the Sun StorEdge A3500FC array or a Sun StorEdge T3 array Supported Hardware In a single switch configuration the switch is connected to the host through a fiber optic cable to a Sun StorEdge PCI Fibre Channel Network Adapter The other ports of the switch are connected to storage devices through a fiber optic cable In a cascaded configuration two switches are connected together by way of InterSwitchLinks ISL A hard zone and a name server zone span both switches 12 TABLE 2 1 Supported Hardware Part Number 540 4026 540 4027 950 3475 X4748A X6727A x6799A X6731A X7637A X978A X6746A SG XSW16 32P Description Sun StorEdge A3500FC FC AL controller for Sun StorEdge A3500FC Array with Sun StorEdge D1000 tray Sun StorEdge A3500FC FC AL controller for Sun StorEdge A3000 array with Sun StorEdge RSM tray Sun StorEdge T3 Array Sun StorEdge PCI Dual Fibre Channel Network Adapter Sun StorEdge PCI Dual F
160. rts can be converted to a Fabric Loop Attach FLA compliant FL_Ports by disabling the QuickLoop mode on that port Appendix B Brocade Troubleshooting 155 Current Issues with the Storage Automated Diagnostic Environment and Brocade m You cannot run the brocadetest 1M between the Brocade Silkworm switch F Port and the Sun HBA m ELS ECHO diagnostics are not supported by the Brocade Switch linktest between the HBA and the Brocade Silkworm switch a Brocade Silkworm switches must be added manually or discovered by subnet discovery Do not put Brocade Silkworm switch IP addresses in the etc fcswitch conf file as you can with Sun StorEdge network FC switches a The minimum Brocade Silkworm switch firmware to use with Storage Automated Diagnostic Environment is v2 6 0 Diagnostic Tools The tools available for troubleshooting differ from the original release of the Sun StorEdge SAN 3 0 Since then Sun StorEdge StorTools 4 x and Network Storage Agent 2 1 have had their functionality combined into a single diagnostic package called the Storage Automated Diagnostic Environment Brocade Silkworm switches also have their own GUI called WebTools Storage Automated Diagnostic Environment and Brocade With the Storage Automated Diagnostic Environment 2 0 release Brocade is now supported under the same diagnostic framework as the rest of the Sun SAN product family This support includes monitoring for error and alert conditions revision chec
161. s Chapter 4 Diagnosing and Troubleshooting the Switch 61 62 Using the Test Mode Switch The test mode switch is a small rotary switch located on the back of the switch chassis The test mode switch enables the switch chassis to perform the following functions Normal Operation Performs POST diagnostics once at the time of startup and then proceeds to normal operation m Force PROM Used to gain access to the PROM when flash memory or the resident configuration file is disabled The test mode switch position determines which functions are performed when the switch chassis is powered on See FIGURE 4 1 for test mode switch functions and positions Normal operation is indicated by the alignment of the small notch on the test mode switch with the dot on the faceplate Caution Use the test mode switch on the back panel while performing maintenance tasks only Data may be corrupted if the test mode switch is used while the switch chassis is operating Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Front Panel Switch Modes The following are the settings for the 10 position rotary switch 0 Normal operations 1 Continuous test 2 Test bypass 3 Operator test 4 Normal operation initial test with force PROM mode 5 Continuous test with force PROM mode 6 Test bypass with force PROM 7 Operator test with force PROM 8 Normal operation initial test with watchdog timer disabled 9
162. s 256 fabric wide zones each name server zone defines which ports or devices receive name server information as defined by the FC GS3 document If hard zones are enabled name server zones do not communicate across defined hard zone boundaries Up to 16 name server zones are possible within one hard zone Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only SL Zones SL zones on the switch allow the division of the fabric into zones that define the ports that can communicate with each other A particular port may be placed in only one SL zone no overlapping SL zones If hard zones are enabled SL zones do not communicate across defined hard zone boundaries The switch does not prevent the user from creating an SL zone spanning more than one switch however the current release of the Sun StorEdge FC switch 3 1 requires the user not to extend the SL zone beyond one switch Zone Type Supported Behavior Hard Zone Supports Segmented Loop zones and name server zones Segmented Loop SL ports only zone Name Server Zone F Ports T Ports and TL Ports 1 The Segmented Loop zone and the name server zones cannot co exist in the same hard zone Zoning Rules a A minimum of one switch port per zone A maximum of 16 zones per 16 port switch a A maximum of 30 zones for cascading 16 port to 16 port switches a Only port based zoning is supported m Zones are non overlapping except in cascading sw
163. s at the same time sh t3 172 20 67 168 slr mi 370 3990 01 e f0 016101 monitored by diag22 XY Layout Save Graphics Turn On Layout Change Vertical FIGURE B 8 Error free Topology AppendixB Brocade Troubleshooting 185 186 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only APPENDIX C Brocade Error Messages This appendix explains the error message format and possible errors This section includes m Error message formats m Error message modules Error Message Formats There are two error message formats depending on whether you are gathering information from the front panel or via Telnet The front panel shows an abbreviated message and by way of Telnet displays the entire error message In all cases the last error encountered is the first error displayed Up to 32 messages are held in a buffer If the 32 message limit is exceeded the messages are overwritten in a first in first out sequence Note Error messages are stored in volatile RAM and are lost whenever power is removed from the switch Access the error message log to view error messages before removing power 187 Front Panel Message Formats The Brocade Silkworm switch s front panel displays error messages The first line includes the error s date and time The beginning of each second line on the front panel display starts with the module name error name a
164. s port Number of primitive sequence protocol errors An error indicates that a sequence protocol violates the FC 2 signaling protocol 76 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only TABLE 5 1 Port Display Window Counters Counter Name in port display Reject Frames Reserved Retry LIPs Short Frame Errors Smoothing Overflow Errors Sync Loss Sync losses 100 ms Description Number of frames from devices that have been rejected Frames can be rejected for any of a large number of reasons N A Currently not used Number of times a frame shorter than 36 bytes was received Number of times that a violation of FC rules on the incoming signal were detected An example of a violation is an insufficient number of idles received between frames Number of synchronization losses detected through reception of invalid transmission words on the port Number of synchronization losses gt 100 ms detected by this port A loss of synchronization is detected by receipt of an invalid transmission word Chapter 5 Switch Counter Information 77 78 Web GUI File Edit View Special Help e switch Faceplate Display Undo Zoning Refresh Previous Next TT a GEE 3c PEE occ APE EEN EE Soe Soa i amp Sun j Prom sw 30200 mrayta Delete blo Power NORMAL i S 3 Temp 26 ok Fand ok lt sysName undefin
165. sS amp Sun microsystems Sun StorEdge Network FC Switch 8 and Switch 16 Field Troubleshooting Guide Sun Microsystems Inc 4150 Network Circle Santa Clara CA 95054 U S A 650 960 1300 Part No 816 1701 12 April 2002 Revision A Sun StorEdge SAN 3 1 Release Send comments about this document to docfeedback sun com Copyright 2002 Sun Microsystems Inc 4150 Network Circle Santa Clara CA 95054 USA All rights reserved This product or document is protected by copyright and distributed under licenses restricting its use copying distribution and decompilation No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors if any Third party software including font technology is copyrighted and licensed from Sun suppliers Parts of the product may be derived from Berkeley BSD systems licensed from the University of California UNIX is a registered trademark in the U S and other countries exclusively licensed through X Open Company Ltd For Netscape Communicator the following notice applies Copyright 1995 Netscape Communications Corporation All rights reserved Sun Sun Microsystems the Sun logo AnswerBook2 docs sun com Sun StorEdge network FC switch 8 and Solaris are trademarks registered trademarks or service marks of Sun Microsystems Inc in the U S and other countries All SPARC trademarks are used under license and are trad
166. st likely problem In this example the cable was replaced and the connection was recabled Step 6 Verify the fix 182 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only To verify the fix after replacing the cable Check the cfgadm output to see if the device appears back in the fabric CODE EXAMPLE B 10 cfgadm Output cfgadm al c2 fc fabric connected configured unknown c2 50020f2300003d2c disk connected configured unknown c3 fc fabric connected configured unknown 3 50020f2300003ee5 disk connected configured unknown The c2 device is connected Return the path to service using the Sun StorEdge Traffic Manager failover commands Check the status of the device Appendix B Brocade Troubleshooting 183 luxadm failover primary dev rdsk c6t60020F2000003EE53AAF7A09000DA257d0s2 luxadm display dev rdsk c6t60020F2000003ERE53AAF7A09000DA257d0s2 DEVICE PROPERTIES for disk dev rdsk c6t60020F2000003EE53AAF7A09000DA257d0s2 Status Port A O K Status Port B O K Vendor SUN Product ID T300 WWN Node 50020 2000003d2c WWN Port A 50020 2300003d2c WWN Port B 50020 2300003ee5 Revision 0118 Serial Num Unsupported Unformatted capacity 488642 000 MBytes Write Cache Enabled Read Cache Enabled Minimum prefetch 0x0 Maximum prefetch 0x0 Device Type Disk device Path s dev rdsk c6t60020F2000003EE53AAF7A09000DA257d0s2 devices scsi_vh
167. stmap port targetid addr_type lun volume owner access ulpl 1 hard 0 vO ul primary ulp1 1 hard 1 v1 ul failover u2p1 2 hard 0 v0 ul failover u2pl 2 hard 1 v1 ul primary diag190 lt 3 gt sys stat Unit State Role Partner y ONLINE Master 2 2 ONLINE AlterM 1 This output confirms what you have already discovered A failover has occurred and it is affecting the AltMaster only The controller itself is ONLINE so the problem most likely is outside the Sun StorEdge T3 array 112 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only 4 Check the Switch Status a Concentrate on the Host to Switch link The HBA in the affected path is pci 6 4000 SUNW qlc 2 fpQ 0 0 b Use the Storage Automated Diagnostic Environment GUI to see what switch is C connected to this HBA Look at the output of discman below pci 6 4000 SUNW qlc 2 fp 0 0 corresponds to qlc 0 opt SUNWstade diags bin discman HBA Port 0 Path devices pci 6 4000 SUNW qlc 2 fp 0 0 devct1 RegisterName qlc0 LGroup StorEdge QLC HostBusAdaptors PGroup StorEdge IO Board Slot 3 Slot on IO Board 0 Port on Card 0 odeWWN 200000e 08b032a01 PortWWN 210000e08b032a01 ode online DriverName fp Instance 0 al_pa 31 DeviceType 3 Looking further in the discman output the first switch in this path is 172 20 67 155 as shown below Also evident is that the HBA connects to this swi
168. tch on port 3 as an F Port Chapter 6 Examples of Fault Isolation 113 Device 0 LogicalPath PhysPath HBA node path RegisterName qlc0 sw0 f3 LGroup StorEdge 8P Switches qlc0 PGroup StorEdge qlc0 odeWWN Port WWN 210000e 08b032a01 Status 0 sw_PortNum 3 sw_PortType F_Port sw_ipAddr LIQ 20667 2155 114 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Check Capture for some more details on port 3 KKKKKKKKKKKK Port Status KKKKKKKKKKKK Port Port Type Admin State Oper State Status Loop Mode 1 F_Port online offline Not logged in 2 T_Port online online logged in 3 F_Port online offline Not logged in KKKKKKKKKKK port count KKKKKKKKKKK Port Number 1 Inframes Outframes 0 LinkFails I OLS Out Port Number 2 Inframes 30 Out frames 1046115 InvalidTxWds 1 OLS Out 3 Total LIP Revd 102 LIPELE T 102 Port Number 3 Inframes 1594519 Outframes 41204 Discards 402 LinkFails 13895 Prim Seq Errs 10252 InvalidTxWds 3 OLS In 10251 OLS Out 3139 KKKKKKKKKKKK Name Server KKKKKKKKKKKK Port Address Type PortWWN Node WWN FC 4 Types Database is empty From the above information notice that port 3 is currently not logged in has a relatively high number of errors and currently has no information in the Nameserver You can also look at the SANsurfer GUI or have the customer look and relate the information
169. ted Loop Zone 1 Ports 3 4 6 All Ports are SL Switch 0 and Switch 1 Nameserver Loop Zone 1 and 1 respectively Ports 5 F 7 TL empty 8 TL FIGURE 2 9 Single Host Cascading Switches Connected to Sun StorEdge T3 Array and Local Storage Sun StorEdge A5200 Array and Sun StorEdge T3 Array E z Zone 1 M 4 2h 3 4447 5 CB Zone 1 7 8 T 9 10 1 2 11 12 3 4 13 14 5 _ 6 15 16 Switch 2 Switch O IBA IBB Zone 1 IBA IBB 1 2 Zone 1 3 4 a J Lf 2 5 ae Ge a 8 3 7 _ E 5 6 Switch 1 7 8 9 10 11 12 13 14 15 16 Switch 3 Chapter 2 Configurations 33 Multihost FIGURE 2 10 shows an example of a multihost configuration two hosts connected through fiber optic cables using switches to two Sun StorEdge A3500FC controller modules FIGURE 2 10 shows two hosts connect to a maximum of four Sun StorEdge A3500 FC arrays FIGURE 2 11 shows two hosts connected to three Sun StorEdge A5200 arrays FIGURE 2 12 shows two hosts connected to four Sun StorEdge T3 array partner pairs FIGURE 2 13 shows two hosts connected to a Sun StorEdge T3 array Partner Group in which each host maintains separate non shared storage FIGURE 2 14 shows a multihost configuration with cascading switches connected to an FC tape library a Sun StorEdge T3 array and a Sun
170. tests good the heartbeat LED blinks three times between the three second pauses No port Logged in LEDs blink You may load new flash control code via the Switch Management port See the Switch Management manual for a description of how to load new flash code 66 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Flash Checksum Failure Switch Management port Ethernet Failure Four Blinks The switch is not operable The flash checksum test verifies the integrity of the flash data If the flash data is corrupt the POST checks the Switch Management port to find out if it is functional The Switch Management port is the load path for loading new flash data If the Switch Management ports tests bad the heartbeat LED blinks four times between the three second pauses No port Logged in LEDs blink This means that the flash control code is corrupt and the Switch Management port may not operate well enough to load new flash code Force PROM Mode in Effect Five Blinks This is an alarm Five blinks indicate that the processor is reading the default configuration from PROM instead of from flash memory The test mode switch is in the force PROM position This error never occurs unless you are using the force PROM button The heartbeat LED blinks five times between the three second pauses Switch ASIC Test Failure Six Blinks The switch is not operable The switch ASIC test verifies the base
171. the switch received a LIP while it was already in the initialization state This LIP is used to re initialize the loop An L_port identified by AL_PS may have noticed a performance degradation and is trying to restore the loop A loop initialization primitive frame used to acquire an AL_PA This LIP denotes a loop failure detected by the L_port identified by AL_PS Currently not used Number of loop initialization primitive frames received The LISM primitive is used to select a temporary loop master for initialization This counter shows the number of times the switch was unable to establish itself as the loop master Currently undefined Number of times the switch was unable to transmit a frame within the R_T_TOV value Number of times a frame longer than the maximum frame size was received Number of signal losses detected for this port Number of offline sequences received An OLS is issued for link initialization a Receive amp Recognize Not_Operation NOS state or to enter the offline state Number of offline sequences issued by this port An OLS is issued for link initialization a Receive amp Recognize Not_Operation NOS state or to enter the offline state The switch may issue an OLS to perform offline diagnostics or to power down Number of times a device on the loop didn t accept an open primitive This usually indicates a device error Number of class 2 and class 3 frames transmitted by thi
172. ther the same information by querying the Storage Automated Diagnostic Environment that you can gather using the sanbox API These methods are completely supported unlike command line sanbox API usage 52 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Power Switch FC Switch LEDs and Back Panel Controls FIGURE 3 4 and FIGURE 3 5 identify the parts of the switch chassis back Port numbers are marked on the chassis Port Number Switch Management j tor RJ4 Activity LED Connector RJ45 ernet f Traffic LED Yellow Der Logged In LED Green Link Status LED Ethernet MAC Address Label TU iC 203 ye ELI WY SEI YY SEI o e e e Ld o omen HEED HE ie EN Switch Logic Power npt AC Over Good LED Green Fuses Power emperature Ph LED Red g Heartbeat LED Test Mode Switch Fan Fail Yellow LED Red Logged In LED Green Traffic LED Yellow Port Number FIGURE 3 4 Chassis Back 8 Port Switch Chapter 3 Diagnostic Tools 53 Port N
173. to Switch Switch to Switch cascaded or Switch to Storage Step 3 Check Array Status m Open a telnet session to the Sun StorEdge T3 array m Refer to the luxadm display output for Sun StorEdge A5200 arrays m Raid Manager Healthcheck for the Sun StorEdge A3500FC arrays m Storage Automated Diagnostic Environment instrumentation reports m LED status a Explorer T3Extractor output 170 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Step 4 Check Switch Status m Explorer output support Show output m WebTools GUI LED Status m Storage Automated Diagnostic Environment instrumentation reports m Port Modes F L E online offline m Nameserver Information nsshow nsallshow Step 5 Start Testing FRUS a Storage Automated Diagnostic Environment tests such as the brocadetest 1M qictest 1M t3test 1M linktest 1M and a5ktest 1M m Brocade CLI tests loopPortTest spinFab crossPortTest m Sun StorEdge T3 Array tests such as T30FDG 1M m Sun StorEdge A3500FC arrays Healthcheck Note The conclusion of these tests isolate the problem to a FRU to be replaced Follow the appropriate hardware manual for proper FRU replacement procedures Step 6 Verify the fix m var adm messages path online multipath informational messages m Storage Automated Diagnostic Environment status m Sun StorEdge Traffic Manager or VxDMP to return the path to its normal state Appendix B B
174. tput Options Test List 4 03 19 13 35 02 0 0 44 T Target 72 20 67 167 Device 1 2 172 20 67 167 Ended 2002 03 19 13 35 46 RC 0 Called with options dev 2 172 20 67 167 iterations 1000 passwd xxxexxx Connect to 172 20 67 167 Opened 172 20 67 167 Logged into 172 20 67 167 i portis in loopback mode i Clear port errors send diagClearError 2 Port errors cleared Running command CrossPortTest 1000 1 Note You should only have a loopback on port 2 If you have more than one loopback installed i this test may report false errors Test Passed i i Loopback took 34 seconds to run Fan 1is OK speed is 8640 RPM i Fan 2 is OK speed is 8760 RPM Fan 3 is OK speedis 8940 RPM Fan 4 is OK speed is 8820 RPM Fan 5 is OK speed is 8820 RPM Fan 6 is OK speed is 8850 RPM LEE E E E E E E E GOOG ORR OK E Detected possible bad Power supply Power Supply 1 is absent DSSS SOSA EOS AITO Power Supply 2 is OK i Close 172 20 67 167 FIGURE B 6 Test Detail Showing Test Passed This suggests that the GBIC and switch port are most likely not the cause of the error Test the HBA by inserting the loopback plug into the HBA and running the HBA test qlctest shown in FIGURE B 7 AppendixB Brocade Troubleshooting 181 Netscape Syncing 17091 on diag229 central sun com ST 17091 gictest es pei ii Ended 2002 03 19 13 41 14 RC 0 qlctest called with options dev devices pci 9 600000
175. tween three second pauses No port Logged in LEDs blink Switch Auto Route Test Failure 10 Blinks The switch is operable The switch auto route test verifies the auto route capability of individual ports to route frames to the other ports in the chassis The heartbeat LED blinks 10 times between three second pauses the switch disables the failing ports or port pairs and blinks their Logged in LEDs The ports whose Logged in LEDs are not blinking have passed the test Chapter 4 Diagnosing and Troubleshooting the Switch 69 70 Eleven and Twelve Blinks Not Used Arbitrated Loop Test Failure 13 Blinks The switch is operable The arbitrated loop test verifies the ability of the arbitrated loop ports to initialize properly The heartbeat LED blinks 13 times between three second pauses The switch disables the failing ports and blinks their Logged LEDs The ports whose Logged in LEDs are not blinking have passed the test Switch Management Port Failure 14 Blinks The switch is operable The switch management port test verifies the functionality of the Ethernet data bus A failure indicates that communication over the Ethernet port will probably be adversely affected The heartbeat LED blinks 14 times between three second pauses No port Logged in LEDs blink NVRAM Test Failure 15 Blinks The switch is not operable The Non Volatile Memory NVRAM test verifies the status of the NVRAM battery not low performs a ch
176. twork Power Checks and Troubleshooting The following procedure assumes the Power Good LED does not illuminate Check that The power switch is in the ON l position The AC power outlet has the proper voltage The power cable has continuity and is plugged into both the AC power outlet and the switch chassis The input fuses are functioning properly If the logged in LED is off and the device attached to the port is a host be sure the host is powered on and booted 59 m If the logged in LED is off and the device attached to the port is a storage unit be sure it is powered on and is operating normally You can verify the status of your array from the array s front LEDs and from RM6 Refer to the Sun StorEdge array manuals for information Power On Self Test POST At startup the switch runs a series of Power On Self Test diagnostics These POST diagnostics check for proper switch operation excluding the GBICs If no fatal errors are encountered the switch becomes operational During the POST the switch logs any errors encountered Some POST errors are fatal others are non fatal A fatal error disables the switch so that it does not operate A non fatal error allows the switch to operate but with some decrease in performance until the problem is corrected m A PROM checksum failure is an example of a fatal error It indicates that the PROM firmware is corrupt and the switch does not operate m A failure associ
177. ually from the command line luxadm failover secondary dev rdsk c6t60020F20000003C53B699ABCO00006C4Cd0s2 100 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only 4 Watch for successful failover in var adm messages shown below Mar 10 19 23 41 diag226 Central Sun COM Failover operation completed successfully for device ssd GUID 60020 20000003c53b699abc00006c4c failed over from primary to secondary 3 Check the Array Status Skip this step for now as you are concentrating on the switch to switch link You can return to this step if needed 4 Check the Switch Status You can use either Capture or the SANsurfer GUI to view the port status Pay particularly close attention to the T Ports between the two switches Refer to FIGURE 6 3 to view Port 2 Counter Status Chapter 6 Examples of Fault Isolation 101 SANSurfer Alm Ha 4 Zoning Refresh Delete PreviousNext A Sp e a E Ce E T ite a aooo t words ecv meae oei pe a E beeen o in Beea Ii ooo ut err eaa FSFT peg Lip Fe ALPS Lip AL a PS AL Init Eros OAL Tn At ree ee Enors Short Frame Enors onr e oa LaserFauts o ooo O ee me p o e AAAA LOF Timeout ELS O OFT S oo Y oe ag ekg fern rts O EE bal i TOK Rok Dis piri Bts Ofs No reply fiom the Switch switch 156 F
178. umber Switch Traffic LED Yellow Management Connector Logged RJ45 AC Power LED Green MAC Address Plug i Power Switch Label ON T 73 EEn SEED EED SEED Ee SEED EED SEEN A SEn E SEET CLN SELE CL SEE Over Heartbeat ao aag re LED Force RED Yellow aon utton Fan Fail Switch Logic Logged In LED LED RED Power Good Green LED Green Traffic LED Yellow Port N umber FIGURE 3 5 Chassis Back 16 Port Switch Power Switch FIGURE 3 4 and FIGURE 3 5 show the location of the power switch The power switch is a rocker switch Press the right side labeled 1 to turn it ON press the left side labeled 0 to turn it OFF When you press the power switch and turn it ON there is a two second delay before the fans start and the Power Good LED on the back of the chassis illuminates The Power Good light indicates that the switch logic is receiving power within the proper voltage
179. upported You must be using PCI dual Fibre Channel Network Adapter and PCI single Fibre Channel Network Adapter HBAs Chapter 2 Configurations 13 Arrays m Sun StorEdge T3 array m Sun StorEdge A3500FC array Supported Host Bus Adapters a PCI Single Fibre Channel network adapter a PCI Dual Fibre Channel network adapter a CPCI Dual Fibre Channel network adapter FC Tape m Sun StorEdge L180 Tape Library with StorageTek 9840 tape drives m Sun StorEdge L700 FC Tape Library with StorageTek 9840 tape drives 14 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only Packages and Patches The packages and patches found in TABLE 2 2 are mandatory and must be installed in the order listed The assumption is that you are starting from an updated Solaris 8 6 01 installation Refer to the most current Sun StorEdge SAN release 3 1 Release Notes and Sunsolve for the most recent patches and packages beyond the levels listed below TABLE 2 2 Solaris 8 6 01 Minimum Required Packages and Patches 8 Recommended Solaris 8 Recommended and Security Patches SUNWsan SAN Foundation package 111847 04 SUNWsan patch 108982 09 Solaris Driver patch 108984 08 Solaris Driver patch 111412 07 Sun StorEdge Traffic Manager patch 111095 07 fcp fp fclt usoc drivers patch 111096 03 fcip driver patch 111097 07 qlc driver patch 109529 06 luxadm libg_fc liba5k patch 111413 07 luxadm libg_fc liba5k patch SUNWcfpl cfgadm_
180. urs de Sun Dez partie de ce produit pourront tre d riv es des systemes Berkeley BSD licences par l Universite de Californie Sun Sun Microsystems le logo Sun et Sun StorEdge sont je marque de fabrique ou des marque deposees de Sun Microsystems Inc aux Etats Unis et dans d autres pays This site can be used to configure and maintain the agents It can also be used to monitor and diagnose Sun storage products See the Help page for more details Site Info Sun Microsystems Network STORAGE Installation 2 hosts 10 devices Devices 3 A5ks 4 Switches 3 T3s Notifications Http NetConnect Email 1 brad derolf sun cor Last Event 2002 02 24 17 25 09 Last Alert No alerts Basic Installation Steps eview w Create Site Info i i i Topology i Agents FIGURE 3 1 Storage Automated Diagnostic Environment Main Window Chapter 3 Diagnostic Tools 43 Storage Automated Diagnostic Environment Diagnostic Functionality Diagnostic tests have been integrated into the Storage Automated Diagnostic Environment for device diagnostics and field replaceable unit FRU isolation Each test can be run individually from the command line or from the Storage Automated Diagnostic Environment user interface The following tests are described in the Storage Automated Diagnostic Environment Diagnostics chapter Sun StorEdge A3500FC Array Test a3500fctest Sun StorEdge A5000 Array Enclosure Test a5ksestest Sun StorEdge A5000 Array Test a
181. vent EventTime 08 10 2001 14 30 09 port 7 in SWITCH csw 194 ip 172 20 67 194 is now Not Available status state changed from Online to Offline Source Agent diag226 central sun com Category T3message EventType AlarmEvent EventTime 08 10 2001 14 30 10 Message s found in logfile var adm messages t300 on diag226 central sun com id 80fee746 Mar 10 21 16 50 diagl90 Central Sun COM ISR1 1 W ulctr ISP2100 2 Received LOOP DOWN async event Mar 10 21 17 16 diagl190 Central Sun COM MNXT 2 W u2ctr starting lun 0 failover Note The Message Event from above is extracted from the var adm messages file It is possible that this problem would be initially discovered by watching the var adm messages file instead of Storage Automated Diagnostic Environment emails This will depend on customer configuration For example the T3message Event may be noted by monitoring the T3 syslog 90 Sun StorEdge SAN release 3 1 Field Troubleshooting Guide April 2002 For Internal Use Only What is known at this point m The physical path pci 6 4000 SUNW qlc 3 fp 0 0 fp1 is in degraded mode m Port 7 on the switch at 172 20 67 194 went offline m There was a LUN failover to u2ctr 2 Determine the extent of the problem a cfgadm al output cfgadm al c4 fc fabric connected configured unknown c4 50020f23000003q5 disk connected configured unknown c5 fc fabric connected configured unknown
182. vice Pack Type HBAs Firmware Patches Patches Sun Amber 1 12 1 13 Leadville 5C Recommende x6799A d it Ex20 111095 07 Pen a Crystal 1 13 Solaris 8 u6 patches 6727A Ex50 x 111096 03 Leadville Diamond 1 13 unbundled Ex80R x6748A 111097 07 packages rs Ex500 Ivory x6757A 1 13 06 111412 07 SUNWsan SUNWcfpl Fx800 111413 07 SUNWcfplx F4810 111846 03 E10000 111847 01 Appendix B Brocade Troubleshooting 153 v To generate the most recent patch list specific to the Brocade SAN 1 Access the PatchPro Realization Table 2 Select SAN Brocade SAN 3 1 3 Select the Brocade SAN Release icon for installations and upgrades TABLE B 2 Disk Array Supportability Matrix Dynamic addition of target to a zone Host Disk Arrays Disk Firmware Add First Additional Max targets per zone T3A WG ES 1 18 Yes Yes 8 4 Solaris 8 T3B WG ES 2 01 Yes Yes 8 4 A3500FC 03 01 03 54 or later No Yes 4 FC Tape L180 Drive Firmware No Yes 4 StorageTek 9840 Rev 28 127 TABLE B 3 Fibre Channel Switch Supportability Matrix Host FC Switches Firmware Switch Software Licenses Brocade Silkworm v2 6 0a Fabric OS Zoning Quickloop 2400 v2 6 0a Webtools lari solaris S Brocade Silkworm v2 6 0a Fabric OS Zoning Quickloop 2800 v2 6 0a Webtools Brocade Silkworm v3 0 2a Fabric OS Zoning Quickloop 3800 v3 0 2a Webtools Notes 1 Brocade Silkworm 3800 FC Switches are supported in 1 GB mode only 2 Brocade Silkworm 2400 2800 and 3800 FC Switch
183. ying Upgrade Compliance Before starting the upgrade insure that your system is stable and that all required software and patches are accessible Refer to Appendix B for the supportability matrix v To Upgrade Software If you have multiple hosts on your SAN you can upgrade them simultaneously or one at a time without affecting your SAN environment Hosts that are not being upgraded will not be affected during the upgrade You can upgrade the host software one host at a time or several hosts in parallel Caution Your system will be unavailable to users and the SAN during the upgrade procedure The order in which the SAN components should be upgraded is as follows 1 Familiarize yourself with the required software components versions and patches Refer to Appendix B for the supportability matrix 2 Back up all data Appendix A Brocade Upgrades and Installations 145 Volume Management If you are using VERITAS Volume Manager refer to the VERITAS Volume Manager Installation Guide for special instructions about upgrading your system There are several steps that need to be followed prior to your system upgrade 1 Upgrade the Solaris Operating System To take advantage of full Fabric support for your SAN you must upgrade to at least Solaris 8 10 01 upgrade 6 For information on how to upgrade your systems refer to Solaris 8 Installation Supplement part number 806 5182 available at http docs sun com Sun StorEdge SA

Sun StorEdge™ Network FC Switch-8 and Switch

Contents

Download Pdf Manuals

Related Search

Related Contents

Sun StorEdge&trade; Network FC Switch-8 and Switch

Contents

Download Pdf Manuals

Related Search

Related Contents

Sun StorEdge™ Network FC Switch-8 and Switch