FC60 Disk Array Service Manual
Contents
1. Code Description Comments Cause Solution 0x91 Boe CCCs Intermodule error call to flash Firmware file is bad Download Firmware function passed bad flash type with a good file Attempt to download Firmware again If the problem persists replace the controller module 0x92 Invalid address in flash device Firmware file is bad Download Firmware with a good file Attempt to download Firmware 0x93 sc cose Length too long for flash device again If the problem persists replace the controller module 0x95 03 Dese Unable to fully erase boot flash Controller module segments reseat or replace 0x96 5a Ceed Unable to fully erase file flash segments 0x97 ecce Cesa Unable to program boot flash after 3 cycles of erase and write 0x98 o ess Unable to program application file flash after 3 cycles of erase and write OxA1 ced Coe Start Boot Firmware Boot Funct Status OxA2 063 Coed Initialize core hardware OxA3 eed Cose Determine hardware configuration OxA4 eced Oecd Determine CDC memory size OxA5 063 Dese Determine RAID Parity Assist memory size OxA6 063 Ceed Initialize BIOS timer OxA7 eced Cees Initialize serial interface OxA8 eed 6c General software initialization OxA9 ened 6000 Display Boot banner OxAA eed 0003 Test Boot Menu invocation steady OxAB 8003 sa Test Boot Menu memory Boot Funct Status Memory problem reseat or replace Controller Enclosure Troubleshooting 523 Buljooysajqnoly2. 0x01 0000 ooa 0x03 Good Dosa 0x07 OOd Deeei Ox0F COC eee Ox1F 0006 seed Controller Enclosure Troubleshooting 511 Buljyooysajqnoly 0x3F Cose seed 0x7F eee cece OxFF 0000 0000 Ox7F 0006 006 Ox3F 0066 seed 0x1F Coos eee Ox0F 0000 peee 0x07 OOd peee 0x03 CoG Dossi 0x01 0000 Doa 0x03 etc O Hose LED patterns are also displayed when the controller programs Flash EEPROM with downloaded code See Table 64 Different patterns are displayed when the download occurs These LED patterns are also displayed during automatic code synchronization Note Note that a indicates the LED is On or Flashing Table 64 EEPROM LEDs Description LED Pattern In between programming steps 0x80 sco ed Erasing boot flash segments 0x81 s Ce Writing boot flash segments 0x82 e coe Verifying erase of boot segments 0x83 eccq Cose Verifying write of boot segments 0x84 eca pec Erasing file flash segments 0x89 eqn esce Writing file flash segments 0x8A secca ced 512 Controller Enclosure Troubleshooting Verifying erase of file flash segments 0x8B eco aces Verifying write of file flash segments Ox8C gang een Controller Start Of Day Process During a reset the controller performs a complete internal selftest sequence known as
3. Controller Enclosure Troubleshooting Connector damage Table 67 Controller Enclosure Troubleshooting Flowchart Sheet 5 of 5 Zon Go to N E A ae 4 Yes Has the new BBU fully charged in 7 hours Is the Fault A or Fault B BBU LED on The BBU has failed Replace the BBU Yes No BBU Full Charge A Has the BBU Wait 7 hours AND No been charging for No for BBU to fully No Full Charge B 7 hours charge EDs are on Yes SO LI CI if pee 1 Faulty Battery charging circuit or harness a Replace both Power Supplies b Replace the Battery Harness Y 2 DOA BBU Go to 3 Midplane A Controller Enclosure Troubleshooting 535 Burooys jqnoIL Table 68 Controller Troubleshooting Symptom Controller LED front cover is ON and the fan LED is off Possible Cause A Controller missing or unplugged Procedure Check the Power LEDs on both controller modules If one Power LED is off make sure that the module is plugged in correctly and its handles are locked in place B Controller failed C One or more memory modules failed If the Fault LED remains ON after replacing the Controller go to cause C Replace the memory modules If the Fault LED remains ON after replacing the memory go to cause D D Controller enclosure midplane failed Replace the Midplane If the Faul
4. OxE6 eeed Feed Controller Enclosure Troubleshooting 517 Buljyooysajqnoly Table 65 Firmware Download LEDs cont d Kernel missing or kernel CRC mismatch The user must download a OxE7 0003 Coes matching combination of controller Bootware and controller Application Firmware Application Firmware missing or Application CRC mismatch The user 0xE8 eed 6033 must download a version of controller Application Firmware that matches the currently loaded controller Bootware Controller Fault LED The Controller Fault LED indicates a number of conditions Power On When power is first applied to the board the Controller Fault LED is turned ON by the controller hardware This is a temporary state and the LED will turn OFF after the controller has powered up If there is an error condition in the controller the LED will remain On Firmware Download When the downloadable firmware is initialized it takes control of the Controller Fault LED and continues to drive the LED on The LED will remain ON not blinking until the board completes diagnostics with no fatal errors If a fatal error occurs the Controller Fault LED will remain on It will take approximately 6 seconds to complete level 0 diagnostics Controller Failure A timeout timer watchdog timer also turns ON the Controller Fault LED if it does not get serviced periodically by the firmware indicating that there has been a cata
5. Table 66 Controller Status LEDs codes cont d Cause Solution Controller module reseat or replace Memory problem reseat or replace Controller module reseat or replace Code Description Comments OxAC ced 6603 Load Boot Menu Boot Funct Status OxAD eced 0600 Invoke Boot Menu OxAE eced weed Exit Boot Menu OxAF eced sees Test Diagnostics Manager memory OxB0 cee OOO Load Diagnostics Manager 0xB1 aces oa Invoke Diagnostics Manager 0xB2 aces 500d Clear work memory 0xB3 000 Ses Clear extended memory 0xB4 aces 6000 Kernel set up 0xB5 Bose Cece Enable Level 2 cache OxB6 eces Coed Load kernel 0xB7 eces Cece Load network manager 0xB8 ces 600d Load Application code symbol table 0xB9 eces 6008 Invoke kernel OxBE 000 0000 Unexpected return from kernel OxBF 500 0006 Rebooting after Boot Firmware download OxE3 5000 Cose No host channel devices on the HW Init status PCI bus PTT REBT devices onthe PCI BUS OxE5 5000 Hace No RPA device on the PCI bus 524 Controller Enclosure Troubleshooting Controller module reseat or replace Memory problem Controller module reseat or replace Table 66 Controller Status LEDs codes cont d Code Description Comments Cause Solution OxE6 eeed 0003 Software load failur
6. Controller Enclosure Troubleshooting Introduction This chapter discusses how to identify interface problems how to identify a controller failure and how to service the controller modules and the memory modules within the controller enclosure as shown in Figure 92 For troubleshooting procedures refer to the Master Troubleshooting Table on page 565 Controller Slot A Controller Slot B Figure 92 Controller Modules in Controller Enclosure Controller Enclosure Troubleshooting 505 Buljyooysajqnoly Controller Enclosure LEDs Figure 93 shows the locations of the status LEDs for the controller enclosure Table 61 summarizes the operating LED states for all components within the controller enclosure Power On LED Power Fault LED Fan Fault LED Controller Fault LED Fast Write Cache LED Controller Power LED Controller Fault LED Heartbeat LED Status LEDs Fault BLED Full Charge B LED Fault ALED Full Charge A LED Power 1 LED Power 2 LED Fan Power LED Q Fan Fault LED A B C D E F G H l J K L M N O P Figure 93 Controller Enclosure LEDs 506 Controller Enclosure Troubleshooting Table 61 Normal LED Status for Controller Enclosure Module LED Normal State Controller Power On On green Enclosure Power Fault
7. 0x80 0x00 No Possible Overheating or Midplane Problem Check 1 Environment 2 Power Supply Fan 3 Controller Fan If OK then replace 1 Midplane 2 Wiring Harness Controller Enclosure Troubl La tano lodule status S L lt OxAB OxAF or n OxAF x 082 7 SRI eshooting Go to Contact the Response Center I No Controller Faul Fixed Yes gt No gt THIS Controller is running diagnostics or has failed POST If the LED status is changing itis running diags If the status is latched THIS Controller ha failed See the status LED section of the service manual for more informati on the exact LED meanings Otherwise 1 Reseat Controller 2 Replace Controller 3 Replace Memory 4 Repalce Midplane 5 Replace Wiring Harness Table 67 Controller Enclosure Troubleshooting Flowchart Sheet 3 of 5 S Ce i Pe n A _ Is the FRONT lt COVER Fan Fault gt _No J Ye Fan Subsystem FAULT Pa S Is a Bad Power Supply Fan lt Power Supply Fan gt Yes module Replace the Fault LED si a A bd No v Reseat the Power Supply fan Module LED on gt power supply fan module Front Cover Fan Fault Controller Fan module Controller Fan module has failed Replace the _ Yes Front Cov
8. Channel 2 53C810 0x02 600d Bosa diagnostics running 0x6B peed ose Drive SCSI Channel 3 53C810 0x03 E Bossi diagnostics running 0x6B peed 6008 Drive SCSI Channel 4 53C810 0x04 DOSI bead diagnostics running 0x6B Deed exes Drive SCSI Channel 5 53C810 0x05 BOSS Gece diagnostics running 0x6B Desa 6000 Drive SCSI Channel 1 53C825 0x11 008 GOO diagnostics running 0x6B peed ose Drive SCSI Channel 2 53C825 0x12 6008 Bosa diagnostics running 0x6B peed ose Drive SCSI Channel 3 53C825 0x13 COSE Bossi diagnostics running 0x6B peed 6008 Drive SCSI Channel 4 53C825 0x14 Boos bead diagnostics running 0x6B Deed 60ee Drive SCSI Channel 5 53C825 0x15 BOSS Gece diagnostics running 0x6B Desa 6000 Drive SCSI Channel 1 53C875 0x21 Dosd Docs diagnostics running 0x6B Desa ose Drive SCSI Channel 2 53C875 0x22 Dosd Bosa diagnostics running 0x6B Desa Sosa Drive SCSI Channel 3 53C875 0x23 Ea Bossi diagnostics running 0x6B peed ose Drive SCSI Channel 4 53C875 0x24 O becd diagnostics running 0x6B Deed 60ee Drive SCSI Channel 5 53C875 0x25 Eo Gece diagnostics running Comments HW Diag status Cause Solution Controller module reseat or replace Controller Enclosure Troubleshooting 521 Buljooysajqnoly Table 66 Controller Status LEDs codes cont d Code Description Comments Ox6C Desa 660
9. Good 0x20 Dosc 60070 0x40 Geog pood 0x80 8003 food x01 etc O00 Bacal 510 Controller Enclosure Troubleshooting Note These status codes are displayed only momentarily and are typically not visible if the controller is operating normally They are only visible if the firmware hangs while executing these functions Kernel Initialization After the hardware is initialized and the Boot Menu has been given a chance to be invoked the controller will initialize the kernel as the final part of the Boot Firmware initialization Table 66 lists the status codes that are displayed Note These status codes are displayed only momentarily and are typically not visible if the controller is operating normally They are only visible if the firmware hangs while executing these functions Table 66 lists the LED patterns while diagnostics are run or when diagnostic failures occur Firmware Download Flash Programming Patterns The controller displays specific LED patterns during firmware download If firmware is being downloaded to the controller the LEDs are all turned ON one by one until all LEDs are on They are then all turned OFF one by one The speed at which the LEDs are turned ON or OFF depends on the rate at which code is being downloaded The LED pattern displayed is shown in Table 63 Note Note that a indicates the LED is On or Flashing Table 63 Firmware Download LED Patterns
10. all logical units The controller will set the Power up Reset Unit Attention for all logical units and the first non inquiry host command to the logical unit will be returned with a Check Condition along with the Controller Enclosure Troubleshooting 513 Buljooysajqnoly 10 11 12 13 14 15 514 sense data indicating that a power on or reset has occurred The Sense Key ASC and ASCQ will be 0x06 0x29 and 0x00 respectively Mode Select Commands disabled The controller will now disable Mode Select Commands until the Start Of Day process is complete This is required so that subsystem configuration changes are prohibited until the controller has finished booting Mode Select commands issued by a host during this time period will be returned with an error by the controller indicating that the addressed logical unit is currently not ready and is in the process of becoming ready Host Side selection enabled The controller will now enable selection on the host side allowing host to select the controller On a bus reset this step is completed within 250ms of receiving the bus reset On power up this step is completed within 10 seconds Respond to host s Inquiry commands The controller is now able to respond to host Inquiry commands based on the knowledge of previously existing logical units as well as their ownership information An Inquiry issued by a host to a logical unit will return information indicating w
11. 3 Drive SCSI Channel 1 i HW Diag status 0x01 600d Goo Turnaround diagnostics running Ox6C peed 6603 Drive SCSI Channel 2 0x02 600d Bosa Turnaround diagnostics running Ox6C Deed 8603 Drive SCSI Channel 3 0x03 Good Does Turnaround diagnostics running Ox6C peed 0003 Drive SCSI Channel 4 0x04 DOSI bead Turnaround diagnostics running Ox6C peed 6603 Drive SCSI Channel 5 0x05 6000 Gece Turnaround diagnostics running Ox6E Desa seed Passive controller Normal during OxEE Bosa seed state after power ON in this case 0x80 8053 cond In between programming steps 0x80 o oa Active controller Normal State 0x00 Eo posa 0x81 o Hcoe Erasing boot flash segments 0x82 809 o Writing boot flash segments 0x83 sco Cose Verifying erase of boot segments 0x84 o 0603 Verifying write of boot segments 0x89 o se Erasing file flash segments Ox8A BOSA 8069 Writing file flash segments 0x8B 8033 a Verifying erase of file flash segments 0x8C 8033 sec Verifying write of file flash segments 522 Controller Enclosure Troubleshooting Cause Solution Controller module reseat or replace Firmware download The controller may be in this use the AM60 software to set to Active state Firmware file is bad Download Firmware with a good file Firmware file is downloading or Firmware file is bad Download Firmware with a good file Table 66 Controller Status LEDs codes cont d
12. Off Fan Fault Off Controller Fault Off Fast Write Cache On green while data is in cache Controller Controller Power On green Controller Fault Off Heartbeat Blink green Status Green There are 8 status LEDs The number and pattern of these LEDs depend on how your system is configured Controller Fault B Off Battery Full Charge B Gn aren Fault A Off Full Charge A On green Controller Power 1 On green pale Power 2 On green Controller Fan Power On green hasan Fan Fault Off 1 Both Full Charge A and Full Charge B LEDs are ON after batteries are fully charged The LEDs flash while charging is in progress and remain on when charging is complete Controller Enclosure Troubleshooting 507 Buljooysajqnoly Controller Status LEDs A bank of eight status LEDs plus a Fault LED and a Power LED on the controller module display status information Each controller module displays only its own status and fault information it does not display information about the other controller module if it is installed Figure 94 shows the location of the controller status LEDs Heartbeat Controller me C Figure 94 Controller Module Status LEDs Normal Active Active LED Patterns The Heartbeat LED is in the most significant position of the 8 status LEDs The pattern displayed on the Status LED bank for an active controller alternates between 0x00 anA to0x80 B Boca A passive controller s status LED
13. Start Of Day The following is an overview of the processes that occur during a Start Of Day 1 Hardware diagnostics This process performs diagnostics on specific hardware components and is only performed during a power on reset If there is a critical component failure when running diagnostics the controller firmware will halt diagnostics execution and flash an error code on the LEDs indicating the component that failed 2 Bootware loaded The controller s boot firmware is loaded from Flash memory into Processor memory 3 Application firmware loaded The controller application firmware is loaded from Flash memory into processor memory 4 Ethernet and Fibre Channel If the controller hardware supports the integrated Ethernet controller and or the Fibre Channel host interface the firmware components to support these interfaces are loaded The Ethernet capability must specifically be enabled via an option in the User configurable region of NVSRAM offset 0x28 bit 3 5 Controller Heartbeat The controller heartbeat LED will begin to turn on and off The LED pattern displayed will be 0x00 0x80 0x00 0x80 0x00 etc The absence of the heartbeat LED is an indication that a fatal error has occurred 6 Controller firmware components to handle host operations The controller will initialize the host operations to allow the controller to receive and handle operations issued by a host system 7 Power up Reset Unit Attention for
14. e Firmware file is bad 5 Download Firmware 0x66 RPA parity error i peed psec ae with a good file OxE7 eeed Cose Kernel missing or kernel CRC mismatch OxE8 ell eee Application Firmware missing or Application CRC mismatch OxEA eeed 6003 Swapped controller SETS is in wrong OxF 1 Dese CSA Processor set up HW Init status Controller module reseat or replace OxF2 eos Coed CDC chip set up OxF3 bee Kose RAID Parity Assist chip set up OxF4 5590 0000 Memory Test OxF5 bose Cece Load Boot Firmware OxF7 5500 Cose Clear Boot memory OxF8 bose 0000 Enable Protected mode OxF9 Dese 6008 Memory test failure OxFA Dese 6003 Exception error OxFB Bess 5500 Start mode determination OxFC bose 6000 Invoke Boot Firmware OxFF Dese cece Hardware reset Hardware reset is in progress or there is a bad controller module Controller Enclosure Troubleshooting 525 Buljooysajqnoly Identifying Interface Problems Types of Interface Problems Interface problems include any malfunctions that delay interrupt or prevent successful input output I O activity between the hosts and other devices This includes transmissions between the controller enclosure and disk enclosures attached to it For the purpose of this discussion the controller enclosure s interface components include the following e Internal components Two Fibre Channel controller modules Controller enclosure card cage includes midplane w SCSI connectors e Exter
15. e least significant byte The displays cycle approximately once every 2 seconds If a fatal diagnostic error occurs the controller will not boot and the LEDs will continue to display a code that gives an indication of the failed component See Table 66 on page 519 for these codes Controller Enclosure Troubleshooting 509 Buljyooysajqnoly Hardware Initialization Hardware initialization codes are displayed on the LED bank as the hardware is initialized during a power up or reset sequence If the controller encounters errors during hardware initialization error patterns are displayed on the LEDs These error patterns are listed in Table 66 on page 519 Note These status codes are displayed only momentarily and are typically not visible if the controller is operating normally They are only visible if the firmware hangs while executing these functions Boot Menu Execution Prior to the Boot Menu actually executing the controller will display a number of status codes indicating various functions being executed see Table 66 on page 519 Once the Boot Menu is invoked the controller will cycle through all of the LEDs one by one and the LED pattern displayed is shown in Table 62 Note Note that a indicates the LED is ON or Flashing Table 62 LEDs Prior to Boot Menu 0x01 Oo Gade 0x02 Ooa Doea 0x04 6000 Deca 0x08 Cog loca 0x10 0006
16. entify the function that is working incorrectly If the problem occurred without an apparent software related activity check the operating system and storage management software for error messages and associated procedures This may help determine if it is a software or hardware problem Check the controller modules for faults Check all the interface cables particularly the host Fibre Channel cables to make sure that they are securely connected and undamaged If you moved the controller enclosure to another host or attached new devices to it check the following Loop ID settings for both controller modules Make sure these settings are unique and do not conflict with other devices Change the settings if necessary SCSI ID settings on all attached disk enclosure BCC modules Change the settings as necessary Make sure all switch settings are set the same for both BCCs in each disk enclosure Interface cable connections Make sure that all cables are routed correctly Change the cable connections as necessary For cable connection information for other devices refer to applicable hardware manuals Make sure that the Split Full bus switch is set on the disk enclosure for the desired configuration Problems resulting from a defective host adapter board controller module memory module or controller enclosure midplane may be difficult to detect If checking all the items listed above does not identify the problem try Rep
17. er Fan Fault J Circuitry failed or power LED burned out on Power Supply fan Module Replace Power Supply Fan Module Yes Front Cover Fan Fault Yes Contact the Response Center Go to Controller Enclosure Troubleshooting 533 Buljooysajqnoly Table 67 Controller Enclosure Troubleshooting Flowchart Sheet 4 of 5 534 Is the FRONT COVER Yes Power Supply Fault LED No on Inspect both power supply module powe LEDs Is the Yes FRONT COVER No power supply Fault LED on x 7 NS we N e Power Supply AN N No x power LED on o a Ca yy n gl y cat I Possible Causes be Go to 1 Power Switch on this supply is OFF Pi DI Yes 2 No input power to supply Sonat gt Yes A 3 Power Supply Module has failed Pa ON a x A gt N Pai N Yes yee a lt Power Supply B 5 No Po 4 power LED on Yes Check that each power supply is well seated in the midplane by firmly pushing the supply into its slot Do not pull a power supply out of the array enclosure wjth a Front Panel Power FAULT condition unless you know the remaining supply in the array is functioning ba e 2 No Possible causes 1 Possible Midplane 2 Power supply to Midplane CAUTION If a supply is unseated and its the only remaining functioning supply the array enclosure will abruptly power down
18. erns are displayed after downloading firmware The patterns are shown in Table 65 Note Note that a indicates the LED is On or Flashing Table 65 Firmware Download LEDs write Replace the controller board Error LED Pattern Download file is lacking header record 0x61 Desa oe Download file header fails checksum test 0x64 esa bead Download file header contains unexpected download type 0x65 ceso Dese The user should attempt to download a known good file 0x67 Desa cess Intermodule error call to flash function passed bad flash type There is 0x91 o ce a mismatch between the Boot Firmware and the Application Firmware The user must download matched versions of both controller Bootware and controller Application Firmware Invalid address in flash device 0x92 cos ooo Unable to fully erase boot flash segments bad voltage or device 0x95 38 Cede Replace the controller board Unable to fully erase file flash segments bad voltage or device 0x96 Soa Desa Replace the controller board Unable to program boot flash after 3 cycles of erase and write 0x97 scoa Dese Replace the controller board Unable to program application file flash after 3 cycles of erase and 0x98 eae e Software load failure There is a mismatch between the Boot Firmware and the Application Firmware The user must download matched versions of both code types
19. hether the logical unit exists and if it is owned by this controller If the controller receives a non Inquiry command it delays the command a period of time determined by the value set in NVSRAM before returning a Not Ready logical unit is in process of coming ready error Drive operations initialized The controller will now initialize firmware components to handle drive operations allowing the controller to issue commands to the drives LUN ownership read The controller will read LUN ownership from the NVSRAM Loaded firmware modules displayed The controller will display a list of loaded firmware modules on the serial port Spin up of drives The controller reads NVSRAM information for previously existing drives and spins up the drives following the Drive Spin up algorithm as defined in NVSRAM If the Drive Spin up algorithm defined in the NVSRAM requires the controller to wait for a Start Unit command from the host before spinning up the drives the Start Of Day process will be suspended at this time until the host issues a Start Unit command Read drive and LUN configuration information from dacStore The controller will now read the drive and LUN configuration information from the dacStore on the drives Controller Enclosure Troubleshooting 16 17 18 19 20 21 22 23 and the appropriate structures are created in memory If these logical units had dirty data in cache the controller wil
20. l flush the data to the drive media If the controller believes that it s Battery Backup Unit has failed it will attempt to recover any mirrored data from the alternate controller Hosts are now allowed access to these previously existing logical units Non Inquiry commands will no longer fail with a Not Ready error The controller will attempt to spin up all remaining drives in the subsystem If new drives are discovered their configuration information is read and any LUNs on these drives are brought on line If the controller finds LUN information on these new drives for a LUN that already exists in the array subsystem then the LUN number will be changed to the next available LUN number Please refer to the LUN Migration document for additional information regarding moving drives Read array subsystem configuration The controller will read the array subsystem configuration from the dacStore on the drives to confirm the correct mode of redundant controller operation Drive and controller hot swap enable The controller now initializes the firmware components to handle drive and controller hot swaps Subsystem component polling The controller now initializes subsystem component polling Restart interrupted drive re constructions The controller will restart any drive re construction processes that were interrupted due to power fail or resets With the current release of controller FW the reconstruction process will be restarted fr
21. lacing the host adapter and appropriate interface cable to each host Controller Enclosure Troubleshooting Buljooysajqnoly e Replacing the disk array controller module including memory e Replacing the controller enclosure midplane controller card cage Controller Servicing Notes Here are a few suggestions to consider when servicing disk array controller modules e Always use proper precautions against electrostatic discharge when removing and handling disk array components e Always read pertinent documentation This includes software instructions on replacing failed interface components and documentation shipped with the replacement FRUs particularly the kit instructions Kit instructions often contain the most current information regarding servicing e Always stop all I O activity to the controller and associated disk modules before replacing the suspect or failed component unless alternate LVM links are properly configured e Memory modules controller modules and the controller enclosure card cage assembly which includes the midplane are not user replaceable These components must be serviced by a qualified trained service technician only e A failed controller module can be hot swapped if the failed controller Is one of a redundant pair e If cache mirroring is enabled and one controller module fails the remaining controller module will assume operation of the disk array but write cache will be disabled e Remo
22. lt periencing intermitent 14 Ly Seran ff or interface gt No End ONENE 2 problems S 2 E a 4 Check the Host for errors using STM AM60 and Yes syslog log Intermittnet problems can be cause by the following 1 Missing patches Install latest driver patches 2 Firmware level Install latest Firmware levels 3 Unsupported configurations 4 Host adapter 5 FC Cables 6 Controller Module Controller Enclosure Troubleshooting Go to 531 Buljooysajqnoly Table 67 Controller Enclosure Troubleshooting Flowchart Sheet 2 of 5 532 Is the FRONT PANEL Controller Fault LED on Controller Subsystem FAULT Open door and look at each Controller modi for Power LED Fault LED 8 Status LEDs Yes LC Is lt Controller A or B power LED off Pa A Pane Is ate 1 Reseat Controller A Controller AorB gt Yes gt 7 Replace Controller v fault LED E THIS Controller is unseated or has failed 1 Reseat Controller Yes 2 Replace Controller 3 Replace Midplane 4 Replace Memory 5 Replace Wiring Harness THIS Controller has failed POST diagnostics 3 Replace Midplane 4 Replace Wiring Harness Memory failure No 1 Replace memory on THIS Controller Module 2 Replace Controller Module 3 Replace Memory and Controller Is Controller A or B status LEDs other tha
23. nal components Fibre Channel host adapters cables and hub or switching devices SCSI cables terminators BCC modules in the disk enclosure Interface problems can be caused by either software or hardware e Software problems which indicate operating system or disk array application errors typically involve one or more of the following Host operating system software error Disk array or other application error Incorrect configuration settings e Hardware problems which indicate defective equipment include the following Loose disconnected or damaged interface cables or connectors Improper SCSI termination on disk enclosure bus or defective terminators Improper interface ID settings hardware switches Failed controller modules memory modules or controller enclosure midplane Failed disk modules host adapter boards or other devices on the Fibre Channel network Failed disk modules 526 Controller Enclosure Troubleshooting Hints for Troubleshooting Interface Problems The first step in troubleshooting interface problems is determining whether the problem is caused by hardware or software The following information should aid in making this determination If the problem occurred during or immediately following a software activity try to undo whatever the software did then step through each software function in smaller increments until the problem occurs again This will id
24. om the beginning of the appropriate LUN The controller has no knowledge of how much of the reconstruction process was completed prior to the reset or power fail Copy Back initiated If a Global Hot Spare GHS drive has been sparing for a Failed drive and the originally Failed drive has been replaced the controller will now initiate a copy back from the GHS to the replaced drive Discover new GHS drives If the controller discovers a new GHS has been inserted into the subsystem it will start using the new GHS drive should a Failed drive be found Discover missing drives The controller will display a list of drives that had LUN configuration on them but were cold removed during the last power down Controller Enclosure Troubleshooting Buljooysajqnoly 24 Spin down Failed Drives Drives that are marked Failed are now spun down and their Fault LEDs are lighted 25 Restart LUN binding The controller will discover and restart LUNs that were in the process of being bound when the reset or power fail occurred 26 LUN 0 created If no LUNs are discovered a default LUN is created 27 Enable Mode Select commands The controller will now enable Mode Select commands allowing users to make configuration changes such as adding or deleting LUNs changing controller modes or drive states 28 Controller Start Of Day completed 516 Controller Enclosure Troubleshooting Errors During After Firmware Download Different patt
25. roller Enclosure Troubleshooting 529 Buljooysajqnoly Controller Enclosure Troubleshooting Introduction This section describes procedures to troubleshoot the controller enclosure See Figure 95 For troubleshooting procedures refer to Table 68 or the Master Troubleshooting Table on page 565 Controller Slot A Controller Slot B Heartbeat Status Figure 95 Controller FRU Slots and LEDs 530 Controller Enclosure Troubleshooting Table 67 Controller Enclosure Troubleshooting Flowchart Sheet 1 of 5 Look at the Array Controller s Power LED on FRONT PANEL BS LEDs Array Enclosure Power Problem Check for 7 v No Power Cords Power is being Power Switch applied to the Array PDU Enclosure AIC Breaker BOTH Power Supplies are bad BOTH Power Supplies overheated Thermal Shutdown 8 Midplane is bad 9 Wiring harness is bad NOGAWNA Controller Fan module is unplugged BBU may be faulted or charging proceed to E No Data in Write Cache Fast Write Indicates one of the Cache LED on or No following conditions _ _ gt flashing 1 No I O 2 Cache disabled Yes 4 Data is in Write Cache Go to SL LEE Fi Normal Array ar KERONEN z tA Enclosure LED NOT on besides Power or tatus N Cache e ache Se J Yes i 7 e Fault
26. s will alternate between 0x6E Deed peed to OxEE feed seec 508 Controller Enclosure Troubleshooting Errors During Normal Operations It is possible for the controller to encounter errors from which it is unable to recover during normal operations Specific status LED patterns are used to give a visual indication of any problem The error pattern returns to the normal setting after the problem has been fixed These error patterns are listed in Table 66 on page 519 If the controller encounters an unexpected processor exception the error pattern of 0xE1 eed 6008 will briefly be displayed before the controller reboots itself Note Note that a indicates the LED is ON or Flashing If the controller encounters a PCI bus fault while running the error pattern of 0xAA will be displayed solid ON the LED bank oedecec Controller Hardware Reset Pattern Whenever the controller is held in a hardware reset state all LEDs are on Therefore the pattern displayed by the LEDs is 0xFF eee eee Hardware Diagnostics Errors Hardware diagnostics are run during power ON or when a controller module is reset As diagnostics are run the LEDs display information indicating which diagnostics are running The patterns displayed by the LEDs correspond to the Component Code of the component being tested Note that the codes are 16 bit and the LEDs display the most significant byte first followed by th
27. strophic controller failure If this error happens contact a trained service representative to upload new firmware 518 Controller Enclosure Troubleshooting Controller Status LED Codes Summary Table 66 lists a summary of the various controller Status LED codes Note Note that a indicates the LED is On or Flashing Table 66 Controller Status LEDs codes Code Description Comments Cause Solution 0x20 Coed 6000 Kernel Start Kernel Init status Controller module 0x21 Cosa Kernel cache initialization reseat or replace 0x22 Coed Coed Kernel IDT set up 0x23 Coed Dose Kernel hardware initialization 0x24 Coed Dec Kernel initialization 0x25 Cog cec Kernel Task Manager start up 0x26 Losa beed Kernel memory initialization 0x27 Cq Gees Kernel clock start 0x28 Cog ec Kernel service initialization 0x29 Cog 006 Kernel symbol table initialization Ox2A Coq 0ed Kernel network initialization 0x2B Cog ecse Kernel flash EEPROM file system initialization 0x2C Coq 600 Kernel NMI enabled 0x2D Cos eos Kernel page management initialization Ox2E Cos ee Kernel shell initialization Ox2F Coeg wees Kernel application load 0x30 Does 000 Kernel application start 0x33 Bose Cose Non volatile Memory 8 Kbytes HW Diag status 0x00 Boog Good diagnostics running Controller Enclosure Troubleshoo
28. t LED remains ON after replacing the midplane call the factory service center Software issued a Controller error message A Controller failed Check the Fan LED on the front cover If itis on go to Troubleshooting Controller Fan Module Problems on page 547 If not continue at the next step Replace the failed Controller Controller enclosure and Fan LED front cover are on A Controller enclosure fan failure caused one or both Controller s to overheat 536 Controller Enclosure Troubleshooting Stop all activity to the Controller module and turn OFF the power Replace the failed Controller enclosure fan module Allow the Controller to cool down then turn ON the power Check both Controllers for fault LEDs If a Controller Fault LED turns on replace the failed Controller
29. ting 519 Buljyooysajqnoly Table 66 Controller Status LEDs codes cont d Code Description 0x36 Processor DRAM diagnostics 0x00 running 0x37 RPA DRAM diagnostics running 0x00 0x38 Processor Level 2 Cache 0x00 diagnostics running 0x44 CDC diagnostics running 0x01 0x44 SIO diagnostics running 0x02 0x44 RPA diagnostics running 0x03 0x44 SIO Interrupt diagnostics running 0x12 0x54 ICON diagnostics running 0x00 0x55 FReD diagnostics running 0x00 0x61 Download file is lacking header record 0x64 Download file header fails checksum test 0x65 Download file header contains unexpected download type 0x65 Host SCSI Channel 53C825 0x00 diagnostics running 0x65 Host SCSI Channel 53C875 0x10 diagnostics running 0x65 Host Fibre Channel Interface 0x20 diagnostics running 0x67 CRC failure on downloaded data 520 Controller Enclosure Troubleshooting Comments HW Diag status Cause Solution Controller module reseat or replace Firmware file is bad Download Firmware with a good file Y Controller module reseat or replace Table 66 Controller Status LEDs codes cont d Code Description 0x6B Desa 6000 Drive SCSI Channel 1 53C810 0x01 Dod Goo diagnostics running 0x6B Desa Sosa Drive SC SI
30. ve the front cover to service the controller modules or to view the LEDs on each module e Make sure that the new controller module has the same amount of memory as the one you are replacing Note Do not swap a controller when the power is off e SCSI interface cables are not hot swappable 528 Controller Enclosure Troubleshooting A controller fault may be due to a failed memory module Memory Module Servicing Notes CAUTION Memory modules must be serviced by a trained service technician ONLY Before replacing a failed SIMM or DIMM remember the following tips Always use proper precautions against electrostatic discharge before removing and handling controller modules SIMMs and DIMMs Always use the same type and size of memory module to replace a failed module The Fault LED on the affected controller module will turn ON if one of its memory modules fail This is the same LED used to indicate a controller failure The controllers do not contain an LED or other mechanism for identifying individual memory module failures The following information should help you determine if the problem is memory or controller related Remove the failed controller module and replace a failed memory module while the other controller module is running If cache mirroring is enabled and one controller module fails the remaining controller module will assume operation of the disk array but write cache will be disabled Cont
Download Pdf Manuals
Related Search