Home

SPARC T5-8 Server Service Manual

1. lt gt 100 10GbE No Port Label Links 1 Network management NET MGT This Ethernet port enables you to connect the server to your local network so that you can manage the server from a remote location 2 Serial management SER MGT This serial port enables you to connect directly to the SP 3 Ethernet network NETx Where x is the number of the port You can use these Ethernet ports to connect the server to local or wide area networks 4 USB These ports provide access for input devices 5 Video This port provides access to the console output on the SP Related Information m Servicing the Rear I O Module m Chassis Subassembly Components on page 22 Identifying Components 21 Chassis Subassembly Components Chassis Subassembly Components Description Links Ur W N e o oa Hard drives 8 Front I O assembly Main module System controls and indicators Processor modules 4 fully populated or processor filler modules slot 1 and slot 2 Chassis Rear chassis subassembly Fan modules 4 PCle carriers 16 Rear I O module Power supplies 4 Servicing Hard Drives Servicing the Front I O Assembly Servicing the Main Module Front Panel Components Service on page 14 Servicing Processor Modules Servicing the Rear Chassis Subassembly Servicing the Rear Chassis Subassemb
2. none TABLE 4 HOST diag hw_change_level Value Description max Runs the maximum set of tests after a hardware change min Runs the minimum set of tests after a hardware change TABLE 5 HOST diag hw_change_verbosity Value Description min default Displays the minimum level of output during the hardware change tests max Displays information for each step normal Displays a moderate amount of information including component names and test results SPARC T5 8 Server Service Manual e November 2015 Configuring POST Value Description debug Displays extensive debugging information none Disables the output TABLE 6 HOST diag power on level Value Description max default Runs the maximum set of tests min Runs the minimum set of tests TABLE 7 HOST diag power on verbosity Value Description min default Displays the minimum level of output max Displays information for each step normal Displays a moderate amount of information including component names and test results debug Displays extensive debugging information none Disables the output TABLE 8 HOST diag error reset level Value Description max default Runs the maximum set of tests min Runs a minimum set of tests TABLE 9 HOST diag error reset verbosity Value Description min default Displays the minimum level of output max Displa
3. ie 91 y Install the Main Module i 95 Y Verify the Main Module iii 97 Servicing the Storage Backplanes erre 99 Y Remove a Storage Backplane ie 99 Y Install a Storage Backplane e 103 Y Verify the Storage Backplane ice eer err ree acerca rrenan 107 Servicing the Service Processor Card renas 109 Y Determine if the Service Processor Card Is Faulty eres 109 Y Remove the Service Processor Card c i 110 Y Install the Service Processor Card ir rreearererereenna 112 Y Verify the Service Processor Card ei 114 Servicing the System Configuration PROM en 117 Y Remove the System Configuration PROM i 118 Y Install the System Configuration PROM ni rrene 119 Y Verify the System Configuration PROM ii rreeee 120 Servicing the Battery e 123 Y Remove the Battery cccsccseccsccsccescceeesceescensseesscsseeseeesceeseseeeneenses 123 V Installithe Battery secas iaia aio 125 Contents 8 Y Verify the Battery iu oe Shed edad errare eni Peba ARE 126 Servicing the Front I O Assembly erre reereeeeeaa 127 Y Remove the Front I O Assembly 127 Y Install the Front I O Ass
4. f you have another person to assist you go to Step 8 m If you are alone go to Step 7 Servicing the Rear Chassis Subassembly 183 Remove the Rear Chassis Subassembly 7 Place a mechanical lift under the chassis and remove the screws that fasten it to the rack 8 Remove the chassis from the rack and place it on an appropriate surface 184 SPARC T5 8 Server Service Manual e November 2015 Install the Rear Chassis Subassembly 9 Using a Phillips screwdriver loosen the five screws that secure the rear chassis subassembly to the server chassis and then slide the rear chassis subassembly out and away from the chassis 10 Slide the rear chassis subassembly out and away from the chassis Related Information Determine if the Rear Chassis Subassembly Is Faulty on page 181 Install the Rear Chassis Subassembly on page 185 Y Install the Rear Chassis Subassembly 1 Ifyou have not already done so remove the faulty rear chassis subassembly See Remove the Rear Chassis Subassembly on page 182 Servicing the Rear Chassis Subassembly 185 Install the Rear Chassis Subassembly 2 Slide the new rear chassis subassembly into the chassis and then using a Phillips screwdriver tighten the five green screws to secure the rear chassis subassembly in the server 3 Determine your next step m f you have another person to assist you go to Step 5 If you are alone go to Step 4
5. Determine Which PCle Card Is Faulty on page 155 Determine if the Rear I O Module Is Faulty on page 173 Related Information Understanding Diagnostics on page 25 Managing Faults on page 41 30 SPARC T5 8 Server Service Manual e November 2015 Interpreting LEDs Front Panel Controls and LEDs POWER 6 No LED Icon or Label Description 1 Locator LED and You can turn on the Locator LED to identify a particular server When lit the LED button white blinks rapidly Turn on the Locator LED by pressing the Locator button or see Locate the Server on page 53 2 Service Required The fmadm faulty command provides details about any faults that cause this LED amber indicator to light See Check for Faults on page 41 Under some fault conditions individual component fault LEDs are lit in addition to the Service Required LED 3 Indicates these conditions Power OK LED m Off Server is not running in its normal state Server power might be off The SP might be running m Steady on Server is powered on and is running in its normal operating state No service actions are required m Fast blink Server is running in standby mode and can be quickly returned to full function Detecting and Managing Faults 31 Interpreting LEDs No LED Icon or Label Description m Slow blink A normal but transitory activity is taking place Slow blinking
6. Steps Description Links 1 Determine if the rear I O module is faulty Determine if the Rear I O Module Is Faulty on page 173 2 Replace the rear I O module m Remove the Rear I O Module on page 174 m Install the Rear I O Module on page 176 3 Verify the replacement Verify the Rear I O Module on page 178 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Y Determine if the Rear I O Module Is Faulty 1 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 2 Start the faultmgmt shell gt start SP faultmgmt shell Servicing the Rear I O Module 173 Remove the Rear I O Module Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the main module is operating normally Ifa fault was detected see Diagnostics Process on page 25 If no fault was detected the main module is operating normally Related Information Remove the Rear I O Module on page 174 Install the Rear I O Module on page 176 m Verify the Rear I O Module on page 178 Y Remove the Rear I O Module 174 The rear I O module is a cold service component that can be replaced only after you have powered off the server For the locatio
7. Installing a PCle card carrier while the primary domain is at the Open Boot prompt is not supported 2 Insert the PCle card carrier into the card cage until it stops A Caution Do not press on the PCIe back panel or force the PCIe card carrier into the card cage 3 Close the latch to lock the carrier Servicing PCle Cards 169 Install a Card Carrier Rotate the handle up until it latches into place 4 Reconnect all of the cables and any transceivers if applicable to the PCle card 5 Determine your next step If you replaced or installed a PCIe card in a server that is running if you hot swapped the card go to Step 6 If you replaced or installed a PCIe card in a powered down server power on the server using the instructions provided in Returning the Server to Operation then go to Step 7 6 Use one of the following methods to bring the PCle card online a Press the Attention button The card s Power OK LED should illuminate indicating that the card is online b At the Oracle Solaris prompt power on and enable the PCle card Example hotplug poweron pci 340 pci 1 pci 0 pci 6 PCIE3 hotplug enable pci 340 pci 1 pci 0 pci 6 PCIE3 7 Verify the card s installation See Verify the PCle Card on page 171 170 SPARC T5 8 Server Service Manual e November 2015 Verify the PCle Card Related Information Determine Which PCle Card Is Faulty on page 155 Remove a C
8. on page 70 m Server Upgrade Process on page 59 Y Verify the Processor Module 70 Ensure that you have completed the following m Applied power to the server See Connect the Power Cords on page 191 Started the system See Power On the Server Oracle ILOM on page 192 If you replaced a faulty PM log in to the fmadm shell and use the fmadm faulty command to determine if a fault on the PM is shown gt start SP faultmgmt shell Do you want to start the SP faultmgmt shell y n y faultmgmtsp gt fmadm faulty a If the output shows the replacement PM as enabled go to Step 3 b If the output shows the replacement PM as disabled go to Detecting and Managing Faults to clear the fault from the server Verify that the OK LED is lit on the PM and that the Fault LED is not lit See Processor Module LEDs on page 61 Verify that the front and rear Service Required LEDs are not lit See Front Panel Controls and LEDs on page 31 and Rear Panel Controls and LEDs on page 33 Perform one of the following tasks based on your verification results Ifa fault was detected see Diagnostics Process on page 25 If no fault was detected then the processor module was installed successfully SPARC T5 8 Server Service Manual e November 2015 Servicing DIMMs DIMMs are cold service components that can be replaced after you remove the processor module from the system For the
9. For the location of the PCle card carriers see Rear Panel Components Service on page 15 Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail Take the necessary ESD precautions See Prevent ESD Damage on page 57 Start this procedure with the primary domain at the Oracle Solaris prompt Note Removing a PCle card carrier while the primary domain is at the Open Boot prompt is not supported Locate the PCle card carrier at the rear of the server m See Rear Panel Components Service on page 15 for the locations of the carriers in the server m See Determine Which PCle Card Is Faulty on page 155 to locate a faulty PCle card Determine if you are removing a card carrier from a running server If you are removing a PCIe card from a server that is running that is if you are hot swapping the card go to Step 5 If you are removing a card from a powered down server go to Step 7 Take the card offline You can use the Attention button or the CLI to hot swap the card from the server m Attention button a Press the button to bring the card offline SPARC T5 8 Server Service Manual e November 2015 Remove a Card Carrier The Power OK LED should go off indicating that the card is ready to be removed b Goto Step 7 CLI a At the Oracle Solaris prompt list all of the PCle dev
10. chassis NAC name 23 parts breakdown 22 clearing faults PSH detected faults 44 components accessible from front 14 accessible from rear 15 DIMMs 71 service categories 23 configuration reference DIMMs 71 fan modules 20 hard drives 18 PCle slots 153 configuring how POST runs 39 D diag level parameter 35 diag mode parameter 35 diag trigger parameter 35 diag verbosity parameter 35 DIMMs capacities 71 configuration 71 failures 73 Fault Remind button 75 faulty 66 general error message 76 installing 78 locating faulty using DIMM Fault Remind button 75 usingfmadm faulty command 73 NAC name 23 orientation 79 removing 76 verifying functionality 80 dmesg command 45 E electrostatic discharge see ESD 50 ESD measures 50 preventing using an antistatic mat 50 preventing using an antistatic wrist strap 50 F fan modules configuration reference 20 installing 145 locating 15 locating faulty 142 NAC name 23 201 Index removing 143 verifying functionality 146 fault detection checking for 41 fmadm command 44 fmadm faulty command 44 fmdump command 41 front I O assembly installing 130 locating 17 NAC name 23 removing 127 front panel components 14 front panel system controls and LEDs 31 H hard drives configuration reference 18 installing 85 locating 14 locating faulty 82 NAC name 23 removing 83 verifying functionality 86 hot serv
11. is reversed 2a 4 4 Tp qua CIS Push the DIMM into the connector until the ejector tabs lock the DIMM in place If the DIMM does not easily seat into the connector check the DIMM s orientation Determine your next step If you replaced a faulty DIMM go to Step 9 If you are upgrading the server repeat Step 5 through Step 7 until all of the new DIMMs are installed Then go to Server Upgrade Process on page 59 Install the PM See Install a Processor Module or Processor Filler Module on page 67 Related Information DIMM Configuration on page 71 Remove a DIMM or DIMM Filler Panel on page 76 m Verify the DIMM on page 80 m Server Upgrade Process on page 59 Servicing DIMMs 79 Verify the DIMM Y Verify the DIMM Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 Started the system See Power On the Server Oracle ILOM on page 192 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected the server is not operating normally See Diagnostics Process on page 25 If no fault was detected the DIMM was insta
12. on page 41 Related Information Check for Faults on page 41 Clear a Fault on page 44 Y Check for Faults The fmadm faulty command displays the list of faults detected by PSH You can run this command from either the host or through the Oracle ILOM fault management shell Detecting and Managing Faults 41 Check for Faults Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Check for PSH diagnosed faults This example shows how to check for faults through the Oracle ILOM fault management shell gt start SP faultmgmt shell Are you sure you want to start SP faultmgmt shell y n y faultmgmtsp gt fmadm faulty 2012 08 27 19 46 26 4ec16c8d 5cdb c6ca c949 e24d3637ef27 PCIEX 8000 8R Major Problem Status solved Diag Engine unknown System Manufacturer Oracle Corporation Name SPARC T5 8 Part_Number 12345678 11 1 Serial Number XXXXXXXXXX Suspect 1 of 1 Fault class fault io pciex device interr corr Certainty 100 Affects hc chassis 0 motherboard 0 cpuboard 0 chip 0 hostbridge 0 pciexrc 0 Status faulted but still in service FRU Status faulty Location 1 SYS PMO Manufacturer Oracle Corporation Name TLA PN NRM T5 1 2 Part_Number 7061001 Revision 01 Serial Number 465769T 12445102WR Chassis Manufacturer Oracle Corporation Name SPARC T5 8 Part Number 12345678 13 2 Serial Number XXXXXXXXXX Description A fault has been diagnosed by
13. 186 SPARC T5 8 Server Service Manual e November 2015 Install the Rear Chassis Subassembly 4 Place the chassis on a mechanical lift 5 Install the chassis into the rack and secure the chassis with the four screws that were removed 6 Atthe rear of the server install these components Component Link Rear I O module PCle carriers Fan modules Cables Install the Rear I O Module on page 176 Install a Card Carrier on page 168 Install a Fan Module on page 145 According to their lables 7 Atthe front of the server install these components Servicing the Rear Chassis Subassembly 187 Verify the Rear Chassis Subassembly Components Links Power supplies Install a Power Supply on page 137 Main module Install the Main Module on page 95 Processor modules bottom to top Install a Processor Module or Processor Filler Module on page 67 Cables According to their lables Power on the server See Returning the Server to Operation Verify the installation See Verify the Rear Chassis Subassembly on page 188 Related Information Remove the Rear Chassis Subassembly on page 182 m Verify the Rear Chassis Subassembly on page 188 Y Verify the Rear Chassis Subassembly 188 1 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 m Started the system See Power
14. 4 Install the rear chassis subassembly Install the Rear Chassis Subassembly on page 185 5 Return the server to operation Returning the Server to Operation 6 Verify the installation Verify the Rear Chassis Subassembly on page 188 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Y Determine if the Rear Chassis Subassembly Is Faulty 1 Log in to Oracle ILOM Servicing the Rear Chassis Subassembly 181 Remove the Rear Chassis Subassembly See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the rear chassis subassembly is operating normally Ifa fault was detected see Diagnostics Process on page 25 If no fault was detected the rear chassis subassembly is operating normally Related Information Remove the Rear Chassis Subassembly on page 182 Install the Rear Chassis Subassembly on page 185 Y Remove the Rear Chassis Subassembly 182 The rear chassis subassembly is a cold service component that can be replaced only by qualified service personnel For its location see Rear Panel Components Service on page 15 Cau
15. 5 Verify the replacement Verify the Service Processor Card on page 114 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Y Determine if the Service Processor Card Is Faulty When a fault occurs with the service processor card the following LEDs are lit Servicing the Service Processor Card 109 Remove the Service Processor Card m System Service Required LEDs on the front panel and rear I O module Server SP LED on the main module and rear I O module Determine if the System Service Required LED is lit on the front panel or the rear I O module See Interpreting LEDs on page 29 Check the SP LED on the main module or the rear I O module See Front Panel Controls and LEDs on page 31 or Rear Panel Controls and LEDs on page 33 The SP LED will be lit amber if the service processor card needs to be replaced Remove the faulty service processor card See Remove the Service Processor Card on page 110 Related Information Remove the Service Processor Card on page 110 Install the Service Processor Card on page 112 m Verify the Service Processor Card on page 114 Y Remove the Service Processor Card 110 A The service processor card is a cold service component that can be replaced only after you have
16. Card 4 Insert the PCle card into the carrier until the bottom connector is firmly seated in the carrier s connector A Caution Do not twist or turn the PCle card as you insert it into the carrier The card s connector must be fully seated in the carrier s slot before you attempt to close the top cover 166 SPARC T5 8 Server Service Manual e November 2015 Install a PCle Card 5 Ensure that the card s bulkhead notch is aligned properly with the carrier s bulkhead pin so that the cover enages the pin Servicing PCle Cards 167 Install a Card Carrier 6 Close and latch the top cover on the carrier 7 Install the card carrier into the server See Install a Card Carrier on page 168 Related Information Install a Card Carrier on page 168 m Verify the PCIe Card on page 171 Y Install a Card Carrier The SPARC T5 8 server supports single wide and double wide card carriers The installation steps are the same for both carrier widths This topic includes illustrations only for the single wide carrier 168 SPARC T5 8 Server Service Manual e November 2015 Installa Card Carrier A Caution This procedure involves handling circuit boards that are extremely sensitive to static electricity Ensure that you follow ESD preventative practices to avoid damaging the circuit boards See Prevent ESD Damage on page 57 1 Ensure that the primary domain is at the Oracle Solaris prompt Note
17. Carrier tree cerci rece nice rien ie nice ricerineo 155 Y Remove a PCle Card ui 159 Y Remove a Carrier Extension sccsccsscsscoscsvcnscvcrsesssossnssosssssossavensenes 162 Y Install a Carrier Extension 164 Y Install a PCle Gardiner no Sesau asia nadas di nad untado 165 Y Install a Card Gatrier iaia ira 168 Y Verify the PGle Gard pirla a a ale parer 171 Servicing the Rear WO Module erre ereeerrererrenanoa 173 Y Determine if the Rear I O Module Is Faulty ires 173 SPARC T5 8 Server Service Manual November 2015 Contents Y Remove the Rear I O Module e 174 Y Install the Rear I O Module e 176 Y Verify the Rear I O Module ee 178 Servicing the Rear Chassis Subassembly sirene 181 Y Determine if the Rear Chassis Subassembly Is Faulty ee ee 181 Y Remove the Rear Chassis Subassembly e 182 w Install the Rear Chassis Subassembly 185 Y Verify the Rear Chassis Subassembly 188 Returning the Server to Operation rece eeree errando 191 y Connect the Power GOTAS siriana 191 Y Power On the Server Oracle ILOM iii 192 Y Power On the Server Power Button 0 193 Glossary erann G tenaaa a rali ala ai
18. Diag Run POST Alternatively you can use the System target gt start SYS Are you sure you want to start SYS y n y Starting SYS Start the system console to view the output of the tests gt start HOST console Related Information POST Overview on page 35 m Oracle ILOM Properties That Affect POST Behavior on page 35 Configure POST on page 39 Oracle ILOM Documentation Library SPARC T5 8 Server Service Manual e November 2015 Managing Faults Managing Faults These topics describe the Predictive Self Healing PSH feature PSH Overview on page 41 Check for Faults on page 41 Clear a Fault on page 44 PSH Overview PSH provides problem diagnosis on the SP and the host Regardless of where a fault occurs you can view and manage the fault diagnosis from the SP or the host When possible PSH initiates steps to take the component offline PSH also logs the fault to the syslogd daemon and provides a fault notification with a message ID You can use the message ID to get additional information about the problem from the knowledge article database A PSH console message provides this information about each detected fault Type Severity Description Automated response Impact Suggested action for system administrator If PSH detects a faulty component use the fmadm faulty command to display information about the fault See Check for Faults
19. Front I O Assembly Servicing the Main Module Identifying Components 23 Component Service Task Reference Component Max NAC Name SDM Name Notes Links m Disk drives all PCle carriers 16 SYS RCSA PCIEX CAR Servicing PCIe Cards PCIe cards 16 SYS RCSA PCIEX CAR System PCI Devices Add Servicing PCle Cards CARD on Device x Processor 4 SYS PMx System CPU Modules 4 for fully Servicing Processor modules CPU Module x populated Modules configuration 2 for half populated Server Upgrade configuration with Process on page 59 2 processor filler modules Rear I O module 1 SYS RIO Servicing the Rear I O Module Power supplies 4 SYS PSx System Power Servicing Power Supplies Power Supplies Power Supply x Rear chassis 1 SYS RSCA Servicing the Rear Chassis subassembly Subassembly SCC PROM 1 SYS MB SCC Servicing the System Configuration PROM SP 1 SP SP Servicing the Service Processor Card Storage 2 SYS SASB PX Servicing the Storage backplanes Backplanes Related Information Front Panel Components Service on page 14 Rear Panel Components Service on page 15 Chassis Subassembly Components on page 22 Server Upgrade Process on page 59 SPARC and Netra SPARC T5 Series Servers Administration Guide 24 SPARC T5 8 Server Service Manual e November 2015 Detecting and Managing Faults These topics explain how to
20. Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Y Connect the Power Cords 1 Connect a terminal device if necessary Find the AC connectors on the rear of the server See Rear Panel Components Service on page 15 Connect the power cords Note As soon as power is applied standby power is applied and the SP boots Depending on Oracle ILOM policy settings the host might also boot in parallel Returning the Server to Operation 191 Power On the Server Oracle ILOM Start the system See Power On the Server Oracle ILOM on page 192 Related Information Power On the Server Oracle ILOM on page 192 Power On the Server Power Button on page 193 Y Power On the Server Oracle ILOM 192 The start up process depends on the last power state of the server and on Oracle ILOM policy settings You must check to see if the system is running before you issue the start SYS command Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Determine if the system has been started gt Show System power state Ifthe system is running go to the verification task for the component that you replaced Ifthe system is not running go to Step 3 Start the system gt start System Determine your next step If you are servicing a component r
21. Main Module on page 95 Related Information Remove the Front I O Assembly on page 127 m Verify the Front I O Assembly on page 131 Y Verify the Front I O Assembly 1 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 Started the system See Power On the Server Oracle ILOM on page 192 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y Servicing the Front I O Assembly 131 Verify the Front I O Assembly faultmgmtsp gt 4 Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 Ifno fault was detected the front I O assembly was installed successfully Related Information Remove the Front I O Assembly on page 127 Install the Front I O Assembly on page 130 132 SPARC T5 8 Server Service Manual e November 2015 Servicing Power Supplies The power supplies are hot service components that can be replaced at any time There are no slot restrictions for power supplies Any supported power supply may be installed in any of the four power supply slots However in order to maintain full power redundancy all four power supply slots must be occupied with functioning power supp
22. Manual e November 2015 Install a Fan Module Related Information Determine Which Fan Module Is Faulty on page 142 Install a Fan Module on page 145 m Verify the Fan Module on page 146 Y Install a Fan Module 1 Insert the fan module into the empty fan module slot Servicing Fan Modules 145 Verify the Fan Module 2 Push the fan module into the slot until it clicks into place to completely seat the fan module into the slot 3 Power on the server if necessary If you had to power off the server before removing and installing a new fan module see Returning the Server to Operation to power on the server again 4 Verify the fan module functionality See Verify the Fan Module on page 146 Related Information Determine Which Fan Module Is Faulty on page 142 Remove a Fan Module on page 143 Verify the Fan Module on page 146 Y Verify the Fan Module 1 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 146 SPARC T5 8 Server Service Manual e November 2015 Verify the Fan Module m Started the system See Power On the Server Oracle ILOM on page 192 Check the front or rear panel LEDs for the following indications Green System OK LED illuminated Amber System Fault LED not illuminated Amber System Fan Fault LED not illuminated See Front Panel Con
23. On the Server Oracle ILOM on page 192 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 Ifno fault was detected the rear chassis subassembly was installed successfully SPARC T5 8 Server Service Manual e November 2015 Verify the Rear Chassis Subassembly Related Information Determine if the Rear Chassis Subassembly Is Faulty on page 181 m Install the Rear Chassis Subassembly on page 185 Servicing the Rear Chassis Subassembly 189 190 SPARC T5 8 Server Service Manual e November 2015 Returning the Server to Operation These topics explain how to return the server from Oracle to operation after you have performed service procedures Steps Description Links 1 Apply power to the server 2 Start the system 3 Either return to the service operation that you were performing if applicable or return to the upgrade process Connect the Power Cords on page 191 m Power On the Server Oracle ILOM on page 192 m Power On the Server Power Button on page 193 Component Service Categories on page 52 Server Upgrade Process on page 59 Related
24. Oracle ILOM Properties That Affect POST Behavior on page 35 Configure POST on page 39 Run POST With Maximum Testing on page 40 Oracle ILOM Properties That Affect POST Behavior Note The value of keyswitch_state must be normal when individual POST parameters are changed Caution Setting the verbosity values to max will result in POST taking a longer amount of time to complete its testing of the server Detecting and Managing Faults 35 Configuring POST 36 TABLE 1 HOST keyswitch state Value Description normal The server can power on and run POST based on the other parameter settings This parameter overrides all other commands diag The server runs POST based on predetermined settings standby The server cannot power on locked The server can power on and run POST but no flash updates can be made TABLE 2 HOST diag mode Value Description off POST does not run normal POST runs according to diag level value max If diag mode normal runs all the minimum tests plus extensive processor and memory tests min If diag mode normal runs minimum set of tests TABLE 3 HOST diag trigger Value Description hw change default Runs POST following a FRU replacement or an AC power cycle all resets Runs POST on all resets error reset power on reset Runs POST on all error resets Runs POST on every power on Does not run POST on reset
25. Perform one of the following tasks based on your verification results m If the previous steps did not verify the drive see Diagnostics Process on page 25 m If the previous steps indicate that the drive is functioning properly perform the tasks required to configure the drive These tasks are covered in the Oracle Solaris OS administration documentation For additional drive verification you can run the Oracle VTS software Refer to the Oracle VTS documentation for details Servicing Hard Drives 87 Verify the Hard Drive Related Information Determine Which Hard Drive Is Faulty on page 82 Install a Hard Drive on page 85 88 SPARC T5 8 Server Service Manual e November 2015 Servicing the Main Module The main module is a cold service component that can be replaced only after you have powered off the server For the location of the main module see Front Panel Components Service on page 14 Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics explain how to service the main module Steps Description Links 1 Replace the main module m Remove the Main Module on page 91 m Install the Main Module on page 95 2 Remove the main module as part of another Remove the Main Module on page 91 component s service operation 3 Install the main module as
26. Servicing the System Configuration PROM The system configuration PROM is a cold service component that can be replaced only by qualified service personnel The system configuration PROM stores the host ID and MAC address If you have to replace the motherboard be sure to move the System configuration PROM from the old motherboard to the new motherboard This step will ensure that the server will retain its original host ID and MAC address Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe service procedures for the system configuration PROM in the server Steps Description Links 1 Determine if the system configuration PROM is Interpreting LEDs on page 29 faulty 2 Remove the main module so that you can replace the Remove the Main Module on page 91 system configuration PROM 3 Replace the system configuration PROM m Remove the System Configuration PROM on page 118 m Install the System Configuration PROM on page 119 4 Install the main module after you have replaced the Install the Main Module on page 95 system configuration PROM 5 Verify the installation Verify the System Configuration PROM on page 120 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Serv
27. a Hard Drive on page 85 Related Information Determine Which Hard Drive Is Faulty on page 82 m Install a Hard Drive on page 85 Y Install a Hard Drive 1 Align the replacement drive to the drive slot and slide the drive in untilitis seated Servicing Hard Drives 85 Verify the Hard Drive 2 8 Drives are physically addressed according to the slot in which they are installed If you are replacing a drive install the replacement drive in the same slot as the drive that was removed Close the latch to lock the drive in place Verify the installation See Verify the Hard Drive on page 86 Y Verify the Hard Drive 86 Determine if you replaced or installed a hard drive in a running server or not If you replaced or installed a hard drive in a server that is running if you hot plugged the hard drive then no further action is necessary The Oracle Solaris OS will automatically configure the hard drive If you replaced or installed a hard drive in a powered down server then continue with these steps to configure the hard drive If the OS is shut down and the drive you replaced was not the boot device boot the OS Depending on the nature of the replaced drive you might need to perform administrative tasks to reinstall software before the server can boot Refer to the Oracle Solaris OS administration documentation for more information At the Oracle Solaris prompt type the cfgadm
28. al command to list all drives in the device tree including any drives that are not configured cfgadm al SPARC T5 8 Server Service Manual e November 2015 Verify the Hard Drive This command helps you identify the drive you installed Example Ap id Type Receptacle Occupant Condition c2 scsi sas connected configured unknown c2 w5000cca00a76d1f5 0 disk path connected configured unknown c3 scsi sas connected configured unknown c3 sd2 disk path connected unconfigured unknown c4 scsi sas connected configured unknown c4 w5000cca00a59b0a9 0 disk path connected configured unknown Configure the drive using the cfgadm c configure command Example cfgadm c configure c2 w5000cca00a76d1f5 0 Replace c2 w5000cca00a76d1f5 0 with the drive name for your configuration Verify that the blue Ready to Remove LED is no longer lit on the drive that you installed At the Oracle Solaris prompt type the cfgadm al command to list all drives in the device tree including any drives that are not configured cfgadm al The replacement drive is now listed as configured Example Ap_id Type Receptacle Occupant Condition c2 scsi sas connected configured unknown c2 w5000cca00a76d1f5 0 disk path connected configured unknown c3 scsi sas connected configured unknown c3 w5000cca00a772bd1 0 disk path connected configured unknown c4 scsi sas connected configured unknown c4 w5000cca00a59b0a9 0 disk path connected configured unknown
29. and System Messages 45 Y Check the Message Buffer ei 45 Y View Log Files Oracle Solaris ri rrerrreea 46 Y View Log Files Oracle ILOM rrenan 46 Preparing for Service co cic ccceccescscavcencscvveceessuvceuccannicvscasaveneesinvecevsauaceucauaniovevaanaebes 49 Safety Informationi s irrita ira 49 Safety Symbols sauen i hi ie lola ao 50 ESD Measues sussa spiuesserssue os pontas use cdsho Il iene pais dese ias 50 Antistatic Wrist Strap Use ciccia zi iii iii 50 Antistatic Mat i oriali lidia lan iaia leale 50 Tools Needed for Service ss criar iraniana air 51 Filler Panels and Modules ii 51 Component Service Categories eee 52 Y Find the Server Serial Number e erereeereeenreennea 52 V Locate the Server spa pi aio Ialia 53 Removing Power From the Server ui 53 Y Prepare to Power Off the Server rrenan 54 Y Power Off the Server SP Command i 54 Y Power Off the Server Power Button Graceful 55 Y Power Off the Server Emergency Shutdown i 55 Y Disconnect the Power Cords ve 56 Y Prevent ESD Damage Lineare ned erre parete E ss trad os 57 Servicing Processor Modules i 59 Server Upgrade Process iii 59 Process
30. faulty DIMM to light for a few minutes Confirm that the DIMM next to the illuminated DIMM Fault LED is the same DIMM that was reported to be faulty by the fmadm faulty command See Determine Which DIMM Is Faulty FMA on page 73 Visually check to ensure that all of the other DIMMS are seated properly in their slots Related Information Determine Which DIMM Is Faulty FMA on page 73 Remove a DIMM or DIMM Filler Panel on page 76 DIMM Configuration Fault Messages When the system boots system firmware checks the memory configuration against the rules described in DIMM Configuration on page 71 If it discovers any faults one or more rule specific messages will be displayed in the POST output indicating the type of configuration fault that has been discovered Related Information DIMM Configuration on page 71 Determine Which DIMM Is Faulty FMA on page 73 Determine Which DIMM Is Faulty LEDs on page 75 Remove a DIMM or DIMM Filler Panel on page 76 Y Remove a DIMM or DIMM Filler Panel 76 A DIMMs are cold service components that can be replaced after you remove the processor module from the server Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail Before beginning this procedure ensure that you are familiar with the cautions and safety instructions descri
31. from the server Related Information Determine Which PCle Card Is Faulty on page 155 Install a Card Carrier on page 168 158 SPARC T5 8 Server Service Manual e November 2015 Remove a PCle Card Y Remove a PCle Card A Caution This procedure involves handling circuit boards that are extremely sensitive to static electricity Ensure that you follow ESD preventative practices to avoid damaging the circuit boards See Prevent ESD Damage on page 57 1 Ensure that you have already taken antistatic measures See Prevent ESD Damage on page 57 2 Determine your next step a If a carrier extension is installed go to Remove a Carrier Extension on page 162 To provide adequate access to the card you must remove the carrier extension if one is installed b If no carrier extension is installed go to Step 3 Servicing PCle Cards 159 Remove a PCle Card 3 Unlatch and open the top cover of the carrier 160 SPARC T5 8 Server Service Manual e November 2015 Remove a PCle Card 4 Carefully remove the PCle card from the carrier A Caution Avoid twisting or pulling unevenly on the card which could damage the card and the carrier slot 5 Place the PCle card on an antistatic mat or into its antistatic packaging 6 Determine your next step If you are replacing the PCIe card go to Install a PCIe Card on page 165 m If you are removing the PCIe card return the c
32. removed the main module Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail Determine your first step a If you cannot access the service processor card go to Step 2 b If you can access the service processor card back up the SP configuration information gt cd SP config gt dump destination uri target where The acceptable values for uri are tftp ftp sftp scp http and https SPARC T5 8 Server Service Manual e November 2015 Remove the Service Processor Card m target is the remote location where you want to store the configuration information For example gt dump destination tftp 129 99 99 99 pathname Take the necessary ESD precautions See Prevent ESD Damage on page 57 Remove the main module from the server See Remove the Main Module on page 91 Locate the service processor card on the main module See Main Module Internal Component Locations on page 17 Grasp the service processor card by the two grasp points and lift up to disengage the service processor card from the connectors on the motherboard Servicing the Service Processor Card 111 Install the Service Processor Card 6 Lift the service processor card up and away from the motherboard Related Information Determine if the Service Processor Card Is Faulty on page 109 m Instal
33. this indicator to light See Check for Faults on page 41 Under some fault conditions individual component fault LEDs are lit in addition to the Service Required LED Indicates these conditions m Off Server is not running in its normal state System power might be off The SP might be running m Steady on Server is powered on and is running in its normal operating state No service actions are required m Fast blink Server is running in standby mode and can be quickly returned to full function m Slow blink A normal but transitory activity is taking place Slow blinking might indicate that system diagnostics are running or that the system is booting Indicates these conditions Off AC power might have been connected to the power supplies Steady on green SP is running in its normal operating state No service actions are required Blink green SP is initializing the Oracle ILOM firmware Steady on amber An SP error has occurred and service is required This button can be used to prove physical presence in the case of log in recovery Indicates these conditions m Off Indicates a steady state no service action is required m Steady on Indicates that a temperature failure event has been acknowledged and a service action is required Indicates these conditions m Off Indicates a steady state no service action is required m Steady on Indicates that a temperature fail
34. 1 m Started the system See Power On the Server Oracle ILOM on page 192 2 Start the faultmgmt shell gt start SP faultmgmt shell Servicing the Main Module 97 Verify the Main Module Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt 3 Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 m Ifno fault was detected the main module was installed successfully Related Information Determine if the Main Module Is Faulty on page 91 Install the Main Module on page 95 98 SPARC T5 8 Server Service Manual e November 2015 Servicing the Storage Backplanes The storage backplances are cold service components that can be replaced after you remove the main module For the location of the storage backplanes see Main Module Internal Component Locations on page 17 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe service procedures for the storage backplanes in the server Steps Description Links 1 Remove a storage backplane Remove a Storage Backplane on page 99 2 Install a storage backplane Install a Storage Backplane on page 103 3 Verify the installation Verify the Storage Backplane on page 107 Related Informa
35. 15 Install the Rear I O Module 2 With the levers in the extended position insert the rear I O module into the slot at the rear of the server Servicing the Rear I O Module 177 Verify the Rear I O Module 3 Close the extraction levers until they click into place to fully seat the rear I O module into the server 4 Connect the cables to the appropriate ports on the rear I O module 5 Power on the server See Returning the Server to Operation 6 Verify the installation See Verify the Rear I O Module on page 178 Related Information Determine if the Rear I O Module Is Faulty on page 173 Remove the Rear I O Module on page 174 Verify the Rear I O Module on page 178 Y Verify the Rear I O Module 1 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 178 SPARC T5 8 Server Service Manual e November 2015 Verify the Rear I O Module m Started the system See Power On the Server Oracle ILOM on page 192 Verify that the System Service Required LED on the rear I O module is not lit See Rear Panel Controls and LEDs on page 33 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determin
36. 2 2 Replace a hard drive m Determine Which Hard Drive Is Faulty on page 82 m Remove a Hard Drive on page 83 m Install a Hard Drive on page 85 m Verify the Hard Drive on page 86 3 Add storage m Install a Hard Drive on page 85 n Verify the Hard Drive on page 86 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Servicing Hard Drives 81 Hard Drive LEDs Component Service Task Reference on page 23 Returning the Server to Operation Hard Drive LEDs Som a No LED Icon Description 1 Ready to Remove Indicates that a drive can be removed during a hot service blue 4 operation 2 Service Required Indicates the drive s availability for use amber m On Read or write activity is in progress m Off Drive is idle and available for use 3 Okay OK Indicates normal operation Blinking indicates that the drive is in use Related Information Determine Which Hard Drive Is Faulty on page 82 Remove a Hard Drive on page 83 Y Determine Which Hard Drive Is Faulty The following LEDs are lit when a hard drive fault is detected m System Service Required LEDs on the front panel and rear I O module Service Required LED on the faulty drive 82 SPARC T5 8 Server Service Manual e November 2015 Remove a Hard Drive 1 Determine if the System Service Requi
37. 3 Remove the Battery 1 Take the necessary ESD precautions See Prevent ESD Damage on page 57 2 Remove the main module See Remove the Main Module on page 91 3 Locate the system battery in the main module See Main Module Internal Component Locations on page 17 4 Push the top edge of the battery against the spring and lift it out of the carrier Related Information Install the Battery on page 125 Verify the Battery on page 126 124 SPARC T5 8 Server Service Manual e November 2015 Install the Battery Y Install the Battery 1 Insert the new battery in the main module with the positive side facing out 2 Install the main module back into the server See Install the Main Module on page 95 If the SP is configured to synchronize with a network time server using the Network Time Protocol the Oracle ILOM clock will be reset as soon as the server is powered on and connected to the network Otherwise proceed to the next step 3 Ifthe SP is not configured to use NTP use the Oracle ILOM clock command to set the day and time The following example sets the date to June 17 2013 gt set SP clock datetime 061716192010 gt show SP clock SP clock Targets Properties datetime Wed JUN 17 16 19 56 2013 timezone GMT GMT usentpserver disabled Commands cd Servicing the Battery 125 Verify the Battery set show Verify that the new ba
38. At the Oracle Solaris prompt type the cfgadm al command to list all drives in the device tree including drives that are not configured cfgadm al This command lists dynamically reconfigurable hardware resources and shows their operational status In this case look for the status of the drive you plan to remove This information is listed in the Occupant column Example Ap id Type Receptacle Occupant Condition c2 scsi sas connected configured unknown c2 w5000cca00a76d1f5 0 disk path connected configured unknown c3 scsi sas connected configured unknown c3 w5000cca00a772bd1 0 disk path connected configured unknown c4 scsi sas connected configured unknown c4 w5000cca00a59b0a9 0 disk path connected configured unknown You must unconfigure any drive whose status is listed as configured as described in Step 3b Unconfigure the drive using the cfgadm c unconfigure command Example cfgadm c unconfigure c2 w5000cca00a76d1f5 0 Replace c2 w5000cca00a76d1f5 0 with the drive name that applies to your situation Verify that the drive s blue Ready to Remove LED is lit Press the drive release button to unlock the drive SPARC T5 8 Server Service Manual e November 2015 Install a Hard Drive 5 Pull the drive out of the server A Caution The latch is not an ejector Do not force the latch too far to the right Doing so can damage the latch 6 Install the replacement drive or a filler tray See Install
39. Is Faulty on page 62 Remove a Processor Module or Processor Filler Module on page 63 Install a Processor Module or Processor Filler Module on page 67 m Verify the Processor Module on page 70 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Server Upgrade Process The SPARC T5 8 server supports two configurations Servicing Processor Modules 59 Server Upgrade Process Fully populated four processor modules Half populated two processor modules and two processor filler modules Processor modules and processor filler modules are cold service components that can be replaced only after you remove all power from the system For the location of the processor modules see Front Panel Components Service on page 14 Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail This table contains the steps for upgrading the server to a fully populated configuration You can also view an animated demonstration of the upgrade process at Flash http download oracle com SPARC T5 8 Server SPARC T5 8 Server T5 8 Demo html MP4 http download oracle com SPARC T5 8 Server SPARC T5 8 Server Upgrade mp4 Step Descriptio
40. M filer panels and replace them with DIMMs before installing the new processor modules DIMM filler panels are not supported in running processor modules PCle card carriers these function like filler panels when a card is not installed Processor filler modules located in slot 1 and slot 2 in a half populated server A Caution To maintain the proper air flow all filler panels and modules must remain in the server unless you remove one to install a functioning component at the same time Related Information Safety Information on page 49 Component Service Categories on page 52 Server Upgrade Process on page 59 Preparing for Service 51 Component Service Categories Component Service Categories The following table identifies the server components that are replaceable Component AIC Power Status Authorized Remove and Replace Instructions for Removal Service Personnel Only Battery Off Servicing the Battery DIMMs Off Servicing DIMMs Fan modules On or off Servicing Fan Modules Front I O assembly Off Servicing the Front I O Assembly Hard drives On or off Servicing Hard Drives Main module Off Servicing the Main Module PCIe cards On or off Servicing PCle Cards Power supplies On or off Servicing Power Supplies Processor modules Off Servicing Processor Modules Rear I O module Off Servicing the Rear I O Module Rear chassis Off X Servicing
41. S and STORAGE were replaced with System You can still use the 3 0 legacy names in commands at any time but to expose the legacy names in the output you must enable them This manual uses the legacy names in the command examples and shows the names in the output examples For more information about the new name spaces see the Oracle ILOM documentation Related Information Diagnostics Process on page 25 Tool Availability on page 27 Oracle ILOM Service Related Tools on page 29 SPARC T5 8 Server Service Manual e November 2015 Interpreting LEDs m Matching Devices to Device Names in SPARC and Netra SPARC T5 Series Servers Administration Guide Oracle ILOM Documentation Library Oracle ILOM Service Related Tools You can use these Oracle ILOM shell commands when performing service related tasks Oracle ILOM Command Description help command set HOST send break action break start HOST console show HOST console history set HOST bootmode property value stop System or stop SYS start System or start SYS reset System or reset SYS reset SP Displays a list of all available commands with syntax and descriptions Specifying a command name as an option displays help for that command Takes the host server from the OS to either kmdb or OpenBoot prompt equivalent to a Stop A depending on the mode in which the Oracle Solaris OS was booted Connects to th
42. SPARC T5 8 Server Service Manual Part No E35082 14 November 2015 ORACLE Part No E35082 14 Copyright O 2013 2015 Oracle and or its affiliates All rights reserved This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws Except as expressly permitted in your license agreement or allowed by law you may not use copy reproduce translate broadcast modify license transmit distribute exhibit perform publish or display any part in any form or by any means Reverse engineering disassembly or decompilation of this software unless required by law for interoperability is prohibited The information contained herein is subject to change without notice and is not warranted to be error free If you find any errors please report them to us in writing If this is software or related documentation that is delivered to the U S Government or anyone licensing it on behalf of the U S Government then the following notice is applicable U S GOVERNMENT END USERS Oracle programs including any operating system integrated software any programs installed on the hardware and or documentation delivered to U S Government end users are commercial computer software pursuant to the applicable Federal Acquisition Regulation and agency specific supplemental regulations As such use duplication disclosure modification and adaptati
43. See Remove a Power Supply on page 135 Related Information Remove a Power Supply on page 135 m Verify the Power Supply on page 138 Y Remove a Power Supply The power supplies are hot service components that can be replaced at any time For the location of the power supplies see Front Panel Components Service on page 14 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail 1 Locate the power supply in the server that you want to remove m See Front Panel Components Service on page 14 for the locations of the power supplies in the server m See Determine Which Power Supply Is Faulty on page 134 to locate a faulty power supply 2 Goto the rear of the server and locate the AC power connector at the rear of the server that supplies power to the faulty power supply See Broken Link Target ID Z400022C1500027 3 Disconnect that power cord Servicing Power Supplies 135 Remove a Power Supply 4 Atthe front of the server squeeze the release latches together on the power supply to be removed then pull the extraction lever toward you to disengage the power supply CIS A OS a a 5 Pull the power supply out of the server 136 SPARC T5 8 Server Service Manual e November 2015 Install a Power Supply 6 Install the replacement power supply See Install
44. Service Manual e November 2015 Remove the Front I O Assembly 5 Disconnect the cable 6 Loosen the retaining screw that secures the front I O assembly to the motherboard 7 Gently pull the front I O assembly toward the rear of the main module until the ports at the front of the assembly clear the front of the main module and then remove the front I O assembly from the main module Related Information Install the Front I O Assembly on page 130 m Verify the Front I O Assembly on page 131 Servicing the Front I O Assembly 129 Install the Front I O Assembly Y Install the Front I O Assembly A Caution This procedure requires that you handle components that are sensitive to static discharge Static discharges can cause the components to fail 1 Take the necessary ESD precautions See Prevent ESD Damage on page 57 2 Insert the front I O assembly into position in the main module m Gently slide the front I O assembly into position with the ports inserted into the port holes in the front of the main module Lower the rear of the front I O assembly so that the captive screw is aligned with the screw hole on the motherboard 130 SPARC T5 8 Server Service Manual e November 2015 Verify the Front I O Assembly 3 Tighten the retaining screw to secure the front I O assembly to the motherboard Connect the cable Install the main module back into the server See Install the
45. Sometimes referred to as an HBA See HBA Supports the creation of RAID volumes on drives Serial attached SCSI System configuration chip Serial management port A serial port on the server SP the server module SP and the CMM Modular component that provides the main compute resources CPU and memory in a modular system Server modules also might have on board storage and connectors that hold FEMs Service processor In the server or server module the SP is a card with its own OS The SP processes Oracle ILOM commands providing lights out management control of the host See host 198 SPARC T5 8 Server Service Manual e November 2015 WWN SSD SSH storage module TIA Tma U S NEC UCP UI UL UTC UUID WWN Solid state drive Secure shell Modular component that provides computing storage to the server modules Telecommunications Industry Association Netra products only Maximum ambient temperature United States National Electrical Code Universal connector port User interface Underwriters Laboratory Inc Coordinated Universal Time Universal unique identifier World wide name A unique number that identifies a SAS target Glossary 199 200 SPARC T5 8 Server Service Manual e November 2015 Index A AC power connectors locating 15 adding additional processor modules 59 antistatic wrist strap 50 B battery installing 125 locating 17 NAC name 23 removing 123 Cc
46. X 16256 MB memory available Serial 487304604 Ethernet address Host ID KKKK Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 Ifno fault was detected the System Configuration PROM was installed successfully For additional verification run specific commands to display data stored in the System Configuration PROM Use the Oracle ILOM show command to display the MAC address SPARC T5 8 Server Service Manual e November 2015 Verify the System Configuration PROM gt show HOST macaddress HOST Properties macaddress 1 kkk 4x Use Oracle Solaris OS commands to display the hostid and Ethernet address hostid 8534299c ifconfig a 100 flags 2001000849 lt UP LOOPBACK RUNNING MULTICAST IPv4 VIRTUAL gt mtu 8232 index 1 inet 127 0 0 1 netmask ff000000 igb0 flags 201004843 lt UP BROADCAST RUNNING MULTICAST IPv4 gt mtu 1500 index 2 inet 10 6 88 150 netmask fffffe00 broadcast 10 6 89 255 ether 8 px pot pot ok Related Information Remove the System Configuration PROM on page 118 Install the System Configuration PROM on page 119 Servicing the Sy
47. a Power Supply on page 137 Related Information Determine Which Power Supply Is Faulty on page 134 Install a Power Supply on page 137 Y Install a Power Supply The power supply is a hot service component that can be replaced by a customer A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components 1 Open the latch on the replacement power supply and align the power supply with the empty bay 2 Slide the power supply into the chassis Servicing Power Supplies 137 Verify the Power Supply 3 Press the lever against the power supply to fully seat the power supply in the server E ERROS TOSENN Red SLI NI 4 Verify the installation See Verify the Power Supply on page 138 Related Information Remove a Power Supply on page 135 Verify the Power Supply on page 138 W Verify the Power Supply 1 Verify that the power supply Power OK and AC Present LEDs are lit and that the Fault LED is not lit See Rear Panel Controls and LEDs on page 33 2 Verify that the front and rear Service Required LEDs are not lit See Interpreting LEDs on page 29 3 Loginto Oracle ILOM See Log In to Oracle ILOM Service on page 28 138 SPARC T5 8 Server Service Manual e November 2015 Verify the Power Supply Start the faultmgmt shell gt start SP faul
48. aa 13 Front Panel Components Service sccescceeceeecceecceecceeccecceeceeeeeseeeeeenseeeseees 14 Rear Panel Components Service e 15 DIMM Locations as ppi Meds e n ea RE aE 16 Main Module Internal Component Locations 0 17 Supported Storage Devices 18 Fani Module EGCAtions mirri 20 Rear I O Module Port Locations i 21 Chassis Subassembly Components i 22 Component Service Task Reference i 23 Detecting and Managing Faults erre 25 Understanding Diagnostics ee 25 Diagnostics Process siii lata ii E rei 25 Tool Availability nali i alia 27 Y Log In to Oracle ILOM Service ee 28 Oracle ILOM Service Related Tools i 29 Interpreting LEDS nre ieri 29 Front Panel Controls and LEDS e 31 Rear Panel Controls and LEDS ei 33 Configuring POST ila I ari 35 POST Overview o lilla liana elia aaa 35 Oracle ILOM Properties That Affect POST Behavior eee 35 Y Configure POST raidue iena alii 39 Y Run POST With Maximum Testing ccceecceecceeceeeceeeeeeeeeeeeneeenes 40 Managing Faults ss iure ggi n rit 41 PSH OVERVIEW sora elencare reina 41 y Check for Fauls oroia pae 41 Contents y Clear a Fault casone denin plein 44 Interpreting Log Files
49. al e November 2015 Remove a Storage Backplane 6 Disconnect the two storage backplane cables from the storage backplane that you want to replace No Description 1 Data cable 2 Power cable Servicing the Storage Backplanes 101 Remove a Storage Backplane 7 Lift up on the plastic retaining panel for the storage backplane that you want to remove to disengage the plastic panel from the top of the hard drive assembly 8 Push the plastic panel toward the rear of the main module and remove the plastic panel from the main module 102 SPARC T5 8 Server Service Manual e November 2015 Install a Storage Backplane 9 Push the top edge of the storage backplane slightly toward the rear of the main module then lift the storage backplane up and remove it from the main module Related Information m Install a Storage Backplane on page 103 m Verify the Storage Backplane on page 107 Y Install a Storage Backplane 1 Position the storage backplane in the main module Servicing the Storage Backplanes 103 Install a Storage Backplane 2 Lower the storage backplane into place 104 SPARC T5 8 Server Service Manual e November 2015 Install a Storage Backplane 3 Slide the plastic retaining panel into place over the storage backplane so that the two notches in the panel slide underneath the two metal mounting studs on the hard drive assembly 4 Press on the press point on the retai
50. ard Carrier on page 155 m Verify the PCIe Card on page 171 Y Verify the PCle Card 1 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 m Started the system See Power On the Server Oracle ILOM on page 192 2 Verify that the Fault LED is not lit on the PCle card 3 Verify that the System Service Required LEDs on the front panel and rear I O module are not lit See Interpreting LEDs on page 29 4 Verify that the System PCle Fault LED on the front panel is not lit See Front Panel Controls and LEDs on page 31 5 Perform one of the following tasks based on your verification results m If any of the LEDs are lit see Diagnostics Process on page 25 Ifnone of the LEDs are lit go to Step 6 6 Verify that the green Power LED is lit on the card that you installed 7 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 8 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt 9 Use the fmadm faulty command to determine if the server is operating normally Servicing PCle Cards 171 Verify the PCle Card Ifa fault was detected see Diagnostics Process on page 25 Ifno fault was detected the PCIe card was installed successfully 10 At the Oracle Solaris prompt ensure that the card i
51. ard carrier to the server See Install a Card Carrier on page 168 Related Information Remove a Carrier Extension on page 162 Install a PCIe Card on page 165 Servicing PCle Cards 161 Remove a Carrier Extension Y Remove a Carrier Extension If you are replacing the PCIe card you must remove the carrier extension to provide enough clearance to access the card Also if you are reconfiguring the server to remove a double wide carrier you must remove the carrier extension so that you can re install the main carrier 1 Unlock the carrier extension at the front panel CON QUI CON 162 SPARC T5 8 Server Service Manual e November 2015 Remove a Carrier Extension Hold the latch open and swing the carrier extension away from the main carrier separating the two components Determine your next step m If you are replacing the PCIe card with another card go to Install a PCIe Card on page 165 m If you are removing the PCIe card from the server go to Step 4 Set the carrier extension aside remove the PCle card and replace the two main PCle card carriers that you removed to install the carrier extension See Install a Card Carrier on page 168 Caution You must reinstall both main PCle card carriers to maintain the proper air flow Related Information Remove a PCle Card on page 159 Servicing PCle Cards 163 Install a Carrier Extension Install a Carr
52. ate the main module in the server See Front Panel Components Service on page 14 4 Squeeze the green latches together on the two extraction levers and pull the extraction levers out to disengage the main module from the server 92 SPARC T5 8 Server Service Manual e November 2015 Remove the Main Module Pull the main module halfway out of the server Press the levers back together toward the center of the main module This will keep the levers from getting damaged when you pull the main module out Remove the main module completely from the server Press down on the green button at the top of the cover to disengage the cover from the main module and push the cover toward the rear of the module as you lift the cover up and away from the chassis Determine your next step Servicing the Main Module 93 Remove the Main Module a If you are replacing a main module due to a faulty motherboard remove all of these internal components and transfer them to the new motherboard Component Link Front I O subassembly Remove the Front I O Assembly on page 127 Hard drives Remove a Hard Drive on page 83 Storage backplanes Remove a Storage Backplane on page 99 System battery Remove the Battery on page 123 System configuration PROM Remove the System Configuration PROM on page 118 System processor card Remove the Service Processor Card on page 110 b I
53. ated Information PSH Overview on page 41 Clear a Fault on page 44 Server Upgrade Process on page 59 Detecting and Managing Faults 43 Clear a Fault Clear a Fault When PSH detects faults the faults are logged and displayed on the console In most cases after the fault is repaired the corrected state is detected by the server and the fault condition is repaired automatically However this repair should be verified In cases where the fault condition is not automatically cleared you must clear the fault manually After replacing a faulty FRU power on the server See Returning the Server to Operation At the host prompt determine whether the replaced FRU still shows a faulty state See Check for Faults on page 41 m If no fault is reported you do not need to do anything else Do not perform the subsequent steps Ifa fault is reported continue to Step 3 Clear the fault from all persistent fault records In some cases even though the fault is cleared some persistent fault information remains and results in erroneous fault messages at boot time To ensure that these messages are not displayed type this PSH command faultmgmtsp gt fmadm acquit UUID If required reset the server In some cases the output of the fmadm faulty command might include this message for the faulty component faulted and taken out of service If this message appears in the output you must reset t
54. ates Tool Oracle ILOM Prompt OpenBoot Prompt Oracle Solaris Prompt Status LEDs Yes Yes Yes PSH commands Yes No Yes Oracle ILOM logs and commands Yes No No OpenBoot commands No Yes No Oracle Solaris logs and commands No No Yes Oracle VTS No No Yes if installed Third party software No No Yes if installed Detecting and Managing Faults 27 Log In to Oracle ILOM Service 28 Related Information Diagnostics Process on page 25 Log Into Oracle ILOM Service on page 28 m Oracle ILOM Service Related Tools on page 29 Oracle ILOM Documentation Library Log In to Oracle ILOM Service At the terminal prompt type ssh root lP address Password password Oracle R Integrated Lights Out Manager Version 3 2 1 2 rXXXXX Copyright c 2013 Oracle and or its affiliates All rights reserved gt Note To enable first time login and access to Oracle ILOM a default Administrator account and its password are provided with the system To build a secure environment you must change the default password changeme for the default Administrator account root after your initial login to Oracle ILOM If this default Administrator account has since been changed contact your system administrator for an Oracle ILOM user account with Administrator privileges Enable the Oracle ILOM 3 0 legacy name spaces gt set SP cli legacy_targets enabled Note In Oracle ILOM 3 1 the name spaces for SY
55. atic strap to your wrist and then to a metal area on the chassis Related Information Safety Information on page 49 Servicing Processor Modules Servicing DIMMs Servicing Hard Drives Servicing the Main Module Servicing the Storage Backplanes m Servicing the Service Processor Card Servicing the System Configuration PROM Servicing the Battery m Servicing the Front I O Assembly Servicing PCle Cards m Servicing the Rear I O Module Servicing the Rear Chassis Subassembly Preparing for Service 57 58 SPARC T5 8 Server Service Manual e November 2015 Servicing Processor Modules The SPARC T5 8 server supports two configurations Fully populated four processor modules Half populated two processor modules and two processor filler modules Processor modules and processor filler modules are cold service components that can be replaced only after you remove all power from the system For the location of the processor modules see Front Panel Components Service on page 14 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe how to service the processor modules Server Upgrade Process on page 59 m Processor Module LEDs on page 61 Determine Which Processor Module
56. aulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 If no fault was detected the storage backplane was installed successfully Related Information Remove a Storage Backplane on page 99 Install a Storage Backplane on page 103 Servicing the Storage Backplanes 107 108 SPARC T5 8 Server Service Manual e November 2015 Servicing the Service Processor Card The service processor card is a cold service component that can be replaced only after you have removed the main module For the location of the service processor card see Main Module Internal Component Locations on page 17 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe how to service the service processor card Steps Description Links 1 Determine if the service processor card is faulty Determine if the Service Processor Card Is Faulty on page 109 2 Remove the main module so that you can replace the Remove the Main Module on page 91 service processor card 3 Replace the service processor card m Remove the Service Processor Card on page 110 m Install the Service Processor Card on page 112 4 Install the main module after you have replaced the Install the Main Module on page 95 service processor card
57. bed in Safety Information on page 49 SPARC T5 8 Server Service Manual e November 2015 Remove a DIMM or DIMM Filler Panel Caution Do not leave DIMM slots empty All of the DIMM slots must have a DIMM Take the necessary ESD precautions See Prevent ESD Damage on page 57 Remove the PM with the faulty DIMM See Remove a Processor Module or Processor Filler Module on page 63 Locate the DIMMS that need to be replaced See Determine Which DIMM Is Faulty FMA on page 73 or Determine Which DIMM Is Faulty LEDs on page 75 Push down on the ejector tabs on each side of the DIMM until the DIMM is released Caution DIMMs and heat sinks on the motherboard might be hot Grasp the top corners of the faulty DIMM and lift it out of its slot Place the DIMM on an antistatic mat Repeat Step 4 through Step 6 for any other DIMMs that you intend to remove Determine your next step m f you are replacing a faulty DIMM see Install a DIMM on page 78 Servicing DIMMs 77 Install a DIMM vy 78 All of the replacement DIMMs must be the same size and type See DIMM Configuration on page 71 m f you are upgrading the server continue to remove all of the DIMM filler panels from the new processor modules DIMM filler panels are not supported in running processor modules After you have removed all of the DIMM filler panels you can install the new DIMMs See Install a DIMM o
58. ber 2015 Verify the Service Processor Card Verify that the front and rear Service Required LEDs are not lit See Interpreting LEDs on page 29 If the Service Required LED is lit see Diagnostics Process on page 25 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 If no fault was detected the SP was installed successfully Verify that the SP date is correct gt show SP clock SP clock Targets Properties datetime Tue Jan 01 23 15 59 1970 timezone GMT GMT uptime 0 days 00 01 18 usentpserver disabled Commands cd set show Set the datetime property if necessary gt set SP clock datetime MMDDhhmmYYYY Set the timezone property if necessary gt set SP clock timezones 3 to 4 characters where the timezone value equals a three or four character timezone abbreviation such as EST or AHST Servicing the Service Processor Card 115 Verify the Service Processor Card Related Information Determine if the Service Processor Card Is Faulty on page 109 Install the Service Processor Card on page 112 116 SPARC T5 8 Server Service Manual e November 2015
59. cessor Module or Processor Filler Module on page 63 SPARC T5 8 Server Service Manual e November 2015 Remove a Processor Module or Processor Filler Module m Verify the Processor Module on page 70 Understanding PCle Root Complex Connections on page 149 Y Remove a Processor Module or Processor Filler Module The SPARC T5 8 server supports two configurations Fully populated four processor modules Half populated two processor modules in PMO and PMI and two processor filler modules in PM2 and PM3 The removal steps are the same for both components Processor modules and processor filler modules are cold service components that can be replaced only after you power off the system For the location of the modules see Front Panel Components Service on page 14 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail 1 Remove all of the power from the system See Removing Power From the Server on page 53 2 Take the necessary ESD precautions See Prevent ESD Damage on page 57 3 Determine which module you need to remove If you are replacing a faulty processor module or upgrading the memory remove that specific processor module If you are upgrading the server to a fully populated configuration start by removing the processor filler module in slot 1 Servicing Process
60. ch would reduce total system memory capacity Action Use fmadm faulty to provide a more detailed view of this event Please refer to the associated reference document at http support oracle com msg SPSUN4V 8000 CQ for the latest service procedures and policies regarding this diagnosis Related Information Determine Which DIMM Is Faulty LEDs on page 75 74 SPARC T5 8 Server Service Manual e November 2015 Determine Which DIMM Is Faulty LEDs Remove a DIMM or DIMM Filler Panel on page 76 Oracle ILOM documentation Y Determine Which DIMM Is Faulty LEDs 1 Check that the Service Required LED is lit on the front of the server See Front Panel Controls and LEDs on page 31 2 Check that the Service Required LED is lit on one of the processor modules See Processor Module LEDs on page 61 3 Remove the PM with the faulty DIMM See Remove a Processor Module or Processor Filler Module on page 63 4 Locate the DIMM Fault Remind button on the front right corner of the motherboard 5 Verify that the DIMM Fault Remind Power LED next to the button is lit An illuminated DIMM Fault Remind Power LED indicates that there is power available to light the faulty DIMM LED after you have pressed the DIMM Fault Remind button Servicing DIMMs 75 DIMM Configuration Fault Messages Press the DIMM Fault Remind button on the processor module This will cause DIMM Fault LED associated with the
61. complexes leaving slots under root complexes 300 and 4c0 as the last to be populated Note If you are upgrading the server you are done with the process Related Information Root Complex Connections Four Processor Modules on page 150 Root Complex Connections Two Processor Modules on page 151 m Server Upgrade Process on page 59 PCle Carrier LEDs and Controls No LED Icon Description 1 CAR PCle carrier 2 Attention ATTN Indicates an operational fault You can also press this button to alert the server to a newly Required button installed PCIe card 154 SPARC T5 8 Server Service Manual e November 2015 Determine Which PCle Card Is Faulty No LED Icon Description 3 Power OK LED Indicates these conditions green m Off PCIe card is not running You can safely remove the card or install a new card Steady on PCle card is powered on and is running in its normal operating state No service actions are required You cannot remove the PCIe card m Blinking PCle card is powering up or powering down You cannot insert or remove the card 4 Service Required The Service Required LED indicates these conditions LED amber m Off PCIe card is operating normally m On PCle card requires service Y Determine Which PCle Card Is Faulty The following LEDs are lit when a fault is detected m System Service Required LEDs on the front panel and rear I O module System Rear PCl
62. d rear panel Service Required LEDs are also lit if Locator LED the server detects a power supply fault white The Locator LED indicates these conditions m Off PCle card is operating normally On PCIe slot has been identified by a user or the button has been pushed 2 OK green Lights when the power supply DC voltage from the PSU to the server OK is within tolerance 3 AC Present green Lights when the AC voltage is applied to the power supply Related Information Determine Which Power Supply Is Faulty on page 134 Remove a Power Supply on page 135 m Verify the Power Supply on page 138 Y Determine Which Power Supply Is Faulty The following LEDs are lit when a power supply fault is detected m System Service Required LEDs on the front panel and rear I O module 134 SPARC T5 8 Server Service Manual e November 2015 Remove a Power Supply Fault LED on the faulty power supply 1 Determine if the System Service Required LEDs are lit on the front panel or the rear I O module See Interpreting LEDs on page 29 2 From the front of the server check the power supply Fault LEDs to identify which power supply needs to be replaced When the power cord is connected to the server the associated AC power connector is lit green indicating that the power cord is connected to an AC wall socket and is supplying power to the AC power connector 3 Remove the faulty power supply
63. dule m Remove a Fan Module on page 143 m Install a Fan Module on page 145 4 Verify the replacement Verify the Fan Module on page 146 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Fan Module LED Each fan has a single LED that is green when the fan is operating normally or amber when the fan has a fault Servicing Fan Modules 141 Determine Which Fan Module Is Faulty Y Determine Which Fan Module Is Faulty 142 The following LEDs are lit when a fan module fault is detected m System Service Required LEDs on the front panel and rear I O module a Server Fan Fail LED on the rear I O module m Service Required LED on the faulty fan module Determine if the System Service Required LEDs are lit on the front panel See Interpreting LEDs on page 29 Determine if the Server Fan Fail LED on the front panel is lit See Front Panel Controls and LEDs on page 31 From the rear of the server check the fan module LEDs to identify which fan module needs to be replaced See Fan Module LED on page 141 Remove the faulty fan module See Remove a Fan Module on page 143 Related Information Remove a Fan Module on page 143 Install a Fan Module on page 145 m Verify the Fan Modu
64. dule so that you can replace the Remove the Main Module on page 91 front I O assembly 2 Replace the I O assembly m Remove the Front I O Assembly on page 127 m Install the Front I O Assembly on page 130 3 Install the main module after you have replaced the Install the Main Module on page 95 T O assembly 4 Verify the installation Verify the Front I O Assembly on page 131 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Y Remove the Front I O Assembly The front I O assembly is a cold service component that can be replaced only after you have removed the main module Servicing the Front I O Assembly 127 Remove the Front I O Assembly A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail 1 Take the necessary ESD precautions See Prevent ESD Damage on page 57 2 Remove the main module from the server See Remove the Main Module on page 91 3 Locate the front I O assembly on the main module See Main Module Internal Component Locations on page 17 4 Locate the cable that connects the front I O assembly to the motherboard No Description 1 Front I O assembly 128 SPARC T5 8 Server
65. e LED on the front panel m Service Required LED on the faulty PCIe card 1 Determine if the System Service Required LEDs are lit on the front panel or the rear I O module See Interpreting LEDs on page 29 2 Determine if the rear PCle LED is lit on the front panel See Front Panel Controls and LEDs on page 31 3 From the rear of the server find the PCle card that has an amber Service Required LED lit 4 Remove the faulty PCle card See Remove a Card Carrier on page 155 Y Remove a Card Carrier The server supports single wide and double wide card carriers The removal steps are the same for both carrier widths This topic includes illustrations only for the single wide carrier Note If you are installing a PCIe card that requires a double wide carrier you must remove two single wide carriers Servicing PCle Cards 155 Remove a Card Carrier 156 A PCIe expansion cards with their carriers are hot service components that can be replaced at any time if the card is not in use Caution To remove a PCle card that is assigned to an I O domain first remove the device from the I O domain Then add the device to the root domain before you physically remove the device from the system These steps enable you to avoid a configuration that is unsupported by the Direct I O or SR IOV feature For more information about making hardware changes to an T O domain refer to the Oracle VM for SPARC documentation
66. e Service Processor Card 113 Verify the Service Processor Card b Download the system firmware Follow the firmware download instructions in the Oracle ILOM documentation Note You can load any supported system firmware version including the firmware revision that had been installed prior to the replacement of the service processor card Restore the SP configuration information gt cd SP config gt load source uri target where The acceptable values for uri are tftp ftp sftp scp http and https m target is the remote location where you stored the configuration information For example gt load source tftp 129 99 99 99 pathname Verify the installation of the service processor card See Verify the Service Processor Card on page 114 Related Information Remove the Service Processor Card on page 110 m Verify the Service Processor Card on page 114 Y Verify the Service Processor Card 114 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 m Started the system See Power On the Server Oracle ILOM on page 192 Verify that the SP LED on the main module or rear I O module is lit green See Front Panel Controls and LEDs on page 31 or Rear Panel Controls and LEDs on page 33 If the SP LED is lit see Diagnostics Process on page 25 SPARC T5 8 Server Service Manual e Novem
67. e Which DIMM Is Faulty FMA The FMA fmadm faulty command displays current server faults including DIMM failures O Type fmadm faulty at the faultmgmtsp prompt gt start SP faultmgmt shell Are you sure you want to start SP faultmgmt shell y n y faultmgmtsp gt fmadm faulty Servicing DIMMs 73 Determine Which DIMM Is Faulty FMA Time UUID msgid Severity 2013 01 18 21 04 40 7040d859 5b03 4a58 8dfd e3a80875d62f SPSUN4V 8000 CQ MAJOR Problem Status solved Diag Engine fdd 1 0 System Manufacturer Oracle Corporation Name SPARC T5 8 Part_Number 12345678 11 1 Serial Number 1 XXXXXXXXXX System Component Manufacturer Oracle Corporation Name SPARC T5 8 Part Number 12345678 11 1 Serial Number XXXXXXXXXX Suspect 1 of 1 Fault class fault memory dimm ue Certainty 100 Affects SYS PMO CM1 CMP BOBO CHO DO Status faulted but still in service FRU Status faulty Location SYS PM CM1 CMP BOBQ CH DO Manufacturer Samsung Name 8192MB DDR3 SDRAM DIMM Part Number 07042208 M393B1K70DHO YKO Revision 04 Serial Number 00CE02121585C74755 Chassis Manufacturer Oracle Corporation Name T5 chassis Part Number abcd Serial Number dbca Description The number of correctable errors associated with this memory module has exceeded acceptable levels Response An attempt will be made to remove the affected memory from service Impact The dimm may be deconfigured at system restart whi
68. e host Displays the contents of the host s console buffer Controls the method of booting for the host server s firmware The value of property can be state config or script Powers off the host server Powers on the host server Generates a hardware reset on the host server Reboots the SP Related Information Diagnostics Process on page 25 Tool Availability on page 27 Log In to Oracle ILOM Service on page 28 Interpreting LEDs Oracle ILOM Documentation Library Use these steps to determine if an LED indicates that a component has failed in the server Detecting and Managing Faults 29 Interpreting LEDs Steps Description Links 1 Check the LEDs on the front and rear of the server m Front Panel Controls and LEDs on page 31 m Rear Panel Controls and LEDs on page 33 2 Check the LEDs on the individual components m Determine if the Main Module Is Note Component LEDs might not be lit even though the component is faulty Use the instructions in these links to determine if the component has been diagnosed as being faulty Faulty on page 91 Determine W hich Processor Module Is Faulty on page 62 Determine W LEDs on pa Determine W hich DIMM Is Faulty ge 75 hich Hard Drive Is Faulty on page 82 Determine Wi hich Power Supply Is Faulty on page 134 Determine Wi hich Fan Module Is Faulty on page 142
69. e if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 If no fault was detected the rear I O module was installed successfully Related Information Determine if the Rear I O Module Is Faulty on page 173 Remove the Rear I O Module on page 174 Install the Rear I O Module on page 176 Servicing the Rear I O Module 179 180 SPARC T5 8 Server Service Manual e November 2015 Servicing the Rear Chassis Subassembly The rear chassis subassembly is a cold service component that can be replaced only by qualified service personnel You must remove all of the power from the server before you attempt to remove the rear chassis subassembly For the location of the rear chassis subassembly see Rear Panel Components Service on page 15 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe service procedures for the rear chassis subassembly in the server Steps Description Links 1 Determine if the rear chassis subassembly is faulty Determine if the Rear Chassis Subassembly Is using the system software Faulty on page 181 2 If the rear chassis subassembly is faulty shut the Removing Power From the Server on page 53 server down 3 Remove the rear chassis subassembly Remove the Rear Chassis Subassembly on page 182
70. e logiciel ou mat riel et la documentation qui l accompagne peuvent fournir des informations ou des liens donnant acc s des contenus des produits et des services manant de tiers Oracle Corporation et ses affili s d clinent toute responsabilit ou garantie expresse quant aux contenus produits ou services manant de tiers sauf mention contraire stipul e dans un contrat entre vous et Oracle En aucun cas Oracle Corporation et ses affili s ne sauraient tre tenus pour responsables des pertes subies des co ts occasionn s ou des dommages caus s par l acc s des contenus produits ou services tiers ou leur utilisation sauf mention contraire stipul e dans un contrat entre vous et Oracle Accessibilit de la documentation Pour plus d informations sur l engagement d Oracle pour l accessibilit la documentation visitez le site Web Oracle Accessibility Program l adresse http www oracle com pls topic lookup ctx acc amp id docacc Acc s au support lectronique Les clients Oracle qui ont souscrit un contrat de support ont acc s au support lectronique via My Oracle Support Pour plus d informations visitez le site http www oracle com pls topic lookup ctx acc amp id info ou le site http www oracle com pls topic lookup ctx acc amp id trs si vous tes malentendant Contents Using This Documentation vii 11 Identifying Components ee rreeereree aan erea
71. e processor module 9 Determine your next step m Ifyou are replacing DIMMs see Servicing DIMMs m f you are installing new processor modules to upgrade the server return to Server Upgrade Process on page 59 m If you are replacing a faulty processor module follow these steps a Remove all of the DIMMs from the faulty processor module and set them in a safe place See Remove a DIMM or DIMM Filler Panel on page 76 b Install the DIMMs into the new processor module See Install a DIMM on page 78 c Install the processor module See Install a Processor Module or Processor Filler Module on page 67 66 SPARC T5 8 Server Service Manual e November 2015 Install a Processor Module or Processor Filler Module Related Information Determine Which Processor Module Is Faulty on page 62 Install a Processor Module or Processor Filler Module on page 67 Verify the Processor Module on page 70 Server Upgrade Process on page 59 Y Install a Processor Module or Processor Filler Module 1 Determine your first step If you are replacing the cover as part of the upgrade process or if you are servicing a processor module go to Step 2 If you are installing new processor modules as part of the upgrade process go to Step 4 2 Place the cover back onto the processor module and slide the cover forward until the latch clicks into place 3 Determine your next step If
72. embly ei 130 Y Verify the Front I O Assembly eee 131 Servicing Power Supplies ccccccccececceececeeeceeeeeceeeeeeeeeeeaeeeseaneeaeeseeaa ees 133 Power Supply LEEDS tran arri 134 Y Determine Which Power Supply Is Faulty 134 Y Remove a Power Supply erre eee rece niee rien eee n ice rice rineo 135 Y Installa Power Supply oseere rai ina ra 137 Y Verify the Power Supply eens eeeeeeeeeeseeeeeeees 138 Servicing Fan Modules erre rerrer err re rare rrare ce rerenanea 141 Fan Module LED aaa ie 141 Y Determine Which Fan Module Is Faulty ie erre rear 142 Y Remove a Fan Module L iii 143 Y Install a Fan Module nette a vien eci zione 145 Y Verify the Fan Module e 146 Servicing PCle Cards nannan NERA AI AVRA Lasso as cag atas Uia asa 149 Understanding PCIe Root Complex Connections e 149 Root Complex Connections Four Processor Modules 150 Root Complex Connections Two Processor Modules 151 PCIe Card Installation Guidelines ei 153 PCIe Carrier LEDs and Controls sccseccssccncceeseesccesscnsccusscusceesceesenesenees 154 Y Determine Which PCle Card Is Faulty er ereerereranea 155 Y Remove a Card
73. ervice component or if you are upgrading the server disconnect the power cords See Disconnect the Power Cords on page 56 Related Information Power Off the Server SP Command on page 54 Disconnect the Power Cords on page 56 Configuring Boot and Restart Behavior in SPARC and Netra SPARC T5 Series Servers Administration Guide Power Off the Server Emergency Shutdown This procedure places the system in the power standby mode To service cold replaceable components you must remove the power Preparing for Service 55 Disconnect the Power Cords 56 A Caution All applications and files are closed abruptly without saving changes File system corruption might occur Press and hold the Power button for four seconds If you are servicing a cold service component or if you are upgrading the server disconnect the power cords See Disconnect the Power Cords on page 56 Related Information Prepare to Power Off the Server on page 54 Disconnect the Power Cords on page 56 Configuring Boot and Restart Behavior in SPARC and Netra SPARC T5 Series Servers Administration Guide Disconnect the Power Cords Ensure that you have shut down the system See m Power Off the Server SP Command on page 54 Power Off the Server Power Button Graceful on page 55 Disconnect all of the power cords Caution Because 3 3v standby power is always prese
74. essor module is in standby mode If the server is powered on then this indicates that the processor module is powered down the blue Ready to Remove LED will be lit in this case Related Information Remove a Processor Module or Processor Filler Module on page 63 m Verify the Processor Module on page 70 Y Determine Which Processor Module Is Faulty 62 The following LEDs are lit when a processor module fault is detected Front and rear System Fault Service Required LEDs Service Required LED on the faulty processor module Note A faulty processor module at PMO results in server shutdown and failure to reboot If your server experiences a fault at PMO and you do not have a replacement processor module available you can move one of the other processor modules to PMO and then boot the server in a degraded state Caution In order to maintain system cooling all four processor module slots must be occupied either with a processor module or a processor filler module Determine if the Service Required LEDs are lit on the front panel or the rear I O module See Interpreting LEDs on page 29 From the front of the server check the processor module LEDs to identify which processor module needs to be replaced See Processor Module LEDs on page 61 Remove the faulty processor module See Remove a Processor Module or Processor Filler Module on page 63 Related Information Remove a Pro
75. eturn to the procedure for that component If you are upgrading the server return to Server Upgrade Process on page 59 Related Information m Verify the Battery on page 126 m Verify the DIMM on page 80 m Verify the Processor Module on page 70 m Verify the Rear I O Module on page 178 m Verify the Service Processor Card on page 114 Verify the System Configuration PROM on page 120 m Server Upgrade Process on page 59 SPARC T5 8 Server Service Manual November 2015 Power On the Server Power Button Y Power On the Server Power Button If you are local to the server you can use the Power button to start the system However the start up process depends on the last power state of the system and on Oracle ILOM policy settings After pressing the Power button you will need to check the system to ensure that it is running 1 Connect a terminal device to the server if necessary 2 Momentarily press and release the Power button on the front panel See Front Panel Controls and LEDs on page 31 for the location of the Power button 3 Ensure that the system has been started gt show System power state Ifthe system is running go to the verification task for the component Ifthe system is not running go to Step 3 4 Start the system gt start System 5 Determine your next step If you are servicing a component return to the procedure for that comp
76. f you are replacing a component inside the main module use one of the following links Servicing the Service Processor Card Servicing the Battery Servicing the System Configuration PROM Servicing the Front I O Assembly Servicing the Storage Backplanes Related Information Determine if the Main Module Is Faulty on page 91 Install the Main Module on page 95 94 SPARC T5 8 Server Service Manual e November 2015 Install the Main Module Y Install the Main Module 1 Place the cover back onto the main module and slide the cover forward until the latch clicks into place 2 Open the levers so that they are fully open 3 Insert the main module back into its slot in the server until the levers begin to engage Servicing the Main Module 95 Install the Main Module 4 Press the levers back together toward the center of the module and then press the levers firmly against the module to fully seat the module back into the server The levers should click into place when the module is fully seated in the server 5 Determine your next step a If you replaced an internal component return to the procedure for that component Verify the Battery on page 126 Verify the Front I O Assembly on page 131 m Verify the Service Processor Card on page 114 Verify the Storage Backplane on page 107 m Verify the System Config
77. g this procedure ensure that you are familiar with the cautions and safety instructions described in Safety Information on page 49 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail 1 Take all necessary ESD precautions See Prevent ESD Damage on page 57 2 Orient the system configuration PROM properly onto the main module 3 Press down on the system configuration PROM until it is completely seated on the main module 4 Insert the main module back into the server See Install the Main Module on page 95 Servicing the System Configuration PROM 119 Verify the System Configuration PROM Related Information Remove the System Configuration PROM on page 118 m Verify the System Configuration PROM on page 120 Y Verify the System Configuration PROM 120 1 Ensure that you have completed the following m Applied power to the server See Connect the Power Cords on page 191 m Started the system See Power On the Server Oracle ILOM on page 192 Verify that the banner display includes an Ethernet address and a host ID value The Ethernet address and host ID values are read from the System Configuration PROM Their presence in the banner verifies that the service processor and the host can read the System Configuration PROM SPARC T5 8 No Keyboard OpenBoot X X
78. he server after you manually repair the fault faultmgmtsp gt exit gt reset System Are you sure you want to reset System y Resetting System Clear the fault in the Oracle Enterprise Manager Ops Center software if applicable Clearing a fault with the fmadm aquit command does not clear that fault in the Oracle Enterprise Manager Ops Center software You must manually clear the fault that is incident For more information see 9 9 10 Marking an Incident Repaired in the Oracle Enterprise Manager Ops Center Feature Reference Guide at 44 SPARC T5 8 Server Service Manual e November 2015 Interpreting Log Files and System Messages http ww oracle com pls topic lookup ctx 0c122 6 Determine your next step If you are servicing a component return to the procedure for that component If you are upgrading the server return to Server Upgrade Process on page 59 Related Information PSH Overview on page 41 m Check for Faults on page 41 m Server Upgrade Process on page 59 Interpreting Log Files and System Messages With the OS running on the server you have the full complement of Oracle Solaris OS files and commands available for collecting information and for troubleshooting If PSH does not indicate the source of a fault check the message buffer and log files for notifications for faults Drive faults are usually captured by the Oracle Solaris message files These topics explain how to
79. heal a 195 ING OX esis sates leale O A ri 201 10 SPARC T5 8 Server Service Manual e November 2015 Using This Documentation Overview Describes how to troubleshooot and maintain the server Audience Technicians system administrators and authorized service providers Required knowledge Advanced experience troubleshooting and replacing hardware Product Documentation Library Late breaking information and known issues for this product are included in the documentation library at http www oracle com goto T5 8 docs Feedback Provide feedback about this documentation at http www oracle com goto docfeedback Using This Documentation 11 12 SPARC T5 8 Server Service Manual e November 2015 Identifying Components These topics identify key components of the server including major modules and subassemblies as well as front and rear panel features Front Panel Components Service on page 14 Rear Panel Components Service on page 15 DIMM Locations on page 16 Main Module Internal Component Locations on page 17 Supported Storage Devices on page 18 Fan Module Locations on page 20 Rear I O Module Port Locations on page 21 Chassis Subassembly Components on page 22 Component Service Task Reference on page 23 Related Information Detecting and Managing Faults Preparing for Service Returning the Server to Operatio
80. i 6c0 pci 1 pci 0 pci 4 Note If you are upgrading the server return to Server Upgrade Process on page 59 Related Information Root Complex Connections Four Processor Modules on page 150 PCIe Card Installation Guidelines on page 153 Server Upgrade Process on page 59 PCle Card Installation Guidelines All 16 slots support PCle cards with the following characteristics Type hot plug low profile adapters Mode x8 Genl x8 Gen2 and x8 Gen3 cards Note If you install a 16 lane card in any slot electrical support is provided to the card s lowest 8 lanes Note Some PCle cards are restricted to specific I O slots to meet system cooling requirements Other I O cards provide better performance when installed in particular slots For more information about PCle slot restrictions for specific devices see I O Slot and Quantity Restrictions in SPARC T5 8 Server Product Notes To achieve optimal load balancing in a fully populated server use the following order to install PCle cards To achieve optimal load balancing in a half populated server use the following order to install PCIe cards You might choose to populate the slots differently due to Oracle VM software or redundant failover considerations but load balancing might not be as efficient You should distribute the Servicing PCle Cards 153 PCle Carrier LEDs and Controls PCle cards evenly across the available root
81. ice Task Reference on page 23 Returning the Server to Operation Servicing the System Configuration PROM 117 Remove the System Configuration PROM Y Remove the System Configuration PROM 118 The system configuration PROM is a cold service component that can be replaced only by qualified service personnel Before beginning this procedure ensure that you are familiar with the cautions and safety instructions described in Safety Information on page 49 For the location of the system configuration PROM see Main Module Internal Component Locations on page 17 Caution This procedure involves handling circuit boards that are extremely sensitive to static electricity Ensure that you follow ESD preventative practices to avoid damaging the circuit boards Remove the main module from the server See Remove the Main Module on page 91 Take the necessary ESD precautions See Prevent ESD Damage on page 57 Locate the system configuration PROM on the main module See Main Module Internal Component Locations on page 17 Grasp the system configuration PROM and lift it up to remove it from the main module SPARC T5 8 Server Service Manual e November 2015 Install the System Configuration PROM Related Information Install the System Configuration PROM on page 119 m Verify the System Configuration PROM on page 120 Y Install the System Configuration PROM Before beginnin
82. ice operations storage 81 l T O root complexes default 150 illustrated parts breakdown 22 installing battery 125 DIMMs 78 fan modules 145 front I O assembly 130 hard drives 85 main module 95 PCle carriers 168 power supplies 137 processor modules 67 rear chassis subassembly 185 rear I O module 176 202 SPARC T5 8 Server Service Manual e November 2015 SP card 112 storage backplanes 103 system configuration PROM 119 K Knowledge Base articles 41 L LEDs front panel 31 NET Link and Activity 33 Net Management Link and Activity 33 Net Management Speed 33 NET Speed 33 processor modules 61 rear I O module 33 SP 33 storage 82 System Locator 31 33 System Overtemp 31 33 System Power OK 31 33 System Service Required 31 33 locating AC power connectors 15 fan modules 15 PCIe cards 15 rear I O module 15 locating faulty DIMMs using Fault Remind button 75 usingfmadm faulty command 73 fan modules 142 hard drives 82 PCle cards 155 power supplies 134 processor modules 62 SP 109 log files viewing 46 M main module Index installing 95 locating 14 NAC name 23 removing 91 maximum testing with POST 40 message buffer checking the 45 message identifier 41 motherboard locating 17 NAC name 23 N NET Link and Activity LED 33 Net Management Link and Activity LED 33 Net Management Speed LED 33 NET Speed LED 33 O Oracle Solaris OS fi
83. ices in the device tree hotplug list l grep PCIE This command lists dynamically reconfigurable PCIe resources and shows their operational status In this case look for the status of the card you plan to remove Example pci 300 pci 1 pci 0 pci 6 pci 340 pci 1 pci 0 pci 6 pci 380 pci 1 pci 0 pci a pci 3c0 pci 1 pci 0 pci e pci 400 pci 1 pci 0 pci c pci 440 pci 1 pci 0 pci c pci 480 pci 1 pci 0 pci 4 pci 4c0 pci 1 pci 0 pci 8 pci 500 pci 1 pci 0 pci e pci 540 pci 1 pci 0 pci e pci 580 pci 1 pci 0 pci a pci 5c0 pci 1 pci 0 pci 8 pci 600 pci 1 pci 0 pci 8 pci 640 pci 1 pci 0 pci 8 pci 680 pci 1 pci 0 pci 4 pci 6c0 pci 1 pci 0 pci 4 b Disable the PCle card Example PCIE1 PCIE3 PCIE9 PCIE11 PCIE2 PCIE4 PCIE10 PCIE12 PCIE5 PCIE7 PCIE13 PCIE15 PCIE6 PCIE8 PCIE14 PCIE16 EMPTY ENABLED EMPTY EMPTY EMPTY EMPTY EMPTY EMPTY EMPTY ENABLED EMPTY EMPTY EMPTY EMPTY EMPTY EMPTY hotplug disable pci 340 pci 1 pci 0 pci 6 PCIE3 Power off the PCle card Example hotplug poweroff pci 340 pci 1 pci 0 pci 6 PCIE3 Disconnect any transceivers if applicable and all of the cables connected to the PCle card Tip Label the cables to ensure proper connection to the replacement card Servicing PCle Cards 157 Remove a Card Carrier 8 Pull the carrier s handle down to disengage the carrier from the card cage 9 Remove the carrier
84. ier Extension on page 164 Y Install a Carrier Extension On double wide PCle card carriers the carrier extension provides the additional airflow that is required for proper cooling 1 Ensure that you have removed two PCle card carriers from the server See Remove a Card Carrier on page 155 Retain the extra card carrier in a suitable storage space in case you want to remove the carrier extension 2 Install the PCle card See Install a PCle Card on page 165 3 Align the rear flange on the carrier extension with the rear wall of the main carrier and swing the two components together 164 SPARC T5 8 Server Service Manual e November 2015 Install a PCle Card Related Information Install a PCIe Card on page 165 Install a Card Carrier on page 168 Y Install a PCle Card A Caution This procedure involves handling circuit boards that are extremely sensitive to static electricity Ensure that you follow ESD preventative practices to avoid damaging the circuit boards See Prevent ESD Damage on page 57 Determine your first step a If you are installing a new card and need an empty carrier see Remove a Card Carrier on page 155 b If you are replacing a faulty card and have already removed its carrier from the server go to Step 3 Unlatch and swing open the top of the PCle card carrier Remove the PCle card from its packaging Servicing PCle Cards 165 Install a PCle
85. l the Service Processor Card on page 112 Y Install the Service Processor Card 1 Take appropriate ESD measures See Prevent ESD Damage on page 57 112 SPARC T5 8 Server Service Manual e November 2015 Install the Service Processor Card 2 Lower the side of the service processor card with the Align Tab sticker down on the service processor tab on the motherboard Lower the other side of the service processor card down and press down on the card to seat it into the connectors on the motherboard Install the main module back into the server See Install the Main Module on page 95 Connect a terminal or a terminal emulator PC or workstation to the SER MGT port Return the server to operation See Returning the Server to Operation If the replacement service processor card detects that the SP firmware is not compatible with the existing host firmware further action is suspended and the following message is delivered over the SER MGT port Unrecognized Chassis This module is installed in an unknown or unsupported chassis You must upgrade the firmware to a newer version that supports this chassis If you see this message go on to Step 7 If you do not see this message go to Step 8 Download the system firmware a Configure the network port on the service processor card to enable the firmware image to be downloaded For instructions refer to the Oracle ILOM documentation Servicing th
86. le on page 146 SPARC T5 8 Server Service Manual e November 2015 Remove a Fan Module Y Remove a Fan Module The fan modules are hot service components that can be replaced if there are at least nine operating fan modules For the location of the fan modules see Fan Module Locations on page 20 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail 1 Locate the faulty fan module that you want to remove from the server m See Rear Panel Components Service on page 15 for the locations of the fan modules in the server m See Determine Which Fan Module Is Faulty on page 142 to locate a faulty fan module 2 Determine if you can remove the fan module with the server running You can remove a fan module with the server running only if one fan module is faulty and if you have a replacement fan module If more than one fan module is faulty you must shut down the server If you can remove the fan module with the server running go to Step 3 If you cannot remove the fan module with the server running see Removing Power From the Server on page 53 to power down the server before continuing Servicing Fan Modules 143 Remove a Fan Module 3 Press the release button to disengage the fan latch 4 Pull out on the fan module to remove it from the server 144 SPARC T5 8 Server Service
87. les and commands 45 Oracle Solaris PSH checking for faults 41 clearing faults 44 overview 41 Oracle VTS 25 27 87 overview server upgrade 59 P PCle cards characteristics 153 locating 15 locating faulty 155 NAC name 24 verifying functionality 171 PCle carriers installing 168 locating 15 NAC name 24 numbering 153 removing 155 PCle slots configuration reference 153 port ID values 150 ports front panel 14 rear I O module 21 POST configuration examples 39 configuring 39 overview 35 running in Diag Mode 40 power cords connecting to server 191 power supplies installing 137 locating 14 locating faulty 134 NAC name 24 removing 135 verifying functionality 138 powering off emergency shutdown 55 gracefully with power button 55 using SP command 54 powering on using power button 193 using start SYS command 192 processor modules cover 65 DIMMs 66 installing 67 locating 14 locating faulty 62 NAC name 24 removing 63 verifying functionality 70 PSH checking for faults 41 clearing faults 44 DIMMs 73 80 97 message ID 41 overview 41 storage backplane 107 R Ready to Remove LED storage 82 203 Index rear chassis subassembly components 16 installing 185 NAC name 24 rear components 15 Rear Fan Module Fault LED 31 rear I O module installing 176 LEDs 33 locating 15 NAC name 24 ports 21 removing 174 verifying func
88. lies and all four power cords must be plugged in For the location of the power supplies see Front Panel Components Service on page 14 Note To take full advantage of the server s power redundancy plug the power supplies into separate power sources wired to two separate circuits using a 2 2 configuration Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause failure of server components These topics describe service procedures for the power supplies in the server Steps Description Links 1 Understand the power supply LEDs Power Supply LEDs on page 134 1 Determine which power supply is faulty Determine Which Power Supply Is Faulty on page 134 25 Replace the power supply m Remove a Power Supply on page 135 m Install a Power Supply on page 137 3 Verify the replacement Verify the Power Supply on page 138 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Servicing Power Supplies 133 Power Supply LEDs Power Supply LEDs L OVEA S Q 17 gt No LED Icon Description 1 Service The Service Required LED lights when the power supply is faulty RequiredLED amber and Note The front an
89. lled successfully Related Information DIMM Configuration on page 71 DIMM Configuration Fault Messages on page 76 m Install a DIMM on page 78 80 SPARC T5 8 Server Service Manual e November 2015 Servicing Hard Drives The storage devices in the server are hot serviceable meaning that the devices can be removed and inserted while the server is powered on depending on the state of the device and the configuration of the data on that device A hard drive is hot pluggable if the drive is in slot 1 to 7 The hard drive in slot O cannot be removed without shutting down the server unless it is configured with an alternative I O path Taking a drive offline prevents any applications from accessing it and removes the logical software links to it The following situations inhibit your ability to hot service a drive Ifthe drive contains the operating system and the operating system is not mirrored on another drive m If the drive cannot be logically isolated from the online operations of the server If either of these conditions apply to the drive being serviced you must take the server offline shut down the operating system before you replace the drive For the location of the hard drives see Supported Storage Devices on page 18 These topics describe service procedures for the hard drives in the server Step Description Links 1 Understand the hard drive LEDs Hard Drive LEDs on page 8
90. llustrates the root complex connections between the four processor modules and the 16 PCle I O slots in a fully populated server Each CPU supports two I O root complexes Each root complex connects to two I O slots through two of five multiplexing switches ET gn o A PMO PM1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 PCle Slots The port ID values shown in the diagram correspond to the pci values reported in the OpenBoot show devs command output For example PM CPU Switch HO Slot Root Complex Path 0 0 0 1 pci 300 pci l pci 0 pci 6 SPARC T5 8 Server Service Manual e November 2015 Understanding PCle Root Complex Connections PM CPU Switch HO Slot Root Complex Path 0 0 1 3 pci 340 pci 1 pci 0 pci 6 0 1 2 9 pci 380 pci 1 pci 0 pci a 0 1 3 11 pci 3c0 pci 1 pci 0 pci e 1 2 0 2 pci 400 pci 1 pci 0 pci c 1 2 1 4 pci 440 pci 1 pci 0 pci c 1 3 2 10 pci 480 pci 1 pci 0 pci 4 T 3 3 12 pci 4c0 pci 1 pci 0 pci 8 2 4 1 5 pci 500 pci 1 pci 0 pci e 2 4 2 7 pci 540 pci 1 pci 0 pci e 2 5 3 13 pci 580 pci 1 pci 0 pci a 2 gt 4 15 pci 5c0 pci 1 pci 0 pci 8 3 6 1 6 pci 600 pci 1 pci 0 pci 8 3 6 2 8 pci 640 pci 1 pci 0 pci 8 3 7 3 14 pci 680 pci 1 pci 0 pci 4 3 7 4 16 pci 6c0 pci 1 pci 0 pci 4 Note If you are upgrading the server return to Server Upgrade Process on page 59 Related Information Root Complex Connections Two Processor Modules on page 151 PCle Card Ins
91. location of DIMMs see DIMM Locations on page 16 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe service procedures for the DIMMs in the server Step Description Links 1 Understand how to configure the DIMM Configuration on page 71 DIMMs 2 Locate a faulty DIMM m Determine Which DIMM Is Faulty FMA on page 73 m Determine Which DIMM Is Faulty LEDs on page 75 m DIMM Configuration Fault Messages on page 76 3 Replace a DIMM m Remove a DIMM or DIMM Filler Panel on page 76 m Install a DIMM on page 78 m Verify the DIMM on page 80 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation DIMM Configuration Consider these topics when installing upgrading or replacing DIMMs Servicing DIMMs 71 DIMM Configuration DIMM Guidelines You must follow these guidelines m Use either 16 or 32 Gbyte DDR3 DIMM capacity DIMMs Use Oracle qualified DIMMs Fully populate 32 DIMMs all processor modules Caution If you ordered processor modules without memory to upgrade the server from a half populated configuration to a fully populated configuration you must install the same t
92. ly Servicing Fan Modules Servicing PCle Cards Servicing the Rear I O Module Servicing Power Supplies 22 SPARC T5 8 Server Service Manual e November 2015 Component Service Task Reference Component Service Task Reference Related Information Front Panel Components Service on page 14 Rear Panel Components Service on page 15 Component Service Task Reference on page 23 This table lists the names of server components that you can service It also lists the system names and task locations for the components Component Max NAC Name SDM Name Notes Links Battery 1 SYS MB BAT Servicing the Battery Chassis 1 SYS System Refer to the SPARC T5 Server Installation Guide DIMMs 128 SYS System Memory DIMMs 16 or 32 GB Servicing DIMMs PMx CMx CMP BOBx CHx DO DIMM x Disk drives 8 SYS SASBPx HDDx System Storage Disks SAS 300 GB or Servicing Hard Drives Fan modules Front I O assembly Main module motherboard Disks x 10 SYS RCSA FBDx FMx Fan x 1 SYS FIO 1 SYS MB System Cooling Fans 600 GB or SSD 100 GB or 300 GB These components are incuded m FIO board with FRU PROM m VGA board no FRU PROM m FIO enclosure with cables These internal components must be reused E Front I O module m Service processor m Disk backplanes 2 SCC PROM Battery Servicing Fan Modules Servicing the
93. ment Follow all cautions and instructions marked on the equipment and described in the documentation shipped with your server Follow all cautions and instructions marked on the equipment and described in the SPARC T5 8 Server Safety and Compliance Guide Ensure that the voltage and frequency of your power source match the voltage and frequency inscribed on the equipment s electrical rating label m Follow the ESD safety practices as described in this section Preparing for Service 49 Safety Information Safety Symbols Note the meanings of the following symbols that might appear in this document Caution There is a risk of personal injury or equipment damage To avoid personal injury and equipment damage follow the instructions Caution Hot surface Avoid contact Surfaces are hot and might cause personal injury if touched Caution Hazardous voltages are present To reduce the risk of electric shock and danger to personal health follow the instructions gt gt gt ESD Measures ESD sensitive devices such as networking adapters hard drives and DIMMs require special handling A Caution Circuit boards and hard drives contain electronic components that are extremely sensitive to static electricity Ordinary amounts of static electricity from clothing or the work environment can destroy the components located on these boards Do not touch the components along their connector edges A Caution You mu
94. might indicate that server diagnostics are running or that the server is booting 4 Power button The recessed Power button toggles the server on or off See Power Off the Server Q Power Button Graceful on page 55 5 System Overtemp Indicates these conditions LED amber m Off Indicates a steady state no service action is required m Steady on Indicates that a temperature failure event has been acknowledged and a service action is required 6 Fan Module Fault Rear FM Indicates these conditions LED amber m Off Indicates a steady state no service action is required m Steady on Indicates that a fan module failure event has been acknowledged and a service action is required on at least one of the fan modules 7 PCle Card Fault Rear PCle Indicates these conditions LED amber m Off Indicates a steady state no service action is required m Steady on Indicates that a failure event has been acknowledged and a service action is required on at least one of the PCIe cards 32 Related Information Rear Panel Controls and LEDs on page 33 Understanding Diagnostics on page 25 SPARC T5 8 Server Service Manual e November 2015 Interpreting LEDs No Rear Panel Controls and LEDs Icon or Label Description 6 AC 0 left and AC 1 right Indicates these conditions power LED m Off No power is applied to the ser
95. n Identifying Components 13 Front Panel Components Service Front Panel Components Service No Description Links 1 Hard drives 8 Servicing Hard Drives Supported Storage Devices on page 18 2 Processor modules 0 to 3 bottom to top or Servicing Processor Modules processor filler modules slot 1 and slot 2 Server Upgrade Process on page 59 3 Main module Servicing the Main Module 4 Power supplies 0 to 3 left to right Servicing Power Supplies Related Information Rear Panel Components Service on page 15 14 SPARC T5 8 Server Service Manual e November 2015 Rear Panel Components Service Chassis Subassembly Components on page 22 Rear Panel Components Service After you install the server into the rack you must access these components from the rear of the server These components are not part of the rear chassis subassembly You must remove these components to access the rear chassis subassembly No Description Links Fan modules 10 AC power connectors 0 to 3 right to left Rear I O module PCle carriers 1 to 16 left to right Servicing Fan Modules Fan Module Locations on page 20 Servicing the Rear Chassis Subassembly Servicing the Rear I O Module Servicing PCle Cards Iden
96. n Link 1 Remove the upgrade components from their packaging and place them on an antistatic mat di Remove the covers from the new processor modules Step 8 in Remove a Processor Module or Processor Filler Module on page 63 3 Remove all of the DIMM filler panels in the processor modules Remove a DIMM or DIMM Filler Panel on page 76 The steps to remove DIMM filler panels are the same as the steps for removing DIMMs 4 Install the DIMMs All of the DIMMs must be either 16 or 32 GB Install a DIMM on page 78 and they must match the size and capacity of the DIMMs that are already installed in the server 5 Replace the covers on the new processor modules Step 1 in Install a Processor Module or Processor Filler Module on page 67 6 Check for faults If any fault is present you must correct the fault Check for Faults on page 41 and clear it from the server Vi Shut down the server Removing Power From the Server on page 53 8 Remove the processor filler modules from slot 1 and slot 2 Remove a Processor Module or Processor Filler Module on page 63 9 Install the new processor modules in slot 1 and slot 2 Install a Processor Module or Processor Filler Module on page 67 10 Return the server to operation Returning the Server to Operation 11 Check for faults If any fault is present you must correct the fault Check for Faults on page 41 and clear it from the server 12 Review the r
97. n of the rear I O module see Rear I O Module Port Locations on page 21 Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail Take the necessary ESD precautions See Prevent ESD Damage on page 57 Remove all of the power from the server See Removing Power From the Server on page 53 Locate the rear I O module See Rear Panel Components Service on page 15 for the location of the rear I O module in the server Label the cables connected to the ports on the rear I O module and then disconnect the cables from the ports You will reconnect the cables to the same ports on the replacement rear I O module SPARC T5 8 Server Service Manual e November 2015 Remove the Rear I O Module 5 Press the green buttons on the rear I O module ejection levers and spread the levers open to eject the rear I O module Servicing the Rear I O Module 175 Install the Rear I O Module 6 Pull the module away from the server to remove it Related Information Determine if the Rear I O Module Is Faulty on page 173 Install the Rear I O Module on page 176 Verify the Rear I O Module on page 178 Y Install the Rear I O Module 1 Take the necessary ESD precautions See Prevent ESD Damage on page 57 176 SPARC T5 8 Server Service Manual e November 20
98. n page 78 Related Information DIMM Configuration on page 71 Determine Which DIMM Is Faulty FMA on page 73 Determine Which DIMM Is Faulty LEDs on page 75 Install a DIMM on page 78 Server Upgrade Process on page 59 Install a DIMM A Before beginning this procedure ensure that you are familiar with the information provided in these topics m Safety Information on page 49 DIMM Configuration on page 71 Take the necessary ESD precautions See Prevent ESD Damage on page 57 Ensure that you have removed the processor module if you are replacing a faulty DIMM See Remove a Processor Module or Processor Filler Module on page 63 Ensure that you have removed the faulty DIMM See Remove a DIMM or DIMM Filler Panel on page 76 Unpack the replacement DIMM and place it on an antistatic mat Caution If you ordered processor modules without memory to upgrade the server from a half populated configuration to a fully populated configuration you must install the same size and capacity of DIMMs that are already in the existing processor modules SPARC T5 8 Server Service Manual e November 2015 Install a DIMM Ensure that the ejector tabs on the connector that will receive the DIMM are in the open position Align the DIMM notch with the key in the connector Caution Ensure that the orientation is correct The DIMM might be damaged if the orientation
99. nfiguration PROM 7 Battery Servicing the Battery Supported Storage Devices The server supports these storage devices Fibre channel arrays SATA FC flash and SAS 2 m SAS arrays SAS 2 ZFS appliances SAS 2 The server also supports these types of tape backup and restore devices 18 SPARC T5 8 Server Service Manual e November 2015 Supported Storage Devices TCP IP Fibre channel m SAS LVD SCSI You can install a mixture of storage devices but the server requires at least one storage device to be installed and operational 2 9 Description Link Drive 1 Servicing Hard Drives Drive 0 Drive 3 Drive 2 Drive 5 Drive 4 Drive 7 Drive 6 o Ny Do A WU Ne Identifying Components 19 Fan Module Locations Fan Module Locations No Description Link Fan module 5 Servicing Fan Modules Fan module 0 Fan module 6 Fan module 1 Fan module 7 Fan module 2 Fan module 8 Fan module 3 Oo AN DH BW NY Fan module 9 jat o Fan module 4 20 SPARC T5 8 Server Service Manual e November 2015 Rear I O Module Port Locations Rear I O Module Port Locations The rear I O module is located on the rear panel above the AC power connectors The rear I O module provides access to all of the internal PCle ports for the server and the video and USB ports for input devices
100. ning panel to secure it to the top of the hard drive assembly Servicing the Storage Backplanes 105 Install a Storage Backplane 5 Connect the two storage backplane cables to the storage backplane and the motherboard No Description 1 Data cable 2 Power cable 6 Install the main module back into the server See Install the Main Module on page 95 7 Install the hard drives that you removed back into the main module Refer to the notes that you took when removing the hard drives to install them back into their original slots See Install a Hard Drive on page 85 8 Power on the server See Returning the Server to Operation 9 Verify the installation See Verify the Storage Backplane on page 107 106 SPARC T5 8 Server Service Manual e November 2015 Verify the Storage Backplane Related Information Remove a Storage Backplane on page 99 m Verify the Storage Backplane on page 107 Y Verify the Storage Backplane 1 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 m Started the system See Power On the Server Oracle ILOM on page 192 2 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 3 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt 4 Use the fmadm f
101. nse terms and license restrictions applicable to the programs No other rights are granted to the U S Government Ce logiciel ou mat riel a t d velopp pour un usage g n ral dans le cadre d applications de gestion des informations Ce logiciel ou mat riel n est pas con u ni n est destin tre utilis dans des applications a risque notamment dans des applications pouvant causer des dommages corporels Si vous utilisez ce logiciel ou mat riel dans le cadre d applications dangereuses il est de votre responsabilit de prendre toutes les mesures de secours de sauvegarde de redondance et autres mesures n cessaires a son utilisation dans des conditions optimales de s curit Oracle Corporation et ses affili s d clinent toute responsabilit quant aux dommages caus s par utilisation de ce logiciel ou mat riel pour ce type d applications Oracle et Java sont des marques d pos es d Oracle Corporation et ou de ses affili s Tout autre nom mentionn peut correspondre des marques appartenant d autres propri taires qu Oracle Intel et Intel Xeon sont des marques ou des marques d pos es d Intel Corporation Toutes les marques SPARC sont utilis es sous licence et sont des marques ou des marques d pos es de SPARC International Inc AMD Opteron le logo AMD et le logo AMD Opteron sont des marques ou des marques d pos es d Advanced Micro Devices UNIX est une marque d pos e d The Open Group C
102. nt in the server you must unplug the power cords before accessing any cold serviceable components See Component Service Categories on page 52 Determine your next step m If you are servicing a component return to the procedure for that component If you are upgrading the server return to Server Upgrade Process on page 59 Related Information Prepare to Power Off the Server on page 54 Power Off the Server SP Command on page 54 Power Off the Server Power Button Graceful on page 55 Power Off the Server Emergency Shutdown on page 55 SPARC T5 8 Server Service Manual e November 2015 Prevent ESD Damage Prevent ESD Damage on page 57 Server Upgrade Process on page 59 Y Prevent ESD Damage Many components housed within the chassis can be damaged by ESD To protect these components from damage perform the following steps before opening the chassis for service 1 Prepare an antistatic surface to set parts on during the removal installation or replacement process Place ESD sensitive components such as the printed circuit boards on an antistatic mat The following items can be used as an antistatic mat Antistatic bag used to wrap a replacement part ESD mat A disposable ESD mat shipped with some replacement parts or optional server components 2 Attach an antistatic wrist strap When servicing or removing server components attach an antist
103. on of the programs including any operating system integrated software any programs installed on the hardware and or documentation shall be subject to license terms and license restrictions applicable to the programs No other rights are granted to the U S Government This software or hardware is developed for general use in a variety of information management applications It is not developed or intended for use in any inherently dangerous applications including applications that may create a risk of personal injury If you use this software or hardware in dangerous applications then you shall be responsible to take all appropriate fail safe backup redundancy and other measures to ensure its safe use Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications Oracle and Java are registered trademarks of Oracle and or its affiliates Other names may be trademarks of their respective owners Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc AMD Opteron the AMD logo and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices UNIX is a registered trademark of The Open Group This software or hardware and documentation may provide access to or information about content products and
104. onent If you are upgrading the server return to Server Upgrade Process on page 59 Related Information m Verify the Battery on page 126 m Verify the DIMM on page 80 m Verify the Processor Module on page 70 m Verify the Rear I O Module on page 178 m Verify the Service Processor Card on page 114 Verify the System Configuration PROM on page 120 m Server Upgrade Process on page 59 Returning the Server to Operation 193 194 SPARC T5 8 Server Service Manual e November 2015 A ANSISIS ASF AWG B blade blade server BMC BOB C chassis CMA CMM CMP Glossary American National Standards Institute Status Indicator Standard Alert standard format Netra products only American wire gauge Generic term for server modules and storage modules See server module and storage module Server module See server module Baseboard management controller Memory buffer on board For servers refers to the server enclosure For server modules refers to the modular system enclosure Cable management assembly Chassis monitoring module server modules only The CMM is the service processor in the modular system that contains server modules Oracle ILOM runs on the CMM providing lights out management of the components in the modular system chassis See modular system and Oracle ILOM Chip multiprocessor Glossary 195 DHCP D DHCP disk module o
105. oot complex changes Root Complex Connections Four Processor Modules on page 150 13 Review the PCle card load balancing changes Even though the PCle Card Installation Guidelines on page 153 load balancing guidelines change with the upgrade you do not need to move any existing PCle cards 60 SPARC T5 8 Server Service Manual e November 2015 Processor Module LEDs Related Information Remove a DIMM or DIMM Filler Panel on page 76 m Install a DIMM on page 78 m Check for Faults on page 41 Removing Power From the Server on page 53 Remove a Processor Module or Processor Filler Module on page 63 Install a Processor Module or Processor Filler Module on page 67 Returning the Server to Operation Root Complex Connections Four Processor Modules on page 150 PCIe Card Installation Guidelines on page 153 Processor Module LEDs No LED Icon Description 1 Ready to Remove blue Indicates that a processor module can be Va removed 2 Service Required amber Indicates that the processor module has experienced a fault condition 3 OK green Indicates if the processor module is available for use m On The server is running and the processor module is powered up Servicing Processor Modules 61 Determine Which Processor Module Is Faulty No LED Icon Description m Off The server is powered down and the proc
106. or Module LEEDS iaia ia la 61 Y Determine Which Processor Module Is Faulty iii 62 Y Remove a Processor Module or Processor Filler Module 63 v Install a Processor Module or Processor Filler Module 67 Y Verify the Processor Module ee 70 Servicing DIMMS eei ilaele iau 71 DIMM Gonfiguration lt il 71 DIMM Guidelinesz p lla eri ii 72 DIMM Locations siii ini 72 Y Determine Which DIMM Is Faulty FMA eee eeererana 73 Y Determine Which DIMM Is Faulty LEDS ii 75 6 SPARC T5 8 Server Service Manual November 2015 Contents DIMM Configuration Fault Messages 76 Y Remove a DIMM or DIMM Filler Panel rena 76 y Install a DIMM tasas ieii he taki leslie la iaia alla 78 Verify the DIMM su ccc asso oa lie studio asas a do mudados o ad aaa 80 Servicing Hard Drives saias nisi ea aci 81 Hard Drive LEDS uma leleine iaia 82 Y Determine Which Hard Drive Is Faulty er ereree means 82 Y Remove a Hard Drive i 83 y Inst l Hard Drive ki iii lie iti 85 yY Verify the Hard Drive srein iriso ieiki sue EEEE ET E AAEE EAEE 86 Servicing the Main Module i 89 Main Module LEEDS x lunario 90 Y Determine if the Main Module Is Faulty ser ererraaa 91 Y Remove the Main Module
107. or Modules 63 Remove a Processor Module or Processor Filler Module 4 Squeeze the release latches together on the two extraction levers and pull the extraction levers out to disengage the processor module or processor filler module from the server 5 Pull the processor module or processor filler module halfway out of the server and then close the levers 64 SPARC T5 8 Server Service Manual e November 2015 Remove a Processor Module or Processor Filler Module This will keep the levers from getting damaged when you remove the module from the server Caution Do not touch the connectors at the rear of the module Using two hands completely remove the processor module or processor filler module and place the module on an antistatic mat Determine your next step If you are replacing DIMMs in an existing processor module go to Step 8 If you are upgrading the server to a fully populated configuration repeat Step 4 through Step 6 to remove the second processor filler module in slot 2 and then go to Server Upgrade Process on page 59 Remove the cover Servicing Processor Modules 65 Remove a Processor Module or Processor Filler Module a Press down on the green button at the top of the cover to disengage the cover from the processor module or modules b Keeping the button pressed down push the cover toward the rear of the processor module and lift the cover up and away from th
108. part of another Install the Main Module on page 95 component s service operation Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Servicing the Main Module 89 Main Module LEDs Main Module LEDs No LED Icon Description 1 Service Required LED Indicates that service is required amber The fmadm faulty command provides details about any faults that cause this indicator to light Under some fault conditions individual component fault LEDs are illuminated in addition to the Service Required LED 2 Power OK LED green Indicates these conditions OK m Off System is not running in its normal state System power might be off The SP might be running m Steady on System is powered on and is running in its normal operating state No service actions are required m Fast blink System is running in standby mode and can be quickly returned to full function m Slow blink A normal but transitory activity is taking place Slow blinking might indicate that system diagnostics are running or that the system is booting 3 SP LED SP Indicates these conditions m Off The AC power might have been disconnected to the power supplies m Steady on green SP is running in its normal operating state No service action
109. perties That Affect POST Behavior on page 35 If the virtual keyswitch is set to normal and you want to define the mode level verbosity Or trigger set the respective parameters Syntax set HOST diag property value See Oracle ILOM Properties That Affect POST Behavior on page 35 for a list of parameters and values Examples gt set HOST diag mode normal gt set HOST diag verbosity max View the current values for settings Example gt show HOST diag HOST diag Targets Properties error reset level max error reset verbosity normal hw change level max hw change verbosity normal level min mode normal power on level max power on verbosity normal trigger hw change error reset verbosity normal Detecting and Managing Faults 39 Run POST With Maximum Testing 40 Commands cd set show Related Information m POST Overview on page 35 Oracle ILOM Properties That Affect POST Behavior on page 35 Run POST With Maximum Testing on page 40 Oracle ILOM Documentation Library Run POST With Maximum Testing This procedure describes how to configure the server to run the maximum level of POST Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Set the virtual keyswitch to diag so that POST runs in service mode Alternatively you can use the System target gt set HOST keyswitch_state diag Set keyswitch state to
110. r SP Command You can use the SP to perform a graceful shutdown of the system This type of shutdown ensures that all of your data is saved and that the system is ready for restart Note Additional information about powering off the system is provided in the SPARC T5 Series Servers Administration Guide Log in as superuser or equivalent Depending on the type of problem you might want to view system status or log files You also might want to run diagnostics before you shut down the system SPARC T5 8 Server Service Manual e November 2015 Power Off the Server Power Button Graceful Switch from the system console to the Oracle ILOM prompt by typing the Hash Period key sequence At the Oracle ILOM prompt type the stop System command If you are servicing a cold service component or if you are upgrading the server disconnect the power cords See Disconnect the Power Cords on page 56 Related Information Power Off the Server Power Button Graceful on page 55 Disconnect the Power Cords on page 56 Configuring Boot and Restart Behavior in SPARC and Netra SPARC T5 Series Servers Administration Guide Power Off the Server Power Button Graceful This procedure places the system in the power standby mode To service cold replaceable components you must remove the power Press and release the recessed Power button The Power OK LED blinks rapidly If you are servicing a cold s
111. r modifier breveter transmettre distribuer exposer ex cuter publier ou afficher le logiciel m me partiellement sous quelque forme et par quelque proc d que ce soit Par ailleurs il est interdit de proc der toute ing nierie inverse du logiciel de le d sassembler ou de le d compiler except des fins d interop rabilit avec des logiciels tiers ou tel que prescrit par la loi Les informations fournies dans ce document sont susceptibles de modification sans pr avis Par ailleurs Oracle Corporation ne garantit pas qu elles soient exemptes d erreurs et vous invite le cas ch ant a lui en faire part par crit Si ce logiciel ou la documentation qui l accompagne est conc d sous licence au Gouvernement des Etats Unis ou a toute entit qui d livre la licence de ce logiciel ou utilise pour le compte du Gouvernement des Etats Unis la notice suivante s applique U S GOVERNMENT END USERS Oracle programs including any operating system integrated software any programs installed on the hardware and or documentation delivered to U S Government end users are commercial computer software pursuant to the applicable Federal Acquisition Regulation and agency specific supplemental regulations As such use duplication disclosure modification and adaptation of the programs including any operating system integrated software any programs installed on the hardware and or documentation shall be subject to lice
112. r disk blade DTE EIA ESD FEM FRU HBA host hot pluggable hot swappable ID PROM IP Dynamic Host Configuration Protocol Interchangeable terms for storage module See storage module Data terminal equipment Electronics Industries Alliance Electrostatic discharge Fabric expansion module server modules only FEMs enable server modules to use the 10GbE connections provided by certain NEMs See NEM Field replaceable unit Host bus adapter The part of the server or server module with the CPU and other hardware that runs the Oracle Solaris OS and other applications The term host is used to distinguish the primary computer from the SP See SP Describes a component that can be replaced with power applied but the component must be prepared for removal Describes a component that can be replaced with power applied and no preparation is required Chip that contains system information for the server or server module Internet Protocol 196 SPARC T5 8 Server Service Manual e November 2015 OBP KVM LwA M MAC MAC address modular system MSGID N name space NEBS NEM NET MGT NIC NMI OBP Keyboard video mouse Refers to using a switch to enable sharing of one keyboard one display and one mouse with more than one computer Sound power level Machine access code Media access controller address The rackmountable chassis that holds server modules
113. rade overview 59 Index upgrading to a fully populated configuration 59 USB ports 21 using Oracle VTS 25 27 UUID 41 V var adm messages file 46 verifying functionality DIMMs 80 fan modules 146 hard drives 86 PCle cards 171 power supplies 138 processor modules 70 rear I O module 178 SP 114 video port 21 viewing system message log files 46 205 206 SPARC T5 8 Server Service Manual e November 2015
114. red LEDs are lit on the front panel or the rear I O module See Interpreting LEDs on page 29 2 From the front of the server check the drive LEDs to identify which drive needs to be replaced See Hard Drive LEDs on page 82 3 Remove the faulty drive See Remove a Hard Drive on page 83 Related Information Remove a Hard Drive on page 83 m Verify the Hard Drive on page 86 Y Remove a Hard Drive Hard drives are hot service components if they are in slots 1 to 7 The hard drive in slot O cannot be removed unless it has an alternate I O path A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail 1 Locate the drive in the server that you want to remove m See Front Panel Components Service on page 14 for the locations of the drives in the server m See Determine Which Hard Drive Is Faulty on page 82 to locate a faulty drive 2 Determine if you need to shut down the OS to replace the drive and perform one of the following actions m Ifthe drive cannot be taken offline without shutting down the OS follow instructions in Power Off the Server SP Command on page 54 and then go to Step 4 m Ifthe drive can be taken offline without shutting down the OS go to Step 3 3 Take the drive offline Servicing Hard Drives 83 Remove a Hard Drive 84 Cc
115. rnatively you can turn off the Locator LED by running the Oracle ILOM set SYS LOCATE value of f command Removing Power From the Server These topics describe different methods for removing power from the chassis Prepare to Power Off the Server on page 54 Preparing for Service 53 Prepare to Power Off the Server 54 m Power Off the Server SP Command on page 54 Power Off the Server Power Button Graceful on page 55 Power Off the Server Emergency Shutdown on page 55 Disconnect the Power Cords on page 56 Prepare to Power Off the Server Notify any affected users that the server will be shut down Refer to the Oracle Solaris system administration documentation for additional information Save any open files and quit all running programs Refer to your application documentation for specific information for these processes Shut down all logical domains Refer to the Oracle VM system administration documentation for details Shut down the Oracle Solaris OS Refer to the Oracle Solaris administration documentation for details Power off the server Related Information m Power Off the Server SP Command on page 54 Power Off the Server Power Button Graceful on page 55 Power Off the Server Emergency Shutdown on page 55 Configuring Boot and Restart Behavior in SPARC and Netra SPARC TS Series Servers Administration Guide Power Off the Serve
116. s are required m Blink green SP is initializing the Oracle ILOM firmware 90 SPARC T5 8 Server Service Manual e November 2015 Determine if the Main Module Is Faulty No LED Icon Description m Steady on amber A SP error has occurred and service is required Related Information Determine if the Main Module Is Faulty on page 91 Remove the Main Module on page 91 Y Determine if the Main Module Is Faulty Check the Service Required and SP LEDs on the main module See Main Module LEDs on page 90 Related Information Remove the Main Module on page 91 m Verify the Main Module on page 97 Y Remove the Main Module The main module is a cold service component that can be replaced only after you have powered off the server A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail 1 Optional If you are replacing a faulty main module you must back up ILOM configuration settings a Configure the SER MGT port to enable the configuration parameters to be uploaded Refer to the ILOM documentation for network configuration instructions b Back up the ILOM configuration parameters See Oracle ILOM documentation 2 Shut down the server Servicing the Main Module 91 Remove the Main Module See Removing Power From the Server on page 53 3 Loc
117. s enabled hotplug list l grep PCIE Example pci 300 pci 1 pci 0 pci 6 pci 340 pci 1 pci 0 pci 6 pci 380 pci 1 pci 0 pci a pci 3c0 pci 1 pci 0 pci e pci 400 pci 1 pci 0 pci c pci 440 pci 1 pci 0 pci c pci 480 pci 1 pci 0 pci 4 pci 4c0 pci 1 pci 0 pci 8 pci 500 pci 1 pci 0 pci e pci 540 pci 1 pci 0 pci e pci 580 pci 1 pci 0 pci a pci 5c0 pci 1 pci 0 pci 8 pci 600 pci 1 pci 0 pci 8 pci 640 pci 1 pci 0 pci 8 pci 680 pci 1 pci 0 pci 4 pci 6c0 pci 1 pci 0 pci 4 Related Information PCIE1 EMPTY PCIE3 ENABLED PCIE9 EMPTY PCIE11 EMPTY PCIE2 EMPTY PCIE4 EMPTY PCIE10 EMPTY PCIE12 EMPTY PCIE5 EMPTY PCIE7 ENABLED PCIE13 EMPTY PCIE15 EMPTY PCIE6 EMPTY PCIE8 EMPTY PCIE14 EMPTY PCIE16 EMPTY Determine Which PCIe Card Is Faulty on page 155 Remove a Card Carrier on page 155 Install a Card Carrier on page 168 172 SPARC T5 8 Server Service Manual e November 2015 Servicing the Rear I O Module The rear I O module is a cold service component that can be replaced only after you have removed all of the power from the server For the location of the PCIe cards see Rear I O Module Port Locations on page 21 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe service procedures for the rear I O module in the server
118. services from third parties Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third party content products and services unless otherwise set forth in an applicable agreement between you and Oracle Oracle Corporation and its affiliates will not be responsible for any loss costs or damages incurred due to your access to or use of third party content products or services except as set forth in an applicable agreement between you and Oracle Documentation Accessibility For information about Oracle s commitment to accessibility visit the Oracle Accessibility Program website at http www oracle com pls topic lookup ctx acc amp id docacc Access to Oracle Support Oracle customers that have purchased support have access to electronic support through My Oracle Support For information visit http www oracle com pls topic lookup ctx acc amp id info or visit http www oracle com pls topic lookup ctx acc amp id trs if you are hearing impaired R f rence E35082 14 Copyright O 2013 2015 Oracle et ou ses affili s Tous droits r serv s Ce logiciel et la documentation qui l accompagne sont prot g s par les lois sur la propri t intellectuelle Ils sont conc d s sous licence et soumis des restrictions d utilisation et de divulgation Sauf stipulation expresse de votre contrat de licence ou de la loi vous ne pouvez pas copier reproduire traduire diffuse
119. st disconnect all power supplies before servicing any of the components that are inside the chassis Antistatic Wrist Strap Use Wear an antistatic wrist strap and use an antistatic mat when you are handling components such as hard drive assemblies circuit boards or networking adapters When servicing or removing server components attach an antistatic strap to your wrist and then to a metal area on the chassis Following this practice equalizes the electrical potentials between you and the server Antistatic Mat Place ESD sensitive components such as motherboards memory and other PCBs on an antistatic mat 50 SPARC T5 8 Server Service Manual e November 2015 Tools Needed for Service Related Information Removing Power From the Server on page 53 Tools Needed for Service on page 51 Tools Needed for Service You will need the following tools for most service operations Antistatic wrist strap Antistatic mat a No 2 Phillips screwdriver Mechanical lift for rear chassis subassembly removal if only one person is present Related Information Component Service Categories on page 52 Filler Panels and Modules on page 51 Filler Panels and Modules Depending on the configuration the server can include the following types of filler panels and modules Hard drive filler panels DIMM filler panels these are used only to ship new processor modules You must remove all of the DIM
120. stem Configuration PROM 121 122 SPARC T5 8 Server Service Manual e November 2015 Servicing the Battery The battery is a cold service component that can be replaced only after you have removed the main module Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe service procedures for the battery in the server Steps Description Links 1 Determine if the battery is faulty PSH Overview on page 41 2 Remove the main module so that you can replace the Remove the Main Module on page 91 battery 3 Replace the battery m Remove the Battery on page 123 m Install the Battery on page 125 4 Install the main module after you have replaced the Install the Main Module on page 95 battery 5 Verify the replacement Verify the Battery on page 126 Related Information Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Y Remove the Battery A The battery is a cold service component that can be replaced only after you have removed the main module Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail Servicing the Battery 12
121. storage modules NEMs and PCI EMs server modules only The modular system provides Oracle ILOM through its CMM Message identifier Top level Oracle ILOM target Network Equipment Building System Netra products only Network express module server modules only NEMs provide Ethernet and SAS connectivity to storage modules Network management port An Ethernet port on the server SP the server module SP and the CMM Network interface card or controller Nonmaskable interrupt OpenBoot PROM Sometimes OBP is used in file names and messages to indicate a relationship to OpenBoot Glossary 197 Oracle ILOM Oracle ILOM Oracle ILOM CMM Oracle Solaris OS PCI PEM POST PROM PSH REM S SAS SCC SER MGT server module SP Oracle Integrated Lights Out Manager Oracle ILOM firmware is preinstalled on a variety of Oracle systems Oracle ILOM enables you to remotely manage your Oracle servers regardless of the state of the host system Oracle ILOM that runs on the CMM server modules only See Oracle ILOM Oracle Solaris operating system Peripheral component interconnect PCle ExpressModule server modules only Modular components that are based on the PCI Express industry standard form factor and offer I O features such as Gigabit Ethernet and Fibre Channel Power on self test Programmable read only memory Predictive self healing RAID expansion module server modules only
122. t log gt show SP logs event list View the audit log gt show SP logs audit list Related Information Check the Message Buffer on page 45 SPARC T5 8 Server Service Manual e November 2015 View Log Files Oracle ILOM m View Log Files Oracle Solaris on page 46 Detecting and Managing Faults 47 48 SPARC T5 8 Server Service Manual e November 2015 Preparing for Service These topics explain how to prepare to service the server Step Description Links o Si po Mon E Review safety and handling information Gather the tools for service Locate the server to be serviced Find the server serial number Locate the component service information For cold service operations shut down the OS and remove the power from the server Prevent ESD damage before you handle any server component Safety Information on page 49 Tools Needed for Service on page 51 Locate the Server on page 53 Find the Server Serial Number on page 52 Component Service Categories on page 52 Removing Power From the Server on page 53 Prevent ESD Damage on page 57 Related Information Identifying Components Detecting and Managing Faults Component Service Task Reference on page 23 Returning the Server to Operation Safety Information For your protection observe the following safety precautions when setting up your equip
123. tallation Guidelines on page 153 m Server Upgrade Process on page 59 Root Complex Connections Two Processor Modules This diagram illustrates the root complex connections between the two processor modules and the 16 PCle I O slots in a half populated server Each CPU supports two I O root complex fabrics Each root complex connects to four I O slots through two of five multiplexing switches Servicing PCle Cards 151 Understanding PCle Root Complex Connections PMO PFM1 PFM2 FINI TI 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 PCle Slots The port ID values shown in the diagram correspond to the pci values reported in the showdevs command output For example PM CPU Switch IO Slot Root Complex Path 0 0 0 1 pci 300 pci 1 pci 0 pci 6 0 0 0 2 pci 300 pci 1 pci 0 pci c 0 0 1 3 pci 340 pci 1 pci 0 pci 6 0 0 1 4 pci 340 pci 1 pci 0 pci c 3 2 1 5 pci 600 pci 1 pci 0 pci e 3 2 1 6 pci 600 pci 1 pci 0 pci 8 3 2 2 7 pci 640 pci 1 pci 0 pci e 3 2 2 8 pci 640 pci 1 pci 0 pci 8 0 1 2 9 pci 380 pci 1 pci 0 pci a 0 1 2 10 pci 380 pci 1 pci 0 pci 4 0 1 3 11 pci 3c0 pci 1 pci 0 pci e 0 1 3 12 pci 3c0 pci 1 pci 0 pci 8 3 3 3 13 pci 680 pci 1 pci 0 pci a 3 3 3 14 pci 680 pci 1 pci 0 pci 4 3 3 4 15 pci 6c0 pci 1 pci 0 pci 8 152 SPARC T5 8 Server Service Manual e November 2015 PCle Card Installation Guidelines PM CPU Switch IO Slot Root Complex Path 3 3 4 16 pc
124. tep Diagnostic Action Possible Outcome Links 1 Confirm that the Power OK If these LEDs are not lit check the power source and Interpreting LEDs on page 29 and AC OK LEDs are lit power connections to the server 2 Check the server for detected Use these tools to check for faults Check for Faults on page 41 faults System LEDs on the front and rear panels fmadm faulty from the Oracle Solaris prompt or through the Oracle ILOM fault management shell E show faulty from the Oracle ILOM prompt or through the Open Problems BUI m Datacenter management tools such as Oracle Enterprise Manager Ops Center 3 Check the log files for fault If system messages indicate a faulty component replace Interpreting Log Files and System information it Messages on page 45 4 RunOracle VTS software To run Oracle VTS the server must be running the m Refer to the Oracle VTS software Oracle Solaris OS m If Oracle VTS reports a faulty component replace it m If Oracle VTS does not report a faulty component run POST documentation Configuring POST on page 35 Contact technical support if the problem persists Related Information m Tool Availability on page 27 Log In to Oracle ILOM Service on page 28 Oracle ILOM Service Related Tools on page 29 Oracle ILOM Documentation Library Tool Availability This table describes what tools are available at the different states in which the server oper
125. the Host Operation System Response The service required LED on the chassis and on the affected FRU may be illuminated Impact No SP impact Action Refer to the associated reference document at http support oracle com msg PCIEX 8000 8R for the latest service procedures and policies regarding this diagnosis 42 SPARC T5 8 Server Service Manual e November 2015 Check for Faults faultmgmtsp gt In this example a fault is displayed that includes these details Date and time of the fault 2012 08 27 19 46 26 m UUID 4e16c8d 5cdb c6ca c949 e24d3637ef27 which is unique to each fault m Message identifier PCIEX 8000 8R which can be used to obtain additional fault information from Knowledge Base articles Consider your next step m f you are checking for faults while adding additional processor modules and no faults were detected return to Broken Link Target ID Z40019D01512366 Ifa fault is detected proceed to Step 4 Use the message ID to obtain more information about this type of fault a Obtain the message ID from console output b Go to https support oracle com and search on the message ID in the Knowledge tab Follow the suggested actions to repair the fault Determine your next step If you found a fault that must be removed manually go to Clear a Fault on page 44 If you are upgrading the server and found no faults return to Server Upgrade Process on page 59 Rel
126. the Oracle VM for SPARC documentation These topics describe service procedures for the PCIe cards that are installed in the PCIe card carriers in the server Understanding PCIe Root Complex Connections on page 149 PCIe Card Installation Guidelines on page 153 PCIe Carrier LEDs and Controls on page 154 Determine Which PCIe Card Is Faulty on page 155 Remove a Card Carrier on page 155 Remove a PCle Card on page 159 Remove a Carrier Extension on page 162 Install a Carrier Extension on page 164 Install a PCle Card on page 165 Install a Card Carrier on page 168 m Verify the PCIe Card on page 171 Understanding PCle Root Complex Connections A root complex is the CMP circuitry that provides the base to a PCle I O fabric Each PCle I O fabric consists of the PCIe switches PCIe slots and leaf devices associated with the root complex Understanding the relationship of the PCIe root complexes to the PCIe I O fabrics will help you to properly assign devices when configuring Oracle VM Server for SPARC logical domains These topics explain the root complex configurations Servicing PCle Cards 149 Understanding PCle Root Complex Connections 150 Root Complex Connections Four Processor Modules on page 150 Root Complex Connections Two Processor Modules on page 151 Root Complex Connections Four Processor Modules This diagram i
127. the Rear Chassis Subassembly subassembly SP off Servicing the Service Processor Card Storage backplanes off Servicing the Storage Backplanes SCC PROM off X Servicing the System Configuration PROM Related Information Removing Power From the Server on page 53 Returning the Server to Operation Y Find the Server Serial Number 52 If you need technical support for your server you must to provide the server s serial number Use one of the following options to find the serial number a Locate the manufacturing sticker on the front of the server or on the sticker on the side of the server b At the Oracle ILOM prompt type gt show System SPARC T5 8 Server Service Manual e November 2015 Locate the Server System Targets Open Problems 0 CPU modules Processors Memory Power Cooling Storage Networking PCI Devices Firmware Log Propertiies health OK health details open problems count 0 type Rack Mount model SPARC T5 8 qpart id Q9527 part number 12345678 11 1 serial number XXXXXXXXXX Y Locate the Server 1 Atthe Oracle ILOM prompt type gt set SYS LOCATE value Fast_Blink Alternatively you can type gt set System locator_indicator on The white Locator LEDs one on the front panel and one on the rear panel blink 2 After locating the server with the blinking Locator LED turn it off by pressing the Locator button Note Alte
128. tifying Components 15 DIMM Locations This illustration shows the rear chassis subassembly removed from the server chassis The rear chassis subassembly is removed and serviced as a single unit No Description Links 1 Server chassis Servicing the Rear Chassis Subassembly 2 Midplane 3 Rear chassis subassembly Related Information Front Panel Components Service on page 14 Chassis Subassembly Components on page 22 DIMM Locations DIMMs are located in each of the processor modules 16 SPARC T5 8 Server Service Manual e November 2015 Main Module Internal Component Locations This figure shows a processor module with all of the DIMM slots populated with DIMMs All of the DIMM slots must be populated with the same type of DIMMs For guidelines see DIMM Configuration on page 71 No Description Link 1 DIMMs Servicing DIMMs Main Module Internal Component Locations These components are accessible after you remove the main module from the front of the server Identifying Components 17 Supported Storage Devices No Description Links 1 Hard drives 8 Servicing Hard Drives 2 Front I O module Servicing the Front I O Assembly 3 Storage backplanes 2 Servicing the Storage Backplanes 4 Motherboard Servicing the Main Module 5 SP Servicing the Service Processor Card 6 System configuration PROM Servicing the System Co
129. tion Identifying Components Detecting and Managing Faults Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation Y Remove a Storage Backplane The storage backplances are cold service components that can be replaced after you remove the main module A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail 1 Power off the server and disconnect the power cords Servicing the Storage Backplanes 99 Remove a Storage Backplane See Removing Power From the Server on page 53 2 Take the necessary ESD precautions See Prevent ESD Damage on page 57 3 Remove all the hard drives from the front of the server for the storage backplane that you want to replace Note the locations of the drives before removing them so that you can install them in their original slots You have to remove only hard drives 0 3 or drives 4 7 depending on which storage backplane you want to replace See Remove a Hard Drive on page 83 4 Remove the main module from the server See Remove the Main Module on page 91 5 Locate the storage backplane that you want to remove No Description 1 Storage backplane for drives 4 through 7 SAS BP1 2 Storage backplane for drives O through 3 SAS BPQ 100 SPARC T5 8 Server Service Manu
130. tion This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail Caution Do not attempt to remove the rear chassis subassembly while the server is in the rack You must remove the chassis to service the rear chassis subassembly Power down the server See Removing Power From the Server on page 53 Disconnect the power coards See Disconnect the Power Cords on page 56 Take the necessary ESD precautions See Prevent ESD Damage on page 57 At the front of the server remove all of these components SPARC T5 8 Server Service Manual e November 2015 Remove the Rear Chassis Subassembly Component Link Cables Processor modules top to bottom Main module Power supplies Ensure that you label the cables Remove a Processor Module or Processor Filler Module on page 63 Remove the Main Module on page 91 Remove a Power Supply on page 135 5 At the rear of the server remove all of these components Component Link Cables Fan modules PCle carriers Rear I O module Ensure that you lable the cables Remove a Fan Module on page 143 Remove a Card Carrier on page 155 Remove the Rear I O Module on page 174 Caution Do not attempt to remove the chassis alone without the aide of another person or a A mechanical lift 6 Determine your next step
131. tionality 178 reconfigurable hardware 84 removing battery 123 DIMMs 76 fan modules 143 front I O assembly 127 hard drives 83 main module 91 PCle carriers 155 power supplies 135 processor modules 63 rear I O module 174 SP 110 storage backplanes 99 system configuration PROM 118 root complex topologies default 150 running POST in Diag Mode 40 S safety information and symbols 49 SCC PROM locating 17 NAC name 24 server connecting power cords 191 powering off emergency shutdown 55 gracefully with power button 55 using SP command 54 powering on 204 SPARC T5 8 Server Service Manual e November 2015 using power button 193 usingstart SYS command 192 server upgrade overview 59 Service Required LED storage 82 SP installing card 112 LEDs 33 locating 17 locating faulty 109 NAC name 24 removing 110 verifying functionality 114 storage device tree 86 hot service operations 81 LEDs 82 NAC name 23 Oracle VTS 87 physical addresses 85 unconfiguring 84 storage backplanes installing 103 locating 17 removing 99 system configuration PROM installing 119 removing 118 system controls front panel 31 system LEDs storage 82 System Locator LED 31 33 system message log files viewing 46 System Overtemp LED 31 33 System Power button 31 System Power OK LED 31 33 System Service Required LED 31 33 T tools needed for service 51 U upg
132. tmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 Jf no fault was detected the power supply was installed successfully Related Information Determine Which Power Supply Is Faulty on page 134 Install a Power Supply on page 137 Servicing Power Supplies 139 140 SPARC T5 8 Server Service Manual e November 2015 Servicing Fan Modules The fan modules are hot service components that can be replaced at any time The server will continue to operate at full capacity with nine fan modules installed in the server The server will not operate with fewer than nine fan modules If the server is operating with nine fan modules and one or more of those modules fails the server will power down to keep from overheating For the location of the fan modules see Fan Module Locations on page 20 A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe service procedures for the fan modules in the server Steps Description Links 1 Understand the fan module LEDs Fan Module LED on page 141 2 Determine which fan module is faulty Determine Which Fan Module Is Faulty on page 142 3 Replace the fan mo
133. trols and LEDs on page 31 and Rear Panel Controls and LEDs on page 33 for the LED locations If these conditions are not met perform the actions described in Diagnostics Process on page 25 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 Jf no fault was detected the fan module was installed successfully Related Information Determine Which Fan Module Is Faulty on page 142 Install a Fan Module on page 145 Servicing Fan Modules 147 148 SPARC T5 8 Server Service Manual e November 2015 Servicing PCle Cards PCle cards are hot service components that can be replaced at any time if the card is not currently in use For the location of the PCle cards see Rear Panel Components Service on page 15 A Caution To remove a PCle card that is assigned to an I O domain first remove the device from the I O domain Then add the device to the root domain before you physically remove the device from the system These steps enable you to avoid a configuration that is unsupported by the Direct I O or SR IOV feature For more information about making hardware changes to an T O domain refer to
134. ttery is functioning properly See Verify the Battery on page 126 Related Information Remove the Battery on page 123 m Verify the Battery on page 126 Y Verify the Battery 126 1 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 191 Started the system See Power On the Server Oracle ILOM on page 192 Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Start the faultmgmt shell gt start SP faultmgmt shell Are you sure you want to start the faultmgmt shell y n y faultmgmtsp gt Use the fmadm faulty command to determine if the server is operating normally Ifa fault was detected see Diagnostics Process on page 25 If no fault was detected the battery was installed successfully Related Information Remove the Battery on page 123 Install the Battery on page 125 SPARC T5 8 Server Service Manual e November 2015 Servicing the Front I O Assembly The front I O assembly is a cold service component that can be replaced only after you have removed the main module A Caution This procedure requires that you handle components that are sensitive to electrostatic discharge This discharge can cause server components to fail These topics describe service procedures for the front I O assembly in the server Steps Description Links 1 Remove the main mo
135. uration PROM on page 120 b If you replaced the entire main module see Verify the Main Module on page 97 96 SPARC T5 8 Server Service Manual e November 2015 Verify the Main Module 6 Ifthe you are replacing the main module with a new one connect a terminal or a terminal emulator PC or workstation to the SER MGT port The following message is delivered over the serial management port Unrecognized Chassis This module is installed in an unknown or unsupported chassis You must upgrade the firmware to a newer version that supports this chassis 7 Download the system firmware a Configure the SER MGT port to enable the firmware image to be downloaded Refer to the Oracle ILOM documentation for network configuration instructions b Download the system firmware Follow the firmware download instructions in the Oracle ILOM documentation Note You can load any supported system firmware version including the firmware revision that had been installed prior to the replacement of the main module However Oracle strongly recommends installing the newest version of the system firmware 8 Power on the server See Returning the Server to Operation Related Information Remove the Main Module on page 91 m Verify the Main Module on page 97 Y Verify the Main Module 1 Ensure that you have completed the following Applied power to the server See Connect the Power Cords on page 19
136. ure event has been acknowledged and a service action is required Related Information Front Panel Controls and LEDs on page 31 Understanding Diagnostics on page 25 34 SPARC T5 8 Server Service Manual e November 2015 Configuring POST Configuring POST PIN These topics explain how to configure POST as a diagnostic tool m POST Overview on page 35 Oracle ILOM Properties That Affect POST Behavior on page 35 Configure POST on page 39 Run POST With Maximum Testing on page 40 POST Overview POST is a group of PROM based tests that run when the server is powered on or when it is reset POST checks the basic integrity of the critical hardware components in the server You can also set other Oracle ILOM properties to control various other aspects of POST operations For example you can specify the events that cause POST to run the level of testing POST performs and the amount of diagnostic information POST displays These properties are described in Oracle ILOM Properties That Affect POST Behavior on page 35 If POST detects a faulty component the component is disabled automatically If the server is able to run without the disabled component the server boots when POST completes its tests For example if POST detects a faulty processor core the core is disabled POST completes its test sequence and the server boots using the remaining cores Related Information
137. use various diagnostic tools to monitor server status and troubleshoot faults in the server The examples use the PSH fmadm faulty command Understanding Diagnostics on page 25 Interpreting LEDs on page 29 Configuring POST on page 35 Managing Faults on page 41 Interpreting Log Files and System Messages on page 45 Related Information Identifying Components Preparing for Service Component Service Task Reference on page 23 Returning the Server to Operation m Oracle ILOM Documentation Library Understanding Diagnostics These topics explain the diagnostic process and tools Diagnostics Process on page 25 Tool Availability on page 27 Log In to Oracle ILOM Service on page 28 Oracle ILOM Service Related Tools on page 29 Diagnostics Process Depending on the fault you might need to perform all of the steps or just some of them You also might have to run diagnostic software that needs to be installed or enabled Detecting and Managing Faults 25 Understanding Diagnostics Faulty hardware suspected Note The diagnostic tools you use and the order in which you use them depend on the nature of the problem you are troubleshooting However for descriptive purposes this table follows the steps given in the illustration 26 SPARC T5 8 Server Service Manual e November 2015 Understanding Diagnostics S
138. ver m Green Power is applied to the server Net MGT port link LED Indicates these conditions m Off No link is established On or blinking A link is established Net MGT port speed LED Indicates these conditions m Off The link is operating as a 10 Mbps connection m On or blinking The link is operating as a 100 Mbps connection Network port link LED Indicates these conditions m Off No link is established m Blinking A link is established Network port speed LED Indicates these conditions m Off The link is operating as a 10 Mbps connection or there is no link Amber on The link is operating as a 100 Mbps connection Green on The link is operating as a Gigabit connection 1000 Mbps m Amber on The link is operating as a 100 Mbps connection AC 2 left and AC 3 right Indicates these conditions power LEDs Detecting and Managing Faults 33 Interpreting LEDs No LED Icon or Label Description 7 Locator LED and button white 8 Service Required LED amber 9 Power OK LED green 10 SP LED SP 11 Physical presence button 12 Overtemp LED amber r m Off No power is applied to the server m Green Power is applied to the server Tum on the Locator LED by pressing the Locator button or see Locate the Server on page 53 When lit the LED blinks rapidly The fmadm faulty command provides details about any faults that cause
139. view the log files and system messages Check the Message Buffer on page 45 Understanding Diagnostics on page 25 Managing Faults on page 41 Y Check the Message Buffer The dmesg command checks the system buffer for recent diagnostic messages and displays them 1 Login as superuser 2 Type dmesg Related Information m View Log Files Oracle Solaris on page 46 Detecting and Managing Faults 45 View Log Files Oracle Solaris 46 m View Log Files Oracle ILOM on page 46 View Log Files Oracle Solaris The error logging daemon syslogd automatically records various system warnings errors and faults in message files These messages can alert you to system problems such as a device that is about to fail The var adm directory contains several message files The most recent messages are in the var adm messages file After a period of time usually every week a new messages file is automatically created The original contents of the messages file are rotated to a file named messages 1 Over a period of time the messages are further rotated to messages 2 and messages 3 and then deleted Log in as superuser Type more var adm messages To view all logged messages type more var adm messages Related Information m Check the Message Buffer on page 45 m View Log Files Oracle Solaris on page 46 View Log Files Oracle ILOM View the even
140. you are installing a new processor module upgrading the memory or replacing a faulty DIMM go to Step 4 If you are replacing the covers on the new processor modules to upgrade the server return to Server Upgrade Process on page 59 Servicing Processor Modules 67 Install a Processor Module or Processor Filler Module 4 Open the latches on the processor module or processor filler module and insert the module into the empty processor module slot in the server 5 Push the levers together toward the center of the processor module or processor filler module and press the levers firmly against the module to fully seat the module back into the server 68 SPARC T5 8 Server Service Manual e November 2015 Install a Processor Module or Processor Filler Module The levers should click into place when the module is fully seated in the server Determine your next step m If you replaced a faulty processor module or DIMM see Returning the Server to Operation Ifyou installed new processor modules to upgrade the server return to Server Upgrade Process on page 59 Determine your next step m f you replaced DIMMs see Verify the DIMM on page 80 m f you replaced a processor module see Verify the Processor Module on page 70 Servicing Processor Modules 69 Verify the Processor Module Related Information Servicing DIMMs m Verify the Processor Module
141. ype and size of DIMMs that are already in the existing processor modules If you are reviewing this information because you are upgrading the server return to Server Upgrade Process on page 59 DIMM Locations DIMM addresses and consequently their NAC names are based on their location on the processor module motherboard as well as the slot in which the processor is installed For example the full address for the DIMM that is installed in the front left corner of the processor module that is installed in slot 0 is System Memory DIMMs DIMM 0 or SYS PMO CM1 CMP BOBO CHO DO This illustration shows the DIMM layout 72 SPARC T5 8 Server Service Manual e November 2015 Determine Which DIMM Is Faulty FMA BOB0 CH0 D0 ERES BOB 0 BOB 2 BOB 3 TT RES EEE n BOB2 CH0 D0 BOB3 CHO DO CM1 CMP BOB4 CHO DO BOB6 CHO DO OT Eee OT EE BOB7 CHo Do BOB5 CHO DO BOB3 CH1 D0 ET E ES BOB 2 BOB 0 DT O 08 1 CH11 D0 AO BOB1 CHO DO Iii BOBO cHo DO CM0 CMP ir BOB4 CHO DO Fe ai RES sO ms E 5055 CH1 D0 BOB 4 BOB 6 BOB 7 al BEE a TO RSZETIOO L BOB6 CHO DO tal BOB7 CHo Do BOB2 CHo DO DE O n BOB 3 Related Information DIMM Configuration Fault Messages on page 76 Determine Which DIMM Is Faulty FMA on page 73 Determine Which DIMM Is Faulty LEDs on page 75 m Installa DIMM on page 78 m Server Upgrade Process on page 59 Y Determin
142. ys information for each step normal Displays a moderate amount of information including component names and test results debug Displays extensive debugging information none Disables the output TABLE 10 HOST diag verbosity Value Description normal Displays all test and informational messages in POST output Detecting and Managing Faults 37 Configuring POST 38 Value Description min Displays functional tests with a banner and pinwheel in POST output max Displays all test informational and some debugging messages in POST output debug Displays extensive debugging information none Does not display POST output This flowchart illustrates the same set of Oracle ILOM set command variables Related Information POST Overview on page 35 Configure POST on page 39 Host boot settings Run POST With Maximum Testing on page 40 SPARC T5 8 Server Service Manual e November 2015 Configure POST Configure POST Log in to Oracle ILOM See Log In to Oracle ILOM Service on page 28 Set the virtual keyswitch to the value that corresponds to the POST configuration you want to run This example sets the virtual keyswitch to normal which configures POST to run according to other parameter values gt set HOST keyswitch state normal Set keyswitch state to Normal For possible values for the keyswitch state parameter see Oracle ILOM Pro

SPARC T5-8 Server Service Manual

Contents

Download Pdf Manuals

Related Search

Related Contents