Home

Sun Blade 6048 Modular System Service Manual

1. cd CMM network A 2 Sun Blade 6048 Modular System Service Manual December 2009 A 1 2 3 Type show to view the switchconf variable setting For example show CMM network Targets Properties commitpending Cannot show property ipaddress 10 6 153 71 ipdiscovery dhcp ipgateway 10 6 152 1 ipnetmask 255 255 252 0 macaddress 00 14 4F 6B 6F C1 pendingipaddress 10 6 153 71 pendingipdiscovery dhcp pendingipgateway 10 6 152 1 pendingipnetmask 255 255 252 0 switchconf port0 Commands cd set show switchconf portl In this example the switchconf variable is set to porto a To activate port 1 and disable port 0 type switchconf port1 To activate port 1 and keep port 0 active type switchconf trunk 4 Remove the CMM and reinstall it into the chassis The active port is now NET MGT port 1 or both NET MGT ports To Enable Ethernet Ports Through the Web Interface To enable an Ethernet port via the web interface 1 Open a web browser and type the IP address of the server SP or CMM The Login page for the ILOM web interface appears 2 In the ILOM Login page enter a user name and password then click OK The ILOM web interface appears Appendix A Sun Blade 6048 ILOM 2 0 Information A 3 3 Click on CMM in the left panel 4 Click the Configuration tab then the Network tab FIGURE A 1 ILOM Web Interface Network Screen File Edit View Hist
2. December 2009 5 Pull the ejector lever toward you to pull the PCI EM forward until you can pull the rest of the module out by hand 4 1 5 2 Installing PCI EMs FIGURE 4 12 shows how to install the PCI EMs FIGURE 4 12 Installing a PCI EM Chapter 4 Replacing System Components 4 15 1 Do one of the following depending on the type of PCI EM that you are installing If you are not installing a Gigabit Ethernet PCI EM go to Step 2 a If you are installing a Gigabit Ethernet PCI EM continue this step a Press the Release button on the side of the GbE PCI EM to release its cover then remove the cover b Locate the PCI EM MAC address which is visible on the PCI EM with its cover removed c Record the MAC address and the slot number into which you are about to install the GbE PCI EM Keep the data for future reference d Replace the GbE PCI EM cover by sliding the cover into place 2 Align the PCI EM with the vacant PCI EM slot Ensure that the indicator lights on the front panel of the PCI EM are facing toward you and that the PCI EM ejector lever on the bottom is fully opened 3 Slide the PCI EM into the vacant PCI EM chassis slot The ejector lever starts to pop up as the module engages with the system slot 4 Complete the installation by closing the ejector lever to secure the PCI EM in the chassis 5 Notify the operating system that a new PCI EM is installed See the server module blade documentati
3. PCI EMs and blades refer to the documentation for that component 2 2 Sun Blade 6048 Modular System Service Manual December 2009 2 2 224 2 2 1 1 2 2 1 2 Monitoring Faults There are three ways to tell when a fault has occurred somewhere in the system m The amber Service Action Required LEDs on the failed component and on the system chassis shelf are illuminated see Section 2 2 1 Monitoring the Service Action Required LEDs on page 2 3 Component status information available through the ILOM web interface and CLI registers that the component is in a faulted state see Section 2 2 2 Monitoring Faults From the Management Interfaces on page 2 4 m The fault is recorded in the system event log see Section 2 2 3 Monitoring the Event Log on page 2 7 Monitoring the Service Action Required LEDs When a component experiences a hardware failure enters a fault state fault management illuminates the Service Action Required amber LED on that component In addition fault management illuminates the Service Action Required LEDs on the system chassis shelf both front and back when any system component is in a faulted state When Service Action Required LEDs Are Turned Off Since a Service Action Required LED indicates a hardware failure it remains illuminated until fault management detects that the failed hardware has been replaced or repaired The chassis shelf Service Action Required LEDs whi
4. m Section 2 2 Monitoring Faults on page 2 3 m Section 2 3 Determining That Hardware Has Failed on page 2 11 m Section 2 4 Replacing a Faulted Component on page 2 19 2 1 About Fault Management The fault management software of the Sun Blade 6048 modular system monitors hardware health and diagnoses and reports hardware failures on system components Fault management also monitors environmental conditions and reports when the systems environment is outside acceptable parameters Various sensors on the system chassis shelf the power supplies PSUs server modules and fans are continuously monitored When a sensor registers a problem the fault management software which runs on the chassis management module CMM is notified Fault management then diagnoses the problem If it determines that a hardware or environmental failure has occurred fault management lights the Service Action Required LED on the affected component The ILOM management interfaces are updated to reflect the failure the fault and the failure is recorded as a fault in the event log Note The Sun Blade 6048 modular system s fault management software is entirely unrelated to Solaris Fault Management Architecture FMA Fault management is part of the system management software and does not interact with the server module hosts or their operating systems 2 1 2 1 1 External Compared With Internal Faults When a system component e
5. If the temperature observed by a sensor falls below or rises above a set threshold the monitoring software subsystem lights the amber Service Required indicators on the front and back of the system If the temperature condition persists and reaches a critical threshold the system might initiate a graceful system shutdown See Section 1 6 6 LED Indicators and Buttons on page 1 16 for more information about the system LEDs All error and warning messages are sent to the chassis management module CMM and are logged in the Sun ILOM log file Additionally some customer replaceable units CRUs such as power supplies and fans provide LEDs that indicate a failure within the CRU See Section 2 3 1 Chassis Shelf Faults on page 2 11 for more information about chassis fault LEDs 1 14 Sun Blade 6048 Modular System Service Manual December 2009 1 6 5 RAS Features Summary TABLE 1 6 RAS Features Summary Feature Description Power supplies Hot swappable redundant integrated into the chassis making the server modules more reliable Airflow and cooling Fans are integrated into the chassis e Two hot swappable front fan modules located within the power supply cool the PCI EMs NEMs and power supplies e Eight hot swappable rear fan modules cool the server modules Server modules Hot pluggable servicing can be done without affecting cabling or blades I O configuration 1 0 modules Hot swappable network express modu
6. Place the brackets on the top of the Sun Blade 6048 cabinet see panel 2 The open part of the bracket should correspond with the opening in the back of the cabinet 3 Use two of the M6 screws for each bracket to fasten the expansion module brackets see panel 3 a Place two M6 screws in the two outermost screw holes of the right bracket b Insert a screwdriver into the hole on top of the bracket that is directly above the screw and tighten the screw c Repeat steps a and b for the left bracket You can now mount a 2 rack unit RU system onto the expansion module using the rack rails included with the system 4 32 Sun Blade 6048 Modular System Service Manual December 2009 4 6 Floor Tie Down Installation This section describes how to install floor tie downs onto the Sun Blade 6048 chassis Note This procedure describes a generic procedure for installing floor tie downs that are supplied by the customer Refer to FIGURE 4 23 for the reference on the installation procedure FIGURE 4 23 Floor Tie Down Locations and Installation 1 Remove the metal plates from the bottom shelf on the front and back of the system The brackets are installed in four places two on the front two on the back Chapter 4 Replacing System Components 4 33 2 Install the four brackets in the in two outer holes on each side of the chassis front and back Refer to FIGURE 4 23 for locations where the brackets should
7. offline or when the system is powered off Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 7 1 4 3 1 8 See the Sun Integrated Lights Out Manager 2 0 User s Guide for more information about the CMM ILOM System Nomenclature The figures and tables in this section describe the paths in the CMM ILOM CLI that you can use to locate component information You can also use the ILOM web interface to access this information Refer to the Sun Integrated Lights Out Manager 2 0 User s Guide for more information Use the show command with the component location shown in this section to view information about the component For example show CH PSO gives you the following information about power supply module 0 gt show CH PSO CH PSO Targets STATUS so S1 S2 T_AMB FANO FANT FAN2 FAN3 FAN _FAIL V 12V V 3_3V T 3_3V V 3_3V_FAULT T_AMB FAULT T_AMB WARN Properties type Power Supply Commands cd show The information available for each component varies depending on the component sensor type See Appendix A for ILOM sensor information Sun Blade 6048 Modular System Service Manual December 2009 FIGURE 1 3 and TABLE 1 2 show the nomenclature used in the CMM ILOM to access information for front chassis components FIGURE 1 3 Front Panel BLO BL1 BL2 BL3 BL4 BL5 BL6 BL7 BL8 BL9 BL10 BL11 TABLE 1 2 Front Panel Nomenclature FRU Description Location n Blade Mod
8. CP Ce 0e 0e Le Ce digi did AJAS AC 1 LED AC 0 LED Front Fan Service LED 2 14 Sun Blade 6048 Modular System Service Manual December 2009 20 22 2 3 2 3 Power Supply Fault LED Functions TABLE 2 2 shows the operation of the LEDs during normal operation or when a fault has occurred Refer to the appropriate sensor table to find the location of the fault in the ILOM CLI TABLE 2 2 Power Supply Fault LED Functions PSU Fan AC 0 AC 1 AC 2 DC Service Service LED LED LED LED LED LED Condition Green Green Green Green Amber Amber Sensor Table Normal operation On On On On Off Off n a 3 cord configuration Normal operation On On Off Off On Off See 2 cord configuration Appendix A Over current over On On On Off On Off TABLE 2 4 voltage or over temperature warning fault AC 0 failed Off On On Off Off Off TABLE 2 4 AC 1 failed On Off On Off Off Off TABLE 2 4 AC 2 failed On On Off Off Off Off TABLE 2 4 Front fan failed On On On On Off On TABLE 2 5 Viewing Power Supply Faults in ILOM If the power supply module LEDs indicate that a power supply or front fan failure has occurred you can verify the fault by viewing the appropriate sensor through the ILOM CLI See the Sun Integrated Lights Out 2 0 Manager User s Guide and Appendix A for details on locating and reading the sensors in the ILOM Note In the tables below the variable n represents one of the following values power supply module 0 PS0 power supp
9. Server Modules 4 7 Contents v 4 1 3 Replacing the Front Indicator Module 4 9 4 1 3 1 Removing the Front Indicator Module 4 9 4 1 3 2 Installing the Front Indicator Module 4 10 4 1 4 Replacing NEMs 4 11 4 1 4 1 Removing the NEMs 4 11 4 1 4 2 Installing the NEMs 4 12 4 1 5 Replacing PCIEMs 4 13 4 1 5 1 Removing the PCIEMs 4 14 4 1 5 2 Installing PCIEMs 4 15 4 1 6 Replacing the Rear Fan Modules 4 17 4 1 6 1 Removing the Rear Fan Modules 4 17 4 1 6 2 Installing Rear Fan Modules 4 17 417 Replacing the Fan Board 4 19 4 1 7 1 Removing the Fan Board 4 19 4 1 7 2 Installing a Fan Board 4 19 4 1 8 Replacing the CMM 4 20 4 1 8 1 Removing the CMM 4 20 4 1 8 2 Installing the CMM 4 21 4 1 9 Installing the Cooling Doors 4 23 4 2 Field Replaceable Units 4 23 42 1 Replacing the Midplane 4 23 4 2 1 1 Removing the Midplane 4 23 4 2 1 2 Installing the Midplane 4 26 43 Servicetool FRU Update Procedure 4 27 44 Powering On the System After Service 4 28 441 Power Cord Connections 4 29 442 Powering On the System Automatically 4 29 44 3 Powering On the System Using the CLI 4 30 vi Sun Blade 6048 Modular System Service Manual December 2009 444 Powering On the System Using the Web Interface 4 30 45 Expansion Module Installation 4 31 4 6 Floor Tie Down Installation 4 33 Sun Blade 6048 ILOM 2 0 Information A 1 AZ Activating CMM Ethernet Ports A 2 A 1 1 To Enable Ethernet Ports Through the CLI A 2 A 1 2 To Enable Ethernet Ports Through the Web Interfa
10. Unis Tous droits r serv s Sun Microsystems Inc a les droits de propri t intellectuels relatants a la technologie qui est d crit dans ce document En particulier et sans la limitation ces droits de propri t intellectuels peuvent inclure un ou plus des brevets am ricains num r s a http www sun com patents et un ou les brevets plus suppl mentaires ou les applications de brevet en attente dans les Etats Unis et dans les autres pays Ce produit ou document est prot g par un copyright et distribu avec des licences qui en restreignent l utilisation la copie la distribution et la d compilation Aucune partie de ce produit ou document ne peut tre reproduite sous aucune forme par quelque moyen que ce soit sans l autorisation pr alable et crite de Sun et de ses bailleurs de licence s il y en a Le logiciel d tenu par des tiers et qui comprend la technologie relative aux polices de caract res est prot g par un copyright et licenci par des fournisseurs de Sun Des parties de ce produit pourront tre d riv es des syst mes Berkeley BSD licenci s par l Universit de Californie UNIX est une marque d pos e aux Etats Unis et dans d autres pays et licenci e exclusivement par X Open Company Ltd Sun Sun Microsystems le logo Sun Java Solaris et Sun Blade sont des marques de fabrique ou des marques d pos es de Sun Microsystems Inc aux Etats Unis et dans d autres pays Toutes les marques SPARC s
11. and the chassis shelf Temperature Fail LEDs are illuminated when the external condition represents a potential hazard to the system It is possible for an external fault to force a shutdown of the entire system Chapter 2 Fault Management 2 11 The Chassis Shelf Service Action Required LED also lights when there is a fault on a chassis shelf component 2 3 1 1 Chassis Shelf LED Locations FIGURE 2 3 and FIGURE 2 4 show the location of the LEDs on the front and rear of the chassis FIGURE 2 3 Front Chassis Shelf Fault Indicators CHASSIS Locate LED Service Action SHORTT oe oe Required LED A Power OK LED and Bae tad bos tas bad Cod Loe tos bad tan tai HE PREERPEEEE EE Ce e LE OO d Temperature Fail LED FIGURE 2 4 Rear Chassis Shelf Fault Indicators T LOW 13N 0 19W LAN Service Action Required LED Chassis Power OK LED CHE sISSvH Locate LED Temperature Fail LED CMM Power OK LED 2 12 Sun Blade 6048 Modular System Service Manual December 2009 2 3 1 2 2 3 1 3 2 2 Checking Other LEDs If the Service Action Required LED is lit on the FIM or CMM check the indicators on the power supplies and fan modules to see if one of these is also lit Refer to the following sections for more information m Section 2 3 2 Power Supply Module Faults on page 2 13 m Section 2 3 3 Rear Fan Faults on page 2 18 If a blade Service Action Required LED is lit refer to
12. automatically detect the new module 4 18 Sun Blade 6048 Modular System Service Manual December 2009 4 1 7 4 1 7 1 4 1 7 2 Replacing the Fan Board You will need to replace a fan if the board fails You can remove and replace a fan board from a powered off system Removing the Fan Board FIGURE 4 15 shows how to remove the fan boards FIGURE 4 15 Removing a Fan Board 1 Power off the chassis See Section 3 3 Powering Off the System Before Service on page 3 6 2 Remove the two rear fan modules that are in front of the fan indicator module that you want to remove See Section 4 1 6 Replacing the Rear Fan Modules on page 4 17 3 Loosen the captive screw on the fan indicator module until you can pull the top of the controller forward Then lift the controller up and off the chassis This screw needs to be fully unscrewed because the screw disengages the fan controller connector from the midplane Installing a Fan Board FIGURE 4 16 shows how to install the fan boards Chapter 4 Replacing System Components 4 19 FIGURE 4 16 Installing a Fan Board 1 Place the bottom notch on the fan board into the hole in the chassis 2 Push the top captive screw into place and tighten it 3 Replace the two rear fan modules that you removed See Section 4 1 6 Replacing the Rear Fan Modules on page 4 17 4 Power on the chassis See Section 4 4 Powering On the System After Service on p
13. blink Helps you locate a specific system in a data center Off Normal operating state You can illuminate the Locator indicator either by executing a software command on the CMM or by momentarily pressing the associated Locator button that is integrated with the LED Located on the power supply module this LED indicates whether the current and voltage for the power supply module is within accepted Steady Voltage temperature and current for the power supply module are within the accepted range e Off Indicates that there is a fault with the power supply module Located on the CMM this LED provides these indications Steady on Lights when external Ethernet connection is active e Off Indicates that there is no external Ethernet connection Sun Blade 6048 Modular System Service Manual December 2009 17 Upgraded Chassis An updated chassis is now available for the Sun Blade 6048 modular system This chassis enables installation of the Sun Blade cooling doors and contains a PCIe 2 0 compatible midplane You can identify the updated chassis in two ways m The updated chassis has air baffles installed on the top and bottom of the chassis FIGURE 1 6 shows the location of the top and bottom air baffles FIGURE 1 6 Air Baffles on Updated Chassis Chassis air baffles m You can use the chassis CMM to identify the chassis part number with one of the following me
14. following sections contain the power consumption values assumed by the CMM ILOM These are the values you should use for planning your system The actual power consumption of real world components may be significantly lower than the ILOM s assumed values However you must plan using the ILOM s assumed values B 1 B 1 1 B 1 2 B 2 in order to avoid triggering the safety features of the ILOM s power management system See Section B 2 ILOM Power Management Features on page B 6 for more information Power Required by the Chassis The chassis requires 20W This figure includes power used by the CMM but does not include power used by blades modules or fans Power Required by Blades A blade s power requirement is the sum of the following m The base power requirement for the blade This value includes power required by the blade motherboard and service processor but does not include power required by any optional components m The power requirements of any processors m The power requirements of any DIMMs m The power requirements of any disks m The power requirements of any attached PCI Express Modules PCI EMs m The power requirements of any attached Raid Expansion Modules REMs m The power requirements of any attached Expansion Modules FEM For example suppose you are estimating the power requirement for a maximally configured Sun Blade X6250 with the highest powered processors Its power consump
15. for the Sun Blade 6048modular system go to http www sun com downloads This site contains updates for firmware and drivers as well as CD ROM iso images Related Documentation For a description of the document set for the Sun Blade 6048 modular system see the Where to Find Documentation sheet that is packed with your system and also posted at the product s documentation site See the following web site http docs sun com app docs prod blade 6048mod Translated versions of some of these documents are available at the documentation web site in French Simplified Chinese Traditional Chinese Korean and Japanese English documentation is revised more frequently and might be more up to date than the translated documentation x Using UNIX Commands This document might not contain information about basic UNIX commands and procedures such as shutting down the system booting the system and configuring devices Refer to the following for this information a Software documentation that you received with your system a Solaris Operating System documentation which is at http docs sun com Sun Blade 6048 Modular System Service Manual December 2009 Typographic Conventions Typeface Meaning Examples AaBbCc123 The names of commands files Edit your Login file and directories onscreen Use 1s a to list all files computer output You have mail AaBbCc123 What you type when contrasted su wi
16. is present 2400 Entity Presence bl1 prsnt Device Present Blade 1 is present 2500 Entity Presence bl2 prsnt Device Present Blade 2 is present 2600 Entity Presence bl3 prsnt Device Present Blade 3 is present 2700 Entity Presence bl4 prsnt Device Present Blade 4 is present 2800 Entity Presence bl5 prsnt Device Present Blade 5 is present 2900 Entity Presence bl6 prsnt Device Present Blade 6 is present 2a00 Entity Presence bl7 prsnt Device Present Blade 7 is present 2b00 Entity Presence b18 prsnt Device Present Blade 8 is present Appendix A Sun Blade 6048 ILOM 2 0 Information A 15 TABLE A 5 AC Cables Re inserted SEL Readings Continued Sensor ID Device State Description 2c00 Entity Presence bl9 prsnt Device Present Blade 9 is present 2d00 Entity Presence bl10 prsnt Device Present Blade 10 is present 2e00 Entity Presence bl11 prsnt Device Present Blade 11 is present A 3 6 4 Stop CH When the stop CH command is applied the SEL displays the readings shown in TABLE A 6 TABLE A 6 Stop CH SEL Readings Sensor ID Device State Description 2f00 Power Supply Predictive Failure Not enough power since the ch pwrctl state Asserted PSU shuts off 3000 ps0 pstate_0 State Deasserted PSU 0 side 0 is out 3100 psi pstate_1 State Deasserted PSU 1 side 1 is out 3200 psi pstate_0 State Deasserted PSU 1 side 0 is out 3400 ps0 pstate_1 State Deasserted PSU 0 side 1 is out A 3 6 5 Start CH When the start C
17. the chassis shelf showing that the ambient temperature for the chassis shelf has moved above an acceptable range The chassis shelf Service Action Required LEDs serve as summary indicators notifying you that a hardware failure has occurred on one or more of the components in the chassis shelf m The sensor information in the CMM ILOM identifies which component has experienced a hardware failure The following topics in this section describe the fault sensors that are activated with component faults m The fault associated with the hardware failure is recorded in the system event log See Section 2 2 3 Monitoring the Event Log on page 2 7 or the Sun Integrated Lights Out Manager 2 0 User s Guide for more information about reading component sensors and the event log The following sections contain further details on identifying faults in the system or specific components m Section 2 3 1 Chassis Shelf Faults on page 2 11 m Section 2 3 2 Power Supply Module Faults on page 2 13 m Section 2 3 3 Rear Fan Faults on page 2 18 Chassis Shelf Faults Chassis shelf faults are external faults There is no hardware failure but an external condition exists that can adversely affect the operation of the system Because they are external chassis shelf faults are auto clearing when fault management detects that the external condition has returned to within normal parameters it clears the fault A fault is declared
18. the procedures in Section 4 1 Customer Replaceable Units on page 4 1 Power on the chassis See Section 4 4 Powering On the System After Service on page 4 28 Servicetool FRU Update Procedure Caution The SunService account is for the use of Sun service representatives only Do not use the SunService account unless you are instructed to do so in a procedure developed by Sun Microsystems 1 Use SSH to log in to the SunService account The default password is changeme ssh lt SP IP address gt 1 sunservice lt SP IP Address gt s password changeme Chapter 4 Replacing System Components 4 27 2 At the prompt enter the servicetool command with options The options are defined in the table below servicetool fru_update serviceprocessor lt OfherOptions gt lt value gt Other Options Value fru_product_part_number Write a new part number to the FRU fru_product_serial_number Write a new serial number to the FRU fru_chassis_serial_number Write a new chassis serial number to the FRU fru_asset_tag Write a new asset tag to the FRU 3 Watch the output from the command and respond to the confirmation prompts for continuing the update and rebooting the server Servicetool is going to collect system information for the service processor for future part swaps The following preconditions must be true for this to work The new service processor must be installed Do you
19. value Predictive Failure Asserted shows the faulted power supply Since one of the power supplies in power supply module 0 has failed the entire power supply module will need to be replaced Obtaining Sensor Readings Using the Web Interface In the ILOM web interface you can obtain instantaneous sensor readings about system FRUs field replaceable units or other system inventory on the System Monitoring gt Sensor Readings page To obtain sensor readings from the ILOM web interface 1 Open a web browser and type the IP address of the server SP or CMM The Login page for the ILOM web interface appears 2 In the ILOM Login page enter a user name and password and then click OK The ILOM web interface appears 3 In the web interface page click System Monitoring gt Sensors Readings The Sensor Readings page appears Chapter 2 Fault Management 2 5 FIGURE 2 1 Sensor Readings Page ser Administrator root 8P Hostname DarrenRevF fita Sun Integrated Lights Out Manager 2 lt Sensor Readings View readings for system sensors Click on a sensor name for more information including threshold values Sensor Readings a Name Type Reading sSYSIPO_PRSNT Entity Presence Present J sSYSIP1_PRSNT Entity Presence Present ISYSIP2_PRSNT Entity Presence Present 18YSIP3_PRSNT Entity Presence Present I8YS EM 0_PRSNT Entity Presence Absent ISYSIEM 1 _PRSNT Entity REST
20. web interface 1 Open a web browser and type the IP address of the server CMM The Login page for the ILOM web interface appears 2 In the ILOM Login page enter a user name and password and then click OK The ILOM web interface appears 3 In the web interface page select System Monitoring gt Event Logs The Event Log page appears FIGURE 2 2 ILOM Web Interface Event Log S sun Integrated Lights Out Manager Mozilla Firefox m X Ele gdt View History Bookmarks Tools Help EX C mozilla org Latest Builds fit O https 10 6 153 126 iPages suntab asp_ ISJ G Chassis User macam E Sensor Readings Indicators Event Logs Power Management E Blade 0 Blade 1 E Event Log 3 a Blade 3 Displays every event in the CMM including chassis IPMI Audit and FMA events Click the Clear Log button to delete all current log entries jade F Blade 4 E E Blade 5 Event Log E seseo rat non E Blade 7 a Blade Event ID class Type Severity Date Time Description a 51975 Audit Log minor Thu Jan 8 root Open Session object sessionitype value www success E Blade 9 02 37 03 1970 51974 Audit Log minor Thu Jan 8 root Close Session object session type value www success E plage 11 02 26 01 1970 51973 Audit Log minor Thu Jan 8 root Open Session object sessionitype v
21. 0W at 12V to power the server modules NEMs and PCI EMs The power supply modules also provide 3 3V standby power to power the system controller Installed from the front the front fan modules are located within the Hot swap power supply modules These fans cool the power supplies NEMs and PCI EMs Installed from the front this module provides LEDs to indicate Hot swap chassis activity Chassis management Installed from the rear the chassis accommodates a CMM ina Hot swap module CMM standby configuration The CMM provides partial chassis management functions Chapter 3 Preparing for Service 3 3 TABLE 3 1 CRU Features Continued Hot swap or CRU Description Hot plug Network express Installed from the rear the chassis supports up to two NEMs The Hot swap modules NEMs initial NEMs installed in the Sun Blade 6048 are switchless NEMs that provide a dual Gigabit NIC for each server module with up to 20 RJ 45 interfaces at the chassis rear PCI ExpressModules Installed from the rear the chassis holds up to 20 PCI EMs The PCI Hot plug PCI EMs EMs provide dedicated I O on a per server module basis Each server module connects through PCI Express to two PCI EMs The PCI EMs are hot pluggable under direct control of the server module hardware and software Rear fan modules Installed from the rear the chassis provides six fan modules that cool Hot swap the server modules Rear fan boards Installed from
22. 1 12 Sun Blade 6048 Modular System Service Manual December 2009 1 6 1 1 6 2 refers to the system s ability to operate continuously without failures and to maintain data integrity Availability refers to the ability of the system to recover to an operational state after a failure with minimal impact Serviceability relates to the time it takes to restore a system to service following a component failure Together the RAS features of the Sun Blade 6048 modular system provide for near continuous operation This section includes the following topics m Section 1 6 1 Hot Pluggable Components on page 1 13 m Section 1 6 2 Hot Swappable Components on page 1 13 m Section 1 6 3 Redundant Components on page 1 14 m Section 1 6 4 Environmental Monitoring on page 1 14 m Section 1 6 5 RAS Features Summary on page 1 15 m Section 1 6 6 LED Indicators and Buttons on page 1 16 Hot Pluggable Components Sun Blade 6048 modular system hardware supports hot plugging of the chassis mounted server modules blades Using the proper software commands you can install or remove these components while the system is running Hot plug technology significantly increases the system s serviceability and availability by providing the ability to replace these components without service disruption Refer your server module documentation for information about software commands for hot pluggable components Hot Swappab
23. 10 removal 4 9 replacement 4 9 front panel components CLI ILOM nomenclature 1 9 front panel of system 1 3 H hardware failure chassis 2 11 front fan 2 17 identifying 2 11 power supply module 2 13 rear fan 2 18 hot pluggable components 1 12 3 3 hot swappable components 1 13 3 3 l I O connectivity 1 5 installing chassis management module CMM 4 21 fan boards 4 19 front indicator module FIM 4 10 midplane 4 26 network express modules NEMs 4 12 PCI ExpressModules PCI EMs 4 15 power supplies 4 4 rear fan modules 4 17 server modules 4 7 Integrated Lights Out Manager ILOM CLI nomenclature 1 8 description 1 7 monitoring system faults 2 4 introduction to system 1 1 L LEDs chassis faults 2 12 CLI ILOM nomenclature 1 11 description 1 16 1 17 rear fan faults 2 18 Service Action Required 2 3 M midplane installation 4 26 removal 4 23 replacement 4 23 monitoring faults 2 3 N network express module NEM installation 4 12 removal 4 11 replacement 4 11 nomenclature CMM ILOM chassis LEDs 1 11 description 1 8 front panel components 1 9 rear panel components 1 10 P PCI ExpressModules PCI EMs description 1 5 installation 4 15 removal 4 14 replacement 4 13 power cord connections 4 29 power supply faults description 2 4 viewing in ILOM 2 15 installation 4 4 LEDs 2 14 module faults 2 13 removal 4 2 replacement 4 2 warnings 2 16 Index 2 Sun
24. 3 I O Connectivity 1 5 1 4 CMM System Management 1 6 141 CMM Integrated Lights Out Manager ILOM Features 1 6 142 Sun Integrated Lights Out Manager 1 7 1 4 3 System Nomenclature 1 8 1 5 Feature Summary 1 12 1 6 Reliability Availability and Serviceability Features 1 12 1 6 1 Hot Pluggable Components 1 13 1 6 2 Hot Swappable Components 1 13 1 6 3 Redundant Components 1 14 1 6 4 Environmental Monitoring 1 14 1 6 5 RAS Features Summary 1 15 1 6 6 LED Indicators and Buttons 1 16 17 Upgraded Chassis 1 19 v View Component Information Using the CLI 20 v View Component Information Using the Web Interface 20 2 Fault Management 2 1 2 1 About Fault Management 2 1 2 1 1 External Compared With Internal Faults 2 2 2 12 Components Monitored by Fault Management 2 2 2 2 Monitoring Faults 2 3 2 2 1 Monitoring the Service Action Required LEDs 2 3 2 2 1 1 When Service Action Required LEDs Are Turned Off 2 3 2 212 When Only the Chassis Shelf LEDs Are Lit 2 3 2 2 1 3 About Power Supply Faults 2 4 2 2 2 Monitoring Faults From the Management Interfaces 2 4 2 2 2 1 Obtaining Sensor Readings Using the CLI 2 4 2 2 2 2 Obtaining Sensor Readings Using the Web Interface 2 5 2 2 3 Monitoring the Event Log 2 7 2 2 3 1 Viewing or Clearing the ILOM Event Log Using the CLI 2 7 22 32 Viewing or Clearing the ILOM Event Log Using the Web Interface 2 9 2 3 Determining That Hardware Has Failed 2 11 2 3 1 Chassis Shelf Faults 2 11 2 3 1 1 Chassis Shelf
25. 6 faults 2 11 features summary 1 12 front panel 1 3 LEDs CLI ILOM nomenclature 1 11 locations 2 12 rear panel 1 5 chassis management module CMM CLI component interface 1 7 CLI ILOM nomenclature 1 8 ILOM 1 7 installation 4 21 removal 4 20 replacement 4 20 system management 1 6 comments and suggestions xi components accessing information 3 2 description 1 2 hot pluggable 3 3 hot swappable 3 3 locating 3 5 replaceable 3 1 replacing failed 2 19 customer replaceable units CRUs features 3 3 replacement procedures 4 1 D documentation related x driver updates ix E environmental system monitoring 1 14 ESD requirements 3 8 event log monitoring 2 7 F fan board installation 4 19 removal 4 19 replacement 4 19 fan faults front 2 17 rear 2 18 viewing in ILOM 2 18 fan module replacement rear fan 4 17 fault LEDs chassis 2 12 power supply 2 14 rear fan 2 18 fault management components monitored 2 2 description 2 1 external and internal faults 2 1 monitoring faults 2 3 Index 1 monitoring the event log 2 7 power supply faults 2 4 replacing a component 2 19 Service Action Required LEDs 2 3 using ILOM CLI 2 4 features list 1 12 field replaceable units FRUs features 3 4 replacement procedures 4 23 field replaceable units FRUs obtaining sensor readings 2 5 firmware updates ix front fan module faults 2 17 front indicator module FIM installation 4
26. 8 modular system The Sun Blade 6048 modular system is a blade server system optimized for high performance applications that place great demands on CPU performance memory capacity and I O bandwidth The system supports up to 48 server modules also known as blades per chassis The system design provides a power and cooling infrastructure to support current and future CPU and memory configurations ensuring that the chassis lifecycle will span multiple generations of server module upgrades The system provides a unified yet flexible architecture that enables you to consolidate multiple operating environments and applications This chapter includes the following sections m Section 1 1 System Components on page 1 2 m Section 1 2 Server Modules on page 1 5 m Section 1 3 I O Connectivity on page 1 5 m Section 1 4 CMM System Management on page 1 6 m Section 1 5 Feature Summary on page 1 12 m Section 1 6 Reliability Availability and Serviceability Features on page 1 12 m Section 1 7 Upgraded Chassis on page 1 19 1 1 1 2 System Components The Sun Blade 6048 modular system includes four chassis shelves Each shelf contains slots for 12 server modules accessible from the front of the chassis along with two power supplies and two front fan modules located inside the power supplies At the rear of the each chassis shelf are up to 24 PCI ExpressModules PCI EMs two network express mo
27. A 2 1 ILOM Behavior With Two Power Cord Configuration This section specifies how CMM and server module firmware behave when only two of the three power plugs are connected to an A231 power supply module There are three plugs on the back of each A231 PSU These plugs are named ACO ACI and AC2 Each allow connection of a 220V power cord When only two of the available three plugs is connected to the A231 PSUs this will provide 5600 watts to the entire chassis If you connect only two of the total three plugs connect them to ACO and AC1 AC2 should not be connected For details on ILOM refer to the Sun Integrated Lights Out Manager 2 0 User s Guide CMM Behavior The following topics describe the CMM behavior when two power cords are connected to the power supply module You can use the following interfaces to view the power supply information m Section A 2 1 1 IPMItool on page A 6 m Section A 2 1 2 CLI on page A 6 m Section A 2 1 3 Web Inferface on page A 6 Note The command samples in this section use the variable X to represent power supply 0 or power supply 1 Appendix A Sun Blade 6048 ILOM 2 0 Information A 5 A 2 1 1 IPMItool You can use the following ipmitool commands to view power supply information m ipmitool H SPIPaddress U root P changeme sdr This command should display the following output psX status 0x08 ok psX pstate_0 0x01 ok psX pstate_1 0x01 ok psX pstate_2 0
28. Blade 6048 Modular System Service Manual December 2009 powering off system 3 6 powering on the system 4 28 automatically 4 29 using the CLI 4 29 preparing for service 3 1 powering off the system 3 6 tools required 3 8 product updates ix R rear fan faults 2 18 LEDs 2 18 viewing faults in ILOM 2 18 rear fan module installation 4 17 removal 4 17 replacement 4 17 rear panel components CLI ILOM nomenclature 1 10 rear panel of system 1 5 redundant components 1 14 related documentation x reliability availability and serviceability RAS features 1 12 removing chassis management module CMM 4 20 fan board 4 19 front indicator module FIM 4 9 midplane 4 23 network express modules NEMs 4 11 PCI ExpressModules PCI EMs 4 14 power supply modules 4 2 rear fan modules 4 17 server modules 4 6 replacing a failed component 2 19 chassis management module CMM 4 20 fan board 4 19 front indicator module FIM 4 9 midplane 4 23 network express modules NEMs 4 11 PCI ExpressModules PCI EMs 4 13 power supplies 4 2 rear fan modules 4 17 server modules 4 6 S safety guidelines ix sensor readings obtaining using the web interface 2 5 Server Action Required LEDs 2 3 server module installation 4 7 removal 4 6 replacement 4 6 system components 1 2 system features summary 1 12 system management description 1 6 fault management 2 1 T third party Web sites xi
29. E Absent SYSIHDDO_PRSNT Entity Presence Absent SYS HDD1_PRSNT Entity Presence Absent 1SYS T_AMB Temperature 29 000 degrees ISYSIPOT_AMB Temperature 58 000 degrees sSYSIP4 T_AMB Temperature 54 000 degrees 18YSIP2IT_ANE Temperature 51 000 degrees C 8YSIP3 T_AMB Temperature 61 000 degrees ISYSN_0_ 48V Voltage 48 400 Volts ISYSN_1_ 48V Voltage 48 400 Volts ISYSN_2_ 48V Voltage 48 000 Volts I8YSIPON_VCORE Voltage 1 344 Volts I8YSIPON VTT Voltage 0 896 Volts 18YS POW_VDDIO Voltage 1 794 Volts 8YSIP1M_VCORE Voltage 1 344 Volts ISYSIPIN NTT Voltage 0 896 Volts 18YS P1A_VODIO Voltage 1 794 Volts ISYSIP2N_VCORE Voltage 1 358 Volts I8YSIP2N VTT Voltage 0 896 Volts m Note If the server is powered off many components will appear as no reading 4 In the Sensor Readings page do the following a Locate the name of the sensor you want to view b Click the name of the sensor to view the property values associated with that sensor For specific details about the type of discrete sensor targets you can access as well as the paths to access them consult the user documentation provided with the Sun server platform 2 6 Sun Blade 6048 Modular System Service Manual December 2009 2 2 9 2 2 3 1 Monitoring the Event Log Faults are recorded in the system event log which can be viewed from the ILOM CLI or web interface m Viewing or Clearing the ILOM Event Log Using the CLI on page 2 7
30. H command is applied the SEL displays the readings shown in TABLE A 7 TABLE A 7 Start CH SEL Readings Sensor ID Device State Description 3500 Voltage ps1 pstate_0 State Asserted PSU 1 side 0 is on 3700 ps0 pstate_1 State Asserted PSU 0 side 1 is on 3800 Power Supply ch pwrctl state Predictive Failure Now there is enough power Deasserted for the chassis 3900 Voltage ps0 pstate_0 State Asserted PSU 0 side 0 is on 3a00 Voltage ps1 pstate_1 State Asserted PSU 1 side 1 is on A 16 Sun Blade 6048 Modular System Service Manual December 2009 A 3 6 6 A 3 6 7 One PSU Is Removed When one PSU is removed and there is too much power consumption in the chassis to support PSU redundancy the SEL displays the readings shown in TABLE A 8 TABLE A 8 PSU Removed SEL Readings Sensor ID Device State Description 3b00 Power Supply ch pwrctl state Predictive Failure Not enough power is 3c00 Entity Presence ps1 prsnt Asserted Device Absent available for the chassis PSU 1 is absent from the system PSU Is Re inserted TABLE A 9 shows the SEL readings as a PSU is reinserted into the system and the system recognized that power has been re applied TABLE A9 PSU Re inserted SEL Readings Sensor ID Device State Description 3d00 Entity Presence ps1 prsnt Device Present Re insert PSU 1 3e00 Voltage ps1 pstate_0 State Deasserted PSU 1 power was off 3f00 Voltage ps1 pstate_0 State Asserted PSU 1 power is on 4000
31. LED Locations 2 12 2 3 1 2 Checking Other LEDs 2 13 2 3 1 3 Viewing Chassis Shelf Faults in ILOM 2 13 2 3 2 Power Supply Module Faults 2 13 2 3 2 1 Power Supply LED Locations 2 14 2 3 2 2 Power Supply Fault LED Functions 2 15 2 3 2 3 Viewing Power Supply Faults in ILOM 2 15 iv Sun Blade 6048 Modular System Service Manual December 2009 Rear Fan Faults 2 18 2 3 3 1 Rear Fan LED Location 2 18 2 3 3 2 Rear Fan Fault LED Functions 2 18 2 3 3 3 Viewing Rear Fan Faults inILOM 2 18 2 4 Replacing a Faulted Component 2 19 3 Preparing for Service 3 1 3 1 About Replaceable Components 3 1 3 1 1 3 1 2 3 1 3 3 1 4 3 1 5 Accessing Component Information 3 2 Diagnosing and Detecting Component Faults 3 2 Hot Swappable and Hot Pluggable Components 3 3 CRU Features 3 3 FRU Features 3 4 3 2 Locating Front and Rear Components 3 5 3 3 Powering Off the System Before Service 3 6 3 3 1 3 3 2 Powering Off the System Using the CLI 3 7 Powering Off the System Using the Web Interface 3 7 3 4 Required Installation Tools and ESD 3 8 3 4 0 1 Attaching the Antistatic Wrist Strap 3 8 Replacing System Components 4 1 41 Customer Replaceable Units 4 1 4 1 1 Replacing Power Supplies and Front Fan Modules 4 2 4 1 1 1 Removing the Power Supply Modules and Front Fan Modules 4 2 4 1 1 2 Installing the Power Supply Modules and Front Fan Modules 4 4 Replacing Server Modules 4 6 4 1 2 1 Removing Server Modules 4 6 412 2 Installing
32. M N A 0 FEM N A 0 TABLE B 4 Power Requirements for Sun Blade X6250 and its Components Possible Requirement Per Number of Component Component Components Blade Base 50W 1 Requirement Processor 50W 80W 120W 1 2 PCI EMs 25W 0 2 DIMMs 11W 2 16 Disks 12W 0 4 REM 14W 0 1 FEM 25W 0 1 TABLE B 5 Power Requirements for Sun Blade T6300 and its Components Possible Requirement Per Number of Component Component Components Blade Base 250W 1 Requirement Processor N A 0 PCI EMs 25W 0 2 DIMMs 6W 2 16 Disks N A 0 REM N A 0 FEM N A 0 B 4 Sun Blade 6048 Modular System Service Manual December 2009 TABLE B 6 Power Requirements for Sun Blade T6320 and its Components Possible Requirement Per Number of Component Component Components Blade Base 550W 1 Requirement Processor N A 0 PCI EMs 25W 0 2 DIMMs N A 0 Disks N A 0 REM N A 0 FEM N A 0 TABLE B 7 Power Requirements for Sun Blade 6000 Storage Module and its Components Possible Requirement Per Number of Component Component Components Blade Base 50W 1 Requirement Processor N A 0 PCI EMs N A 0 DIMMs N A 0 Disks 12W 0 8 REM N A 0 FEM N A 0 Appendix B Power Management B 5 B 1 3 B 1 4 Power Estimates for Network Express Modules NEMs A chassis can contain 1 or 2 NEMs which must be the same model Power requirements for each model are specified in TABLE B 8 TABLE B 8 NEM Power Requirements NEM Type Power Requirement for Each NEM Bas
33. MB_FAULT 65 75 C This sensor causes the power supply to shut down 12V power output has exceeded 14V for more than CH PSn Sn V 12V_FAULT 400 milliseconds 2 16 Sun Blade 6048 Modular System Service Manual December 2009 TABLE 2 4 Power Supply Module Faults Fault Type CLI Path to Sensor 3V power output reaches the following range CH PSn V 3_3V_FAULT 3 7 4 3V 12V_n output current exceeds 240 amps for more than CH PSn Sn 1 12V_FAULT 60 seconds or 12V_n output current exceeds 275A for 20 msec 3 3V output current exceeds 13A for more than 20 CH PSn I 3_3V_Fault msec TABLE 2 5 Front Fan Faults Fault Type CLI Path to Sensor Front fan has failed CH PSN FAN_FAIL Chapter 2 Fault Management 2 17 2 3 3 2 3 3 1 200 2 2 3 3 3 2 18 Rear Fan Faults The Sun Blade 6048 chassis shelf contains six rear fans Rear Fan LED Location FIGURE 2 6 Rear Fan LED Location Fan Service Action Required LED Rear Fan Fault LED Functions The rear fan fault LEDs indicate when a failure has occurred on a fan module The source of the failure could be mechanical electrical or the result of a midplane controller failure Viewing Rear Fan Faults in ILOM Use the following command to view the sensor for a rear fan fault show CH FMn FAIL The variable n represents the fan module number For example CH FM1 FAIL indicates a fan failure in fan module 1 See the Sun Integrated Lig
34. Manual December 2009 1 4 2 m Provides inventory information and status of components inside the chassis This does not include components managed by the server module SP m Provides an interface to view FRU SEEPROM data and for the following components power supplies fan modules chassis NEMs CMM and server module motherboards The CLI provides an interface to view the status of the components shown in TABLE 1 1 TABLE 1 1 CLI Component Status Component Status Power supply e Presence e Power state e Voltage and current e Fan speed e Temperature e Failure state e Fault state e Warning state Fan module e Presence Fan speed e Failure state Chassis Power state NEMs Presence Server module Presence blade Sun Integrated Lights Out Manager The Sun Integrated Lights Out Manager ILOM firmware is preinstalled on the CMM and it initializes as soon as power is applied to the chassis ILOM provides a command line interface CLI and a web interface that you can use to administer and diagnose local or remote systems Out of band communication is provided through the serial port for CLI console access and through a dedicated Ethernet for CLI via SSH or the web interface In band communication is provided via the host operating system ILOM runs on the CMM independently of the rest of the system using the system s standby power Therefore ILOM continues to function when the system s operating system goes
35. Power Supply ch pwrctl state Predictive Failure Now there is enough power Deasserted for chassis Appendix A Sun Blade 6048 ILOM 2 0 Information A 17 A 18 Sun Blade 6048 Modular System Service Manual December 2009 APPENDIX B Power Management The amount of power used by the blades and other components cannot exceed the amount of power provided by the power supply units PSUs If demand exceeds supply the PSUs might shut down completely and without warning There are two things you can do to avoid this situation m When planning the system calculate the power needs of your blades and other components and verify that they do not exceed the power available The power consumption data you need is provided in Section B 1 Determining Power Requirements on page B 1 m Use the power management system of the chassis ILOM to monitor power usage and to configure backup power supplies See Section B 2 ILOM Power Management Features on page B 6 B 1 Determining Power Requirements The power requirement of a blade system is the sum the following a Power requirement of the chassis itself including the CMM but not including any blades modules or fans m Total power required for the blades including power for internal options processors and DIMMs as well as power for external components attached to each blade m Total power required by Network Express Modules NEMs m Total power required by fans The
36. System Chassis PCI ExpressModules PCI EMs S B 2s a ms ms me eS mem a ms Vi fam Ya fa om Piel ojoon 1 1 5 sea 2 eof og 2 elelee esfe esf 2 o2 eo ee Chassis management module CMM 0 Network express module NEM Rear fans 3 3 3 6 Powering Off the System Before Service Although most components of the Sun Blade 6048 modular system can be removed while the system is powered on you must remove power from the system to service the following components a Midplane m Internal power cables You can power off the system in one of two ways as described in the following sections m Section 3 3 1 Powering Off the System Using the CLI on page 3 7 m Section 3 3 2 Powering Off the System Using the Web Interface on page 3 7 Sun Blade 6048 Modular System Service Manual December 2009 3 3 1 JAZ Powering Off the System Using the CLI You can use the command line interface to power off the system 1 Log in to the ILOM CLI on the CMM 2 Type the following command stop f CH This command powers off the server without a graceful shutdown of the host operating systems Caution Using the f option could result in data loss unless the host operating systems have already been shut down at the server module level For instructions on powering on the system after ser
37. act with the chassis the bottom lever will start to rise and the top lever will start to lower 5 Simultaneously close the ejector levers at the top and bottom of the server module lower the top ejector and raise the bottom ejector until the levers clamp in place against the front of the server module As the ejector levers are closed the server module moves back to engage with the chassis 6 Verify that the server module is powered on and operational After you insert a server module into a powered on system the server module s internal service processor SP is brought online automatically and the server module is powered on by default 7 For each remaining server module to be installed repeat Step 1 through Step 6 4 8 Sun Blade 6048 Modular System Service Manual December 2009 4 1 3 4 1 3 1 Replacing the Front Indicator Module The front indicator module FIM is located in the front of the chassis You can remove and replace a FIM from a powered off system or from a powered on system using hot plug procedures Removing the Front Indicator Module To remove the front indicator module 1 Remove power supply 0 from the chassis See Section 4 1 1 Replacing Power Supplies and Front Fan Modules on page 4 2 FIGURE 4 6 shows how to remove the FIM FIGURE 4 6 Removing the FIM 2 Push back the power supply air baffle 3 Press the button on the right side of the front indicator module FIM Chap
38. age 4 28 4 1 8 Replacing the CMM You will need to replace a chassis management module CMM if the module fails You can remove and replace a CMM from a powered off system or from a powered on system using a hot swap procedure The chassis will be unmanageable while you replace the CMM hardware Note The system fans will run at 100 per cent speed when the CMM is removed from the system as there will be no fan management while the CMM is out of the system 4 1 8 1 Removing the CMM FIGURE 4 17 shows how to remove the CMM 4 20 Sun Blade 6048 Modular System Service Manual December 2009 4 1 8 2 FIGURE 4 17 Removing the CMM 1 In the rear of the chassis locate the CMM 2 Remove any cables on the CMM 3 Press together and hold the ejector button on the module ejector lever 4 To unlatch the module from the chassis pull the ejector lever away from the module and up 5 Pull the ejector lever toward you until you are able to pull the rest of the module out by hand Installing the CMM FIGURE 4 18 shows how to install the CMM Chapter 4 Replacing System Components 4 21 FIGURE 4 18 Installing the CMM 1 Align the CMM with the CMM slot Ensure that the CMM indicator lights of the CMM on the front panel are on the top of the module and facing you 2 Slide the CMM into the vacant CMM chassis slot Ensure that the CMM engages with the system chassis guidance system 3 Complete the installation by closing
39. alue www success 01 46 09 1970 51972 Audit Log minor Thu Jan 8 root Close Session object session type value www success 01 40 59 1970 51971 Audit Log minor Thu Jan 8 root Open Session object sessionitype value www success 01 24 06 1970 61970 audit Lion Lrninor Thu Jang root Close Session ohiert tseccinnihme value ont suce ix Done 10 6 153 126 6 4 In the Event Log page perform any of the following m Page through entries Use the page navigation controls at the top and the bottom of the table to navigate forward and backward through the available data in the table Note that selecting a larger number of entries might cause the web interface to respond more slowly than selecting a smaller number of entries Chapter 2 Fault Management 2 9 2 10 m View the entries in the display by scrolling through the list The following table provides descriptions of the columns that appear in the log Column Label Description Event ID Date Time Class Type Severity Description The number of the event in sequence from number 1 The day and time the event occurred If the Network Time Protocol NTP server is enabled to set the ILOM time the ILOM clock will use Universal Coordinated Time UTC e Audit Log Commands that result in a configuration change Description includes user command command parameters and success or fail e IPMI Log Any event that is p
40. and away from the power supply 3 Align the power supply unit with the power supply slot Ensure that the power supply unit is facing up and that the five LED holes appear on the left when you are installing the power supply module 4 In a smooth motion slide the power supply into the power supply slot until the unit engages with the internal connectors Chapter 4 Replacing System Components 4 5 4 1 2 4 1 2 1 4 6 Replacing Server Modules You can remove and replace a server module blade from a powered off system or from a powered on system using hot plug procedures Caution Do not operate system with empty slots Always insert a filler into an empty slot to reduce the possibility of module shutdown Removing Server Modules 1 Initiate an orderly shutdown of the operating system See the documentation for your server module for information about how to do this Power down the server module to standby power See the documentation associated with the server module for detailed instructions FIGURE 4 3 Removing a Server Module Press the latches on the ejectors at the top and bottom of the server module Swing out both ejector latches simultaneously raise the top ejector and lower the bottom ejector Use the ejector latches to start pulling the server module out of its slot in the chassis Sun Blade 6048 Modular System Service Manual December 2009 C
41. ange the I O configuration you will need to replace a PCI EM You can remove and replace a PCI EM from a powered on system using a hot swap procedure as well as from a powered off system Chapter 4 Replacing System Components 4 13 4 1 5 1 4 14 Caution If you are not immediately replacing the PCI EM install an PCI EM filler panel to ensure proper system operation Removing the PCI EMs Note Check your server module operating system documentation to determine if software commands are needed for the system to recognize PCI EM removal or installation FIGURE 4 11 shows how to remove the PCI EMs FIGURE 4 11 Removing a PCI EM In the rear of the chassis locate the PCI EM that you want to remove If the amber Attention indicator on a PCI EM is lit this indicates a problem with the PCI EM and you might choose to replace it Otherwise if you want to change the 1 0 configuration you can choose any PCI EM to replace If you are hot plugging the PCI EM prepare the operating system You will need to prepare for PCI EM removal through the associated server module operating system See the server module blade or operating system documentation for more information Verify that the PCI EM Power indicator is off If you have successfully prepared the PCI EM for a hot removal the Power indicator turns off Place one finger on top of ejector lever handle Sun Blade 6048 Modular System Service Manual
42. assis In general this applies to any component with a FRU ID or present or not present indicators For more information about accessing component information see the Sun Integrated Lights Out Manager 2 0 User s Guide You can view the following component information m Part number m Revision level m Serial number m Any fault data associated with the component Component information is accessible even when the component itself is powered off When you remove or replace a component the management interfaces report and log the change in the component inventory within 10 seconds Diagnosing and Detecting Component Faults The Sun Blade 6048 modular system uses fault management software to monitor hardware health and to diagnose and report hardware failures on system components When a sensor on the system chassis power supplies or fans registers a problem the fault management software diagnoses the problem and if it determines that a hardware failure has occurred the software lights the component s Service Action Required LED The ILOM management interfaces are also updated to reflect the hardware failure and the failure is recorded as a fault in the event log For further information about the fault management software and how to monitor faults see Chapter 2 Sun Blade 6048 Modular System Service Manual December 2009 3 1 3 3 1 4 Hot Swappable and Hot Pluggable Components Within the Sun Blade 6048 modular system most
43. aution Do not use the ejector latches to carry the server module The ejector latches should be only used to unseat the server module from the chassis The latches are not designed to support the weight of the server module 6 Use your hands to finish pulling the server module out of the chassis supporting the weight of the server module from the bottom 7 Install a filler panel into the server module slot if the server module will be out of the chassis for longer than one minute 4 1 2 2 Installing Server Modules To install server module 1 Remove the filler panel if necessary FIGURE 4 4 Removing a Filler Panel 2 Rotate the lower ejector lever on the filler panel downward and pull the filler panel out of the chassis Note Do not discard the filler panel You will need to reinstall the filler panel if the server module needs to be serviced or removed at a later time Chapter 4 Replacing System Components 4 7 FIGURE 4 5 shows how to install a server module FIGURE 4 5 Installing a Server Module 3 Align the server module with the vacant server module slot Ensure the following a Server module indicator lights on the front panel are facing toward you a Server module ejector levers are fully opened Server module ejectors are on the right side of the module 4 Slide the server module into the vacant server module chassis slot As the ejectors make cont
44. be installed on the chassis 3 Secure the bottom of the each bracket to the hole in the platform underneath the chassis 4 34 Sun Blade 6048 Modular System Service Manual December 2009 APPENDIX A Sun Blade 6048 ILOM 2 0 Information This appendix contains ILOM 2 0 information that is specific to the Sun Blade 6048 modular system For additional information on using ILOM with the Sun Blade modular system see the Sun Integrated Lights Out Manager 2 0 User s Guide 820 1188 The following topics are covered in this appendix m Section A 1 Activating CMM Ethernet Ports on page A 2 m Section A 2 ILOM Behavior With Two Power Cord Configuration on page A 5 m Section A 3 ILOM Sensor Information on page A 9 A 1 Activating CMM Ethernet Ports By default Ethernet Port 0 is enabled on the CMM You can enable Port 1 or enable both ports through the CLI or the web interface network if you activate both Ethernet ports on the CMM Before you activate both ports make sure the external switch supports trunk mode The upstream Ethernet switch needs to be configured correctly so that no Ethernet traffic loop is created This is done usually by the spanning tree algorithm Caution You can cause Ethernet networking problem and bring down the external A 1 1 To Enable Ethernet Ports Through the CLI To enable port 1 via the CLI 1 Log in to the ILOM CLI ssh root ipaddress 2 Type
45. ce A 3 A2 ILOM Behavior With Two Power Cord Configuration A 5 A 2 1 CMM Behavior A 5 A 2 1 1 IPMitool A 6 A 2 12 CLI A 6 A 2 13 Web Inferface A 6 A 2 2 Server Module Behavior A 7 A 2 2 1 IPMitool A 7 A 2 2 2 CLI A 8 A 2 2 3 Web Interface A 8 A 3 ILOM Sensor Information A 9 A 3 1 Basic Status Sensors A 9 A 3 2 Power Supply Sensors A 9 A 3 3 Fan Sensors A 10 A 3 4 Server Module Present A 12 A 3 5 Chassis Sensors A 12 A 3 6 Readings for Specific Cases A 13 A 3 6 1 CMM Boots A 13 A 3 6 2 AC Cables Are Disconnected A 14 A 3 6 3 All the AC Cables Get Disconnected Then Plugged Back In A 15 A3 64 Stop CH A 16 Contents vii A 3 6 5 Start CH A 16 A 3 6 6 One PSU Is Removed A 17 A 3 6 7 PSU Is Re inserted A 17 B Power Management B 1 B 1 Determining Power Requirements B 1 B 1 1 Power Required by the Chassis B 2 B 12 Power Required by Blades B 2 B 1 3 Power Estimates for Network Express Modules NEMs B 6 B 1 4 Power Requirements for Fans B 6 B 2 ILOM Power Management Features B 6 Index Index 1 viii Sun Blade 6048 Modular System Service Manual December 2009 Preface This Sun Blade 6048 Modular System Service Manual contains information and procedures for maintaining and upgrading the modular system Before You Read This Document It is important that you review the safety guidelines in the Sun Blade 6048 Modular System Safety and Compliance Guide Product Updates For product updates that you can download
46. ch serve as summary indicators for all component faults remain illuminated as long as any system component remains in a faulted state When Only the Chassis Shelf LEDs Are Lit If the chassis Service Action Required LEDs are illuminated but no other system component has a lit Service Action Required LED then fault management has diagnosed an external fault a problem outside the system that potentially affects the system as a whole For example if the external ambient air temperature exceeds 45 C a fault is declared and the system shuts down although there is nothing physically wrong with any system hardware Refer to Section 2 3 1 Chassis Shelf Faults on page 2 11 for information about the external conditions that can cause these chassis faults Chapter 2 Fault Management 2 3 2 2 1 3 222 2224 2 4 About Power Supply Faults The power supply units PSUs are a special case they monitor their own fault status and control their own Service Action Required LEDs The fault management software cannot turn the PSU LEDs on or off However because fault management is monitoring sensors on the PSUs it is notified when a PSU fault occurs Fault management illuminates the chassis shelf Service Action Required LEDs and notes the fault occurrence in the ILOM management interfaces and in the event log Note that it is possible for a PSU to extinguish its Service Action Required LED declare that the fault is cleared but for faul
47. customer replaceable units CRUs and field replaceable units FRUs are hot swappable and some components are hot pluggable Both hot swappable and hot pluggable components can be removed and installed while the system is running without affecting the rest of the system s operations However for hot pluggable components you must prepare the operating system prior to the hot plug procedure by performing certain system administration tasks The system continues to operate normally during a hot swap or hot plug operation of redundant components However there are constraints as to the period of time you can perform removal and replacement of certain components Refer to specific service procedures for that information CRU Features CRUs are components that customers can replace on site without the assistance of authorized service personnel The following table describes the components and indicates whether they are hot swappable or hot pluggable TABLE 3 1 CRU Features CRU Hot swap or Description Hot plug Server modules blades Power supply modules Front fan modules Front indicator module FIM Installed from the front the server modules contain the core CPU and Hot plug memory subsystem The server modules also contain the PCI Express I O connectivity to the NEMs and PCI EMs Installed from the front the chassis supports two power supply Hot swap modules Each power supply module provides approximately 280
48. d Lower Non Deassert Yes Fan speed has returned to normal from lower non Recoverable recoverable This indicates that the fan has returned to normal or has been replaced fmn fail These sensors report fan module failure status TABLE6 fmn fail Readings Reading State Event Description 0x0001 Predictive Failure No This state indicates that the fan module has not Deasserted failed 0x0002 Predictive Failure Yes This state indicates that the fan module has Asserted failed Appendix A Sun Blade 6048 ILOM 2 0 Information A 11 A 3 4 A 3 5 A 12 Server Module Present bln prsnt These sensors report whether server modules are present in the specified slots Slots are numbered from 0 TABLE7 Server Module Present States Reading State Event Description 0x0001 State Deasserted No This state indicates that the server module is not present 0x0002 State Asserted No This state indicates that the server module is present Chassis Sensors ch t_ovrd This sensor reports if any blade is requesting full fan speed due to high temperature TABLE A 1 Full Fan Speed State Reading State Event Description 0x0001 State Deasserted No This state indicates that no blade is requesting full fan speed 0x0002 State Asserted No This state indicates that one ore more blades is requesting full fan speed ch t_amb_n Two sensors ch t_am_0 and ch t_amb_1 report ambient temperature in the chassis midp
49. dules NEMs one chassis management module CMM and six fan modules All active chassis components that are critical to system operation are configured for redundancy The Sun Blade 6048 modular system is designed for ease of service by either the customer for user upgradeable components or by authorized service personnel FIGURE 1 1 shows the front and rear views of the chassis Sun Blade 6048 Modular System Service Manual December 2009 FIGURE 1 1 Front and Rear Views of Sun Blade Modular System Chassis HE AE PREEE EEE EEE E TU ft Pe ju ann HoA aag PEPEEEEELEEE Bi Ue Poh Pr I A 1 PEEEEEE 1 Le Ce Ce EE S Us CP a CE UM Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 3 FIGURE 1 2 Single Shelf Component View Power supplies Front indicator module FIM Server or storage modules blades PCI ExpressModules PCI EMs Chassis management medula OMM oooooo Network express modules NEMs Rear fans 1 4 Sun Blade 6048 Modular System Service Manual December 2009 12 Server Modules The Sun Blade 6048 modular system architecture includes capacity for up to 48 hot pluggable server modules Refer to the documentation that comes with the server module for specific information about each server module 1 35 I O Connectivity All 1 0 is PCI Express based on customer replaceable units CRUs called PCI ExpressModules PCI EMs and ne
50. e following procedure to prevent static damage when you are accessing the server modules and server module options Caution Printed circuit boards and hard drives contain electronic components that are extremely sensitive to static electricity Ordinary amounts of static from your clothes or the work environment can destroy components Do not touch the components themselves or any metal parts without taking proper antistatic precautions 1 Use an antistatic mat or similar surface When installing an option onto the server module such as a DIMM or disk drive place those static sensitive parts on an antistatic surface You can use the following items as an antistatic surface a The bag used to wrap a Sun part The shipping container used to package a Sun part Sun electrostatic discharge ESD mat part number 250 1088 available through your Sun sales representative Disposable ESD mat shipped with some Sun parts and options 2 Attach an antistatic wrist strap Attach one end of the wrist strap to the chassis sheet metal and the other end to your wrist Refer to the instructions that come with the strap 3 Detach both ends of the strap when you are finished installing the parts Sun Blade 6048 Modular System Service Manual December 2009 CHAPTER 4 Replacing System Components This chapter contains information and procedures for servicing the Sun Blade 6048 server module hardware including component r
51. ed with the fan module before replacing the module Do not leave a power supply slot vacant Power supplies have integrated fans that will continue to supply cooling air even when the power supply is off You can remove and replace a power supply from a powered off system or from a powered on system using a hot swap procedure Note When removing a power supply from the system wait for all LED indicators on the power supply to turn off before reinstalling the power supply If you do not follow this procedure the system might log a false power supply fault condition Use the following procedures to install a power supply into the corresponding vacant slot Removing the Power Supply Modules and Front Fan Modules FIGURE 4 1 shows how to remove the power supplies and front fan modules Note Remove only one power supply at a time One power supply must be in the system at all times to supply power to the system Sun Blade 6048 Modular System Service Manual December 2009 FIGURE 4 1 Removing a Power Supply and Front Fan Module Note You will need to replace the power supply that you removed within one minute to ensure proper system operation 1 In the front of the chassis locate the power supply to be removed or the power supply that contains the front fan module that you plan to replace 2 Press and hold the green button on top of the powe
52. emoval and replacement procedures The following topics are covered in this chapter m Section 4 1 Customer Replaceable Units on page 4 1 m Section 4 2 Field Replaceable Units on page 4 23 m Section 4 3 Servicetool FRU Update Procedure on page 4 27 m Section 4 4 Powering On the System After Service on page 4 28 m Section 4 5 Expansion Module Installation on page 4 31 m Section 4 6 Floor Tie Down Installation on page 4 33 4 1 Customer Replaceable Units The following procedures describe how to replace customer replaceable unit CRU components m Section 4 1 1 Replacing Power Supplies and Front Fan Modules on page 4 2 m Section 4 1 2 Replacing Server Modules on page 4 6 m Section 4 1 3 Replacing the Front Indicator Module on page 4 9 m Section 4 1 4 Replacing NEMs on page 4 11 m Section 4 1 5 Replacing PCI EMs on page 4 13 m Section 4 1 6 Replacing the Rear Fan Modules on page 4 17 m Section 4 1 7 Replacing the Fan Board on page 4 19 m Section 4 1 8 Replacing the CMM on page 4 20 m Section 4 1 9 Installing the Cooling Doors on page 4 23 4 1 4 1 1 4 1 1 1 4 2 Replacing Power Supplies and Front Fan Modules You will need to replace a front fan module or power supply module if the module fails A front fan module is located within each of the power supply modules so you will need to remove the power supply associat
53. er is supplied to the fan array server modules and the network express modules NEMs Chapter 4 Replacing System Components 4 29 4 4 3 Powering On the System Using the CLI 1 Log in to the ILOM CLI ssh root ipaddress 2 Type start CH This command powers on the chassis and is available only to administrators 4 4 4 Powering On the System Using the Web Interface 1 Open a web browser and type the IP address of the server SP or CMM The Login page for the ILOM web interface appears 2 In the ILOM Login page enter a user name and password then click OK The ILOM web interface appears 3 Select the CMM view from the left panel 4 From the Remote Control tab click the Remote Power Control tab 4 30 Sun Blade 6048 Modular System Service Manual December 2009 FIGURE 4 22 Remote Control Tab Sun Integrated Lights Out Manager Mozilla Firefox Ele Edit View History Bookmarks Tools Help ministrator root CMM Hostname 5 Integrated Lights Out Manager E Blade o E Blade 1 E Blade 3 E Brave 4 Blades E grade Blade 7 H slaves F Blades E Blase 11 gt amp ft Li https 10 6 153 126 iPages suntab asp_ Remote Control E Name Current State Remote Power Control Control the chassis power from this page To change the chassis power state select the radio button next to CH then choose an optio
54. g them outward 5 Holding the opened ejector levers pull the NEM toward you until you can pull the rest of the module out by hand 4 1 4 2 Installing the NEMs FIGURE 4 9 shows how to install the NEMs Panel 1a shows installation of a dual slot NEM and Panel 1b shows installation of a single slot NEM FIGURE 4 9 Installing a NEM 4 12 Sun Blade 6048 Modular System Service Manual December 2009 4 1 5 1 Align the NEM with the vacant NEM slot Ensure that The RJ 45 port connectors of the NEM are facing you a The NEM ejector levers are fully opened 2 Slide the NEM into the vacant NEM chassis slot until you feel it stop 3 Complete the installation by closing the ejector levers to secure the NEM in the chassis Replacing PCI EMs A pair of PCI EMs is assigned to each server module in the chassis All PCI EMs connect to a chassis midplane The midplane provides an interconnect between the server modules and the PCI EMs installed in the chassis The midplane connections between the server module and the PCI EM are logically defined by the midplane as shown in FIGURE 4 10 FIGURE 4 10 PCI EM to Server Module Correspondence BL BL BL BL BL BL BL BL B 11 10 9 8 7 6 5 4 3 2 GO O0 BCE e ue SP e W PE C O If the PCI Express Module PCI EM fails or if you choose to ch
55. hen Sensor Readings tab The following readings will be displayed SYS PS1 VINOK2 State Deasserted SYS PS1 PWROK2 State Deasserted SYS PSO VINOK2 State Deasserted SYS PSO PWROK2 State Deasserted A 8 Sun Blade 6048 Modular System Service Manual December 2009 A 3 A 3 1 A 3 2 ILOM Sensor Information The chassis includes a number of sensors that generate entries in the system event log SEL when the sensor crosses a threshold Many of these readings are used to adjust the fan speeds and perform other actions such as illuminating LEDs and powering off the chassis These sensors can also be configured to generate IPMI PET traps as described in the Integrated Lights Out Manager ILOM Administration Guide 820 0280 Caution Do not use any interface other than the ILOM CLI or web GUI to alter the state or configuration of any sensor or LED Doing so could void your warranty Basic Status Sensors nemn prsnt These sensors report whether the network express modules NEMs are present NEMs are numbered from 0 TABLE1 nemn prsnt Readings Reading State Event Description Action 0x0001 Device Absent Yes NEM is absent none 0x0002 Device Present Yes NEM is present none Power Supply Sensors There are two power supplies in the chassis Each power supply has two AC input lines and two DC output lines All are numbered from 0 Appendix A Sun Blade 6048 ILOM 2 0 Information A 9 psn prsn
56. hts Out Manager 2 0 User s Guide and Appendix A for more information about reading this and other rear fan sensors Sun Blade 6048 Modular System Service Manual December 2009 2 4 Replacing a Faulted Component When a fault indicates a hardware failure the recommended method for clearing the fault is to replace the failed component To replace a failed component 1 Determine which system component has experienced a hardware failure Look at the Service Action Required LEDs and the event log to get information about the component failure See Section 2 3 Determining That Hardware Has Failed on page 2 11 2 Remove and replace the failed component Refer to the instructions in Chapter 4 3 Monitor the component LEDs to confirm that the fault is cleared Chapter 2 Fault Management 2 19 2 20 Sun Blade 6048 Modular System Service Manual December 2009 CHAPTER 3 Preparing for Service This chapter contains information on preparing to service the Sun Blade 6048 modular system The following sections are in this chapter m Section 3 1 About Replaceable Components on page 3 1 m Section 3 2 Locating Front and Rear Components on page 3 5 m Section 3 3 Powering Off the System Before Service on page 3 6 m Section 3 4 Required Installation Tools and ESD on page 3 8 3 1 About Replaceable Components The Sun Blade 6048 modular system is designed for ease of use by both cust
57. ic NEM 0 NEM 24W IB NEM 173W Hydra NEM 80W Power Requirements for Fans A chassis has two kinds of fans m Fans in fan modules Each module requires 65W m Fans in power supplies Each power supply requires 72W B 2 B 6 ILOM Power Management Features The CMM ILOM power management system performs two functions m It reports values for power consumption and PSU power efficiency m It determines the power requirements of installed components and only allows a blade to power up if doing so does not increase power requirements past power available The power requirements assumed by the CMM ILOM are documented in Section B 1 Determining Power Requirements on page B 1 You can access power data and also configure the power management system through the CMM ILOM WebGUI under the System Monitoring Power Management tab as shown in FIGURE B 1 Sun Blade 6048 Modular System Service Manual December 2009 FIGURE B 1 Power Management Tab in CMM ILOM WebGUI G Chassis System Monitoring Indicators Sensor Readings Event Logs Power Management Blade 0 Blade 1 Power Management Blade 2 View and configure power metrics Blade 3 Blade 4 ene Total Power Supplies 2 Blade 6 Reserve Power Supplies O Blade 7 Blade 8 Cone Blade 9 Blade 10 Blade 11 Name Unit Chassis DC Consumed Power Watts Chassis AC Consumed Power Watts PSU 0 AC Consumed Power Watt
58. ice indicators The slow blink amber function for the PCI Express Modules is a Locate function that is specified by the PCI ExpressModule standard The LED states are displayed as follows m Steady on a Off m Slow blink Blinks once per second a Fast blink Blinks four times per second a Standby Turns on for 100 ms and off for 2900 ms TABLE 1 7 LED Indicators Rear Front Power Indicator Color State CMM PCIEM Fan Fan Supply OK Normal Green Off X X X Standby X On X Fault Amber Off X X X X Slow X X Blink On X X X X Locate White Off X Fast blink X Sun Blade 6048 Modular System Service Manual December 2009 TABLE 1 8 shows the functions of the system LEDs and buttons TABLE 1 8 LED and Button Descriptions Button or Indicator Color Description Locator indicator White This LED provides the following indications chassis e Fast Blink Helps you locate a specific system in a data center Off Normal operating state You can illuminate the Locator indicator either by executing a software command on the CMM or by momentarily pressing the associated Locator button that is integrated with the LED Locator button n a This button provides the following function chassis e Quick press Lights the Locator indicator Fault Service Action Amber This LED provides the following indications Required indicator e Steady on or slow blink indicates that the chassis rear fans component or a component that a
59. laced in the IPMI SEL is also put in the management log e Chassis State For changes to the inventory and general system state e Chassis Action Category for shutdown events for server module or chassis hot insert or removal of a FRU and Reset Parameters button pushed e FMA Fault For Fault Management Architecture FMA faults Description gives time of fault as detected by FMA and suspect component e FMA Repair For FMA repairs Description gives component Critical Major or Minor A description of the event m Clear the event log To clear the event log click the Clear Event Log button A confirmation dialog box appears In the confirmation dialog box click OK to clear the entries Note The ILOM event log accumulates many types of events including copies of IPMI entries Clearing the ILOM event log clears all entries in the log including the IPMI entries However clearing the ILOM event log entries does not clear the actual entries posted directly to an IPMI log Sun Blade 6048 Modular System Service Manual December 2009 2 5 24 1 Determining That Hardware Has Failed When a hardware failure occurs the following actions take place m One of the following fault LEDs are illuminated a The amber Service Action Required LED is illuminated on the failed component and the chassis shelf Service Action Required LEDs both front and back are illuminated a The Temperature Fail LED is illuminated on
60. lane These are for monitoring only no threshold is defined ch pwrctl state Reports power supply oversubscription state Sun Blade 6048 Modular System Service Manual December 2009 A 3 6 A 3 6 1 TABLE A 2 Power supply oversubscription state Reading State Event Description 0x0001 State Deasserted No Power supplies are not oversubscribed blades are allowed to turn on 0x0002 State Asserted No Power supplies are oversubscribed blades are not allowed to turn on Readings for Specific Cases This section shows some of the sensor readings for the SEL in cases that are specific to the Sun Blade 6048 modular system CMM Boots When the CMM boots up the SEL displays the readings shown in TABLE A 3 TABLE A 3 CMM Boot SEL Readings Sensor ID Device State Description 100 Entity Presence nem0 prsnt Device Present NEM 0 is present 200 Entity Presence ps0 prsnt Device Present Power supply PSU 0 is present 300 Entity Presence ps1 prsnt Device Present PSU 1 is present 400 Voltage ps0 pstate_0 State Asserted All the 4 ACs are connected and DC output fine 500 Voltage ps0 pstate_1 State Asserted All the 4 ACs are connected and DC output fine 600 Voltage ps1 pstate_0 State Asserted All the 4 ACs are connected and DC output fine 700 Voltage psl pstate_1 State Asserted All the 4 ACs are connected and DC output fine 800 Entity Presence bl0 prsnt Device Present Blade 0 is present 900 Entity Presence bl1 p
61. le Components Sun Blade 6048 modular system hardware supports hot swapping of the network express modules NEMs PCI ExpressModules PCI EMs chassis management module CMM both front and rear fan modules and power supply modules You can install or remove these components while the system is running without using any software commands Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 13 1 6 3 Redundant Components The Sun Blade 6048 modular system provides redundant components that enable the system to continue operations if one of the associated components fails This separation of functions minimizes the impact of component problems and servicing The redundant components include the following m Power supply units m System fans front and rear The following modules might be redundant depending on the system configuration m Server modules m PCI ExpressModules m Network express modules 1 6 4 Environmental Monitoring The Sun Blade 6048 modular system features an environmental monitoring subsystem designed to protect components against the following m Extreme temperatures m Lack of adequate airflow throughout the system m Power supply failures m Hardware faults Temperature sensors located throughout the system monitor the ambient temperature of the chassis and internal components The software and hardware ensure that the temperatures within the chassis do not exceed predetermined safe operating ranges
62. les and PCI ExpressModules Chassis management Hot swappable standby operation module CMM ILOM and Intelligent per blade and chassis wide management functions CMM system management ILOM continues to function when the operating system goes offline or the system is powered off Hardware upgrades No tools required to access user upgradeable modules Power on and restart Automatic server restart network based booting capability Troubleshooting Troubleshooting includes Environmental monitoring e Rapid response lighting e Service LED indicators e System error logging including logging to the system event log SEL Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 15 1 6 6 1 16 LED Indicators and Buttons The Sun Blade 6048 chassis system indicators follow the ANSI Vita 40 2003 Status Indicator Standard as well as Sun Microsystems Service Indicator Application Guidelines There are three types of indicators for chassis components Normal green Fault amber and Locate white although not every customer replaceable unit CRU will have all indicator types The subset of indicators and functionality chosen for each component depends on what is appropriate for a given CRU See the documentation for the server modules for information about server module LEDs TABLE 1 7 summarizes all service indicators and their modes in the Sun Blade 6048 system The indicators on the CMM are the same as the chassis serv
63. ling the Midplane Caution Thermal damage to the chassis can occur Failure to remove the gasketing material prior to installing the midplane might result in damage to chassis FIGURE 4 21 shows how to install the midplane FIGURE 4 21 Installing the Midplane 4 26 Sun Blade 6048 Modular System Service Manual December 2009 4 3 Before installing the midplane remove any gasketing material from the chassis that is present in the midplane area See Step 7 in the Removing the Midplane section for more information Tilt the midplane to the side slightly to fit it into the chassis When you reach the back wall of the chassis straighten the midplane so that it is flush with the back of the chassis and lift it so that the guidepin holes are aligned with the chassis guidepins Push the midplane backward slightly to engage the chassis guidepins Fasten the 16 screws to secure the midplane to the chassis Replace the air baffle a Align the air baffle so that the fastening snaps are facing away from the inside of the chassis a Place the back edge of the air baffle in the slot on the back of the chassis b Rotate the air baffle upward until the snaps fasten into place at the back of the power supply cage Replace all of the components that were removed from the chassis before you install the midplane except for the gasketing material the you removed See
64. ly module 1 PS1 12V output 0 S0 12V output 1 S1 or 12V output 1 S2 For example CH PS0 S1 represents 12 V output 1 located within power supply module 0 Unless noted otherwise the sensors shown in the following tables will display the following value if a fault has occurred value Predictive Failure Asserted Chapter 2 Fault Management 2 15 Note If you are using two power cords per power supply the ILOM readings will be different Refer to Section A 2 ILOM Behavior With Two Power Cord Configuration on page A 5 for more information TABLE 2 3 Power Supply Module Warnings Fault Type CLI Path to Sensor Power supply input lost or out of range Possible values CH PSn STATUS are e Presence detected e Power supply failure detected e Predictive failure e Power supply input lost AC DC e Power supply input lost or out of range e Power supply input out of range but present This sensor shows the ambient temperature of the CH PSn T_AMB power supply The CMM LED turns on when ambient reaches 40 C and the chassis shelf shuts down when the temperature reaches 45 C 12V_n output current exceeds 240A for 100 msec CH PSn Sn 1 12V_WARN Ambient temperature reaches the following range CH PSn T_AMB_WARN 509 600 C TABLE 2 4 Power Supply Module Faults Fault Type CLI Path to Sensor Power supply has failed CH PSn Sn AcFAIL Ambient temperature reaches the following range CH PSn T_A
65. m Viewing or Clearing the ILOM Event Log Using the Web Interface on page 2 9 Viewing or Clearing the ILOM Event Log Using the CLI To view or clear events in the system event log using the ILOM CLI 1 Establish a local serial console connection or SSH connection to the CMM and log in to the ILOM 2 Type the following command paths to set the working directory cd CMM logs event 3 Type the following command path to display the event log list show list The contents of the event log appears An example follows ID Date Time Class Type Severity 50611 Wed Aug 15 16 55 56 2007 Audit Log minor root Open Session object session type value shell success 50610 Wed Aug 15 16 44 44 2007 Audit Log minor root Open Session object session type value shell success 50609 Tue Aug 14 18 03 45 2007 Audit Log minor 4 In the event log perform any of the following tasks m Scroll down the list to view entries Press any key except q The following table provides descriptions of the columns that appear in the log Chapter 2 Fault Management 2 7 Column Label Description Event ID The number of the event in sequence from number 1 Date Time The day and time the event occurred If the Network Time Protocol NTP server is enabled to set the ILOM time the ILOM clock will use Universal Coordinated Time UTC Class Type e Audit Log Commands that result in a configura
66. module contains power supplies front is broken and needs to be replaced On the FIM or fans and PCI EMs CMM this LED provides a summary of all other Service Action Required FRU and CRU indicators in the chassis It remains steady on as long as any Service Action Required indicator on any FRU or CRU in the chassis is lit in either a slow blink or steady on state e Off Service is not required Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 17 TABLE 1 8 LED and Button Descriptions Continued Button or Indicator Color Description OK indicator chassis Green power supplies CMM and PCI EMs Locator indicator White chassis DC OK power Green supply module Ethernet link activity Green CMM This LED provides the following indications Standby blink Indicates that the unit is being managed The unit is not operational but it is ready to be activated On the FIM or CMM this LED indicates that the CMM has begun initialization When the CMM has completed its initialization and has begun to actively manage the chassis the OK indicator changes to the standby blink state At this state only the chassis summary indicators and CMM indicators are active Steady on Indicates that the unit is operating normally e Off Remains off before the chassis has powered on and before the chassis has reached the standby blink state This LED provides the following indications Fast
67. n Microsystems Inc in the U S and in other countries All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc in the U S and in other countries Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems Inc AMD Opteron is a trademark or registered trademark of Advanced Microdevices Inc The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems Inc for its users and licensees Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry Sun holds a non exclusive license from Xerox to the Xerox Graphical User Interface which license also covers Sun s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun s written license agreements U S Government Rights Commercial use Government users are subject to the Sun Microsystems Inc standard license agreement and applicable provisions of the FAR and its supplements DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Copyright 2009 Sun Microsystems Inc 4150 Network Circle Santa Clara Californie 95054 Etats
68. n from the Action drop down list immediate Power Off cuts power to the system components including the blades Graceful Shutdown and Power Of attempts to bring the OSs down gracefully then cuts power to the system components Power On gives full power to the chassis and blades subject to system policies Power Cycle brings the system to power off then automatically powers the system back on not applicable to CH Remote Power Control Actions iv lt m IE 10 6 153 126 amp 5 Select the radio button next to CH chassis 6 Use the drop down list to select a new power state for the chassis The chassis current power state appears in the Current State column a Select the Power On value to power on the host b In the confirmation dialog box click OK to confirm the change 4 5 Expansion Module Installation This section describes installation of a 2U expansion module for the Sun Blade 6048 cabinet This expansion module is offered as a separate orderable option The module will enable you to mount a 2 RU system on top of the Sun Blade 6048 system The following illustrations shows the procedure for installing the top cap on a Sun Blade 6048 cabinet Chapter 4 Replacing System Components 4 31 D 1 Unpack the contents of the expansion module kit see panel 1 The expansion module kit contains two expansion module mounting brackets and four M6 screws 2
69. omers and authorized service personnel All components in the Sun Blade 6048 modular system are removable and serviceable by a single person Most components are customer replaceable units CRUs however some components are field replaceable units FRUs which must be removed and installed only by authorized Sun service personnel See Section 3 1 4 CRU Features on page 3 3 and Section 3 1 5 FRU Features on page 3 4 to determine which components are CRUs and which components are FRUs When a component fails its associated amber Service Action Required LED indicator lights See Section 3 1 2 Diagnosing and Detecting Component Faults on page 3 2 The Sun Blade 6048 modular system is designed with a midplane to which various replaceable units are installed from the front and rear of the chassis Components that are hot pluggable and hot swappable do not require tools for removal or replacement When you replace a component an indicator light goes on or an audible click sounds so that you know the component is properly installed For 3 1 aL ada 3 2 a list of hot pluggable and hot swappable components and those components that require the system or associated module to be powered off see Section 3 1 3 Hot Swappable and Hot Pluggable Components on page 3 3 Accessing Component Information You can use the CMM ILOM CLI interface or web interface to access information about some components installed in the ch
70. on for more information 6 Verify that the PCI EM Power indicator is On If the hot insertion procedure was successful the green Power indicator will light 7 For each remaining PCI EM to be installed repeat Step 1 through Step 6 4 16 Sun Blade 6048 Modular System Service Manual December 2009 4 1 6 4 1 6 1 4 1 6 2 Replacing the Rear Fan Modules You will need to replace a rear fan module if the module fails You can remove and replace a rear fan from a powered off system or from a powered on system using a hot swap procedure Removing the Rear Fan Modules FIGURE 4 13 shows how to remove rear fan modules FIGURE 4 13 Removing a Fan Module 1 In the rear of the chassis locate the fan module that you want to remove 2 Press and hold the green button on the fan handle 3 Pull out the rear fan module in a smooth motion until it is free from the chassis Installing Rear Fan Modules FIGURE 4 14 shows how to install rear fan modules Chapter 4 Replacing System Components 4 17 FIGURE 4 14 Installing a Fan Module 1 In the back of the system chassis locate a vacant fan module slot 2 Using both hands align the fan module with the vacant slot Make sure that the green button on the handle of the fan module is at the top of the fan 3 Slide the fan module into the vacant slot until the front panel meets the chassis After you install the rear fan module the chassis management module CMM will
71. ont utilis es sous licence et sont des marques de fabrique ou des marques d pos es de SPARC International Inc aux Etats Unis et dans d autres pays Les produits portant les marques SPARC sont bas s sur une architecture d velopp e par Sun Microsystems Inc AMD Opteron est une marque de fabrique ou une marque deposee de Advanced Microdevices Inc L interface d utilisation graphique OPEN LOOK et Sun a t d velopp e par Sun Microsystems Inc pour ses utilisateurs et licenci s Sun reconna t les efforts de pionniers de Xerox pour la recherche et le d veloppement du concept des interfaces d utilisation visuelle ou graphique pour l industrie de l informatique Sun d tient une license non exclusive de Xerox sur l interface d utilisation grephique Xerox cette licence couvrant galement les licenci es de Sun qui mettent en place l interface d utilisation graphique OPEN LOOK et qui en outre se conforment aux licences crites de Sun LA DOCUMENTATION EST FOURNIE EN L TAT ET TOUTES AUTRES CONDITIONS DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE A L APTITUDE A UNE UTILISATION PARTICULIERE OU A L ABSENCE DE CONTREFA ON Si Ca Adobe PostScript Contents Preface ix Introduction to the Sun Blade 6048 Modular System 1 1 1 1 System Components 1 2 1 2 Server Modules 1 5 1
72. ory Bookmarks Tools Help Se gt S A Ditttesii10 6 153 126 agestsuntab asp FE D naag D Latest Builds 7 p nistrator root CM ame 00144F689672 Integrated Lights Out Manager Configuration EE 0 System Management Access Alert Management Network Serial Port Clock Settings Syslog SMTP Client H biade 1 Network Settings Viewthe MAC address and configure network settings for the Chassis Monitoring Module from this page DHCP is the default mode but you can manually E Biade3 configure the Netmask Gateway and IP address Selectthe radio button next to the appropriate mode then enter settings as needed The CMM Ethernet a Blade 4 Management Network switch setting can also be configured E Blades a Blade 6 MAC Address 00 1 4 4F 6B 96 72 lE Blade 7 Obtain an IP Address Automatically E Blades use DHCP E Badeg Use the Following IP Address le IP Address 10 6 153 126 E Biade1t Subnet Mask 255 255 252 0 Gateway 10 6 152 1 CMM Management Network Por D Active Switch om 10 6 153 126 amp In the CMM Management Network Switch drop down list select the port that you want to activate or Trunking Link Aggregation to activate both ports 5 Remove the CMM and reinstall it into the chassis The active port is now NET MGT port 1 or both NET MGT ports Sun Blade 6048 Modular System Service Manual December 2009 A 2
73. r supply handle frame 3 Rotate the power supply handle away from the power supply and down 4 Pull the power supply out of the chassis approximately 100 cm 5 Choose the instructions below that correspond to the module that you want to replace a If you are replacing the whole power supply While supporting the bottom of the power supply module with both your hands pull the power supply all the way out of the chassis Chapter 4 Replacing System Components 4 3 a If you are planning to replace the fan module only a You do not need to remove the power supply from the system any further Caution The fan will continue spinning for a moment after the power supply is disconnected Be careful not to insert a finger or other object into the fan while it is still spinning b With one hand press together the two latches on top of the fan module and pull the module upward c Grasp the module with the other hand as soon as possible and remove the module from the power supply 4 1 1 2 Installing the Power Supply Modules and Front Fan Modules FIGURE 4 2 shows how to install the power supplies 4 4 Sun Blade 6048 Modular System Service Manual December 2009 FIGURE 4 2 Installing a Power Supply Module and Front Fan Module 1 If you are replacing a front fan module insert the fan module into the power supply 2 If necessary pull the power supply handle out
74. rsnt Device Present Blade 1 is present a00 Entity Presence bl2 prsnt Device Present Blade 2 is present b00 Entity Presence bl3 prsnt Device Present Blade 3 is present Appendix A Sun Blade 6048 ILOM 2 0 Information A 13 TABLE A3 CMM Boot SEL Readings Continued Sensor ID Device State Description c00 Entity Presence bl4 prsnt Device Present Blade 4 is present doo Entity Presence bl5 prsnt Device Present Blade 5 is present e00 Entity Presence bl6 prsnt Device Present Blade 6 is present f00 Entity Presence bl7 prsnt Device Present Blade 7 is present 1000 Entity Presence bl8 prsnt Device Present Blade 8 is present 1100 Entity Presence bl9 prsnt Device Present Blade 9 is present 1200 Entity Presence bl10 prsnt Device Present Blade 10 is present 1300 Entity Presence bl11 prsnt Device Present Blade 11 is present A 3 6 2 AC Cables Are Disconnected When three of the AC cables get disconnected the SEL displays the readings shown in TABLE A 4 Note The order of the events might not the real time event exactly because that is based on how the sensors are being scanned TABLE A 4 AC Cable Disconnect SEL Readings Sensor ID Device State Description 1400 Voltage ps0 pstate_0 State Deasserted PSU 0 side 0 DC output is out because AC is unplugged 1500 Voltage ps0 ac1_fail Predictive Failure PSUD side 1 AC is Asserted disconnected 1600 Power Supply ps0 status Presence detected Presence of PSU 1 is Dea
75. s PSU 0 DC Consumed Power Watts PSU 0 Power Efficiency Percent PSU 1 AC Consumed Power Watts PSU 1 DC Consumed Power Watts PSU 1 Power Efficiency Percent Appendix B Power Management B 7 B 8 This information and configuration is also available through the ILOM command line TABLE 0 1 gt show CMM powermgmt CMM powermgmt Targets 1 Chassis DC Consumed Power Chassis AC Consumed Power PSU 0 AC Consumed Power PSU 0 DC Consumed Power PSU 0 Power Efficiency PSU 1 AC Consumed Power 1 1 YAO amp amp N PSU DC Consumed Power 8 PSU Power Efficiency Properties reserve_psus total_psus 2 0 For more information refer to the documentation for your version of the ILOM The power management system automatically determines the amount of power available from installed PSUs You can set aside one PSU as a reserve by setting the Reserve Power Supplies field in the WebGUI to 1 You can also set the reserve_psus property from the command line Setting aside a reserve PSU reduces available power by 50 Caution Before setting aside a reserve PSU verify that currently powered up blades will have their power requirements met by the single remaining PSU Sun Blade 6048 Modular System Service Manual December 2009 Index B buttons 1 16 1 17 C chassis adding modules and options network express modules NEMs 4 11 PCI ExpressModules PCI EMs 4 1
76. sS amp o SUN microsystems Sun Blade 6048 Modular System Service Manual Sun Microsystems Inc www sun com Part No 820 2863 13 December 2009 Revision A Submit comments about this document at http www sun com hwdocs feedback Copyright 2009 Sun Microsystems Inc 4150 Network Circle Santa Clara California 95054 U S A All rights reserved Sun Microsystems Inc has intellectual property rights relating to technology fiat is described in this document In particular and without limitation these intellectual property rights may include one or more of the U S patents listed at http www sun com patents and one or more additional patents or pending patent applications in the U S and in other countries This document and the product to which it pertains are distributed under licenses restricting their use copying distribution and decompilation No part of the product or of this document may be reproduced in any form by any means without prior written authorization of Sun and its licensors if any Third party software including font technology is copyrighted and licensed from Sun suppliers Parts of the roue may be derived from Berkeley BSD systems licensed from the University of California UNIX is a registered trademark in the U S and in other countries exclusively licensed through X Open Company Ltd Sun Sun Microsystems the Sun logo Java Solaris and Sun Blade are trademarks or registered trademarks of Su
77. serial interface and two RJ 45 Ethernet connectors The CMM is also compatible with Sun N1 System Manager SM software and some third party management software The CMM provides the following a Automatic control of chassis fan speed A command line interface CLI and web interface that provides the ability to control and view the status of components present in the chassis The CLI and web interface enable the following a Display component presence a Display component FRU SEEPROM data a Display component status Display and configure CMM network parameters Display and configure server module service processor SP network parameters Connect to server module SP CLIs through SSH CLI only m Support for IPMItool IMPItool is an open source simple command line interface CLI utility for managing and configuring IPMI enabled devices IPMItool can manage the IPMI functions of either the local system or a remote system You can use the IPMItool utility to perform IPMI functions with a kernel device driver or over a LAN You can download IPMItool from this site http ipmitool sourceforge net CMM Integrated Lights Out Manager ILOM Features The CMM ILOM provides the following features m Is accessed through the CMM serial and network ports m Detects the presence of replaceable components in the chassis This is limited to components that are not managed by the server module SP Sun Blade 6048 Modular System Service
78. sserted detected 1700 Power Supply ps0 status Power Supply AC lost At least one AC input is Asserted lost to power supply 0 1800 Voltage ps0 acO_fail Predictive Failure PSU 0 side 0 AC is Asserted disconnected A 14 Sun Blade 6048 Modular System Service Manual December 2009 A 3 6 3 AC Cable Disconnect SEL Readings Continued TABLE A 4 Sensor ID Device State Description 1900 Voltage ps0 pstate_1 1a00 Power Supply ch pwrctl state 1b00 Voltage psl pstate_1 State Deasserted Predictive Failure Asserted State Deasserted PSU 0 side 1 DC output is out There is now not enough power for chassis PSU 1 side 1 DC output is out All the AC Cables Get Disconnected Then Plugged Back In When all of the AC cables get disconnected then plugged back in the SEL displays the readings shown in TABLE A 5 TABLE A 5 AC Cables Re inserted SEL Readings Sensor ID Device State Description 1c00 Entity Presencenem0 prsnt Device Present NEM 0 is present 1d00 Entity Presence ps0 prsnt Device Present PSU 0 is present 1e00 Entity Presence ps1 prsnt Device Present PSU 1 is present 1f00 Voltage ps0 pstate_0 State Asserted PSU 0 side 0 is connected 2000 Voltage ps0 pstate_1 State Asserted PSU 0 side 1 is connected 2100 Voltage psl pstate_0 State Asserted PSU 1 side 0 is connected 2200 Voltage ps1 pstate_1 State Asserted PSU 1 side 1 is connected 2300 Entity Presence bl0 prsnt Device Present Blade 0
79. surpass those of conventional servers These features are summarized in TABLE 1 5 Additional system specifications can be found in the Sun Blade 6048 Modular System Site Planning Guide TABLE 1 5 Sun Blade 6048 Modular System Features Feature Description Chassis dimensions e Chassis height 81 7 inches 2075 mm e Chassis depth 40 35 inches 1025 mm e Chassis width 23 78 inches 604 mm Front modules e Up to 12 server modules blades per shelf Two power supply modules per shelf One front indicator LED module FIM per shelf e Two front fan modules located in power supply per shelf Rear modules Up to 24 PCI ExpressModules 2 per server module per shelf e Up to two NEMs per shelf e One chassis management module CMM per shelf e Eight high performance fan modules to cool the server modules per shelf System Management Chassis management module CMM provides Connection to the system wide management network e 10 100 1000BASE T external connections Serial port access to CMM e Sun Integrated Lights Out Manager ILOM firmware 1 6 Reliability Availability and Serviceability Features The Sun Blade 6048 modular system includes many server module centric and chassis wide features that increase reliability availability and serviceability RAS These RAS features are aspects of a system s design that affect its ability to operate continuously and to minimize the time necessary to service the system Reliability
80. t Reports the presence of the power supply TABLE2 psn prsnt Readings Reading State Event Description Action 0x0001 Device Absent Yes Power Supply is absent none 0x0002 Device Present Yes Power Supply is present none psn vinokn Reports the status of the AC input lines for each power supply TABLE3 psn vinokn Readings Reading State Event Description Action 0x0001 State Deasserted Yes Power Supply is disconnected System Alert LED from AC power is SLOW 0x0002 State Asserted Yes Power Supply is connected to System Alert LED AC power is OFF psn pwrokn Reports status of DC output lines for each power supply TABLE4 psn pwrokn Readings Reading State Event Description 0x0001 State Deasserted Yes DC output is off none 0x0002 State Asserted Yes DC output is on none A 3 3 Fan Sensors The chassis contains eight fan modules Each module contains two fans Modules and fans are numbered from 0 A 10 Sun Blade 6048 Modular System Service Manual December 2009 fmn fn speed These sensors report individual fan speeds All fan speed sensors are configured to generate the same events and all faults are handled in the same way They are not monitored when the chassis is powered off TABLE5 Fan Speed Events Threshold Direction Event Description Lower Non Assert Yes Fan speed has decreased below lower non recoverable Recoverable threshold This indicates that the fan has failed or has been remove
81. t management to continue to assert that the PSU is still in a faulted state If this happens the ILOM management interfaces the chassis shelf Service Action Required LEDs and the event log reflect that the faulted state is ongoing Refer to Section 2 3 2 Power Supply Module Faults on page 2 13 for more information Monitoring Faults From the Management Interfaces You can monitor chassis shelf and component faults from the ILOM CLI or the web interface Note Refer to the Sun Integrated Lights Out Manager 2 0 User s Guide for information about the object namespace and how to identify the targets and properties that might pertain to faults m Section 2 2 2 1 Obtaining Sensor Readings Using the CLI on page 2 4 m Section 2 2 2 2 Obtaining Sensor Readings Using the Web Interface on page 2 5 Obtaining Sensor Readings Using the CLI To obtain sensor readings using the CLI 1 Establish a local serial console connection or SSH connection to the CMM and log in to the ILOM 2 Issue the appropriate show command to display information about system components Sun Blade 6048 Modular System Service Manual December 2009 2 25202 For example if a power supply AC 1 light is lit you would issue the following command gt show CH PS0 S1 AC FAIL CH PSO S1 AC_FATL Targets Properties type Voltage class Discrete Sensor value Predictive Failure Asserted Commands cd show The
82. t swappable or hot pluggable component You will need to power off the chassis before replacing this component Removing the Midplane To remove the midplane 1 Power off the chassis See Section 3 3 Powering Off the System Before Service on page 3 6 FIGURE 4 19 shows how to remove the midplane Chapter 4 Replacing System Components 4 23 FIGURE 4 19 Removing the Midplane 2 Remove all the modules or filler panels that are installed in the chassis See the procedures in Section 4 1 Customer Replaceable Units on page 4 1 3 Remove the air baffle by pressing the snaps from underneath the power supply cage and tilting the front of the air duct downward to free it from the chassis 4 Loosen the 16 captive screws attaching the midplane to the back of the chassis 4 24 Sun Blade 6048 Modular System Service Manual December 2009 Caution Do not touch the connectors on the midplane Doing so could result in damaged or bent connector pins 5 Pull the midplane away from the chassis guidepins then down to clear the power supply cage 6 Tilt the midplane slightly to the side to and remove it from the chassis 7 If present pull off the gasketing material located on the left and right side of the chassis in the midplane area See FIGURE 4 20 FIGURE 4 20 Removing the Midplane Gasketing Material Chapter 4 Replacing System Components 4 25 4 2 1 2 Instal
83. ted at 16A or 20A depending on geographic location and must have a plug that meets the requirements of the data center installation The connection to the AC inlet on the system is location specific a AMER L6 20P to IEC 320 C19 2 4m Sun part number 180 2005 02 a EMEA IEC 309 to IEC 320 C19 2 4m Sun part number 180 2004 02 Powering On the System Automatically 1 Verify that the power cords are properly connected to the AC inlets on the chassis and to the cabinet power distribution unit PDU or other power source When the chassis is first connected to AC power the following sequence of events occurs The power supplies produce 3V auxiliary standby power The CMM receives 3V auxiliary standby power and comes out of reset The CMM starts to boot and sets the OK indicator on the CMM to a slow blink state When the CMM has booted the CMM sets the OK indicator on the front indicator module FIM and CMM to the standby blink state indicating that the chassis is now ready for full power on The chassis then automatically powers on by default Note The chassis powers on only when at least one power supply module is energized and at least one AC input is receiving power 2 Verify that the green OK indicator on the FIM or CMM is in the Steady On state When the chassis has been fully powered on the OK indicator remains illuminated The steady on OK LED indicates that the power supplies are energized and the 12V pow
84. ter 4 Replacing System Components 4 9 4 1 3 2 4 10 4 Remove the FIM from the chassis Installing the Front Indicator Module To install the front indicator module 1 Align the FIM with the FIM slot Make sure that the release button is on the right side of the module FIGURE 4 7 shows how to install the FIM FIGURE 4 7 Installing the FIM 2 Slide the FIM into the FIM slot 3 Reinstall power supply 0 into the chassis See Section 4 1 1 Replacing Power Supplies and Front Fan Modules on page 4 2 Sun Blade 6048 Modular System Service Manual December 2009 4 1 4 Replacing NEMs If a network express module NEM fails you will need to replace the NEM You can remove and replace a NEM from a powered on system using a hot swap operation as well as from a powered off system Caution If you are not immediately replacing the NEM install an NEM filler panel to ensure proper system operation 4 1 4 1 Removing the NEMs FIGURE 4 8 shows how to remove the NEMs Panel 3a shows removal of a dual slot NEM and Panel 3b shows removal of a single slot NEM FIGURE 4 8 Removing a NEM 1 In the rear of the chassis locate the NEM that you want to remove Chapter 4 Replacing System Components 4 11 2 Remove all cables from the NEM 3 Press together and hold the ejector buttons on both the right and left ejector levers 4 To unlatch the NEM from the chassis open the ejector levers by extendin
85. th onscreen computer output Password AaBbCc123 Book titles new words or terms Read Chapter 6 in the User s Guide words to be emphasized These are called class options Replace command line variables You must be superuser to do this with real names or values To delete a file type rm filename The settings on your browser might differ from these settings Third Party Web Sites Sun is not responsible for the availability of third party web sites mentioned in this document Sun does not endorse and is not responsible or liable for any content advertising products or other materials that are available on or through such sites or resources Sun will not be responsible or liable for any actual or alleged damage or loss caused by or in connection with the use of or reliance on any such content goods or services that are available on or through such sites or resources Sun Welcomes Your Comments Sun is interested in improving its documentation and welcomes your comments and suggestions You can submit your comments by going to http www sun com hwdocs feedback Please include the title and part number of your document with your feedback Sun Blade 6048 Modular System Service Manual part number 820 2863 13 Preface xi xii Sun Blade 6048 Modular System Service Manual December 2009 CHAPTER 1 Introduction to the Sun Blade 6048 Modular System This chapter contains an overview of the Sun Blade 604
86. the blade documentation for servicing the blade Viewing Chassis Shelf Faults in ILOM The chassis shelf Temperature Fail LED light turns on when at least one of the ambient temperature sensors in the power supply modules reaches 40 C and shuts down the chassis shelf when the temperature reaches 45 C See TABLE 2 3 for information about viewing this sensor information See the Sun Integrated Lights Out Manager 2 0 User s Guide for more information about reading this and other chassis shelf sensors Power Supply Module Faults There are three power supplies located within each power supply module The AC 0 LED corresponds to power supply 0 within the power supply module AC 1 corresponds to power supply 1 and AC 2 corresponds to power supply 2 If you do not need the full 8400W of power from the power supplies you can connect only two of the total three plugs to the ACO and AC1 connectors for each power supply Do not connect AC2 When only two of the available three plugs is connected to the power supplies 5600 W of power will be supplied to the chassis The LEDs and ILOM will show different readings than for the three power cord connections See the notes in the following sections for the differences in configurations Chapter 2 Fault Management 2 13 2 3 2 1 Power Supply LED Locations FIGURE 2 5 9000W Power Supply LED Location poes ea Vus vus Vans Dons bus vas Vas bus vais tas M PSU Service LED DC LED AC 2 LED
87. the ejector lever to secure the CMM in the chassis After you physically install a CMM in a powered on system the CMM automatically begins its initialization If you have installed the CMM in a powered off system the CMM goes through the following sequence of operations a CMM receives 3 volt auxiliary standby power and comes out of reset a While the CMM boots up the green OK indicator on the CMM is set to the Slow Blink state a When the CMM has booted the CMM sets the green OK indicator on the front indicator FIM and chassis management module CMM to the Standby Blink state indicating that the chassis is now ready for full power on 4 22 Sun Blade 6048 Modular System Service Manual December 2009 4 1 9 Installing the Cooling Doors There are now two cooling doors available for the Sun Blade 6048 modular system The cooling doors can only be installed on a Sun Blade 6048 upgraded chassis To install a rear door you will need the upgraded Sun Blade 6048 module system See Section 1 7 Upgraded Chassis on page 1 19 for information on how to identify the new chassis Refer to the Sun Cooling Door 5200 Installation and User s Guide or the Sun Cooling Door 5600 Installation Guide for detailed information on installing the cooling doors 4 2 42 1 4 2 1 1 Field Replaceable Units The following procedures should be performed only by Sun Service personnel Replacing the Midplane The midplane is not a ho
88. the rear the rear fan boards connect the fans to the N A midplane There is one fan board for every two rear fan modules 3 1 5 FRU Features FRUs are components that must be removed and replaced only by authorized Sun service personnel These components are neither hot swappable or hot pluggable The system must be powered off before you replace the components TABLE 3 2 FRU Features Hot swap or FRU Description Hot plug Midplane Installed from the front You must remove all system modules fan N A connector boards and the midplane air baffle before you remove or install the midplane 3 4 Sun Blade 6048 Modular System Service Manual December 2009 3 2 Locating Front and Rear Components FIGURE 3 1 shows the front components and FIGURE 3 2 shows the rear components on the Sun Blade 6048 chassis Internal components that are not visible in the illustrations are in the following locations m Front fan modules x2 Within the power supplies m Rear fan boards x3 Behind the fan modules one per every two fan modules a Midplane In the center of the chassis m Internal power cables Run from the power supplies in the front of the chassis to the power connections in the rear of the chassis FIGURE 3 1 Front View of Sun Blade Modular System Chassis Power supplies Front indicator module FIM Server or storage modules blades Chapter 3 Preparing for Service 3 5 FIGURE 3 2 Rear View of Sun Blade Modular
89. thods Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 19 a View Component Information Using the CLI on page 20 a View Component Information Using the Web Interface on page 20 Vv View Component Information Using the CLI 1 Log in to the ILOM CLI as an Administrator or Operator 2 At the command prompt type gt show CH The chassis manufacturing part number will be displayed in the product_part_number field a If the part number is 594 5971 you have the upgraded chassis If the part number is 594 5110 you have the previous version of the chassis Vv View Component Information Using the Web Interface 1 Log in to the ILOM web interface as an Administrator or Operator 2 Select System Information gt Components The Component Management page appears 3 Click on CH in the Component Management Status table A dialog box appears with information about the chassis The chassis manufacturing part number will be displayed in the product_part_number field a If the part number is 594 5971 you have the upgraded chassis a If the part number is 594 5110 you have the previous version of the chassis 1 20 Sun Blade 6048 Modular System Service Manual December 2009 CHAPTER 2 Fault Management This chapter contains information about the following topics related to fault management on the Sun Blade 6048 modular system m Section 2 1 About Fault Management on page 2 1
90. tion change Description includes user command command parameters and success or fail IPMI Log Any event that is placed in the IPMI SEL is also put in the management log Chassis State For changes to the inventory and general system state Chassis Action Category for shutdown events for server module or chassis hot insert or removal of a FRU and Reset Parameters button pushed e FMA Fault For Fault Management Architecture FMA faults Description gives time of fault as detected by FMA and suspect component e FMA Repair For FMA repairs Description gives component Severity Critical Major or Minor Description A description of the event a Dismiss the event log stop displaying the log Press the q key m Clear entries in the event log Perform these steps a Type set clear true A confirmation message appears b Type one of the following a To clear the entries type y a To cancel clearing the log type n Note The ILOM event log accumulates many types of events including copies of IPMI entries Clearing the ILOM event log clears all entries in the log including the IPMI entries However clearing the ILOM event log entries does not clear the actual entries posted directly to an IPMI log 2 8 Sun Blade 6048 Modular System Service Manual December 2009 2 2 3 2 Viewing or Clearing the ILOM Event Log Using the Web Interface To view or clear events in the ILOM event log using the ILOM
91. tion is given in TABLE B 1 TABLE B 1 Power Requirements for Maximally Configured Sun Blade X6250 Requirement Per Number of Component Component Components Subtotal Blade Base 50W 1 50W Requirement Processors 120W 2 240W PCI EMs 25W 2 50W DIMMs 11W 16 176W Total 603W Sun Blade 6048 Modular System Service Manual December 2009 TABLE B 1 Power Requirements for Maximally Configured Sun Blade X6250 Requirement Per Number of Component Component Components Subtotal Disks 12W 4 48W REM 14W 1 14W FEM 25W 1 25W Total 603W Power requirements for blade products and their components are given in the following tables Note that PCI EMs always require 25W this value is repeated in each table for ease of reference TABLE B 2 Power Requirements for Sun Blade X6220 and Its Components Possible Requirement Per Number of Component Component Components Blade Base 25W 1 Requirement Processor 68W 95W 120W 1 2 PCI EMs 25W 0 2 DIMMs 6W 2 16 Disks 11W 0 4 REM N A 0 FEM N A 0 TABLE B 3 Power Requirements for Sun Blade X6240 and its Components Possible Requirement Per Number of Component Component Components Blade Base 25W 1 Requirement Processor 68W 95W 1 4 PCI EMs 25W 0 2 DIMMs 6W 2 32 Appendix B Power Management B 3 TABLE B 3 Power Requirements for Sun Blade X6240 and its Components Possible Requirement Per Number of Component Component Components Disks N A 0 RE
92. tools required for service 3 8 typographic conventions xi Index 3 Index 4 Sun Blade 6048 Modular System Service Manual December 2009
93. twork express modules NEMs The PCI ExpressModules PCI EMs are based on the PCI Express industry standard form factor for PCI EMs The PCI EMs provide dedicated I O functions on a per server module basis There are two PCI EMs for each server module for up to 24 PCI EMs per chassis shelf You can customize the server module and PCI EM configurations in the chassis For example one server module can be configured with redundant Fibre Channel PCI EMs while another server module can have a single Fibre Channel PCI EM and a single InfiniBand PCI EM The PCI EMs provide a maximum of 4 Gbytes per second of I O per module Up to two NEMs per system shelf offer pass through Gigabit Ethernet connections to the server modules Designed within a Sun proprietary form factor these switchless networking devices provide a dual Gigabit NIC for each server module There are up to two RJ 45 Ethernet interfaces per server module providing 10 100 1000BASE T ports at the chassis rear Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 5 1 4 1 4 1 1 6 CMM System Management The Sun Blade 6048 modular system chassis management module CMM manages a portion of the Sun Blade 6048 chassis This section provides an overview of the CMM functionality For specific information about system management using the CMM Integrated Lights Out Manager ILOM see the Sun Integrated Lights Out Manager 2 0 User s Guide The CMM provides an RJ 45
94. ule n 0 11 CH BLn 0 11 Power Supply Unit n 0 1 CH PSn 0 1 Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 9 FIGURE 1 4 and TABLE 1 3 show the nomenclature used in the CMM ILOM to access information for rear chassis components FIGURE 1 4 Rear Panel CMM FMO FM4 FM1 FM5 FM2 FM6 FM3 FM7 TABLE 1 3 Rear Panel Nomenclature FRU Description Location n Fan module n 0 7 CH FMn 0 7 Chassis management module CMM n a Gigabit Ethernet NEM n 0 1 CH NEMn 0 1 NEM n 0 CH NEMn 0 1 If you have a NEM installed that occupies both NEM slots the ILOM identification for that NEM is NEMO 1 10 Sun Blade 6048 Modular System Service Manual December 2009 FIGURE 1 5 and TABLE 1 4 show the nomenclature used in the CMM ILOM to access information for rear chassis components FIGURE 1 5 Chassis LEDs LOCATE FAULT fe EER eee HT De hv Ce Che Ce De Che Ce De Oe i TEKEN N AA i i T LOW 13N 0 19W LAN FAULT NORMAL TEMPFAIL VR SISSVH LOCATE TABLE 1 4 LED Nomenclature LED Description Location Locate button LED CH LOCATE Fault LED Service action required CH SERVICE Power OK LED CH NORMAL Temperature Fail LED CH TEMPFAIL Chapter 1 Introduction to the Sun Blade 6048 Modular System 1 11 1 5 Feature Summary The Sun Blade 6048 modular system is designed with hardware and software features that
95. use the following interfaces to view the power supply information m Section A 2 2 1 IPMItool on page A 7 m Section A 2 2 2 CLI on page A 8 m Section A 2 2 3 Web Interface on page A 8 This information applies for server modules running ILOM 2 0 Note The command samples in this section use the variable X to represent power supply 0 or power supply 1 IPMItool You can use the following ipmitool commands to view power supply information m ipmitool H SPIPaddress U root P changeme sdr This command should display the following output psX vinok2 0x01 ok psX pwrok2 0x01 ok Appendix A Sun Blade 6048 ILOM 2 0 Information A 7 A 2 2 2 A 2 2 3 m ipmitool H SPIPaddress U root P changeme sel elist This command should display output similar to the following a503 12 27 2005 14 55 45 Power Supply psX vinok2 State Deasserted a603 12 27 2005 14 55 45 Power Supply psX pwrok2 State Deasserted CLI show SP logs command should produce output similar to the following 8665 Thu Mar 19 03 15 44 1970 IPMI Log critical ID 1d pre init timestamp Power Supply psX vinok2 State Deasserted 8667 Thu Mar 19 03 15 44 1970 IPMI Log critical ID 1f pre init timestamp Voltage psX pwrok2 State Deasserted Web Interface 1 Log into the ILOM web interface using the SP IP of the server module 2 Click on System Monitoring tab t
96. vice is complete see Section 4 4 Powering On the System After Service on page 4 28 Powering Off the System Using the Web Interface To powering off the system using the web interface 1 Log in to the ILOM web interface on the chassis management module CMM 2 Select the CMM view from the left panel 3 From the Remote Control tab click the Remote Power Control tab 4 Select the radio button next to CH chassis 5 Use the drop down list to select a new power state for the chassis The chassis current power state appears in the Current State column a Select one of the following values Immediate Power Off Select to power off the server without a graceful shutdown of the host operating systems Using this option could result in data loss as the host operating systems are not powered off gracefully Graceful Shutdown and Power Off Select to gracefully shut down the operating system before the host is powered off b In the confirmation dialog box click OK to confirm the change Chapter 3 Preparing for Service 3 7 3 4 3 4 0 1 3 8 Required Installation Tools and ESD You will need the following tools and supplies to install the initial hardware into the Sun Blade 6048 modular system m No 2 Phillips screwdriver for removing the midplane or fan connector boards a M5 hex nut driver for removing internal power cables m Antistatic wrist strap a Antistatic mat Attaching the Antistatic Wrist Strap Use th
97. want to continue y n y Service processor FRU information ready to be collected You MUST reboot the service processor for to complete this process Allow the service processor to fully boot DO NOT UNPLUG THE SYSTEM WHILE THE SERVICE PROCESSOR IS BOOTING Would you like to reboot the service processor now y n y The system is going down NOW Sending SIGTERM to all processes 4 4 Powering On the System After Service Follow the procedures in this section after you have serviced the following components a Midplane m Internal power cables The chassis is set by default to power on automatically when AC power is connected to the chassis However if you have changed this default setting so that the chassis does not power on automatically you can power on the system by issuing software commands through the CMM command line interface CLI This section includes the following topics 4 28 Sun Blade 6048 Modular System Service Manual December 2009 4 4 1 4 4 2 m Section 4 4 1 Power Cord Connections on page 4 29 m Section 4 4 2 Powering On the System Automatically on page 4 29 m Section 4 4 3 Powering On the System Using the CLI on page 4 30 m Section 4 4 4 Powering On the System Using the Web Interface on page 4 30 Power Cord Connections Ensure that the power cords are connected to the AC power inlets on the rear of the chassis These power cords must be ra
98. x02 ok psX 112V_2 0 00 Amps ok psX ac0_fail 0x01 ok psX acl_fail 0x01 ok psX ac2_fail 0x02 ok m ipmitool H SPIPaddress U root P changeme sel elist This command should display output similar to the following 1100 Pre Init Time stamp Power Supply psX status Presence detected Deasserted 1200 Pre Init Time stamp Power Supply psX status Power Supply AC lost Asserted 1300 Pre Init Time stamp Voltage psX ac2_fail Predictive Failure Asserted A 2 1 2 CLI show CMM logs command should produce output similar to the following 8665 Thu Mar 19 03 15 44 1970 IPMI Log critical ID 1d pre init timestamp Power Supply psX status Power Supply AC lost 8667 Thu Mar 19 03 15 44 1970 IPMI Log critical ID 1f pre init timestamp Voltage psX ac2_fail Predictive Failure Asserted A 2 1 3 Web Inferface 1 Log into the ILOM web interface using the SP IP address for the CMM 2 Click on CMM A 6 Sun Blade 6048 Modular System Service Manual December 2009 A 2 2 A 2 2 1 3 Click on System Monitoring tab then Sensor Readings tab The following readings will be displayed CH PSX STATUS Power Supply input lost AC DC CH PSX S2 PSTATE State Deasserted CH PSX S2 AC_FAIL Predictive Failure Asserted Server Module Behavior The following topics describe the CMM behavior when two power cords are connected to the power supply module You can
99. xperiences a hardware failure it is called an internal fault that is the fault is the result of a problem with the hardware of the Sun Blade 6048 modular system itself Internal faults are cleared when a repair action takes place most likely the replacement of the failed component There are some faults however that are external faults In these cases the system hardware has not failed but a condition external to the system is causing a potential problem If for example the ambient air temperature external to the chassis exceeds a certain threshold it is a fault because it can adversely affect the operation of the system if not corrected External faults are autoclearing they are cleared when the external condition no longer exists Nonetheless an external fault can if not attended to cause components or the system as a whole to shut down 2 12 Components Monitored by Fault Management Fault management monitors the following system component TABLE 2 1 Component Fault Management System Component Refer to This Section System chassis shelf Section 2 3 1 Chassis Shelf Faults on page 2 11 Power supply units Section 2 3 2 Power Supply Module Faults on page 2 13 PSUs Front fans within power Section 2 3 2 Power Supply Module Faults on page 2 13 supply modules Rear Fans Section 2 3 3 Rear Fan Faults on page 2 18 Note For information about faults for other system components such as NEMs

Sun Blade 6048 Modular System Service Manual

Contents

Download Pdf Manuals

Related Search

Related Contents