Home

Netra CP3260 Blade Server Userճ Guide

1. FIGURE A 7 Zone 3 Signal Connectors J31 J32 J33 L TINI LN ZEE 7 oma a van a O a a a a a O o HG Fa fowl LU mL I n iS TW nd Ef ERE xw wmm m mm mm m m mm uw mm m DG D e n e fe e n e n fn fn fe f n n fa fe e o n n PCB C m m we PEIPER jm gm mmm m BG BI eee n m m fu fm in n f e m u n n n n n AN m l xc wi cU jw Oe ie m M A EEN 109 876543231319 87 6 5 43 2 098765 43 2 A 10 Netra CP3260 Blade Server User s Guide April 2009 TABLE A 7 gives the Zone 3 J31 connector signals and pin assignments TABLE A 7 Zone 3 J31 Connector Pin Assignments Row A B Cc D E F 1 Z3 Fl TXI Z3 F1 TXI Z3 Fl RXO Z3 F1 RX0O Z3 F1 TXO Z3 F1 TXO 2 Z3_F1_RX2 Z3_F1_RX2 Z3FITDO Z3_F1_TX2 Z3FIRXI Z3 F1 RXI 3 Z3 F2 TXO Z3 F2 TX0o Z3 F1 RX34 Z3F1RX3 Z3 F1 TX34 Z3 F1 TX3 4 Z3 F2 RXI Z3 F2RXI Z3F2TXI Z83F2TXI Z3 F2RXO Z3 F2 RX0O 5 Z3_F2_TX3 Z3F2TX3 Z3_F2_RX2 Z3_F2_RX2 Z3rF2DO Z3_F2_X2 6 N U N U N U N U Z3 F2 RX3 Z3 F2 RX3 7 N U N U N U N U N U N U 8 N U N U N U N U N U N U 9 N U N U N U N U N U N U 10 N U N U N U N U N U N U N U not used TABLE A 8 gives the Zone 3 J32 connector signals and pin assignments TABLE A 8 Zone 3 J32 Connector Pin Assignments Row A B C D E F 1 XAUI MDIO XAUI MDC N U N U N U N U 2 N U N U N U N U SER_RJ45_RTS SER_RJ45_DTR 3 N U N U N U N U SER_RJ45_TXD SER_RJ45_RXD 4 N U N U N U N U SER_RJ45_DSR SER_RJ45_CTS
2. A 1 Form Factor The Netra CP3260 blade server is a standard 8U form factor a single slot wide It complies with the blade server mechanical dimensions required by the PICMG 3 0 R1 0 Specification m 322 25 mm x 280 mm length x width m 1 2 inch wide front panel A 2 Layout The Netra CP3260 blade server layout is shown in FIGURE A 1 FIGURE A 1 Netra CP3260 Blade Server Layout A 2 i PT Figure Legend Ethernet port connector RJ 45 Dual USB port connector USB 2 0 Serial port connector A OO N a Compact flash connector on o o Zone 1 power connector Zone 2 I O connectors Zone 3 RTM I O connectors Zone 3 RTM power connector Netra CP3260 Blade Server User s Guide April 2009 A 3 A 3 1 A 3 2 Front Panel The single slot wide 8U front panel was designed to meet PICMG 3 0 R 1 0 and other specifications Visual Indicators The Netra CP3260 blade server has the following indicators on the front panel m Green LED Board s healthy status or user programmable ACTIVE m Amber LED Board s fault condition FAULT m Blue LED Indicates safe removal hot swap activity The front panel s Ethernet ports do not have LED indicators Ports The Netra CP3260 blade server has the following ports on the front panel m One 10 100 1000BASE T Ethernet port RJ 45 m Dual USB ports USB 2 0 m One serial port RJ 45 A 4 A 4 1 Connectors and Pinout FIGUR
3. FIGURE 1 4 Netra CP3260 Blade Server Midplane and Netra CP32x0 ARTM 1 12 Netra CP3260 Blade Server User s Guide April 2009 Figure Legend 1 Sun Netra CP3260 blade server 2 Sun Netra CP32X0 ARTM 3 ATCA chassis midplane Note When a Netra CP32x0 ARTM is used with the Netra CP3260 blade server shielded cables are required for serial I O ports Unshielded cables can be used on Ethernet ports to satisfy EMI compliance standards The shields for all shielded cables must be terminated on both ends The customer can order a Netra CP32x0 ARTM build a custom card or buy from an independent hardware vendor IHV A minimal set of I O must provide a boot path for the host blade server and a path for console I O to deliver commands and to read blade server and system status Possible boot and console configurations are described in TABLE 1 2 Sun Microsystems supplies the Netra CP3260 blade server and compatible Netra CP32x0 ARTMs The other configurations require IHV hardware TABLE 1 2 I O Configurations 1 0 Hardware Required Description Ethernet Netra CP3260 blade server Default boot path uses the Ethernet port when the blade server Netra CP32x0 ARTM runs in diskless client configuration ARTM supplied as an option for rear access SAS Netra CP32x0 SAS Storage SAS devices can be used for local booting ARTM Dual HD Serial data Netra CP3260 blade server Serial port on front panel provides a path for a console I O
4. ICACHE other 6f00000fce IMMU other DMMU other DCACHE other 6f 00000fd4 6 00000fda 6f00000fe0 CORE7 other 6 00000fe6 PO Pl P2 P3 PA P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17 P18 P19 P20 P21 P22 P23 SPU other FPU other 6 00000fec 6 00000 2 ICACHE other 6f00000ff8 DCACHE other 6f00000ffe IMMU other 6 00001004 DMMU other 6 0000100a other 6 00001010 other 6 00001016 other 6 0000101c other 6 00001022 other 6 00001028 other 6 0000102e other 6 00001034 other 6 0000103a other 6 00001040 other 6 00001046 other 6 0000104c other 6 00001052 other 6 00001058 other 6 0000105e other 6 00001064 other 6 0000106a other 6 00001070 other 6 00001076 other 6 0000107c other 6 00001082 other 6 00001088 other 6 0000108e other 6 00001094 other 6 0000109a Chapter 4 Firmware and Blade Server Management 4 25 CODE EXAMPLE 4 4 prtpicl Command Output P24 other 6 000010a0 P25 other 6 000010a6 P26 other 6 000010ac P27 other 6 000010b2 P28 other 6 000010b8 P29 other 6 000010be P30 other 6 000010c4 P31 other 6 000010ca P32 other 6 000010d0 P33 other 6 000010d6 P34 other 6 000010dc P35 other 6 000010e2 P36 other 6 000010e8 P37 other 6 000010ee P38 other 6 000010f4
5. Local IP addresses are not needed if they are assigned by a network DHCP server You might need the MAC Ethernet addresses of the local hosts to make nameserver database entries The MAC address can be seen in the console output while booting to the ok prompt It can also be derived from the host ID seen on the barcode label see Section 1 7 1 Part Number Serial Number and Revision Number Identification on page 1 16 3 4 Firmware Updates For information on firmware updates refer to the Netra CP3260 Blade Server Product Notes 820 0455 which is available online at http docs sun com app docs prod netra brds All Sun Netra CP3260 firmware updates can be downloaded from the Sun Download Center SDLC at http www sun com download 3 5 Configuring for 1 GbE or 10 GbE Switches The Fabric interface on the Sun Netra CP3260 blade server is capable of operating at either 1 Gbps or 10 Gbps The Sun Netra CP3260 can be used in the Netra CT 900 server with either the Sun Netra CP3240 10 GbE switch board or the Sun Netra CP3140 1 GbE switch board Therefore it is important that the Solaris driver configuration file be configured accordingly on the Netra CP3260 blade server 3 8 Netra CP3260 Blade Server User s Guide April 2009 The NIU Network Interface Unit driver configuration file nxge conf might need to be modified to employ the proper port configurations The nxge conf file is located in the platform sun4v k
6. P39 other 6 000010fa P40 other 6 00001100 P41 other 6 00001106 P42 other 6 0000110c P43 other 6 00001112 P44 other 6 00001118 P45 other 6 0000111e P46 other 6 00001124 P47 other 6 0000112a P48 other 6 00001130 P49 other 6 00001136 P50 other 6 0000113c P51 other 6 00001142 P52 other 6 00001148 P53 other 6 0000114e P54 other 6 00001154 P55 other 6 0000115a P56 other 6 00001160 P57 other 6 00001166 P58 other 6 0000116c P59 other 6 00001172 P60 other 6 00001178 P61 other 6 0000117e P62 other 6 00001184 P63 other 6 0000118a V VMEML voltage sensor 6 00001190 V VCORE voltage sensor 6f00001197 TTYA other 6 0000119e For more information on the PICL framework refer to the picld 1M man page 4 26 Netra CP3260 Blade Server User s Guide April 2009 4 8 Multiplexing to Zones 2 and 3 A multiplexer MUX controller and ShMM configuration is available for use on Netra CP3260 blade servers to multiplex 10GbE network interface unit NIU ports to Zone 2 backplane and or to Zone 3 ARTM m MUX control configurations take effect during the blade activation following the configuration change m Both NIU ports are independently configurable m MUX configuration is persistent across reboots resets and hotswaps m MUX configuration remains persistent for the slot so that if a blade is replaced with another Netra
7. Sun part number and dash number for example 501 7658 01 01 is the dash number m Rev Revision number of the part for example Rev 06 The Media Access Control MAC address label contains the MAC address for the blade server in printed and barcode form 1 16 Netra CP3260 Blade Server User s Guide April 2009 FIGURE 1 5 Netra CP3260 Blade Server Barcode Labeling ANIKI i f HT imi SS n nnn HTHTTTTTTTTTEETETTTTTTTTTITTT rrr n 2 a oe oie ok Ue SunSN 1005LCB 0626Wxxx SunSN 1005LCB 0626Wxxx ankaii SunPN 501 7508 01 Rev 03 SunPN 501 7654 01 Rev 03 SunPN 501 7313 07_Rev 06 Note You might find the labels shown in FIGURE 1 5 on other locations on your blade server Your particular blade server configuration might also appear different than the illustration Chapter 1 Introduction 1 17 1 18 Netra CP3260 Blade Server User s Guide April 2009 ca CGHAPTE Hardware Installation This chapter describes the hardware installation procedures for the Netra CP3260 blade server and contains the following sections m Section 2 1 Equipment and Operator Safety on page 2 1 m Section 22 Materials and Tools Required on page 2 3 m Section 2 3 Preparing for the Installation on page 2 3 m Section 24 Installation Procedure Summary on page 2 4 m Section 2 5 Configuring On Board Hardware on page 2 5 m Section 2 6 Installing an Advanced Rear Transition Module Optional on page 2 1
8. 48 probe scsi 4 7 probe scsi all 4 7 show devs 4 8 watch net 4 11 OpenBoot variables auto boot 4 6 displaying 4 14 setting 4 14 operating system 3 1 output power 5 18 P part number 1 16 patches 3 1 PCI Express switch 5 10 PICL 4 19 PICL platform information and control library 4 19 pinouts connectors A 3 to A 13 POST error messages 4 3 4 4 tests 4 3 POST power on self test 4 3 power module 5 18 Power PC 5 3 power on self test POST 4 2 preparation installation 2 3 probe ide OpenBoot command 4 8 probe scsi OpenBoot command 4 7 probe scsi all OpenBoot command 4 7 processor block diagram 5 5 cores 5 2 5 4 speed 5 4 UltraSPARC T2 5 2 5 4 R rear transition module 1 11 removable media device 3 10 removing FB DIMMs 2 10 requirements hardware 1 14 operating system 1 3 safety 2 1 software 1 15 reset manual system 4 5 revision number 1 16 RJ 45 Ethernet A 4 rmformat utility 3 10 RTM 2 18 compatibility 2 18 support 5 4 S SCSI devices diagnosing problems 4 7 SERDES 5 1 5 4 5 10 serial number 1 16 serial port 5 11 A 5 service processor 5 12 shielded cables 1 13 2 24 show devs OpenBoot command 4 8 shutdown Solaris command 4 5 software patches 3 1 PICL 4 19 Solaris OS 3 1 Index 3 Solaris commands World Wide Name probe scsi 4 7 init 4 5 shutdown 4 5 X SPARC 5 2 XAUI 5 1 5 8 5 9 subsystems 5 1 XAUI ports 5 8
9. 5 N U N U N U N U N C N C 6 RTM_LANO_A RTM LANO A RTM TXFRMR RTM TXFRMR RTM LANO B RTM LANO B VOLTAGE VOLTAGE 7 RTM_LANO_C RTM LANO C RTM ACT RTM LINK RTM_LANO_D RTM_LANO_D LED_N LED_N 8 RTM_SLAN_TX RTM_SLAN_ RTM_SLAN_ RTM_SLAN_ N C N C 1 TX1 RX1 RX1 9 RTM_SLAN_TX RTM_SLAN_ RTM_SLAN_ RTM_SLAN_ N U N U 2 TX2 RX2 RX2 10 N U N U N U N U N U N U N C z not connected N U not used Appendix A Physical Characteristics A 11 TABLE A 9 gives the Zone 3 J33 connector signals and pin assignments TABLE A 9 Zone 3 33 Connector Pin Assignments Row A B c D E F 1 RTM_TX0 RTM_TX0 RTM_RX0 RTM RX0 RTM CON RTM CON REF CLK REF CLK 2 RTM_TX1 RTM_TX1 RTM_RX1 RTM RX1 TCLKA TCLKA 3 RTM_TX2 RTM TX2 RTM_RX2 RTM_RX2 TCLKB TCLKB 4 RTM TX34 RTM TX3 RTM RX34 RTM RX3 TCLKC TCLKC 5 RIM_TX4 RTM_TX4 RTM_RX4 RTM_RX4 TCLKD TCLKD 6 RTM_TX5 RTM TX5 RTM_RX5 RTM RX5 N U N U 7 RTM_TX6 RTM TX6 RTM_RX6 RTM RX6 N U N U 8 RTM_TX7 RTM TX7 RTM_RX7 RTM RX7 N U FPGA PRESET RTM 9 N C N C N C N C RTM MMCL PCI CFG 10 N U N U N U N U GND RTM CON EN L N C not connected N U not used FIGURE A 8 Zone 3 Power Connector VA f 1 E fe p Mm m M E b B E bel j ise 4 305 1 TABLE A 10 gives the Zone 3 power connector signals and pin assignments A 12 Netra CP3260 Blade Server User s Guide April 2009 TABLE A 10 Zone 3 Power Connector Pin Assignments R o W Pin 1 Pin 2 P
10. 6 00000d64 RO unknown 6f00000d6f R1 unknown 6 00000d74 SEEPROM other 6 00000d79 PRSNT presence indicator 6f 0000087f T AMB temperature sensor 6 00000d87 CH1 other 6 00000d97 DO module 6 00000d9d RO unknown 6 00000da8 R1 unknown 6 00000dad SEEPROM other 6 00000db2 PRSNT presence indicator 6 00000db8 T AMB temperature sensor 6 00000dc0 other 6 00000dd0 CHO other 6 00000dd6 DO module 6 00000ddc RO unknown 6f00000de7 R1 unknown 6f00000dec SEEPROM other 6f00000df1 PRSNT presence indicator 6 00000df7 T AMB temperature sensor 6f00000dff CH1 other 6f00000e0f DO module 6 00000e15 RO unknown 6 00000e20 R1 unknown 6 00000e25 SEEPROM other 6f 00000e2a PRSNT presence indicator 6 00000e30 T AMB temperature sensor 6 00000e38 other 6 00000e48 Chapter 4 Firmware and Blade Server Management 4 23 4 24 CODE EXAMPLE 4 4 prtpicl Command Output CHO other DO module RO unknown R1 unknown SEEPROM PRSNT presence indicator temperature sensor T_AMB CH1 other DO module RO unknown R1 unknown SEEPROM PRSNT presence indicator temperature sensor 6 00000ec0 T AMB COREO other SPU other FPU other ICACHE other DCACHE other IMMU other DMMU other CORE1 other SPU other FPU other ICACHE other DCACHE other IMMU other DMMU other
11. CORE2 other SPU other FPU other ICACHE other DCACHE other IMMU other DMMU other CORE3 other SPU other FPU other ICACHE DCACHE IMMU other DMMU other COREA other SPU other FPU other 6 00000 14 other other 6 00000e4e 6 00000e54 other 6 00000e87 6 00000e8d other 6 00000ec6 6 00000ecc 6 00000ed2 6 00000ed8 6 00000ede 6 00000ee4 6 00000eea 6 00000ef0 6 00000ef6 6 00000efc 6 00000 02 6 00000 08 6 00000 06 6 00000f1a 6 00000 20 6 00000 26 6 00000 2c 6 00000 32 6 00000 38 6 00000 3e 6 00000 44 6 00000f4a 6 00000 50 6 00000 56 6 00000 f5c 6 00000 62 6 00000 68 6 00000f6e 6 00000 74 6 00000e5 f 6 00000e64 6 00000e69 6 00000e6 6 00000e77 6 00000e98 6 00000e9d 6 00000ea2 6 00000ea8 6 00000eD0 ICACHE other DCACHE other 6 00000 7a 6 00000 80 Netra CP3260 Blade Server User s Guide April 2009 CODE EXAMPLE 4 4 prtpicl Command Output IMMU other DMMU other 6 00000 86 6 00000 8c CORE5 other 6 00000f92 SPU other FPU other 6 00000 98 6 00000 9e ICACHE other 6f00000fa4 IMMU other DMMU other DCACHE other 6f00000faa 6 00000 D0 6 00000 fDb6 CORE6 other 6 00000fbc SPU other FPU other 6 00000 c2 6 00000 c8
12. The probe scsi and probe scsi all commands diagnose problems with the SCSI devices Caution If you used the halt command or the Stop A key sequence to reach the ok prompt issuing the probe scsi or probe scsi all command can hang the system The probe scsi command communicates with all SCSI devices connected to on board SCSI controllers The probe scsi all command also accesses devices connected to any host adapters installed in PCI slots For any SCSI device that is connected and active the probe scsi and probe scsi all commands display its loop ID host adapter logical unit number unique worldwide name WWN and a device description that includes type and manufacturer The following sample output is from the probe scsi all command with a Netra CP32x0 ARTM connected to the Netra CP3260 blade server probe scsi all Command Output to continue y n y 1 05 Firmware Version 1 22 01 00 Chapter 4 Firmware and Blade Server Management 4 7 4 3 3 2 4 3 3 3 4 8 probe ide Command The probe ide command communicates with all Integrated Drive Electronics IDE devices connected to the IDE bus This is the internal system bus for media devices such as the DVD drive Caution If you used the halt command or the Stop A key sequence to reach the ok prompt issuing the probe ide command can hang the system The following shows sample output from the probe ide command CODE EXAMPLE 4 2 probe ide Co
13. You have mail su Password Read Chapter 6 in the User s Guide These are called class options You must be superuser to do this To delete a file type rm filename Note Characters display differently depending on browser settings If characters do not display correctly change the character encoding in your browser to Unicode UTF 8 Related Documentation The Netra CP3260 blade server documentation is listed in the following table Except for the Important Safety Information for Sun Hardware Systems all the documents listed are available online at http docs sun com app docs prod cp3260 brd Title Part Number Netra CP3260 Board Product Notes Netra CP3260 Board Getting Started Guide 820 0455 820 0456 Important Safety Information for Sun Hardware Systems printed version only 816 7190 Preface xv Application Title Part Number Format Location Installation and Sun Netra CP32x0 SAS Storage 820 3147 PDF Online Configuration Advanced Rear Transition Module HTML Installation and Configuration Installation and Configuration Dual HD User s Guide http docs sun com app docs prod cp32x0 sas hic Sun Netra CP32x0 Quad GbE 820 3148 PDF Online Dual Fibre Channel Advanced Rear HTML Transition Module User s Guide http docs sun com app docs prod cp32x0 4gbefc l en hic Sun Netra CP32x0 10GbE 820 3150 PDF Online Advanced Rear Transition Module HTML Dual Po
14. located on the front panel of the Netra CP3260 blade server FIGURE 1 1 blinks when a hot swap is initiated and lights steadily when the blade server is ready to be removed from the system Unlatching the bottom latch on the Netra CP3260 blade server initiates the hot swap sequence The LED lights steadily when the blade server can be safely removed from the system The reverse is true when a Netra CP3260 blade server is installed into the system Once the Netra CP3260 blade server is installed into the system and the bottom latch is latched the blue Hot Swap LED blinks until the blade server is ready and then turns off The green LED lights steadily when the blade server is ready FIGURE 2 9 shows the hot swap latch and Hot Swap LED Netra CP3260 Blade Server User s Guide April 2009 FIGURE 2 9 Hot Swap Latch and Hot Swap LED Chapter 2 Hardware Installation 2 25 2 26 Netra CP3260 Blade Server User s Guide April 2009 cdeGHA PTE Software Installation This chapter contains the following sections m Section 3 1 Operating Systems and Patches on page 3 1 m Section 3 2 Configuring Payload OS NIU Driver for Multiplexing to Zones 2 and 3 on page 3 2 m Section 33 Installing as a Diskless Clients on page 3 4 m Section 34 Firmware Updates on page 3 8 m Section 3 5 Configuring for 1 GbE or 10 GbE Switches on page 3 8 m Section 3 6 Downloading and Installing SunVTS Software on page 3 9 m Section 37 Fo
15. 14 vi Netra CP3260 Blade Server User s Guide April 2009 5 2 8 4 ATCA Hot Swap Latch 5 14 5 2 8 5 LEDs 5 15 5 2 8 6 Power Control 5 15 5 2 8 7 System Monitor ADM1026 and Thresholds 5 15 5 2 8 8 FRUID PROMs 5 17 5 29 I O Subsystem Resets 5 17 5 2 10 ATCA Power Module 48V to 12V 5 18 5 2 11 TOD Clock Battery 5 18 A Physical Characteristics A 1 A 1 A 2 A 3 AA Form Factor A 1 Layout A 1 Front Panel A 3 A 3 1 Visual Indicators A 3 A 3 2 Ports A 3 Connectors and Pinout A 3 A 4 1 Front Panel Connectors A 3 A 4 1 1 Ethernet Port A 4 A 4 1 2 Dual USB Port A 4 A 4 1 3 SerialPort A 5 A 4 2 Compact Flash Connector A 6 A 4 3 Midplane Power Connector Zone 1 A 6 A 4 4 Data Transport Connector Zone 2 A 8 A 4 5 Sun Netra ARTM Connectors Zone 3 A 10 A 4 6 TOD Clock Battery Holder A 13 B Sun OEM IPMI Commands B 1 B 1 B 2 Get Version Command B 2 Get RTM Status Command B3 Contents vii B3 Solaris OS Graceful Shutdown Commands B 4 B 4 Send Sensor State Command B 5 Index Index 1 viii Netra CP3260 Blade Server User s Guide April 2009 FIGURE 1 1 FIGURE 1 2 FIGURE 1 3 FIGURE 1 4 FIGURE 1 5 FIGURE 2 1 FIGURE 2 2 FIGURE 2 3 FIGURE 2 4 FIGURE 2 5 FIGURE 2 6 FIGURE 2 7 FIGURE 2 8 FIGURE 2 9 FIGURE 5 1 FIGURE 5 2 FIGURE A 1 FIGURE A 2 FIGURE A 3 FIGURE A 4 Figures Netra CP3260 Blade Server Front Panel 1 4 Netra CP3260 Blade Server Top View 1 5 Netra CP3260 Blade Server in an ATCA She
16. 2 14 blade server installation 2 1 2 21 layout A 1 locking screws 2 22 block diagram 5 2 C chip multithreading CMT 5 4 CMT chip multithreading 5 4 Compact Flash 1 8 1 13 1 14 2 5 2 13 5 12 adding 2 13 formatting 3 10 location 2 14 replacing 2 13 compliance NEBS 1 3 PICMG specifications 1 3 configurations Ethernet 1 13 I O 1 13 Netra switches 3 9 SAS 1 13 connectors pinouts A 3 to A 13 console port 5 4 5 11 cores processor 5 2 D device information retrieving 4 19 device paths 4 8 diagnostics POST 4 3 SunVTS software 3 9 diskless clients 3 4 adding 3 5 boot server 3 4 Dual Star topology 5 1 E electric shock caution 2 1 Electronic Fuse Efuse 5 7 Ethernet 10 Gb XAUI 5 1 base interface 5 1 Index 1 fabric interface 5 1 management port 5 4 5 11 RJ 45 A 4 SERDES 5 1 F fabric interface 5 1 5 3 5 9 5 10 fan tray upgrade kit 1 14 FB DIMMs 2 5 5 8 description 2 5 installing 2 11 location 2 6 removing 2 10 requirements 2 5 features 1 2 CPU 1 2 hot swap 1 3 IPMI system management 1 3 memory 1 2 operating system 1 3 power requirements 1 2 Field Programmable Gate Array FPGA 5 12 firmware updates 3 8 flash update 1 3 form factor A 1 front panel A 3 A 5 I O 1 3 5 11 FRUID PROMs 5 17 frutree entries and properties 4 19 hierarchy 4 19 G graceful shutdown 4 5 H hardware architecture 5 1 descriptions
17. 2 Remove this FB DIMM for access to battery 3 TOD clock battery location with FB DIMM removed 2 6 2 18 Installing an Advanced Rear Transition Module Optional For rear I O access a compatible RTM must be used with the Netra CP3260 blade server Note If you are using a Netra CP32x0 ARTM refer to the appropriate Netra CP32x0 ARTM User s Guide for specific installation instructions In general install the RTM from the rear of the server Look at the front of the server and locate the slot number where the Netra CP3260 blade server is installed Then go to the back of the server and install the RTM in that particular slot Netra CP3260 Blade Server User s Guide April 2009 FIGURE 2 6 Installing a Netra CP32X0 ARTM Figure Legend 1 Netra CP3260 blade server 2 Netra CP32X0 ARTM 3 ATCA chassis midplane Chapter 2 Hardware Installation 2 19 2 7 2 20 Installing the Netra CP3260 Blade Server Caution The Netra CT 900 server fan tray upgrade kit 594 4953 must be installed in the chassis before the Netra CP3260 blade server is installed This fan tray upgrade is required to provide adequate cooling and to prevent the system from overheating or shutting down due to an over temperature condition that can occur with the older fan trays To verify that the fan tray upgrade is installed on a Netra CT 900 server log into the Shelf Manager and issue the following command for fan trays 1 2 and 3
18. 6 Push the blade server slightly into the midplane connectors and then close the latches to seat the blade server in the connectors FIGURE 2 8 7 When the lower latch is closed the blue Hot Swap LED blinks while the blade server is initializing The blue LED turns off and the green OK LED lights when the blade server is ready 8 Tighten the locking screws to ensure the blade server is secured into the shelf FIGURE 2 8 Chapter 2 Hardware Installation 2 21 FIGURE 2 8 Netra CP3260 Blade Server Latches and Locking Screws 2 8 Connecting External I O Cables External I O cables are connected to the Netra CP3260 blade server or to the Netra CP32x0 ARTM when a rear transition module is used 2 22 Netra CP3260 Blade Server User s Guide April 2009 Note Shielded cables are required for serial I O ports Unshielded cables can be used on Ethernet ports to satisfy EMI compliance standards The shields for all shielded cables must be terminated on both ends Information on connecting each of these cables follows m For Ethernet connections category 5e or better network cable is required One end of the Ethernet cable is connected to a suitable 10 100 1000BASE T switch and the other end to one of the Ethernet ports on the Netra CP3260 blade server A single 10 100BASE T Ethernet port is also available on a Netra CP32x0 ARTM when installed Refer to the appropriate Netra CP32x0 ARTM User s Guide for more information Us
19. CPU and memory 5 1 I O 5 1 IPMI 5 1 power 5 1 service processor 5 1 Sun Download Center SDLC 3 1 Sun OEM IPMI commands Get RTM Status B 3 Sun Services 1 15 SunVTS software 3 9 support 1 15 switch configuration Netra CP3140 1 GbE 3 8 Netra CP3240 10 GbE 3 8 system flash PROM 4 2 system monitor 5 15 T temperature monitoring 5 16 TOD clock battery 5 18 A 13 battery holder A 13 TOD clock battery adding 2 14 replacing 2 15 tools needed installation 2 3 topology Dual Star 5 1 U UItraSPARC T2 1 1 1 2 1 8 5 2 5 4 5 10 5 12 5 18 upgrade kit fan tray 1 14 USB port 5 12 V voltage monitoring 5 15 voltage sensor thresholds 5 16 Ww warranty 1 15 Index 4 Netra CP3260 Blade Server User s Guide April 2009
20. Handling Summary 4 15 Automatic System Recovery 4 16 4 5 1 Enabling and Disabling Automatic System Recovery 4 17 4 5 1 1 To Enable Automatic System Recovery 4 17 4 5 1 2 To Disable Automatic System Recovery 4 17 Network Device Aliases 4 18 Retrieving Device Information 4 19 Multiplexing to Zones 2and 3 4 27 Hardware Functional Descriptions 5 1 5 1 5 2 Hardware Architecture 5 1 5 1 1 UltraSPARC T2 Processor and Memory 5 2 512 I O 5 3 5 13 Base and Fabric Interfaces 5 3 5 14 Additionall O 5 3 5 15 PowerPC 5 3 5 1 6 ARTM Support 5 4 Hardware Modules 5 4 Contents v 5 2 1 UltraSPARC T2 Processor 5 4 5 2 1 1 Electronic Fuse 5 7 5 2 12 Cores 5 8 5 2 1 3 L2Cache 5 8 5 2 1 4 Memory Controller 5 8 5 2 1 5 I O Interface 5 8 5 2 2 Memory Subsystem 5 9 5 2 2 1 Memory Capacity 5 9 5 2 2 2 Memory Speed 5 9 5 23 I O Subsystem 5 9 5 2 3 1 PCI Express Switch 5 10 5 2 3 2 Base Interface 5 10 5 2 3 3 Fabric Interface 5 10 5 2 3 4 Common ARTM 5 10 5 24 Other ARTM Interfaces 5 11 5 2 4 1 Serial Ports 5 11 5 24 2 Ethernet Management Port 5 11 5 25 Front PanelI O 5 11 5 2 5 1 Ethernet Management Port 5 11 5 2 5 2 Serial Port 5 11 5 2 5 3 Dual USB Ports 5 12 5 2 6 Compact Flash Socket 5 12 5 2 7 Service Processor MPC885 5 12 5 27 1 Field Programmable Gate Array 5 12 5 2 8 Intelligent Platform Management Controller 5 13 5 2 8 1 Intelligent Platform Management Bus 5 14 5 2 8 2 Interface to the PPC 5 14 5 2 8 3 IPMB L Interface 5
21. Netra CP3260 blade server The ADM1206 and H8 firmware reports a minor major or critical alarm when the temperature of the CPU goes beyond the thresholds listed in TABLE 5 2 Also if the temperature of the CPU goes beyond the Emergency H8 Shutdown value listed in TABLE 5 2 the H8 initiates a event warning blade server to the shelf manager and shuts down TABLE 5 2 CPU Temperature Alarms Sensor Sensor Minor Alarm Major Alarm Critical Alarm Emergency Number Name UNC UC UNR H8 Shutdown 4 CPU Temp 1 gt 80 C gt 90 C gt 102 C gt 102 C 5 CPU Temp 2 gt 80 C gt 90 C gt 102 C gt 102 C 6 Board Temp gt 60 C gt 70 C gt 88 C gt 88 C 5 16 Netra CP3260 Blade Server User s Guide April 2009 5 2 8 8 5 29 The H8 temperature alarm equivalents are Minor Alarm Upper Non Critical UNC Major Alarm Upper Critical UC Critical Alarm Upper Non Recoverable UNR Emergency H8 Shutdown EMR UNR Caution These voltage and temperature thresholds should not be changed under normal operating conditions FRUID PROMs There are two FRUID PROMs on the Sun Netra CP3260 and both are 64 Kbyte SEEPROMSs One contains Sun FRU information and the other contains IPMI FRU information Each device stores static information such as the part number and manufacturing date and dynamic data that can be updated by the host system such as operational statistics and failure information I O S
22. Netra CP32x0 ARTMs The serial port can be used on either the Netra CP3260 blade Compact Flash server or Netra CP32x0 ARTM but both ports cannot be used at the same time If the serial port on Netra CP3260 blade server is used the serial port on the Netra CP32x0 ARTMs should not be used and vise versa Sun Compact Flash card The Compact Flash connector can be used to add an optional Sun 8 Gbyte or 16 Gbyte user flash type I II memory card Chapter 1 Introduction 1 13 1 5 Hot Swap Support There are three hot swap models described in the PICMG ATCA specification basic hot swap full hot swap and high availability HA hot swap Refer to the PICMG ATCA Specification which provides a detailed description of this subject The Netra CP3260 blade server supports basic hot swap The hot swap process uses hardware connection control to disconnect and connect the hardware in an orderly sequence 1 6 System Requirements and Options This section contains the system level hardware and software requirements and options for the Netra CP3260 blade server 1 6 1 Hardware Requirements and Options Sun provides the following items for customer order m Netra CP3260 blade server m The Netra CT 900 server fan tray upgrade kit part number 594 4953 This upgrade kit must be installed in the Netra CT 900 server chassis to provide adequate cooling Refer to the Netra CT 900 Server Upgrade Guide 820 3255 for more information Also see
23. Some of the salient features of the module are m 210 watts output power m Input to output isolation m 12V with current rating of 17 5A m 3 3V standby at 1 82A The IPMC draws power from IPMI Power 3 3V STBY so it can remain functional even if back end logic is powered down m Isolated remote ON OFF m Isolated A and B bus detect signals m High efficiency typically 88 percent m Operating input voltage 39V to 72V m The IC interface monitors the status of fuse input voltages output voltages and temperature and sends an alarm if any of the parameters are outside the programmable threshold m FRU information TOD Clock Battery The TOD clock battery must be type CR1632 with a minimum of 4ma abnormal charging current rating for example a Renata CR1632 Caution Risk of explosion if battery is replaced by an incorrect type Dispose of batteries properly in accordance with manufacturer s instructions and local regulations See Section 2 5 3 Adding or Replacing TOD Clock Battery on page 2 14 for battery location and installation instructions Netra CP3260 Blade Server User s Guide April 2009 APPENDIX A Physical Characteristics Specifications for the Netra CP3260 blade server are provided in the following sections m Section A 1 Form Factor on page A 1 m Section A 2 Layout on page A 1 m Section A3 Front Panel on page A 3 m Section A4 Connectors and Pinout on page A 3
24. cause the parameter changes to take effect type ok reset all The system permanently stores the parameter changes and boots automatically when the OpenBoot configuration variable auto boot is set to true its default value Note To store parameter changes you can also power cycle the system using the front panel Power button To Disable Automatic System Recovery 1 At the ok prompt type Ok setenv auto boot on error false 2 To cause the parameter changes to take effect type Ok reset all The system permanently stores the parameter change Note To store parameter changes you can also power cycle the system using the front panel Power button Chapter 4 Firmware and Blade Server Management 4 17 After you disable the automatic system recovery ASR feature it is not activated again until you re enable it 4 6 4 18 Network Device Aliases A device alias is a shorthand representation of a device path The Solaris OS provides some predefined device aliases for the network devices so that you do not need to type the full device path name TABLE 4 3 lists the network device aliases the default Solaris OS device names and associated ports for the Netra CP3260 blade server You can use the devalias command to display the device aliases TABLE 4 3 Network Device Aliases Device Alias Default Solaris 10 OS Device Name Port Description net netO ne
25. error messages are displayed on a console The format of the these messages is the following Core ID Strand ID ERROR TEST test name Core ID Strand ID H W under test description Core ID Strand ID Repair Instruction Core ID Strand ID MSG error message body Core ID Strand ID END ERROR The following is an example of a POST error message 3 2 gt ERROR TEST L2 Cache Functional 3 2 gt H W under test Core 12 Cache 3 2 gt Repair Instructions Replace items in order listed by H W under test above 3 2 gt MSG No way found to match tag address 00000000 00600000 state 3 3 2 2END ERROR 4 3 4 4 OpenBoot Firmware The Solaris OS installed operates at different run levels For a full description of run levels refer to the Solaris system administration documentation Most of the time the OS operates at run level 2 or run level 3 which are multiuser states with access to full system and network resources Occasionally you might operate the system at run level 1 which is a single user administrative state However the lowest operational state is run level 0 When the OS is at run level 0 the ok prompt appears This prompt indicates that the OpenBoot firmware is in control of the system There are a number of scenarios under which OpenBoot firmware control can occur By default before the operating system is installed the system comes up under OpenBoot firmware control m W
26. from the PROM 5 2 8 Intelligent Plattorm Management Controller The Renesas H8S 2166 provides the IPM controller IPMC function on the Netra CP3260 blade server The IPMC provides PICMG 3 0 blade server management functionality and it interfaces to the host CPU through a serial interface The IPMC subsystem is powered from the standby power The IPMC is responsible for the following m Dual buffered IPMB interfaces to connect to IPMB 0 m Serial payload interface to the host through PPC m IPMI L interface to communicate with MMC on ARTMs m Power control of the entire Netra CP3260 blade server m Power and reset control of the AMC board m Hot swap latch input and LED control m Power control m E Keying control Chapter 5 Hardware Functional Descriptions 5 13 5 2 8 1 5 2 8 2 5 2 8 3 5 2 8 4 5 14 m Environmental monitoring m Access to all environmental PC devices m Access to all IC devices when MPC is reset Intelligent Plattorm Management Bus The BMR HS8S provides dual buffered Intelligent Platform Management Bus IPMB interfaces to the IPMB 0 bus on the PICMG 3 0 midplane The I2C channels on the H8S are connected the IPMB A and IPMB B through the PC buffers The I C buffers allow the blade server I C to be isolated from the midplane until the blade server is fully seated and the 2C bus on the midplane is idle Interface to the PPC The BMR HS8S provides one serial payload interface to the PPC This interface
27. in Section 2 5 1 2 To Install FB DIMM Memory Modules on page 2 11 To Install FB DIMM Memory Modules The following procedure provides a general guide for installing additional memory However for directions on the installation process of the memory FB DIMMs on the Netra CP3260 blade server refer to the documentation that shipped with the memory module Caution Do not remove the FB DIMM from its antistatic container until you are ready to install it on the card Handle the module only by its edges Do not touch module components or metal parts Always wear a grounded antistatic wrist strap when handling modules Chapter2 Hardware Installation 2 11 1 Locate the FB DIMM connectors on the Netra CP3260 blade server Select the connectors where you will install the memory module FIGURE 2 1 If you need to replace an existing memory module with a new module see Section 2 5 1 1 To Remove FB DIMM Memory Modules on page 2 10 for instructions on removing the FB DIMM module 2 Remove the FB DIMM from its protective packaging holding the module only by the edges 3 Insert the bottom edge of the FB DIMM into the bottom of the slot s hinge style connector see O in FIGURE 2 3 The socket and module are both keyed which means the module can be installed one way only With even pressure push simultaneously on both upper corners of the FB DIMM until its bottom edge the edge with the gold fingers is firmly seated in
28. management hot swap control and some board hardware The IPMC configuration is controlled by separate firmware The Netra CP3260 blade server boots from the 4 Mbyte system flash PROM device that includes the power on self test POST and OpenBoot firmware A systems firmware progress sensor SFPS is available on the Sun Netra CP3260 blade server The purpose of the sensor is to model the firmware running on the payload and provide various states to the external management software ShMM on Netra CT 900 servers This occurs via a standard IPMI event mechanism The firmware states are Progress Hang and Error with various substates The sensor generates an IPMI event message for each state You can verify the messages by using clia sel command on the ShMM through HPI event and SNMP traps for each state of a sensor event For more information see Section B 4 Send Sensor State Command on page B 5 For detailed sensor command syntax and options refer to the Netra CT 900 Software Developer s Guide 819 1178 Even if you are using a third party chassis the SFPS commands and options apply and this document is available online http docs sun com app docs prod n900 srvr hic Netra CP3260 Blade Server User s Guide April 2009 4 2 4 2 1 Power On Self Test Diagnostics Power on self test POST is a firmware program that helps determine whether a portion of the system has failed POST verifies the core functionality
29. of the system including the CPU modules motherboard memory and some on board I O devices The software then generates messages that can be useful in determining the nature of a hardware failure POST can run even if the system is unable to boot If POST detects a faulty component it is disabled automatically preventing faulty hardware from potentially harming any software If the system is capable of running without the disabled component the system boots when POST is complete For example if one of the processor cores is deemed faulty by POST the core is disabled and the system boots and runs using the remaining cores POST diagnostic and error message reports are displayed on a console POST Test Coverage The POST diagnostics include the following tests CPU Basic Test CPU Functional Test CPU D Cache RAM CPU D Cache Tags CPU I Cache RAM Test CPU I Cache Tag RAM CPU L2 Cache UA Array Test CPU L2 Cache UA Array Test CPU Interrupt Handler CPU IMMU Memory Management Unit CPU DMMU Memory Management Unit FPU Data movement and functionality I O Controller register access I O Controller PCI configuration access Block Memory Test Network Interfaces register access and basic funtionality Modular Arithemetic Memory parity errors Test SPU Security Processing Unit Tests Chapter 4 Firmware and Blade Server Management 4 3 4 2 2 POST Diagnostic and Error Message Format POST diagnostic and
30. on the Fabric extended interface m Synchronization Clock interface supporting CLK1A B and CLK2A 2B FIGURE A 6 shows the Zone 2 connectors A 8 Netra CP3260 Blade Server User s Guide April 2009 FIGURE A 6 Zone 2 Connectors J23 J20 OOL ALH I A S E a E HG Cm m m m m TER m HG Qm m m m m m m m m m G u m m m m u t e EST Gjim mim mm im im m m m FG pr m im 7 7 m FG rf ie e fe fm m fe RE ERE iia e REESE gis ej x x xx rv f DG p s fu alee Ru m Clim w lm m m m mimiomom G mmm mim mmi jm BG g Cm Cm im m im fm fm m m fm BG gm m im fm fm m m x A jum oe oe oe oe ie m A Um io Yom jm jm ee m 1 CU x ZN TEN ZES 10 987654321 109 8765 43 2 1 TABLE A 5 gives the Zone 2 J23 connector pin assignments TABLE A 5 Zone 2 J23 Connector Pin Assignments Row A B C D E F G H 1 P23_F2_TX2 P23 F2 TX2 P23 F2 RX24 P23 F2 RX2 P23 F2 TX34 P23 F2 TX3 P23 F2 RX34 P23 F2 RX3 2 P23 F2 TX0 P23 F2 TX0 P23_F2_RX0 P23 F2 RXO P23_F2_TX1 P23 F2 TXI P23_F2_RX1 P23 F2 RX1 3 P23 Fl TX24 P23 Fl TX2 P23 Fl RX24 P23 Fl RX2 P23_F1_TX3 P23 Fl TX3 P23 Fl RX34 P23 F1 RX3 4 P23 Fl TX0 P23 Fl TX0 P23_F1_RX0 P23 Fl RXO P23_F1_TX1 P23 Fl TX1 P23_F1_RX1 P23 Fl RX1 5 P23 BI DA1 P23 BI DA1 P23_BI_DB1 P23 BI DBl P23 B
31. or fan trays 0 1 and 2 for older chassis clia fruinfo fan tray 1 grep Product Part If Product Part Model Number 371 3033 xx is displayed an upgraded fan tray is installed If Product Part Model Number is not 371 3033 an old fan tray is installed and the upgrade is required Refer to the Netra CT 900 Server Upgrade Guide 820 3255 for more information To Install the Netra CP3260 blade server perform the following steps 1 If you have installed a Netra CP32x0 ARTM go to the front of the system and locate the card slot where you installed the ARTM at the rear of the system 2 Remove the filler panel if necessary The filler panel is secured to the card cage using two screws one at the top of the filler panel the other at the bottom Store the filler panel in a safe place you might need to use it again if you have to remove a card for an extended period of time 3 Prepare the blade server by opening the injector ejector latches FIGURE 2 8 4 Carefully align the edges of the blade server with the card guides in the appropriate slot FIGURE 2 7 It might be helpful to look into the enclosure to verify correct alignment of the rails in the guides 5 Taking care to keep the blade server aligned in the guides slide the blade server in until the injector ejector latches engage the card cage Netra CP3260 Blade Server User s Guide April 2009 FIGURE 2 7 Installing Blade Server Into Chassis Slot
32. to Zone3 name SUNW niusl parent niu 80 unit address 0 phy type xgsd name SUNW niusl parent niu 80 unit address 1 phy type xgf CODE EXAMPLE 3 4 Zone 2 and Zone 3 Connectivity First NIU Port nxge0 to Zone 3 at 10Gbps Speed and the Second NIU Port nxge1 to Zone2 name SUNW niusl parent niu 80 unit address 0 phy type xgf name SUNW niusl parent niu 80 unit address 1 phy type xgsd Chapter 3 Software Installation 3 3 3 3 Installing as a Diskless Clients The following procedures describe how to create a boot server for diskless clients and how to add new diskless clients to the patched boot server For additional instructions on installing diskless clients refer to the appropriate Solaris Documentation Collection at the Sun Documentation web site at http docs sun com app docs prod solaris You must have a superuser password on your diskless server to perform the tasks in the following sections 3 3 1 Creating a Boot Server for Diskless Clients This procedure sets up a boot server by starting the operating environment services required for diskless clients Once you have set up the boot server see Section 3 3 2 Adding a Diskless Client on page 3 5 for instructions on adding diskless clients to the boot server To create a boot server for diskless clients 1 Verify that the IP addresses for all other network interfaces on the
33. 03 cpu cpu 6 00000211 cpu cpu 6 0000021 cpu cpu 6 0000022d cpu cpu 6 0000023b cpu cpu 6 00000249 cpu cpu 6 00000257 cpu cpu 6 00000265 cpu cpu 6 00000273 cpu cpu 6 00000281 cpu cpu 6 0000028f cpu cpu 6 0000029d cpu cpu 6 000002ab cpu cpu 6 000002b9 cpu cpu 6 000002c7 cpu cpu 6 000002d5 cpu cpu 6 000002e3 cpu cpu 6 000002f1 cpu cpu 6 000002ff cpu cpu 6 0000030d cpu cpu 6 0000031b cpu cpu 6 00000329 cpu cpu 6 00000337 cpu cpu 6 00000345 cpu cpu 6 00000353 cpu cpu 6 00000361 cpu cpu 6 0000036f cpu cpu 6 0000037d cpu cpu 6 0000038b Netra CP3260 Blade Server User s Guide April 2009 CODE EXAMPLE 4 4 prtpicl Command Output cpu cpu 6 00000399 cpu cpu 6 000003a7 cpu cpu 6 000003b5 cpu cpu 6 000003c3 cpu cpu 6 000003d1 cpu cpu 6 000003dE cpu cpu 6 000003ed cpu cpu 6 000003fb cpu cpu 6 00000409 cpu cpu 6 00000417 cpu cpu 6 00000425 cpu cpu 6 00000433 cpu cpu 6 00000441 cpu cpu 6 0000044f cpu cpu 6 0000045d cpu cpu 6 0000046b cpu cpu 6 00000479 cpu cpu 6 00000487 cpu cpu 6 00000495 cpu cpu 6 000004a3 cpu cpu 6 000004b1 cpu cpu 6f 000004bf cpu cpu 6 000004cd cpu cpu 6 000004db pci pciex 6 000004e9 pci pciex 6 0000050e pci pciex 6 00000533 network netw
34. 107100 FBDIMM E L gt UltraSPARC 8 FBDIMM SLOTS T2 es Processor ae DC Blocking caps HOST OBP FLASH PPC lec PPC Parallel Bus Payload RS232 Serial RS232 PORT 10 pt SMM15 X4 Payload FLASH_SEL FRONTREAR 12C Devices _ OBP HV SERIAL SP NVRAM CONSOLE SUNFRUID T y SCC MAC 2 2G 8 L Selector o S A S ie Flash bi nm Ethernet as RMII 12C Devices Dual GBE Dual FEE TET L FBDIMM SPD O 7 GBE Bo UE Up 120 ss Ic o 7 12C Netconsole 2C Li H8 z 12C RTM ADM1026 lac IPMC_l2c gt IPMI FRUID Debug Serial 12C Devices SEL EEPROM DC DC XAULRTM E Front board header Telco Daughter Card al x FRONT REAR m SERIAL CONSOLE EE bb amp 3 z i io bb LE 10 100 1000 FJ45 i Midplane Zone 2 MGMNT Zone 3 RTM Si Midplane Zone 1 FRONT PANEL ETHERNET NOTES 1 Functions in orange are for Power PC 2 Functions in yellow are replacing Southbridge 5 1 UItraSPARC T2 Processor and Memory The UltraSPARC T2 processor has eight SPARC physical processor cores Each SPARC physical processor core has full hardware support for eight strands two integer execution pipelines one floating point execution pipeline and one memory pipeline The eight SPARC cores are connected through a crossbar to an on chip unified 4 Mbyte 16 way associative L2 cache 5 2 Netra CP3260 Blade Server User s Guide April 2009 5 1 2 5 1 3 5 1 4 541 5
35. 5 1 5 18 device paths 4 8 host adapter probe scsi 4 7 hot swap LED 2 25 support 1 14 hot swap latch 2 25 l I O 5 3 dual USB ports 5 3 front panel 1 3 5 11 IDE port Compact Flash 5 3 resets 5 17 I2C interface 5 18 IDE bus 4 8 init Solaris command 4 5 injector ejector latches 2 21 install patches 3 5 installation collecting network information 3 7 FB DIMMs 2 5 2 11 overall procedure 2 4 preparation 2 3 software 3 1 summary 2 4 tools needed 2 3 installing blade server 2 22 integrated drive electronics IDE 4 8 Intelligent Platform Management Bus IPMB 5 14 Intelligent Plattorm Management Controller IPMC 4 2 Internet Assigned Numbers Authority IANA B 1 IPMB L 5 14 IPMC 4 2 5 13 IPMI system management 1 3 L L2 cache 5 8 labels 1 16 MAC address 1 16 part number 1 16 serial number 1 16 latch hot swap 2 25 LEDs active A 3 fault 5 15 A 3 front panel A 3 healthy status 5 15 hot swap 2 25 5 15 A 3 local network information 3 7 logical unit number probe scsi 4 7 loop IDs probe scsi 4 7 Index 2 Netra CP3260 Blade Server User s Guide April 2009 M MAC address 3 8 label 1 16 management port Ethernet 5 11 manual system reset 4 5 memory 1 2 5 8 FB DIMMs 2 5 5 9 N netfunction NetFn B 1 Netra CP32X0 ARTM 1 8 1 11 1 13 1 14 2 20 network boot 1 8 network device aliases 4 18 O OpenBoot ok prompt 4 5 OpenBoot commands probe ide
36. 8 m Section 2 8 Connecting External I O Cables on page 2 23 2 1 Equipment and Operator Safety Refer to Important Safety Information for Sun Hardware Systems 816 7190 for general safety information Read the safety statements specific to the Netra CP3260 blade server carefully before you install or remove any part of the system Caution Depending on the particular chassis design operations with open equipment enclosures can expose the installer to hazardous voltages with a consequent danger of electric shock Ensure that line power to the equipment is disconnected during operations that make high voltage conductors accessible 2 1 2 2 The installer must be familiar with commonly accepted procedures for integrating electronic systems and with the general practice of Sun systems integration and administration Although parts of these systems are designed for hot swap operation other components must not be subjected to such stresses Work with power connected to a chassis only when necessary and follow these installation procedures to avoid equipment damage This equipment is sensitive to damage from electrostatic discharge ESD from clothing and other materials Use the following antistatic measures during an installation m If possible disconnect line power from the equipment chassis when servicing a system or installing a hardware upgrade If the chassis cannot be placed upon a grounded antistatic mat connect a g
37. 9 2 5 1 1 2 10 To Remove FB DIMM Memory Modules You might need to remove a FB DIMM module from the Netra CP3260 blade server if you are returning the FB DIMM module or the blade server for service or if you are replacing a module with another FB DIMM module Note Safely store the original factory shipped FB DIMM and related FB DIMM packaging You might wish to store any removed FB DIMM in the new FB DIMM packaging or use the packaging for service To remove a FB DIMM from the Netra CP3260 blade server perform the following steps 1 Take antistatic precautions attach and electrically ground the wrist strap Caution Always wear a grounded antistatic wrist strap when handling modules 2 Place the Netra CP3260 blade server on an antistatic mat or on the blade server s antistatic bag if you do not have a mat available 3 For the FB DIMM you wish to remove simultaneously pull both FB DIMM retainer clips outward from the slot see in FIGURE 2 2 4 Grasp the FB DIMM by the edges and carefully pull it out of its connector see O in FIGURE 2 2 A slight rocking motion might be required Caution Take extra care to prevent damage when removing the outer most FB DIMMs 5 Place it in an antistatic bag Netra CP3260 Blade Server User s Guide April 2009 FIGURE 2 2 Removing an FB DIMM Memory Module 2 5 1 2 6 If you are replacing the module you removed with a new FB DIMM install it as described
38. A 2 USB Connector Pin Assignments Pin Signal Description Pin Signal Description Al 5 V fused B1 5 V fused A2 USB2 B2 USB3 A3 USB2 B3 USB3 A4 Ground B4 Ground Serial Port FIGURE A 4 contains the connector pin assignments for the front panel serial port Appendix A Physical Characteristics A 5 A 4 2 A 4 3 A 6 FIGURE A 4 Front Panel Serial Port Diagram 10101 12345678 TABLE A 3 shows the serial port connector pin assignments TABLE A 3 Serial Port RJ 45 Connector Pinouts Signal Signal Pin Name Function Pin Name Function 1 RTS Request To Send 5 GND Ground 2 DTR Data Terminal Ready 6 RXD Receive Data 3 TXD Transmit Data 7 DSR Data Set Ready 4 GND Ground 8 CTS Clear To Send Compact Flash Connector The Compact Flash connector is a type I II connector Midplane Power Connector Zone 1 The Netra CP3260 blade server uses a Zone 1 power connector It provides support for the following signals m Two 48 volt DC power feeds four signals each eight signals total m Two IPMB ports two signals each four signals total m Geographic address eight signals The analog test and ring voltage pins are left unconnected Netra CP3260 Blade Server User s Guide April 2009 FIGURE A 5 shows the pin assignments FIGURE A 5 Power Distribution Connector Zone 1 P10 13 9 9 8 25 9 14 10 22 18 9 15 11 9 38 26 OKO 24 20 76 i2 TABLE A 4 lists the power connect
39. CP3260 blade server the new blade assumes the previous configuration Note The host must be configured to match the MUX configuration For customers using blade servers in a Netra CT 900 chassis a complete end to end solution is provided The MUX feature is implemented through the ShMM firmware and IPMI commands on the IPMC These commands extend MUX configuration access to the management software so that during blade server hot swaps the MUX configuration is persistent across blade server activations and deactivations Customers who use Sun Netra CP3260 blade servers in a third party chassis which does not have the Netra CT 900 chassis ShMM management software implemented can save MUX configurations in a configuration file or in a persistent storage managed by system management software When the system management software detects blade server activation it sends the command to set MUX to the programmed state Because management software sends the command during every blade server activation the configuration is persistent across blade server deactivation and activation Refer to the following documentation m For ShMM commands refer to the Netra CT 900 Server Administration and Reference Manual 819 1177 m For IPMI commands refer to the Netra CT 900 Software Developer s Guide 819 1178 m For payload configuration see Section 3 2 Configuring Payload OS NIU Driver for Multiplexing to Zones 2 and 3 on page 3 2 C
40. E A 1 shows all the basic I O connectors to the front and the rear of the Netra CP3260 blade server Front Panel Connectors The front panel has the following connectors m One 10 100 1000BASE T Ethernet port RJ 45 m One serial port RJ 45 m Dual USB ports USB 2 0 Appendix A Physical Characteristics A 3 A 4 1 1 A 4 1 2 Ethernet Port The Ethernet connector is a RJ 45 connector The controller autonegotiates to either 10BASE T 100BASE T or 1000BASE T The Ethernet connector pin numbering is shown in FIGURE A 2 FIGURE A 2 Ethernet RJ 45 Connector lt e o o gt 12345678 Activity LED yellow _ BEL Link LED green TABLE A 1 shows the Ethernet connector pin assignments TABLE A 1 Ethernet Port Connector Pin Assignments Signal Signal Pin Name Function Pin Name Function BI DA Bi directional pair A BI_DA Bi directional pair A BI_DC Bi directional pair C BI DB Bi directional pair C BI_DB Bi directional pair B BI_LDD Bi directional pair D Ae WO N HL oN OA A BI_DC Bi directional pair B BI_DD Bi directional pair D Dual USB Port The dual port Universal Serial Bus USB connector pin numbering is shown in FIGURE A 3 A 4 Netra CP3260 Blade Server User s Guide April 2009 A 4 1 3 FIGURE A 3 Dual USB Connector For USB connector signals see TABLE A 2 TABLE
41. I DC1 P23 BI DCl P23 BI DD1 P23 BI DDI 6 P23 BI DA2 P23 BI DA2 P23 BI DB2 P23 BI DB2 P23_BI_DC2 P23 BI DC2 P23_BI_DD2 P23 BI DD2 7 N C N C N C N C N C N C N C N C 8 N C N C N C N C N C N C N C N C 9 N C N C N C N C N C N C N C N C 10 N C N C N C N C N C N C N C N C TABLE A 6 gives the Zone 2 J20 connector pin assignments TABLE A 6 Zone 2 J20 Connector Pin Assignments Row A B Cc D E F G H 1 CLK1A P CLK1A N CLKIB P CLK1B_N CLK2A_P CLK2A_N CLK2B_P CLK2B_N 2 N C N C N C N C CLK3A_P CLK3A_N CLK3B_P CLK3B_N 3 N C N C N C N C N C N C N C N C Appendix A Physical Characteristics A 9 TABLE A 6 Zone 2 J20 Connector Pin Assignments Continued Row A B c D E F G H 4 N C N C N C N C N C N C N C N C 5 N C N C N C N C N C N C N C N C 6 N C N C N C N C N C N C N C N C 7 N C N C N C N C N C N C N C N C 8 N C N C N C N C N C N C N C N C 9 N C N C N C N C N C N C N C N C 10 N C N C N C N C N C N C N C N C For information about multiplexing to Zone 2 or Zone 3 see Section 4 8 Multiplexing to Zones 2 and 3 on page 4 27 A 4 5 Sun Netra ARTM Connectors Zone 3 The Netra CP3260 blade server provides all the I O and power connections for rear access through the Zone 3 RTM connectors The Zone 3 I O connectors are shown in FIGURE A 7 and the Zone 3 power connector is shown in FIGURE A 8 The pin numbers and signal assignments are listed in the associated tables
42. P3260 Blade Server Feature Summary Feature Description PICMG Single wide ATCA module compliance ATCA 8U form factor Single wide ATCA module 6 HP slot width PICMG 3 0 R1 0 PICMG 3 1 R1 0 Node board Functions as a CPU node board with the Solaris operating system and support software packages Operating Solaris 10 8 07 OS and subsequent compatible versions with supported system Netra patches Internal I O connections to ATCA midplane External I O IPMI system management Hot swap support Front panel access I O Rear access interface Compact Flash support Building compliance Flash update Dual 10 100 1000BASE T Ethernet for Base interfaces 10 Gb XAUI or 1 Gb SERDES Ethernet interfaces as Fabric interfaces Dual IPMI channel connects to the midplane for communicating with the Shelf Management card One 10 100 1000BASE T Ethernet maintenance port on front panel One asynchronous serial port for console on front panel Dual USB 2 0 compliant ports on the front panel Rear access available on Netra CP32x0 ARTMs One 10 100 1000BASE T Ethernet port RJ 45 One asynchronous serial port RJ 45 Note With a Netra CP32x0 ARTM installed both the front panel and ARTM serial ports are active Either serial port can be used but only one of the serial ports can be used at a time Uses IPMI communications with baseboard management controller BMC performs advanced system monitoring ASM on local
43. RU type bridge fhs The prtpicl v command shows the condition of all devices in the PICL tree Sample output from the prtpicl command on the Netra CP3260 blade server is shown in CODE EXAMPLE 4 4 CODE EXAMPLE 4 4 prtpicl Command Output prtpicl picl 6 00000001 platform sun4v 6 00000005 scsi_vhci devctl 6 00000022 memory memory 6 000000cd memory segment memory segment 6 00000c10 memory bank memory bank 6 00000c13 memory bank memory bank 6f00000c1ic memory bank memory bank 6 00000c25 memory bank memory bank 6f00000c2e virtual devices virtual devices 6f000000df channel devices channel devices 6 000000f2 virtual channel obp device 6f 000000ff virtual channel client obp device 6 00000109 virtual channel obp device 6 00000113 Chapter 4 Firmware and Blade Server Management 4 19 4 20 CODE EXAMPLE 4 4 prtpicl Command Output virtual channel client obp device 6 0000011d n2cp obp device 6 00000127 ncp obp device 6 00000134 random number generator obp device 6 00000141 flashprom flashprom 6 0000014d console serial 6 00000154 rtc obp device 6 00000161 cpu cpu 6 00000169 cpu cpu 6 00000177 cpu cpu 6 00000185 cpu cpu 6 00000193 cpu cpu 6 000001a1 cpu cpu 6f 000001af cpu cpu 6 000001bd cpu cpu 6 000001cb cpu cpu 6 000001d9 cpu cpu 6 000001e7 cpu cpu 6 000001f5 cpu cpu 6 000002
44. SS 4v SUN microsystems Sun Netra CP3260Blade Server User s Guide Sun Microsystems Inc www sun com Part No 820 0457 11 April 2009 Revision 01 Submit comments about this document at http www sun com hwdocs feedback Copyright 2009 Sun Microsystems Inc 4150 Network Circle Santa Clara California 95054 U S A All rights reserved This distribution may include materials developed by third parties Parts of the o may be derived from Berkeley BSD systems licensed from the University of California UNIX is a registered trademark in the U S and in other countries exclusively licensed through X Open Company Ltd Sun Sun Microsystems the Sun logo Netra Sun Ray the Netra logo and the Solaris logo are trademarks or registered trademarks of Sun Microsystems Inc and its subsidiaries in the U S and other countries All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc in the U S and other countries Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems Inc Use of any spare or replacement CPUs is limited to repair or one for one D procement of CPUs in products exported in compliance with U S export laws Use of CPUs as product upgrades unless authorized by the U S Government is strictly prohibited DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING AN
45. Section 2 7 Installing the Netra CP3260 Blade Server on page 2 21 for fan tray upgrade verification information m Optional Netra CP32x0 ARTMs An RTM enables rear system I O access to the network to a boot device or to a console terminal The Netra CP32x0 ARTMs also provide a variety of I O and storage solutions an AMC like interface and hot swap capability The Netra CP32x0 ARTMs are optional and must be ordered separately Contact your Sun representative for information on the Netra CP32x0 ARTMs and compatibility with the Netra ATCA node boards m Optional Compact Flash card An IDE Compact Flash card is optional and must be ordered separately Acquire the following components if needed 1 14 Netra CP3260 Blade Server User s Guide April 2009 1 6 2 m Serial terminal or terminal emulation for console output m Cables for terminal and network connections m Optional AMC disks and PCI E cards The following are ATCA and other minimum requirements met by the Netra CP3260 blade server m ATCA system enclosure for 8U boards includes chassis midplane power supply m Console output device or serial terminal m Boot device such as hard drive network or Compact Flash card m Peripheral device for network access m Intelligent Plattorm Management Controller IPMC Software Requirements The Netra CP3260 blade server supports Solaris 10 8 07 OS and subsequent compatible versions Refer to the Netra CP3260 Blade
46. Server Product Notes 820 0455 for more Solaris OS information including a list of the required software patches You can view and download the latest version of this manual at the following web site http docs sun com app docs prod netra brds 1 7 Technical Support and Warranty Should you have any technical questions or support issues that are not addressed in the Netra CP3260 blade server documentation set or on the web site contact your local Sun Services representative This hardware carries a one year return to depot warranty For customers in the US or Canada call 1 800 USA 4SUN 1 800 872 4786 For customers in the rest of the world find the World Wide Solution Center nearest you by visiting our web site http www sun com service contacting solution html When you call Sun Services be sure to indicate if the Netra CP3260 blade server was purchased separately and is not associated with a system Have the proper blade server identification information ready Be prepared to give the representative the blade server part number serial number and date code FIGURE 1 5 Chapter 1 Introduction 1 15 1 7 1 Part Number Serial Number and Revision Number Identification The Netra CP3260 blade server part number serial number and revision can be found on labels located on the card FIGURE 1 5 The Sun barcode labels provide the following information m SunSN Sun serial number for example 1005LCB 0626WM001M m SunPN
47. There are four on chip memory controllers that interface directly to FB DIMM memory and include eight FB DIMM slots one per channel or two per memory controller In addition there are an on chip PCI Express I O interface and two 10 Gb Ethernet ports The UltraSPARC T2 processor is a highly integrated processor that implements the 64 bit SPARC V9 architecture I O All I O is provided via the UltraSPARC T2 PCI Express interface A PLX switch provides the PCI E interfaces Dual Intel NICs or network adapters supply the two Ethernet interfaces to the front panel and the ARTM for management interfaces Base and Fabric Interfaces The Base interface extends from the Intel 82571 NIC connected to the PCI E switch via a x4 connection The PCI E switch then connects to the Power PC and to the Base interface The Fabric interface uses the processor s XAUI interface and connects to the midplane fabric interface Additional I O The additional I O includes m RS 232 Serial Port m IDE port for Compact Flash card m Dual USB ports A PCI E to PCI bridge provides an on board local PCI bus to a PCI to USB device that provides the USB ports Two USB ports connect to the front panel and one USB port connects to an USB to IDE bridge The USB to IDE bridge supplies the IDE interface for the Compact Flash Power PC The RMII port on the Power PC connects to the Base interface through an Ethernet switch Chapter 5 Hardware Functional Descripti
48. Y IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Copyright 2009 Sun Microsystems Inc 4150 Network Circle Santa Clara California 95054 Etats Unis Tous droits r serv s Cette distribution peut comprendre des composants d velopp s par des tierces parties Des parties de ce produit pourront tre d riv es des syst mes Berkeley BSD licenci s par l Universit de Californie UNIX est une marque d pos e aux Etats Unis et dans d autres pays et licenci e exclusivement par X Open Company Ltd Sun Sun Microsystems le logo Sun Netra Sun Ray le logo Netra et le logo Solaris sont des marques de fabrique ou des marques d pos es de Sun Microsystems Inc ou ses filiales aux Etats Unis et dans d autres pays Toutes les marques SPARC sont utilis es sous licence et sont des marques de fabrique ou des marques d pos es de SPARC International Inc aux Etats Unis et dans d autres pays Les produits portant les marques SPARC sont bas s sur une architecture d velopp e par Sun Microsystems Inc see above L utilisation de pieces detachees ou d unites centrales de remplacement est limitee aux reparations ou a l echange standard d unites centrales pour les produits exportes conformement a la legislation americaine en matiere d exportation Sauf autorisation par les autorites des Etats Unis l utilisation d unit
49. aded 3 7 3 10 Formatting the Optional Compact Flash Card The Compact Flash card is an USB removable media device on the Sun Netra CP3260 blade server Therefore the Solaris OS rmformat utility must be used to format the device The Solaris OS also provides a removable media framework for use with this type of device Refer to the rmformat 1 man page for more information Netra CP3260 Blade Server User s Guide April 2009 cd GHAPTE Firmware and Blade Server Management This chapter contains the following sections Section 4 1 System Firmware on page 4 2 Section 4 2 Power On Self Test Diagnostics on page 4 3 Section 4 3 OpenBoot Firmware on page 4 4 Section 4 4 Error Handling Summary on page 4 15 Section 4 5 Automatic System Recovery on page 4 16 Section 4 6 Network Device Aliases on page 4 18 Section 4 7 Retrieving Device Information on page 4 19 Section 4 8 Multiplexing to Zones 2 and 3 on page 4 27 4 1 4 1 4 2 System Firmware The Netra CP3260 blade server contains a modular firmware architecture that gives you latitude in controlling boot initialization You can customize the initialization test the firmware and even enable the installation of a custom operating system This platform also employs the Intelligent Platform Management Controller IPMC described in Section 5 2 8 Intelligent Platform Management Controller on page 5 13 which controls the system
50. aded boot unless you change this setting to true In addition the system will not attempt a degraded boot in response to any fatal nonrecoverable error even if degraded booting is enabled For examples of fatal nonrecoverable errors see Section 4 3 4 OpenBoot Configuration Variables on page 4 12 OpenBoot Commands You type the OpenBoot commands at the ok prompt The OpenBoot commands that can provide useful diagnostic information include m probe scsi and probe scsi all m probe ide m show devs For a complete list of OpenBoot commands and more information about the OpenBoot firmware refer to the OpenBoot 4 x Command Reference Manual An online version of the manual is included with the OpenBoot Collection AnswerBook that ships with Solaris software Netra CP3260 Blade Server User s Guide April 2009 4 3 3 1 CODE EXAMPLE 4 1 Do you wish MPT Version 0 ok 0 ok probe scsi all This command may hang the system if a Stop A or halt command has been executed Please type reset all to reset the system before executing this command pci 0 pci 0 pci a pci 0 usb 4 2 storage 2 Unit 0 Disk TUSB6250USB20 DISK DRIVE pci 0 pci 0 pci 8 scsi d Target 2 Unit 0 Disk SEAGATE ST914602SSUN146G0400 286739329 Blocks 146 GB SASAddress 5000c5000576b0e5 PhyNum 7 Target 3 Unit 0 Disk SEAGATE ST914602SSUN146G0400 286739329 Blocks 146 GB SASAddress 5000c50005d16415 PhyNum 6 probe scsi and probe scsi all Commands
51. ading and performing the following steps 1 Become familiar with the contents of the documentation referenced in the steps 2 Verify that all listed hardware and software are available see Section 1 6 System Requirements and Options on page 1 14 3 Check power thermal environmental and space requirements see Section 2 3 1 Checking Power Thermal Environmental and Space Requirements on page 2 3 4 Verify that local area network LAN preparations are completed see Section 3 3 3 Determining Local Network IP Addresses and Host Names on page 3 7 5 Ensure that the host names and their network IP addresses are allocated and registered at the site Checking Power Thermal Environmental and Space Requirements Verify that you meet the following requirements m Your enclosure specifications support the sum of the specified maximum blade server power loads Chapter 2 Hardware Installation 2 3 Facility power loading specifications can support the rack or enclosure requirements Your enclosure specifications support the cooling airflow requirements The Netra CP3260 blade server fits a standard ATCA shelf or chassis If your installation requirements are different contact your field application engineer 2 4 Installation Procedure Summary The procedure in this section summarizes the Netra CP3260 blade server installation at a high level Be sure to read the details in Section 2 5 Configuring O
52. ale locale x ns name service x nameserver name seroer Where ip address is the client s IP address ethernet address is the client s Ethernet address host name is the client s host name n is the Solaris OS version you are using swap size is the size of the swap space that you will be using The default is 24 however your swap space should be the same amount as your memory time zone is the client s time zone locale is the client s locale name service is the client S nameservice name seroer is the nameserver s hostname backslash is a line continuation character indicating that the command is continued on the next line Refer to the smdiskless 1M man page for more information and options For example smdiskless add i 129 144 214 99 e 8 0 20 22 b3 aa n client host Xx os sparc sun4v Solaris 10 x root export root client host x swap export swap client host x swapsize 999 x tz US Pacific x localezen US x ns NIS x nameserver nameserver host Where ip address 129 144 214 99 ethernet address 8 0 20 22 b3 aa host name 2 client host n 10 for Solaris 10 swap size 128 3 6 Netra CP3260 Blade Server User s Guide April 2009 m fime zone US Pacific m locale en US m name service NIS m n me server nameserver host m backslash is a line continuation character indicating that the command is continued on the next line You must type your superuser pa
53. ations to suit each end user requirement For example the blade server can be configured to boot from a network as a diskless client with either a front panel or RTM network connection or from an optional Compact Flash card Also a Netra CP32x0 SAS Storage ARTM Dual HD can be installed to provide local disk I O which can be used optionally as a boot path 1 8 Netra CP3260 Blade Server User s Guide April 2009 FIGURE 1 3 Netra CP3260 Blade Server in an ATCA Shelf Enclosure Chapter 1 Introduction 1 9 Kless client that boots through network from a remote server 1 10 Netra CP3260 Blade Server User s Guide April 2009 Figure Legend Remote server Sun Netra CP32X0 Advanced RTM installed from rear Serial connection Console terminal Ethernet connection RJ 45 Sun Netra CP3260 blade server installed from front Netra CT 900 server ATCA shelf oc 5p ON 1 4 Advanced Rear Transition Modules An optional Sun Netra CP32X0 Advanced Rear Transition Module ARTM can be installed into the rear of the ATCA enclosure opposite the Netra CP3260 blade server FIGURE 1 3 The Netra CP32x0 ARTM connects to the Netra CP3260 blade server s Zone 3 rear I O connectors FIGURE 1 2 Contact your Sun representative for information on the Netra CP32x0 ARTMs FIGURE 1 4 shows the physical relationship between the blade server the rear transition module and the midplane in a typical ATCA system Chapter 1 Introduction 1 11
54. board interface for example temperature sense FRU ID and control Board hot swap support One 10 100 1000BASE T Ethernet port RJ 45 Dual USB 2 0 compliant ports One serial port RJ 45 Common rear transition module RTM interface supporting the Sun Netra CP32X0 ARTMs Compact Flash socket to support a Sun 8 Gbyte or 16 Gbyte user flash type I II memory card Network Equipment Building Systems NEBS Level 3 Supported from downloaded file Chapter 1 Introduction 1 3 FIGURE 1 1 Netra CP3260 Blade Server Front Panel 1 4 Netra CP3260 Blade Server User s Guide April 2009 FIGURE 1 2 Netra CP3260 Blade Server Top View Chapter 1 Introduction 1 5 Pel I Ssh un Dr jJ n mra Figure Legend 1 2 3 4 5 6 7 8 Top latch Out of Service LED yellow Ethernet management port RJ 45 OK LED green Dual USB Ports USB 2 0 Serial console port RJ 45 ttya Hot Swap LED blue Hot Swap switch and bottom latch Figure Legend Zone 1 power connector Zone 2 signal connectors Zone 3 RTM signal connectors A U N Zone 3 ARTM power connector Chapter 1 Introduction 1 7 Figure Legend 5 Compact Flash connector 6 UltraSPARC T2 processor under heat sink 7 FB DIMMs 1 3 System Configurations Netra CP3260 blade servers can be installed in an ATCA shelf or chassis as shown in FIGURE 1 3 The blade servers can be deployed in various electrical configur
55. boot server have corresponding host names in the hosts database 2 Log in to the network server as superuser and change to the usr sadm bin directory cd usr sadm bin 3 Use the smosservice command to add boot services to the installation server Note The backslash in the following code boxes is a line continuation character indicating that the command is continued on the next line smosservice add u root p root password x mediapath image_directory x platform sparc sun4v Solaris_n x cluster SUNWCXall x locale locale Where m root password is the root password for the installation server m image directory is the path to the directory where the Solaris install image is stored m nisthe Solaris OS version you are using 3 4 Netra CP3260 Blade Server User s Guide April 2009 locale is the locale that you want to use backslash is a line continuation character indicating that the command is continued on the next line Refer to the smosservice 1M man page for more information and options For example smosservice add u root p root password x mediapath export install x platform sparc sun4v Solaris 10 x cluster SUNWCXall x locale zen US Did Where root password root password image directory export install locale en_US n 10 for Solaris 10 backslash is a line continuation character indicating that the command is continued on the n
56. e virtual memory virtual memory 6 000000d5 iscsi iscsi 6f 0000082b physical platform picl 6f00000c44 SYS chassis 6 00000c47 MB container 6 00000c4d SEEPROM other 6 00000c58 SCC NVRAM other 6 00000c5e GBEO container 6 00000c64 GBE1 container 6 00000c6b PCIE container 6 00000c72 PCIE IO container 6 00000c79 CMPO other 6 00000c80 NIUO other 6 00000c86 NIU1 other 6 00000c8c PIUO other 6 00000c92 MCUO other 6 00000c98 MCU1 other 6 00000c9e MCU2 other 6 00000ca4 MCU3 other 6 00000caa L2_BANKO other 6 00000cb0 L2_BANK1 other 6 00000cb6 L2 BANK2 other 6f00000cbc L2 BANK3 other 6 00000cc2 L2 BANKA other 6f00000cc8 L2 BANK5 other 6f00000cce L2 BANK6 other 6 00000cd4 Netra CP3260 Blade Server User s Guide April 2009 CODE EXAMPLE 4 4 prtpicl Command Output L2_BANK7 other 6 00000cda BRO BR1 BR2 BR3 other 6 00000ce0 CHO other 6 00000ce6 DO module 6 00000cec RO unknown 6f 00000cf7 R1 unknown 6 00000cfc SEEPROM other 6f00000d01 PRSNT presence indicator 6 00000d07 T AMB temperature sensor 6f 00000d0f CH1 other 6 00000d1f DO module 6 00000d25 RO unknown 6 00000d30 R1 unknown 6 00000d35 SEEPROM other 6f00000d3a PRSNT presence indicator 6f 00000d40 T AMB temperature sensor 6 00000d48 other 6 00000d58 CHO other 6 00000d5e DO module
57. e Serial Attached SCSI controller Number of incorrect security password attempts Firmware security level Chapter 4 Firmware and Blade Server Management 4 13 TABLE 4 2 Variable OpenBoot Configuration Variables Possible Values Default Value Description security password ttya mode use nvramrc verbosity 4 3 4 1 variable name none 9600 8 n 1 9600 8 n 1 true false false max min min none normal Firmware security password if security mode is not none never displayed Do not set this directly Serial management port baud rate bits parity stop handshake The serial management port only works at the default values If true execute commands in NVRAMRC during server startup Controls the amount and detail of OpenBoot output Default is min none Only error and fatal messages are displayed on the system console min Notice error warning and fatal messages are displayed on the system console normal Summary progress and operational messages are displayed on the system console in addition to the messages displayed by the min setting max Detailed progress and operational messages are displayed on the system console Viewing and Setting OpenBoot Configuration Variables Halt the server to display the ok prompt m To display the current values of all OpenBoot configuration variables use the printenv command The following exampl
58. e result is that the 8 core processor handles up to 64 active threads concurrently and a 6 core processor handles up to 48 active threads concurrently Netra CP3260 Blade Server User s Guide April 2009 FIGURE 5 2 UltraSPARC T2 Multicore Processor Block Diagram Chapter 5 Hardware Functional Descriptions 5 5 5 6 Netra CP3260 Blade Server User s Guide April 2009 Fully buffered UltraSPARC T2 de cat 40 DIMMs FBD SPARC core sis SPARC core W D e x ER SPARC core S 2h SPARC core S zl SPARC core S zh e E N oo SPARC core E zl 2 O AR 64 B 128 64 SPARC core S zh SPARC core S 2h TCU CCU Efuse EE e Optional dual channel mode 5 2 1 1 Electronic Fuse The Electronic Fuse Efuse block within the UltraSPARC T2 processor contains configuration information that is electronically burned in manufacturing Chapter 5 Hardware Functional Descriptions 5 7 5 2 1 2 5 2 1 3 5 2 1 4 Sul 5 8 Cores The UltraSPARC T2 processor provides 8 physical SPARC processor cores and each physical core is capable of supporting 8 threads for a total of 64 threads L2 Cache The UltraSPARC T2 provides a total of 4 Mbytes of L2 cache banked 8 ways Memory Controller The UltraSPARC T2 supports 4 FB DIMM memory controllers each controller capable of supporting 2 FB DIMM memory channels The UltraSPARC T2 con
59. e shows a short excerpt of this command s output 0 ok printenv Variable Name Value local mac address true fcode debug false scsi initiator id 7 oem logo false auto boot true boot command boot boot file Default Value true false d false true boot 4 14 Netra CP3260 Blade Server User s Guide April 2009 boot device disk net disk net use nvramrc false false error reset recovery boot boot 0 ok m To set or change the value of an OpenBoot configuration variable use the setenv command 0 ok setenv error reset recovery none error reset recovery none m To set OpenBoot configuration variables that accept multiple keywords separate keywords with a space 4 4 Error Handling Summary Error handling during the power on sequence falls into one of the following three cases m If no errors are detected by POST or OpenBoot firmware the system attempts to boot if auto boot is true m If only nonfatal errors are detected by POST or OpenBoot firmware the system attempts to boot if auto boot is true and auto boot on error is true Nonfatal errors include the following SATA Serial Advanced Technology Attachment subsystem failure In this case a working alternate path to the boot disk is required m Ethernet interface failure m Serial interface failure a Memory failure Given a failed FB DIMM the firmware will unconfigure the entire logical bank associated w
60. e software configuration Chapter 4 provides information about the Netra CP3260 firmware Chapter 5 provides hardware and functional descriptions of the Netra CP3260 blade server Appendix A provides information about the physical characteristics of the Netra CP3260 blade server Appendix B describes the Sun specific OEM defined Intelligent Platform Management Interface IPMI commands xiii Using UNIX Commands This document might not contain information on basic UNIX commands and procedures such as shutting down the system booting the system and configuring devices Refer to the following for this information m Software documentation that you received with your system m Solaris Operating System documentation which is at http docs sun com app docs prod solaris xiv Shell Prompts Shell Prompt C shell C shell superuser Bourne shell and Korn shell Bourne shell and Korn shell superuser machine name machine name Sun Netra CP3260 Blade Server User s Guide April 2009 Typographic Conventions Typeface Meaning Examples AaBbCc123 The names of commands files and directories on screen computer output AaBbCc123 What you type when contrasted with on screen computer output AaBbCc123 Book titles new words or terms words to be emphasized Replace command line variables with real names or values Edit your login file Use 1s a to list all files 9
61. e the Ethernet device names shown in TABLE 2 1 when configuring the Ethernet ports TABLE 2 1 Ethernet Device Names Solaris 10 OS Ethernet Ports Device Name Ethernet management port on front panel e1000g4 m A shielded asynchronous serial I O cable can be attached from serial communication devices to the RJ 45 serial port on the Netra CP3260 blade server s front panel A serial port is also provided on the Netra CP32x0 ARTM You can use either the serial port on the Netra CP3260 blade server or the serial port on Netra CP32x0 ARTM but only one of the ports should be used at one time Once a serial cable is connected use the tip utility on the host to establish a full duplex terminal connection with the Netra CP3260 blade server At the UNIX prompt in a command tool or shell tool type tip 9600 dev ttya for serial port m USB 2 0 cables can be attached from USB devices to the dual USB 2 0 compliant ports on the Netra CP3260 blade server s front panel Chapter 2 Hardware Installation 2 23 2 9 2 24 Hot Swapping the Netra CP3260 Blade Server The Netra CP3260 blade server supports hot swapping at the blade server level Note Before hot swapping the blade server the system administrator should gracefully shutdown the applications and operating system and deactivate the blade server After the blade server is replaced the system administrator should activate the new blade server The blue Hot Swap LED
62. e2 00 Byte3 00 Byte4 2A Byte5 CPLD version Byte6 REV1 byte of IPMC firmware Byte7 REV2 byte of IPMC firmware Byte8 Reserved for future use ignore Byte9 Reserved for future use ignore ByteA Reserved for future use ignore Netra CP3260 Blade Server User s Guide April 2009 Example Terminal Mode B8 00 80 00 00 2A Request BC 00 80 00 00 00 2A 02 02 00 00 00 00 Response m The IPMC version is read as lower nibble of REVI high nibble of REV2 low nibble of REV2 In the preceding example the IPMC version is 2 0 0 m The CPLD version is read as lower nibble of CPLD version byte In the example the CPLD version is 2 B 2 Get RTM Status Command You can use the Get RTM Status command to detect the presence of a rear transition module RTM in the system Command NetFn Opcode Reference Get_RTM_Status Ox2E OEM 0x88 CPLD Specification TABLEB 2 Get RTM Status Command Data Bytes Type Byte Data Field Request data Bytel 00 Byte2 00 Byte3 2A Response data Bytel Completion code 00 OK C1 Command not supported CC Invalid data in request Byte2 00 Byte3 00 Byte4 2A Byte5 RTM presence Bits 7 to1 0 Bits 0 RTM presence 0 RTM not detected 1 RTM detected Appendix B Sun OEM IPMI Commands B 3 Example Terminal Mode B8 00 88 00 00 2A Request BC 00 88 00 00 00 2A 01 Response B 3 B 4 Solaris OS Graceful Shutd
63. er Service Processor MPC885 The Netra CP3260 blade server includes a MPC885 service processor subsystem used for Logical Domains LDoms configuration and Hypervisor interaction host system reset and boot support The Netra CP3260 blade server uses the MPC885 to run the vBSC firmware on VxWorks The following I O interfaces provided by MPC885 are used by the Netra CP3260 blade server m 100BASE T Connects to the Base interface switch m I2C Provides interface to I2C devices required by vBSC DIMM SPDs TOD NVRAM m UART1 Provides serial interface to IPMC Payload m UART 2 Provides console interface Field Programmable Gate Array The Field Programmable Gate Array FPGA serves as a gateway between the UltraSPARC T2 and the MPC subsystem and provides support for the IPMC The FPGA provides the following functionality m 32 Kbyte SRAM integrated used as Mailbox Data Channel and scratch pad for POST The SRAM is accessible both from the UltraSPARC T2 processor through the SSI interface and from H8 through the H8 bus Netra CP3260 Blade Server User s Guide April 2009 m H8 interface providing IPMC support m MPC interface m SSI Interface UltraSPARC T2 m BUS Interface and Arbiter Round Robin m Bus Clock Control m Power sequence control of DC DCs on board m Interrupts m Data Channel Fast Mailbox Control The FPGA configuration is performed after an FPGA reset when the configuration is downloaded
64. er configuration file nxge conf to initialize interfaces to either Zone 2 backplane or Zone 3 ARTM The driver uses phy type property to configure each interface On the Netra CP3260 blade server the phy type can have the following values m phy type gsd for interface to operate at 1Gbps with Zone 2 connectivity m phy type xgsd for interface to operate at 10Gbps with Zone 2 connectivity m phy type xgf for interface to operate at 10Gbps with Zone 3 connectivity For default installations the nxge driver is in the platform sun4v kernel drv directory The following examples show how to configure zone link speed in the nxge driver file for Zone 2 and Zone 3 connectivity when using the Solaris OS CODE EXAMPLE 3 1 Zone 2 Connectivity Both nxge Ports name SUNW niusl parent niu 80 unit address 0 phy type gsd name SUNW niusl parent niu 80 unit address 1 phy type xgsd Note The phy type gsd sets the interface at 1Gbps and the phy type xgsd sets the interface at 10Gbps CODE EXAMPLE 3 2 Zone 3 Connectivity Both nxge Ports name SUNW niusl parent niu 80 unit address 0 phy type xgf name SUNW niusl parent niu 80 unit address 1 phy type xgf Netra CP3260 Blade Server User s Guide April 2009 CODE EXAMPLE 3 3 Zone 2 and Zone 3 Connectivity First NIU Port nxge0 to Zone 2 at 10Gbps Speed and the Second NIU Port nxge1
65. erial 0 ca0000 pci 0 pci 0d pci 0 pci 0 pci a pci 0 pci 0 pci 9 pci 0 pci 0 pci s pci 0 pci 0 pci 2 pci 0 pci 0 pci l pci 0 pci 0 pci a pci 0 pci 0 pci 0 pci a pci 0 usbe 4 show devs Command Output pci 0 pci 0 pci a pci 0 usbe 4 2 pci 0 pci 0 pci a pci 0 usbe4 1 pci 0 pci 0 pci a pci 0 usbe4 pci 0 pci 0 pci a pci 0 usbe 4 2 storage 2 L 2 storage 2 disk pci 0 pci 0 pci 9 network 0 pci 0 pci 0 pci 8 scsi d pci 0 pci 0 pci 9 network 0 1 pci 0 pci 0 pci 8 scsi 0 disk pci 0 pci 0 pci 8 scsi 0 tape pci 0 pci 0 pci 2 network 0 1 pci 0 pci 0 pci 2 network 0d pci 0 pci 0 pci pci 0 pci 0 pci l niu 80 network 1 niu 80 networke 0 virtual devices 1 virtual devices 1 virtual devices 1 virtual devices 1 virtual devices 1 virtual devices 1 1 network 0 1 network 0 00 rtc 5 00 console 1 00 flashprom 0 00 random number generator e 00 ncp 6 00 n2cp 7 Netra CP3260 Blade Server User s Guide April 2009 4 3 3 4 CODE EXAMPLE 4 3 show devs Command Output virtual devices 100 channel devices 200 virtual devices 100 channel devices 200 virtual channel client 1 virtual devices 100 channel devices 200 virtual channel 0 virtual devices 100 channel devices 200 virtual channel client 2 virtual devices 100 channel devices 200 virtual channel 3 openprom client services packages obp tftp packages kbd translator packages SUNW asr packages dropins packag
66. ernel drv directory The following examples show the proper entries for each switch configuration These entries are additional to any other configuration parameters set in the nxge conf file Example 1 If Sun Netra CP3240 10 GbE switches are used in Slot 7 and Slot 8 the nxge conf file should have the following entries name SUNW niusl parent niu 80 unit address 0 phy type xgsd name SUNW niusl parent niu 80 unit address 1 phy type xgsd Example 2 If Sun Netra CP3140 1 GbE switches are used in Slot 7 and Slot 8 the nxge conf file should have the following entries name SUNW niusl parent niu 80 unit address 1 phy type gsd name SUNW niusl parent niu 80 unit address 0 phy type gsd Example 3 If a Sun Netra CP3140 1 GbE switch is used in Slot 7 and a Sun Netra CP3240 10 GbE switch in Slot 8 the nxge conf file should have the following entries name SUNW niusl parent niu 80 unit address 0 phy type gsd name SUNW niusl parent niu 80 unit address 1 phy type xgsd Example 4 If a Sun Netra CP3240 10 GbE switch is used in Slot 7 and a Sun Netra CP3140 1 GbE switch is used in Slot 8 the nxge conf file should have the following entries name SUNW niusl parent niu 80 unit address 0 phy type xgsd name SUNW niusl parent niu 80 unit address 1 phy type gsd 3 6 Downloading a
67. es centrales pour proceder a des mises a jour de produits est rigoureusement interdite LA DOCUMENTATION EST FOURNIE EN L ETAT ET TOUTES AUTRES CONDITIONS DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES DANS LA MESURE AUTORISEE PARLA LOI APPLICABLE Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE A L APTITUDE A UNE UTILISATION PARTICULIERE OU A L ABSENCE DE CONTREFACON 5 cx Ca Adobe PostScript Contents Preface xiii Introduction 1 1 1 1 1 2 1 3 1 4 1 5 1 6 1 7 Overview 1 1 Features 1 2 System Configurations 1 8 Advanced Rear Transition Modules 1 11 Hot Swap Support 1 14 System Requirements and Options 1 14 1 6 1 Hardware Requirements and Options 1 14 1 6 2 Software Requirements 1 15 Technical Support and Warranty 1 15 1 7 1 Part Number Serial Number and Revision Number Identification 1 16 Hardware Installation 2 1 2 1 2 2 2 3 Equipment and Operator Safety 2 1 Materials and Tools Required 2 3 Preparing for the Installation 2 3 2 3 1 Checking Power Thermal Environmental and Space Requirements 2 3 2 4 2 5 2 6 2 7 2 8 2 9 Installation Procedure Summary 2 4 Configuring On Board Hardware 2 5 2 5 1 Adding or Replacing FB DIMM Memory Modules 2 5 2 5 1 1 To Remove FB DIMM Memory Modules 2 10 2 5 1 2 To Install FB DIMM Memory Modules 2 11 2 5 2 Adding or Replacing Compact Flash Card 2 13 2 5 3 Adding or Replacing TOD Clock Batte
68. es terminal emulator packages disk label packages deblocker packages SUNW builtin drivers Using watch net and watch net all Commands to Check the Network The watch net diagnostics test monitors Ethernet packets on the primary network interface The watch net all diagnostics test monitors Ethernet packets on the primary network interface and on any additional network interfaces connected to the system board Good packets received by the system are indicated by a period Errors such as the framing error and the cyclic redundancy check CRC error are indicated with an X and an associated error description To start the watch net diagnostic test type the watch net command at the ok prompt 0 ok watch net 1000 Mbps full duplex Link up Looking for Ethernet Packets is a Good Packet X is a Bad Packet Type any Key to StOD i esses Ve ge de Re ig ovens To start the watch net all diagnostic test type watch net all at the ok prompt 0 ok watch net all pci80 pciQ0 pciG1 networkQ0 1 1000 Mbps full duplex Link up Looking for Ethernet Packets is a Good Packet X is a Bad Packet Type any key to stop Chapter 4 Firmware and Blade Server Management 4 11 4 3 4 TABLE 4 2 pci80 pciQ0 pciG1 networkGe0 1000 Mbps full duplex Link up Looking for Ethernet Packets is a Good Packet X is a Bad Packet Type any key to stop OpenBoo
69. ext line Download and install additional patches Refer to the Netra CP3260 Blade Server Product Notes 820 0455 for the latest information on the patches available for the Sun Netra CP3260 blade server The document can be downloaded from the following web site http docs sun com app docs prod netra brds Follow the instructions in the Netra CP3260 Blade Server Product Notes 820 0455 for downloading and applying patches to a diskless clients boot server After the patches are installed follow the procedure in Section 3 3 2 Adding a Diskless Client on page 3 5 Adding a Diskless Client 1 Prepare a patched boot server for the diskless clients Follow the steps in Section 3 3 1 Creating a Boot Server for Diskless Clients on page 3 4 to create a boot server for the diskless clients Log in to the patched boot server as superuser Collect the following information for the diskless client you are adding Client s IP address Client s Ethernet address Client s host name Chapter 3 Software Installation 3 5 4 Change directories to the usr sadm bin directory cd usr sadm bin 5 Set up the diskless clients For each diskless client type the following command as superuser smdiskless add i ip address e ethernet address n host name N x os sparc sun4v Solaris 5 x root export root host name x swap export swap lost name x swapsize swap size x tz time zone x loc
70. ffer AMB which converts this serial interface into the standard memory interface used by DDR2 memory The UltraSPARC T2 provides 4 FB DIMM memory controllers with each controller supporting two memory channels Memory Capacity The Netra CP3260 provides up to 8 FB DIMM slots two per memory controller This limits the maximum memory to 16 Gbytes if 2 Gbyte DIMMs are used or 8 GBytes if 1 Gbyte DIMMs are used Memory Speed The target speed for FB DIMMS is 667 MHz DDR2 RAMs which results in a channeled speed of 4 0 Gbps I O Subsystem The Netra CP3260 I O subsystem comprises of the following main features m PCI Express switch interface m Dual 10 100 1000BASE T Base interface ports to Zone 2 connector m Dual 1 10 Gbit XAUI ports for Fabric interface to Zone 2 connector Common RIM interface via Zone 3 connector for rear access m Type II Compact Flash socket m Front panel I O Chapter 5 Hardware Functional Descriptions 5 9 5 2 3 1 5 2 3 2 5 2 3 3 5 2 3 4 5 10 PCI Express Switch The Netra CP3260 uses the PLX PCI switch that encompasses a 32 lane PCI Express switch with six configurable ports x1 x2 x4 x8 x16 Base Interface The Netra CP3260 provides dual redundant 10 100 1000BASE T Ethernet links to connect to the midplane Base interface channels A x4 lane port from the PCI Express switch connects to a dual gigabit controller Fabric Interface The dual XAUI ports from UltraSPARC T2 processor are m
71. gure the swap space to another storage device Chapter2 Hardware Installation 2 13 FIGURE 2 4 Compact Flash Card Location Figure Legend 1 Compact Flash connector 2 0 9 Adding or Replacing TOD Clock Battery FIGURE 2 5 shows the location of the TOD clock battery and holder The optional TOD battery must be type CR1632 with a minimum of 4ma abnormal charging current rating for example a Renata CR1632 2 14 Netra CP3260 Blade Server User s Guide April 2009 Caution Risk of explosion if the battery is replaced by an incorrect type Dispose of batteries properly in accordance with manufacturer s instructions and local regulations To install the battery perform the following steps 1 Remove the FB DIMM closest to the front panel To do this see Section 2 5 1 1 To Remove FB DIMM Memory Modules on page 2 10 2 Optional Remove the old battery if necessary 3 Slide the new battery into the holder with the side labeled facing up 4 Re install the FB DIMM To do this see Section 2 5 1 2 To Install FB DIMM Memory Modules on page 2 11 Chapter 2 Hardware Installation 2 15 FIGURE 2 5 TOD Battery Location 2 16 Netra CP3260 Blade Server User s Guide April 2009 Chor E ONG H HE RAR 4 D ee IP ERR A amp tm E i c Jy EN i C He wes p f ATT 3 fie E Om Rl Ii T Cr reg LS LS E Figure Legend 1 TOD battery location
72. hapter 4 Firmware and Blade Server Management 4 27 Be aware of the following possible issues when multiplexing zones a MUX and payload driver configuration could go out of sync requiring a system administrator to make sure they are set to the same configuration m Midplane FRUID record could be corrupted when updating with the MUX configuration m Third party shelf management software might block updates to the midplane FRUID 4 28 Netra CP3260 Blade Server User s Guide April 2009 CHAPTER 5 Hardware Functional Descriptions This chapter contains the following sections m Section 5 1 Hardware Architecture on page 5 1 m Section 5 2 Hardware Modules on page 5 4 5 1 Hardware Architecture The Netra CP3260 blade server is an ATCA node board based on the UltraSPARC T2 processor It is hot swappable to an ATCA midplane and supports dual 10 100 1000BASE T Ethernet interfaces as Base interface and 10 Gb XAUI or 1 Gb SERDES Ethernet interfaces as Fabric interfaces to support a redundant Dual Star topology FIGURE 5 1 is a block diagram of the Netra CP3260 blade server The blade server functions can be broken down into subsystems consisting of m CPU and memory subsystem m Service processor subsystem m I O subsystem m IPMI subsystem m Power subsystem 5 1 FIGURE 5 1 Netra CP3260 Blade Server Block Diagram PPC Ethernet D190 Vcore C Pm Pon M PPC Ethernet PHY y RMII
73. hen the auto boot OpenBoot configuration variable is set to false the system boots to the ok prompt m When the operating system is halted the system transitions to run level 0 in an orderly way Netra CP3260 Blade Server User s Guide April 2009 4 3 1 m When the operating system crashes the system reverts to OpenBoot firmware control m During the boot process when there is a serious hardware problem that prevents the operating system from running the system reverts to OpenBoot firmware control m When a serious hardware problem develops while the system is running the operating system transitions smoothly to run level 0 m When the OS is deliberately placed under the OpenBoot firmware control in order to execute firmware based commands Getting to the ok Prompt There are different ways of reaching the ok prompt The methods are not equally desirable See TABLE 4 1 for details TABLE4 1 Ways of Accessing the ok Prompt Access Method What to Do Graceful shutdown of From a shell or command tool window issue an appropriate the Solaris OS command for example the shutdown or init command as described in Solaris system administration documentation Manual system reset Setting the OBP auto boot variable to false causes the system to stop at the ok prompt the next time the blade server is reset Caution Obtaining the ok prompt suspends all application and operating system software After you issue f
74. in 3 Pin 4 Pin 5 Pin 6 A Logic GND Logic GND Logic GND Shelf GND Shelf GND Shelf GND B Logic GND Logic GND Logic GND 3 3V_STBY_RTM 3 3V_STBY_RTM 3 3V_STBY_RTM C DC RIMCONN SCL DC RTMCONN SCL DC RTMCONN SCL I2C_RTMCONN_SDA ID2C RTMCONN SDA I2C RTMCONN SDA D 12V_RTM 12V_RTM 12V_RTM 12V_RTM 12V_RTM 12V_RTM E RIM_PS1_L RTM_PS1_L RTM_PS1_L N C N C N C For information about multiplexing to Zone 2 or 3 see Section 4 8 Multiplexing to Zones 2 and 3 on page 4 27 A 4 6 TOD Clock Battery Holder FIGURE A 9 shows the location of the TOD clock battery and holder The TOD battery must be type CR1632 with a minimum of 4ma abnormal charging current rating for example a Renata CR1632 Caution Risk of explosion if battery is replaced by an incorrect type Dispose of batteries properly in accordance with manufacturer s instructions and local regulations To install the battery slide the battery into the holder with the side labeled facing up Appendix A Physical Characteristics A 13 FIGURE A 9 TOD Battery Location eer ae Ger gt H c CH 3 z REL rem e E 53E dq E s H ud lop 2 TOD clock battery location with FB DIMM removed 1 TOD battery location Figure Legend Netra CP3260 Blade Server User s Guide April 2009 A 14 APPENDIX B Sun OEM IPMI Commands This appendix contains the following sec
75. ion 1 3 System Configurations on page 1 8 m Section 1 5 Hot Swap Support on page 1 14 m Section 1 6 System Requirements and Options on page 1 14 m Section 1 7 Technical Support and Warranty on page 1 15 1 1 Overview The Netra CP3260 blade server FIGURE 1 1 and FIGURE 1 2 is a high performance single board computer based on one UltraSPARC T2 multicore processor and designed for high availability in a switched network computing environment This blade server is compliant with Advanced Telecommunications Computing Architecture AdvancedTCA ATCA specifications PICMG 3 0 and PICMG 3 1 The PICMG PCI Industrial Computer Manufacturers Group standards committee has developed the new ATCA or PICMG 3 x standard to address the issues posed by previous standards based on cPCI and cPSB PICMG 2 x The PICMG 3 x specification brought the following changes to the existing PICMG 2 x family of products m Larger board space 8U high compared to 6U for cPCI which allows more features and processing power m On board power supplies deriving local power from redundant 48V power from the midplane rather than separate power supplies m 6 HP slot width allowing greater component height m Advanced Mezzanine card AMC support and options m Elimination of PCI connectivity between the blade servers in the system and reallocation of connectivity to serial interconnects eliminating single points of failure a Manda
76. irmware commands and run firmware based tests from the ok prompt the system might not be able to resume where it left off If possible back up system data before starting to access the ok prompt Also exit or stop all applications and warn users of the impending loss of service For information about the appropriate backup and shutdown procedures see Solaris system administration documentation Chapter 4 Firmware and Blade Server Management 4 5 4 3 2 4 3 3 4 6 Auto Boot Options The system firmware stores a configuration variable called auto boot which controls whether the firmware will automatically boot the operating system after each reset The default setting for Sun platforms is true Normally if a system fails power on diagnostics auto boot is ignored and the system does not boot unless an operator boots the system manually An automatic boot is generally not acceptable for booting a system in a degraded state Therefore the Netra CP3260 server OpenBoot firmware provides a second setting auto boot on error This setting controls whether the system will attempt a degraded boot when a subsystem failure is detected Both the auto boot and auto boot on error switches must be set to true to enable an automatic degraded boot To set the switches type 0 ok setenv auto boot true 0 ok setenv auto boot on error true Note The default setting for auto boot on error is false The system will not attempt a degr
77. ith the failed module Another non failing logical bank must be present in the system for the system to attempt a degraded boot Note If POST or OpenBoot firmware detects a nonfatal error associated with the normal boot device the OpenBoot firmware automatically unconfigures the failed device and tries the next in line boot device as specified by the boot device configuration variable m Ifa fatal error is detected by POST or OpenBoot firmware the system does not boot regardless of the settings of auto boot or auto boot on error Fatal nonrecoverable errors include the following Chapter 4 Firmware and Blade Server Management 4 15 Any CPU failed All logical memory banks failed a Flash RAM cyclical redundancy check CRC failure a Critical field replaceable unit FRU PROM configuration data failure a Critical application specific integrated circuit ASIC failure 4 5 4 16 Automatic System Recovery Automatic system recovery ASR consists of self test features and an autoconfiguration capability to detect failed hardware components and unconfigure them By enabling ASR the server is able to resume operating after certain nonfatal hardware faults or failures have occurred If a component is monitored by ASR and the server is capable of operating without it the server automatically reboots if that component develops a fault or fails This capability prevents a faulty hardware component from stopping
78. lf Enclosure 1 9 Netra CP3260 Blade Server Midplane and Netra CP32x0 ARTM 1 12 Netra CP3260 Blade Server Barcode Labeling 1 17 FB DIMM Memory Locations 2 6 Removing an FB DIMM Memory Module 2 11 Installing a FB DIMM Memory Module 2 12 Compact Flash Card Location 2 14 TOD Battery Location 2 16 Installing a Netra CP32X0 ARTM 2 19 Installing Blade Server Into Chassis Slot 2 22 Netra CP3260 Blade Server Latches and Locking Screws 2 23 Hot Swap Latch and Hot Swap LED 2 26 Netra CP3260 Blade Server Block Diagram 5 2 UItraSPARC T2 Multicore Processor Block Diagram 5 5 Netra CP3260 Blade Server Layout A 2 Ethernet RJ 45 Connector A 4 Dual USB Connector A 5 Front Panel Serial Port Diagram A 6 FIGURE A 5 FIGURE A 6 FIGURE A 7 FIGURE A 8 FIGURE A 9 Power Distribution Connector Zone 1 P10 A 7 Zone 2 Connectors A 9 Zone 3 Signal Connectors A 10 Zone 3 Power Connector A 12 TOD Battery Location A 14 x Netra CP3260 Blade Server User s Guide April 2009 TABLE 1 1 TABLE 1 2 TABLE 2 1 TABLE 3 1 TABLE 4 1 TABLE 4 2 TABLE 4 3 TABLE 4 4 TABLE 5 1 TABLE 5 2 TABLE A 1 TABLE A 2 TABLE A 3 TABLE A 4 TABLE A 5 TABLE A 6 TABLE A 7 TABLE A 8 TABLE A 9 TABLE A 10 Tables Netra CP3260 Blade Server Feature Summary 1 2 I O Configurations 1 13 Ethernet Device Names 2 24 Local Network Information 3 7 Ways of Accessing the ok Prompt 4 5 OpenBoot Configuration Variables 4 12 Network Device Aliases 4 18 PICL Fru
79. m logo output device screen columns screen rows scsi initiator id security badlogins security mode boot none sync true false keyboard virtual console 0 n true false protocol key value variable name true false true false screen virtual console 0 n 0 n 0 15 variable name none full command boot false virtual console 16384 true none none false false virtual console 80 34 7 none none Specifies recovery action after an error reset Default is boot none No recovery action boot System attempts to boot sync Firmware attempts to execute a Solaris sync callback routine If true include name fields for plug in device FCodes Power on input device Address If true network drivers use their own MAC address not the server MAC address Arguments to be used by the PROM for network booting Defaults to an empty string network boot arguments can be used to specify the boot protocol RARP DHCP to be used and a range of system knowledge to be used in the process For further information see the eeprom 1M man page or your Solaris Reference Manual Command script to execute if use nvramrc is true If true use custom OEM banner If true use custom OEM logo otherwise use Sun logo Power on output device Sets number of columns on screen Sets number of rows on screen SCSI ID of th
80. mmand Output 0 ok probe ide Device 0 Primary Master ATA Model FUJITSU MHV2040BH Device 1 Primary Slave ATA Model Device 2 Secondary Master Not Present Device 3 Secondary Slave Not Present show devs Command The show devs command lists the hardware device paths for each device in the firmware device tree The following shows some sample output CODE EXAMPLE 4 3 show devs Command Output 0 ok show devs pci performance counters 0 ebus c0 pci d niu880 cpuQ3f cpu 3e cpu 3d cpu 3c cpu 3b cpu 3a cpu 39 cpu 38 Netra CP3260 Blade Server User s Guide April 2009 CODE EXAMPLE 4 3 cpu 3 cpu 3 cpu 3 cpu 3 cpu 3 cpu 3 cpu 3 cpu 3 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu 2 cpu l cpuQ1 cpu l cpu l cpu l cpuQ1 cpu l cpu 1 cpu 1 cpu 1 cpu 1 cpu 1 cpu 1 cpu 1 cpu 1 cpu 1 cpu f cpu e cpu d cpu c cpu b cpu a cpu 9 7 6 5 4 3 2 1 0 f e d c b a 9 8 7 6 5 4 3 2 1 0 f OFPNWRKRUDNINWAUVUOAFAAO show devs Command Output Chapter 4 Firmware and Blade Server Management 4 9 CODE EXAMPLE 4 3 cpu s cpu 7 cpu 6 cpu 5 cpu 4 cpu 3 cpu 2 cpuQ1 cpu 0 virtual devices 100 virtual memory memory m0 8000000 aliases options openprom chosen packages ebus c0 s
81. n Board Hardware on page 2 5 before installing the blade server The procedure to set up and configure a Netra CP3260 blade server in a system includes the following actions 1 Configure the on board physical hardware For example add memory or optional Compact Flash Configure and install a rear transition module RTM if necessary Physically install the Netra CP3260 blade server into the chassis Connect the nodes to a local network Alternatively the blade server can be run as a standalone system without a network connection Install the operating system and patches as necessary See Section 3 1 Operating Systems and Patches on page 3 1 Download firmware updates if needed See Section 3 4 Firmware Updates on page 3 8 Configure the operating system for 1 GbE or 10 GbE Ethernet switches See Section 3 5 Configuring for 1 GbE or 10 GbE Switches on page 3 8 2 4 Netra CP3260 Blade Server User s Guide April 2009 249 2 5 1 Configuring On Board Hardware This section provides the procedures for adding or replacing the on board hardware components such as memory modules a Compact Flash card and the time of day TOD battery Read and perform the procedures as necessary before installing the Netra CP3260 blade server into the chassis Adding or Replacing FB DIMM Memory Modules The Netra CP3260 blade server supports a total of 8 FB DIMMs and a maximum memory capacity
82. nd Installing SunVTS Software Sun Validation Test Suite GunVTS software is a comprehensive suite that tests and validates the Sun Netra CP3260 blade server by verifying the configuration and function of most hardware controllers and devices on the blade server Chapter 3 Software Installation 3 9 SunVTS software is used to validate a system during development production inspection troubleshooting periodic maintenance and system or subsystem stressing SunVTS software can be tailored to run on machines ranging from desktops to servers with modifiable test instances and processor affinity features You can perform high level system testing by using the appropriate version of SunVTS software For detailed information on SunVTS support documentation and downloads refer to the following web site http www sun com oem products vts You will be prompted for your Sun Online Account name and password For the Sun Netra CP3260 blade server use the SunVTS6 4ps2 SPARC or newer Toolkit and Documentation Ensure that the SunVTS software version is compatible with the Solaris OS being used You can find information about the SunVTS software version installed on your system by viewing the content of the version file cat opt SUNWvts bin version Note For security reasons only a superuser is permitted to run SunVTS software Installation and starting instructions are included with the software when it is downlo
83. nects directly to 8 DIMMS one DIMM per channel providing for a total of 8 DIMM slots See Section 5 2 2 Memory Subsystem on page 5 9 for details on the Netra CP3260 memory design I O Interface The UltraSPARC T2 provides the following I O interfaces m x8 PCI Express interface m Two 10 Gbps Ethernet XAUI ports PCI Express The UltraSPARC T2 provides a PCI Express Unit PEU that implements the root complex behavior of the PCI Express Base specification 1 0A The PEU supports x1 x2 x4 and x8 configuration at the data rate of 2 5Bb s in each direction The PEU also supports the lane reversal feature thus easing blade server routing restrictions XAUI Interface The XAUI interface is a high speed point to point serial interface with four differential pairs for transmit TX and four differential pairs for receive RX operating at 3 125 Gbps Netra CP3260 Blade Server User s Guide April 2009 0 2 2 5 2 2 1 5 2 2 2 5 2 3 The XAUI interface is the Fabric interface to the ATCA midplane The XAUI interface is routed to the Zone 2 connector on the midplane see Section A 4 Connectors and Pinout on page A 3 for details on board connector pinout assignments Memory Subsystem The heart of the Memory subsystem is the FB DIMM memory Data from the FB DIMM memory controller is brought out on a high speed serial bus that connects to a single memory DIMM per channel The FB DIMM provides an Advanced Memory Bu
84. of 16 Gbytes using eight 2 Gbyte DIMMs FIGURE 2 1 shows the location of the FB DIMMs The Netra CP3260 blade server accommodates the following m Fight standard FB DIMMs buffered and registered m 1 Gbyte or 2 Gbyte FB DIMM modules Note You cannot mix 1 Gbyte and 2 Gbyte FB DIMMs All eight DIMMs must be the same density that is all 1 Gbyte or all 2 Gbyte FB DIMMs The Netra CP3260 blade server supports FB DIMM memory modules that have the following characteristics m Each DIMM has a 72 bit wide data bus 64 8 ECC and up to 14 address bits m Memory controller supports 128 bit data plus 9 bit error correcting code ECC m Maximum of 16 Gbytes eight 2 Gbyte FB DIMMs or 8 Gbyte eight 1 Gbyte FB DIMMs m FB DIMM 1 55 volts or FB DIMM 1 8 volts For information about sensors refer to the Netra CT 900 Software Developer s Guide 819 1175 For additional information see Section 5 2 2 Memory Subsystem on page 5 9 FIGURE 2 1 shows the location of the DIMMs Chapter 2 Hardware Installation 2 5 FIGURE 2 1 FB DIMM Memory Locations 2 6 Netra CP3260 Blade Server User s Guide April 2009 Chapter 2 Hardware Installation 2 7 HH zu ni nn nmm GRE eee F 2 a EL EL Figure Legend A O N FBOB FBOA FB1B FB1A DIMM Pair 0 DIMM Pair 1 on o oco FB2A FB2B FB3A FB3B DIMM Pair 2 DIMM Pair 3 Chapter 2 Hardware Installation 2
85. of the event message data according to the class of the event type for the sensor threshold discrete or OEM Refer to the IPMI specification Event Data 2 Represents the remainder of the event message data according to the class of the event type for the sensor threshold discrete or OEM Refer to the IPMI specification Event Data 3 Represents the remainder of the event message data according to the class of the event type for the sensor threshold discrete or OEM Refer to the IPMI specification Reserved for future use Write zero B 6 Netra CP3260 Blade Server User s Guide April 2009 TABLE B 3 Send Sensor State Command Data Bytes Continued Type Byte Data Field Byte12 Reserved for future use Write zero Response data Bytel Completion code 00 OK C1 Command not supported CC Invalid data in request Refer to IPMI specification for more completion codes Byte2 00 Byte3 00 Byte4 2A Appendix B Sun OEM IPMI Commands B 7 B 8 Netra CP3260 Blade Server User s Guide April 2009 Index A Advanced Rear Transition Module ARTM 1 11 Advanced Telecommunications Computing Architecture ATCA 1 1 ARTM 1 11 2 18 5 4 5 10 installing 2 19 ATCA AdvancedTCA 1 1 ATCA midplane 2 20 auto boot OpenBoot variable 4 6 automatic system recovery ASR disabling 4 17 enabling 4 17 obtaining recovery information 4 28 B base interface 5 3 5 9 5 10 battery TOD clock
86. ons 5 3 5 1 6 ARTM Support ARTM support is routed to the ARTM through the Zone 3 connectors Typical ARTM support includes m One RS 232 serial console port m One 10 100 100BASE T Ethernet management port Because Netra CP32x0 Advanced RTMs ARTM provide additional functionality the following interfaces are also routed to the ARTM m A x8 PCI E interface m Dual 10 100 1000 SERDES Ethernet channels from Intel NIC Power for the ARTMs is provided through the Zone 3 RTM power connector 9 2 Du 5 4 Hardware Modules This section provides a brief description of the Netra CP3260 hardware modules UItraSPARC T2 Processor The UltraSPARC T2 multicore processor is the basis of the Netra CP3260 blade server The UltraSPARC T2 processor is based on chip multithreading CMT technology that is optimized for highly threaded transactional processing The UltraSPARC T2 processor improves throughput while using less power and dissipating less heat than conventional processor designs It is high performance highly integrated processor that implements the 64 bit SPARC V9 architecture On the Netra CP3260 blade server the UltraSPARC T2 processor operates at 1 2 GHz and contains 16 Kbytes of instruction cache per core and 8 Kbytes of data cache per core FIGURE 5 2 Depending on the model purchased the processor has six or eight UltraSPARC cores Each core equates to a 64 bit execution pipeline capable of running eight threads Th
87. operation of the entire system or causing the system to fail repeatedly If a fault is detected during the power on sequence the faulty component is disabled If the system remains capable of functioning the boot sequence continues To support this degraded boot capability the OpenBoot firmware uses the 1275 client interface by means of the device tree to mark a device as either failed or disabled creating an appropriate status property in the device tree node The Solaris OS does not activate a driver for any subsystem marked in this way As long as a failed component is electrically dormant not causing random bus errors or signal noise for example the system reboots automatically and resumes operation while a service call is made Once a failed or disabled device is replaced with a new one the OpenBoot firmware automatically modifies the status of the device upon reboot Note ASR is not enabled until you activate it see Section 4 5 1 1 To Enable Automatic System Recovery on page 4 17 Netra CP3260 Blade Server User s Guide April 2009 4 5 1 4 5 1 1 4 5 1 2 Enabling and Disabling Automatic System Recovery The automatic system recovery ASR feature is not activated until you enable it Enabling ASR requires changing configuration variables in OpenBoot To Enable Automatic System Recovery 1 At the ok prompt type ok setenv auto boot true ok setenv auto boot on error true 2 To
88. or pin assignments TABLE A 4 Power Distribution Connector Pin Assignments Pin Number Name Description 1 Reserved Reserved 2 Reserved Reserved 3 Reserved Reserved 4 Reserved Reserved 5 HAO HAO Hardware Address Bit 0 6 HA1 HA1 Hardware Address Bit 1 7 HA2 HA2 Hardware Address Bit 2 8 HA3 HA3 Hardware Address Bit 3 9 HA4 HA4 Hardware Address Bit 4 10 HA5 HA5 Hardware Address Bit 5 11 HA6 HA6 Hardware Address Bit 6 12 HA7 P HA7 P Hardware Address Bit 7 Odd Parity Bit 13 SCL_A IPMB Clock Port A 14 SDA_A IPMB Data Port A 15 SCL_B IPMB Clock Port B 16 SDA_B IPMB Data Port B Appendix A Physical Characteristics A 7 A 4 4 TABLE A 4 Power Distribution Connector Pin Assignments Continued Pin Number Name Description 17 Unused 18 Unused 19 Unused 20 Unused 21 Unused 22 Unused 23 Unused 24 Unused 25 SHELF_GND Shelf Ground 26 LOGIC_GND Logic Ground 27 ENABLE_B Enable B 28 VRTN_A Voltage Return A 29 VRTN B Voltage Return B 30 EARLY A 48V Early A 31 EARLY B 48V Early B 32 ENABLE A Enable A 33 48V A 48V A 34 48V B 48V B Data Transport Connector Zone 2 The data transport connector consists of one 120 pin HM Zd connector labeled P23 with 40 differential pairs This is called the Zone 2 connector The Zone 2 connector provides the following signals m Two 1000BASE T Ethernet Base interface channels m Two 1 Gb SERDES on the Fabric extended interface m 10 Gb XAUI
89. ork 6 0000055a network network 6 00000582 pci pciex 6 000005aa network network 6 000005d1 network network 6f 000005f9 pci pciex 6 00000621 pci pciex 6 00000647 network network 6 0000066e network network 6 00000696 pci pciex 6 000006be pci pci 6 000006e4 usb ohci 6 0000070e usb ohci 6 00000736 usb ehci 6 0000075e storage obp device 6 00000786 disk block 6 000007b0 ebus ebus 6 000007ca serial serial 6 000007db pci performance counters obp device 6f 000007ea niu sun4v 6 000007 5 Chapter 4 Firmware and Blade Server Management 4 21 4 22 CODE EXAMPLE 4 4 network network os io console 6 0000 pseudo devctl 6f 0000 zconsnex devctl network network prtpicl Command Output 6 00000804 6 00000814 0824 0843 6 0000084c obp picl 6 0000001 packages packages 6 00000034 SUNW builtin drivers SUNW builtin drivers 6 0000003a deblocker deblocker 6 00000040 disk label disk label 6 00000047 terminal emulator terminal emulator 6 0000004d dropins dropins 6 00000054 SUNW asr SUNW asr 6 0000005b kbd translator kbd translator 6 00000061 obp tftp obp tftp 6 00000067 ufs file system ufs file system 6 0000006d chosen chosen 6 00000074 openprom openprom 6 00000081 client services client services 6 0000008b options options 6 00000091 aliases aliases 6f000000b
90. own Commands Users can change the graceful shutdown time out of Solaris OS with an IPMI command executed from ShMM or a user application Note that this setting is not persistent the Solaris FSM will overwrite it after every Solaris OS boot SUNWctfsm is the Solaris FSM for ATCA graceful shutdown and reboot For installation and requirements refer the README file included in the release package and the Sun Netra CP3260 Blade Server Product Notes 820 0455 The commands and parameters are described in the following table TABLE1 Solaris OS Graceful Shutdown Parameters in etc fsmd conf Parameter Description and Parameters GS_SHUTDOWN_TIMEOUT Sets the payload graceful shutdown timeout in increments of 100 milliseconds The legal value range is from 0 to Oxffff 0 second to 6553 seconds The default value is 1200 which is 120 seconds This 1200 will overwrite the default value 300 in IPMC Solaris might panic during graceful shutdown if this parameter value is too short Conversely if this value is too long the shutdown could take too long See Note that follows this table GR_COMPLETION_NOTICE Determines if Solaris FSM will send a reboot succeed notice to IPMC The legal values are yes and no The default value is no IPMC might or might not support use it GR_TIMER_DISARM Determines if Solaris FSM will disarm graceful reboot timer The legal values are yes and no The default value is no Set to yes only if the blade ser
91. rmatting the Optional Compact Flash Card on page 3 10 3 1 Operating Systems and Patches The Sun Netra CP3260 blade server uses the Solaris 10 8 07 OS and subsequent compatible versions with associated patches Refer to the Netra CP3260 Blade Server Product Notes 820 0455 for the Solaris OS version and the patches and patch installation procedures Theses notes are available online at http docs sun com app docs prod netra brds The Solaris OS software and patches can be downloaded from SunSolve or the Sun Download Center SDLC at http www sun com download Note For information on versions of the Solaris OS including installation see the appropriate Solaris Documentation Collection at the Sun Documentation web site at http docs sun com app docs prod solaris 3 1 Depending the blade server s hardware configuration the Sun Netra CP3260 blade server can be used as a diskless client with the Solaris OS installed on a boot server see Section 3 3 Installing as a Diskless Clients on page 3 4 The OS can also be downloaded to a boot device like the Netra CP32x0 SAS Storage ARTM Dual HD or an optional Compact Flash Ow 3 2 Configuring Payload OS NIU Driver for Multiplexing to Zones 2 and 3 The MUX configuration by system management software and payload driver configuration must be in sync for proper functioning of both 10Gbps Ethernet nxge interfaces The payload driver relies on the driv
92. rounding strap between the facility electrical input ground usually connected to the equipment chassis and facility electrical service ground m Use an antistatic wrist strap when Removing a blade server from its antistatic bag Connecting or disconnecting blade servers or peripherals The other end of the strap lead should be connected to one of the following A ground mat Grounded chassis metalwork a A facility electrical service ground m Keep blade servers in the antistatic bags until they are needed m Place circuit blade servers that are out of their antistatic bags on an antistatic mat if one is available The mat must be grounded to a facility electrical service ground Do not place blade servers on top of an antistatic bag unless the outside of the bag also has antistatic protective properties m Remove a blade server from its antistatic bag only when wearing a properly connected ground strap Netra CP3260 Blade Server User s Guide April 2009 2 2 Materials and Tools Required This section provides information on the materials and tools required to perform installation The minimum tools required to perform installation are m Phillips screwdrivers No 1 No 2 optional m Antistatic wrist strap m Terminal console See Section 1 6 1 Hardware Requirements and Options on page 1 14 for information on hardware requirements 2 3 2 3 1 Preparing for the Installation Prepare for installation by re
93. rt User s Guide http docs sun com app docs prod cp32x0 10gbee 1l en hic Documentation Support and Training Sun Function URL Documentation http www sun com documentation Support http www sun com support Training http www sun com training Third Party Web Sites Sun is not responsible for the availability of third party web sites mentioned in this document Sun does not endorse and is not responsible or liable for any content advertising products or other materials that are available on or through such sites xvi Sun Netra CP3260 Blade Server User s Guide April 2009 or resources Sun will not be responsible or liable for any actual or alleged damage or loss caused by or in connection with the use of or reliance on any such content goods or services that are available on or through such sites or resources Sun Welcomes Your Comments Sun is interested in improving its documentation and welcomes your comments and suggestions You can submit your comments by going to http www sun com hwdocs feedback Please include the title and part number of your document with your feedback Sun Netra CP3260 Blade Server User s Guide part number 820 0457 11 Preface xvii xviii Sun Netra CP3260 Blade Server User s Guide April 2009 ctheGHA PTE Introduction This chapter contains the following sections m Section 1 1 Overview on page 1 1 m Section 1 2 Features on page 1 2 m Sect
94. ry 2 14 Installing an Advanced Rear Transition Module Optional 2 18 Installing the Netra CP3260 Blade Server 2 21 Connecting External I O Cables 2 23 Hot Swapping the Netra CP3260 Blade Server 2 25 3 Software Installation 3 1 3 1 3 2 3 3 3 4 3 5 3 6 3 7 Operating Systems and Patches 3 1 Configuring Payload OS NIU Driver for Multiplexing to Zones 2and3 3 2 Installing as a Diskless Clients 3 4 3 3 1 Creating a Boot Server for Diskless Clients 3 4 3 8 2 Adding a Diskless Client 3 5 3 3 3 Determining Local Network IP Addresses and Host Names 3 7 Firmware Updates 3 8 Configuring for 1 GbE or 10 GbE Switches 3 8 Downloading and Installing SunVTS Software 3 9 Formatting the Optional Compact Flash Card 3 10 4 Firmware and Blade Server Management 4 1 4 1 4 2 System Firmware 4 2 Power On Self Test Diagnostics 4 3 4 2 1 POST Test Coverage 4 3 4 2 2 POST Diagnostic and Error Message Format 44 iv Netra CP3260 Blade Server User s Guide April 2009 4 3 4 4 4 5 4 6 4 7 4 8 OpenBoot Firmware 4 4 4 3 1 Getting to the ok Prompt 4 5 43 2 Auto BootOptions 4 6 4 83 53 OpenBoot Commands 4 6 4 3 3 1 probe scsi and probe scsi all Commands 4 7 4 3 3 2 probe ide Command 4 8 4 3 3 3 show devs Command 4 8 43 3 4 Using watch net and watch net all Commands to Check the Network 4 11 4 3 4 OpenBoot Configuration Variables 4 12 4 3 A1 Viewing and Setting OpenBoot Configuration Variables 4 14 Error
95. se sensors Voltage Monitoring The ADM1026 measures most of the blade server voltages The ADM1026 has one temperature sensor embedded in the device itself and supports two remote sensing channels that require external diodes for temperature sensing These two remote sensors are used for measuring the CPU thermal diode temperature The ADM1026 measures the voltages listed in TABLE 5 1 The ADM1206 and H8 firmware monitors these voltages When the voltages are within regulations the blade server functions with no warnings When any voltage goes out of regulation beyond approximately 7 a Critical Warning is generated and presented to the shelf manager Chapter 5 Hardware Functional Descriptions 5 15 TABLE 5 1 Voltage Sensor Thresholds ADM1026 Generated Critical Warning H8 Initiated Shutdown San Sendi approx 7 approx 10 Number Name Voltage Lower Upper Lower Upper 7 12 0V 12V 11 16V 12 84V 11V 13 2V 8 5 0V 5V 4 8V 5 2V 4 78V 5 23V 9 3 3V 3 3V 3 07V 3 53V 3V 3 6V 10 3 3V STBY 3 3V 3 07V 3 53V 2 97V 3 63V 11 3V VBAT STBY 3V 2 79V No UC no no shutdown shutdown 12 1 0V 1V 0 93V 1 07V 0 9V 1 1V 13 1 1V CPU 1 1V 1 02V 1 18V 0 99V 1 21V 14 VDD 1 1V 1 1V 1 02V 1 18V 0 99V 1 21V 15 1 5V 1 5V 1 4V 1 61V 1 35V 1 65V 16 VDD 1 8V 1 8V 1 67V 1 93V 1 62V 1 98V 17 VDD 2 5V 2 5V 2 37V 2 68V 2 36V 2 75V 18 VDD_IO 1 2V 1 5V 1 4V 1 61V 1 35V 1 65V Temperature Monitoring The ADM1026 also monitors the CPU diode temperature on the
96. ssword again after typing this command The installation process should take roughly 5 minutes per client and about 15 30 minutes for the operating environment service to install however no progress is displayed on screen while the process is running Do not cancel or kill the process until the process has successfully completed You should see messages similar to the following after a few minutes confirming that the command went through successfully the second time Login to client host as user root was successful Download of com sun admin osservermgr cli OsServerMgrCli from client host was successful 6 Boot the diskless client 3 09 Determining Local Network IP Addresses and Host Names Collect the following information to connect hosts to the local area network LAN Ask your network administrator for help if necessary You can use TABLE 3 1 to record this information This information is not needed for a standalone installation TABLE 3 1 Local Network Information Information Needed Your Information IP address and host name for each Sun Netra CP3260 client Domain name Type of name service and corresponding name server names and IP addresses for example DNS and NIS or NIS Subnet mask Chapter 3 Software Installation 3 7 TABLE 3 1 Local Network Information Information Needed Your Information Gateway router IP address NFS server names and IP addresses Web server URL
97. supports hardware flow control RTS Request To Send and CTS Clear To Send IPMB L Interface The H8S provides a local IPMI interface wired to the ARTM s MMC IPMB L The MMC IPMI interface is isolated through FET and controlled by H8 GPIO pins ATCA Hot Swap Latch Hot swap is supported by monitoring of the hot swap handle switch The handle switch goes directly to one of the GPIO pins on the H8S PE5 Payload Shutdown in Response to Hot Swap The hardware supports both non graceful shutdown and graceful shutdown of payload in response to a hot swap event In case of non graceful shutdown firmware on the IPMC disables the voltage rails before turning the blue LED on Optionally the IPMC can send a message across the serial payload interface to the PPC Netra CP3260 Blade Server User s Guide April 2009 5 2 8 5 5 2 8 6 5 2 8 7 LEDs The Netra CP3260 blade server supports three LEDs compliant with ATCA specification m Green LED Healthy status m Amber LED Fault condition m Blue LED Indicates that blade server is safe for removal when on and hot swapping in progress when blinking The LEDs are controlled by H8 Power Control The BMR HSS is able to control enable disable power rails to the payload System Monitor ADM1026 and Thresholds The Analog Devices ADM1026 is used for system monitoring functions The ADM1026 is interfaced with the IPMC and the IPMC firmware is responsible for monitoring the
98. t port When the ARTM serial port is not used console data is routed to the H8 for netconsole or to the front panel serial port provided a serial device is connected into the port Ethernet Management Port A single 10 100 1000BASE T Ethernet port is routed from the Intel 82671 Dual Gigabit Ethernet controller to rear panel RJ 45 connectors A second port goes to the front panel RJ 45 management port Front Panel I O The front panel I O consists of the following m Ethernet Management port m Serial port m Dual USB ports Ethernet Management Port A single 10 100 1000BASE T Ethernet port is routed from the Intel 82671 Dual Gigabit Ethernet controller to front panel RJ 45 connector A second management port goes to the ARTM Serial Port The serial console port from the PPC is routed to the front panel and terminated in an RJ 45 connector Chapter 5 Hardware Functional Descriptions 5 11 5 2 5 3 5 2 6 S27 5 2 7 1 5 12 Note A serial port is also provided on the Netra CP32x0 ARTM You can use either the serial port on the front panel or the serial port on Netra CP32x0 ARTM but only one of the ports should be used at one time Dual USB Ports Dual USB ports are 2 0 compliant and routed from the USB hub to the front panel Compact Flash Socket The Netra CP3260 blade server provides a Type I II compact flash socket for removal media The Compact Flash socket interface is derived from the USB to IDE controll
99. t Configuration Variables The OpenBoot configuration variables are stored in the OBP flash PROM and determine how and when OpenBoot tests are performed This section explains how to access and modify OpenBoot configuration variables For a list of important OpenBoot configuration variables see TABLE 4 2 Changes to OpenBoot configuration variables take effect at the next reboot OpenBoot Configuration Variables Variable ansi terminal auto boot auto boot on error boot command boot file boot device diag switch Possible Values Default Value Description true false true If true enable ANSI terminal emulation true false true If true boots automatically after power on or reset true false false If true boot automatically after system error variable name boot Action following a boot command variable name none File from which to boot if diag switch is false variable name disk net Device s from which to boot if diag switch is false true false false If true After a boot request boot diag file from diag device If false 4 12 Netra CP3260 Blade Server User s Guide April 2009 After a boot request boot boot file from boot device TABLE 4 2 Variable Possible Values OpenBoot Configuration Variables Default Value Description error reset recovery fcode debug input device load base local mac address network boot arguments nvramrc oem banner oe
100. t1 net2 net3 net4 net5 e1000g0 e1000g1 e1000g4 e1000g5 e1000g2 e1000g3 Base Interface Ethernet 0 Base Interface Ethernet 1 Management Ethernet Ethernet port on front panel Management Ethernet Rear Access Ethernet port on ARTM Rear Access ARTM Ethernet 0 Rear Access ARTM Ethernet 1 Netra CP3260 Blade Server User s Guide April 2009 4 7 Retrieving Device Information You can use the Solaris platform information and control library PICL framework for obtaining the state and condition of the Netra CP3260 blade server The PICL framework provides information about the system configuration that it maintains in the PICL tree Within this PICL tree is a subtree named frutree which represents the hierarchy of system field replaceable units FRUs with respect to a root node in the tree called chassis The frutree represents physical resources of the system The PICL tree is updated whenever a change occurs in a device s status TABLE 4 4 shows the frutree entries and properties that describe the condition of the Netra CP3260 blade server TABLE 4 4 PICL Frutree Entries and Description for the Netra CP3260 Board Frutree Entry Property Entry Description Example of Condition CPU location State State of the receptacle or slot connected CPU fru Condition of the blade server or ok Condition occupant CPU fru State State of the blade server or occupant configured CPU fru FRUType F
101. tFn Opcode Reference Get_Version Ox2E OEM 0x80 TABLE B 3 Send Sensor State Command Data Bytes Type Byte Data Field Request data Bytel 00 Byte2 00 Byte3 2A Appendix B Sun OEM IPMI Commands B 5 TABLE B 3 Send Sensor State Command Data Bytes Continued Type Byte Data Field Byte4 Byte5 Byte6 Byte7 Byte8 Byte9 Bytel0 Bytel1 Sensor type byte Indicates event class or type of sensor that generates the event message Refer to IPMI specification for sensor type codes Sensor optional Write to 0 Unique number representing the sensor within the management controller that generates the event message The BIOS OBP ILOM and OS does not have to know and send the sensor number for the sensors that are maintained and reported by the IPMC Reserved for future use EventDir EventType 1 bit for EventDir which indicates the event transition direction A value of 0 means assertion event a value of 1 means deassertion event 7 bits for EventType which indicates the type of threshold crossing or state transition trigger that produces the event Encode using the Event Reading Type code Refer to IPMI specification for type codes Sensor offset Tells the sensor generic or sensor specific offset as defined in the IPMI specification IPMC uses this data to define the state of the sensors Refer to the IPMI specification Event Data 1 Represents the remainder
102. the connector FIGURE 2 3 Installing a FB DIMM Memory Module Caution Do not rock the FB DIMM into place Ensure that all contacts engage at the same time You will feel or hear a click when the FBDIMM properly seats in the connector 2 12 Netra CP3260 Blade Server User s Guide April 2009 2 0 2 4 Press the top edge of the FB DIMM toward the blade server until the retainer clips click into place see in FIGURE 2 3 The small retainer clips on each side of the FB DIMM slot click into place in the notches on the sides of the FB DIMM Adding or Replacing Compact Flash Card You can install an optional Sun Compact Flash card on the Netra CP3260 blade server The Compact Flash card is not hot swappable and there is no access to the card once the blade server is installed in an ATCA chassis To install the Compact Flash card use the arrow on the card s label as a guide and insert the card into the Compact Flash connector FIGURE 2 4 Note On the Netra CP3260 the Compact Flash is an USB removable media device Therefore you must use the Solaris rmformat utility to format the device Refer to the rmformat 1 man page for more information Note Sun Compact Flash cards have a life of 2 000 000 write erase cycles Users are responsible for ensuring that the operating system and applications do not exceed this limitation If the Solaris OS is installed on the Compact Flash card Sun recommends that you reconfi
103. tions m Section B 1 Get Version Command on page B 2 m Section B 2 Get RTM Status Command on page B 3 m Section B 3 Solaris OS Graceful Shutdown Commands on page B 4 m Section B 4 Send Sensor State Command on page B 5 The commands described in this appendix are specific to Netra CP3260 blade server designed by Sun Microsystems The Internet Assigned Numbers Authority IANA number assigned to Sun Microsystems is 42 Refer to http www iana org assignments enterprise numbers for more information about IANA number assignments The netfunction NetFn used for these commands is 0x2E which is the OEM netfunction as defined in the IPMI specification For this netfunction the first three data bytes in the request packet must be this IANA number and the first three bytes in the response packet following the completion code are the IANA number For Sun ATCA node boards these three bytes are 00 00 2A B 1 B 1 B 2 Get Version Command Get Version returns the IPM controller IPMC firmware version and Standby CPLD version Bytes 8 9 and A are reserved for future use Command NetFn Opcode Reference Get_Version 0x2E OEM 0x80 TABLE B 1 Get Version Command Data Bytes Type Byte Data Field Request data Bytel 00 Byte2 00 Byte3 2A Response data Bytel Completion code 00 OK C1 Command not supported CC Invalid data in request Refer to IPMI specification for more completion codes Byt
104. tory use of Intelligent Platform Management Interface IPMI management interfaces m Flexible user I O m Power and thermal management guidelines enforced by the management infrastructure m Separation of control and data traffic by supporting the Base PICMG 3 0 and Fabric PICMG 3 1 interfaces The ATCA standard consists of the PICMG 3 0 PICMG 3 1 PICMG 3 2 and PICMG 3 3 specifications The Netra CP3260 blade server complies with m PICMG 3 0 the base specification that defines the mechanical power distribution system management data transport and regulatory guidelines m PICMG 3 1 which builds upon the PICMG 3 0 base specification and the IEEE 802 3 2003 standard 1 2 1 2 Features TABLE 1 1 provides a summary of Netra CP3260 features TABLE 1 1 Netra CP3260 Blade Server Feature Summary Feature Description CPU One UltraSPARC T2 processor 6 or 8 cores 8 threads core CPU core speed 1 2 GHz Memory Fully Buffered DDR2 based memory DIMMs FB DIMMs Eight FB DIMM slots one per channel or two per memory controller 1 Gbyte or 2 Gbyte FB DIMMs for a total memory of 16 Gbytes Service Service processor MPC885 for CPU reset boot partition and fault processor management architecture FMA Power 200 watts maximum requirement Dual redundant input 48V to derive on board power Cooling e 32 35 cfm at 55C requirement Netra CP3260 Blade Server User s Guide April 2009 TABLE 1 1 Netra C
105. tree Entries and Description for the Netra CP3260 Board 4 19 Voltage Sensor Thresholds 5 16 CPU Temperature Alarms 5 16 Ethernet Port Connector Pin Assignments A 4 USB Connector Pin Assignments A 5 Serial Port RJ 45 Connector Pinouts A 6 Power Distribution Connector Pin Assignments A 7 Zone 2 J23 Connector Pin Assignments A 9 Zone 2 J20 Connector Pin Assignments A 9 Zone 3 J31 Connector Pin Assignments A 11 Zone 3 J32 Connector Pin Assignments A 11 Zone 3 J33 Connector Pin Assignments A 12 Zone 3 Power Connector Pin Assignments A 13 xi TABLE B 1 Get Version Command Data Bytes B 2 TABLE B 2 Get RTM Status Command Data Bytes B 3 TABLE 1 Solaris OS Graceful Shutdown Parameters in etc fsmd conf B 4 TABLE B 3 Send Sensor State Command Data Bytes B 5 xii Netra CP3260 Blade Server User s Guide April 2009 Preface The Sun Netra CP3260 Blade Server User s Guide provides information about features installation configuration functional hardware components and physical properties of this blade server The Sun Netra CP3260 Blade Server User s Guide is written for system integration engineers field applications and service engineers and others involved in the integration of these blade servers into systems How This Document Is Organized Chapter 1 provides an overview of the Sun Netra CP3260 blade server Chapter 2 provides instructions on hardware installation Chapter 3 provides instructions on th
106. ubsystem Resets The UltraSPARC T2 provides a PCI Express reset out on the PEX RESET L pin This pin connects to the FPGA and in turn this is used by the FPGA to reset the I O subsystem which includes the PLC PCI Express switch The PCIe switch in turn provides an inband reset output pin which via the FPGA resets the m Dual GbE for the Base m Dual GbE for RTM m GbE for management ports m RTM PCIe reset m Reset to PCI E to PCI bridge In addition to the processor resetting the I O subsystem each I O component has reset software write able by the PPC to reset individual components The Ethernet switch is reset able via software by the PPC This allows the Ethernet path between the PPC and the Base to remain active in the event the processor is being reset Chapter 5 Hardware Functional Descriptions 5 17 5 2 10 5 2 11 5 18 A reset mask register is provided in the FPGA to allow the masking of resets to individual I O components For example a PEX RESET L reset from the UltraSPARC T2 processor resets only those I O subcomponents not masked by the reset mask register ATCA Power Module 48V to 12V The Netra CP3260 blade server uses the Artesyn ATCA power module solution The Artesyn power module provides an integrated ATCA power solution that meets PICMG 3 0 requirements including dual bus input DC isolation hold up hot plug and management power 3 3V standby It provides a 12V intermediate bus as backend power
107. ultiplexed between the midplane Zone 2 connector that is the fabric interface The XAUI interface operate at 10 Gbps or 1 Gbps Common ARTM The Netra CP3260 provides a Zone 3 interface to a common ARTM which essentially appears as an AMC card The common ARTM supports the AMC 1 and AMC 2 compliant interfaces An X8 PCI Express channel is routed to the RTM slot The ARTM interface provides an Ethernet SERDES interface for AMC 2 compliant functionality These ports are sourced from the dual Gigabit Ethernet controller that connects to the PCI Express switch using a x4 lane PCI E channels See Section 5 2 4 Other ARTM Interfaces on page 5 11 for other non AMC interfaces routed to the ARTM ARTM Power and Management The common ARTMs require two voltages m 3 3V STBY for management current limited to 100 mA m 12V power for payload on ARTM Maximum 25 Watts Power management is handled by the ATCA shelf manager s Intelligent Platform Management Controller IPMC Netra CP3260 Blade Server User s Guide April 2009 5 2 4 5 2 4 1 5 2 4 2 5 2 0 5 2 5 1 5 2 5 2 Other ARTM Interfaces In addition to the AMC type interface to the ARTM additional interfaces are routed to the ARTM to provide for rear access to the Netra CP3260 I O Serial Ports The console port from the PPC is routed to the ARTM for rear access When a serial device is connected into the ARTM s serial port the console data is output to tha
108. ver s firmware has no graceful reboot support Any parameter changes you make will become effective only after you execute the Solaris command svcadm restart fsm Netra CP3260 Blade Server User s Guide April 2009 Note As of the R3U2 release Netra CP3260 sysfw 7 2 1 has graceful shutdown support Note To use the graceful shutdown effectively users must determine a component s shutdown time cost which depends on hardware software and configuration For example the default 120 seconds could be too short for a Sun Netra CP3260 blade server with a heavy I O configuration Conversely it could be too long for a Sun Netra CP3220 blade server Ignorance of the shutdown time could cause a panic during graceful shutdown For more information refer to the following documentation m Netra CT 900 Server Administration and Reference Manual 819 1177 for ShMM commands m Netra CT 900 Software Developer s Guide 819 1178 for IPMI commands Even if you are using a third party chassis the commands and options apply and these documents are available online http docs sun com app docs prod n900 srvr hic B 4 Send Sensor State Command Send Sensor State is used by the firmware progress sensor or any other sensor that intends to send sensor states to the IPM controller IPMC The IPMC uses this information to put a sensor in the correct state and send the event data to the system event log Command Ne

Netra CP3260 Blade Server Userճ Guide

Contents

Download Pdf Manuals

Related Search

Related Contents