Home

Siemens M9000 User's Manual

image

Contents

1. Reserve 02 03 0000 Current XSBHO1 2 02 02 0000 Reserve 02 03 0000 Current XSBHO1 3 02 02 0000 Reserve 02 03 0000 Current In case that the OpenBoot PROM version of the XSB to which the resource of the target CMU has been assigned is not displayed as 02 03 0000 contact a service engineer 36 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 14 15 16 17 18 19 20 Turn off the power to the target domain XSCF gt poweroff d domain_id Upgrade an on CMU SPARC64 VI processors to SPARC64 VII processors or add SPARC64 VII processors to an existing CMU m For upgrading the CPU operate by hot replacement referring to 6 2 Active Replacement and Hot Replacement in SPARC Enterprise M8000 M9000 Servers Service Manual m For adding the CPU operate by hot replacement referring to 6 2 Active Replacement and Hot Replacement in SPARC Enterprise M8000 M9000 Servers Service Manual Note Do not fail to execute the diagnosis of newly mounted CMU in the maintenance menu of replacefru 8 command Confirm that the mounted CPU module has been recognized by the server and the error indicator asterisk is not displayed XSCF gt showhardconf M Confirm no abnormality occurred by using showlogs error vand showstatus 8 commands XSCF gt showlogs error v XSCF gt showstatus If you encounter any hardw
2. output in the console log SUNW MSG ID FMD 8000 2K TYPE VER 1 SEVERITY Minor Defect EVENT TIME Fri Apr 4 21 41 57 PDT 2008 PLATFORM SUNW SPARC Enterprise CSN 2020642002 HOSTNAME lt hostname gt SOURCE fmd self diagnosis REV 1 0 EVENT ID 6b2e15d7 aa65 6bcc bcb1 cb03a7dd77e3 DESC A Solaris Fault Manager component has experienced an error that required the module to be disabled Refer to http sun com msg FMD 8000 2K for more information AUTO RESPONSE The module has been disabled Events destined for the module will be saved for manual diagnosis IMPACT Automated diagnosis and response for subsequent events associated with this module will not occur REC ACTION Use fmdump v u lt EVENT ID gt to locate the module Use fmadm reset lt module gt to reset the module command on the domain to recover svcadm clear fmd Then restart cpumem diagnosis fmadm restart cpumem diagnosis Information about Software 25 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6660197 DR might cause the domain to hang if either of This has been fixed in patch 138397 01 the following conditions exist e A domain contains 256 or more CPUs Workaround e Memory error occurred and the DIMM has 1 Set the following parameterin the system been degraded specification file etc system set drmach drmach disable mcopy 1 2 R
3. rm f var fm fmd ckpt cpumem diagnosis cpumem diagnosis 14 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 4 Solaris OS Issues and Workarounds Continued CRID Description Workaround 6498283 Using the DR deleteboard 8 command This bug has been fixed in Solaris 10 8 07 while psradm operations are running on a For Solaris 10 11 06 this has been fixed in domain might cause a system panic patch 120011 07 There is no workaround 6499304 CPU isn t offlined and unexpected message is This bug has been fixed in Solaris 10 8 07 displayed on console when many correctable For Solaris 10 11 06 this has been fixed in error CE occurs patch 125369 05 Example SUNW MSG ID FMD 8000 11 TYPE Workaround Defect VER 1 SEVERITY Minor Check CPU status on XSCF EVENT TIME Fri Feb 2 18 31 07 JST 2007 PLATFORM SPARC Enterprise CSN BE80601035 HOSTNAME FF2 35 0 6502204 Unexpected error messages may be displayed This bug has been fixed in Solaris 10 8 07 on console on booting after CPU UE panic For Solaris 10 11 06 this has been fixed in Example patch 125369 05 SUNW MSG ID FMD 8000 11 TYPE Defect VER 1 SEVERITY Minor Workaround EVENT TIME Tue Jan 9 20 45 08 JST Jf you see unexpected messages use the XSCF 498 command showdomainstatus 8 to check PLATFORM SUNW SPARC Enterprise system status on XSCF CSN 2030636002 HOSTNAME P2 DC1 16 d0 6502750 No
4. tape config list SEAGATE DAT DAT72 000 SEAGATE_DA DAT72 000 SEAGATE_DA DAT72 000 SEAGATE DAT DAT72 000 1 0x34 0 0x9639 4 0x00 0x8c 0x8c Ox8c 3 There are four spaces between SEAGATE DAT and DAT72 000 6466617 Performing a hot plug operation with the PCI Pause a few seconds between the issue of each Express slot too quickly interrupts a PCI leaf cfgadm c command reset and occurs the following error cfgadm Component system is busy error 6472153 If you create a Solaris OS Flash archive on a This bug has been fixed in Solaris 10 5 08 non SPARC Enterprise M4000 M5000 sun4u server and install it on a SPARC Enterprise M4000 M5000 sun4u server the console s TTY flags will not be set correctly This can cause the console to lose characters during stress For Solaris 10 8 07 or earlier this has been fixed in patch 137046 01 Workaround Just after installing Solaris OS from a Solaris Flash archive telnet into the SPARC Enterprise M4000 M5000 server to reset the console s TTY flags a follows sttydefs r console sttydefs a console i 9600 hupcl opost onlcr crtscts f 9600 This procedure is required only once Information about Software 13 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6481002 Installing the Solaris OS from the network If you are using a Sun PCI E Dual Gigabit using certain PCI Express cards may cause a Ethernet Adapter MMF card or a Sun PCI E
5. Issues and Workarounds ID Description Workaround RTIF1 While XSCF is running a process may go Check that XSCF is started to be translated If 070418 009 down a watchdog timeout may occur or a not started stop all the domains and then hang up may occur After this XSCF may execute the system power off on AC reset OFF ON To turn on the system power that you turned off wait at least 30 seconds before power on RTIF1 While XSCF is running watchdog timeout Check that XSCF is started If not started stop 070528 002 may occur and XSCF may reboot all the domains and then execute the system power off on AC OFF ON To turn on the system power that you turned off wait at least 30 seconds before power on 8 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 3 XCP Issues and Workarounds Continued ID Description Workaround RTIF1 Using the XSCF Web when you selected SSH To specify the value which exceeds the 070823 001 on the snapshot screen the maximum number maximum number of character input for the of character input for Host Directory ID and XSCF Web use XSCF Shell Password doesn t correspond to the maximum number of character input on the XSCF Shell RTIF1 On the XSCF Web when you select Domain Select Domain Mode Configuration one more 070824 002 Mode Configuration to perform various time and perform the settings Or once settings the pop up screen may not appear but term
6. LA MESURE AUTORISEE PAR LA LOI APPLICABLE Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE A L APTITUDE A UNE UTILISATION PARTICULIERE OU A LABSENCE DE CONTREFACON Contents Preface vii Technical Support vii Software Resources vii Accessing Documentation viii Fujitsu Siemens Computers Welcomes Your Comments viii General Information about XCP 1 Supported Firmware and Operating System 1 Solaris OS Patch Information 2 Patches for SPARC64 VI Processors 2 Patches for SPARC64 VII Processors 2 Updating toXCP 3 Resetting the XSCF Firmware 3 Updating from a Version Earlier Than XCP 1050 3 Updating from a Version Earlier Than XCP 1070 4 Functionality Issues and Limitations 4 Limitations for SPARC64 VII Processors 4 General Functionality Issues and Limitations 4 Information about Hardware 7 Hardware Issues and Workarounds 7 Sun Crypto Accelerator 6000 Cards 7 Information about Software 8 XCP Issues and Workarounds 8 Solaris OS Issues and Workarounds 12 Software Documentation Updates 27 Identifying Degraded Memory in a System 29 Identifying Different Memory Sizes in a System Board 29 Using the showdevices Command 29 Using the prtdiag Command to Identify Memory Size 30 Identifying Permanent Memory in a Target Board 31 CPU Upgrade 32 Installation Notes 32 Updating the OpenBoot PROM Firmware With Processor Upgrade 32 Adding SPARC64 VII Processors to a Domain Using DR 32 Adding a New CMU Equipped with
7. SPARC64 VII as a New Domain 33 Upgrading an On CMU SPARC64 VI to SPARC64 VII or Adding SPARC64 VII to an Existing CMU to an Existing Domain Configured with SPARC64 VI 35 Adding a New CMU Equipped with SPARC64 VII to an Existing Domain Configured with SPARC64 VI 38 vi SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 Preface These product notes contain late breaking information about the SPARC Enterprise M8000 M9000 server hardware software or documentation that became known after the documentation set was published Technical Support If you have technical questions or issues that are not addressed in the SPARC Enterprise M8000 M9000 servers documentation contact a sales representative or a certified service engineer Software Resources The Solaris Operating System and Sun Java Enterprise System software are preinstalled on your SPARC Enterprise M8000 M9000 servers Contact a sales representative or a certified service engineer for software resources for your SPARC Enterprise M8000 M9000 servers Note For latest patch information go to http support fujitsu siemens com Installation information and README files are included in the patch download vii Accessing Documentation Instructions for installing administering and using your SPARC Enterprise M8000 M9000 servers are provided in the SPARC Enterprise M8000 M9000 servers documentation set T
8. ae aper 1 ieabit Eth Re configure the affected network interfaces i paler A di za 10 Gigabit Ethernet after the completion of the DR operation For Piber XER Low profleAdapter basic network configuration procedures refer to the ifconfig man page for more information 6568417 After a successful CPU DR deleteboard 8 This bug has been fixed in Solaris 10 5 08 For operation the system panics when the Solaris 10 8 07 or earlier this has been fixed in following network interfaces are in use patch 127111 02 X4447A Z PCI e Quad port Gigabit Ethernet Adapter UTP Workaround e X1027A Z1 PCI e Dual 10 Gigabit Ethernet Add the following line to etc system and Fiber XFP Low profile Adapter reboot the system set ip ip soft rings cnt 0 6571370 Use of the following cards have been observed This bug has been fixed in Solaris 10 5 08 For to cause data corruption in stress test under Solaris 10 8 07 this has been fixed in patch laboratory conditions 127741 01 X4447A Z PCI e Quad port Gigabit Ethernet Adapter UTP Workaround e X1027A Z1 PCI e Dual 10 Gigabit Ethernet Add the following line in etc system and Fiber XFP Low profile Adapter reboot the system set nxge nxge rx threshold hi 0 6572827 On SPARC Enterprise M8000 M9000 servers There is no workaround one of the columns in the IO Devices section of the output from prtdiag v is Type This reports PCIe PCIx PCI or UNKN for each device The algorithm used to compute this valu
9. e boot r at open boot prompt e devfsadm C at Solaris prompt e cfgadm twice at Solaris prompt 6531036 The error message network There is no workaround This message can be initialization failed appears repeatedly after a boot net installation safely ignored 18 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6531668 System hangs when executing parallel hot There is no workaround plug operation with SP DR in suspend phase 6532215 volfs or dscp service may fail when domain is Restart the service if the failure is observed To booted avoid the problem issue the following commands svc platform sun4u dscp default Method lib svc method svc dscp svccfg s dscp setprop start failed with exit status 95 start timeout seconds count 300 svccfg s volfs setprop svc system filesystem volfs defaul start timeout_seconds count 300 t Method or service exit timed svcadm refresh dscp out Killing contract 59 svcadm refresh volfs 6533686 DR deleteboard 8 and moveboard 8 This has been fixed in patch 138397 01 operations might fail if the target board includes permanent memory Workaround Example for messages on domain Try DR operations again grmach VARNING DR DESIRE eos To avoid this issue before performing DR SC operations using deleteboard 8 and dcs lt xxxx gt config change state mo
10. on an account with platform administrative privileges Use the showstatus 8 command to confirm that a component in Faulted or Deconfigured status doesn t exist XSCF gt showstatus If no failures found the following message appears No failures found in System Initialization In case of other messages contact a service engineer before proceeding to the next step Change the key position on the operator panel from Locked to Service Collect an XSCF snapshot to archive system status prior to upgrade This will be help in case any problem occurred in this procedure XSCF gt snapshot t user host directory Update the XCP version to 1071 or later Before updating the XCP be sure to see Updating to XCP on page 3 For the XCP updating procedures see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide After updating the XCP reset the XSCE XSCF gt rebootxscf After resetting the XSC log in to the XSCF again Mount the CPU module CPUM on the CMU for add on For the procedure see the description about the CPU module installation in Section 6 4 1 Replacing a CPU module in the SPARC Enterprise M8000 M9000 Servers Service Manual Mount the CMU described in step 8 on the server a Executing the addfru 8 command and select CMU IOU from the maintenance menu XSCF gt addfru b Perform the operation according to the instruct
11. panic Dual Gigabit Ethernet Adapter UTP card do not install the Solaris using either of these cards Instead use other network devices such as the onboard Gigabit Ethernet or another network device 6485555 On board Gigabit Ethernet NVRAM This bug has been fixed in Solaris 10 8 07 corruption could occur due to a race condition For Solaris 10 11 06 this has been fixed in The window of opportunity for this race patch 120011 08 condition is very small There is no workaround 6496337 The cpumem diagnosis module may fail to This bug has been fixed in Solaris 10 8 07 load after uncorrectable error UE panic Systems will function correctly but events normally automatically diagnosed by FMA using this module will require manual diagnosis Example SUNW MSG ID FMD 8000 2K TYPE Defect VER 1 SEVERITY Minor EVENT TIME Thu Feb 15 15 46 57 JST 2007 PLATFORM SUNW SPARC Enterprise CSN BE80601007 HOSTNAME col2 ff em7 d0 For Solaris 10 11 06 this has been fixed in patch 125369 05 Workaround If problem occurred implement the following workaround 1 Remove the following file rm var fm fmd ckpt cpumem diagnosis cpumem diagnosis 2 Restart fmd service svcadm restart fmd To avoid this problem in advance add rm f var fm fmd ckpt cpumem diagnosis cpumem diagnosis in 1ib svc method svc dumpadm file as below We haven t run savecore on a dump device yet savedev none
12. setprop client memlim 40m This bug has been fixed in Solaris 10 8 07 For Solaris 10 11 06 this has been fixed in patch 120011 01 Workaround Do not exceed 170 CPU threads Limit the number of CPU threads to one per CPU core by using the Solaris psradm command to disable the excess CPU threads For example disable all odd numbered CPU threads This bug has been fixed in Solaris 10 8 07 For Solaris 10 11 06 this has been fixed in patch 120011 08 Workaround Use multiple NIC cards to split network connections This bug has been fixed in Solaris 10 8 07 For Solaris 10 11 06 this has been fixed in patch 120011 07 There is no workaround 12 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6449315 The Solaris OS cfgadm 1M command does Disable the Volume Management Daemon not unconfigure a DVD drive from a domain vold before unconfiguring a DVD drive with on a SPARC Enterprise M8000 M9000 server the cfgadm 1M command To disable vold stop the daemon by issuing the command etc init d volmgt stop After the device has been removed or inserted restart the daemon by issuing the command etc init d volmgt start 6459540 The DAT72 internal tape drive on SPARC Add the following definition to Enterprise M8000 M9000 may time out during kernel drv st conf tape operations
13. the target domain stopped XSCF gt showlogs power Change the key position on the operator panel from Locked to Service Information about Software 35 10 11 12 13 Collect an XSCF snapshot to archive system status prior to upgrade This will be help in case any problem occurred in this procedure XSCF gt snapshot t user host directory Update the XCP version to 1072 Before updating the XCP be sure to see Updating to XCP on page 3 For the XCP updating procedures see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide After updating the XCP reset the XSCE XSCF gt rebootxscf After resetting the XSC log in to the XSCF again Power on the target domain and apply OpenBoot PROM firmware XSCF gt poweron d domain_id It stops at ok prompt It is not necessary to start Solaris OS Check the updated OpenBoot PROM version The OpenBoot PROM version of XCP 1071 is 02 03 0000 XSCF gt version c cmu v DomainID 00 02 02 0000 DomainID 01 02 02 0000 DomainID 02 02 03 0000 DomainID 03 02 03 0000 DomainID 15 02 02 0000 XSB 00 0 02 02 0000 Current 01 01 0000 Reserve XSBHOO 1 02 02 0000 Current 01 01 0000 Reserve XSBHOO 2 02 02 0000 Current 01 01 0000 Reserve XSBHOO 3 02 02 0000 Current 02 01 0000 Reserve XSBHO1 0 02 02 0000 Reserve 02 03 0000 Current XSBHO1 1 02 02 0000
14. 0000 XSCF gt version c cmu v DomainID 00 02 02 0000 DomainID 01 02 02 0000 DomainID 02 02 03 0000 DomainID 03 02 03 0000 DomainID 15 02 02 0000 XSB 00 0 02 02 0000 Current 01 01 0000 Reserve XSBHOO 1 02 02 0000 Current 01 01 0000 Reserve XSB 00 2 02 02 0000 Current 01 01 0000 Reserve XSB 00 3 02 02 0000 Current 01 01 0000 Reserve XSBH01 0 02 02 0000 Reserve 02 03 0000 Current XSBHO1 1 02 02 0000 Reserve 02 03 0000 Current XSBH01 2 02 02 0000 Reserve 02 03 0000 Current XSBH01 3 02 02 0000 Reserve 02 03 0000 Current Information about Software 39 In case that the OpenBoot PROM version of the XSB to which the resource of the target CMU has been assigned is not displayed as 02 03 0000 contact a service engineer 14 Turn off the target domain XSCF gt poweroff d domain_id 15 Mount the CPU module CPUM on the CMU for add on For the procedure see the description about the CPU module installation in Section 6 4 1 Replacing a CPU module in the SPARC Enterprise M8000 M9000 Servers Service Manual 16 Mount the CMU described in step 15 on the server a Executing the addfru 8 command and select CMU IOU from the maintenance menu XSCF gt addfru b Perform the operation according to the instructions displayed in the maintenance menu referring to 6 2 Active Replacem
15. 41 42 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008
16. CH PRODUCT OR TECHNOLOGY OR THIS DOCUMENT WHICH ARE ALL PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Unless otherwise expressly set forth in such agreement to the extent allowed by applicable law in no event shall Fujitsu Limited Sun Microsystems Inc or any of their affiliates have any liability to any third party under any legal theory for any loss of revenues or profits loss of use or data or business interruptions or for any indirect special incidental or consequential damages even if advised of the possibility of such damages DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID SS com ca Adobe PostScript Copyright 2008 Sun Microsystems Inc 4150 Network Circle Santa Clara California 95054 U S A et FUJITSU LIMITED 1 1 Kamikodanaka 4 chome Nakahara ku Kawasaki shi Kanagawa ken 211 8588 Japon Tous droits r serv s Entr e et revue tecnical fournies par Fujitsu Limited sur des parties de ce mat riel Sun Microsystems I
17. LE 2 Hardware Issues and Workarounds CR ID Description Workaround 6433420 The domain console might display a Mailbox Issue a reset all command from the timeout or IOCB interrupt timeout error during OpenBoot PROM OK prompt and reboot boot 6557379 Power cables are not redundant on single On servers that have single power feed all power feed servers without the dual power power cables must be connected and powered feed option on at all times Sun Crypto Accelerator 6000 Cards If you are not using the correct version of the Sun Crypto Accelerator SCA 6000 card driver hot plug operations on SCA 6000 cards can cause Sun SPARC Enterprise M4000 M5000 servers to panic or hang Version 1 1 of the SCA6000 driver and firmware supports hot plug operations after the required bootstrap firmware upgrade has been performed Version 1 0 of the SCA6000 driver does not support hot plug and should not be used Information about Hardware 7 Information about Software This section describes the special instructions and the issues about the SPARC Enterprise M8000 M9000 servers software XCP Issues and Workarounds Solaris OS Issues and Workarounds Software Documentation Updates Identifying Degraded Memory in a System Identifying Different Memory Sizes in a System Board Identifying Permanent Memory in a Target Board CPU Upgrade XCP Issues and Workarounds TABLE 3 lists known XCP issues and possible workarounds TABLE 3 XCP
18. The corrections for SPARC Enterprise M4000 M5000 M8000 M9000 servers XSCF Reference Manual if not otherwise specified also apply to the man pages which XSCF provides And they supersede the information on the man pages TABLE 5 lists known documentation updates TABLE 5 Software Documentation Updates Title Page Number Update AII SPARC Enterprise M8000 M9000 servers documentation SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF Reference Manual Page 2 30 Page 2 91 Page 8 18 adduser 8 command console 8 command When there is the section SPARC Enterprise M8000 M9000 Servers Documentation in the Preface the following document will be added SPARC Enterprise PRIMEQUEST Common Installation Planning Manual C120 H007 The maximum length of the user name is 31 characters Table 2 4 User Accounts Adminstration erroneously documents a maximum user name length of 32 characters Table 2 22 DCL Information Configuration policy s System Removal of a domain domain deactivated is now refered to as below Hardware is degraded in units of domains or the relevant domain is stopped without degradation First paragraph in 8 1 11 is now refered as below If the system hangs or any of the messages shown below is output during the firmware update retry the firmware update It might be completed successfully The maximu
19. WARNING PCI Expansion ROM is not accessible will be seen on the console shortly before the system panic The following cards are affected by this defect e X4447A Z PCI e Quad port Gigabit Ethernet Adapter UTP e X1027A Z1 PCI e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter For Solaris 10 8 07 this has been fixed in patch 127741 01 Workaround Perform cfgadm c disconnect to completely remove the card After waiting at least 10 seconds the card may be configured back into the domain using the cfgadm c configure command Information about Software 21 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6559504 Messages of the form nxge NOTICE This bug has been fixed in Solaris 10 5 08 nxge_ipp_eccue_valid_check rd_ptr nnn For Solaris 10 8 07 this has been fixed in wr_ptr nnn will be observed on the console patch 127741 01 with the following cards X4447A Z PCI e Quad port Gigabit Workaround Ethernet Adapter UTP NOTY These messages can be safely ignored e X1027A Z1 PCI e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter 6564934 Performing a DR deleteboard 8 operation This bug has been fixed in Solaris 10 5 08 on a board which includes Permanent Memory For Solaris 10 8 07 this has been fixed in when using the following network cards patch 127741 01 results in broken connections e X4447A Z PCI e Quad port Gigabit Workaround Eth t Adapter UTP
20. are abnormality of the XSCF contact a service engineer Move the key position on the operator panel from Service to Locked In case of addition sets the following for the CMU m Set up XSB m Set up the LSB m Add the XSB to the domain For each setting see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCE User s Guide Set up and confirm the CPU operational mode of the domain For detail see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide Information about Software 37 21 Power on the target domains XSCF gt poweron d domain id 22 Confirm that the target domain has been correctly started XSCF gt showlogs power 23 Confirm no abnormality occurred by using showlogs error vand showstatus 8 commands XSCF gt showlogs error v XSCF gt showstatus If you encounter any hardware abnormality of the XSCF contact a service engineer Adding a New CMU Equipped with SPARC64 VII to an Existing Domain Configured with SPARC64 VI 1 When the Solaris OS version is Solaris 10 8 07 upgrade it to Solaris 10 5 08 or apply the mandatory patches described in Solaris OS Patch Information on page 2 2 To use SPARC64 VII processors apply the relevant patches to the software in use if necessary 3 Log in to the XSCF on an account with platform administrative privileges 4 Use the showstatus 8 command to confirm that a component in Faulted or Deco
21. ate the single user mode for Solaris OS do not switch from the domain console to the XSCF Shell m We recommend the domain to use the XSCF Unit as NTP server In this case pay attention to the following points u XSCF must be connected to an external NTP server a When you connect one or more NTP servers in addition to XSCF connect the same NTP server as XSCF is using For details on NTP server contact a service engineer For details on NTP settings refer to the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCE User s Guide General Information about XCP 1072 5 6 On the SPARC Enterprise M8000 M9000 servers with XCP 1050 or later the dual XSCF Unit feature is working Therefore you can not downgrade SPARC Enterprise M8000 M9000 servers with XCP 1050 or later to XCP 1040 or XCP 1041 which does not support dual XSCF Unit feature You cannot use the following user account names as they are reserved for system use root bin daemon adm operator nobody sshd rpc rpcuser ldap apache ntp admin and default The Firefox 3 is not supported in XSCF Web Using the XSCF Web when you import XCP or update the firmware Session ID error may be displayed on the web browser And in the Autologout setting when you specify the timeout period as over 30 minutes Internal Server Error might be displayed when you perform the firmware update Please close the current browser and open the new browser to reconnect to XSCF Web When you us
22. atisfiable 6510861 When Dual Channel Ultra320 SCSI Card This bug has been fixed in Solaris 10 8 07 SEOX7SC2F SEOX7SC2X is mounted For Solaris 10 11 06 this has been fixed in correctable errors CE occur and system may patch 120011 08 panic Workaround Add the following entry to the etc system file and then reboot the system set pcie pcie aer ce mask 0x31cl1 6511374 Unexpected error messages may be displayed This message can be safely ignored on console after changing the system configuration Example WARNING Translation error source LSB0 B0 0 PA 3c000000000 target LSB0 B0 20000000 16 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6515648 Replumb Failed error appears when Once the DR operation is complete it can be dr 0 SB1 memory fails plumbed up manually Example steps to re plumb the interface manually ifconfig interface plumb xxx XXX XXX XXX netmask broadcast up ifconfig interface group group name ifconfig interface addif XxXx XXX XXX XXX failover deprecated up This workaround assumes that the etc hostname interface file is correctly configured for the IPMP group and does not need any modification The IP addresses used in the example above should match what was previously used and what matches the etc hostname lt interface gt file 6516135 Ap_Id for
23. bout Software 31 CPU Upgrade This section describes the procedure to mount the SPARC64 VII processor on the SPARC Enterprise M8000 M9000 server m Adding a New CMU Equipped with SPARC64 VII as a New Domain m Upgrading an On CMU SPARC64 VI to SPARC64 VII or Adding SPARC64 VII to an Existing CMU to an Existing Domain Configured with SPARC64 VI m Adding a New CMU Equipped with SPARC64 VII to an Existing Domain Configured with SPARC64 VI Installation Notes Note Before upgrading firmware to XCP 1071 or later refer to Updating to XCP on page 3 Caution You must complete the upgrades to the XCP firmware and to Solaris before inserting SPARC 64 VII processors into the chassis Updating the OpenBoot PROM Firmware With Processor Upgrade If you have added SPARC64 VII processors to a domain and upgraded from an XCP version earlier than 1070 you must restart the domain to complete the update to the OpenBoot PROM firmware in the target domain Adding SPARC64 VII Processors to a Domain Using DR Before adding SPARC64 VII CPUs to a domain using dynamic configuration DR for the first time you must already have performed the following steps 1 Upgrade to XCP 1071 or later and a compatible version of Solaris 2 Reboot the domain 32 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 Adding a New CMU Equipped with SPARC64 VII as a New Domain Log in to the XSCF
24. cannot use Dynamic Reconfiguration m The maximum number of IOUA Base I O Card cards per domain is limited to six cards m Do not use the internal CD RW DVD RW drive unit and the TAPE drive unit at the same time m For this XCP release the XSCF browser user interface XSCF Web does not support the External I O Expansion Unit Manager feature m The XSCF does not support the Log Archiving feature m When using XSCF as the NTP server of the domain configure it so as not to block the ICMP protocol of the DNS server and the NTP server which the XSCF refers to m When you use the external power control interface of the external power controller the following notification signals are not supported u the OS panic or the server hardware error signal CPUN RTNU u the server hardware error signal power fail temperature error and fan error ALARM m For 1027A Z X1027A Z PCle Dual 10 Gigabit Ethernet Fiber XFP cards these limits apply u Do not use more than two cards per domain u Do not use these cards in an External I O Expansion Unit m For 4447A Z X4447A Z PCIe Quad port Gigabit Ethernet Adapter UTP cards these maximum limits apply u No more than two cards in an External I O Expansion Unit u No more than eight cards in a SPARC Enterprise M8000 M9000 servers m Incase that Solaris OS is a single user mode if you switch from the domain console to the XSCF Shell Solaris OS might start up in multi user When you oper
25. co FUJITSU COMPUTERS SIEMENS SPARC Enterprise M8000 M9000 Servers Product Notes For XCP version 1072 Order No U41813 J Z816 1 76 Part No 820 6008 10 September 2008 Revision A Copyright 2008 Sun Microsystems Inc 4150 Network Circle Santa Clara California 95054 U S A and FUJITSU LIMITED 1 1 Kamikodanaka 4 chome Nakahara ku Kawasaki shi Kanagawa ken 211 8588 Japan All rights reserved Sun Microsystems Inc and Fujitsu Limited each own or control intellectual property rights relating to products and technology described in this document and such products technology and this document are protected by copyright laws patents and other intellectual property laws and international treaties The intellectual property rights of Sun Microsystems Inc and Fujitsu Limited in such products technology and this document include without limitation one or more of the United States patents listed at http www sun com patents and one or more additional patents or patent applications in the United States or other countries This document and the product and technology to which it pertains are distributed under licenses restricting their use copying distribution and decompilation No part of such product or technology or of this document may be reproduced in any form by any means without prior written authorization of Fujitsu Limited and Sun Microsystems Inc and their applicable licensors if any The furnishing of this docume
26. e eee ee ee ee ee ee Memory Available Memory DIMM of Mirror Interleave LSB Group Size Status Size DIMMs Mode Factor 00 A 32768MB okay 2048MB 16 no 8 way 00 B 32768MB okay 2048MB 16 no 8 way 03 A 8192MB okay 2048MB 4 no 2 way 03 B 8192MB okay 2048MB 4 no 2 way 04 A 8192MB okay 204 8MB 4 no 2 way 04 B 8192MB okay 2048MB 4 no 2 way 05 A 8192MB okay 2048MB 4 no 2 way 05 B 8192MB okay 2048MB 4 no 2 way 06 A 8192MB okay 2048MB 4 no 2 way 30 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 Identifying Permanent Memory ina Target Board 1 Log in to XSCE 2 Execute the following command XSCF gt showdevices d domain_id The following example shows a display of the showdevices d command where 0 is the domain_id XSCF gt showdevices d 0 Memory board perm base domain target deleted remaining DID XSB mem MB mem MB address mem MB XSB mem MB mem MB 00 00 0 8192 0 0x0000000000000000 24576 00 00 2 8192 1674 0x000003c000000000 24576 00 00 3 8192 0 0x0000034000000000 24576 The entry for column 4 perm mem MB indicates the presence of permanent memory if the value is non zero The example shows permanent memory on 00 2 with 1674 MB If the board includes permanent memory when you execute the deleteboard 8 command or the moveboard 8 command the following notice appears System may be temporarily suspended proceed y n Information a
27. e is incorrect It reports PCI for PCI X leaf devices and UNKN for legacy PCI devices 22 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6584984 On SPARC Enterprise M8000 M9000 servers This bug has been fixed in Solaris 10 5 08 For busstat 1M command may cause rebooting Solaris 10 8 07 or earlier this has been fixed in of domains patch 127127 11 There is no workaround Do not use busstat 1M command Check for the availability of a patch for this defect 6588555 XSCF failover during DR operation to the Do not start an XSCF failover while a DR permanent memory might cause domain operation is running Wait for a DR operation panic to finish before starting the failover If you start the failover first wait for the failover to finish before starting the DR operation 6588650 On occasion the system is unable to DR after There is no workaround an XSCF failover or XSCF reboot 6589546 prtdiag 8 command does not show allI O This bug has been fixed in Solaris 10 5 08 devices of the following cards For Solaris 10 8 07 or earlier this has been e XSEFC402AF Sun StorageTek Enterprise fixed in patch 127127 11 Class 4Gb Dual Port Fibre Channel PCI E HBA Workaround e XSEFC401AF Sun StorageTek Enterprise Use prtdiag v for full output Class 4Gb Single Port Fibre Channel PCI E HBA 6589644 After add
28. e the XSCF Web if a plug in such as the search tool installed with the browser remove the plug in or disable the pop up blocking XSCF LAN is compliant with auto negotiation When you connect XSCF LAN and the network device which has been fixed to the full duplex mode according to the IEEE 802 3 rule XSCF LAN communicates in the half duplex mode Due to this network communication speed may slow down or communication error may occur Do not fail to set the network device which connects with XSCF LAN to the auto negotiation mode While you are performing DR operation to a COD board do not execute the addcodlicense 8 deletecodlicense 8 setcod 8 command The restoredefaults 8 command is not supported in this release If your domain is running one of the following versions of Solaris OS u Solaris 10 5 08 OS u An earlier version of Solaris 10 OS with patch ID 127127 11 Then you must set the following parameter in the system specification file etc system set heaplp use _stlb 0 Then reboot the domain For more information refer to CR 6718173 in TABLE 4 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 Information about Hardware This section describes the special instructions and the issues about the SPARC Enterprise M8000 M9000 servers hardware m Hardware Issues and Workarounds Hardware Issues and Workarounds TABLE 2 lists known hardware issues and possible workarounds TAB
29. eboot the domain 6679370 The following message may be output on the Add the following to etc system and then console during the system booting the reboot the domain External I O Expansion Unit adding by hotplug or the FMEMA operating by DR set pcie expected ce mask 0x2001 SUNW MSG ID SUN4 8000 75 TYPE Fault VER 1 SEVERITY Critical DESC A problem was detected in the PCI Express subsystem Refer to http sun com msg SUN4 8000 75 for more information 6565553 DR deleteboard 8 and moveboard 8 There is no workaround 6674266 operations might fail Try DR operations again Example for messages on domain drmach WARNING Device driver failure pci dcs lt xxxx gt config change state Hardware specific failure unconfigure SB1 Device driver failure pci 6718173 If your domain is running one of the following Set the following parameter in the system versions of Solaris OS the system might panic trap during normal operation e Solaris 10 5 08 OS e An earlier version of Solaris 10 OS with patch ID 127127 11 specification file etc system set heaplp_use_stlb 0 Then reboot the domain 26 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 Software Documentation Updates This section contains late breaking software information that became known after the documentation set was published and corrections in the SPARC Enterprise M8000 M9000 servers software documentation
30. ed the system board by DR when the The console can be recovered by pressing switching occurred on the redundant XSCF Ctrl q the Ctrl key and the q key Units the domain console may hang up 6592302 Unsuccessful DR operation leaves memory Try deleteboard 8 again partially configured Information about Software 23 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6614737 The DR deleteboard 8 and moveboard 8 This has been fixed in patch 137111 01 operations might hang if any of the following conditions exist Workaround A DIMM has been degraded Avoid performing DR operations if any of the The domain contains system boards with listed conditions exist different memory size To determine whether the system contains degraded memory use the XSCF showstatus 8 command For sample output see Identifying Degraded Memory in a System on page 29 To determine whether the domain contains system boards with different memory sizes display the list of memory sizes using the XSCF showdevices 8 command or the prtdiag 8 command on the domain For sammple of output see Identifying Different Memory Sizes in a System Board on page 29 If a DR command hangs reboot the domain to recover 6619224 For Solaris domains that include SPARC 64 VII This has been fixed in patch 137111 03 processors a single domain of 256 threads 128 Ram PA a for an is Workaround SAONA RAE GOA For So
31. ent and Hot Replacement in SPARC Enterprise M8000 M9000 Servers Service Manual Note Do not fail to execute the diagnosis of newly mounted CMU in the maintenance menu of addfru 8 command 17 Confirm that the mounted CPU module has been recognized by the server and the error indicator asterisk is not displayed XSCF gt showhardconf M 18 Confirm no abnormality occurred by using showlogs error v and showstatus 8 commands XSCF gt showlogs error v XSCF gt showstatus If you encounter any hardware abnormality of the XSCF contact a service engineer 19 Move the key position on the operator panel from Service to Locked 20 Sets the following for the CMU m Set up XSB m Set up the LSB m Add the XSB to the domain 40 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 21 22 23 m Set up the CPU operational mode of the domain For each setting see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCE User s Guide Power on the target domains XSCF gt poweron d domain id Confirm that the target domain has been correctly started XSCF gt showlogs power Confirm no abnormality occurred by using showlogs error vand showstatus 8 commands XSCF gt showlogs error v XSCF gt showstatus If you encounter any hardware abnormality of the XSCF contact a service engineer Information about Software
32. he documentation set is available for download from the following website http manuals fujitsu siemens com Note Information in these product notes supersedes the information in the SPARC Enterprise M8000 M9000 servers documentation set Solaris documentation is available at http www sun com documentation viii Fujitsu Siemens Computers Welcomes Your Comments If you have any comments or requests regarding this manual or if you find any unclear statements in the manual please state your points specifically and forward it to a sales representative or a certified service engineer Please include the title and part number of your document with your feedback SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 General Information about XCP 1072 This section describes the general information about XCP 1072 m Supported Firmware and Operating System m Updating to XCP m Functionality Issues and Limitations Supported Firmware and Operating System The following firmware and operating system OS are supported in this release TABLE 1 Firmware and Operating System Versions Firmware and Operaing System Version XSCF Control Package XCP SolarisTM Operating System SPARC64 VI processors SPARC64 VII processors 1072 10 8 07 or later 10 11 06 with required patches 10 5 08 or later 10 8 07 with required patches Note By us
33. her countries Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems Inc SPARC64 is a trademark of SPARC International Inc used under license by Fujitsu Microelectronics Inc and Fujitsu Limited The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems Inc for its users and licensees Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry Sun holds a non exclusive license from Xerox to the Xerox Graphical User Interface which license also covers Sun s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun s written license agreements United States Government Rights Commercial use U S Government users are subject to the standard government user license agreements of Sun Microsystems Inc and Fujitsu Limited and the applicable provisions of the FAR and its supplements Disclaimer The only warranties granted by Fujitsu Limited Sun Microsystems Inc or any affiliate of either of them in connection with this document or any product or technology described herein are those expressly set forth in the license agreement pursuant to which the product or technology is provided EXCEPT AS EXPRESSLY SET FORTH IN SUCH AGREEMENT FUJITSU LIMITED SUN MICROSYSTEMS INC AND THEIR AFFILIATES MAKE NO REPRESENTATIONS OR WARRANTIES OF ANY KIND EXPRESS OR IMPLIED REGARDING SU
34. inate the XSCF Web and then perform the Undefined may be displayed on the screen settings RTIF1 On the XSCF Web when you changed the Ignore the pop up and close the screen 070824 005 Refresh Interval value of the Monitor Message Frame the invalid pop up Confirmation Domain mode may appear RTIF1 On the tab browser to the same host when When you use the tab browser do not perform 070824 006 you perform multiple log in with the user multiple log in to the same host accounts of different user privileges the user privilege of the last log in user account will be applied to those pages which you ve already logged in RTIF1 While using FireFox2 in the Configuration From the Menu select the Domain 070824 011 policy setting on the Domain Configuration Configuration page again screen when you specify a domain which is in operation an error display pop up appears When you click on the Back button on this error display pop up and click on the Cancel button on the inquiry screen to re display the data the system remains in the error message screen RTIF1 CLIs should display Permission denied when Only the following CLIs can be executed on 070904 001 it is executed in Standby XSCF the Standby XSCF snapshot 8 switchscf 8 Do not attempt to run any other CLI on the Standby Such attempts will report various errors RTIF1 The latest communication field in Disabling and re enabling archiving refreshes 070904 004 showarchiving 8 c
35. ing the Solaris 10 8 07 installation DVD you cannot boot the domain mounted with the SPARC64 VII processors When you newly install Solaris OS to a domain mounted with the SPARC64 VII processors use the Solaris 10 5 08 installation DVD to install Solaris 10 5 08 For XCP you can download the latest files of firmware at the following websites Global Site http www fujitsu com sparcenterprise firmware Japanese Site http primeserver fujitsu com sparcenterprise download firmware Solaris OS Patch Information This section lists mandatory patches for the SPARC Enterprise M8000 M9000 servers Patches for SPARC64 VI Processors These patches are required for servers running Solaris 10 11 06 118833 36 Install 118833 36 before 125100 04 125100 04 or later 120068 03 or later 123839 07 or later 125424 01 or later 125075 01 or later 125670 02 or later The patches are not required for servers running Solaris 10 8 07 OS or later Patches for SPARC64 VII Processors These patches are required for servers running Solaris 10 8 07 and mounting SPARC64 VII processors Note Apply the patches in the following order For the procedures of CPU upgrade including the patches see CPU Upgrade on page 32 2 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 119254 51 or later 125891 01 or later 127755 01 or later 127127 11 The patches are not required for servers running So
36. ing to XCP 1050 or later This account name is reserved in XCP 1050 and higher Use the deleteuser 8 command to delete the account Updating from a Version Earlier Than XCP 1070 On a domain which has been in operation during the XCP update to XCP 1071 or later when you perform Dynamic Reconfiguration DR to add or replace the SPARC64 VII processors you need to update the OpenBoot PROM firmware The OpenBoot PROM firmware is updated as you update the XCP and restart the domain For this reason it is strongly recommended to restart all the domains after you update the firmware to XCP 1071 or later regardless of whether you added or replaced the SPARC64 VII processors A Functionality Issues and Limitations This section describes the known issues and limitations at the time of this release Limitations for SPARC64 VII Processors Caution You must complete the upgrades to the XCP firmware and to Solaris OS before inserting the CPU memory board unit of SPARC 64 VII processors into the chassis On a SPARC Enterprise M9000 server if a single domain includes 16 boards containing SPARC64 VII processors then SunVTS 7 0ps2 software will not start CR ID 6720922 4 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 General Functionality Issues and Limitations Caution For dynamic reconfiguration DR and hot plug issues see TABLE 4 m Domains using the ZFS file system
37. ions displayed in the maintenance menu referring to 6 2 Active Replacement and Hot Replacement in SPARC Enterprise M8000 M9000 Servers Service Manual Information about Software 33 Note Do not fail to execute the diagnosis of newly mounted CMU in the maintenance menu of addfru 8 command 10 Confirm that the mounted CPU module has been recognized by the server and the error indicator asterisk is not displayed XSCF gt showhardconf M 11 Confirm no abnormality occurred by using showlogs error v and showstatus 8 commands XSCF gt showlogs error v XSCF gt showstatus If you encounter any hardware abnormality of the XSCF contact a service engineer 12 Move the key position on the operator panel from Service to Locked 13 Sets the following for the CMU m Set up XSB m Set up the domain m Set up the CPU operational mode of the domain For each setting see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide 14 Use the setdomainmode 8 command disable the autoboot function of the domain For detail see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide 15 Power on the domains XSCF gt poweron d domain id 16 Confirm that the target domain has been correctly started XSCF gt showlogs power 17 Confirm no abnormality occurred by using showlogs error v and showstatus 8 commands XSCF gt sho
38. ixed in Solaris 10 8 07 fails For Solaris 10 11 06 this has been fixed in patch 120011 09 There is no workaround 6545143 When kcage thread is expanding the kcage This bug has been fixed in Solaris 10 5 08 For area if the user stack exists in the expanded Solaris 10 8 07 or earlier this has been fixed in area its area is demapped and might cause a patch 127111 08 ptl_1 panic during the flushw handler execution There is no workaround 6545685 If the following message displayed on the OS This bug has been fixed in Solaris 10 5 08 console memory degradation or XSB For Solaris 10 8 07 or earlier this has been deconfiguration may occur in the subsequent fixed in patch 127111 08 reboot Workaround the following to etc system and then Example Add the following to etc d th mc opl WARNING mc opl rewrite reboot the domain ee ea set mc opl mc max rewrite loop 20000 6546188 The system panics when running hotplug This bug has been fixed in Solaris 10 5 08 cfgadm 1M and DR operations For Solaris 10 8 07 this has been fixed in addboard 8 and deleteboard 8 on the patch 127741 01 following cards X4447A Z PCI e Quad port Gigabit Eth t Adapter UTP ave Te Sr There is no workaround e X1027A Z1 PCI e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter 6551356 The system panics when running hotplug This bug has been fixed in Solaris 10 5 08 cfgadm 1M to configure a previously unconfigured card The message
39. laris 10 5 08 OS or later Note See Software Resources on page vii for information on how to find the latest patches Installation information and README files are included in the patch download Patches for Emulex PCI E Cards The following Emulex cards require drivers supplied in patch 120222 26 m XSEFC402AF Sun StorageTek Enterprise Class 4Gb Dual Port Fibre Channel PCI E HBA m XSEFC401AF Sun StorageTek Enterprise Class 4Gb Single Port Fibre Channel PCI E HBA Updating to XCP 1071 or Later To update your XCP to XCP 1071 or later please pay attention to the following points depending on your current XCP version You can upgrade to XCP 1071 or later from XCP version 1050 or higher Refer to the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide for instructions Resetting the XSCF Firmware After updating the XCP firmware to 1071 or later do not fail to reset the XSCF by using the reboot xscf 8 command General Information about XCP 1072 3 Updating from a Version Earlier Than XCP 1050 m You cannot update to XCP 1071 or later directly If you are currently running a version earlier than XCP 1050 you must first update to an interim version of XCP between 1050 and 1070 inclusive before updating to XCP 1071 or later Refer to the product notes document for the interim version for instructions m Delete any accounts named admin Any accounts named admin must be deleted prior to updat
40. laris domains that include SPARC 64 VII situations Upon recovery the uptime cae processors do not exceed a domain size of 256 command will show extremely high load g averages virtual CPUs in a single Solaris domain This means a maximum of 32 CPU chips in a single domain configuration maximum configuration for an M8000 server 6623226 The Solaris command lockstat 1M or the Do not use the Solaris lockstat 1M dtrace lockstat provider might cause a system command or the dtrace lockstat provider panic 6625734 Systems with large number of processors ina Use processor sets to bind application single domain environment may have suboptimal performance with certain workloads processes or LWPs to groups of processors Refer to the psrset 1M man page for more information 24 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6632549 fmd service on domain might fail to This has been fixed in patch 138050 01 maintenance mode after DR operations Workaround If fmd service fails issue the following commands on the domain to recover svcadm clear fmd 6660168 If aubc piowbeue cpu error occurs ona If FMA service fails issue the following domain the Solaris Fault Management cpumem diagnosis module might fail causing an interruption in FMA service If this happens you will see the following
41. m length of the user name is 31 characters The OPERANDS of the adduser 8 command erroneously documents a maximum user name length of 32 characters The note in DESCRIPTION is changed as follows When you return to XSCF shell console without logging out from the domain the return causes automatically logging out from the domain Information about Software 27 TABLE 5 Software Documentation Updates Continued Title Page Number Update SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF Reference Manual SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF Reference Manual and XSCF manpages sendbreak 8 command setdscp 8 commands viewaudit 8 command setdcl 8 command setaudit 8 command The sendbreak 8 command will not work when the secure mode is set to on while the mode switch on the operator panel is set to locked Refer to the setdomainmode 8 for more information The references to site planning guide of the caution in EXAMPLES are now referenced to as administration guide The viewaudit 8 man pages show incorrect output for Example 5 and Example 6 The action when option is set to policy and when value is set to system will be changed as below Deactivates the target system for an error detected by a diagnosis By using the t option at most four values may be set instead of two 28 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 Septembe
42. mat and devices may not be shown Use the following operations to display all of correctly by cfgadm 1M the PCI slots 1 devfsadn at Solaris prompt 2 cfgadm 6519290 Large amounts of I O on swap devices can Set the following to etc system and then cause the system to appear hung by reboot the domain overwhelming the I O system The amount of set maxfastscan 0x2000 I O required can be generated through a number of ways eg memory shortage heavy use of tmp etc 6520990 Domain may cause a panic when This bug has been fixed in Solaris 10 8 07 deleteboard 8 command for kernel board For Solaris 10 11 06 this has been fixed in by using Dynamic Reconfiguration DR patch 120011 08 Workaround To mask this error add the following entry to the etc system file set drmach fmem_timeout 30 6522017 DR and ZFS may not be used in the same Reduce the amount of kernel memory that ZFS domain can allocate by setting the zfs_arc_max parameter in the etc system file The following example sets the maximum size to 512 Mbytes set zfs arc max 0x20000000 Information about Software 17 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6522433 After the CPU hardware error occurred the This bug has been fixed in Solaris 10 5 08 For fmdump 1M command on the domain may Solaris 10 8 07 or earlier this has been fixed in display an incorrect faulty component patch 127127 11 Workaround Check system
43. nc et Fujitsu Limited d tiennent et contr lent toutes deux des droits de propri t intellectuelle relatifs aux produits et technologies d crits dans ce document De m me ces produits technologies et ce document sont prot g s par des lois sur le copyright des brevets d autres lois sur la propri t intellectuelle et des trait s internationaux Les droits de propri t intellectuelle de Sun Microsystems Inc et Fujitsu Limited concernant ces produits ces technologies et ce document comprennent sans que cette liste soit exhaustive un ou plusieurs des brevets d pos s aux Etats Unis et indiqu s l adresse http www sun com patents de m me qu un ou plusieurs brevets ou applications brevet es suppl mentaires aux Etats Unis et dans d autres pays Ce document le produit et les technologies aff rents sont exclusivement distribu s avec des licences qui en restreignent l utilisation la copie la distribution et la d compilation Aucune partie de ce produit de ces technologies ou de ce document ne peut tre reproduite sous quelque forme que ce soit par quelque moyen que ce soit sans l autorisation crite pr alable de Fujitsu Limited et de Sun Microsystems Inc et de leurs ventuels bailleurs de licence Ce document bien qu il vous ait t fourni ne vous conf re aucun droit et aucune licence expresses ou tacites concernant le produit ou la technologie auxquels il se rapporte Par ailleurs il ne contient ni ne repr
44. nfigured status doesn t exist XSCF gt showstatus If no failures found the following message appears No failures found in System Initialization In case of other messages contact a service engineer before proceeding to the next step 5 Turn off the power to a domain assigned the CMU of which you add the CPU XSCF gt poweroff d domain_id 6 Confirm that the target domain stopped XSCF gt showlogs power 38 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 10 11 12 13 Change the key position on the operator panel from Locked to Service Collect an XSCF snapshot to archive system status prior to upgrade This will be help in case any problem occurred in this procedure XSCF gt snapshot t user host directory Update the XCP version to 1071 or later Before updating the XCP be sure to see Updating to XCP on page 3 For the XCP updating procedures see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide After updating the XCP reset the XSCF XSCF gt rebootxscf After resetting the XSCE log in to the XSCF again Power on the target domain and update OpenBoot PROM firmware XSCF gt poweron d domain_id It stops at ok prompt It is not necessary to start Solaris OS Check the updated OpenBoot PROM version The OpenBoot PROM version of XCP 1071 is 02 03
45. nt to you does not give you any rights or licenses express or implied with respect to the product or technology to which it pertains and this document does not contain or represent any commitment of any kind on the part of Fujitsu Limited or Sun Microsystems Inc or any affiliate of either of them This document and the product and technology described in this document may incorporate third party intellectual property copyrighted by and or licensed from suppliers to Fujitsu Limited and or Sun Microsystems Inc including software and font technology Per the terms of the GPL or LGPL a copy of the source code governed by the GPL or LGPL as applicable is available upon request by the End User Please contact Fujitsu Limited or Sun Microsystems Inc This distribution may include materials developed by third parties Parts of the product may be derived from Berkeley BSD systems licensed from the University of California UNIX is a registered trademark in the U S and in other countries exclusively licensed through X Open Company Ltd Sun Sun Microsystems the Sun logo Java Netra Solaris Sun Ray Answerbook2 docs sun com OpenBoot and Sun Fire are trademarks or registered trademarks of Sun Microsystems Inc in the U S and other countries Fujitsu and the Fujitsu logo are registered trademarks of Fujitsu Limited All SPARC trademarks are used under license and are registered trademarks of SPARC International Inc in the U S and ot
46. of 080512 001 hostname of the sethostname 8 command the sethostname 8 command and reset XSCF by using the applynetwork 8 and the rebootxsc 8 commands a process goes down in XSCF RTIF1 When the system is stressed with many faults Reboot the service processor using the XSCF 080526 001 the fmd process on the service processor might command rebootxscf 8 hang Once this happens fma commands on the service processor can fail or hang RTIF1 In setsnmp addv3traphost when the There is no workaround 080725 001 authentication failed due to the reasons such Confirm that the SNMP trap host is working as the trap host not working or the wrong and re execute the setsnmp 8 command user name or password the subsequent SNMP using the correct user name traps will not be notified RTIF1 When the SNMPv3 trap has been set after the There is no workaround 080725 002 watchdog timeout occurred in XSCF and XSCF reset the subsequent SNMP traps will not be notified RTIF1 The initial value that set with the In the OpenBoot PROM prompt ok gt execute 080725 003 setdomparam set defaults option differs the set defaults command to restore the from the initial value of OpenBoot PROM OpenBootPROM environmental variables to parameters current value expected value the initial value diag level none 0x00 min 0x20 auto boot false 0x00 true 0xff RTIF1 After set the Daylight Saving Time by using There is no workaround 080725 004 the XSCF
47. ommand is not updated the Latest communication field in regularly showarchiving 8 output RTIF1 When the XSCF switching performed there There is no workaround 070914 003 may be SCF Board control error DMA timeout stored in the error log This message can be safely ignored Information about Software 9 TABLE 3 XCP Issues and Workarounds Continued ID Description Workaround RTIF1 When you execute XCP Sync on the Firmware There is no workaround However the XCP 070914 025 Update page after 15 minutes the error Sync process has been continuously executed message Another flashupdate is now Check the XSCF update completion message processing or The page cannot be displayed on the monitoring message to confirm the may appear completion of Sync process RTIF1 The snmp daemon might quit To restart the snmp daemon issue the 071102 002 command setsnmp enable RTIF1 After using the addfru 8 or replceafru 8 Do not fail to execute the diagnosis in the 071116 001 command for active replacement without the maintenance menu of the addfru 8 or diagnostic test in the maintenance menu replacefru 8 command In case of missing further DR operation might fail with a the diagnostic test in the maintenance menu of misleading message regarding the board being the addfru 8 or the replacefru 8 unavailable for DR command execute the testsb 8 command or delete the CPU memory board unit using the deletefru 8 command and then ret
48. onal Inc aux Etats Unis et dans d autres pays Les produits portant les marques SPARC sont bas s sur une architecture d velopp e par Sun Microsystems Inc SPARC64 est une marques d pos e de SPARC International Inc utilis e sous le permis par Fujitsu Microelectronics Inc et Fujitsu Limited L interface d utilisation graphique OPEN LOOK et Sun a t d velopp e par Sun Microsystems Inc pour ses utilisateurs et licenci s Sun reconna t les efforts de pionniers de Xerox pour la recherche et le d veloppement du concept des interfaces d utilisation visuelle ou graphique pour l industrie de l informatique Sun d tient une license non exclusive de Xerox sur l interface d utilisation graphique Xerox cette licence couvrant galement les licenci s de Sun qui mettent en place l interface d utilisation graphique OPEN LOOK et qui en outre se conforment aux licences crites de Sun Droits du gouvernement am ricain logiciel commercial Les utilisateurs du gouvernement am ricain sont soumis aux contrats de licence standard de Sun Microsystems Inc et de Fujitsu Limited ainsi qu aux clauses applicables stipul es dans le FAR et ses suppl ments Avis de non responsabilit les seules garanties octroy es par Fujitsu Limited Sun Microsystems Inc ou toute soci t affili e de l une ou l autre entit en rapport avec ce document ou tout produit ou toute technologie d crit e dans les pr sentes correspondent aux garanties exp
49. r 2008 Identifying Degraded Memory ina System 1 Log in to XSCE 2 Type the following command XSCF gt showstatus The following example identifies DIMM number 00A on CMU 3 has degraded memory XSCF gt showstatus CMU 3 Status Normal MEM 00A Status Degraded Identifying Different Memory Sizes in a System Board To identify if the domain contains system boards with different memory sizes you can use either of the following commands to display the list of memory sizes m showdevices 8 command on the XSCF m prtdiag 1M command on the domain Using the showdevices Command 1 Log in to XSCE Information about Software 29 2 Type the following command XSCF gt showdevices d domain id The following example displays 00 0 has 64GB of memory while the other system boards have 16GB XSCF gt showdevices d 1 Memory board perm base domain target deleted remaining DID XSB mem MB mem MB address mem MB XSB mem MB mem MB 01 00 0 63680 0x0000004000000000 260288 01 03 0 16384 7384 0x0000034000000000 260288 01 03 1 16384 0x0000030000000000 260288 01 03 2 16384 0x000002c000000000 260288 01 03 3 16384 0x0000028000000000 260288 Using the prtdiag Command to Identify Memory Size m On the domain execute the prtdiag command prtdiag The following example displays different memory sizes prtdiag Memory Configuration wor
50. ress ment stipul es dans le contrat de licence r gissant le produit ou la technologie fourni e SAUF MENTION CONTRAIRE EXPRESSEMENT STIPULEE DANS CE CONTRAT FUJITSU LIMITED SUN MICROSYSTEMS INC ET LES SOCIETES AFFILIEES REJETTENT TOUTE REPRESENTATION OU TOUTE GARANTIE QUELLE QU EN SOIT LA NATURE EXPRESSE OU IMPLICITE CONCERNANT CE PRODUIT CETTE TECHNOLOGIE OU CE DOCUMENT LESQUELS SONT FOURNIS EN L TAT EN OUTRE TOUTES LES CONDITIONS REPRESENTATIONS ET GARANTIES EXPRESSES OU TACITES Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE A L APTITUDE A UNE UTILISATION PARTICULIERE OU A L ABSENCE DE CONTREFACON SONT EXCLUES DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE Sauf mention contraire express ment stipul e dans ce contrat dans la mesure autoris e par la loi applicable en aucun cas Fujitsu Limited Sun Microsystems Inc ou l une de leurs filiales ne sauraient tre tenues responsables envers une quelconque partie tierce sous quelque th orie juridique que ce soit de tout manque a gagner ou de perte de profit de probl mes d utilisation ou de perte de donn es ou d interruptions d activit s ou de tout dommage indirect sp cial secondaire ou cons cutif m me si ces entit s ont t pr alablement inform es d une telle ventualit LA DOCUMENTATION EST FOURNIE EN LETAT ET TOUTES AUTRES CONDITIONS DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES DANS
51. ry the addfru 8 command RTIF1 While system power on when the XSCF There is no workaround This message can be 071116 005 switching performed by switchsc 8 safely ignored command PANIC might be generated in XSCF before it switches and SHUTDOWN timeout may be stored in the error log RTIF1 When the server is being installed and the Turn off the system AC power then turn it on 071126 002 mainline switch is turned on for the first time again these error messages might be displayed When the error log is displayed again turn off PSU shortage the system AC power then turn it on again To XSCF watchdog timeout turn on the system power that you turned off XSCFU hang up is detected wait at least 30 seconds before power on XSCF process down detected RTIF1 An error log cannot be registered in XSCF There is no workaround 071129 003 when the following message is displayed on Solaris OS by the External I O Expansion Unit error SUNW MSG ID SUN4 8000 4P TYPE Fault RTIF1 When you use the command setsnmpusm Always set a password of at least eight 080404 002 passwd to set a password if you set a characters password of fewer than eight characters a segmentation fault occurs 10 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 3 XCP Issues and Workarounds Continued ID Description Workaround RTIF1 When you specify localhost to the Do not specify localhost to the hostname
52. sente aucun engagement de quelque type que ce soit de la part de Fujitsu Limited ou de Sun Microsystems Inc ou des soci t s affili es Ce document et le produit et les technologies qu il d crit peuvent inclure des droits de propri t intellectuelle de parties tierces prot g s par copyright et ou c d s sous licence par des fournisseurs A Fujitsu Limited et ou Sun Microsystems Inc y compris des logiciels et des technologies relatives aux polices de caracteres Par limites du GPL ou du LGPL une copie du code source r gi par le GPL ou LGPL comme applicable est sur demande vers la fin utilsateur disponible veuillez contacter Fujitsu Limted ou Sun Microsystems Inc Cette distribution peut comprendre des composants d velopp s par des tierces parties Des parties de ce produit pourront tre d riv es des syst mes Berkeley BSD licenci s par l Universit de Californie UNIX est une marque d pos e aux Etats Unis et dans d autres pays et licenci e exclusivement par X Open Company Ltd Sun Sun Microsystems le logo Sun Java Netra Solaris Sun Ray Answerbook2 docs sun com OpenBoot et Sun Fire sont des marques de fabrique ou des marques d pos es de Sun Microsystems Inc aux Etats Unis et dans d autres pays Fujitsu et le logo Fujitsu sont des marques d pos es de Fujitsu Limited Toutes les marques SPARC sont utilis es sous licence et sont des marques de fabrique ou des marques d pos es de SPARC Internati
53. shell XSCF Web does not show the Use the showlogs 8 command of the XSCF correct time in the Logs menu shell RTIF1 On the cluster system using the Refer to the manual of the PRIMECLUSTER 080808 001 PRIMECLUSTER software when there are 10 software to check the cluster status If no or more RCI units the RCI busy status may be temporarily unreleased and output the following syslog message SA_pprcir so to test host host_name failed problem found on the status this message can be safely ignored If any problem follow the instructions on the manual to solve the problem Information about Software 11 TABLE 4 Solaris OS Issues and Workarounds TABLE 4 lists known Solaris OS issues and possible workarounds Solaris OS Issues and Workarounds CR ID Description Workaround 5076574 6303418 6416224 6441349 A PCle error can lead to an invalid fault diagnosis on a large M8000 M9000 domain A SPARC Enterprise M9000 server with a single domain and 11 or more fully populated system boards might hang under heavy stress System performance can degrade using a single NIC card with more than 5 000 connections The system may hang if there is an I O error in the system This bug has been fixed in Solaris 10 5 08 For Solaris 10 8 07 or earlier this has been fixed in patch 127127 11 Workaround Create a file etc fm fmd fmd conf containing the following lines setprop client buflim 40m
54. status on XSCF 6527781 The cfgadm command fails while moving the This bug has been fixed in Solaris 10 8 07 DVD DAT drive between two domains For Solaris 10 11 06 this has been fixed in patch 125081 06 There is no workaround To reconfigure DVD Tape drive execute reboot r from the domain exhibiting the problem 6527811 The showhardconf 8 command on the XSCF This bug has been fixed in Solaris 10 5 08 cannot display PCI card information that is For Solaris 10 8 07 or earlier this has been installed in External I O Expansion Unit ifthe fixed in patch 128346 01 External I O Expansion Unit is configured using PCI hotplug There is no workaround 6529714 Warning messages occur while trying to There is no workaround configure more than four X4447A Z or X1027A Z1 cards into one I O Boat 6530178 DR addboard 8 command can hang Once This bug has been fixed in Solaris 10 8 07 problem is observed further DR operations are For Solaris 10 11 06 this has been fixed in blocked Recovery requires reboot of the patch 120011 07 domain There is no workaround 6530288 Ap_Id format may not be shown correctly by This bug has been fixed in Solaris 10 8 07 cfgadm 1M command For Solaris 10 11 06 this has been fixed in patch 120011 07 There is no workaround 6530753 Some of the PCI slots in the External I O Use one of the following operations to display Expansion Unit PCI slots are not displayed all of the PCI slots during a normal boot operation
55. tification message for inserted or removed This bug has been fixed in Solaris 10 8 07 card by PCI hot plug may not output For Solaris 10 11 06 this has been fixed in patch 120011 08 There is no workaround 6505921 Correctable error on the system PCIe bus This bug has been fixed in Solaris 10 5 08 controller generates an invalid fault For Solaris 10 8 07 or earlier this has been fixed in 120011 14 and 125369 05 Workaround Create a file etc fm fmd fmd conf containing the following lines setprop client buflim 40m setprop client memlim 40m Information about Software 15 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6508432 Many correctable errors CE may occur and This bug has been fixed in Solaris 10 8 07 despite these are the correctable errors For Solaris 10 11 06 this has been fixed in domain may panic patch 120011 08 Workaround Set the following to etc system and then reboot the domain set pcie pcie aer ce mask 0x2001 6508434 The domain may panic when an additional This bug has been fixed in Solaris 10 8 07 PCI X card is installed or a PCI X card is For Solaris 10 11 06 this has been fixed in replaced by using PCI hot plug patch 120011 08 Workaround Do not insert a different type of PCI X card on the same PCI slot by using PCI hot plug 6509337 s10s_u3 wanboot fails The server returned This bug has been fixed in Solaris 10 8 07 416 Requested Range Not S
56. ur gt gt Workaround To avoid this problem issue the following commands on the domain cd usr platform SUNW SPARC Enterprise lib fm topo plugins mv ioboard so ioboard so orig svcadm restart fmd If the following messages are displayed on the domain contact a service engineer Example SUNW MSG ID SUNOS 8000 1L TYPE Defect VER 1 SEVERITY Minor EVENT TIME Sun May 6 18 22 24 PDT 2007 PLATFORM SUNW SPARC Enterprise CSN BE80601007 HOSTNAME sparc 6537511 Bluetooth partner is hung during security tests Restart application server execution 6539084 Under Solaris 10 11 06 there is a low This bug has been fixed in Solaris 10 8 07 probability of a domain panic during reboot when the Sun Quad GbE UTP x8 PCIe There no workaround X4447A Z card is present in a domain 6539909 Under Solaris 10 11 06 do not use the Use an alternate type of network card or following I O cards for network access when you are using the boot net install command to install the Solaris OS e X4447A Z X4447A Z PCIe Quad port Gigabit Ethernet Adapter UTP e X1027A Z X1027A Z PCIe Dual 10 Gigabit Ethernet Fiber XFP onboard network device to install the Solaris OS via the network 20 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6542632 Memory leak in PCIe module if driver attach This bug has been f
57. veboard 8 commands on a board that Hardware specific failure includes permanent memory follow these unconfigure SBl DR parellel copy steps Pameour 1 Add the following line to the System Configuration file etc system set drmach drmach disable mcopy 1 2 Reboot the domain 6534471 Domains might panic This bug has been fixed in Solaris 10 8 07 6535018 In Solaris domains that include SPARC64 VII This has been fixed in patch 137111 01 processors workloads that make heavy use of the Solaris kernel might not scale as expected Workaround when you increase the thread count to a value gt For Solaris domains that include SPARC64 VII greater than 256 s processors limit domains to a maximum of 256 threads 6535564 PCI hot plug to PCI slot 0 1 or External I O This bug has been fixed in Solaris 10 8 07 Expansion Unit may fail on XSB added by DR For Solaris 10 11 06 this has been fixed in patch 120011 08 Workaround Use DR instead of PCI hot plug if need to add or remove PCI card on the XSB Information about Software 19 TABLE 4 Solaris OS Issues and Workarounds Continued CR ID Description Workaround 6536564 showlogs 8 and showstatus 8 command This bug has been fixed in Solaris 10 5 08 on XSCF might report wrong I O component For Solaris 10 8 07 or earlier this has been due to wrong diagnosis by Solaris Fault fixed in patch 125369 05 management Architecture when faults in I O devices occ
58. wlogs error v XSCF gt showstatus If you encounter any hardware abnormality of the XSCF contact a service engineer 34 SPARC Enterprise M8000 M9000 Servers Product Notes for XCP Version 1072 September 2008 18 19 Install Solaris 10 5 08 Use the setdomainmode 8 command enable the autoboot function of the domain For detail see the SPARC Enterprise M4000 M5000 M8000 M9000 Servers XSCF User s Guide The autoboot function is applied by a domain reboot Upgrading an On CMU SPARC64 VI to SPARC64 VIL or Adding SPARC64 VII to an Existing CMU to an Existing Domain Configured with SPARC64 VI When the Solaris OS version is Solaris 10 8 07 upgrade it to Solaris 10 5 08 or apply the mandatory patches described in Solaris OS Patch Information on page 2 To use SPARC64 VII processors apply the relevant patches to the software in use if necessary Log in to the XSCF on an account with platform administrative privileges Use the showstatus 8 command to confirm that a component in Faulted or Deconfigured status doesn t exist XSCF gt showstatus If no failures found the following message appears No failures found in System Initialization In case of other messages contact a service engineer before proceeding to the next step Turn off the power to a domain assigned the CMU of which you upgrade or add the CPU XSCF gt poweroff d domain_id Confirm that

Download Pdf Manuals

image

Related Search

Related Contents

MEDIA CENTER CONTROL – BEDIENUNGSANLEITUNG  電動丸(9000 Beast Master ZB) 取扱説明書 - Shimano    Analizador FTTx - PROLITE-77B  

Copyright © All rights reserved.
Failed to retrieve file