Home

Sun Blade T6300 Server Module Administration Guide

image

Contents

1. Storing FRU Information The setfru command enables you to store information on FRU PROMs For example you might store information identifving the server in which the FRUs have been installed w To Store Information in Available FRU PROMs e Atthe sc gt prompt type setfru c data 30 Sun Blade T6300 Server Module Administration Guide April 2007 APPENDIX A OpenBoot Configuration Variables TABLE A 1 describes the OpenBoot firmware configuration variables stored in non volatile memory on the system The OpenBoot configuration variables are printed here in the order in which they appear when you issue the showenv command TABLE A 1 OpenBoot Configuration Variables Stored on the System Processor Variable local mac address fcode debug scsi initiator id oem logo oem banner ansi terminal screen columns screen rows ttya rts dtr off ttva ignore cd Possible Values true true 0 15 true false false false false false false false Default Value true false false false true 80 34 false true Description If true network drivers use their own MAC address not the server MAC address If true includes name fields for plug in device FCodes SCSI ID of the Serial Attached SCSI controller If true uses custom OEM logo otherwise use Sun logo If true uses custom OEM banner If true enables ANSI terminal emulation
2. This chapter explains what the system console is describes the different ways of configuring it on your server and helps you understand its relation to the system controller Topics covered in this chapter include Communicating With the Server on page 1 m Accessing the System Controller on page 3 Switching Between the ALOM System Controller and the System Console on page 9 ALOM sc gt Prompt on page 10 OpenBoot ok Prompt on page 12 m System Console OpenBoot Configuration Variable Settings on page 17 Communicating With the Server To install your system software or to diagnose problems you need some way to interact at a low level with the system The system console is the facility for this interaction You use the system console to view messages and issue commands There can be only one system console per computer The system console must be accessed through the ALOM CMT system controller during initial system installation After installation you can configure the system console to accept input from and send output to different devices 2 What the Svstem Console Does The svstem console displavs status and error messages generated bv firmware based tests during system startup After running those tests you can enter special commands that affect the firmware and alter system behavior After the operating system is booted the system console displays UNIX system messages and acce
3. long off period 2 9 sec Steady on Steady state Status normal System or component functioning with no service actions required Slow blink A transitory temporary event is taking place for which direct proportional feedback is not needed or not feasible Controlling the Locator LED You control the Locator LED from the sc gt prompt or by the Locator button on the front of the server module V To Turn On the Locator LED From the ALOM System Controller Command Prompt e Type sc gt setlocator on V To Turn Off the Locator LED From the ALOM System Controller Command Prompt e Type sc gt setlocator off Chapter 2 Managing RAS Features and System Firmware 21 Y To Display the State of the Locator LED From the ALOM Svstem Controller Command Prompt e Tvpe sc gt showlocator Locator LED is on Note You do not need user permissions to use the setlocator and showlocator commands Automatic Svstem Recoverv Automatic Svstem Recoverv functionalitv enables the svstem to resume operation after experiencing certain nonfatal hardware faults or failures When ASR is enabled the svstem s firmware diagnostics automaticallv detect failed hardware components An autoconfiguring capabilitv designed into the system firmware enables the system to unconfigure failed components and to restore system operation As long as the system is capable of operating without the failed component the
4. Information Use the following procedure to retrieve information about the status of system components affected by ASR v To Obtain ASR Information e At the sc gt prompt type sc gt showcomponent In the showcomponent command output any devices marked disabled have been manually unconfigured using the system firmware The showcomponent command also lists devices that have failed firmware diagnostics and have been automatically unconfigured by the system firmware 26 Sun Blade T6300 Server Module Administration Guide April 2007 Unconfiguring and Reconfiguring Devices To support a degraded boot capabilitv the ALOM firmware provides the disablecomponent command which enables you to unconfigure system devices manually This command flags the specified device as disabled by creating an entry in the ASR database v To Unconfigure a Device Manually e At the sc gt prompt type sc gt disablecomponent asr key The asr key is one of the device identifiers from TABLE 2 5 Note The device identifiers are not case sensitive You can type them as uppercase or lowercase characters TABLE 2 5 Device Identifiers and Devices Device Identifiers Devices MB CMPcpu number Pstrand number CPU strand Number 0 31 MB PCIEa PCle leaf A pci 780 MB PCIEb PCIe leaf B pci 7c0 MB CMPO CHchannel number Rrank number Ddimm number DIMMs v To Reconfigure a Device Manually At the sc gt prompt ty
5. 0276 HTML Online Module Service Manual and PDF Safety and Sun Blade T6300 Server 820 0279 HTML Online compliance Module Safety and and PDF Compliance Manual ALOM Advanced Lights Out 819 7981 HTML Online Management ALOM CMT and PDF v1 3 Guide Preface xiii Documentation Support and Training Sun Function URL Documentation http www sun com documentation Support http www sun com support Training http www sun com training Third Party Web Sites Sun is not responsible for the availability of third party web sites mentioned in this document Sun does not endorse and is not responsible or liable for any content advertising products or other materials that are available on or through such sites or resources Sun will not be responsible or liable for any actual or alleged damage or loss caused by or in connection with the use of or reliance on any such content goods or services that are available on or through such sites or resources Sun Welcomes Your Comments Sun is interested in improving its documentation and welcomes your comments and suggestions You can submit your comments by going to http www sun com hwdocs feedback Please include the title and part number of your document with your feedback Sun Blade T6300 Server Module Administration Guide part number 820 0277 10 xiv Sun Blade T6300 Server Module Administration Guide April 2007 CHAPTER 1 Configuring the System Console
6. ASR features enable the system to reboot automatically without operator intervention Note ASR is not activated until you enable it See Enabling and Disabling Automatic System Recovery on page 25 AutoBoot Options The system firmware stores a configuration variable called auto boot7 which controls whether the firmware automatically boots the operating system after each reset The default setting for Sun platforms is true Normally if a system fails power on diagnostics auto boot is ignored and the system does not boot unless an operator boots the system manually An automatic boot is generally not acceptable for booting a system in a degraded state Therefore the server s OpenBoot firmware provides a second setting auto boot on error This setting controls whether the system will attempt a degraded boot when a subsystem failure is detected Both the auto boot7 and auto boot on error switches must be set to true to enable an automatic degraded boot 22 Sun Blade T6300 Server Module Administration Guide April 2007 V To Enable an Automatic Degraded Boot Set the switches by typing ok setenv auto boot true ok setenv auto boot on error true Note The default setting for auto boot on error is false The system will not attempt a degraded boot unless you change this setting to true In addition the system will not attempt a degraded boot in response to any fatal nonrecoverable error even if degrad
7. See the following for this information a Software documentation that you received with your system m Solaris OS documentation which is at http docs sun com xii Shell Prompts Shell Prompt C shell C shell superuser Bourne shell and Korn shell Bourne shell and Korn shell superuser machine name machine namet Sun Blade T6300 Server Module Administration Guide April 2007 Typographic Conventions Tvpeface Meaning Examples AaBbCc123 The names of commands files Edit vour login file and directories on screen Use 1s a to list all files computer output You have mail AaBbCc123 What you type when contrasted su with on screen computer output password AaBbCc123 Book titles new words or terms Read Chapter 6 in the User s Guide words to be emphasized These are called class options Replace command line variables You must be superuser to do this with real names or values To delete a file type rm filename The settings on your browser might differ from these settings Related Documentation The documents listed as online are available at http www sun com documentation Application Title Part Number Format Location Late breaking news Sun Blade T6300 Server 820 0278 HTML Online Module Product Notes and PDF Installation Sun Blade T6300 Server 820 0275 HTML Online Module Installation Guide and PDF Service Sun Blade T6300 Server 820
8. Standbv blink The svstem is functioning at a minimal level and readv to resume full function Slow blink Transitorv activitv or new activitv represented bv the color is taking place Fast blink Attention is required Feedback flash Activitv is taking place commensurate with the flash rate such as disk drive activitv The LEDs have assigned meanings described in TABLE 2 2 TABLE 2 2 LED Behaviors With Assigned Meanings Color Behavior Definition Description White Off Steady state Fast blink 4 Hz repeating This indicator helps you to locate a particular sequence equal enclosure board or subsystem for example the intervals on Locator LED and off Blue Off Steady state Steady on Steady state If blue is on a service action can be performed on the applicable component with no adverse consequences for example the OK to Remove LED Yellow or Off Steady state Amber Steady on Steady state This indicator signals the existance of a fault condition Service is required for example the Service Required LED Green Off Steady state 20 Sun Blade T6300 Server Module Administration Guide April 2007 TABLE 2 2 LED Behaviors With Assigned Meanings Continued Color Behavior Definition Description Standbv blink Repeating The svstem is running at a minimum level and is sequence ready to be quickly revived to full function for consisting ofa example the System Activity LED brief 0 1 sec on flash followed by a
9. licence et sont des marques de fabrique ou des marques d pos es de SPARC International Inc aux Etats Unis et dans d autres pays Les produits portant les marques SPARC sont bas s sur une architecture d velopp e par Sun Microsystems Inc L interface d utilisation graphique OPEN LOOK et Sun M a t d velopp e par Sun Microsystems Inc pour ses utilisateurs et licenci s Sun reconna t les efforts de pionnier de Xerox pour la recherche et le d veloppement du concept des interfaces d utilisation visuelle ou graphique pour l industrie de l informatique Sun d tient une license non exclusive de Xerox sur l interface d utilisation graphique Xerox cette licence couvrant galement les licenci es de Sun qui mettent en place l interface d utilisation graphique OPEN LOOK et qui en outre se conforment aux licences crites de Sun LA DOCUMENTATION EST FOURNIE EN L TAT ET TOUTES AUTRES CONDITIONS DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE A L APTITUDE A UNE UTILISATION PARTICULIERE OU A L ABSENCE DE CONTREFA ON SA CA Adobe PostScript Contents Preface xi Configuring the Svstem Console 1 Communicating With the Server 1 What the Svstem Console Does 2 What the ALOM Svstem Controller Console Does 2 Using the Svstem Console 2 Default System Console Connection Through the S
10. on the chassis refer to the Sun Blade T6000 Chassis Installation Guide for instructions Accessing the System Console Through a Terminal Server The following procedure assumes that you are accessing the ALOM system controller by connecting a terminal server to the serial port 4 Sun Blade T6300 Server Module Administration Guide April 2007 Y To Access The System Console Through a Terminal Server 1 Complete the phvsical connection from the serial port to vour terminal server The serial port on the server is a data terminal equipment DTE port The pinouts for the serial port correspond with the pinouts for the RJ 45 ports on the serial interface breakout cable supplied by Cisco for use with the Cisco AS2511 RJ terminal server If you use a terminal server made by another manufacturer check that the serial port pinouts of the server match those of the terminal server you plan to use m If the pinouts for the server serial ports correspond with the pinouts for the RJ 45 ports on the terminal server vou have two connection options Connect a serial interface breakout cable to the dongle cable See Accessing the Svstem Controller on page 3 a Connect a serial interface breakout cable to a patch panel and use a straight through patch cable to connect the patch panel to the server m If the pinouts for the serial port do not correspond with the pinouts for the RJ 45 ports on the terminal server you must make a crossover cable t
11. the serial port m Terminal server Alphanumeric terminal or similar device a TIP line connected to another Sun computer Sun Blade T6300 Server Module Administration Guide April 2007 These constraints provide for secure access at the installation site Using a TIP line enables vou to use windowing and operating svstem features on the system making the connection to your server m For instructions on accessing the system console through a terminal server see Accessing the System Console Through a Terminal Server on page 4 m For instructions on accessing the system console through an alphanumeric terminal see Accessing the System Console Through an Alphanumeric Terminal on page 8 m For instructions on accessing the system console through a TIP line see Accessing the System Console Through a TIP Connection on page 6 After the network management port has been assigned an IP address by a DHCP server you can connect to the ALOM system controller using Secure Shell SSH As an alternative to the default DHCP configuration you can configure the network management port with a static IP address and change the communication protocol from SSH to Telnet Up to eight simultaneous connections to the system controller sc gt prompt are available through the network management port For more information see Activating the Network Management Port on page 4 Accessing the System Controller This section describ
12. vou issue an administrative command you must create a password for the default username admin for subsequent access After this initial configuration vou will be prompted to enter a user name and password everv time vou access the ALOM svstem controller For more information about navigating between the svstem console and the ALOM svstem controller see Obtaining the ok Prompt on page 15 Access Through Multiple Controller Sessions Up to nine ALOM svstem controller sessions can be active concurrentiv one session through the serial port and up to eight sessions through the network management port Users of each of these sessions can issue commands at the sc gt prompt For more information see the following m Accessing the System Controller on page 3 a Activating the Network Management Port on page 4 Note Only one user has active control of the system console at any time Any additional ALOM system controller sessions afford passive views of system console activity until the active user of the system console logs out However the console f command enables users to seize access to the system console from one another For more information see the Advanced Lights Out Management ALOM CMT v1 3 Guide Chapter 1 Configuring the System Console 11 Reaching the sc gt Prompt There are several ways to obtain the sc gt prompt m You can log in directly to the ALOM system controller from a device connected to the
13. 2 contain an etc remote file with the appropriate hardwire entrv However if the Sun svstem is running an older version of Solaris OS software or if the etc remote file has been modified you might need to edit the file See Modifying the etc remote File on page 7 for details 3 In a shell tool window on the Sun system type tip hardwire The Sun system responds by displaving connected The shell tool is now a TIP window directed to the Sun Blade T6300 server module through the Sun svstem s serial port This connection is established and maintained even when the server is completelv powered off or just starting up Note Use a shell tool or a CDE terminal such as dtterm not a command tool Some TIP commands might not work properlv in a command tool window 6 Sun Blade T6300 Server Module Administration Guide April 2007 Modifving the etc remote File This procedure might be necessarv if vou are accessing the server module using a TIP connection from a Sun svstem running an older version of the Solaris OS software You might also need to perform this procedure if the etc remote file on the Sun svstem has been altered and no longer contains an appropriate hardwire entry Log in as superuser to the system console of a Sun system that you intend to use to establish a TIP connection to the server module To Modify the etc remote File Determine the release level of Solaris OS softw
14. SN R SUN microsystems Sun Blade T6300 Server Module Administration Guide Sun Microsystems Inc www sun com Part No 820 0277 10 April 2007 Revision A Submit comments about this document at http www sun com hwdocs feedback Copyright 2007 Sun Microsystems Inc 4150 Network Circle Santa Clara California 95054 U S A All rights reserved Sun Microsvstems Inc has intellectual propertv rights relating to Pl INC fiat is described in this document In particular and without limitation these intellectual property rights may include one or more of the U S patents listed at http www sun com patents and one or more additional patents or pending patent applications in the U S and in other countries This document and the product to which it pertains are distributed under licenses restricting their use copving distribution and decompilation No part of the product or of this document may be reproduced in any form by any means without prior written authorization of Sun and its licensors if any Third party software including font technology is copyrighted and licensed from Sun suppliers Parts of the s produet may be derived from Berkeley BSD systems licensed from the University of California UNIX is a registered trademark in the U S and in other countries exclusively licensed through X Open Company Ltd Sun Sun Microsystems Sun Blade the Sun logo docs sun com Sun StorageTek OpenBoot and Solaris are trade
15. Sets number of columns on screen Sets number of rows on screen If true operating system does not assert rts request to send and dtr data transfer ready on serial management port If true operating system ignores carrier detect on serial management port 31 TABLE A 1 OpenBoot Configuration Variables Stored on the System Processor Continued Variable Possible Values Default Value Description ttya mode 9600 8 n 1 9600 8 n 1 Serial management port baud rate bits parity stop handshake The serial management port only works at the default values output device virtual virtual Power on output device console ttva console input device virtual virtual Power on input device console ttva console auto boot on error true false false If true boost automatically after system error load base O n 16384 Address auto boot true false true If true boots automatically after power on or reset boot command variable name boot Action following a boot command boot file variable name none File from which to boot if diag switch7 is false boot device variable name disk net Devices from which to boot if diag switch is false use nvramrc true false false If true executes commands in NVRAMRC during server startup nvramrc variable name none Command script to execute if use nvramrc is true security mode none command No default Firmware security level full security password variable name No default Firmware securit
16. ablecomponent 27 alphanumeric terminal accessing system console from 8 setting baud rate 8 auto boot OpenBoot configuration variable 12 22 Automatic Svstem Recoverv ASR about 22 commands 24 disabling 26 enabling 25 obtaining recoverv information 26 B break sc gt command 14 Break kev alphanumeric terminal 16 C Cisco L2511 terminal server connecting 5 command prompts explained 10 communicating with the system about 1 console sc gt command 14 console configuration connection alternatives explained 10 console f sc gt command 11 D device identifiers 27 reconfiguration 27 unconfiguration 27 disablecomponent ALOM command 27 dtterm Solaris utility 6 E enablecomponent ALOM command 27 error handling summarv 23 escape sequence ALOM system controller 12 F sck Solaris command 15 G go OpenBoot command 15 graceful system halt 13 16 H halt graceful advantages of 13 16 init Solaris command 13 16 35 input device OpenBoot configuration variable 17 K kevboard sequences L1 A 13 14 16 L L1 A keyboard sequence 13 14 16 LEDs Locator system status LED 21 system interpreting 20 Locator system status LED controlling 21 controlling from sc gt prompt 21 22 M manual device reconfiguration 27 manual device unconfiguration 27 manual system reset 14 16 multiple ALOM sessions 11 N network management por
17. are installed on the Sun system Type uname r The system responds with a release number Do one of the following depending on the number displayed a If the number displayed by the uname r command is 5 0 or higher The Solaris OS software shipped with an appropriate entry for hardwire in the etc remote file If you suspect that this file was altered and the hardwire entry modified or deleted check the entry against the following example and edit the file as needed hardwire dv dev term b br 9600 el C S Q U D ie 0e D Note If you intend to use the Sun system s serial port A rather than serial port B edit this entry by replacing dev term b with dev term a m If the number displayed by the uname r command is less than 5 0 Check the etc remote file and add the following entry if it does not already exist hardwire dv dev ttyb br 9600 el C S Q0 U D ie 0e D Chapter 1 Configuring the System Console 7 Note If vou intend to use the Sun svstem s serial port A rather than serial port B edit this entry by replacing dev ttyb with dev ttva If you have redirected the system console to TTYB and want to change the system console settings back to use the serial port and network management ports see System Console OpenBoot Configuration Variable Settings on page 17 Accessing the System Console Through an Alphanumeric Terminal Use this pro
18. ariable is set to false the system boots to the ok prompt When the operating system is halted the system transitions to run level 0 in an orderly way When the operating system crashes the system reverts to OpenBoot firmware control m During the boot process when there is a serious hardware problem that prevents the operating system from running the system reverts to OpenBoot firmware control 12 Sun Blade T6300 Server Module Administration Guide April 2007 When a serious hardware problem develops while the system is running the operating system transitions smoothly to run level 0 When you deliberately place the system under firmware control in order to execute firmware based commands The last of these scenarios most often concerns you as an administrator since there will be times when you need to reach the ok prompt The section Methods to Reach the ok Prompt on page 13 lists several ways For detailed instructions see Obtaining the ok Prompt on page 15 Methods to Reach the ok Prompt There are several ways to reach the ok prompt depending on the state of the system and the means by which you are accessing the system console Note These methods of reaching the ok prompt work only if the system console has been redirected to the appropriate port For details see System Console OpenBoot Configuration Variable Settings on page 17 The methods are Graceful shutdown ALOM system controll
19. ate path to maintain availability This capability is known as automatic failover To take advantage of multipathing capabilities you must configure the server with redundant hardware such as redundant network interfaces or two host bus adapters connected to the same dual ported storage array For your server three different types of multipathing software are available m Solaris IP Network Multipathing software provides multipathing and load balancing capabilities for IP network interfaces a VERITAS Volume Manager VVM software includes a feature called Dynamic Multipathing DMP which provides disk multipathing as well as disk load balancing to optimize I O throughput m Sun StorageTek Traffic Manager is an architecture fully integrated within the Solaris OS beginning with the Solaris 8 release that enables I O devices to be accessed through multiple host controller interfaces from a single instance of the I O device For More Information on Multipathing Software For instructions on how to configure and administer Solaris IP Network Multipathing software consult the IP Network Multipathing Administration Guide provided with vour specific Solaris release Chapter 2 Managing RAS Features and System Firmware 29 For information about VVM and its DMP feature refer to the documentation provided with the VERITAS Volume Manager software For information about Sun StorageTek Traffic Manager refer to vour Solaris OS documentation
20. cedure when you access the system console by connecting the serial port of an alphanumeric terminal to the serial port of the server Y To Access the System Console Through an Alphanumeric Terminal 1 Attach one end of the serial cable to the alphanumeric terminal s serial port Use a null modem serial cable or an RJ 45 serial cable and null modem adapter Connect this cable to the terminal s serial port connector 2 Attach the opposite end of the serial cable to the serial port on the dongle cable 3 Connect the alphanumeric terminal s power cord to an AC outlet 4 Set the alphanumeric terminal to receive m 9600 baud a 8 bits No parity m 1 stop bit No handshake protocol Refer to the documentation accompanying your terminal for information about how to configure the terminal You can issue system commands and view system messages using the alphanumeric terminal Continue with your installation or diagnostic procedure as needed When you are finished type the alphanumeric terminal s escape sequence For more information about connecting to and using the ALOM system controller refer to the Advanced Lights Out Management ALOM CMT v1 3 Guide 8 Sun Blade T6300 Server Module Administration Guide April 2007 Switching Between the ALOM Svstem Controller and the Svstem Console If the svstem console is directed to the virtual console device its default configuration the serial port and the network manageme
21. ed booting is enabled For examples of fatal nonrecoverable errors see Error Handling Summary on page 23 Error Handling Summary Error handling during the power on sequence falls into one of the following three cases a If no errors are detected by POST or OpenBoot Diagnostics the system attempts to boot if auto boot is true If only nonfatal errors are detected by POST or OpenBoot Diagnostics the system attempts to boot if auto boot7 is true and auto boot on error is true Nonfatal errors include the following a Ethernet interface failure a Serial interface failure PCI Express card failure Memory failure When a DIMM fails the firmware unconfigures the entire logical bank associated with the failed server module Another nonfailing logical bank must be present in the system for the system to attempt a degraded boot Note that certain DIMM failures might not be diagnosable to a single DIMM These failures are fatal and result in both logical banks being unconfigured Note If POST or OpenBoot Diagnostics detect a nonfatal error associated with the normal boot device the OpenBoot firmware automatically unconfigures the failed device and tries the next in line boot device as specified by the boot device configuration variable Chapter 2 Managing RAS Features and System Firmware 23 a Ifa fatal error is detected by POST or OpenBoot Diagnostics the system does not boot regardless of the settings of auto boot
22. er break and console command pair L1 A Stop A keys or Break key Manual system reset A discussion of each method follows For step by step instructions see Obtaining the ok Prompt on page 15 Note As a rule before suspending the operating system you should back up files warn users of the impending shutdown and halt the system in an orderly manner However it is not always possible to take such precautions especially if the system is malfunctioning Graceful Shutdown The preferred method of reaching the ok prompt is to shut down the operating system by issuing an appropriate command for example the shutdown init or uadmin command as described in Solaris system administration documentation You can also use the system Power button to initiate a graceful system shutdown Chapter 1 Configuring the System Console 13 Gracefullv shutting down the svstem prevents data loss enables vou to warn users beforehand and causes minimal disruption Xou can usuallv perform a graceful shutdown provided the Solaris OS is running and the hardware has not experienced serious failure ALOM System Controller break or console Command Typing break from the sc gt prompt forces a running server to drop into OpenBoot firmware control If the operating system is already halted you can use the console command instead of break to reach the ok prompt Caution After forcing the system into OpenBoot firmware control be aware that is
23. erial Port and Network Management Ports 2 Accessing the System Controller 3 Using the Serial Port 3 v To Use the Serial Port 4 Activating the Network Management Port 4 Accessing the System Console Through a Terminal Server 4 v To Access The System Console Through a Terminal Server 5 Accessing the System Console Through a TIP Connection 6 v To Access the System Console Through a TIP Connection 6 Modifying the etc remote File 7 v To Modify the etc remote File 7 Accessing the System Console Through an Alphanumeric Terminal 8 v To Access the System Console Through an Alphanumeric Terminal Switching Between the ALOM System Controller and the System Console 9 ALOM sc gt Prompt 10 Access Through Multiple Controller Sessions 11 Reaching the sc gt Prompt 12 OpenBoot ok Prompt 12 Methods to Reach the ok Prompt 13 Graceful Shutdown 13 ALOM System Controller break or console Command 14 L1 A Stop A Keys or Break Key 14 Manual System Reset 14 For More Information About OpenBoot Firmware 15 Obtaining the ok Prompt 15 v To Obtain the ok Prompt 16 System Console OpenBoot Configuration Variable Settings 17 2 Managing RAS Features and System Firmware 19 Interpreting System LEDs 20 Controlling the Locator LED 21 v To Turn On the Locator LED From the ALOM System Controller Command Prompt 21 v To Turn Off the Locator LED From the ALOM System Controller Command Prompt 21 v To Display the State of the Locator LED From the ALOM System C
24. es ways of accessing the system controller Using the Serial Port When you are accessing the ALOM system controller using a device connected to the serial port you will see the output of the ALOM diagnostics when AC power is first applied or when ALOM resets After the diagnostics have completed the serial port is available for login For more information about the ALOM system controller card refer to the Advanced Lights Out Management ALOM CMT v1 3 Guide Chapter 1 Configuring the System Console 3 w To Use the Serial Port 1 Ensure that the serial port on vour connecting device is set to the following parameters 9600 baud 8 bits No parity 1 stop bit No handshaking 2 Establish an ALOM system controller session For instructions on how to use the ALOM system controller see the Advanced Lights Out Management ALOM CMT v1 3 Guide Activating the Network Management Port The network management port is configured on the chassis to retrieve network settings using DHCP and to allow connections using SSH You might need to modify these settings for your network If you are unable to use DHCP and SSH on your network you must connect to the ALOM system controller using the serial port Note There is no default password when connecting to the ALOM system controller for the first time You must assign a password during initial system configuration For instructions on how to activate the network management port
25. hat takes each pin on the serial port to the corresponding pin in the terminal server s serial port TABLE 1 1 shows the crossovers that the cable must perform TABLE 1 1 Pin Crossovers for Connecting the Server to a Terminal Server Dongle Serial Port RJ 45 Connector Pin Terminal Server Serial Port Pin Pin 1 RTS Pin 1 CTS Pin 2 DTR Pin 2 DSR Pin 3 TXD Pin 3 RXD Pin 4 Signal Ground Pin 4 Signal Ground Pin 5 Signal Ground Pin 5 Signal Ground Pin 6 RXD Pin 6 TXD Pin 7 DSR DCD Pin 7 DTR Pin 8 CTS Pin 8 RTS 2 Open a terminal session on the connecting device and type telnet IP address of terminal server port number Chapter 1 Configuring the System Console 5 For example for a server connected to port 10000 on a terminal server whose IP address is 192 20 30 10 vou would tvpe telnet 192 20 30 10 10000 Accessing the Svstem Console Through a TIP Connection Use this procedure to access the server svstem console bv connecting the dongle serial port to the serial port of another Sun svstem W To Access the Svstem Console Through a TIP Connection 1 Connect the RJ 45 serial cable and if required a DB 9 or DB 25 adapter The cable and adapter connect between another server s serial port tvpicallv TTXB and the dongle serial port 2 Ensure that the etc remote file on the Sun svstem contains an entrv for hardwire Most releases of Solaris OS software shipped since 199
26. l You can obtain an online version of the manual at http www sun com documentation Obtaining the ok Prompt This procedure provides several ways of reaching the ok prompt For details about when to use each method see OpenBoot ok Prompt on page 12 Chapter 1 Configuring the System Console 15 Caution Forcing the server to the ok prompt suspends all application and operating system software After you issue firmware commands and run firmware based tests from the ok prompt the system might not be able to resume where it left off If at all possible back up system data before starting this procedure Also exit or stop all applications and warn users of the impending loss of service For information about the appropriate backup and shutdown procedures see Solaris system administration documentation v To Obtain the ok Prompt 1 Decide which method you need to use to reach the ok prompt See OpenBoot ok Prompt on page 12 for details 2 Follow the appropriate instructions in TABLE 1 2 TABLE 1 2 Ways of Accessing the ok Prompt Access Method What to Do Graceful shutdown of From a shell or command tool window issue an appropriate the Solaris OS command for example the shutdown or init command as described in Solaris system administration documentation L1 A Stop A keys or From a Sun keyboard connected to the dongle cable press the Break key Stop and A keys simultaneously From an alpha
27. marks or registered trademarks of Sun Microsystems Inc in the U S and in other countries All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc in the U S and in other countries Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems Inc The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems Inc for its users and licensees Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry Sun holds a non exclusive license from Xerox to the Xerox Graphical User Interface which license also covers Sun s licensees viho implement OPEN LOOK GUIs and otherwise comply with Sun s written license agreements U S Government Rights Commercial use Government users are subject to the Sun Microsystems Inc standard license agreement and applicable provisions of the FAR and its supplements DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Copyright 2007 Sun Microsystems Inc 4150 Network Circle Santa Clara Californie 95054 Etats Unis Tous droits r serv s Sun Microsystem
28. might be necessary to issue the escape sequence Hash Period to return to ALOM To reach the system console from the ALOM system controller m Type the console command at the sc gt prompt To reach the ALOM system controller from the system console m Type the system controller escape sequence By default the escape sequence is Hash Period For more information about communicating with the ALOM system controller and system console see the following Communicating With the Server on page 1 ALOM sc gt Prompt on page 10 OpenBoot ok Prompt on page 12 Accessing the System Controller on page 3 Advanced Lights Out Management ALOM CMT v1 3 Guide ALOM sc gt Prompt The ALOM system controller runs independently of the server module and regardless of system power state When you install the server module the ALOM system controller immediately starts up and begins monitoring the system You can log in to the ALOM system controller at any time regardless of system power state as long as you have a way of interacting with the system The sc gt prompt indicates that you are interacting with the ALOM system controller directly The sc gt prompt is the first prompt you see when you log in to the system through the serial port or network management port 10 Sun Blade T6300 Server Module Administration Guide April 2007 Note When vou access the ALOM svstem controller for the first time and
29. nt port provide access to both the svstem console and the ALOM svstem controller see FIGURE 1 1 Network management or Serial management port mi lt console pen System console ALOM system controller FIGURE 1 1 Switching Between ALOM System Controller and System Console If the system console is configured to use the virtual console device when you connect through one of these ports you can access either the ALOM command line interface or the system console You can switch between the ALOM system controller and the system console at any time but you cannot access both at the same time from a single terminal or shell tool Chapter 1 Configuring the System Console 9 The prompt displaved on the terminal or shell tool indicates which channel vou are accessing m The or prompt indicates that you are at the system console and that the Solaris OS is running m The ok prompt indicates that you are at the system console and that the server is running under OpenBoot firmware control The sc gt prompt indicates that you are at the ALOM system controller Note If no text or prompt appears it might be that no console messages were recently generated by the system If this happens pressing the terminal s Enter or Return key should produce a prompt If the ALOM session has timed out pressing the terminal s Enter or Return key might not be effective In that case it
30. numeric terminal configured to access the system console press the Break key ALOM system 1 From the sc gt prompt type the break command The break controller break and command should put the system in a state in which the console commands operating system software is not running and the server is under OpenBoot firmware control 2 Issue the console command Manual system reset e From the sc gt prompt type the reset command Requires the OpenBoot configuration variable input device keyboard For more information see System Console OpenBoot Configuration Variable Settings on page 17 16 Sun Blade T6300 Server Module Administration Guide April 2007 Svstem Console OpenBoot Configuration Variable Settings Certain OpenBoot configuration variables control from where svstem console input is taken and to where its output is directed The table below shows how to set these variables in order to use the serial port and network management port TABLE 1 3 OpenBoot Configuration Variables That Affect the System Console OpenBoot Configuration Variable Name Serial and Network Management Ports output device virtual console input device virtual console Note The sc gt prompt and POST messages are only available through the serial management port and network management port In addition to the OpenBoot configuration variables described in TABLE 1 3 there are other variables that affect and determine s
31. ontroller Command Prompt 22 Automatic System Recovery 22 AutoBoot Options 22 v To Enable an Automatic Degraded Boot 23 Error Handling Summary 23 Reset Scenarios 24 iv Sun Blade T6300 Server Module Administration Guide April 2007 8 Automatic Svstem Recoverv User Commands 24 Enabling and Disabling Automatic System Recovery 25 v To Enable Automatic System Recovery 25 v To Disable Automatic System Recovery 26 Obtaining Automatic System Recovery Information 26 v To Obtain ASR Information 26 Unconfiguring and Reconfiguring Devices 27 v To Unconfigure a Device Manually 27 v To Reconfigure a Device Manually 27 Displaying System Fault Information 28 v To Display System Fault Information 28 Multipathing Software 29 For More Information on Multipathing Software 29 Storing FRU Information 30 v To Store Information in Available FRU PROMs 30 OpenBoot Configuration Variables 31 Contents v vi Sun Blade T6300 Server Module Administration Guide April 2007 FIGURE 1 1 Figures Switching Between ALOM Svstem Controller and Svstem Console 9 vii viii Sun Blade T6300 Server Module Administration Guide April 2007 Tables TABLE 1 1 TABLE 1 2 TABLE 1 3 TABLE 2 1 TABLE 2 2 TABLE 2 3 TABLE 2 4 TABLE 2 5 Pin Crossovers for Connecting the Server to a Terminal Server 5 Wavs of Accessing the ok Prompt 16 OpenBoot Configuration Variables That Affect the System Console 17 LED Behavior and Meaning 20 LED Behavio
32. or auto boot on error Fatal nonrecoverable errors include the following a Any CPU failure Any logical memory bank failure a Flash RAM cyclical redundancy check CRC failure a Critical field replaceable unit FRU PROM configuration data failure Critical system configuration SEEPROM read failure Critical application specific integrated circuit ASIC failure Reset Scenarios Three ALOM configuration variables diag_mode diag_level and diag_trigger control whether the system runs firmware diagnostics in response to system reset events The standard system reset protocol bypasses POST completely unless the virtual keyswitch or ALOM variables are set as follows TABLE 2 3 Virtual Keyswitch Setting for Reset Scenario Keyswitch Value virtual keyswitch diag TABLE 2 4 ALOM Variable Settings for Reset Scenario Variable Value Default diag_mode normal or service normal diag_level min or max min diag trigger power on reset error reset power on reset Therefore ASR is enabled by default For instructions see Enabling and Disabling Automatic System Recovery on page 25 Automatic System Recovery User Commands ALOM commands are available for enabling and disabling ASR and for obtaining ASR status information For more information see 24 Sun Blade T6300 Server Module Administration Guide April 2007 Unconfiguring and Reconfiguring Devices on page 27 m To Reconfigure a Device Man
33. pe sc gt enablecomponent asr key where asr key is any device identifier from TABLE 2 5 Chapter 2 Managing RAS Features and System Firmware 27 Note The device identifiers are not case sensitive Vou can tvpe them as uppercase or lowercase characters Xou can use the ALOM enablecomponent command to reconfigure anv device that you previously unconfigured with the disablecomponent command Displaying System Fault Information ALOM software enables you to display current valid system faults The showfaults command displays the fault ID the faulted FRU device and the fault message to standard output The showfaults command also displays POST results v To Display System Fault Information e Type showfaults For example sc gt showfaults ID FRU Fault 0 FTO F2 SYS FAN at FTO F2 has FAILE 28 Sun Blade T6300 Server Module Administration Guide April 2007 Adding the v option displavs additional information sc gt showfaults v ID Time FRU Fault 0 MAY 20 10 47 32 FTO F2 SYS_FAN at FTO F2 has FAILED For more information about the showfaults command refer to the Advanced Lights Out Management ALOM CMT v1 3 Guide Multipathing Software Multipathing software enables you to define and control redundant physical paths to I O devices such as storage devices and network interfaces If the active path to a device becomes unavailable the software can automatically switch to an altern
34. pts UNIX commands You can access the system console using the ALOM console command What the ALOM System Controller Console Does The ALOM system controller displays the results of the ALOM boot diagnostics and initialization If it receives no user input within 60 seconds the ALOM system controller console automatically connects to the system console To return to the ALOM system controller type the console escape sequence Hash Period Using the System Console To use the system console you need to attach an I O device to the server module or the chassis Initially you might have to configure that hardware and load and configure appropriate software as well You must also ensure that the system console is directed to the appropriate port generally the one to which your hardware console device is attached You do this by setting the input device and output device OpenBoot configuration variables Default System Console Connection Through the Serial Port and Network Management Ports On your server the system console comes preconfigured to allow input and output only by means of the ALOM CMT system controller The ALOM CMT system controller must be accessed either through the serial port or the network management port By default the network management port is configured to retrieve network configuration using DHCP and to allow connections using Secure Shell SSH Typically you connect one of the following hardware devices to
35. rs With Assigned Meanings 20 Virtual Keyswitch Setting for Reset Scenario 24 ALOM Variable Settings for Reset Scenario 24 Device Identifiers and Devices 27 x Sun Blade T6300 Server Module Administration Guide April 2007 Preface The Sun Blade T6300 Server Module Administration Guide is written for experienced svstem administrators This guide includes general descriptive information about the Sun Blade T6300 server module and detailed instructions for configuring and administering the server module To use the information in this guide vou must have working knowledge of computer network concepts and terms and advanced familiaritv with the SolarisTM Operating Svstem Solaris OS How This Book Is Organized This guide is divided into the following chapters m Chapter 1 describes the system console and how to access it m Chapter 2 describes the tools used to configure system firmware including Sun Advanced Lights Out Manager ALOM CMT system controller environmental monitoring Automatic System Recovery ASR and multipathing software In addition this chapter describes how to unconfigure and reconfigure a device manually m Appendix A provides a list of all OpenBoot configuration variables and a short description of each xi Using UNIX Commands This document might not contain information on basic UNIX commands and procedures such as shutting down the system booting the system and configuring devices
36. rver s file svstems although the fsck command usually restores them Use this method only when nothing else works Caution Accessing the ok prompt suspends the Solaris OS When you access the ok prompt from a functioning server you are suspending the Solaris OS and placing the system under firmware control Any processes that were running under the operating system are also suspended and the state of such processes might not be recoverable After a manual system reset the system can be configured to boot automatically if the OpenBoot auto boot configuration variable is set to true See System Console OpenBoot Configuration Variable Settings on page 17 If the server begins to boot automatically after a reset you must abort the boot with the ALOM system controller break command or perform a graceful shutdown of the Solaris Operating System after the boot has completed The commands you run from the ok prompt have the potential to affect the state of the system This means that it is not always possible to resume execution of the operating system from the point at which it was suspended Although the go command will resume execution in most circumstances in general each time you obtain the ok prompt you should expect to have to reboot the system to get back to the operating system For More Information About OpenBoot Firmware For more information about the OpenBoot firmware refer to the OpenBoot 4 x Command Reference Manua
37. s Inc a les droits de propriete intellectuels relatants la technologie qui est d crit dans ce document En particulier et sans la limitation ces droits de propri t intellectuels peuvent inclure un ou plus des brevets am ricains num r s a http www sun com patents et un ou les brevets plus suppl mentaires ou les applications de brevet en attente dans les Etats Unis et dans les autres pays Ce produit ou document est prot g par un copyright et distribu avec des licences qui en restreignent l utilisation la copie la distribution et la d compilation Aucune partie de ce produit ou document ne peut tre reproduite sous aucune forme par quelque moyen que ce soit sans l autorisation pr alable et crite de Sun et de ses bailleurs de licence s il y ena Le logiciel d tenu par des tiers et qui comprend la technologie relative aux polices de caract res est prot g par un copyright et licenci par des fournisseurs de Sun Des parties de ce produit pourront tre d riv es des syst mes Berkeley BSD licenci s par l Universit de Californie UNIX est une marque d pos e aux Etats Unis et dans d autres pays et licenci e exclusivement par X Open Company Ltd Sun Sun Microsystems Sun Blade le logo Sun docs sun com Sun StorageTek OpenBoot et Solaris sont des marques de fabrique ou des marques d pos es de Sun Microsystems Inc aux Etats Unis et dans d autres pays Toutes les marques SPARC sont utilis es sous
38. serial management port SERIAL MGT acceptable console device connections 2 36 Sun Blade T6300 Server Module Administration Guide April 2007 as default communication port on initial startup 1 configuration parameters 4 using 3 SERIAL MGT See serial management port setlocator sc gt command 21 showenv OpenBoot command 31 shutdown Solaris command 13 16 Solaris commands fsck 15 init 13 16 shutdown 13 16 tip 6 uadmin 13 uname 7 uname r 7 suspending the operating system software 15 system console accessing through alphanumeric terminal 8 accessing through terminal server 4 accessing through TIP connection 6 accessing via TIP connection 6 alphanumeric terminal connection 8 default configuration explained 1 default connections 2 defined 1 multiple view sessions 11 sc gt prompt switching between 9 setting OpenBoot configuration variables for 17 system reset scenarios 24 system status LEDs interpreting 20 Locator 21 T terminal server accessing system console from 2 4 connection through patch panel 5 pinouts for crossover cable 5 tip Solaris command 6 TIP connection accessing system console 6 accessing terminal server 6 U uadmin Solaris command 13 uname Solaris command 7 uname r Solaris command 7 Index 37 38 Sun Blade T6300 Server Module Administration Guide April 2007
39. serial port See Accessing the System Controller on page 3 m You can log in directly to the ALOM system controller using a connection through the network management port See Activating the Network Management Port on page 4 m If you have logged in to the ALOM system controller and then directed the system console to the serial port and network management ports you can return to the prior ALOM session by typing the ALOM system controller escape sequence OpenBoot ok Prompt The server with the Solaris OS installed is capable of operating at different run levels A synopsis of run levels follows For a full description of run levels refer to the Solaris system administration documentation Most of the time you operate the server at run level 2 or run level 3 which are multiuser states with access to full system and network resources Occasionally you might operate the system at run level 1 which is a single user administrative state However the lowest operational state is run level 0 At this state it is safe to turn off power to the system When your server is at run level 0 the ok prompt appears This prompt indicates that the OpenBoot PROM firmware is in control of the system There are a number of scenarios under which OpenBoot firmware control can occur m By default before the operating system is installed the system comes up under OpenBoot firmware control When the auto boot OpenBoot configuration v
40. suing certain OpenBoot commands such as probe scsi probe scsi all or probe ide might hang the system L1 A Stop A Keys or Break Key When it is impossible or impractical to shut down the system gracefully you can get to the ok prompt by typing the L1 A Stop A key sequence from a keyboard connected to the server that is if OpenBoot input device kevboard If you have an alphanumeric terminal attached to the server press the Break key Caution After forcing the system into OpenBoot firmware control be aware that issuing certain OpenBoot commands such as probe scsi probe scsi all or probe ide might hang the system Manual System Reset This section explains how to execute a manual reset and what happens when a manual reset occurs Caution Forcing a manual system reset results in loss of system state data and should be attempted only as a last resort After a manual system reset all state information is lost which inhibits troubleshooting the cause of the problem until the problem reoccurs Use the ALOM system controller reset command or poweron and poweroff commands to reset the server Reaching the ok prompt by performing a manual system reset or by power cycling the system should be the method of last resort 14 Sun Blade T6300 Server Module Administration Guide 7 April 2007 Using these commands results in the loss of all svstem coherence and state information A manual svstem reset could corrupt the se
41. t activating 4 O ok prompt about 12 accessing through ALOM break command 13 accessing through Break key 13 14 accessing through graceful system shutdown 13 accessing through L1 A Stop A keys 13 14 accessing through manual system reset 13 14 accessing via ALOM break command 14 risks in using 15 suspension of Solaris Operating System 15 ways to access 13 15 OpenBoot commands go 15 probe ide 14 probe scsi 14 probe scsi all 14 showenv 31 OpenBoot configuration variables auto boot 12 22 described table 31 input device 17 output device 17 system console settings 17 OpenBoot firmware scenarios for control 12 operating system software suspending 15 output device OpenBoot configuration variable 17 P paritv 8 patch panel terminal server connection 5 poweroff sc gt command 14 poweron sc gt command 14 probe ide OpenBoot command 14 probe scsi OpenBoot command 14 probe scsi all OpenBoot command 14 R reset manual system 14 16 scenarios 24 reset sc gt command 14 run levels explained 12 ok prompt and 12 S sc gt commands break 14 console 14 console f 11 poweroff 14 poweron 14 reset 14 setlocator 21 showlocator 22 sc gt prompt about 10 accessing from network management port 12 accessing from serial management port 12 multiple sessions 11 system console escape sequence 12 system console switching between 9 ways to access 12
42. ually on page 27 Obtaining Automatic System Recovery Information on page 26 Enabling and Disabling Automatic System Recovery The ASR feature is not activated until you enable it Enabling ASR requires changing configuration variables in ALOM as well as OpenBoot firmware To Enable Automatic System Recovery At the sc gt prompt type sc gt setsc diag mode normal sc gt setsc diag level min sc gt setsc diag trigger power on reset At the ok prompt type ok setenv auto boot true ok setenv auto boot on error true To cause the parameter changes to take effect type ok reset all The system permanently stores the parameter changes and boots automatically when the OpenBoot configuration variable auto boot is set to true its default value Note To store parameter changes you can also power cycle the system using the front panel Power button Chapter 2 Managing RAS Features and System Firmware 25 Y To Disable Automatic System Recovery 1 At the ok prompt type ok setenv auto boot on error false 2 To cause the parameter changes to take effect type ok reset all The system permanently stores the parameter change Note To store parameter changes you can also power cycle the system using the front panel Power button After you disable the ASR feature it is not activated again until you re enable it Obtaining Automatic System Recovery
43. vstem behavior These variables are discussed in more detail in Appendix A Chapter 1 Configuring the System Console 17 18 Sun Blade T6300 Server Module Administration Guide April 2007 CHAPTER 2 Managing RAS Features and Svstem Firmware This chapter describes how to manage reliabilitv availabilitv and serviceabilitv RAS features and system firmware including the Sun Advanced Lights Out Manager ALOM system controller and Automatic System Recovery ASR In addition this chapter describes how to unconfigure and reconfigure a device manually and introduces multipathing software This chapter contains the following sections Interpreting System LEDs on page 20 Automatic System Recovery on page 22 Unconfiguring and Reconfiguring Devices on page 27 LI LI E Multipathing Software on page 29 Note This chapter does not cover detailed troubleshooting and diagnostic procedures For information about fault isolation and diagnostic procedures refer to the Sun Blade T6300 Server Module Service Manual Interpreting Svstem LEDs The behavior of LEDs on vour server conforms to the American National Standards Institute ANSI Status Indicator Standard SIS These standard LED behaviors are described in TABLE 2 1 TABLE 2 1 LED Behavior and Meaning LED Behavior Meaning Off The condition represented bv the color is not true Steadv on The condition represented by the color is true
44. y password if security mode is not none never displayed Do not set this directly security badlogins variable name No default Number of incorrect security password attempts 32 Sun Blade T6300 Server Module Administration Guide April 2007 TABLE A 1 OpenBoot Configuration Variables Stored on the System Processor Continued Variable Possible Values Default Value Description diag switch error reset recoverv network boot arguments true false boot sync none protocol key value false boot none If true e OpenBoot verbosity is set to maximum e After a boot request boot diag file from diag device If false e OpenBoot verbosity is set to minimum After a boot request boot boot file from boot device Command to execute following a system reset generated by an error Arguments to be used by the PROM for network booting Defaults to an empty string Use network boot arguments to specify the boot protocol RARP DHCP and a range of system knowledge to be used in the process For further information see the eeprom 1M man page or your Solaris reference manual Appendix A OpenBoot Configuration Variables 33 34 Sun Blade T6300 Server Module Administration Guide April 2007 Index Svmbols etc remote file 6 modifving 7 A Advanced Lights Out Manager ALOM escape sequence 12 multiple connections to 11 ALOM commands disablecomponent 27 en

Download Pdf Manuals

image

Related Search

Related Contents

Progress Lighting P4325-20 Installation Guide    DeLOCK 0.1m SATA Cable  USER`S MANUAL - Mitsubishi Heavy Industries  User Manual  Mai 2015 URGENT – Transmettez immédiatement cette lettre aux    View - Uniwell EPOS systems  取扱説明書/592KB - Kikusui Electronics Corp.  la cartographie des métiers  

Copyright © All rights reserved.
Failed to retrieve file