User`s Manual
Contents
1. FW Version 2 7 000 Rom Info type GPXE version 1 5 5 devid 25448 Device ID 25448 Chip Revision AO Description Portl Port2 MACs 0002c90000bb 0002c90000bc Board ID n a MT 0920110004 VSD n a PSID MT 0920110004 Assuming that ConnectX EN PXE is connected via Port 1 then the MAC address is 00 02 c9 00 00 bb 12 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Extracting the MAC Address Method II The six bytes of MAC address can be captured from the display upon the boot of the ConnectX device ses sion as shown in the figure below Mellanox ConnectX EN PXE v1 5 5 gPXE 0 9 9 Open Source Boot Firmware net0 00 07 9 05 cf f6 on PCI02 00 0 open CLink up TX TXE 0 RX 0 RXE 0 DHCP net 00 02 c9 05 cf f6 ok Placing MAC Addresses in etc dhepd conf The following is an excerpt of a etc dhcpd conf example file showing the format of representing a cli ent machine for the DHCP server running on a Linux machine host hostl next server 11 4 3 7 filename pxelinux 0 fixed address 11 4 3 130 hardware ethernet 00 02 c9 00 00 bb Mellanox Technologies 13 J Rev 1 2 TFTP Server 4 TFTP Server If you have set the filename parameter in your DHCP configuration to a non empty filename you need to install TFTP Trivial File Transfer Protocol TFTP is a simple FTP like file transfer protocol used to trans fer files from the TFTP server to the boot cli2. 7 3 6 echo Echoes an environment variable Example gPXE gt echo root path 7 3 7 dhcp A network interface attempts to open the network interface and then tries to connect to and com municate with the DHCP server to obtain the IP address and filepath from which the boot will occur Example gPXE gt dhcp netl 7 3 8 help Displays the available list of commands 7 3 9 exit Exits from the command line interface Mellanox Technologies 19 J Rev 1 2 Diskless Machines 8 8 1 Diskless Machines Mellanox ConnectX EN PXE supports booting diskless machines To enable using an Ethernet driver the remote kernel or init rd image must include and be configured to load the driver This can be achieved either by compiling the adapter driver into the kernel or by adding the device driver module into the initrd image and loading it The Ethernet driver requires loading the following modules in the specified order see Section 8 1 for an example e mlx4 core ko e mlx4 en ko Example Adding an Ethernet Driver to initrd Linux Prerequisites The ConnectX EN PXE image is already programmed on the adapter card 2 The DHCP server is installed and configured as described in Section 3 1 Configuring the DHCP Server and connected to the client machine 3 An initrd file 4 To add an Ethernet driver into initrd you need to copy the Ethernet modules to the diskless image Your machin
3. Mellanox ConnectX EN PXE first initializes the network adapter device Then it connects to a DHCP server to obtain its assigned IP address and network parameters and also to obtain the source location of the kernel OS to boot from The DHCP server instructs Mellanox ConnectX EN PXE to access the kernel OS through a TFTP server an iSCSI target or other service The binary code is exported by the device as an expansion ROM image 1 1 Supported Mellanox Adapter Hardware The package supports all ConnectX and ConnectX 2 Network Adapter family devices and cards Specif ically adapter products responding to the following PCI Device IDs are supported e Decimal 25408 Hexadecimal 6340 Decimal 25418 Hexadecimal 634a e Decimal 26418 Hexadecimal 6732 e Decimal 26428 Hexadecimal 673c e Decimal 26438 Hexadecimal 6746 e Decimal 25448 Hexadecimal 6368 Decimal 26448 Hexadecimal 6750 Decimal 26478 Hexadecimal 676e Decimal 26488 Hexadecimal 6778 Decimal 25458 Hexadecimal 6372 Decimal 26458 Hexadecimal 675A 1 2 Supported Firmware These supported hardware products require the firmware package fw 25408 which is available via www mellanox com gt Downloads gt Firmware gt Customized Firmware Check Mellanox ConnectX EN PXE Release Notes for the appropriate firmware version 13 Tested Platforms See Mellanox ConnectX EN Family PXE Release Notes ConnectX EN PXE release notes txt 8 Mellanox Tec
4. PCI 90 0 open CLink up TX 0 TXE 0 RX 0 RXE 0 If the Ethernet link comes up successfully the client attempts connecting to the DHCP server to obtain an IP address and the source location of the kernel OS to boot from The client waits up to 30 seconds for a DHCP server response 16 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 MLNX NIC 1 5 5 PCI 02 00 0 starting execution MLNX NIC 1 5 5 initialising devices Mellanox ConnectX EN PXE v1 5 5 gPXE 0 9 9 Open Source Boot Firmware netO 00 02 c9 05 cf f6 on PCI02 00 0 Copen CLink up 0 TKE 0 RX 0 RXE 01 DHCP neti 00 02 c9 05 cf f6 ok netO 11 4 3 130 255 255 255 0 gu 0 0 0 0 Booting from filename pxeboot 0 tftp 11 4 3 7 pxeboot 0 Next ConnectX EN PXE attempts to boot as directed by the DHCP server Mellanox Technologies 17 J Rev 1 2 Command Line Interface CLI 7 Command Line Interface CLI 7 1 Invoking the CLI When the boot process begins the computer starts its Power On Self Test POST sequence Shortly after completion of the POST the user will be prompted to press CTRL B to invoke Mellanox ConnectX EN PXE CLI The user has few seconds to press CTRL B before the message disappears see figure Mellanox C ctx EN PX A therboot 0 0 CB8O PC PnP BBS PMM002C004 CB8O to configure MLNX oo CPCI 0 0 Alternatively you may skip invoking CLI right after POST and invoke it instead ri
5. Root Password Network Customer Center Online Update Service Users Clean Up Release Notes Hardware Configuration Help Gi Clock and Time Zone Region Time Zone Europe Canada Central and South America Russia Asia Australia Africa Pacific Global Etc Alaska Aleutian Arizona Central Hawaii Indiana Starke Michigan Mountain Pacific Samoa Hardware Clock Set To UTC Abort Time and Date izj 07 52 06 24 03 2008 Change East Indiana 28 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Step 10 In the Installation Settings window click Partitioning to get the Suggested Partitioning window Preparation v Language v License Agreement v Disk Activation Click headli k he Ch bel v System Analysis ick any headline to make changes or use the ange menu below v Time Zone Overview Expert Installation m Installation Settings Installation Summary Keyboard Layout e Perform Installation English US Configuration e e Hostname Partitioning etico Hassard Create boot partition dev sdal 70 5 MB with ext2 SIETE Create swap partition dev sda2 502 0 MB Customer Center Create root partition dev sda3 7 4 GB with reiserfs e Online Update e Service Software e Users e Clean Up SUSE Linux Enterprise Server 10 Release Notes X Window System GNOME Desktop En
6. back and check the settings if you are unsure Show Release Nc Change v Step 19 At the end of the file copying stage the Finishing Basic Installation window will pop up and ask for confirming a reboot You can click OK to skip count down See image below Note Assuming that the machine has been correctly configured to boot from ConnectX EN PXE via its connection to the iSCSI target make sure that MLNX_EN has the highest priority in the BIOS boot sequence 34 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Preparation v Language v License Agreement v System Analysis v a Finishing Basic Installation Time Zone Copy files to installed system Installation v Installation Summary Save configuration gt Perform Installation Install boot manager Configuration Hostname e Root Password Prepare system for initial boot Network e Customer Center e Online Update Service Users e Clean Up e Release Notes The system will reboot now e Hardware Configuration Save installation settings 8 Finished ex EES Step 20 Once the boot is complete the Startup Options window will pop up Select SUSE Linux Enterprise Server 10 then press Enter SUSE Linux Enterprise Server 10 Floppy SUSE Linux Enterprise Server 10 Failsafe Boot Options Step 21 The Hostname and Domain Name window will pop up Continue configuring yo
7. configuration start from S Linux Enterp i d ne scratch or reread the Floppy Other chainloader dev fd0 configuration saved on Failsafe SUSE Linux Enterprise Server 10 SP2 Image append showopts ide n your disk If you have multiple Linux systems installed YaST can try to find them and merge their menus J gag Add Edit Delete Setas Default ra ca 32 Mellanox Technologies J Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Step 16 In the Optional Kernel Command Line Parameter field append the following string to the end of the line ibft_mode off include a space before the string Click OK and then Finish to apply the change Section Name o 4 Use Section Name to specify the boot loader section name The section name must be unique Section Settings Selecting Do not verify Filesystem before Booting will skip all file system checks Optional Kernel Command Line Parameter lets you define additional parameters to passto the kernel Kernel Image defines the kernel to boot Either enter the name directly or choose via Browse Initial RAM Disk if not empty defines the initial ramdisk to use Either enter the path and file name directly or choose by using Browse Root Device sets the device to passtothe 2 x kernel as root device J Boot Loader Settings Section Manag
8. img Type fileio Tip The following is an example of an iSCSI target ign line Target ign 2007 08 7 3 4 10 iscsiboot Mellanox Technologies 37 10 WinPE Mellanox ConnectX EN PXE enables WinPE boot via TFTP For instructions on preparing a WinPE image please see http etherboot org wiki winpe 38 Mellanox Technologies
9. 3 iSCSI Boot Example of SLES 10 SP2 OS 23 9 4 Windows 2008 iSCSI Boot 36 10 WinPE 38 4 Mellanox Technologies J Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Mellanox Technologies 5 J Rev 1 2 Revision History Printed on December 21 2009 Rev 1 2 December 21 2009 e Added in Chapter 2 Burning the PXE ROM Image instructions for upgrading the image on Con nectX EN devices with a firmware version earlier than 2 7 000 e Updated Chapter 9 iSCSI Boot to include Windows 2008 iSCSI Boot Rev 1 1 December 6 2009 e Added support for more ConnectX family devices see Section 1 1 Supported Mellanox Adapter Hardware on page 8 e Modified the firmware upgrade instructions in Chapter 2 Burning the PXE ROM Image e Added Chapter 7 Command Line Interface CLI page 18 Rev 1 0 February 2009 First release 6 Mellanox Technologies J Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Mellanox Technologies 7 J Rev 1 2 Overview 1 Overview This chapter describes Mellanox ConnectX EN PXE the software for Boot over Mellanox Technologies network adapter devices supporting Ethernet Mellanox ConnectX EN PXE enables booting kernels or oper ating systems OSes from remote servers in compliance with the PXE specification Mellanox ConnectX EN PXE is based on the open source project Etherboot gPXE available at http www etherboot org
10. AA Mellanox TECHNOLOGIES Mellanox ConnectX EN Family PXE User s Manual www mellanox com Rev 1 2 NOTE THIS INFORMATION IS PROVIDED BY MELLANOX FOR INFORMATIONAL PURPOSES ONLY AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIM ITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FORA PARTICULAR PURPOSE ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIA BLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSE QUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS HARDWARE EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies Mellanox Technologies Ltd 350 Oakmead Parkway Suite 100 PO Box 586 Hermon Building Sunnyvale CA 94085 Yokneam 20692 U S A Israel www mellanox com Tel 972 4 909 7200 Tel 408 970 3400 Fax 972 4 959 3245 Fax 408 970 3403 Copyright 2009 Mellanox Technologies Inc All Rights Reserved Mellanox ConnectX InfiniBlast InfiniBridge InfiniHost InfiniRISC InfiniScale and InfiniPCI are registered trademarks of Mellanox Technologies Ltd BridgeX PhyX and Virtual Protocol Interconnect are trademarks of Mellanox Technol
11. Booting Preparation v Language w License Agreement y Disk Activation W System Analysis v Time Zone Overview Expert a Installation Settings Click any headline to make changes or use the Change menu below Installation gt Installation Summary Keyboard Layout Peruri Inslallativni English US Configuration Se Root Password Partitioning e Hostname Create swap partition dev sdal 502 0 MB Network Create root partition dev sda2 7 5 GB with reiserfs e Customer Center e Online Update Add On Products e Service n ere No add on product selected for installation e Clean Up Release Notes Software e Hardware Configuration SUSE Linux Enterprise Server 10 SP2 Server Base System KDE Desktop Environment for Server C C Compiler and Tools X Window System Size of Packagesto Install 1 6 GB Booting Boot Loader Type GRUB Location dev sda2 boot Sections SUSE Linux Enterprise Server 10 SP2 default Floppy Failsafe SUSE Linux Enterprise Show Release Notes Server 10 SP2 Help Back Abort Mellanox Technologies 31 Rev 1 2 ISCSI Boot Step 15 Click Edit in the Boot Loader Settings window Section List Boot Loader Settings From Other you can manually edit the boot Section Management Boot Loader Installation loader configuration files clear the current configuration and m propose a new Section Summary
12. dure Step 1 Load the SLES 10 SP2 installation disk and enter the following parameters as boot options netsetup 1 WithISCSI 1 Boot from Hard Disk Installation ACPI Disabled Installation Local APIC Disabled Installation Safe Settings Rescue System Memory Test Boot Options netsetup i WithISCSI 1 Mellanox Technologies 23 Rev 1 2 iSCSI Boot Step 2 Continue with the procedure as instructed by the installation program until the iSCSI Initiator Overview window appears Preparation V Language V License Agreement Disk Activation System Analysis Service Connected Targets Time Zone a iSCSI Initiator Overview Portal Address Target Name Start Up Installation Installation Summary Perform Installation Configuration Root Password Hostname Network e Customer Center e Online Update e Service Users Clean Up Release Notes Hardware Configuration Step3 Click the Add tab in the iSCSI Initiator Overview window An iSCSI Initiator Discovery window will pop up Enter the IP Address of your iSCSI target and click Next Preparation V Language V License Agreement Disk Activation e System Analysis Time Zone a iSCSI Initiator Discovery Installation Installation Summary e Perform Installation IP Address Port zeo Configuration rp Root Password Hostname Network IX No Authentication Custom
13. e needs to be pre installed with a MLNX EN Linux Driver available for download from www mellanox com gt Products gt Ethernet SW Drivers that is appropriate for the kernel version the diskless image will run Adding the Ethernet Driver to the initrd File Warning The following procedure modifies critical files used in the boot procedure It must be exe cuted by users with expertise in the boot process Improper application of this procedure may prevent the diskless machine from booting Step 1 Back up your current initrd file Step 2 Make a new working directory and change to it host1 mkdir tmp initrd en hostl cd tmp initrd en Step 3 Normally the initrd image is zipped Extract it using the following command host1 gzip dc lt initrd image gt cpio id The initrd files should now be found under tmp initrd_en Step 4 Create a directory for the ConnectX EN modules and copy them host1 mkdir p tmp initrd en lib modules mlnx en host1 cd lib modules uname r updates kernel drivers hostl cp net mlx4 mlx4 core ko tmp initrd en lib modules mlnx en 20 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 hostl cp net mlx4 mlx4 en ko tmp initrd en lib modules mlnx en Step 5 To load the modules you need the insmod executable If you do not have it in your initrd please add it using the following command host1 cp sbin insmod tmp initrd_en sbin Step 6 If you plan to give y
14. ectX EN 25448 ROM X X XXX rom 1 Depending on the OS the device name may be superceded with a prefix Mellanox Technologies 11 Rev 1 2 DHCP 3 DHCP The DHCP server plays a major role in the boot process by assigning IP addresses for ConnectX EN PXE clients and instructs the clients where to boot from When the ConnectX EN PXE boot session starts the PXE firmware attempts to bring up a ConnectX net work link port If it succeeds to bring up a connected link the PXE firmware communicates with the DHCP server The DHCP server assigns an IP address to the PXE client and provides it with the location of the boot program 3 1 Configuring the DHCP Server When a ConnectX EN PXE client boots it sends the DHCP server various information including its DHCP hardware Ethernet address MAC The MAC address is 6 bytes long and it is used to distinguish between the various DHCP sessions Extracting the MAC Address Method I Run the following commands Note The following MFT commands assume that the Mellanox Firmware Tools MFT package has been installed on the client machine host1 mst start host1 mst status The device name will be of the form dev mst mt lt dev_id gt _pci _cr0 conf0 Use this device name to obtain the MAC address via a query command flint d lt MST DEVICE NAME gt q Example with ConnectX EN as the network adapter device hostl flint d dev mst mt25448 pci cr0 q Image type ConnectX
15. ement Section Editor Section Name SUSE Linux Enterprise Server 10 SP2 Section Settings _ Do not verify Filesystem before Booting Optional Kernel Command Line Parameter resume dev sdal splash silent showopts ibft mode off Kernel Image w Browse Initial RAM Disk boot initrd x Browse Root Device idevisda2 xj Vga Mode 0x332 Back Abort ok Step 17 If you wish to change additional settings click the appropriate item and perform the changes and click Accept when done Mellanox Technologies 33 Rev 1 2 iSCSI Boot Step 18 In the Confirm Installation window click Install to start the installation See image below Preparation v Language v License Agreement v Disk Activation v System Analysis v Time Zone Installation Settings Click any headline to make changes or use the Change menu below Overview Expert Installation Installation Sum Perform Installatic Confirm Installation Configuration Hostname Root Password Network e Customer Center e Online Update Service Users e Clean Up Release Notes e Hardware Config All information required for the base installation is now complete If you continue now partitions on your hard disk will be formatted erasing any existing data in those partitions according to the installation settings in the previous dialogs Go
16. ent as part of the boot process 14 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 5 BIOS Configuration The expansion ROM image presents itself to the BIOS as a boot device As a result the BIOS will add MLNX NIC lt ver gt to the list of boot devices The priority of this list can be modified through BIOS setup Mellanox Technologies 15 J Rev 1 2 Operation 6 6 1 6 2 Operation Prerequisites e Make sure that your client is connected to the server s e The ConnectX EN PXE image is already programmed on the adapter card see Chapter 2 e Configure and start the DHCP server as described in Section 3 1 e Configure and start at least one of the services iSCSI target see Section 9 and TFTP see Section 4 Starting Boot Boot the client machine and enter BIOS setup to configure MLNX NIC lt ver gt to be the first on the boot device priority list see Section 5 Note On dual port network adapters the client first attempts to boot from Port 1 If this fails it switches to boot from Port 2 If MLNX NIC was selected through BIOS setup the client will boot from ConnectX EN PXE The client will display ConnectX EN PXE attributes and will attempt to bring up a port link MLNX NIC 1 5 5 PCI 02 00 0 starting execution MLNX NIC 1 5 5 initialising devices Mellanox Connect EN PXE v1 5 5 gPXE 0 9 9 Open Source Boot Firmware net0 00 02 c9 05 cf f6 on
17. er Center Online Update Service Users Clean Up Username Password Incoming Authentication Release Notes Hardware Configuration Outgoing Authentication Username Password Help Back Abort 24 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Step 4 Details of the discovered iSCSI target s will be displayed in the iSCSI Initiator Discovery window Select the target that you wish to connect to and click Connect rat iSCSI Initiator Discovery y License Agreement Disk Activation _ System Analysis Portal Address Target Name Connected e Time Zone 10 4 3 7 3260 1 iqn 2007 08 7 3 4 10 iscsiboot False Installation Installation Summary Perform Installation Configuration Root Password e Hostname Network Customer Center e Online Update Service e Users e Clean Up Release Notes e Hardware Configuration Connect E If no iSCSI target was recognized then either the target was not properly installed or no connection was found between the client and the iSCSI target Open a shell to ping the iSCSI target you can use CTRL ALT F2 and verify that the target is or is not accessible To return to the graphical installation screen press CTRL ALT F7 Mellanox Technologies 25 Rev 1 2 iSCSI Boot Step 5 The iSCSI Initiator Discovery window will now request authentication to access
18. ght after Con nectX EN PXE starts booting Once the CLI is invoked you will see the following prompt gPXE gt 7 2 Operation The CLI resembles a Linux shell where the user can run commands to configure and manage one or more PXE port network interfaces Each port is assigned a network interface called neti where i is 0 1 2 lt of interface gt Some commands are general and are applied to all network inter faces Other commands are port specific therefore the relevant network interface is specified in the command 7 3 Command Reference 7 3 1 ifstat Displays the available network interfaces in a similar manner to Linux s ifconfig gt on PCIOZ 9 0 closed Link down 2 0 RK 0 9 Link status Unknown 0x1a086001 1 neti 00 02 c9 05 cf f7 on PCIO2 00 0 closed Link doun TX 0 TXE 0 RX 0 RXE 01 Link status Unknown 0x1a086001 1 GPXED 7 3 2 ifopen net lt x gt Opens the network interface net lt x gt The list of network interfaces is available via the ifstat command 18 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 7 3 3 ifclose net lt x gt Closes the network interface net lt x gt The list of network interfaces is available via the ifstat command 7 3 4 autoboot Starts the boot process from the device s 7 3 5 sanboot Starts the boot process of an iSCSI target Example gPXE gt sanboot iscsi 11 4 3 7 ign 2007 08 7 3 4 11 iscsiboot
19. hnologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 1 4 Package Contents The ConnectX EN PXE package is provided as a tarball tgz extension containing the following files 1 A PXE ROM image file for each of the supported Mellanox network adapter devices For example ConnectX EN PCI DevID 25448 CONNECTX EN 25448 ROM lt version gt rom Note Please refer to the release notes file for the exact contents 2 A docs directory includes the following files Mellanox ConnectX EN PXE User Manual pdf this user s manual ConnectX EN PXE release notes txt release notes 15 Reference Documents and Downloads e The Mellanox Firmware Tools MFT package and documentation can be downloaded from www mel lanox com gt Downloads gt Firmware Tools e The MLNX EN Ethernet Driver and documentation can be downloaded from www mellanox com gt Products gt Ethernet SW Drivers gt MLNX_EN Ethernet Driver Mellanox Technologies 9 J Rev 1 2 Burning the PXE ROM Image 2 Burning the PXE ROM Image 2 1 Burning the Image on ConnectX EN ConnectX 2 EN Note This section is applicable only to ConnectX EN ConnectX 2 EN devices with firm ware versions 2 7 000 or later For earlier firmware versions please follow the instructions in Section 2 2 on page 10 2 1 1 Prerequisites 1 Expansion ROM Image The expansion ROM images are provided as part of
20. ing the kernel and initrd for Linux via ConnectX EN PXE and the second is for loading other parts of the OS Note Linux distributions such as SuSE Linux Enterprise Server 10 SPx and Red Hat Enterprise Linux 5 1 can be directly installed on an iSCSI target At the end of this direct installation initrd is capable to continue loading other parts of the OS on the iSCSI target Other dis tributions may also be suitable for direct installation on iSCSI targets If you choose to continue loading the OS after boot through the adapter device driver please verify that the initrd image includes the adapter driver as described in Section 8 1 Configuring the DHCP Server to Boot From an iSCSI Target Configure DHCP as described in Section 3 1 Configuring the DHCP Server Example on Linux Edit your DHCP configuration file etc dhcpd conf and add the following lines for the machine s you wish to boot from the iSCSI target Filename option root path iscsi iscsi target ip iscsi target ign The following is an example for configuring an Ethernet device to boot from an iSCSI target host hostl filename hardware ethernet 00 02 c9 00 00 bb option root path iscsi 11 4 3 7 ign 2007 08 7 3 4 10 iscsiboot Configuring an iSCSI Target for a Linux Environment Prerequisites Step 1 Make sure that an iSCSI target is installed on your server side Tip You can download and install an iSCSI target from the fol
21. lowing location http sourceforge net project showfiles php group id 108475 amp package id 117141 Step 2 Dedicate a partition on your iSCSI target on which you will later install the operating system 22 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Step 3 Configure your iSCSI target to work with the partition you dedicated If for example you choose partition dev sda5 then edit the iSCSI target configuration file etc ietd conf to include the following line under the iSCSI target iqn line Lun 0 Path dev sda5 Type fileio Tip The following is an example of an iSCSI target iqn line Target iqn 2007 08 7 3 4 10 iscsiboot Step 4 Start your iSCSI target Example host1 etc init d iscsitarget start 9 3 iSCSI Boot Example of SLES 10 SP2 OS This section provides an example of installing the SLES 10 SP2 operating system on an iSCSI target and booting from a diskless machine via ConnectX EN PXE Note that the procedure described below assumes the following e The client s LAN card is recognized during installation e The iSCSI target can be connected to the client via a LAN and a ConnectX Ethernet Prerequisites See Section 6 1 on page 16 Warning The following procedure modifies critical files used in the boot procedure It must be exe cuted by users with expertise in the boot process Improper application of this procedure may prevent the diskless machine from booting Proce
22. ogies Ltd All other marks and names mentioned herein may be trademarks of their respective companies 2 Mellanox Technologies Document Number 3026 Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Blank Page Mellanox Technologies 3 J Rev 1 2 Table of Contents Table of Contents 4 Revision History 6 1 Overview 8 1 1 Supported Mellanox Adapter Hardware 8 1 2 Supported Firmware 8 13 Tested Platforms 8 1 4 Package Contents 9 1 5 Reference Documents and Downloads 9 2 Burning the PXE ROM Image 10 2 1 Burning the Image on ConnectX EN ConnectX 2 EN 10 2 1 1 Prerequisites 10 2 1 2 Burning the Image 10 2 2 Updating the Image on ConnectX EN Devices with Legacy Firmware 10 2 2 1 Prerequisites 10 2 2 2 Image Burning Procedure 11 3 DHCP 12 3 1 Configuring the DHCP Server 12 TFTP Server 14 BIOS Configuration 15 Operation 16 6 1 Prerequisites 16 6 2 Starting Boot 16 7 Command Line Interface CLI 18 7 1 Invoking the CLI 18 7 2 Operation 18 7 3 Command Reference 18 7 3 1 ifstat 18 7 3 2 ifopen net lt x gt 18 7 3 3 ifclose net lt x gt 19 7 3 4 autoboot 19 7 3 5 sanboot 19 7 3 6 echo 19 7 3 7 dhcp 19 7 3 8 help 19 7 3 9 exit 19 8 Diskless Machines 20 8 1 Example Adding an Ethernet Driver to initrd Linux 20 9 iSCSI Boot 22 9 1 Configuring the DHCP Server to Boot From an iSCSI Target 22 9 1 1 Example on Linux 22 9 2 Configuring an iSCSI Target for a Linux Environment 22 9
23. our Ethernet device a static IP address then copy ifconfig Otherwise skip this step hostl cp sbin ifconfig tmp initrd_en sbin Step 7 Now you can add the commands for loading the copied modules into the file init Edit the file tmp initrd en init and add the following lines at the point you wish the Ethernet driver to be loaded Warning The order of the following commands for loading modules is critical echo loading Mellanox ConnectX EN driver sbin insmod lib modules mlnx en mlx4 core ko sbin insmod lib modules mlnx en mlx4 en ko Step 8 Now you can assign a static or dynamic IP address to your Mellanox ConnectX EN network inter face Step 9 Save the init file Step 10 Close initrd hostl cd tmp initrd en hostl find cpio H newc o gt tmp new initrd en img hostl gzip tmp new init en img Step 11 At this stage the modified initra including the Ethernet driver is ready and located at tmp new init ib img gz Copy it to the original initrd location and rename it properly Mellanox Technologies 21 Rev 1 2 iSCSI Boot 9 9 1 9 1 1 9 2 iSCSI Boot Mellanox ConnectX EN PXE enables an iSCSI boot of an OS Linux or Windows located on a remote iSCSI target It has a built in iSCSI initiator which can connect to the remote iSCSI target and load from it the kernel and on Linux initrd There are two instances of connection to the remote iSCSI target the first is for gett
24. rtitioner disks This is intended for experts If you are not i Size Type Mount Mount By Start End Used By Label Devi familiar with the 8 0 GB IET VIRTUAL DISK O 1045 scsi concepts of hard disk 705 MB partitions and how to use them you might wantto go back and selectautomatic partitioning F Linux native Ext2 o boot K 0 8 7 4 GB F Linux native Reiser 73 1045 Please note that nothing will be written to your hard disk until you confirm the entire installation in the last installation dialog Until that point you can safely abort the installation You have not assigned a swap partition There is nothing wrong with that but in most cases it is highly recommended to create and assign a swap partition Swap partitions on your system are listed in the main window with the type Linux Swap An assigned swap partition has the mount point swap You can assign more than one swap partition if desired For LVM setup using a non LVM root device and a non LVM swap device is inca recommended Other no than the root and swap devices you should have partitions managed by LVM Do you want to change this The table to the right shows the current partitions on all your Create Edit Delete Resize hard disks T aa E Lvm l EVMS BAID v CryptFile Expert v izim Tapan Step 14 Select the Expert tab and click
25. tandard Mellanox products ini files are included in the firmware mlx packages For help in identifying the correct ini file of your adapter hardware please refer to MFT User s Manual see Section 1 5 3 Expansion ROM Image The expansion ROM images are provided as part of the SW package and are listed in the release notes file ConnectX EN PXE release notes txt 4 Firmware Burning Tools You need to install the Mellanox Firmware Tools MFT package version 2 6 0 or later in order to burn the PXE ROM image To download MFT see Firmware Tools under www mellanox com gt Downloads Specifically you will be using the mlxburn tool to create and burn a composite image from an adapter device s firmware and the PXE ROM image onto the same Flash device of the adapter 2 2 2 Image Burning Procedure To burn the composite image perform the following steps 1 Obtain the MST device name Run mst start mst status The device name will be of the form mt lt dev id gt pci cr0 conf0 2 Create and bum the composite image Run mlxburn d lt mst device name gt fw lt FW mlx file gt conf lt ini file gt exp rom lt expansion ROM image gt Example on Linux mlxburn dev dev mst mt25448 pci cr0 fw fw 25408 xX X XXX mlx conf MNEH28 XTC ini exp rom ConnectX EN 25448 ROM X X XXX rom Example on Windows mlxburn dev mt25448 pci cr0 fw fw 25408 X X XXX mlx conf MNEH28 XTC ini exp rom Conn
26. the DHCP server as described in Section 3 1 and Section 9 1 e Configure and start at least one of the services iSCSI target see Section 9 and TFTP see Section 4 Booting Procedure The following steps describe the procedure for booting Windows 2008 from an iSCSI target Step 1 Step 2 Step 3 Step 4 Step 5 Tip Install Windows 2008 on a local machine as instructed on the Etherboot Web page http etherboot org wiki sanboot win2k8_physical Install the MLNX_EN for Windows driver available from Mellanox s Web page http www mellanox com content pages php pg products dyn amp product family 28 amp menu section 35 Prepare an image from your installed partition The following Web location shows how to do this using a Linux USB key http etherboot org wiki sanboot transfer Copy the prepared image to the iSCSI target Make sure that an iSCSI target is installed on your server side You can download and install an iSCSI target from the following location http sourceforge net project showfiles php group id 108475 amp package id 117141 36 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Step 6 Configure your iSCSI target to work with the file copied in Step 4 If for example you choose the file name w2k8 boot img then edit the iSCSI target configuration file etc ietd conf to include the following line under the iSCSI target ign line Lun 0 Path w2k8 boot
27. the Mellanox ConnectX EN PXE package and are listed in Section 1 4 2 Firmware Burning Tools You need to install the Mellanox Firmware Tools MFT package version 2 6 0 or later in order to burn the PXE ROM image To download MFT see Firmware Tools under www mellanox com gt Downloads 2 1 2 Burning the Image To burn the composite image perform the following steps 1 Obtain the MST device name Run mst start mst status The device name will be of the form mt lt dev id gt pci cr0 conf0 2 Create and burn the composite image Run flint dev lt mst device name gt brom lt expansion ROM image gt Example on Linux flint dev dev mst mt25448 pci cr0 brom ConnectX EN 25448 ROM X X XXX rom Example on Windows flint dev mt25448 pci cr0 brom ConnectX EN 25448 ROM X X XXX rom 2 2 Updating the Image on ConnectX EN Devices with Legacy Firmware Note This section is applicable only to ConnectX EN devices with firmware versions earlier than 2 7 000 2 2 1 Prerequisites 1 Firmware packages 1 Depending on the OS the device name may be superseded with a prefix 10 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 The appropriate ConnectX fw 25408 firmware mlx package can be downloaded from Mellanox Technologies Web site see www mellanox com gt Downloads gt Firmware gt Customized Firmware 2 Firmware Configuration ini Files For s
28. the iSCSI target Click Next to continue without authentication unless authentication is required Preparation V Language License Agreement Disk Activation System Analysis Time Zone iSCSI Initiator Discovery Installation e Installation Summary Perform Installation Configuration Root Password Hostname Network e Customer Center Incoming Authentication Online Update Username Password Service Users e Clean Up Release Notes e Hardware Configuration X No Authentication Outgoing Authentication Username Password i Help Back P Abort Step 6 The iSCSI Initiator Discovery window will show the iSCSI target that got connected to Note that the Connected column must indicate True for this target Click Next See figure below 26 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Preparation v Language License Agreement Disk Activation System Analysis Portal Address Target Name Connected Time Zone 043 ign 2007 08 0 siboot True iSCSI Initiator Discovery Installation Installation Summary Perform Installation Configuration Root Password Hostname Network Customer Center Online Update Service Users Clean Up Release Notes Hardware Configuration Ce Ces EE Step 7 The iSCSI Initiator Overview window will pop up Click Toggle Start Up to change start
29. ts If you are not familiar with the concepts of hard disk partitions and how to use them you might want to go back and select automatic partitioning Device Size F Type Mount Mount By Start End Used By Label Devi dev sda 8 0 GB IET VIRTUAL DISK 0 1045 scsi dev sdal 70 5 MB F Linux native Ext2 boot K 0 8 scsi dev sda2 502 0 MB F Linux swap swap 9 dev sda3 7 4 GB F Linux native Reiser Please note that nothing will be written to your hard disk until you confirm Really delete device dev sda2 the entire installation ni in the last installation dialog Until that point you can safely abort the installation For LVM setup using a non LVM root device and a non LVM swap device is recommended Other than the root and swap devices you should have partitions managed by LVM The table to the right showsthe current n partitions on all your Create Edit Delete Resize dt hard disks EVMS RAID Crypt File Expert v Back Abort Hard disks are Step 13 In the pop up window click No to approve deleting the swap partition You will be returned to Installation Settings window See image below 30 Mellanox Technologies Mellanox ConnectX EN Family PXE User s Manual Rev 1 2 Partition yourhard Expert Pa
30. up from manual to automatic Click Finish Preparation V Language y License Agreement Disk Activation System Analysis Service Connected Targets Time Zone a iSCSI Initiator Overview Portal Address Target Name Installation 0 4 3 1 ign 2007 0 3 oot manual Installation Summary Perform Installation Configuration Root Password e Hostname Network e Customer Center e Online Update Service e Users e Clean Up Release Notes e Hardware Configuration Add Log Out Toggle Start Up Help Abort Mellanox Technologies 27 Preparation v Language w License Agreement Disk Activation System Analysis Time Zone Installation e Installation Summary Perform Installation Configuration Root Password Hostname Network Customer Center Online Update Service Users Clean Up Release Notes Hardware Configuration Help Step 9 Select the appropriate Region and Time Zone in the Clock and Time Zone window then click Fin ish t Installation Mode Rev 1 2 ISCSI Boot Step 8 Select New Installation then click Finish in the Installation Mode window Select Mode New Installation Abort Preparation v Language v License Agreement v Disk Activation v System Analysis gt Time Zone Installation Installation Summary Perform Installation Configuration Hostname
31. ur machine until the operating system is up then you can start running the machine in normal operation mode Mellanox Technologies 35 Rev 1 2 iSCSI Boot Step 22 Optional If you wish to have the second instance of connecting to the iSCSI target go through the Ethernet driver copy the initra file under boot to a new location add the Ethernet driver into it after the load commands of the iSCSI Initiator modules and continue as described in Section on page 19 Warning Pay extra care when changing initrd as any mistake may prevent the client machine from booting It is recommended to have a back up iSCSI Initiator on a machine other than the client you are working with to allow for debug in case initrd gets corrupted Next edit the init file that is in the initrd zip and look for the following string if iSCSI TARGET IPADDR then iscsiserver iSCSI TARGET IPADDR fi Now add before the string the following line iSCSI TARGET IPADDR lt Ethernet IP Address of iSCSI target gt Example iSCSI TARGET IPADDR 11 4 3 7 Also edit the file boot grub menu 1st and delete the following string ibft mode off 9 4 Windows 2008 iSCSI Boot ConnectX EN PXE supports booting Windows 2008 from an iSCSI target Prerequisites e Make sure that your client is connected to the server s e The ConnectX EN PXE image is already programmed on the adapter card see Chapter 2 e Configure and start
32. vironment for Server Hardware Configuration 4 Server Base System Novell AppArmor Print Server Size of Packages to Install 1 3 GB Language Primary Language English US Show Release Notes Help Abort Step 11 Select Base Partition Setup on This Proposal then click Next Your hard disks have Suggested Partitioning been checked The partition setup displayed is proposed for your hard drive Create boot partition dev sdal 70 5 MB with ext2 Create swap partition dev sda2 502 0 MB To accept these Create root partition dev sda3 7 4 GB with reiserfs suggestions and continue select Accept Proposal Ifthe suggestion does not fit your needs create your own partition setup starting with the partitions as currently present on the disks For this select Custom Partition Setup This is also the option to choose for advanced options like RAID and LVM Partitioning Accept Proposal Base Partition Setup on This Proposal Create Custom Partition Setup Back Abort Mellanox Technologies 29 Rev 1 2 iSCSI Boot Step 12 In the Expert Partitioner window select from the IET VIRTUAL DISK device the row that has its Mount column indicating swap then click Delete Confirm the delete operation and click Finish Partition yourhard Expert Partitioner disks This is intended for exper
Download Pdf Manuals
Related Search