Home

PEM-FDR

1. IN SEARCH OF INCREDIBLE PEM FDR 56 Gb s FDR InfiniBand card User Guide T d 100000 meos f Im 11100 1000000101001010100 ic im 0 m E9426 First Edition July 2014 Copyright 2014 ASUSTeK COMPUTER INC All Rights Reserved No part of this manual including the products and software described in it may be reproduced transmitted transcribed stored in a retrieval system or translated into any language in any form or by any means except documentation kept by the purchaser for backup purposes without the express written permission of ASUSTeK COMPUTER INC 505 Product warranty or service will not be extended if 1 the product is repaired modified or altered unless such repair modification of alteration is authorized in writing by ASUS or 2 the serial number of the product is defaced or missing ASUS PROVIDES THIS MANUAL AS IS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OR CONDITIONS OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE IN NO EVENT SHALL ASUS ITS DIRECTORS OFFICERS EMPLOYEES OR AGENTS BE LIABLE FOR ANY INDIRECT SPECIAL INCIDENTAL OR CONSEQUENTIAL DAMAGES INCLUDING DAMAGES FOR LOSS OF PROFITS LOSS OF BUSINESS LOSS OF USE OR DATA INTERRUPTION OF BUSINESS AND THE LIKE EVEN IF ASUS HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES ARISING FROM ANY DEFECT OR ERROR IN THIS MANUA
2. When done reboot your system root localhost mnt File Edit View Search Terminal Help Changing max locked memory to unlimited in etc security limits conf Please log out from the shell and login again in order to update this change Read more about this topic in the VMA s User Manual VMA README txt is installed at usr share doc libvma 6 5 9 0 README txt Please refer to VMA journal for the latest changes usr share doc libvma 6 5 9 0 journal txt Preparing mlnxofed docs Preparing mpitests mvapich2 1 9 Preparing mpitests openmpi 1 6 5 Preparing mpitests openmpi 17 4 Installation finished successfully e If your system has the latest firmware no firmware update will occur e To force the firmware update key in the following command force fw update ASUS PEM FDR 3 7 ASUS contact information ASUSTeK COMPUTER INC Address 15 Li Te Road Peitou Taipei Taiwan 11259 Telephone 886 2 2894 3447 Fax 886 2 2890 7798 E mail info asus com tw Web site http www asus com Technical Support Telephone 86 21 38429911 Fax 86 21 58668722 ext 9101 Online Support http support asus com techserv techserv aspx ASUSTeK COMPUTER INC Taiwan Address Telephone Fax E mail Web site Technical Support Telephone Online Support 15 Li Te Road Peitou Taipei Taiwan 11259 886 2 2894 3447 886 2 2890 7798 info asus com tw http www asus com tw 8
3. Z9 and P9 series motherboard system Product introduction This chapter provides information about the PEM FDR card features and the new technologies it supports 1 1 Welcome Thank you and congratulations for buying an ASUS PEM FDR card Before installing the 56 Gb s FDR InfiniBand card check the items in your package with the list below 1 2 Package contents Check your package for the following items Standard Gift Box Pack Standard Bulk Pack Support DVD 1 per carton Full height bracket with screws 1 Low profile bracket with screws 1 Packaging Quantity 1 pc per carton 5 pcs per carton The default small bracket for node is a proprietray bracket that supports ASUS Z10PH D16 series ES4000 G3 series and RS720Q E8 series models The complete and updated list of supported models are subject to change without prior notice Z If any of the above items is damaged or missing contact your retailer 1 3 System requirements Check if your system meets the following requirements ASUS motherboard system with a PCI Express x8 or x16 card slot Supported operating system Windows and Linux operating systems refer to the ASUS website for the latest updates Goto www asus com for the complete and updated list of supported motherboard system models Other requirements Appropriate thermal solution Ensure to update your BIOS to the latest version before using PEM FDR on ASUS Z9
4. vvS 202 LL LLOZ L 9 LA 268 108 10 1102 L Z SA 2 806 106 90 1102 Z SA 1 806 LOE 0 002 2 0 6A 119 108 80 0102 Lv LA 0 008 NA 80 0102 L 9 LA 1 0 008 01 9002 LZ LA 82 008 NA SAPA 31189 93 9 6661 L LOZ LEV 700c 020SS NA 2102 2 2 00019 N3 OL0c vc0SS 2002 LV 1002 21099 NA 6002 27 9002 2 2 00019 N3 1102 0 0102 22099 N3 X H SeAnoeuip 0 eui jo aui 2103 29 801 00 jonpoJd 5 ANVIAH3O 2 N3ONLLVH 08807 22 12 415 LHOXHVH sseppy dalfid WOO Snsv edoung ui eAnejuesaudai pezuouiny NVMIVL 211 I3dlV L PY 91 11 091 ON AY 5 HALNdNOD e1Snsv 318IQ38JNI 30 HIUVIS NI cnl peuDisJepun y IM JO 23 JOA 10 oc UNL WA eqngeusig 7 juoapisag suey JAIS S uos1oqd oAnvjuoso1dow uone1odo poursopun osneo Kew jeu Surpn our 143996 IMAP STY T pue oouaropr
5. 86 2 2894 3447 0800 093 456 http support asus com techserv techserv aspx ASUSTeK COMPUTER INC China Address Telephone Fax Web site Technical Support Telephone Online Support No 508 Chundong Road Xinzhuang Industrial Zone Minhang District Shanghai China 86 21 5442 1616 86 21 5442 0099 http www asus com cn 400 620 6655 http support asus com techserv techserv aspx 9 aneus ueus Aer O39 uonsog 155091 Ajuuojuoo 93 vlog 32 469A 7102 90 02 91eq JIN SHOH f13 S9 ELLOZ X ELOZ ZL9 ON 03 6002 842 ON 6002 zv9 ON 93 8002 8421 7 19 93 52 76002 LLOZ Z LY 2002 99009 LLOZ Z LY 09609 N3 X GA1 03 S6 900eX 40 8002 LL LA 2 62 20 10 8002 LL LA L L62 202 11 8002 LA 2 ZS LOE 60 002 Le LA 92 202 NA 90 2002 2 LA 2 92 ZOE 60 0102 L 9 LA 2 68 108 60 2102 L Z ZA 41 687 105 11 1002 LA 6 687 LOE L1 S002 FELA 2 68 LOE 80 6002 Lv LA 7 687 108 80 2002 Lv LA 5 689 108 60 1 10Z 2 6 LA 1 687 LOE 8002 229 NA 2002 98209 NA 0102 62729 1002 0950 10 6002 LL LA 229 ZOE 10 6002 LL LA Z
6. Do you want to continue 7 uzn y jurning FW image without signatures iestoring signature files on the USB device 5 When the firmware is successfully updated reboot the system ASUS PEM FDR Driver installation This chapter provides instructions in installing the PEM FDR card s drivers on different operating systems 3 1 Mellanox ConnectX 3 Infiniband driver installation This section provides instructions on how install the Mellanox driver 311 Windows operating system To install the Mellanox driver 1 Restart the computer then log on with Administrator privileges 2 Insert the Mellanox Infiniband support DVD into the optical drive The support DVD automatically displays the Drivers menu if Autorun is enabled in your computer 3 Click Mellanox ConnectX Virtual Channel Adapter Driver N lt _InfiniB and_CX3 Support DVD Windows Server 2008 R2 64 bit ilities Manual Contact Drivers Please select items the right side 4 Click Next to start installing the driver 1 MLNX_VPI InstallShield Wizard Welcome to the InstallShield Wizard for MLNX_VPI The InstallShield R Wizard will install MLNX_VPI on your computer To continue click Next WARNING This program is protected by copyright law and 3 2 Chapter 3 Driver installation 5 Tick I accept the terms in the license agreement then click Next to continue 12 MLNX VPI InstallShield Wizard License Agreement
7. L OR PRODUCT SPECIFICATIONS AND INFORMATION CONTAINED IN THIS MANUAL ARE FURNISHED FOR INFORMATIONAL USE ONLY AND ARE SUBJECT TO CHANGE AT ANY TIME WITHOUT NOTICE AND SHOULD NOT BE CONSTRUED AS A COMMITMENT BY ASUS ASUS ASSUMES NO RESPONSIBILITY OR LIABILITY FOR ANY ERRORS OR INACCURACIES THAT MAY APPEAR IN THIS MANUAL INCLUDING THE PRODUCTS AND SOFTWARE DESCRIBED IN IT Products and corporate names appearing in this manual may or may not be registered trademarks or copyrights of their respective companies and are used only for identification or explanation and to the owners benefit without intent to infringe Contents oie lii ise Edom iv How this guide is organized iv Where to find more informatiOn cccccccsecsrccssessecsseessesseesseeseesseeseressas IV Conventions used in this guide tpe bruta tua V ticos slc V PEM FDR card specifications summary eene vi Chapter 1 Product introduction 1 1 Welcome 1 2 1 2 PACKAGE COMING ere 1 2 1 3 System requiremenls e en oer anna anna emunt tma rn nia anni n taa ie E FRE n aka Unus 1 2 1 4 AYO me 1 3 1 5 Replacing the card Bracket nk e sus adieu Eo cix 1 4 Chapter 2 Firmware update 2 1 Updating the firmware t 2 2 Chapte
8. Linux OS Z For users using SUSE Linux Enterprise Server OS follow the general instructions in this section to mount the Mellanox driver on your system To mount the Mellanox driver 1 Login to the system as root 2 Create a folder on the desktop and copy the latest Mellanox driver into the same folder Rename the folder according to your preference then key in the following commands cd mkdir folder name root localhost tmp File Edit View Search Terminal Help root localhost tmp mkdir mnt root localhost tmp J 3 Key in the command mount o ro loop tmp image file name folder name then mount the driver into the system The driver version and the OS label may vary Ensure that the command line matches your driver version and operating system root localhost tmp File Edit View Search Terminal Help root localhost tmp mkdir mnt root localhost tmp mount o ro loop MLNX OFED LINUX 2 1 1 0 6 rhel6 4 x86 64 iso mnt root localhost tmp B ASUS PEM FDR 3 6 4 the following commands cd folder name mlnxofedinstall root localhost mnt File Edit View Search Terminal Help root localhost tmp cd mnt root localhost mnt mlnxofedinstallfj All other Mellanox OEM OFED or distribution IB packages will be removed 5 Press lt Y gt to continue root localhost mnt File Edit View Search Terminal Help root localhost tm
9. Please read the following license agreement carefully Copyright c 2005 2011 Mellanox Technologies All rights reserved Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions are met Redistributions of source code must retain the above copyright notice this list of conditions and the following disclaimer ILC ENT il inna in hinam fnrm mant tha ler 6 Click Change to select your desired destination folder then click Next to continue 7 Tick the check box and click Next to continue ji MLNX VPI InstallShield Wizard Maximum Performance Configure your system for maximum 10GigE performance v ASUS PEM FDR 3 3 3 4 10 Click Install to begin the installation ii VPI InstallShield Wizard E xj Ready to Install the Program The wizard is ready to begin installation 1 MLNX_VPI InstallShield Wizard The InstallShield Wizard has successfully installed MLNX VPI Click Finish to exit the wizard You chose to run performance tuning The log file can be found at eem ng Enable WinSock Direct protocol WSD Enable Socket Direct Protocol SDP Enable NetworkDirect interface ND Show IB release notes Show ETH release notes Drivers Please select items right side MLNX VPI Installer Information Chapter 3 Driver installation 3 1 2 RedHat Enterprise
10. ojur njyurmu osneo jou IMAP siu suonrpuoo 01 P fqns st samy DOH Jo wed 4 IMAP stu uoneurioju euonuojuru g wedqng DOA suoneorgroeds SUTMOT OJ AY 0 surioguo JequinwN pie 19npoJd jonpoud yey sae poop SSSh SO9 OTS LLLE 6EL OTS ON Jua AB AA 008 55 snsy o qrsuodso LLOT DDA Pd ALINYOANOD AO NOLLV2IV IOHG
11. or P9 series platform server motherboard or system Chapter 1 Product introduction 1 4 Card layout Dn N L PH EEHEHBEEHHEEHBHEEHEERBEENI nlala la lala a alaaa la ig HHHHHHHHdi I ey 1 QSFP connector PCI Express Gen3 x8 interface MLED1 MLED2 P N LED indications MLED2 Activity Link LED OFF No device Orange Device plugged in Ready Device plugged in Data transmitting Description Orange Blinking ASUS PEM FDR 18 1 5 Replacing the card bracket The ASUS PEM FDR card is bundled with a full heigth bracket a low profile bracket and an ASUS proprietary bracket By default the card is pre installed with an ASUS proprietary bracket ES4000 G3 series and RS720Q E8 series models The supported list of models are Z The proprietray bracket is a small bracket for node that supports ASUS Z10PH D16 series subject to change without prior notice To replace the card bracket 1 Get the PEM FDR card and place it on a stable and flat surface Ensure that the printed circuit board PCB is facing down 2 Remove the two screws that secures the PEM FDR card to the card bracket then remove the card bracket 3 Get the replacement card bracket 4 X Align and insert the PEM FDR card into the mounting hole of the card bracket Ensure that the screw holes on the PEM FDR card matches the screw holes of the card bracket 5 Replace the
12. p cd mnt root localhost mnt mlnxofedinstall Logs dir tmp MLNX_OFED_LINUX 2 1 1 0 6 9553 1095 This program will install the MLNX OFED LINUX package on your machine Note that all other Mellanox OEM OFED or Distribution IB packages will be rem oved Do you want to continue y N 6 Wait while the system installs the driver File Edit View Search Terminal Help root localhost tmp cd mnt root localhost mnt mlnxofedinstall Logs dir tmp MLNX OFED LINUX 2 1 1 0 6 9553 10gs This program will install the MLNX OFED LINUX package on your machine Note that all other Mellanox OEM OFED or Distribution IB packages will be rem oved Do you want to continue y N y b bin rpm nosignature e allmatches nodeps libibverbs libibverbs utils libm thca libmlx4 libcxgb3 libnes libipathverbs libibcm libibumad libibmad ibsim ibac m librdmacm librdmacm utils opensm libs dapl rds tools ibutils infinipath psm li bibverbs libibverbs utils libmthca libmlx4 libcxgb3 libnes libipathverbs libibcm libibumad libibmad ibsim ibacm librdmacm librdmacm utils opensm libs compat ope nsm libs dapl rds tools infinipath psm opensm libs ibutils ibutils libs fcoe uti ls compat openmpi compat openmpi psm rdma 3 6 1 el6 noarch rdma Starting MLNX OFED LINUX 2 1 1 0 6 installation Installing mlnx ofa kernel kmod mlnx ofa kernel 2 1 RPM Chapter 3 Driver installation e
13. r 3 Driver installation 3 1 Mellanox ConnectX 3 Infiniband driver installation 3 2 3 1 1 Windows operating system es 3 2 3 1 2 Red Hat Enterprise Linux OS seen 3 5 ASUS contact 1 About this guide This user guide contains the information you need when installing and configuring the server management board How this guide is organized This guide contains the following parts Chapter 1 Product introduction This chapter describes the PEM FDR Ethernet card features and the new technologies it supports Chapter 2 Firmware update This chapter provides information in updating the firmware of PEM FDR card Chapter 3 Driver installation This chapter provides instructions in installing the PEM FDR card s drivers on different operating systems Where to find more information Refer to the following sources for additional information and for product and software updates 1 ASUS websites The ASUS website provides updated information on ASUS hardware and software products Refer to the ASUS contact information 2 Optional documentation Your product package may include optional documentation such as warranty flyers that may have been added by your dealer These documents are not part of the standard package Conventions used in
14. screws that you removed earlier in step 2 m m it ELE HA Chapter 1 Product introduction Firmware update This chapter provides information in updating the firmware of PEM FDR card 2 1 Updating the firmware Z You can download the latest firmware of PEM FDR at www asus com To update the firmware 1 Unzip or extract the firmware file then copy it into a bootable USB device 2 Goto the DOS prompt and key in the following command mst start INIBNFDR gt mst start Jib sfdr vmst exe List available mst devices Isage c ib fdr mst exe status 3 the following command mst status CINIBNFDROmst status Found 3 devices mt4899 pciconf B nt4899 pci crB mt48B99 pci uarB Z Take note of the name of the device ee 4 Key in the following command flnnt d device name i file name bin allow psid change nofs burn Chapter 3 Driver installation 0920 bin cwsdpmi flint i2c i2c chck i2c scan mcra mst update Ensure to key in the correct device name A and file name B of the firmware lt NIBNFDROf lint mt4699_pciconfs i 8928 bin allow_psid_change nofs burn Current FW version on flash 2 10 08 New FW version 2 11 5t process will not be failsafe No checks will be perforned iLL flash including the Invariant Sector will be overwritten If this process fails computer may renain in an inoperable state
15. this guide To make sure that you perform certain tasks properly take note of the following symbols used throughout this manual DANGER WARNING Information to prevent injury to yourself when trying to complete a task CAUTION Information to prevent damage to the components when trying to complete a task IMPORTANT Instructions that you MUST follow to complete a task NOTE Tips and additional information to help you complete a gt gt task Typography Bold text Indicates a menu or an item to select Italics Used to emphasize a word or a phrase Key Keys enclosed in the less than and greater than sign means that you must press the enclosed key Example Enter means that you must press the Enter or Return key lt Key1 gt lt Key2 gt lt Key3 gt If you must press two or more keys simultaneously the key names are linked with a plus sign Example lt Ctrl gt lt Alt gt lt Del gt Command Means that you must type the command exactly as shown then supply the required item or value enclosed in brackets Example At the DOS prompt type the command line format a vi PEM FDR card specifications summary PEM FDR Controller Mellanox ConnectX 3 FDR Interface PCI E Gen3 x8 Port QSFP Data transfer rate Up to 56 Gb s FDR InfiniBand Form factor 4 72 in x 1 77 in lower than standard low profile Specifications are subject to change without notice Supports ASUS 210

PEM-FDR

Contents

Download Pdf Manuals

Related Search

Related Contents