InfiniHost PCI-X Low Profile RoHS HCA Adapter Cards User Manual
Contents
1. Example HCA Card MHET2X 2TC OPN Field Decoder Tall Bracket MHTS I XBR w 256MB Memory Mellanox Technologies M H HCA H HCA Card S Express Card SIOM H Media C Cu SDR E Cu SDR Fiber Adapter G Cu DDR E Fiber Adapter currently SDR only F Fiber SDR 5 HCA IC T InfiniHost A InfiniHost III Ex S InfiniHost III T Lx IB ports 1 1 222 A 10 B 11 limit to one character 2 I Interface X PCI X 4 PCle x4 8 PCle x8 x Separator x Memory Size X MemFree 1 128MB 2 256MB 3 512MB 2 B Bracket S Short T Tall N None T R RoHS lt blank gt non RoHS C RoHS w Exemption R RoHS C Lead Free Mellanox Technologies 9 Rev 1 3 Overview 1 3 Finding the GUID and Serial Number on the Adapter Cards All mellanox HCA adapter cards have a label on the printed side of the adapter card that has the card serial number and the card GUID Figure 2 Product Label MT0744X000 P N MHET2X 1SC d n Il II 2F Made in IL 10 Mellanox Technologies InfiniHost Low Profile Adapter Rev 1 3 2 HCA Card Installation 2 1 Hardware and Software Requirements Before installing the HCA Adapter card please make sure that the system meets the hardware and software requirements listed in Table 5 Table 5 Hardware and Soflware Requirements Requirement Description Hardware Software For Windows see http www mellanox com gt Downloads gt Infin2. amp HR cUCUo CORBPIVAPTLEY ERIC LT EFEHinde 216225 862926505089 Co U C F uv Translation This is Class product based the standard of the Voluntary Control Council for Interference by Information Technology Equipment VCCI If this is used near a radio or television receiver in a domestic environment it may cause radio interference Install and use the equipment according to instruction manual A 3 5 MIC Statement Republic of Korea Korea s Regulation for Certification of Information and Communication Equipment requires EMC testing and certification for many electronic products Korean EMC certifications are issued by Radio Research Laboratory RRL which is organized under the Ministry of Information and Communications MIC EMC testing includes electromagnetic emissions EMI and susceptibil ity EMS Certified equipment is labeled with the MIC mark and certification number Class A Statement 7 7 IFF2E SERERE 9 71710 9 4 BHA EE AMR ae BH 987 BR rl TE 9 81918 WE ESOS AA Translation Class A Device This device is registered for EMC requirements for industrial use The seller or buyer should be aware of this If this type was sold or purchased by mistake it should be replaced with a residential use type 28 Mellanox Technologies J InfiniHost Low Profile Adapter Rev 1 3 Class
3. This equipment should be installed replaced or serviced only by trained and A qualified personnel 6 Equipment Disposal Disposal of this equipment should be in accordance to all national laws and N regulations 7 Local and National Electrical Codes This equipment should be installed in compliance with local and national elec trical codes 12 Mellanox Technologies J InfiniHost PCI X Low Profile Adapter Card Rev 1 3 3 Driver Software and Firmware 3 1 Driver Software For Linux or Windows download and install the latest OpenFabrics Enterprise Distribution OFED software package available via the Mellanox OpenFabrics Web site at http www mellanox com gt Downloads gt InfiniBand SW Drivers Follow the installation instruc tions included in the download package 3 2 Updating HCA Card Firmware Each HCA card is shipped with the latest version of qualified firmware at the time of manufactur ing Firmware is updated occasionally and the most recent firmware can be obtained from http www mellanox com through the Support page The HCA firmware download pages include the following firmware update options Single HCA card firmware update InfiniBand cluster firmware update Customized firmware mlx ini firmware update OEMs Only 3 3 Single HCA Card Firmware Update Firmware can be updated on the standalone single card using the flint tool of the Mellanox Firm ware Tools MFT package This
4. ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Mellanox TECHNOLOGIES Mellanox Technologies Mellanox Technologies Ltd 350 Oakmead Parkway Suite 100 PO Box 586 Hermon Building Sunnyvale CA 94085 Yokneam 20692 U S A Israel www mellanox com Tel 972 4 909 7200 Tel 408 970 3400 Fax 972 4 959 3245 Fax 408 970 3403 Copyright 2010 Mellanox Technologies All rights reserved Mellanox BridgeX ConnectX Virtual Protocol Interconnect InfiniBlast InfiniBridge InfiniHost InfiniRISC InfiniScale and InfiniPCI are registered trademarks of Mellanox Technologies Ltd CORE Direct FabricIT and PhyX are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners All other marks and names mentioned herein may be trademarks of their respective companies InfiniHost PCI X Low Profile RoHS HCA Adapter Cards User s Manual 2 Mellanox Technologies Document Number 2680 InfiniHost Low Profile Adapter Rev 1 3 Contents Contents 3 List of Tables 5 Revision History 6 About this Manual 7 Chapter 1 Overview 8 1 1 List of HCA Cards 9 12 HCA Cards Part Numbering Key 9 13 Finding the GUID and Serial Number on the Adapter Cards 10 Chapter 2 Card Installation 11 2 1 Hardware and Software Requirements 11 2 2 Installation Instructions 11 2 3 Safety Warnings 11 Chapter 3 Driver Software and Firmware 13 3 1 3 1 Driver Software 13 3 2 Updating HCA C
5. String Tag 0x02 0x82 1 Length OxA 3 Data Cougar cub Alphanumeric 13 Large Resource Type VPD R Tag 0x10 0x90 14 Length Ox4F 16 VPD Keyword PN Numbers Add in Card Part Number 18 Length 0x15 19 Data MHET2X 1SC VPD Keyword EC Alphanumeric Engineering Change 40 Level of the card rev 42 Length 0x2 43 Data Al Upper level revision 45 VPD Keyword SN Alphanumeric Serial Number 47 Length 0x18 48 Data MTYYWWF SSSSS Mellanox Technologies 19 Rev 1 3 Vital Product Data VPD Format Table 9 VPD Format for MHET2X 1SC Ban Item Value Format Description 72 VPD Keyword V0 Currently not in use 74 Length 0x10 75 Data N A 91 VPD Keyword RV 93 Length 0 1 94 Data Checksum 95 Large Resource Type VPD W Tag 0x11 0x91 96 Length 0x9D 98 VPD Keyword vi EFI Driver version 100 Length 0x6 101 Data N A Number 107 VPD Keyword YA Asset Tag 109 Length 0x20 110 Data N A Alphanumeric N A 142 VPD Keyword RW Remaining read write area 144 Length Ox6e 145 Data Reserved 0x00 255 Small Resource Type END Tag 0x11 0x78 256 Mellanox Read Only Mask 0x0 0 Numbers 356 Mellanox Read Write Mask Ox1 1 Numbers 511 Mellanox Read Only Mask 0x0 Numbers 5 2 PCI VPD Layout for MHET2X 1TC Note A1 was used as the HCA card upper level re
6. bits Mellanox Technologies 7 Rev 1 3 Overview 1 Overview This document is a User s Manual for Mellanox Technologies host channel adapter HCA cards based on the MT23108 InfiniHost HCA IC device Each of the cards described has the following features IBTA v1 2 1 compliant Two 4X InfiniBand copper ports for connecting InfiniBand traffic 4X IB connectors Each IB port supports a 10Gb s transmission rate Single Data Rate or SDR Each IB port includes a media detect circuit supporting external InfiniBand fiber solutions e PCI 2 2 and PCI X 1 0 specifications compliant with a standard 3 3 Volt PCI X edge connector On board DDR SDRAM memory 128MByte or 256MByte EU Restriction of Hazardous Substances RoHS compliant Figure 1 MHET2X 1TC HCA m acis m 8 Mellanox Technologies InfiniHost Low Profile Adapter Rev 1 3 1 1 List of HCA Cards Table 3 lists the HCA cards described in this manual Table 3 HCA Cards List on Short Ordering Part IB Port board Tall RoHS Number OPN Speed Memory Bracket Compliance Size MHET2X 1SC 10Gb s 128 Short RoHS R5 SDR MByte MHET2X 1TC Tall MHET2X 2SC 256 Short MByte MHET2X 2TC Tall 1 2 Mellanox HCA Cards Part Numbering Key Table 4 describes the Mellanox Technologies HCA cards part numbering key Table 4 Mellanox HCA Cards Part Numbering Key
7. link illuminates once VAPI is started and a physical connection is made between two nodes The logical link yellow LED illu minates once the InfiniBand network is discovered over the physical link If the LEDs are not active either the physical or the logical or both connections have not been established Table 7 LEDs Port Number LED Name Port 1 Physical Link Green Logical Link Yellow Port 2 Physical Link Green Logical Link Yellow Mellanox Technologies 15 Rev 1 3 Adapter Card Interfaces Figure 3 shows the common I O panel of all the tall bracket HCA cards covered in this User s Manual Both IB ports are shown along with the physical and activity link LEDs Figure 3 1 0 Panel with Dual Ports and LEDs Tall Bracket Port 1 Physical Link Port 1 Activity Green LED Logical Link ellow LED Port 1 Port 2 Activity Logical Link Yellow LED Port 2 Physical Link Green LED Port 2 4 1 4 12C Interface A three pin header reference name J3 in Figure 5 on page 26 is provided as the I2C interface Figure 4 I2C interface 16 Mellanox Technologies J InfiniHost Low Profile Adapter Rev 1 3 42 Power The HCA board is a 3 3 V compliant design 3 3 V only The board receives 3 3V and 5V power from the PCI X Edge connector All other required power is generated by on board switch mode regulators For power consumption see Specifications on
8. page 26 43 Memory The HCA supports multiple memory devices on board through the DDR SDRAM Flash and I2C interfaces 4 3 1 DDR SDRAM The HCA board is designed with multiple memory configurations to be placed on the board The board is compliant with the JEDEC standard JESD79 Double Data Rate DDR SDRAM Speci fication The board is generally populated with unbuffered 128 256 or 512MB ECC 133 266 or 166 333 MHz memory configurations can vary Memory speed and size are encoded in the standard Mellanox Ordering Part Number OPN Cus tom part numbers may be different An example would be the MHXL CF128 T this is an HCA board with a 166MHz 128MB DDR memory 4 3 2 FLASH ROM The HCA board supports 4MB of FLASH ROM space via the CPU interface of the MT23108 The details of the FLASH supported are 8 bit data 19 bit address Programming via the CPU bus The FLASH Macronix MX29LV033ATC 90G has the following characteristics 32 Mb Uniform 4MB 90 ns access time 4 3 3 EEPROM The board incorporates an EEPROM that is accessible through an I2C interface The EEPROM is used for storing the Vital Product Data VPD VPD format adheres to the PCI Local Bus specifi cation rev 2 3 VPD definition see Chapter 5 on page 19 EEPROM capacity is 256K Byte Mellanox Technologies 17 Rev 1 3 Adapter Card Interfaces 4 4 Jumper Configuration Table 8 Jumper Configuration Ref Default d Name Description O
9. version 100 Length 0x6 101 Data N A Number 107 VPD Keyword YA Asset Tag 109 Length 0x20 110 Data N A Alphanumeric N A 142 VPD Keyword RW Remaining read write area 144 Length Ox6e 145 Data Reserved 0x00 255 Small Resource Type END Tag 0x11 0x78 256 Mellanox Read Only Mask 0x0 0 Numbers 356 Mellanox Read Write Mask Ox1 1 Numbers 511 Mellanox Read Only Mask 0x0 Numbers Mellanox Technologies 23 Rev 1 3 5 4 PCI VPD Layout for MHET2X 2TC Vital Product Data VPD Format Note A1 was used as the HCA card upper level revision Later revisions of the HCA card will have the same format Table 12 VPD Format for MHET2X 2TC Offset atus Decimal Item Value Format Description 0 Large Resource Type ID String Tag 0x02 0x82 1 Length OxA 3 Data Cougar cub Alphanumeric 13 Large Resource Type VPD R Tag 0x10 0x90 14 Length Ox4F 16 VPD Keyword PN Numbers Add in Card Part Number 18 Length 0x15 19 Data MHET2X 2TC VPD Keyword EC Alphanumeric Engineering 40 Change Level of the card rev 42 Length 0x2 43 Data Al Upper level revi sion 45 VPD Keyword SN Alphanumeric Serial Number 47 Length 0x18 48 Data MTYYWWF SSSSS 72 VPD Keyword vo Currently not in use 74 Length 0x10 75 Data N A 91 VPD Keyword RV 93 Length 0 1 94 Data Checksum 95 Large Resource Type
10. 003 class B EN 61000 3 2 2000 A2 2005 EN61000 3 3 1995 A 1 2000 EN 55024 1998 A1 2001 A2 2003 stan dards harmonized under EMC Directive 89 336 EEC TEC EN 60950 1 2001 ETSI EN 300 019 2 2 TEC 60068 2 64 29 32 RoHS R5 30 Mellanox Technologies InfiniHost Low Profile Adapter Rev 1 3 Appendix B Interface Connectors Pinout 1 12C compatible Connector Pinout Figure 7 I2C compatible Connector Table 15 I2C compatible Connector Pinout Connector Pin Number HCA Signal Name 1 SPSDA 2 SPSCL 3 GND 4 NC 5 NC B 2 InfiniBand Connector Pinout Figure 8 InfiniBand 4X Port Connector Table 16 InfiniBand 4X Connector Pinout Sheet 1 of 2 IBtxOp 3 IBtxIp 3 S8 IBtxOn 3 IBtxin 3 E Connector Pin Connector Pin IB Port A Signal IB Port B Signal Number Name Name Name IBtxOp 2 IBtxIp 2 jston2 4X S1 IBtxIp 0 Rx_Al Rx 84 ooi E S2 IBtxIn 0 Rx A0 Rx 14 1 t8beOn t IBtxIn 1 S3 IBtxIp 1 Rx_A3 Rx_B3 319 IBtxOp 0 IBtxIp 0 S4 IBtxIn 1 Rx_A2 Rx_B2 S15 IBtxOn 0 IBtxIn D S5 IBtxIp 2 Rx A5 Rx B5 S6 IBtxIn 2 Rx A4 Rx B4 S7 IBtxIp 3 Rx A7 Rx B7 S8 IBtxIn 3 Rx A6 Rx B6 S9 IBtxOn 3 Tx A6 Tx B6 S10 IBtxOp 3 Tx_A7 Tx_B7 S11 IBtxOn 2 Tx A4 Tx B4 S12 IBtxOp 2 Tx_A5 Tx_B5 S13 IBtxOn 1 Tx A2 Tx B2 Mellanox Technologies 31 Rev 1 3
11. 1 DN 3 4 B A _ o BE U 7 anes 5 J ESF Sonia 7 _ gt 5 PIDE E ui e N N U m E FA Ey pe n Lo p 5 m sa 0188 BB PE Dm 1 us W JF m Y E EL gt lt ER ul 15 014 10 J4 5 382 10 _ P oe Present Not present Jumper 35 391 10 UT 50 84 10 15 602 10 163 09 10 ALL DIMENSIONS ARE IN MM Note Short brackets have the same port and LED footprints as tall brackets See Instructions for Replacing a Tall Bracket with a Short Bracket on HCA Cards on page 33 26 Mellanox Technologies J InfiniHost Low Profile Adapter Rev 1 3 EMC Certification Statements Table 13 lists the approved EMC certification status per HCA card in different regions of the world Table 13 HCA Cards EMC Certification Status FCC Class EN Class ICES Class MIC BCC EN USA Europe Canada Korea MHET2X 1SC B B B MHET2X 1TC B B B MHET2X 28C B B B MHET2X 2TC B B B A 3 1 FCC Statements USA Class B Statements 15 21 Statement Warning Changes or modifications to this equipment not expressly approved by the party responsible for compliance Mellanox Technologies could void the user s authority to operate the equipment 15 105 Statement NOTE This equipment has been tested and found to comply with the limits for a Class B dig ital devi
12. 5 Data Reserved 0x00 255 Small Resource Type END Tag 0x11 0x78 256 Mellanox Read Only Mask 0x0 0 Numbers 356 Mellanox Read Write Mask Ox1 1 Numbers 511 Mellanox Read Only Mask 0x0 Numbers 5 3 PCI VPD Layout for MHET2X 2SC Note A1 was used as the HCA card upper level revision Later revisions of the HCA card will have the same format Table 11 VPD Format for MHET2X 2SC an Item Value Format Description 0 Large Resource Type ID String Tag 0x02 0x82 1 Length OxA 3 Data Cougar cub Alphanumeric 13 Large Resource Type VPD R Tag 0x10 0x90 14 Length Ox4F 16 VPD Keyword PN Numbers Add in Card Part Number 18 Length 0x15 19 Data MHET2X 2SC VPD Keyword EC Alphanumeric Engineering Change 40 Level of the card rev 42 Length 0x2 43 Data Al Upper level revision 45 VPD Keyword SN Alphanumeric Serial Number 47 Length 0x18 22 Mellanox Technologies J InfiniHost Low Profile Adapter Rev 1 3 Table 11 VPD Format for MHET2X 2SC am Item Value Format Description 48 Data MTYYWWF SSSSS 72 VPD Keyword NO Currently not in use 74 Length 0x10 75 Data N A 91 VPD Keyword RV 93 Length 0 1 94 Data Checksum 95 Large Resource Type VPD W Tag 0x11 0x91 96 Length 0x9D 98 VPD Keyword vi EFI Driver
13. B Statement CORE PHLBRESERRSAEMMBRS VCCI DEH KES lt SITABARB RETT COSXIBIZ RERA CIEI 2 amp HR CUCUo ES CORBPIVAPTLEY ERIC LT HAGNSE SMBS FHMC TFCEPHVET PIRES gt CIEL LU C F uv Translation This is a Class B product based on the standard of the Voluntary Control Council for Interference by Information Technology Equipment VCCI If this is used near a radio or television receiver in a domestic environment it may cause radio interference Install and use the equipment according to instruction manual Mellanox Technologies 29 Table 14 lists the specifications for the HCA cards Table 14 HCA Cards Specifications Rev 1 3 4 Cards Specifications Physical Power and Environmental Size 2 5in x 6 6in 6 4cm x 16 8cm Voltage 5V 3 3V Air Flow 200LFM 55 C Maximum Power 12W 4X 10Gb s Connector InfiniBand Copper current rating Temperature 0 C to 55 C 0 5A max with active media adapter On board Memory support Size 128MB MHET2X 1SC 1TC 256MB MHET2X 2SC 2TC Protocol Support Regulatory InfiniBand IBTA v1 2 1 Auto Negotiation 10Gb EMC FCC 47 CER part 15 2005 subpart B s 2 5Gb s class B QoS 8 InfiniBand Virtual Lanes for each Support port ICES 003 2004 Issue 4 class B PCI Yes All Ports PCI X 22 VCCI V 3 2005 04 class B 1 0 Safety Environmental RoHS EN 55022 1998 1 2000 A2 2
14. Images for InfiniHost PCI X HCA Cards 13 Table 7 LEDs 15 Table 8 Jumper Configuration 18 Table 9 Format for MHET2X 1SC 19 Table 10 VPD Format for MHET2X 1TC 21 Table 11 VPD Format for MHET2X 2SC 22 Table 12 VPD Format for MHET2X 2TC 24 Table 13 HCA Cards EMC Certification Status 27 Table 14 Cards Specifications 30 Table 15 I2C compatible Connector Pinout 31 Table 16 InfiniBand Connector Pinout 31 Mellanox Technologies 5 Rev 1 3 Revision History Table 1 Revision History Rev Date Comments Changes Imported new template 13 August Changed IBTA 1 2 to 1 2 1 2010 Fixed links to new Website Added safety warnings in French German and Spanish Replaced Safety warnings and added Safety warnings in French 12 August Removed watermark and changed footer 2008 Added MIC BCC to Cert Table Added Korean Certs Statements December Added EMC VCCI statements to Appendix A Specifications on page 26 1 11 20 2006 November Added EMC Class B statements to Appendix A Specifications on page 26 1 10 27 2006 1 00 November First version of the User s Manual 2 2006 6 Mellanox Technologies InfiniHost Low Profile Adapter Rev 1 3 About this Manual This card User s Manual describes Mellanox Technologies InfiniHost PCI PCI X RoHS compli ant HCA cards It provides details as to the interfaces of the board specifications requi
15. Lisez toutes les instructions d installation avant de brancher le mat riel la source d alimentation lectrique 2 Temp rature excessive Ce mat riel ne doit pas fonctionner dans une zone avec une temp rature ambi ante d passant le maximum recommand de 55 C 131 F En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouvertures de ventilation 3 Orages dangers lectriques Pendant un orage il ne faut pas utiliser le mat riel et il ne faut pas brancher ou d brancher les c bles P 4 Branchement d branchement des c bles InfiniBand en cuivre Les c bles InfiniBand en cuivre sont lourds et ne sont pas flexibles il faut donc faire tr s attention en les branchant et en les d branchant des connecteurs Consultez le fabricant des c bles pour conna tre les mises en garde et les instructions sp ciales gt 5 Montage et entretien sur baie Lorsque ce produit est mont ou entretenu sur baie il faut prendre des pr caut ions sp ciales pour s assurer que le syst me reste stable En g n ral il faut remplir la baie avec du mat riel de bas en haut gt 6 Installation du mat riel Ce mat riel ne doit tre install remplac ou entretenu que par du personnel form et qualifi gt 7 Elimination du mat riel L limination de ce mat riel doit s effectuer dans le respect de toutes les l gisl ations et r glementations nationa
16. Mellanox TECHNOLOGIES InfiniHost PCI X Low Profile ROHS HCA Adapter Cards User Manual P N MHET2X 1SC MHET2X 1TC MHET2X 2SC MHET2X 2TC Rev 1 3 www mellanox com Rev 1 3 NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRODUCT S AND RELATED DOCUMENTATION EVEN IF
17. Table 16 InfiniBand 4X Connector Pinout Sheet 2 of 2 Connector Pin Connector Pin IB Port A Signal IB Port B Signal Number Name Name Name S14 IBtxOp 1 Tx_A3 Tx_B3 S15 IBtxOn 0 Tx A0 Tx BO S16 IBtxOp 0 Tx AI Tx Bl G1 G6 G9 H1 Signal Ground GND GND H2 Sense 3 3V SENSE PI SENSE P2 G8 Vcc MC POWER MC POWER P2 a The Sense 3 3V signal is used to enable the Vcc power supply pin G8 used to provide power to the active media adapter B 3 PCI X Edge Connector Pinout The HCA card uses a standard PCI X 1 0 edge connector See http www pcisig com specifica tions 32 Mellanox Technologies J InfiniHost PCI X Low Profile Adapter Card Rev 1 3 Appendix C Instructions for Replacing a Tall Bracket with a Short Bracket on HCA Cards This appendix provides instructions on how to remove a tall bracket of an HCA card such as the one shown in Figure 9 and replace it with a short one Figure 9 Tall Bracket of a Dual IB Port HCA Card C 1 Removing Tall Bracket Step 1 Remove connector clips Figure 10 shows a connector retention clip and the designated names of Its sections Figure 10 Connector Retention Clip Clip Top Clip Center embraces the connector Clip Hook grips onto the connector Using a small flat head screwdriver gently push up one hook of a connector s clip toward the con nector s top side as shown in Figure 11 a on page 34 The
18. VPD W Tag 0x11 0x91 96 Length 0x9D 98 VPD Keyword V1 EFI Driver ver sion 24 Mellanox Technologies J InfiniHost Low Profile Adapter Rev 1 3 Table 12 VPD Format for MHET2X 2TC Ben Item Value Format Description 100 Length 0x6 101 Data N A Number 107 VPD Keyword YA Asset Tag 109 Length 0x20 110 Data N A Alphanumeric N A 142 VPD Keyword RW Remaining read write area 144 Length Ox6e 145 Data Reserved 0x00 255 Small Resource Type END Tag 0x11 0x78 256 Mellanox Read Only Mask 0x0 0 Numbers 356 Mellanox Read Write Mask Ox1 1 Numbers 511 Mellanox Read Only Mask 0x0 Numbers Mellanox Technologies 25 Rev 1 3 Appendix Specifications A 1 Air Flow Requirement Per the PCI PCI X specifications all the HCA cards covered in this User s Manual require an air flow of 200 LFM at 55 C ambient temperature A 2 Board Mechanical Drawings and Dimensions All the HCA cards covered in this User s Manual have the same mechanical drawing and share the same dimensions as depicted in Figure 5 except for the bracket length short or tall Figure 5 HCA Cards Mechanical Drawing and Dimensions mm 2 167 64 10 a TC Header D SEE 295 ae 87 c SAN NO D JE gt m 7 4 12 AS I uli JEDD _
19. ard Firmware 13 3 33 Single HCA Card Firmware Update 13 3 4 HCA Card Firmware Update as Part of a Cluster Firmware Update 13 3 5 Customized HCA Card Firmware Update OEM Only 14 Chapter 4 Adapter Card Interfaces 15 4 1 I O Interfaces 15 4 1 1 InfiniBand Interface 15 4 1 2 PCI X Interface 15 4 1 3 LED Assignment 15 4 1 4 I2C Interface 16 42 Power 17 43 Memory 17 4 3 1 DDR SDRAM 17 4 3 2 FLASH ROM 17 4 3 8 EEPROM 17 4 4 Jumper Configuration 18 Chapter 5 Vital Product Data VPD Format 19 5 1 PCI VPD Layout for MHET2X 1SC 19 5 2 PCI VPD Layout for MHET2X 1TC 20 5 3 PCI VPD Layout for MHET2X 2SC 22 5 4 PCI VPD Layout for MHET2X 2TC 24 Appendix A Specifications 26 A 1 Air Flow Requirement 26 A 2 Board Mechanical Drawings and Dimensions 26 EMC Certification Statements 27 A 3 1 FCC Statements USA 27 A 3 2 EN Statements Europe 28 A 3 3 ICES Statements Canada 28 A 3 4 VCCI Statements Japan 28 A 3 5 MIC Statement Republic of Korea 28 Appendix B Interface Connectors Pinout 31 Appendix C Instructions for Replacing a Tall Bracket with a Short Bracket on HCA Cards 33 Appendix D Avertissements de s curit d installation 38 Mellanox Technologies 3 InfiniHost Low Profile Adapter Rev 1 3 List of Tables Table 1 Revision History 6 Table 2 Documents List 7 Table 3 Cards List 9 Table 4 HCA Cards Part Numbering Key 9 Table 5 Hardware and Software Requirements 11 Table 6 Firmware
20. bling Short Bracket The short bracket can now be assembled onto the HCA card See Figure 14 Figure 14 HCA Card Ready for Short Bracket Mellanox Technologies 35 Rev 1 3 Step 1 Place short bracket onto card Gently place the bracket onto the card fitting the connectors through the bracket connector holes Make sure the LEDs are aligned into their intended bracket holes Figure 15 Place Tall Bracket onto Card LED Holes Step 2 Attach short bracket to card Insert a screw along with a washer into each of the two holes on the card intended for holding the bracket Use a torque screwdriver to apply up to 2 Ibs in torque on each screw Figure 16 Attach Bracket onto Card using 36 Mellanox Technologies InfiniHost amp PCI X Low Profile Adapter Card Rev 1 3 Step 3 Install Connector Clips Gently push one clip onto the connector Make sure to slide both clip hooks sides around the connector evenly as shown in Figure 17 Figure 17 Sliding Connector Clip Evenly Use a small flat head screwdriver to gently slide the clip s hook towards the connector s base side as shown in Figure 18 Figure 18 Fix Clip Hooks into Place Using Screw Repeat this step for the second clip See Figure 19 Assembled Short Bracket View Figure 19 Assembled Short Bracket View Mellanox Technologies 37 Rev 1 3 Appendix D Avertissements de s curit d installation m Instructions d installation
21. ce pursuant to part 15 of the FCC Rules These limits are designed to provide reason able protection against harmful interference in a residential installation This equipment generates uses and can radiate radio frequency energy and if not installed and used in accor dance with the instructions may cause harmful interference to radio communications However there is no guarantee that interference will not occur in a particular installation If this equip ment does cause harmful interference to radio or television reception which can be determined by turning the equipment off and on the user is encouraged to try to correct the interference by one or more of the following measures Reorient or relocate the receiving antenna Increase the separation between the equipment and receiver Connect the equipment into an outlet on a circuit different from that to which the receiver is con nected Consult the dealer or an experienced radio TV technician for help Mellanox Technologies 27 Rev 1 3 3 2 EN Statements Europe EN55022 Class B Statement No statement is required for Class B products A 3 3 ICES Statements Canada Class B Statement This Class B digital apparatus complies with Canadian ICES 003 Cet appareil num rique de la classe B est conforme la norme NMB 003 du Canada A 3 4 VCCI Statements Japan Class B Statement CORE VCCI DEH KES lt SITABARB RETT CORE CIEI 2
22. chnologies InfiniHost PCI X Low Profile Adapter Card Rev 1 3 Appendix F Advertencias de seguridad para la instalaci n Spanish 1 Instrucciones de instalaci n Antes de conectar el equipo a la fuente de alimentaci n leer todas las instruc clones de instalaci n P 2 Instalaci n en unlugar con acceso restringido 9 Esta unidad ha sido ideada para instalar en lugares de acceso restringido 3 Sobrecalentamiento No se debe utilizar el equipo en un rea con una temperatura ambiente superior a la m xima recomendada 45 C Adem s para garantizar una circulaci n de aire adecuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alre dedor de las aberturas de ventilaci n 4 Cuando hay rayos peligro de descarga el ctrica No utilizar el equipo ni conectar o desconectar cables durante per odos de activ idad de rayos 5 Conexi n y desconexi n del cable Copper InfiniBand Dado que los cables de cobre InfiniBand son pesados y no son flexibles su con exi n a los conectores y su desconexi n se deben efectuar con mucho cuidado Para ver advertencias o instrucciones especiales consultar al fabricante del cable P 6 Montaje y mantenimiento de bastidores Alinstalar o realizar el mantenimiento de este aparato en un bastidor es preciso adoptar precauciones especiales para garantizar que el sistema se mantenga estable En general en un bastidor los equipos se deben instalar comenzando d
23. einen sollten Sie das Gestell von unten nach oben mit Ger ten f llen 6 Ger teinstallation Diese Ger t sollte nur von geschultem und qualifiziertem Personal installiert ausge tauscht oder gewartet werden Mellanox Technologies 39 Rev 1 3 7 Ger teentsorgung Die Entsorgung dieses Ger ts sollte unter Beachtung aller nationalen Gesetze Bestim mungen erfolgen gt 8 Regionale und nationale elektrische Bestimmungen Dieses Ger t sollte unter Beachtung der regionalen und nationalen elektrischen Bestimmungen installiert werden P 9 Richtigen Schutz sicherstellen Geeigneter elekrischer mechanischer und Feuerschutz sind vom Hersteller des Endprodukts oder dem Endbenutzer bereitzustellen A 10 11 Ableitstrom gt 3 SmA LEAKAGE gt 3 5mA WARNUNG Hohe Ableitstrom Earth Verbindung bevor Sie die Verbindung von wesentlicher Bedeutung werden P 12 Add GND Verbindung Informationen Add GND connection information Bevor Sie dieses Ger t an das Stromnetz die Schutzerde Terminal Schrauben dieses Ger t muss an den Schutzleiter in der Geb udeinstallation P 13 INSTALLATION CODES INSTALLATION CODES Dieses Ger t muss installiert sein entsprechend auf die neueste Version des Landes National Electrical Code F r Nordamerika m ssen in bereinstimmung mit den geltenden Vorschriften in der US amerikanischen National Electrical Code und dem Canadian Electrical Code gt 40 Mellanox Te
24. esde abajo hacia arriba P 7 Instalaci n de equipos La instalaci n el reemplazo y el mantenimiento de este equipo estar n a cargo nicamente de personal capacitado y competente gt Mellanox Technologies 41 Rev 1 3 8 Asegurar confinamientos adecuados El fabricante del producto final o el usuario final deber n suministrar un confinamiento adecuado para componentes el ctricos y mec nicos y contra incen dio 9 Eliminaci n de equipos La eliminaci n definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales 10 C digos el ctricos locales y nacionales AN Este equipo se debe instalar conforme a los c digos el ctricos locales y nacionales 11 Fuga gt 3 5 mA ADVERTENCIA Alta corriente de fuga Es esencial efectuar la conexi n a tierra antes de conectar la alimentaci n 12 C digos de instalaci n Este dispositivo se debe instalar conforme a la versi n m s reciente de los c digos el ctricos nacionales del pa s en cuesti n En Am rica del Norte el equipo se debe instalar de acuerdo con las disposiciones vigentes del C digo El ctrico Nacional de los EE UU y del C digo El ctrico de Canad 42 Mellanox Technologies J
25. iBand SW Driverse Operating ForLinux see Mellanox OpenFabrics Enterprise Distribution OFED software package available Systems Distributions via the Mellanox OpenFabrics Web site http www mellanox com gt Downloads gt InfiniBand SW 2 2 Installation Instructions The HCA cards listed in Table 3 on page 9 are standard PCI PCI X cards with a standard PCI X edge connector Please consult the host machine documentation for instructions on how to install a PCI PCI X card 2 3 Safety Warnings 1 Installation Instructions Read all installation instructions before connecting the equipment to the power source gt 2 Over temperature This equipment should not be operated in an area with an ambient temperature exceeding the maximum recommended 55 C 131 F An air flow of 200LFM at this maximum ambient temperature is required To guarantee proper air flow allow at least 8cm 3 inches of clearance around the ventilation openings 3 During Lightning Electrical Hazard During periods of lightning activity do not work on the equipment or connect or disconnect cables 4 Copper InfiniBand Cable Connecting Disconnecting Copper InfiniBand cables are heavy and not flexible as such they should be carefully attached to or detached from the connectors Refer to the cable manu facturer for special warnings and instructions gt Mellanox Technologies 11 Rev 1 3 HCA Card Installation 5 Equipment Installation
26. les en vigueur P 8 Codes lectriques locaux et nationaux Ce mat riel doit tre install dans le respect des codes lectriques locaux et nationaux P 38 Mellanox Technologies InfiniHost PCI X Low Profile Adapter Card Rev 1 3 Anhang E Installation Sicherheitshinweise German 1 Installationsanleitungen Lesen Sie alle Installationsanleitungen bevor Sie das Ger t an die Stromversorgung anschlieBen gt 2 bertemperatur Dieses Ger t sollte nicht in einem Bereich mit einer Umgebungstemperatur ber der maximal empfohlenen Temperatur von C F betrieben werden Es ist ein Luftstrom von 200 LFM bei maximaler Umgebungstemperatur erforderlich AuBerdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen ein wandfreien Luftstrom zu gew hrleisten P 3 Bei Gewitter Elektrische Gefahr Arbeiten Sie w hrend eines Gewitters und Blitzschlag nicht am Ger t schlieBen Sie keine Kabel an oder ab P 4 Anschlie en Trennen von InfiniBand Kupferkabel InfiniBand Kupferkabel sind schwer und nicht flexible Deshalb m ssen sie vorsichtig an die Anschl sse angebracht bzw davon getrennt werden Lesen Sie die speziellen Warnungen und Anleitungen des Kabelherstellers P 5 Rack Montage und Wartung Wenn dieses Produkt in einem Rack montiert oder gewartet wird sind besondere Vor sichtsmaBnahmen zu ergreifen um die Stabilit t des Systems zu gew hrleisten Im Allgem
27. n push the other hook each of the two clip s hook towards the connector s top side see Figure 11 b Finally pull the clip away from its center see Figure 11 c Mellanox Technologies 33 Rev 1 3 Figure 11 Extracting Clip b Gently Push c Pull Clip a Gently Repeat the above actions for the second connector s clip Note The LED holes arrangement on the bracket photos appearing in this appendix may be dif ferent from their arrangement on the actual HCA cards brackets Nevertheless this difference bears no impact on the replacement instructions provided here Step 2 Unscrew bracket screws Unscrew both screws from the card using a torque screwdriver as shown in Figure 12 Figure 12 Unscrew Bracket Screws Step 3 Detach bracket Grip the bracket as shown in Figure 13 placing your thumb on the LED compo nent In a rotating move toward the component side of the card slide the bracket out of the connec tors Figure 13 c gt Gently hold your thumb on the LED component gt At the same time extract the bracket as shown in image 18 34 Mellanox Technologies InfiniHost PCI X Low Profile Adapter Card Rev 1 3 Make sure to protect the LED while extracting the bracket Figure 13 Detach the Bracket in a Rotating a Card without Clips b Grip the Card in Prep for Detachment c Detach the Bracket in Rotating Move toward Component Side C 2 Assem
28. package is available for download along with its user s manual from the single HCA card firmware update page See http www mellanox com under Firm ware A firmware binaries table lists a binary file per HCA card The file name of each such binary is composed by combining the firmware name the firmware release version and the card part num ber See Table 6 Table 6 Firmware Images for InfiniHost PCI X HCA Cards Firmware Image PCI Dev ID Name Decimal fw 23108 3 5 000 MHET2X 1SC Al bin zip is the firmware binary of fw 23108 23108 firmware fw 23108 version 3 5 000 for the RoHS compliant HCA card MHET2X 1SC Rev Al 3 4 HCA Card Firmware Update as Part of a Cluster Firmware Update If the HCA card is part of an InfiniBand cluster its firmware can be updated as part of the entire cluster firmware update using the ibfwmgr tool of the IB administration IBADM tools package IBADM is available for download as part of IB stack distributions such as IB Gold and OFED 1 Currently only the Linux distributions support updating firmware for an entire InfiniBand cluster Mellanox Technologies 13 Rev 1 3 Driver Software and Firmware available via http www mellanox com See Firmware downloads under the same Web page for cluster update instructions 3 5 Customized HCA Card Firmware Update OEM Only Note The procedure described in this section is normally not needed and applies to OEM
29. ption Configura Comments tion 14 FLASH FLASH Present Not No Shorting Block Default No Short Header 1x2 present FLASH Present ing Block Shorting Block present FLASH a Not present m m 4 E Note See Figure 5 HCA Cards Mechanical Drawing and Dimensions mm on page 26 for the Jumper location 18 Mellanox Technologies InfiniHost Low Profile Adapter Rev 1 3 5 Vital Product Data VPD Format The PCI VPD Vital Product Data layout for each of the described Mellanox Technologies HCA cards complies with the format defined in the PCI 2 3 Specification Appendix I All HCA cards share the same PCI VPD layout i e have the same items however the Offset of each item and or its Value may be different from one card s VPD to another The PCI VPD layout for each HCA card is provided in the following sections Section 5 1 PCI VPD Layout for MHET2X 1SC on page 19 Section 5 2 PCI VPD Layout for MHET2X 1TC on page 20 Section 5 3 PCI VPD Layout for MHET2X 2SC on page 22 Section 5 4 PCI VPD Layout for MHET2X 2TC on page 24 5 1 PCI VPD Layout for MHET2X 1SC Note A1 was used as the HCA card upper level revision Later revisions of the HCA card will have the same format Table 9 VPD Format for MHET2X 1SC pu Item Value Format Description 0 Large Resource Type ID
30. red soft ware and firmware for operating the board and relevant documentation Intended Audience This manual is intended for the installer and user of these PCI PCI X HCA cards listed in Over view on page 8 The manual assumes basic familiarity with the InfiniBand architecture specification Related Documentation Table 2 Documents List InfiniHost Programmer Reference Manual Reference describing the interface used by developers to write a Document no 2111PM driver for Mem free Mellanox InfiniHost devices InfiniHost MT23108 Hardware Reference Manual Reference for hardware engineers responsible for designing sys Document no 2112HM l tems and boards incorporating InfiniHost components Mellanox Firmware Tools MFT User s Manual User s Manual describing the set of MFT firmware management Document no 2204UG tools for a single InfiniBand node See www mellanox com under Firmware downloads or Management Tools InfiniBand Administration IBADM Package User s Man User s Manual describing the utilities included in the IBADM ual tools package for system administration of an InfiniBand clus Document no 2130UM ter See www mellanox com under Management Tools Online Resources Mellanox Technologies Web pages http www mellanox com Document Conventions When discussing memory sizes MB and MBytes are used in this document to mean size in mega bytes The use of Mb small b indicates size in mega
31. s only To create a customized firmware binary the firmware image in MLX format needs to be down loaded along with the MFT tools package See http www mellanox com under Firmware Down loads for customized firmware update instructions 14 Mellanox Technologies InfiniHost Low Profile Adapter Rev 1 3 4 Adapter Card Interfaces 4 1 WO Interfaces The HCA board includes the following interfaces Two 4X InfiniBand Copper Connectors e PCI X 64 bit 66 133 MHz Edge Connector Panel LEDs DC 4 1 1 InfiniBand Interface The HCA provides two 4X InfiniBand v1 1 connectors to the MT23108 for external copper cables The MT23108 device is compliant with the IBTA specification 1 2 1 The MT23108 device has two 4X ports A and B which connect to 4X copper connectors 1 and 2 respectively The MT23108 has eight internal SerDes 4 1 2 PCI X Interface The PCI X bus is a PCI X version 1 0a compliant 64 bit 66 133MHz interface The MT23108 can be either a master initiating the PCI bus operations or a slave responding to PCI bus operations The PCI PCI X bus can connect to either a host CPU in an HCA application or to an I O device such as Gigabit Ethernet when used as a Target Channel Adapter 4 1 2 1 PCI X Board Features Low profile short PCI expansion board 2 5 x 6 6 inches 3 3V and 5V up to 133 MHz 4 1 3 LED Assignment The board has only four LEDs located on the I O panel The physical
32. vision Later revisions of the HCA card will have the same format 20 Mellanox Technologies J InfiniHost Low Profile Adapter Rev 1 3 Table 10 VPD Format for MHET2X 1TC Tiu Item Value Format Description 0 Large Resource Type ID String Tag 0x02 0x82 1 Length OxA 3 Data Cougar cub Alphanumeric 13 Large Resource Type VPD R Tag 0x10 0x90 14 Length Ox4F 16 VPD Keyword PN Numbers Add in Card Part Number 18 Length 0x15 19 Data MHET2X 1TC VPD Keyword EC Alphanumeric Engineering Change 40 Level of the card rev 42 Length 0x2 43 Data Al Upper level revision 45 VPD Keyword SN Alphanumeric Serial Number 47 Length 0x18 48 Data MTYYWWF SSSSS 72 VPD Keyword NO Currently not in use 74 Length 0x10 75 Data N A 91 VPD Keyword RV 93 Length 0 1 94 Data Checksum 95 Large Resource Type VPD W Tag 0x11 0x91 96 Length 0x9D 98 VPD Keyword vi EFI Driver version 100 Length 0x6 101 Data N A Number 107 VPD Keyword YA Asset Tag 109 Length 0x20 110 Data N A Alphanumeric N A Mellanox Technologies 21 Rev 1 3 Table 10 VPD Format for MHET2X 1TC Vital Product Data VPD Format ua Item Value Format Description 142 VPD Keyword RW Remaining read write area 144 Length Ox6e 14
Download Pdf Manuals
Related Search