"user manual"
Contents
1. 0 S1 _S15 IBtxOn 0 IBtxIn 0 2 Mellanox Technologies Rev 0 30 28 B 3 PCI Express x8 Connector Pinout These cards use a standard PCI Express x8 edge connector and the PCI Express x8 standard pinout according to the Table 15 Connector Pin To Port Signal Name Connector Pin Connector Pin IB Port A Signal IB Port B Signal Number Name Name Name 51 IBtxIp 0 Rx_Al Rx_Bl S2 IBtxIn 0 Rx A0 Rx BO S3 IBtxIp 1 Rx A3 Rx B3 54 IBtxIn 1 Rx A2 Rx B2 S5 IBtxIp 2 Rx A5 Rx B5 S6 IBtxIn 2 Rx A4 Rx B4 S7 IBtxIp 3 Rx A7 Rx B7 S8 IBtxIn 3 Rx A6 Rx B6 S9 IBtxOn 3 Tx A6 Tx B6 S10 IBtxOp 3 Tx A7 Tx B7 511 IBtxOn 2 Tx_A4 Tx_B4 12 IBtxOp 2 Tx_A5 Tx B5 13 IBtxOn 1 Tx_A2 Tx_B2 S14 IBtxOp 1 Tx_A3 Tx_B3 S15 IBtxOn 0 Tx_AO Tx BO S16 IBtxOp 0 Tx Al Tx BI G1 G6 G9 HI H2 Signal Ground GND GND G7 Sense 3 3V SENSE SENSE P2 G8 MC POWER PI MC POWER P2 1 The Sense 3 3V signal is used to enable the Vcc power supply pin G8 used to provide power to the active media adapter PCI Express 2 0 specification Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 29 Appendix C Replacing Existing Bracket on Adapter Cards Adapter cards come in two bracket heights If your adapter card has a bracket that is not compatible with your plat fo2. FCC ICES C Tick Adapter Card P N Class e ae Class oe Australia and jo ng IEC EN cTUVus CB USA Canada P New Zealand MNEH28 XTC A A A A MNEH29 XSC A A A A MNEH29 XTC A A A A A 2 1 FCC A 2 2 Statements USA Class A Statements 15 21 Statement Warning Changes or modifications to this equipment not expressly approved by the party responsible for compli ance Mellanox Technologies could void the user s authority to operate the equipment 15 105 a Statement NOTE This equipment has been tested and found to comply with the limits for a Class A digital device pursuant to Part 15 of the FCC Rules These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment This equipment generates uses and can radiate radio frequency energy and if not installed and used in accordance with the instruction manual may cause harmful inter ference to radio communications Operation of this equipment in a residential area is likely to cause harmful interfer ence in which case the user will be required to correct the interference at his own expense A 2 3 EN Statements Europe EN55022 Class A Statement RF Emissions Control Warning This is a class A product In a domestic environment this product may cause radio interference in which case the user may be required to take adequate me
3. Date Rev Comments Changes May 2008 0 3 changes according to MRD Requested by Tar April 2008 0 20 Changed link to MLNX OFED page and removed Confidential in footer May 2007 0 10 Preliminary first revision August 2008 1 0 Initial Release Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 7 About this Manual This User s Manual describes Mellanox Technologies Ethernet PCI Express Network Interface Cards It provides details as to the interfaces of the card specifications required software and firmware for operating the card and rele vant documentation Intended Audience This manual is intended for the installer and user of these cards The manual assumes basic familiarity with Ethernet networks Related Documentation Table 2 Documents List Reference describing the interface used by developers to write a device EN MTNIC PRM driver Document Number DOC18348 Connec ix EN Hardware Reference Manual Reference for hardware engineers responsible for designing systems and Document Number 2788HM boards Mellanox Firmware Tools MFT User s Manual User s Manual describing the set of MFT firmware management tools Document Number 2204UG See http www mellanox com under Firmware downloads PCI Express 2 0 Specifications Industry Standard PCI Express 2 0 Card Electromechanical Specifica tion
4. 96 Length Ox1 97 Data 0 96 CSO 98 Large Resource Type VPD W Tag 0x11 0x91 99 Length 7 0 LSB Ox9A 100 Length 15 8 MSB 0x00 101 VPD Keyword VI STR EFI Driver version 103 Length 0x6 104 Data N A STR SPC 110 VPD Keyword YA STR Asset Tag 112 Length 0x20 113 Data N A STR SPC N A 145 VPD Keyword RW STR Remaining read write area 147 Length 0x6B 148 Data STR ZERO Reserved 0x00 255 Small Resource Type END Tag 0x11 0x78 Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual Table 10 VPD Layout for MNEH29 XSC TANE Item Value Format Description 0 Large Resource Type ID String Tag 0x02 0x82 1 Length 7 0 LSB OxD 2 Length 15 8 MSB 0x0 3 Data Eagle 10G ETH STR 16 Large Resource Type VPD R Tag 0x10 0x90 17 Length 7 0 LSB Ox4F 18 Length 15 8 MSB 0x00 19 VPD Keyword PN STR Add in Card Part Number 21 Length 0 15 22 PN PN STR_SPC 43 VPD Keyword EC STR Engineering Change Level of the card rev 45 Length 0x2 46 Revision XI STR PCB revision 48 VPD Keyword SN STR Serial Number 50 Length 0x18 51 Serial Number STR_SPC 00 00XXXX XX 75 VPD Keyword vo STR Misc Information 77 Length 0x10 78 Data PCIe Gen2 x8 STR_SPC 94 VPD Keyword RV STR 96 Length 0x1 97 Data 0 96 CSO 98 Large Resource Type VPD W Ta
5. Ethernet CX4 Network Interface Cards p Data Short Ordering Part Express e Tall RoHS Adapter IC Part Number OPN SERDES Compliance Number NIE LOD n Rate Bracket Speed MNEH28 XSC 2 5 GT s 20 20 Gb s Short RoHS R5 MT25408A0 FCC SE PCIe with Genl exemption MNEH28 XTC 2 5 GT s Tall PCIe MNEH29 XSC 5 0 GT s 40 40 Gb s Short RoHS R5 MT25408A0 FCC TE PCIe with Gen2 exemption MNEH29 XTC 5 0 GT s Tall PCIe Short Gen2 Tall 1 The NICs have a similar form and fit The main visible difference is in the bracket height Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 1 2 Mellanox Part Numbering Legend Table 4 describes the Mellanox Technologies adapter cards part numbering legend Table 4 Mellanox Adapter Cards Part Numbering Key Adapter Card OPN Field Decoder MHTS I XBR M Mellanox Technologies H Adapter Type H InfiniBand Host Channel Adapter N Ethernet Network Interface Card S Express Module T Media 10GBASE CX4 1OGBASE CX4 IOGBASE SR T 10GBASE UTP Twisted Pair P 10GBASE SR SFP with powered connector S Silicon H ConnectX ports 121 222 I Host Interface 8 PCIe x8 9 PCIe SerDes 5 0 GT s G Generation lt blank gt Initial product generation B 2nd Generation board Separator X
6. Device This device is registered for EMC requirements for industrial use The seller or buyer should be aware of this If this type was sold or purchased by mistake it should be replaced with a residential use type Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual A 3 MNEH28 XTC XSC Specifications Table 12 Specifications for MNEH28 XTC XSC Physical Power and Environmental Size 2 54in x 5 37in 64 4mm x 136 47mm Voltage 12V 3 3V Air Flow Typ Power TBD 4X 20Gb s Connector 200LFM 55 C single port operation TBD InfiniBand Copper current rating 0 5A dual port operation 9 63W max with active media adapter support Maximum Power TBD single port operation TBD dual port operation 10 89W Temperature to 55 C Protocol Support Regulatory Ethernet IEEE Std 802 3ae 10 Gigabit Ethernet EMC IEEE Std 802 3ak 10GBASE CX4 IEEE Std 802 3aq 10GBASE LRM TBD Multicast and Jumbo Frame Support TBD TBD MIC BCC Cert QoS 8 Virtual Lanes for each port DMA Support Yes All Ports RoHS COC Double Data Rate PCI Express 2 0 SERDES 2 5 GT s Safety Mellanox Technologies Rev 0 30 A 4 MNEH29 XTC XSC Specifications Table 13 Specifications for MNEH29 XTC XSC Physical Power and Environmental Size 2 54in x 5 37in 64 4mm x 136 47mm Voltage 12V
7. will not provide the optimum data transfer Mellanox Technologies Rev 0 30 14 3 Driver Software and Firmware Drivers and Firmware can be downloaded and installed by using the Mellanox OFED download site within the Mel lanox Website Note The installation requires administrator privileges on the target machine 3 1 Driver Software For Linux download and install the latest OpenFabrics Enterprise Distribution OFED software package available via the Mellanox OpenFabrics Web site at http www mellanox com products ofed php Follow the installation instructions included in the download package For Windows download the appropriate software from https docs mellanox com dm WinIB ReadMe html 3 2 Updating Adapter Card Firmware Each adapter card is shipped with the latest version of qualified firmware at the time of manufacturing Firmware is updated occasionally and the most recent firmware can be obtained from http www mellanox com through the Firmware downloads link 3 3 Single Adapter Card Firmware Update Firmware can be updated on the stand alone single card using the flint tool of the Mellanox Firmware Tools MFT package This package is available for download along with its user s manual from the single adapter card firmware update page See http www mellanox com under Firmware downloads A firmware binaries table lists a binary file per adapter card The file name of each such binary is compos
8. 3 3V Air Flow 200LFM 55 C Typ Power TBD 4X 20Gb s Connector InfiniBand Copper current rating 0 5A single port operation 11 73W max with active media adapter support dual port operation TBD Maximum Power TBD single port operation 13 34W dual port operation 0 C to 55 C Temperature Protocol Support Regulatory Ethernet IEEE Std 802 3ae 10 Gigabit Ethernet EMC IEEE Std 802 3ak 10GBASE CX4 TBD IEEE Std 802 3aq 10GBASE LRM Multicast and Jumbo Frame Support MIC BCC Cert MNEH29 XTC only TBD QoS 8 Virtual Lanes for each port DMA Support Yes All Ports RoHS COC Double Data Rate PCI Express 2 0 SERDES 5 0 GT s Safety Environmental RoHS Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 27 Appendix B Interface Connectors Pinout B 1 PC compatible Connector Pinout Figure 5 I2C Compatible Connector Table 14 I2C Compatible Connector Pinout Connector Pin Number Adapter Card Signal Name 1 SPSDA 2 SPSCL 5 1 2 3 4 3 GND NIA ENE oR 4 NC 5 NC B 2 Ethernet Connector Pinout Figure 6 ConnectX CX4 Copper Connector Pinout J2 Conn 1X S10 iBOp 3 IBtxlp 3 S7 S9 IBxOn 3 IBban 3 2 _S12 BOp 2 IBtxip 2 99 S11 Bone IBtxin 2 6 514 IBtxOp 1 IBtxlp 1 53 __ _S13 IBtxOn 1 1 84 _S16 IBtxOp 0
9. A 17 18 19 19 21 22 25 26 27 28 ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual List of Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Figure 12 Figure 13 Figure 14 Figure 15 Figure 16 Figure 17 Figure 18 Figure 19 Figure 20 Figure 21 Figure 22 Port Numbering Physical and Logical Link Indications I2C Connector Schematic of the Ethernet Adapter Card With CX4 Connectors I2C Compatible Connector ConnectX CX4 Copper Connector Pinout Tall Bracket of a Dual IB Port Adapter Card Connector Retention Clip Extracting Connector Clip Unscrew Bracket Screws Rotate the Bracket to Detach it From the Card Hold Kapton Label With Pincers Place Label on Print Side With Label s and Card s Holes Aligned Ensure That Label is Well attached Adapter Card Ready for New Bracket Place Bracket onto Card Attach Bracket onto Card using Screws Sliding Connector Clip Evenly Fix Clip Hooks into Place Using Screwdriver Spread the Hooks to Slide on the Clip Assembled Bracket View Print Side View After Bracket Assembly With Kapton Label Mellanox Technologies 15 16 17 22 27 27 29 29 30 30 31 31 32 32 33 33 34 34 35 35 35 36 Rev 0 30 4 Revision History This document was printed on 8 7 08 Table 1 Revision History Table
10. AM Mellanox TECHNOLOGIES ConnectX EN Dual Port Ethernet Network Interface Cards User s Manual P N MNEH28 XSC MNEH28 XTC MNEH29 XSC MNEH29 XTC Rev 0 30 Mellanox Technologies 2 Copyright 2008 Mellanox Technologies Inc All Rights Reserved Mellanox Technologies InfiniHost and ConnectX are registered trademarks for Mellanox Technologies Inc All other marks and names mentioned herein may be trademarks of their respective companies ConnectX EN Dual Port Ethernet Adapter Cards User s Manual Document Number 2844 2900 Stender Way Santa Clara CA 95054 U S A www mellanox com Tel 408 970 3400 Fax 408 970 3403 Mellanox Technologies Ltd PO Box 586 Hermon Building Yokneam 20692 Israel Tel 972 4 909 7200 Fax 972 4 959 3245 Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual Table of Contents Revision History About this Manual Chapter 1 Chapter 2 Chapter 3 Chapter 4 AppendixA AppendixB AppendixC AppendixD Overview 1 1 Adapter Cards 1 2 Mellanox Part Numbering Legend 1 3 Finding the Mac Address and Serial Number on the Adapter Card Adapter Card Installation 2 1 Hardware and Software Requirements 2 2 Installation Instructions Driver Software and Firmware 3 1 Driver Software 3 2 Updating Adapter Card Firmware 3 3 Single Adapter Card Firmware Update Adapter Card Int
11. Memory Size X MemFree B Bracket S Short T Tall N None R RoHS blank non RoHS C RoHS w Exemption R RoHS Lead Free For example the part number MNEH28 XTC describes Mellanox Technologies ConnectX EN NIC with dual 1OGBASE CXA ports PCle2 0 x8 2 5GT s interface no on board memory mem free a tall PCI bracket and RoHS R5 compliance Using the legend field M M to indicate a Mellanox Technologies product e field H N to indicate a Network Interface Card e field T E to indicate 10GBase CX4 copper with media adapter support field S 2H to indicate the ConnectX family field Z2 to indicate two ports field I 8 to indicate PCI Express 2 0 x8 running at 2 5GT s field X X to indicate no on board memory field B T to indicate a tall bracket and field R C to indicate RoHS R5 w Exemptions compliance Mellanox Technologies Rev 0 30 10 1 3 Finding the Mac Address and Serial Number on the Adapter Card All Mellanox Ethernet NICs have a label on the printed side of the adapter card that has the card serial number and the card MAC address AL Fer ecza o TR Yu Come Fe acu i vL S N MT0742X01849 MIN REV x1 P N MNKH B XTC E MAC 0002 9003174 PLE a Made In iG sn 9919 9919 te 2919 Rev 0 30 Mellanox Technologies Serial number MAC address Conn
12. Number STR_SP 00 00X XXX XX C 75 VPD Keyword vo STR Misc Information TI Length 0x10 78 Data PCIe x8 STR SPC 94 VPD Keyword RV STR 96 Length 0x1 97 Data 0 96 CSO 98 Large Resource Type VPD W Tag 0x11 0x91 99 Length 7 0 LSB Ox9A 100 Length 15 8 MSB 0x00 101 VPD Keyword VI STR EFI Driver version 103 Length 0x6 104 Data N A STR SPC 110 VPD Keyword YA STR Asset Tag 112 Length 0x20 113 Data N A STR SPC N A 145 VPD Keyword RW STR Remaining read write area 147 Length 0x6B 148 Data STR ZER Reserved 0x00 O 255 Small Resource Type END Tag 0x11 0x78 Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual Table 8 VPD Layout for MNEH28 XSC Mellanox Technologies en Item Value Format Description 0 Large Resource Type ID String Tag 0x02 0x82 1 Length 7 0 LSB OxD 2 Length 15 8 MSB 0x0 3 Data Eagle 10G ETH STR 16 Large Resource Type VPD R Tag 0x10 0x90 17 Length 7 0 LSB Ox4F 18 Length 15 8 MSB 0x00 19 VPD Keyword PN STR Add in Card Part Number 21 Length 0 15 22 PN MNEH28 XSC STR SPC 43 VPD Keyword EC STR Engineering Change Level of the card rev 45 Length 0x2 46 Revision Al STR PCB revision 48 VPD Keyword SN STR Serial Number 50 Length 0x18 51 Serial Number STR_SPC 00 00XXXX XX 75 VPD K
13. Rev 1 3 SFP Module Spec sheet Reference for the Mellanox SFP Module Document Number 2957 Online Resources e Mellanox Technologies Web pages http www mellanox com e Mellanox Technologies Firmware download Web page http www mellanox com under Firmware downloads e Mellanox Technologies Document Distribution System DDS http docs mellanox com requires a customer login account Document Conventions When discussing memory sizes MB and MBytes are used in this document to mean size in mega bytes The use of Mb or Mbits small b indicates size in mega bits Mellanox Technologies Rev 0 30 8 1 Overview This document is a User s Manual for Mellanox Technologies Ethernet Network Interface Cards NICs based on the MT25408 ConnectX EN integrated circuit device The cards described in this manual have the following main fea tures e IEFE 802 3ak compliant e Two 10GBASE CX4 copper ports for connecting Ethernet traffic e PCI Express 2 0 1 1 compatible through an x8 edge connector up to 5GT s e Media detect circuit with powered connectors supporting the use of active cables and external PHY fiber solu tions e EU Restriction of Hazardous Substances RoHS compliant The cards differ in e Bracket height short or tall e PCI Express 2 0 with SerDes speed 2 5 GT s or 5 0 GT s 1 1 Adapter Cards Table 3 on page 8 lists the Ethernet NICs described in this manual Table 3
14. asures Mellanox Technologies Rev 0 30 24 A 2 4 ICES Statements Canada Class A Statement This Class A digital apparatus complies with Canadian ICES 003 Cet appareil num rique de la classe A est conforme la norme NMB 003 du Canada A 2 5 VCCI Statements Japan Class A Statement lt 0 3 PHUBRES TER CE SEU MRA VCCI DAR CE 2 722AATSSRIXIEAEIE CORE ARERR 2 CER Hesse TL EN EURE ONSE 38 XT EIE BEDPBRKENSCECPHVET A 2 6 Translation This is a Class A product based on the standard of the Voluntary Control Council for Interfer ence by Information Technology Equipment VCCI If this equipment is used in a domestic environment radio inter ference may occur in which case the user may be required to take corrective actions A 2 7 MIC Statement Republic of Korea Korea s Regulation for Certification of Information and Communication Equipment requires EMC testing and cer tification for many electronic products Korean EMC certifications are issued by Radio Research Laboratory RRL which is organized under the Ministry of Information and Communications MIC EMC testing includes electromag netic emissions EMI and susceptibility EMS Certified equipment is labeled with the MIC mark and certification number Class A Statement DCE AAW SR ERR 9 71710 21 BHA EE Ol BE 917 pep BH xeu TE PUBS wor 717822 m s uu Translation Class A
15. by gently sweeping your thumb on the label to assure the label is well attached onto board See Figure 14 Figure 14 Ensure That Label is Well attached GUID 0002C90200258384 See also Figure 22 Print Side View After Bracket Assembly With Kapton Label on page 36 Now your card is ready for a new bracket assembly Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual C 3 Assembling a New Bracket The bracket can now be assembled onto the Adapter card See Figure 15 Figure 15 Adapter Card Ready for New Bracket Gently place the bracket onto the card fitting the connectors through the bracket connector holes Make sure the LEDs are aligned into their intended bracket holes Insert a screw along with a washer into each of the two holes on the card intended for holding the bracket Figure 16 Place Bracket onto Card LED Holes Mellanox Technologies Rev 0 30 34 5 Use a torque screwdriver to apply up to 2 Ibs in torque on each screw Figure 17 Attach Bracket onto Card using Screws 6 Gently push one clip onto the connector Make sure to slide both clip hooks sides around the connector evenly as shown in Figure 18 Figure 18 Sliding Connector Clip Evenly Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 7 Use a small flat head screwdriv
16. e sont lourds et ne sont pas flexibles il faut donc faire tr s attention en les branchant et en les d branchant des connecteurs Consultez le fabri cant des c bles pour connaitre les mises en garde et les instructions sp ciales gt 5 Installation du mat riel Ce mat riel ne doit tre install remplac ou entretenu que par du personnel form et qualifi 6 Elimination du mat riel L limination de ce mat riel doit s effectuer dans le respect de toutes les l gislations et r glementations nationales en vigueur Mellanox Technologies Rev 0 30 38 Rev 0 30 7 Codes lectriques locaux et nationaux Ce mat riel doit tre install dans le respect des codes lectriques locaux et nationaux Mellanox Technologies
17. ectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 2 Adapter Card Installation 2 1 Hardware and Software Requirements Before installing the NIC please make sure that the system meets the hardware and software requirements listed in Table 5 Table 5 Hardware and Software Requirements Systems Distributions Requirement Description Hardware Minimum 3 GB of available space PCI Express x8 or x16 slots Software For Windows see WinIB ReadMe at https docs mellanox com dm WinIB ReadMe html Operating For Linux see Mellanox OpenFabrics Enterprise Distribution OFED software package available via the Mellanox OpenFabrics Web site http ww w mel lanox com products ofed php Novell SuSE Linux Enterprise server SLES Red Hat Enterprise Linux RHEL and other Linux distributions Microsoft Windows Server2003 2008 Windows Compute Cluster Server 2003 Mellanox Technologies Rev 0 30 11 12 2 2 Installation Instructions 2 2 1 Installation Safety Warnings 1 Installation Instructions Read all installation instructions before connecting the equipment to the power source 2 Over temperature This equipment should not be operated in an area with an ambient temperature exceed ing the maximum recommended 55 C 131 F Moreover to guarantee proper air flow allow at least 8cm 3 inches of clearance around the ventilation openings 3 Durin
18. ed by com bining the firmware name the firmware release version and the card part number Note Please contact your assigned Field Application Engineer if you cannot find the firmware binary for your adapter card This may happen if the product is not yet available for general distribution Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 15 4 Adapter Card Interfaces 4 1 I O Interfaces Each adapter card includes the following interfaces e Two 4X copper connectors e PCI Express x8 edge connector e panel LEDs PC compatible connector for debug 4 1 1 Ethernet CX4 Interface The ConnectX EN 10GBase CX4 device is compliant with the IEEE Std 802 3ak I0GBASE CX4 It has two com pliant 4X ports 1 and 2 each having four Tx Rx pairs of SerDes Each of the cards listed in Table 3 on page 8 based on this device provides access to these ports by means of two 4X connectors for external copper cables Con nector 1 connects to port 1 of the device while connector 2 connects to port 2 Figure 1 Port Numbering Port 1 Port 2 Each of the NIC cards is embedded with a media detect circuit that supports active cables and external fiber solu tions to be connected to the port connectors Fiber Solutions require the use of active media converters 4 1 2 PCI Express Interface The ConnectX EN 10GBASE CX4 adapter cards support the PCI Expre
19. efault Description Option Only Comments Flash present not present connection open Flash present connection open Header 1x2 connection shorted Flash not present Flash present OSED _FLASH_NOT_PRESENT T 4 4 3 Each board incorporates an EEPROM that is accessible through the I2C compatible interface The EEPROM is used for storing the Vital Product Data VPD The VPD format adheres to the PCI Local Bus specification rev 2 3 VPD definition see VPDs on page 18 The EEPROM capacity is 512 bytes Mellanox Technologies Rev 0 30 18 4 5 VPDs e The PCI VPD Vital Product Data layout for each of the described Mellanox Technologies ConnectX EN 10GBASE CX4 NICs cards comply with the format defined in the PCI 2 3 Table 7 VPD Layout for MNEH28 XTC Value Format Description 0 Large Resource Type ID String Tag 0x02 0x82 1 Length 7 0 LSB OxD 2 Length 15 8 MSB 0x0 3 Data Eagle 10G ETH STR 16 Large Resource Type VPD R Tag 0x10 0x90 17 Length 7 0 LSB Ox4F 18 Length 15 8 MSB 0x00 19 VPD Keyword PN STR Add in Card Part Number 21 Length Ox15 22 PN MNEH28 XTC STR_SP C 43 VPD Keyword EC STR Engineering Change Level of the card rev 45 Length 0x2 46 Revision Al STR PCB revision 48 VPD Keyword SN STR Serial Number 50 Length 0x18 51 Serial
20. er to gently slide the clip s hook towards the connector s base side as shown in Figure 20 Figure 19 Fix Clip Hooks into Place Using Screwdriver Figure 20 Spread the Hooks to Slide on the Clip 8 Repeat this step for the second clip See Figure 21 for the assembled new bracket side view and Figure 22 for the Print Side view showing the Kapton label Figure 21 Assembled Bracket View e the new bracket of the proper height Mellanox Technologies Rev 0 30 36 Figure 22 Print Side View After Bracket Assembly With Kapton Label Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual Appendix D Avertissements de s curit d installation 1 Instructions d installation Lisez toutes les instructions d installation avant de brancher le mat riel la source d alimentation lectrique 2 Temp rature excessive Ce mat riel ne doit pas fonctionner dans une zone avec une temp rature ambiante d passant le maximum recommand de 55 C 131 F En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation 3 Orages dangers lectriques Pendant un orage il ne faut pas utiliser le mat riel et il ne faut pas brancher ou d brancher les cables 4 Branchement d branchement des c bles InfiniBand en cuivre Les c bles InfiniBand en cuivr
21. erfaces 4 1 I O Interfaces 42 Compatible Interface 4 3 Power 44 Memory 45 VPDs Specifications A 1 Board Mechanical Drawing and Dimensions A2 Certification Statements A 3 MNEH28 XTC XSC Specifications 4 MNEH29 XTC XSC Specifications Interface Connectors Pinout B 1 12C compatible Connector Pinout B 2 Ethernet Connector Pinout B 3 PCI Express x8 Connector Pinout Replacing Existing Bracket on Adapter Cards C 1 Removing Bracket C 2 Placing a Kapton Polyimide Label C 3 Assembling a New Bracket Avertissements de s curit d installation Mellanox Technologies O A 11 12 14 14 14 14 15 15 17 17 17 18 22 22 22 25 26 27 27 27 28 29 29 31 33 37 Rev 0 30 8 List of Tables Table 1 Table 2 Table 3 Table 4 Table 5 Table 6 Table 7 Table 8 Table 9 Table 10 Table 11 Table 12 Table 13 Table 14 Table 15 Rev 0 30 Revision History Table Documents List Ethernet CX4 Network Interface Cards Mellanox Adapter Cards Part Numbering Key Hardware and Software Requirements Jumper Configuration VPD Layout for MNEH28 XTC VPD Layout for MNEH28 XSC VPD Layout for MNEH29 XTC VPD Layout for MNEH29 XSC Adapter Cards EMC certification Status Specifications for MNEH28 XTC XSC Specifications for MNEH29 XTC XSC D2C Compatible Connector Pinout Connector Pin To Port Signal Name Mellanox Technologies
22. eyword vo STR Misc Information TI Length 0x10 78 Data PCIe x8 STR SPC 94 VPD Keyword RV STR 96 Length Ox1 97 Data 0 96 CSO 98 Large Resource Type VPD W Tag 0x11 0x91 99 Length 7 0 LSB Ox9E 100 Length 15 8 MSB 0x00 101 VPD Keyword VI STR EFI Driver version 103 Length 0x6 104 Data N A STR SPC 110 VPD Keyword YA STR Asset Tag 112 Length 0x20 113 Data N A STR SPC N A 145 VPD Keyword RW STR Remaining read write area 147 Length 0x6F 148 Data STR ZERO Reserved 0x00 259 Small Resource Type END Tag 0x11 0x78 Table 9 VPD Layout for MNEH29 XTC Pide Item Value Format Description 0 Large Resource Type ID String Tag 0x82 0x02 Rev 0 30 20 Table 9 VPD Layout for MNEH29 XTC Continued Value Format Description 1 Length 7 0 LSB OxD 2 Length 15 8 MSB 0x0 3 Data Eagle 10G STR ETH 16 Large Resource Type VPD R Tag 0x10 0x90 17 Length 7 0 LSB Ox4F 18 Length 15 8 MSB 0x00 19 VPD Keyword PN STR Add in Card Part Number 21 Length 0 15 22 PN PN STR_SPC 43 VPD Keyword EC STR Engineering Change Level of the card rev 45 Length 0x2 46 Revision XI STR PCB revision 48 VPD Keyword SN STR Serial Number 50 Length 0x18 51 Serial Number STR_SPC 00 00XXXX XX 75 VPD Keyword vo STR Misc Information TI Length 0x10 78 Data PCIe Gen2 x8 STR SPC 94 VPD Keyword RV STR
23. g 0x11 0x91 99 Length 7 0 LSB Ox9A 100 Length 15 8 MSB 0x00 101 VPD Keyword VI STR EFI Driver version 103 Length 0x6 104 Data N A STR SPC 110 VPD Keyword YA STR Asset Tag 112 Length 0x20 113 Data N A STR SPC N A 145 VPD Keyword RW STR Remaining read write area 147 Length 0x6B 148 Data STR ZERO Reserved 0x00 255 Small Resource Type END Tag 0x11 0x78 Mellanox Technologies Rev 0 30 22 Appendix A Specifications A 1 Board Mechanical Drawing and Dimensions All the NICs covered in this User s Manual have the same mechanical drawing and share the same dimensions as depicted in Figure 4 Note All dimensions are in millimeters Figure 4 Schematic of the Ethernet Adapter Card With CX4 Connectors 136 47 I 33 35 l gt el 1 90 43 17 57 15 Le 96 3 A 2 EMC Certification Statements J5 I2C Connector J6 Flash Jumper Table 11 lists the approved EMC certification status per adapter card in different regions of the world Table 11 Adapter Cards EMC certification Status FCC ICES C Tick Adapter Card P N Class ae Bin Class Tous Australia and IEC EN cTUVus CB 08 Canada P New Zealand MNEH28 XSC A A A A Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual Table 11 Adapter Cards EMC certification Status
24. g Lightning Electrical Hazard During periods of lightning activity do not work on the equipment or connect or discon nect cables 4 Copper InfiniBand Cable Connecting Disconnecting Copper InfiniBand cables are heavy and not flexible as such they should be carefully attached to or detached from the connectors Refer to the cable manufacturer for special warnings and instructions 5 Equipment Installation This equipment should be installed replaced or serviced only by trained and qualified personnel 6 Equipment Disposal Disposal of this equipment should be in accordance to all national laws and regulations 7 Local and National Electrical Codes This equipment should be installed in compliance with local and national electrical codes Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 13 2 2 2 Installation Instructions The adapter cards listed in Table 3 on page 8 are standard PCI Express x8 cards each with a standard x8 edge connec tor Please consult the host machine documentation for instructions on how to install a PCI Express card Note When more than one PCI Slot is available first make sure to use the PCI slot with the proper configu ration Any PCI slot with the proper configuration is acceptable for connection If the card is installed in a PCI slot with less lanes than the card requires then the adapter card
25. ivity Yellow Blinking indicates Data Transfer Constant on indicates no Data Transfer Note The short bracket has the same port and LED footprint as the tall bracket Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual 17 42 PC Compatible Interface A three pin header on the adapter card is provided as the re compatible interface See Figure 4 on page 22 for the location on the board Figure 3 IC Connector 4 3 Power All adapter cards receive power from the PCI Express Edge connector All other required power voltages are gener ated by on board switch mode regulators For power consumption see Specifications starting on page 22 4 4 Memory The NICs support multiple memory devices through the PCI Express Flash and I2C compatible interfaces 4 4 1 System Memory Each of the NICs utilize the PCI Express interface to store and access connection information and packet data on the system memory 4 4 2 Flash Each of the NICs includes 2MB SPI Flash device P N M25P16 VME6G by ST Microelectronics accessible via the Flash interface of the MT25408 ConnectX EN device There is a jumper on each adapter card that indicates to the device whether an on board Flash device exists is to be used Table 6 provides information on this jumper See Figure 4 on page 22 for the jumper location Table 6 Jumper Configuration acm a Card D
26. nent side of the card slide the bracket out of the connectors Figure 11 c Gently hold your thumb on the LED component Pp n Ap A At the same time extract the bracket as shown in Figure 11 c Make sure to protect the LED while extracting the bracket Rev 0 30 Mellanox Technologies ConnectX EN Dual Port Ethernet Network Interface Cards With PCI Express x8 User s Manual Figure 11 Rotate the Bracket to Detach it From the Card a Card without Clips and Screws b Grip the Card in preparation for Detachment c Rotate the bracket toward the Component Side C 2 Placing a Kapton Polyimide Label Prior to assembling the bracket you need to apply a Kapton polyimide round label on the board s Print Side Note Check to see if the label is already installed as this label may have been installed at the factory Note The label can be provided by Mellanox Technologies P N MEC000821 The following steps are instructions for placing the polyimide label 1 Make sure your working area is ESD protected 2 Hold the label with light pincers See Figure 12 Figure 12 Hold Kapton Label With Pincers Mellanox Technologies Rev 0 30 32 3 Gently place the label as shown in Figure 13 Make sure to align the center hole of the label with the drilled hole in the board Figure 13 Place Label on Print Side With Label s and Card s Holes Aligned 4 After placing the label complete the process
27. rm this appendix provides instructions on how to remove a bracket from a standard Mellanox Technologies adapter card and replace it with a different one It includes the following sections e Removing Bracket e Placing a Kapton Polyimide Label Assembling New Bracket Figure 7 shows the bracket side view of a dual port Adapter card Figure 7 Tall Bracket of a Dual IB Port Adapter Card C 1 Removing Bracket Figure 8 shows a connector retention clip and the designated names of its sections Figure 8 Connector Retention Clip Clip Top Clip Center embraces the connector Clip Hook grips onto the connector 1 Using a small flat head screwdriver gently push up one hook of a connector s clip toward the connector s top side as shown in Figure 9 a on page 30 2 Then push the other hook each of the two clip s hook towards the connector s top side see Figure 9 b Finally pull the clip away from its center see Figure 9 c Mellanox Technologies Rev 0 30 30 Figure 9 Extracting Connector Clip a Gently Push One Hook of Clip b Gently Push Other Hook of Clip c Pull Clip Away 3 Repeat the above actions for the second connector s clip 4 Unscrew both screws from the card using a torque screwdriver as shown in Figure 10 Figure 10 Unscrew Bracket Screws Grip the bracket as shown in Figure 11 placing your thumb on the LED component In a rotating move toward the compo
28. ss 2 0 x8 interface 1 1 compatible The NICs can be either a master initiating the PCI Express bus operations or a slave responding to PCI bus operations 4 1 3 LED Assignment The board has four LEDs located on the I O panel 2 LEDs per port The green LED when lit indicates that the driver is running and a valid physical connection between nodes exists The green LED when blinking indicates that the physical connection between nodes is problematic The yellow LED when lit indicates a valid data activity link this is the logical link The yellow LED illuminates when the network is discovered over the physical link A valid Mellanox Technologies Rev 0 30 16 data activity link without data transfer is designated by a constant yellow LED indication A valid data activity link with data transfer is designated by a blinking yellow LED indication If the LEDs are not active either the physical link or the logical link or both connections have not been established Figure 2 Physical and Logical Link Indications Port Number LED Name Port 1 Physical Link Green P rt 1 Constant on indicates a good physical link Blinking indicates problem with the Physical link Data Activity Yellow Blinking indicates Data Transfer Constant on indicates no Data Transfer Port 2 Physical Link Green Constant on indicates a good physical link Blinking indicates a problem with the Physical link Data Act
Download Pdf Manuals
Related Search