PDF - Lenovo Press
Contents
1. ping latency e Support for QSFP to SFP for 10 GbE support e Traffic steering across multiple cores e Legacy and UEFI PXE network boot support Ethernet mode only The Mellanox ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter has the same features as the ConnectX 3 40GbE FDR IB VPI Adapter with these additions e Mezzanine LOM Generation 2 ML2 form factor e Offers NVGRE hardware offloads e Offers VXLAN hardware offloads Performance Based on Mellanox s ConnectX 3 technology these adapters provide a high level of throughput performance for all network environments by removing I O bottlenecks in mainstream servers that are limiting application performance With the FDR VPI IB E Adapter servers can achieve up to 56 Gb transmit and receive bandwidth Hardware based InfiniBand transport and IP over InfiniBand IPoIB stateless off load engines handle the segmentation reassembly and checksum calculations that otherwise burden the host processor RDMA over InfiniBand and RDMA over Ethernet further accelerate application run time while reducing CPU utilization RDMA allows very high volume transaction intensive applications typical of HPC and financial market firms as well as other industries where speed of data delivery is paramount With the ConnectX 3 based adapter highly compute intensive tasks running on hundreds or thousands of multiprocessor nodes such as climate research molecular modeling and physical simulations can share data and2. synchronize faster resulting in shorter run times In data mining or web crawl applications RDMA provides the needed boost in performance to enable faster search by solving the network latency bottleneck that is associated with I O cards and the corresponding transport technology in the cloud Various other applications that benefit from RDMA with ConnectX 3 include Web 2 0 Content Delivery Network business intelligence database transactions and various cloud computing applications Mellanox ConnectX 3 s low power consumption provides clients with high bandwidth and low latency at the lowest cost of ownership TCP UDP IP acceleration Applications utilizing TCP UDP IP transport can achieve industry leading data throughput The hardware based stateless offload engines in ConnectX 3 reduce the CPU impact of IP packet transport allowing more processor cycles to work on the application Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 6 NVGRE and VXLAN hardware offloads The Mellanox ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter offers NVGRE and VXLAN hardware offload engines which provide additional performance benefits especially for public or private cloud implementations and virtualized environments These offloads ensure that Overlay Networks are enabled to handle the advanced mobility scalability serviceability that is required in today s and tomorrow s data center These offloads dramatically lower CPU consumption thereby r
3. D TO THE IMPLIED WARRANTIES OF NON INFRINGEMENT MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE Some jurisdictions do not allow disclaimer of express or implied warranties in certain transactions therefore this statement may not apply to you This information could include technical inaccuracies or typographical errors Changes are periodically made to the information herein these changes will be incorporated in new editions of the publication Lenovo may make improvements and or changes in the product s and or the program s described in this publication at any time without notice The products described in this document are not intended for use in implantation or other life support applications where malfunction may result in injury or death to persons The information contained in this document does not affect or change Lenovo product specifications or warranties Nothing in this document shall operate as an express or implied license or indemnity under the intellectual property rights of Lenovo or third parties All information contained in this document was obtained in specific environments and is presented as an illustration The result obtained in other operating environments may vary Lenovo may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you Any references in this publication to non Lenovo Web sites are provided for convenience only and do not in any man
4. GbE Adapter Microsoft Windows Server 2008 R2 Microsoft Windows Server 2012 R2 Red Hat Enterprise Linux 5 Server x64 Edition Red Hat Enterprise Linux 6 Server x64 Edition Red Hat Enterprise Linux 7 SUSE Enterprise Linux Server SLES 12 SUSE LINUX Enterprise Server 10 for AMD64 EM64T SUSE LINUX Enterprise Server 11 for AMD64 EM64T VMware vSphere 5 0 ESXi VMware vSphere 5 1 ESXi VMware vSphere 5 5 ESXi VMware vSphere 6 0 ESXi For the Mellanox ConnectX 3 40GbE FDR IB VPI Adapter Microsoft Windows Server 2008 R2 Microsoft Windows Server 2012 R2 Red Hat Enterprise Linux 5 Server x64 Edition Red Hat Enterprise Linux 6 Server x64 Edition Red Hat Enterprise Linux 7 SUSE Enterprise Linux Server SLES 12 SUSE LINUX Enterprise Server 10 for AMD64 EM64T SUSE LINUX Enterprise Server 11 for AMD64 EM64T VMware vSphere 5 0 ESXi VMware vSphere 5 1 ESXi VMware vSphere 5 5 ESXi For the Mellanox ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter Note Microsoft Windows Server 2012 R2 Red Hat Enterprise Linux 6 Server x64 Edition Red Hat Enterprise Linux 7 SUSE Enterprise Linux Server SLES 12 SUSE LINUX Enterprise Server 11 for AMD64 EM64T VMware vSphere 5 1 ESXi VMware vSphere 5 5 ESXi VMware vSphere 6 0 ESXi VXLAN is initially supported only with Red Hat Enterprise Linux 7 NVGRE is initially supported only with Windows Server 2012 R2 Note on VMware support With VMware these adapters are supported o
5. Lenovo LENOVO PRESS Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x Product Guide High performance computing HPC solutions require high bandwidth low latency components with CPU offloads to get the highest server efficiency and application productivity The Mellanox ConnectX 3 and ConnectX 3 Pro network adapters for System x servers deliver the I O performance that meets these requirements Mellanox s ConnectX 3 and ConnectX 3 Pro ASIC delivers low latency high bandwidth and computing efficiency for performance driven server applications Efficient computing is achieved by offloading from the CPU routine activities which makes more processor power available for the application Network protocol processing and data movement impacts such as InfiniBand RDMA and Send Receive semantics are completed in the adapter without CPU intervention RDMA support extends to virtual servers when SR IOV is enabled Mellanox s ConnectX 3 advanced acceleration technology enables higher cluster efficiency and scalability of up to tens of thousands of nodes Figure 1 Mellanox Connect X 3 10GbE Adapter for System x 3U bracket shown Did you know Mellanox Ethernet and InfiniBand network server adapters provide a high performing interconnect solution for enterprise data centers Web 2 0 cloud computing and HPC environments where low latency and interconnect efficiency is paramount In additiona Virtual Protocol Interconn
6. ect VPI offers flexibility in InfiniBand and Ethernet port designations With the new ConnectX 3 Pro adapter you can implement VXLAN and NVGRE offload engines to accelerate virtual LAN ID processing ideal for public and private cloud configurations Check for Updates Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 1 Part number information Table 1 shows the part numbers and feature codes for the adapters Table 1 Ordering part numbers and feature codes 00D9690 A3PM Mellanox ConnectX 3 10 GbE Adapter 00D9550 A3PN Mellanox ConnectX 3 40GbE FDR IB VPI Adapter OOFP650 A5RK Mellanox ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter The part numbers include the following e One adapter with a bracket attached e Additional bracket included in the box e Quick installation guide e Documentation CD e Warranty Flyer e Important Notices Flyer Figure 2 shows the Mellanox ConnectX 3 40GbE FDR IB VPI Adapter the shipping adapter includes a heatsink over the ASIC but the figure does not show this heatsink i Figure 2 Mellanox ConnectX 3 40GbE FDR IB VPI Adapter for System x with 3U bracket required ASIC heatsink not shown Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 2 Supported cables and transceivers The 10GbE Adapter 00D9690 supports the direct attach copper DAC twin ax cables transceivers and optical cables that are listed in the following table Table 2 Supported transceiver
7. educing cloud application cost facilitating the highest available throughput and lowering power consumption Software support All Mellanox adapter cards are supported by a full suite of drivers for Microsoft Windows Linux distributions and VMware ConnectX 3 adapters support OpenFabrics based RDMA protocols and software Stateless offload is fully interoperable with standard TCP UDP IP stacks ConnectX 3 adapters are compatible with configuration and management tools from OEMs and operating system vendors Specifications InfiniBand specifications ConnectX 3 FDR VPI IB E Adapter and ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter e Supports InfiniBand FDR 14 FDR 10 QDR DDR and SDR e IBTA Specification 1 2 1 compliant e RDMA Send Receive semantics e Hardware based congestion control e 16 million I O channels e 256 to 4 KB MTU 1 GB messages e Nine virtual lanes Eight data and one management e NV GRE hardware offloads ConnectX 3 Pro only e VXLAN hardware offloads ConnectX 3 Pro only Enhanced InfiniBand specifications e Hardware based reliable transport e Hardware based reliable multicast e Extended Reliable Connected transport e Enhanced Atomic operations e Fine grained end to end quality of server QoS Ethernet specifications e EEE 802 3ae 10 GbE e EEE 802 3ba 40 GbE ConnectX 3 FDR VPI IB E Adapter and ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter e EEE 802 3ad Link Aggregation e EEE 802 3az Energy Efficien
8. ing US registered or common law trademarks owned by Lenovo at the time this information was published Such trademarks may also be registered or common law trademarks in other countries A current list of Lenovo trademarks is available on the Web at http www lenovo com legal copytrade html The folowing terms are trademarks of Lenovo in the United States other countries or both Flex System Lenovo Lenovo logo Lenovo XClarity System x X5TM The following terms are trademarks of other companies Intel Intel Xeon Intel logo Intel Inside logo and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries Linux is a trademark of Linus Torvalds in the United States other countries or both Microsoft Windows and the Windows logo are trademarks of Microsoft Corporation in the United States other countries or both Other company product or service names may be trademarks or service marks of others Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 15
9. m 6 60 in e Width 69 mm 2 71 in e Depth 17 mm 0 69 in e Weight 208 g 0 46 Ib Approximate shipping dimensions e Height 189 mm 7 51 in e Width 90 mm 3 54 in e Depth 38 mm 1 50 in e Weight 450 g 0 99 Ib Supported servers The adapters are supported in the System x servers that are listed in the following tables Memory requirements Ensure that your server has sufficient memory available to the adapters The first adapter requires 8 GB of RAM in addition to the memory allocated to operating system applications and virtual machines Any additional adapters installed each require 4 GB of RAM Support for servers with Intel Xeon v3 processors Table 4 Support for servers with Intel Xeon v3 processors Part number Description 00D9690 Mellanox ConnectX 3 10 GbE Adapter 00D9550 Mellanox ConnectX 3 40GbE FDR IB VPI Adapter OOFP650 Mellanox ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter T gt N W q N e X o 1 o v x Ke X o vs co v x x3100 M5 5457 nx360 M5 5465 x3250 M5 5458 x3500 M5 5464 x3550 M5 5463 x3650 M5 5462 Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 9 Support for servers with Intel Xeon v2 processors Table 5 Support for servers with Intel Xeon v2 processors cats vN O9ExU Iz ZA 0092 63 viN O9ExP DE A 43 Lvc9 9X OS6EX 9X 0S8 X I gt Lege 9X OS6EX 9X OS8EX ESE 928 vIN 0928x Sene Z928 vIN 09Z X gt g
10. ner serve as an endorsement of those Web sites The materials at those Web sites are not part of the materials for this Lenovo product and use of those Web sites is at your own risk Any performance data contained herein was determined in a controlled environment Therefore the result obtained in other operating environments may vary significantly Some measurements may have been made on development level systems and there is no guarantee that these measurements will be the same on generally available systems Furthermore some measurements may have been estimated through extrapolation Actual results may vary Users of this document should verify the applicable data for their specific environment Copyright Lenovo 2015 All rights reserved This document was created or updated on Sep 30 2015 Send us your comments in one of the following ways e Use the online Contact us review form found at http lenovopress com tips0897 mellanox connectx 3 e Send your comments in an e mail to Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 14 comments lenovopress com This document is available online at http lenovopress com TIPS0897 Trademarks Lenovo the Lenovo logo and For Those Who Do are trademarks or registered trademarks of Lenovo in the United States other countries or both These and other Lenovo trademarked terms are marked on their first occurrence in this information with the appropriate symbol or TM indicat
11. nly in Ethernet mode InfiniBand is not supported Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 11 Regulatory approvals e EN55022 e EN55024 e EN60950 1 e EN 61000 3 2 e EN 61000 3 3 e EC 60950 1 e FCC Part 15 Class A e UL 60950 1 e CSA C22 2 60950 1 07 e VCCI e NZ AS3548 C tick e RRL for MIC KCC e BSMI EMC e IECS 003 2004 Issue 4 Operating environment The adapters are supported in the following environment Operating temperature e 0 55 C 32 to 131 F at 0 914 m 0 3 000 ft e 10 32 C 50 90 F at 914 to 2133 m 3 000 7 000 ft Relative humidity 2096 8096 noncondensing Maximum altitude 2 133 m 7 000 ft Air flow 200 LFM at 55 C Power consumption e Power consumption typical 8 8 W typical both ports active e Power consumption maximum 9 4 W maximum for passive cables only e 13 4 W maximum for active optical modules Warranty One year limited warranty When installed in an System x server these cards assume your system s base warranty and any warranty upgrades Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 12 Related publications For more information refer to these documents e U S Announcement Letter http iom com common ssi cgi bin ssialias infotype dd amp subtype ca amp amp htmlfid 897 ENUS1 13 008 e System x networking adapters http shop lenovo com us en systems servers options systemx networking adapters e Ser
12. s and DAC cables 10GbE adapter Direct Attach Copper DAC cables Lswo TSF Transceiver 10 Gbps SR B 000T8 1 m LC LC Fiber Cable networking Optical 5 m LC LC Fiber Cable networking Optical Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 3 The FDR VPI IB E Adapter 00D9550 and and ML2 Adapter OOFP650 support the direct attach copper DAC twin ax cables transceivers and optical cables that are listed in the following table Table 3 Supported transceivers and DAC cables 40GbE adapters Direct attach copper DAC cables InfiniBand 40Gb Ethernet QSFP to 10Gb Ethernet SFP Conversion 40Gb Ethernet QSFP 40GbE copper uses the QSFP to QSFP cables directly AEXB 1DQ Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 4 The following figure shows the Mellanox ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter Figure 3 Mellanox ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter Features The Mellanox Connect X 3 10GbE Adapter has the following features Two 10 Gigabit Ethernet ports Low profile form factor adapter with 2U bracket 3U bracket available for CTO orders PCI Express 3 0 x8 host interface PCle 2 0 and 1 1 compatible SR IOV support 16 virtual functions supported by KVM and Hyper V OS dependant up to a maximum of 127 virtual functions supported by the adapter Enables Low Latency RDMA over Ethernet supported with both non virtualized and SR IOV enabled virtualized
13. servers latency as low as 1 s TCP UDP IP stateless offload in hardware Traffic steering across multiple cores Intelligent interrupt coalescence Industry leading throughput and latency performance Software compatible with standard TCP UDP IP stacks Microsoft VMQ VMware NetQueue support Legacy and UEFI PXE network boot support Supports iSCSI as a software iSCSI initiator in NIC mode with NIC driver The Mellanox ConnectX 3 40GbE FDR IB VPI Adapter has the following features Two QSFP ports supporting FDR 14 InfiniBand or 40 Gb Ethernet Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x e Low profile form factor adapter with 2U bracket 3U bracket available for CTO orders e PCI Express 3 0 x8 host interface PCle 2 0 and 1 1 compatible e Support for InfiniBand FDR speeds of up to 56 Gbps auto negotiation FDR 10 DDR and SDR e Support for Virtual Protocol Interconnect VPI which enables one adapter for both InfiniBand and 10 40 Gb Ethernet Supports three configurations e 2 ports InfiniBand e 2 ports Ethernet e 1 port InfiniBand and 1 port Ethernet e SR IOV support 16 virtual functions supported by KVM and Hyper V OS dependant up to a maximum of 127 virtual functions supported by the adapter e Enables Low Latency RDMA over 40Gb Ethernet supported with both non virtualized and SR IOV enabled virtualed servers latency as low as 1 s e Microsoft VMQ VMware NetQueue support e Sub 1 us InfiniBand MPI
14. t gt 09 GH YIN OS9Ex gt j fz 99s ag viN osgex Mz ZA 0092 S3 S164 YIN 0S9 X Mx z 00vz 83 8SLZ VIN O 9 X Iz Z 0092 93 vL62 vIN OSSEX DE ZA 00pZ S3 09L2 vIN OESEX gt gt lz 0092 83 8 4 VIN 00SEX gt gt lz Q gt m tc a LL Lu Re 9l e t p x 3 o oO c c Q O x o c S oO Q LO LO o Q e e 2x40GbE FDR VPI Adapter Description 00D9690 Mellanox ConnectX 3 10 GbE Adapter OOFP650 Mellanox ConnectX 3 Pro ML2 Part number Support for servers with Intel Xeon v1 processors Table 6 Support for servers with Intel Xeon v1 processors 0092 83 zL64 VIN O9ExP Da 11 sx 0S8 X gt gt 2 2228 VIN OSLEx gt gt 2 12 SX 069 x DM 0097 93 SL64 vIN osoex gt gt 2 00v2 S3 8S1Z VIN OE9EX xz 0097 93 vL64 VIN OSSEX O B 00pZ S3 0912 bn OEGex Da 0092 Sa egeZ PIN o0sex Ez Z882 viN 00 ex DE E8SZ pW oszex EE cese viN OOLEXx IE VPI Adapter Description 00D9690 Mellanox ConnectX 3 10 GbE Adapter 00D9550 Mellanox ConnectX 3 40GbE FDR IB VPI OOFP650 Mellanox ConnectX 3 Pro ML2 2x40GbE FDR Part number 10 Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x Supported operating systems The adapters support the following operating systems Check Fix Central for currently supported OS kernels http ibm com support fixcentral For the Mellanox ConnectX 3 10
15. t Ethernet e EEE 802 1Q 1P VLAN tags and priority e EEE 802 1Qbg e IEEE P802 1Qaz DO 2 Enhanced Transmission Selection ETS e EEE P802 1Qbb D1 0 Priority based Flow Control e EEE 1588v2 Precision Clock Synchronization Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x e Multicast e Jumbo frame support 9600B e 128 MAC VLAN addresses per port Hardware based l O virtualization e Address translation and protection e Multiple queues per virtual machine e VMware NetQueue support e 16 virtual function SR IOV supported with Linux KVM e VXLAN and NVGRE ConnectX 3 Pro ML2 2x40GbE FDR VPI Adapter SR IOV features e Address translation and protection e Dedicated adapter resources e Multiple queues per virtual machine e Enhanced QoS for vNICs e VMware NetQueue support Additional CPU offloads e TCP UDP IP stateless offload e Intelligent interrupt coalescence e Compliant with Microsoft RSS and NetDMA Management and tools InfiniBand e Interoperable with OpenSM and other third party subnet managers e Firmware and debug tools MFT and IBDIAG Ethernet e MIB MIB II MIB II Extensions RMON and RMON 2 e Configuration and diagnostic tools Protocol support e Open MPI OSU MVAPICH Intel MPI MS MPI and Platform MPI e TCP UDP EoIB and IPoIB e uDAPL Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x Physical specifications The adapters have the following dimensions e Height 168 m
16. verProven compatibility http www lenovo com us en serverproven xseries lan matrix shtml e Mellanox User Manual Mellanox ConnectX 3 40GbE FDR IB VPI Adapter http bit ly YJ3ROx e Mellanox User Manual Mellanox Connect X 3 10GbE Adapter http bit ly YJ40Qa Mellanox ConnectX 3 and ConnectX 3 Pro Adapters for System x 13 Notices Lenovo may not offer the products services or features discussed in this document in all countries Consult your local Lenovo representative for information on the products and services currently available in your area Any reference to a Lenovo product program or service is not intended to state or imply that only that Lenovo product program or service may be used Any functionally equivalent product program or service that does not infringe any Lenovo intellectual property right may be used instead However it is the user s responsibility to evaluate and verify the operation of any other product program or service Lenovo may have patents or pending patent applications covering subject matter described in this document The furnishing of this document does not give you any license to these patents You can send license inquiries in writing to Lenovo United States Inc 7009 Think Place Building One Morrisville NC 27560 USA Attention Lenovo Director of Licensing LENOVO PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITE
Download Pdf Manuals
Related Search