User Manual v1.9.9-0
Contents
1. CHNOL OGIES Connect Accelerate Outperform Mellanox MLNX EN ESX Driver for VMware ESXi 5 1 and ESXi 5 5 User Manual Rev 1 9 9 0 www mellanox com Rev 1 9 9 0 THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO S AND OR THE SYSTEM USING IT THEREFORE MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE DISCLAIMED IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT INDIRECT SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES OF ANY KIND INCLUDING BUT NOT LIMITED TO PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY FROM THE USE OF THE PRODUCT S AND2. 6 Chapter 4 Additional Driver Settings 7 4 1 Changing Driver Default 7 4 1 1 Changing the Driver s Module Parameters Default Values Temporary 7 4 1 2 Changing the Driver s Module Parameters Default Values Permanently 7 4 2 Disabling Enabling Automatic Load of the Driver upon 7 4 3 Adding the Device as an uplink to an Existing Vswitch using the CLI 8 43 1 Locally cc RR EIS uere er e ur OV IU EO e E x 8 4 3 2 Remotely i teo ev OVE UE XR PENES PE 8 4 3 3 Renaming the uplink 6 9 Mellanox Technologies 3 Rev 1 9 9 0 List of Tables Table1 mlx4 core Module 4 Table2 mlx4 en Module Parameters llle e 4 1 Mellanox Technologies Rev 1 9 9 0 1 Overview This document provides instructions for installing the MLNX EN ESX drivers for Mellanox Technologies ConnectX family based network adapter cards in a VMware ESXi 5 1 and ESXi 5 5 server environment The ConnectX family adapters identify on the PCI bus as 25408 25418 25448 26418 26428 26438 26448 26468 26478 26488 25400 27500 27510 27511 27520 27521 27530 27531 27540 27541 27550 27551 27560 27561 1 1 Main Features Overview Single Dual port
3. It s 8 bits bit mask each bit indicates priority 0 7 default 0 Bit value 1 generate pause frames according to the RX buffer threshold on the specified priority 0 never generate pause frames on the specified priority udp rss Performs RSS for incoming UDP traffic 1 enable 0 disable default 0 use rx frags Uses RX frags 1 enable 0 disable default 0 5 Mellanox Technologies Rev 1 9 9 0 3 Firmware Programming 1 Download the bootable binary image md5sum e7b3e9357ca4045fabe2e8a95d951343 from the Mellanox Firmware Tools MFT site 2 Install the image according to the steps described in the README file The following procedure requires custom boot image downloading mounting boot ing from a USB device Mellanox Technologies 6 Rev 1 9 9 0 Additional Driver Settings 4 Additional Driver Settings 41 Changing Driver Default Values The driver s default values can be changed in one of the following methods 4 1 1 Changing the Driver s Module Parameters Default Values Temporary gt To temporary change the module parameters default values 1 Unload the driver gt esxcfg module u mlx4 core gt esxcfg module u mlx4 en 2 Load the driver with the required parameters gt esxcfg module mlx4 core param name gt lt value gt gt esxcfg module mlx4 en param name value For example gt esxcfg module mlx4 en
4. MSI X Int X e NetQueue support Multiple Tx Rx rings Hardware Tx Rx checksum offload Large Send Offload TCP Segmentation Offload VLAN Tx Rx acceleration HW VLAN stripping insertion Ethtool support NAPI support e Wake on Lan WoL only on supported hardware e NC SI Auto moderation Net dump Only for ESX15 5Ux RSS Queues Fixed Pass Through 10G 406 Only for ESXi5 5Ux Mellanox Technologies 2 Rev 1 9 9 0 Driver Software 2 2 1 2 2 2 3 Driver Software VMware uses a file package called a VIB VMware Installation Bundle as the mechanism for installing or upgrading software packages on an ESXi server MLNX EN ESX driver consists of 2 dependant kernel modules mlx4 core ConnectX core driver and mlx4 en ConnectX Ethernet driver each with its own vib file In order to install the driver both of the VIBs need to be installed together For this MLNX EN ESX driver provides a bundle file a zip file that contain each module VIB file and metadata file that describes the dependencies between them The following steps describe how to download install and run the driver Installing and Running the offline bundle Driver on ESXi 5 1 and ESXi 5 5 1 Copy the offline bundle zip file to the ESXi 5 1 or ESXi 5 5 machine 2 Extract its contents 3 Install the driver in one of the following ways a Remove any earlier version of the driver from your VMware ESXi s
5. RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Mellanox TECHNOLOGIES Connect Accelerate Outperform Mellanox Technologies 350 Oakmead Parkway Suite 100 Sunnyvale CA 94085 U S A www mellanox com Tel 408 970 3400 Fax 408 970 3403 Copyright 2012 Mellanox Technologies All rights reserved Mellanox Mellanox Logo BridgeX ConnectX CORE Direct InfiniBridge InfiniHost InfiniScale PhyX SwitchX Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies Ltd FabricIT MLNX OS Unbreakable Link UFM and Unified Fabric Manager are trademarks of Mellanox Technologies Ltd All other trademarks are property of their respective owners Mellanox MLNX EN ESX Driver for VMware ESXi 5 1 and ESXi 5 5 User Manual 2 Mellanox Technologies Document Number 3527 Rev 1 9 9 0 Table of Contents Chapter T Overvieweescbesoecasia s cuo X TE enc than ooh bs 2 1 1 Main Features Overview llle re 2 Chapter 2 Driver 3 2 1 Installing and Running the offline bundle Driver on ESXi 5 1 and ESXi 5 5 3 2 2 Removing the offline bundle 3 2 3 Driver Default Values eee epe Rieke epe a ee enge 3 Chapter3 Firmware
6. byte CQEs EQEs when it is supported by the firmware 1 enabled 0 disabled default 0 enable qos Enables Quality of Service support in the HCA 1 enabled 0 disabled default 0 log mtts per seg Log2 number of MTT entries per segment 1 7 default 3 log num mgm entry size Log2 MGM entry size that defines the number of QPs per MCG Not in use with device managed flow steering 9 10 11 12 default 12 msi x Enables MSI X e 1 enabled e 0 disabled default 1 mtu 4k Enables configuration of 4k MTU 1 enabled 0 disabled default 0 Table 2 mlx4 en Module Parameters Parameter Description Values inline thold Threshold for using inline data 0 104 default 104 netq Uses netqueue 1 enabled 0 disabled default 1 netq num rings per rss Number of rings per RSS netqueue 0 2 4 default 0 pferx Priority based Flow Control policy on RX Per priority bit mask It is 8 bits bit mask each bit indicates priority 0 7 Bit value 1 respect incoming pause frames on the specified priority 0 ignore incoming pause frames on the specified priority 0 255 default 0 Mellanox Technologies 4 Rev 1 9 9 0 Driver Software Table 2 mlx4 en Module Parameters Parameter Description Values pfctx Priority based Flow Control policy on TX Per priority bit 0 255 mask
7. netq 1 upd_rss 1 Unless set again these parameters will be cleared to the driver s default values after k next reload When changing the module parameter only in mlx4_en module it is unnecessary to unload the mlx4 core module 4 1 2 Changing the Driver s Module Parameters Default Values Permanently To permanently change the module parameters default values 1 Query the available module parameters gt esxcli system module parameters list m module name 2 Set the driver with the required parameters d esxcli system parameters set m module name p parameters list For example gt esxcli system module parameters set m mlx4 en p netg 1 udp rss 1 3 Verify that the parameters are set correctly d esxcli system module parameters list m module name gt 4 2 Disabling Enabling Automatic Load of the Driver upon Boot 1 Query the driver auto load status gt esxcli system module list 7 Technologies Rev 1 9 9 0 2 Disable auto load on boot gt esxcli system module set enabled false m mlx4 core gt esxcli system module set enabled false m mlx4 en 3 Enable auto load on boot gt esxcli system module set enabled true m mlx4 core gt esxcli system module set enabled true m mlx4 en 4 3 Adding the Device as an uplink to an Existing Vswitch using the CLI 4 3 1 Locally 1 Log into the ESXi server with root permissions
8. 2 Find your device uplink name under the name column gt esxcli network nic list 3 Addan uplink from a vSwitch gt esxcli network vswitch standard uplink add u uplink name v vswitch name Once you add a device via the CLI it is visible in the vSphere client console thus d removing it can be performed via the UI 4 Check that the uplink was added successfully gt esxcli network vswitch standard list v vswitch name To remove the device locally 1 Log into the ESXi server with root permissions 2 Remove an uplink from a vSwitch gt esxcli network vswitch standard uplink remove u uplink name v vswitch name For additional documents please refer to the VMware site https pubs vmware com vsphere 50 index jsp topic 2Fcom vmware vcli ref doc_50 2Fesxcli_network html 4 3 2 Remotely 1 Download and install VMware vSphere Management Assistant from https my vmware com web vmvware info slug datacenter cloud infrastructure vmware vsphere 5 5 2 Use the command vicfg vswitch from the environment For additional documents please refer to the VMware site https www vmware com support pubs vsphere esxi vcenter server pubs html Mellanox Technologies 8 Rev 1 9 9 0 Additional Driver Settings 4 3 3 Renaming the uplink Name The uplink naming format is in an increasing order e g when working in either Multifunction Mode or Flex10 vmnicO to vm
9. erver machine prior to installing the new version gt esxcli software vib remove n net mlx4 en gt esxcli software vib remove n net mlx4 core b Install the MLX4 EN driver offline bundle package gt esxcli software vib install d lt path gt MLNX EN ESX 1 9 9 0 zip c Reboot ESXi server The driver will be loaded automatically gt To verify that the driver is loaded gt esxcli system module list grep mlx4 core gt esxcli system module list grep mlx4 en gt query network uplinks installed on your machine gt esxcli network nic list The number of uplinks claimed by MLX4 EN module should be displayed Removing the offline bundle Driver gt To remove the offline bundle driver package from the ESXi server machine gt esxcli software vib remove n net mlx4 en gt esxcli software vib remove n net mlx4 core Driver Default Values The below are mlx4 en and mlx4 core module parameters Some of these values can be changed by using module parameters which can be obtained by run ning d esxcli system module parameters list m module name gt 3 Mellanox Technologies Rev 1 9 9 0 For further information please refer to Section 4 1 Changing Driver Default Values on page 7 Table 1 4 core Module Parameters Parameter Description Values debug level Enables debug tracing 1 enabled 0 disabled default 0 enable 64b cqe eqe Enables 64
10. nic7 If the order is disrupted inconsistent and you wish to correct please follow the procedure below Log into the ESXi server with root permissions Open the vi etc vmware esx conf file Locate the device lt PCi device gt vmkname vmnicX Change the vmnic numbers to the desired order Save the file Reboot the server A Two vmnics with the same number cannot exist in the same ESXi server 9 Mellanox Technologies
Download Pdf Manuals
Related Search