Mellanox Technologies ConnectX-3 User Manual
  1. Manuals
  2. Brands
  3. Mellanox Technologies Manuals
  4. Adapter
  5. ConnectX-3 Pro
  6. User manual
Mellanox Technologies ConnectX-3 User Manual

Mellanox Technologies ConnectX-3 User Manual

Vpi single and dual qsfp+ port adapter card
Hide thumbs Also See for ConnectX-3:
Table of Contents

Advertisement

Quick Links

See also: User Manual
ConnectX®-3 VPI Single and Dual QSFP+ Port
Adapter Card User Manual
P/N:
MCX353A-FCBT, MCX353A-FCBS, MCX353A-TCBT, MCX353A-QCBT, MCX354A-FCBT, MCX354A-
FCBS, MCX354A-TCBT, MCX354A-QCBT
Rev 2.5
www.mellanox.com

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the ConnectX-3 and is the answer not in the manual?

Questions and answers

Related Manuals for Mellanox Technologies ConnectX-3

Summary of Contents for Mellanox Technologies ConnectX-3

  • Page 1 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual P/N: MCX353A-FCBT, MCX353A-FCBS, MCX353A-TCBT, MCX353A-QCBT, MCX354A-FCBT, MCX354A- FCBS, MCX354A-TCBT, MCX354A-QCBT Rev 2.5 www.mellanox.com...
  • Page 2 KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY.

  • Page 3: Table Of Contents

    Chapter 4 Driver Installation ........26 Mellanox Technologies...
  • Page 4 7.3 MCX353A-TCBT Specifications ....... .71 7.4 MCX354A-TCBT Specifications ....... .72 Mellanox Technologies...
  • Page 5 Appendix D Avertissements de sécurité d’installation (Warnings in French) 89 Appendix E Sicherheitshinweise (Warnings in German) ....91 Appendix F Advertencias de seguridad para la instalación (Warnings in Spanish) 93 Mellanox Technologies...

  • Page 6: List Of Figures

    Figure 17: Card Product Label ........... . . 86 Mellanox Technologies...

  • Page 7: List Of Tables

    Physical and Logical Link Indications (InfiniBand Mode) ..... . . 76 Table 18: Physical and Logical Link Indication (Ethernet Mode ......) 76 Mellanox Technologies...

  • Page 8: Revision History

    Table 15, “MCX353A-FCB[T/S] Specifications Table,” on page 73 Table 16, “MCX354A-FCB[T/S] Specifications Table,” on page 74 October 2012 • Fixed Table 5, “Jumper Configuration,” on page 17 • Updated figures in Section 4.2, “Windows Driver,” on page 46 by providing ConnectX-3 examples Mellanox Technologies...
  • Page 9 Section 2.4, “I2C-compatible Inter- face,” on page 18 • Formatted specification tables in Appendix 7, “Specifications,” on page 68 October 2011 Added new OPNs based on ConnectX-3 Step A1 devices July 2011 Minor edits July 2011 0.10 First Release Mellanox Technologies...

  • Page 10: About This Manual

    Rev 2.5 About this Manual This User Manual describes Mellanox Technologies ConnectX®-3 VPI Single and Dual QSFP+ port PCI Express x8 adapter cards. It provides details as to the interfaces of the board, specifica- tions, required software and firmware for operating the board, and relevant documentation.

  • Page 11: Technical Support

    The Mellanox support downloader contains software, firmware and knowledge database infor- mation for Mellanox products. Access the database from the Mellanox Support web page, http://www.mellanox.com => Support or use the following link to go directly to the Mellanox Support Download Assistant page, http://www.mellanox.com/supportdownloader/. Mellanox Technologies...

  • Page 12: Chapter 1 Introduction

    Rev 2.5 Introduction This is the User Guide for Mellanox Technologies VPI adapter cards based on the ConnectX®-3 VPI integrated circuit device. These adapters connectivity provide the highest performing and most flexible interconnect solution for PCI Express Gen3 servers used in Enterprise Data Cen-...

  • Page 13: Table 4: Single And Dual-Port Fdr10 Adapter Cards

    Data Transmission Rate Ethernet: 40/56Gb/s Network Connector Types single or dual-port QSFP+ PCI Express (PCIe) SerDes Speed PCIe 3.0 x8 8GT/s RoHS Adapter IC Part Number MT27508A1-FCCR-FV 4099 for Physical Function Device ID (decimal) 4100 for Virtual Function Mellanox Technologies...

  • Page 14: Features And Benefits

    Applications utilizing TCP/UDP/IP transport can achieve industry leading Sockets Acceleration throughput over InfiniBand or 10 or 40GbE. The hardware-based stateless offload engines in ConnectX-3 reduce the CPU overhead of IP packet transport. Sockets acceleration software further increases performance for latency sensitive applica- tions.
  • Page 15 (VM) within the server. SR-IOV I/O virtualization with ConnectX-3 Pro gives data center managers better server utilization while reducing cost, power, and cable complexity. A consolidated compute and storage network achieves significant cost-perfor- mance advantages over multi-fabric networks.

  • Page 16: Operating Systems/Distributions

    • Interoperable with InfiniBand or 10/40 Gb Ethernet switches. • Interoperable with 56GbE Mellanox Switches. • Passive copper cable with ESD protection • Powered connectors for optical and active cable support • QSFP to SFP+ connectivity through QSA module Mellanox Technologies...

  • Page 17: Chapter 2 Interfaces

    . Ethernet traffic is transmitted through the Table 6, “Features,” on page 14 cards' QSFP+ connectors. For connecting to an SFP+ interface, you can use Mellanox QSA (QSFP to SFP+) adapter module (to be ordered separately). Mellanox Technologies...

  • Page 18: Pci Express Interface

    “Mechanical Drawing of the Dual-port MCX354A-[QFT]CBT Adapter Card,” on page 78 for the location on the board. LED Interface There are two I/O LEDs per port. For LED specifications please refer to Section 7.7, “Adapter LED Operation,” on page 76 Mellanox Technologies...

  • Page 19: Chapter 3 Hardware Installation

    . If you need to replace it with the short bracket that is included in the Instructions,” on page 22 shipping box, please follow the instructions in this section. To replace the bracket you will need the following parts: • The new bracket of the proper height • One new gasket Mellanox Technologies...

  • Page 20: Removing The Existing Bracket

    Note: The following instructions relate to the installation of the gasket for adapter cards that are not shipped with a gasket already installed. Mellanox Technologies now provides all ConnectX- 3 adapter cards with the gasket already installed unto the bracket.

  • Page 21: 3.4.2.2 Installing The Bracket

    Do not force the bracket onto the card. You may have to gently push the LEDs using a small screwdriver to align the LEDs with the holes in the bracket. 2. Screw on the bracket using the screws and washers saved from the bracket removal proce- dure above. Mellanox Technologies...

  • Page 22: Card Installation Instructions

    4. Applying even pressure at both corners of the card, insert the adapter card into the slot until it is firmly seated. When the adapter is properly seated, the adapter port connectors are aligned with the slot opening, and the adapter faceplate is visible against the system chassis. Mellanox Technologies...

  • Page 23: Cables And Modules

    4. After plugging in a cable, lock the connector using the latching mechanism particular to the cable vendor. When a logical connection is made the Yellow LED will light. When data is being transferred the yellow led will blink. See Section 7.7, “Adapter LED Operation,” on page 76 Mellanox Technologies...

  • Page 24: Identify The Card In Your System

    6. In the Value display box, check the fields VEN and DEV (fields are separated by ‘&’). In the display example above, notice the sub-string “PCI\VEN_15B3&DEV_1003”: VEN is equal to 0x15B3 – this is the Vendor ID of Mellanox Technologies; and DEV is equal to 1003 – this is a valid Mellanox Technologies PCI Device ID.

  • Page 25: On Linux

    ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 The list of Mellanox Technologies PCI Device IDs can be found in the PCI ID reposi- tory at http://pci-ids.ucw.cz/read/PC/15b3. 3.7.2 On Linux Get the device location on the PCI bus by running lspci and locating lines with the string “Mella- nox Technologies”:...

  • Page 26: Chapter 4 Driver Installation

    ConnectX entries in the display. The following example shows a system with an installed Mellanox HCA: host1# lspci -v | grep Mellanox 02:00.0 InfiniBand: Mellanox Technologies MT25418 [ConnectX IB DDR, PCIe 2.0 2.5GT/s] (rev a0) Download the ISO image to your host.

  • Page 27: Installing Mellanox Ofed

    The following command will create a MLNX_OFED_LINUX ISO image for RedHat 5.6 under the /tmp directory. MLNX_OFED_LINUX-1.5.3-rhel5.6-x86_64/docs/mlnx_add_kernel_support.sh -i /mnt/MLNX_OFED_LINUX-1.5.3-rhel5.6-x86_64.iso All Mellanox, OEM, OFED, or Distribution IB packages will be removed. 1. The firmware will not be updated if you run the install script with the ‘--without-fw-update’ option. Mellanox Technologies...

  • Page 28: 4.1.3.2 Installation Script

    Removing OFED RPMs... Running mkisofs... Created /tmp/MLNX_OFED_LINUX-1.5.3-rhel5.6-x86_64.iso 4.1.3.2 Installation Script The usage of the installation script is described below. You will use it during the installation pro- cedure described in Section 4.1.3.4, “Installation Procedure,” on page 30 Usage ./mnt/mlnxofedinstall [OPTIONS] Mellanox Technologies...

  • Page 29: 4.1.3.3 Mlnxofedinstall Return Codes

    Set quiet - no messages will be printed --umad-dev-rw Grant non root users read/write permis- sion for umad devices instead of default --hugepages-overcommit Set 80% of MAX_MEMORY as overcommit for a huge page allocation 4.1.3.3 mlnxofedinstall Return Codes Mellanox Technologies...

  • Page 30: 4.1.3.4 Installation Procedure

    Failed to start the mst driver 4.1.3.4 Installation Procedure Login to the installation machine as root. Step 1. Mount the ISO image on your machine Step 2. host1# mount -o ro,loop MLNX_OFED_LINUX-<ver>-<OS label>-<CPU arch>.iso /mnt Mellanox Technologies...
  • Page 31 ################################################## mlnx-ofa_kernel ################################################## Installing kmod-mlnx-ofa_kernel RPM Preparing... ################################################## kmod-mlnx-ofa_kernel ################################################## Installing mlnx-ofa_kernel-devel RPM Preparing... ################################################## mlnx-ofa_kernel-devel ################################################## Installing kernel-mft RPM Preparing... ################################################## kernel-mft ################################################## Installing knem RPM Preparing... ################################################## knem ################################################## Installing mpi-selector RPM Preparing... ################################################## mpi-selector ################################################## Mellanox Technologies...
  • Page 32 ################################################## Preparing... ################################################## libibverbs-devel ################################################## Preparing... ################################################## libibverbs-devel-static ################################################## Preparing... ################################################## libibverbs-devel-static ################################################## Preparing... ################################################## libibverbs-utils ################################################## Preparing... ################################################## libmverbs ################################################## Preparing... ################################################## libmverbs ################################################## Preparing... ################################################## libmlx4 ################################################## Preparing... ################################################## libmlx4 ################################################## Preparing... ################################################## libmlx4-devel ################################################## Mellanox Technologies...
  • Page 33 ################################################## Preparing... ################################################## libmlx5-devel ################################################## Preparing... ################################################## libmverbs-devel ################################################## Preparing... ################################################## libmverbs-devel ################################################## Preparing... ################################################## libmqe ################################################## Preparing... ################################################## libmqe ################################################## Preparing... ################################################## libmqe-devel ################################################## Preparing... ################################################## libmqe-devel ################################################## Preparing... ################################################## libibcm ################################################## Preparing... ################################################## libibcm ################################################## Mellanox Technologies...
  • Page 34 ################################################## Preparing... ################################################## libibumad ################################################## Preparing... ################################################## libibumad-devel ################################################## Preparing... ################################################## libibumad-devel ################################################## Preparing... ################################################## libibumad-static ################################################## Preparing... ################################################## libibumad-static ################################################## Preparing... ################################################## libibmad ################################################## Preparing... ################################################## libibmad ################################################## Preparing... ################################################## libibmad-devel ################################################## Preparing... ################################################## libibmad-devel ################################################## Mellanox Technologies...
  • Page 35 ################################################## Preparing... ################################################## ibacm ################################################## Preparing... ################################################## librdmacm ################################################## Preparing... ################################################## librdmacm ################################################## Preparing... ################################################## librdmacm-utils ################################################## Preparing... ################################################## librdmacm-devel ################################################## Preparing... ################################################## librdmacm-devel ################################################## Preparing... ################################################## opensm-libs ################################################## Preparing... ################################################## opensm-libs ################################################## Preparing... ################################################## opensm ################################################## Mellanox Technologies...
  • Page 36 ################################################## Preparing... ################################################## opensm-static ################################################## Preparing... ################################################## compat-dapl ################################################## Preparing... ################################################## compat-dapl ################################################## Preparing... ################################################## compat-dapl-devel ################################################## Preparing... ################################################## compat-dapl-devel ################################################## Preparing... ################################################## dapl ################################################## Preparing... ################################################## dapl ################################################## Preparing... ################################################## dapl-devel ################################################## Preparing... ################################################## dapl-devel ################################################## Mellanox Technologies...
  • Page 37 ################################################## dapl-devel-static ################################################## Preparing... ################################################## dapl-utils ################################################## Preparing... ################################################## perftest ################################################## Preparing... ################################################## mstflint ################################################## Preparing... ################################################## ################################################## Preparing... ################################################## srptools ################################################## Preparing... ################################################## rds-tools ################################################## Preparing... ################################################## rds-devel ################################################## Preparing... ################################################## ibutils2 ################################################## Preparing... ################################################## ibutils ################################################## Mellanox Technologies...
  • Page 38 - To start FCA Manager now, type: /etc/init.d/fca_managerd start - There should be single process of FCA Manager running per fabric. - To start FCA Manager automatically after boot, type: /etc/init.d/fca_managerd install_service - Check /opt/mellanox/fca/share/doc/fca/README.txt for quick start instructions. Mellanox Technologies...
  • Page 39 05:00.0 Ethernet controller: Mellanox Technologies MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0) Link Width is not 8x PCI Link Speed: 5Gb/s Device (07:00.0): 07:00.0 Ethernet controller: Mellanox Technologies MT27500 Fam- ily [ConnectX-3] Link Width: 8x PCI Link Speed: 5Gb/s Installation finished successfully.
  • Page 40 Band link is up. The utility also checks for and displays additional information such as • HCA firmware version • Kernel architecture • Driver version • Number of active HCA ports along with their states • Node GUID Mellanox Technologies...

  • Page 41: Installation Results

    • The OFED and MFT packages are installed under the /usr directory. • The kernel modules are installed under: • InfiniBand subsystem: /lib/modules/`uname -r`/updates/kernel/drivers/infiniband/ • mlx4_core driver: /lib/modules/<kernel_version>/extra/mlnx-ofa_kernel/drivers/net/ether- net/mellanox/mlx4/mlx4_core.ko • mlx4_ib driver: /lib/modules/<kernel_version>/extra/mlnx-ofa_kernel/drivers/infiniband/ hw/mlx4/mlx4_ib.ko • mlx5_core driver: /lib/modules/<kernel_version>/extra/mlnx-ofa_kernel/drivers/net/ether- net/mellanox/mlx5/core/mlx5_core.ko Mellanox Technologies...
  • Page 42 90-ib.rules /etc/udev/rules.d/ • If OpenSM is installed, the daemon is installed under opensmd /etc/init.d/ is installed under opensm.conf /etc • If IPoIB configuration files are included, files will be installed under: ifcfg-ib<n> • on a RedHat machine /etc/sysconfig/network-scripts/ Mellanox Technologies...

  • Page 43: Post-Installation Notes

    If you need to burn an Expansion ROM image, please refer to OFED User Manual. Table 2, “Documents List,” on page 9 The following steps are also appropriate in case you wish to burn newer firmware that http://www.mella- you have downloaded from Mellanox Technologies’ Web site ( nox.com => Support => Firmware Download). Start Step 1.

  • Page 44: Uninstalling Mellanox Ofed

    -dev /dev/mst/mt25418_pci_cr0 -fw /mnt/firmware/fw-25408/ fw-25408-rel.mlx Reboot your machine after the firmware burning is completed. Step 4. 4.1.7 Uninstalling Mellanox OFED Use the script to uninstall the Mellanox OFED package. /usr/sbin/ofed_uninstall.sh The script is part of the RPM. ofed-scripts Mellanox Technologies...

  • Page 45: Port Type Management On Linux

    (InfiniBand or Ethernet). For example, if the first port is connected to an InfiniBand switch and the second to Ethernet switch, the NIC will automatically load the first switch as InfiniBand and the second as Ethernet. Mellanox Technologies...

  • Page 46: 4.1.9.1 Enabling Auto Sensing

    Enter the following command: > echo %PROCESSOR_ARCHITECTURE% On an x64 (64-bit) machine, the output will be “AMD64”. Go to the MLNX_WinOF for Windows Web page at Step 2. http://www.mellanox.com => Products => Software => InfiniBand/VPI Drivers => Windows SW/Drivers. Mellanox Technologies...

  • Page 47: Extracting Files Without Running Installation

    To extract the files without running installation, perform the following steps. Open a CMD console (Click Start-->Run and enter CMD). Step 1. Enter the following command: Step 2. MLNX_VPI_WinOF-<version>_All_<OS>_<arch>.exe /a Click Next to create a server image. Step 3. Mellanox Technologies...
  • Page 48 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 Click Change and specify the location in which the files are extracted to. Step 4. Click Install to extract this folder, or click Change to install to a different folder. Step 5. Mellanox Technologies...

  • Page 49: Installing Mlnx_Winof

    To configure your setup to contain the logs option, please run the following command after opening a CMD console: MLNX_VPI_WinOF-4_40_0_All_win8_x64.exe /v"/l*vx [LogFile]" If you do not want to upgrade your firmware version, run the following command: MLNX_VPI_WinOF-4_40_0_All_win8_x64.exe /v" MT_SKIPFWUPGRD=1" For further help, please run: MLNX_VPI_WinOF-4_40_0_All_win8_x64.exe /v" /h" Mellanox Technologies...
  • Page 50 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 Click Next in the Welcome screen. Step 2. Read then accept the license agreement and click Next. Step 3. Mellanox Technologies...
  • Page 51 If the user has a standard Mellanox card, and the firmware version is older than the one specified in WinOF Installation Guide 4.40, the firmware will be updated accordingly. However, if the user has both OEM card and Mellanox card, only Mellanox card will be updated. Mellanox Technologies...
  • Page 52 Configure your system for maximum performance by checking the maximum performance Step 6. box. This step requires rebooting your machine at the end of the installation. Select a Complete or Custom installation, follow Step a and on, on page 53. Step 7. Mellanox Technologies...
  • Page 53 Band environment. • SDK - contains the libraries and DLLs for developing InfiniBand application over IBAL. • Documentation: contains the User Manual and Installation Guide. • ND FLTR DLLs: contains the files for standalone installation of the mlx4nd provider. Mellanox Technologies...
  • Page 54 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 Click Install to start the installation. Click Finish to complete the installation. Step 8. Mellanox Technologies...

  • Page 55: 4.2.4.2 Unattended Installation

    Windows 2012 driver do not completely uninstall the previous version. • In Windows 2012 (MLNX_WinOF Rev. 4.2 and above), the network configuration is saved upon driver upgrade. • In Windows 2008 R2 the existing configuration files are not saved upon driver upgrade. Mellanox Technologies...

  • Page 56: Installation Results

     To run OpenSM as a service, assuming the package was installed in the default path, use: sc create OpenSM1 binPath= "c:\Program Files\Mellanox\MLNX_VPI\IB\Tools\ opensm.exe --service" start=auto"  To start the service, run: sc start opensm Mellanox Technologies...

  • Page 57: Uninstalling Mlnx_Winof

    1. Click Start-> Control Panel-> Programs and Features-> MLNX_VPI-> Uninstall. (NOTE: This requires elevated administrator privileges.) 2. Double click the .exe and follow the instructions of the install wizard. 3. Click Start-> All Programs-> Mellanox Technologies-> MLNX_WinOF-> Uninstall MLNX- _WinOF. 4.2.8.2 Unattended Uninstall ...
  • Page 58 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 Right-click a Mellanox Local Area Connection and left-click Properties. Step 2. Select Internet Protocol Version 4 (TCP/IPv4) from the scroll list and click Properties. Step 3. Mellanox Technologies...

  • Page 59: 4.2.10 Port Type Management On Windows

    After installing Mellanox WinOF VPI for Windows on your machine, you can change a port's pro- tocol configuration. The following steps describe how to configure the port type: Step 1: Display the Device Manager and expand “System Devices”. Mellanox Technologies...
  • Page 60 Select the Port Protocol tab from the Properties sheet. Note: The “Port Protocol” tab is displayed only if the NIC is a VPI (IB and ETH). If the NIC is either only IB or ETH, the tab will not be shown. Mellanox Technologies...

  • Page 61: Performance Tuning

    Please uninstall any previous Mellanox driver packages prior to installing the new ver- sion.  To install the driver: 1. Log into the ESXi5.0 server with root permissions. 2. Install the driver. #> esxcli software vib install -d <bundle_file> 3. Reboot the machine. Mellanox Technologies...

  • Page 62: Uninstalling Mellanox Infiniband Ofed Driver

    Preboot Execution Environment (PXE) standard specification, and FlexBoot soft- ware is based on the open source iPXE project (see www.ipxe.org). For more information go to http://www.mellanox.com => Products => Software => InfiniBand/VPI Drivers => Download. Mellanox Technologies...

  • Page 63: Chapter 5 Updating Adapter Card Firmware

    Enter your card PSID to display the latest firmware file. The file name of the binary is composed by combin- ing the firmware name, the firmware release version, and the card part number. Note: Please contact Mellanox System Support if you cannot find the firmware binary for your adapter card. Mellanox Technologies...

  • Page 64: Figure 7: Support Download Assistant

    > flint -d /dev/mst/mt4099_pci_cr0 -i <binary image> burn a. Reboot the computer. b. Enter: mst start. c. Verify that the card firmware was updated successfully. > flint -d /dev/mst/mt4099_pci_cr0 Image type: ConnectX FW Version: 2.9.4100 Device ID: 4099 Mellanox Technologies...

  • Page 65: Chapter 6 Troubleshooting

    • Verify that your switch and adapter port are compatible • Check that the latest driver is loaded Link light is on, but • Check that both the adapter and its link are set to the same speed with no and duplex settings communication established Mellanox Technologies...

  • Page 66: Linux

    Mellanox Firmware Tool (MFT) Once installed, run: mst start mst status flint –d <mst_device> q ibstat Ports Information lbv_devinfo Firmware Version To download the latest firmware version refer to Upgrade http://www.mellanox.com/supportdownloader /var/log/messages Collect Log File dmesg > system.logF Mellanox Technologies...

  • Page 67: Windows

    –d <mst_device> –i <firmware_bin_file> b • Event log viewer • MST device logs: • mst start Collect log file • mst status • C:\Users\Administrator> flint –d <mst_device> dc > dump_configura- tion.log • C:\Users\Administrator> mstdump <mst_device> dc > mstdump.log Mellanox Technologies...

  • Page 68: Chapter 7 Specifications

    => Products => Cables and Transceivers Cable Support a. The ConnectX-3 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product.
  • Page 69 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 b. For both operational and non-operational states. c. Air flow is measured ~1” from the heat sink between the heat sink and the cooling air inlet. Mellanox Technologies...

  • Page 70: Mcx354A-Qcbt Specifications

    => Products => Cables and Transceivers Cable Support a. The ConnectX-3 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product.

  • Page 71: Mcx353A-Tcbt Specifications

    => Products => Cables and Transceivers Cable Support a. The ConnectX-3 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product.

  • Page 72: Mcx354A-Tcbt Specifications

    => Products => Cables and Transceivers Cable Support a. The ConnectX-3 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product.

  • Page 73: Mcx353A-Fcb[T/S] Specifications

    => Products => Cables and Transceivers Cable Support a. The ConnectX-3 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product.

  • Page 74: Mcx354A-Fcb[T/S] Specifications

    => Products => Cables and Transceivers Cable Support a. The ConnectX-3 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another Mellanox InfiniBand product.
  • Page 75 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 c. Air flow is measured ~1” from the heat sink between the heat sink and the cooling air inlet. Mellanox Technologies...

  • Page 76: Adapter Led Operation

    If neither LED is lit, then the physical link has not been established Yellow - logical (data • A blinking yellow indicates activity (data transfer) • Stays off when there is no activity activity link) The short bracket has the same port and LED footprints as the tall bracket. Mellanox Technologies...

  • Page 77: Board Mechanical Drawing And Dimensions

    Board Mechanical Drawing and Dimensions All dimensions are in millimeters. All the mechanical tolerances are +/- 0.1mm. Figure 8: Mechanical Drawing of the Single-port MCX353A-[QFT]CBT Adapter Card J1 - I C Connector Flash Jumper 142.24 40.5 49.09 22.61 53.59 45.95 12.75 Mellanox Technologies...

  • Page 78: Figure 9: Mechanical Drawing Of The Dual-Port Mcx354A-[Qft]Cbt Adapter Card

    ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 Figure 9: Mechanical Drawing of the Dual-port MCX354A-[QFT]CBT Adapter Card Mellanox Technologies...

  • Page 79: Bracket Mechanical Drawing

    ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 Bracket Mechanical Drawing Figure 10: Dual-port Tall Bracket Figure 11: Single-port Tall Bracket Mellanox Technologies...

  • Page 80: Figure 12: Dual-Port Short Bracket

    ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 Figure 12: Dual-port Short Bracket Port 1 LEDS Port 2 Figure 13: Single-port Short Bracket LEDs Gasket Port 1 Mellanox Technologies...

  • Page 81: Regulatory Statements

    ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 7.10 Regulatory Statements For regulatory statements for all ConnectX®-3 cards please refer to: http://www.mellanox.com/related-docs/user_manuals/Regulatory_and_Compliance_Guide.pdf Mellanox Technologies...

  • Page 82: Appendix A Interface Connectors Pinout

    Figure 14: Connector and Cage Views Table 19 - Connector Pin Number and Name to Signal Name Map Connector Pin Number Connector Pin Name Port A Signal Name TXN_2 Tx2n TXP_2 Tx2p TXN_4 Tx4n TXP_4 Tx4p ModSelL_Port0 ModSelL ResetL_Port0 ResetL VccRx Mellanox Technologies...
  • Page 83 Connector Pin Name Port A Signal Name RXP_3 Rx3p RXN_3 Rx3n RXP_1 Rx1p RXN_1 Rx1n RXN_2 Rx2n RXP_2 Rx2p RXN_4 Rx4n RXP_4 Rx4p ModPrsl_Port0 Mod PrsL IntL IntL VccTx Vcc1 LPMode_Port0 LPMode TXP_3 Tx3p TXN_3 Tx3n TXP_1 Tx1p TXN_1 Tx1n Mellanox Technologies...

  • Page 84: Figure 15: Pcie Connector Pinout

    ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 PCI Express x8 Connector Pinout The adapter cards use a standard PCI Express x8 edge connector and the PCI Express x8 standard pinout according to the PCI Express 3.0 specification. Figure 15: PCIe Connector Pinout Mellanox Technologies...

  • Page 85: Figure 16: Compatible Connector Plug And Pinout

    ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 C-compatible Connector Pinout Figure 16: Compatible Connector Plug and Pinout Connector Pin Number Signal Name SPSDA SPSCL Mellanox Technologies...

  • Page 86: Appendix B Finding The Guid/ Mac And Serial Number On The Adapter Card

    Figure 17: Card Product Label Port 1 uses the GUID or MAC described on the label. To obtain the GUID or MAC for Port 2 on dual port cards, add 1 to the GUID or MAC of Port 1. Mellanox Technologies...

  • Page 87: Appendix C Safety Warnings

    This equipment should be installed in compliance with local and national electrical codes. 8. Hazardous Radiation Exposure Caution – Use of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure. Mellanox Technologies...
  • Page 88 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 CLASS 1 LASER PRODUCT and reference to the most recent laser standards: IEC 60 825-1:1993 + A1:1997 + A2:2001 and EN 60825-1:1994+A1:1996+ A2:20. Mellanox Technologies...

  • Page 89: Appendix D Avertissements De Sécurité D'installation (Warnings In French)

    L’élimination de ce matériel doit s’effectuer dans le respect de toutes les législations et réglementations nationales en vigueur. 7. Codes électriques locaux et nationaux Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux. Mellanox Technologies...
  • Page 90 PRODUIT LASER DE CLASSE 1 » et références aux normes laser les plus récentes CEI 60 825-1:1993 + A1:1997 + A2:2001 et NE 60825- 1:1994+A1:1996+ A2:2001 Mellanox Technologies...

  • Page 91: Appendix E Sicherheitshinweise (Warnings In German)

    Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestim- mungen erfolgen. 7. Regionale und nationale elektrische Bestimmungen t Dieses Gerät sollte unter Beachtung der regionalen und nationalen elektrischen Bes- timmungen installiert werden. This equipment should be installed in compliance with local and national elec- trical codes. Mellanox Technologies...
  • Page 92 Achtung – Nutzung von Steuerungen oder Einstellungen oder Ausführung von Prozeduren, die hier nicht spezifiziert sind, kann zu gefährlichem Strahlenkon- takt führen.. Klasse 1 Laserprodukt und Referenzen zu den aktuellsten Lasterstandards : ICE 60 825-1:1993 + A1:1997 + A2:2001 und EN 60825-1:1994+A1:1996+ A2:2001 Mellanox Technologies...

  • Page 93: Appendix F Advertencias De Seguridad Para La Instalación (Warnings In Spanish)

    Este equipo se debe instalar conforme a los códigos eléctricos locales y nacionales. 8. Exposición a niveles de radiación peligrosos Precaución: el uso de controles o ajustes o la realización de procedimientos distintos de los que aquí se especifican podrían causar exposición a niveles de radiación peligrosos. Mellanox Technologies...
  • Page 94 ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual Rev 2.5 PRODUCTO LÁSER DE CLASE 1 y referencia a las normas de láser más recientes: IEC 60825-1:2007/03 y EN 60825-1:2007 Mellanox Technologies...

Table of Contents

Save PDF