Page 1
Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide December 2007 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883 Text Part Number: OL-14325-01...
Page 2
OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. CCVP, the Cisco logo, and Welcome to the Human Network are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn is a service mark of Cisco Systems, Inc.;...
Unified Fabric Transparent Topology Redundancy and High-Availability Scalability Cisco SFS 3504 Server Switch Chassis Cards, Modules, and Panels for the Cisco SFS 3504 Chassis InfiniBand Switch Card Fibre Channel Gateway Card Ethernet Gateway Card AC Power-Fan Module Blanking Panels for the Cisco SFS 3504 Server Switch Chassis...
Page 4
Safety Preparing to Install Installing Cisco SFS 3504 Server Switch Components Mounting the Cisco SFS 3504 Server Switch on a Rack Installing with the Power-Fan Modules Flush-front Installing with the Data-Connection Cables Flush-front Attaching a Serial Console Cable to a PC or Terminal...
Page 5
Specifications and Compliance A P P E N D I X General Specifications Electrical Specifications Laser Specifications Acronyms and Abbreviations A P P E N D I X N D E X Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Preface This preface describes who should read the Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide, how it is organized and its document conventions. It includes the following sections: Audience, page vii • Organization, page vii • Conventions, page viii •...
^D in a screen display means hold down the Control key while you press the D key. < > Nonprinting characters, such as passwords are in angle brackets. Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide viii OL-14325-01...
Statement 1071—Warning Definition This section repeats, in multiple languages, the basic warning appropriate for the Cisco SFS 3504 Server Switch. See the Regulatory Compliance and Safety Information for the Cisco Server Fabric Switches: 7000D, 7000P, 7008P, 7012, 7024, 3012R, and 3504 for translations of all warning messages.
Page 10
å forhindre ulykker. Bruk nummeret i slutten av hver advarsel for å finne oversettelsen i de oversatte sikkerhetsadvarslene som fulgte med denne enheten. TA VARE PÅ DISSE INSTRUKSJONENE Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 11
Använd det nummer som finns i slutet av varje varning för att hitta dess översättning i de översatta säkerhetsvarningar som medföljer denna anordning. SPARA DESSA ANVISNINGAR Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 12
Brug erklæringsnummeret efter hver advarsel for at finde oversættelsen i de oversatte advarsler, der fulgte med denne enhed. GEM DISSE ANVISNINGER Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 13
Preface Statement 1071—Warning Definition Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide xiii OL-14325-01...
Preface Related Documentation Related Documentation For additional information about the Cisco SFS 3504 Server Switch and related products and interfaces, refer to the following publications: Cisco SFS 3504 Multifabric Server Switch Installation and Configuration Note • Regulatory Compliance and Safety Information for the Cisco Server Fabric Switches: 7000D, •...
Page 16
Preface Obtaining Documentation and Submitting a Service Request Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Rail Kit for Rack Mounting, page 1-9 Introduction The Cisco SFS 3504 Server Switch is a next generation gateway chassis that offers gateway slots with high performance and high port count gateway line cards. It supports DDR, 24 4xDDR (20 Gbps), switch and backplane IB ports.
• IB Connectivity The Cisco SFS 3504 Server Switch enables a typical deployment of IB products that include a set of computer servers with PCIe IB HCAs, switches, and core IB switches. The server switch enables IB networks to connect to Ethernet and Fibre Channel networks.
System Features Redundancy and High-Availability Two Cisco SFS 3504 Server Switches can be configured to achieve redundancy. A typical example of a redundant configuration is to attach a server with a two-port HCA to two IB switch modules on two...
Cisco SFS 3504 Server Switch Chassis Figure 1-2 displays the data connection end of the Cisco SFS 3504 Server Switch and numbers the slots on the chassis. FRUs with interfaces populate the data connection end of the chassis. Figure 1-2...
Page 21
BE FO RE SE RV IC IN G PS1—AC power-fan module PS2—AC power-fan module The power-fan module slot identification numbers are embossed inside the Cisco SFS 3504 chassis as shown in Figure 1-4. Figure 1-4 Location of AC Power-Fan Module Slot Identification Numbers...
The IB switch card and modules are all field replaceable. If any slot is vacant on the Cisco SFS 3504 Server Switch, a blanking panel is required in the slot to Note ensure proper air flow and cooling of the system.
Ethernet ports, and Ethernet traffic, coming from Ethernet ports, are encapsulated into IB packets and forwarded to the IB fabric. The Cisco SFS 3504 chassis supports between one and four Ethernet gateway cards. The Product ID number for the Ethernet gateway card is SFS-3500-ENGW-1G.
SE RV IC IN G If only one power-fan module is used in a Cisco SFS 3504 Server Switch, it can be installed in either one Note of the two slots. If a power-fan module slot is vacant, a blanking panel is required in the slot to ensure...
Rail Kit for Rack Mounting A rail kit for rack mounting the Cisco SFS 3504 Server Switch ships with the chassis. It is optional to use this rail kit. The rail kit contains two adjustable brackets for the data connection end of the server switch and two fixed brackets (also called rack ear brackets) for the power supply end of the server switch.
Page 26
Cisco SFS 3504 Server Switch Overview Rail Kit for Rack Mounting The Cisco SFS 3504 Server Switch is shipped with two pairs of premounted guide rails that are used to adjust the brackets on the data connection end during installation (see Figure 1-11).
C H A P T E R Preparing to Install Planning a proper location for the Cisco SFS 3504 Server Switch and the proper layout of your equipment rack is essential for successful system operation. This chapter describes how to prepare your...
These sections describe some of the basic site requirements that you should be aware of as you prepare to install your Cisco SFS 3504 Server Switch. Environmental factors can adversely affect the performance and longevity of your system. Planning a proper location for the switch and layout of your equipment rack or wiring closet is essential for successful system operation.
When wires are run for any significant distance in an electromagnetic field, interference can occur between the field and the signals on the wires. This fact has two implications for the construction of plant wiring: Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
If you have had problems of this sort in the past, you may want to consult experts in electrical surge suppression and shielding. Shock and Vibration Cisco SFS 3504 Server Switches have been shock- and vibration-tested for operating ranges, handling, and office vibration and criteria, transportation vibration and criteria, and packaged equipment shock. (See Table A-1 on page A-1.)
EMI shielding and connectors are integral components of the carrier. Although the metal carrier helps to protect the circuit board from ESD, always use an ESD grounding strap when handling modules. Air Flow Air flows through the Cisco SFS 3504 Server Switch from rear to front. (See Figure 2-1.)
Site Preparation Checklist Table 2-1 lists the site planning activities that you should perform prior to installing the Cisco SFS 3504 Server Switch. Completing each activity helps ensure a successful switch installation. Table 2-1 Site Planning Checklist Task No.
Page 33
Site Planning Checklist (continued) Task No. Planning Activity Verified By Time Date Grounding evaluation: Circuit breaker size • EMI evaluation: Distance limitations for signaling • Site wiring • RFI levels • Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 34
Chapter 2 Preparing to Install Site Preparation Checklist Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
C H A P T E R Installing the Cisco SFS 3504 Server Switch This chapter describes how to mount your Cisco SFS 3504 Server Switch on a rack, boot the Cisco SFS 3504 Server Switch, and configure basic services. For advanced configuration information, see the Cisco SFS InfiniBand Software Configuration Guide or the Cisco SFS Product Family Element Manager User Guide.
Install only in accordance with national and local wiring regulations. Statement 1045 Warning Use of controls, adjustments, or performing procedures other than those specified may result in hazardous radiation exposure. Statement 1057 Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Place the chassis on a secure, clean surface. Step 2 Verify that you have the parts for the rail kit for rack mounting (shipped with the Cisco SFS 3504 Server Step 3 Switch as part of the accessory kit or the Shelf Kit (ordered and shipped separately, product number SFS-7008P-RKIT=).
Cisco SFS 3504 Server Switch chassis. Although we recommend that you install components before you mount the Cisco SFS 3504 Server Switch, it is also possible to add or change components after mounting the Cisco SFS 3504 Server Switch chassis in the rack. Instructions for installing or replacing the components appear in Chapter 4, “Installing and Removing Field-Replaceable...
Mounting the Cisco SFS 3504 Server Switch on a Rack Installing with the Power-Fan Modules Flush-front To install the Cisco SFS 3504 Server Switch on a rack so that the power-fan module end is flush mounted to the front rack posts, perform the following steps: Open the bag that contains the mounting hardware.
Page 40
Slide the two adjustable brackets, from the back, between each of the guide rails that are attached on Step 4 either side of the Cisco SFS 3504 Server Switch at the data connection end (see Figure 3-2).
The adjustable bracket should always be fully supported by the guide rails. Installing with the Data-Connection Cables Flush-front To install the Cisco SFS 3504 Server Switch on a rack so that the data-connection cables are flush front to the rack posts, perform the following steps: Step 1 Open the bag that contains the mounting hardware.
Page 42
Installing the Cisco SFS 3504 Server Switch Mounting the Cisco SFS 3504 Server Switch on a Rack Remove the two guide rails that are attached on either side of the Cisco SFS 3504 Server Switch at the Step 2 data-connection end and attach the guide rails similarly at the power-fan modules end (see Figure 3-3).
Page 43
100 /100 0Ba se T TX/R X Link Rack ear brackets Slide the Cisco SFS 3504 Server Switch between the rack posts (see Figure 3-5), until the rack ear Step 4 brackets on the chassis make contact with the front rack posts.
Page 44
Link Slide the two adjustable brackets, from the back, between each of the guide rails that are attached on Step 5 either side of the Cisco SFS 3504 Server Switch at the power-fan modules end (see Figure 3-5). Adjust the adjustable brackets so that the holes in the brackets align with the holes in the rack posts, and...
HCA. Use IB cables to connect the HCA in your host to the IB switch card of your Cisco SFS 3504 Server Switch. To plug in an IB cable with a pinch or a pull connector, push the connector into the interface. In addition, for the pull connector, push the latch until it is engaged after it is in the interface.
Page 46
To remove a cable with a pinch connector, pinch both sides of the back of the connector, as shown in Figure 3-8, and pull the connector away from the port. Figure 3-8 Removing a Pinch Connector Press here Press here Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide 3-12 OL-14325-01...
Pull here Hold here Managing the Cisco SFS 3504 Server Switch You can manage the Cisco SFS 3504 Server Switch by using one of the following methods: • Command-Line Interface (CLI)—a text-based interface accessible through a direct serial connection, Telnet over IP, or SSH over IP.
Page 48
Chapter 3 Installing the Cisco SFS 3504 Server Switch Managing the Cisco SFS 3504 Server Switch Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide 3-14 OL-14325-01...
Removing an InfiniBand Switch Card, page 4-4 An IB switch card can be removed and installed in the Cisco SFS 3504 Server Switch at any time. However, if the system is in operation when the IB switch card is removed, it powers down. It powers up again when an IB switch card is inserted.
M u lt if a b Lin k ri c S w it SFS 350 0 Ser ies Eth ern et Ga tew ay TX/ RX Link 100 /10 00B ase T TX/ RX Link Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
IB switch card spring lock See the Cisco SFS InfiniBand Software Configuration Guide for instructions about booting and configuring the Cisco SFS 3504 Server Switch as well as about updating the software image on the IB switch card. Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide...
Fibre Channel Gateway Card Removing an InfiniBand Switch Card An IB switch card can be removed and installed in the Cisco SFS 3504 Server Switch at any time. Note However, if the system is in operation when the IB switch card is removed, it powers down. It powers up again when an IB switch is inserted.
Lin k TX/ RX Lin k Close both ejector levers simultaneously and completely. Step 4 Push both ejector levers inwards and lock securely with the spring locks (see Figure 4-4). Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 54
Type the show card all CLI command to check whether the software image in the gateway matches the software image used by the chassis. For a description of the show card all command, see the Cisco SFS Product Family Command Reference.
Step 3 Facing the data connection end of the Cisco SFS 3504 Server Switch, pull both ejector levers of the Fibre Channel gateway module simultaneously towards you and all the way out. The ejector levers should automatically lift the securing spring locks as you begin to pull them outward.
Page 56
S w it SF S3 500 Se rie s Eth ern et Ga tew ay TX/ RX Lin k 10 0/1 00 0B as e T TX/ RX Lin k Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 57
Type the show card all CLI command to check whether the software image in the gateway matches the Step 5 software image used by the chassis. For a description of the show card all command, see the Cisco SFS Product Family Command Reference...
Properly ground yourself to avoid potential ESD discharge and damage to the Ethernet gateway card. Step 3 Facing the data connection end of the Cisco SFS 3504 Server Switch, pull both levers of the Ethernet gateway module simultaneously towards you and all the way out.
Properly ground yourself to avoid potential ESD discharge and damage to the hardware. Step 1 Facing the data connection end of the Cisco SFS 3504 Server Switch, slide both wire latches on the Step 2 blanking panel inwards while pulling the panel towards you and out of the chassis.
Statement 1034 Caution Never place your hand inside an empty module bay or anywhere inside the Cisco SFS 3504 Server Switch. Unused module bays should always have an AC power-fan blanking panel over the bay to ensure proper Caution safety, ventilation, and cooling.
Page 61
8-4 A 50 /60 Hz CA UT IO Captive screw N: DI SC ON NE CT PO WE R BE FO RE SE RV IC IN G Captive screw Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide 4-13 OL-14325-01...
Either replace the module immediately or secure a blanking panel to the empty bay to ensure appropriate cooling. Never operate the Cisco SFS 3504 Server Switch with a vacant AC power-fan module slot. Caution Blanking Panel for AC Power-Fan Module Slot...
Page 63
8-4 A 50 /60 Hz CA UT IO N: DI SC ON NE CT PO WE R BE FO RE SE RV IC IN G Captive screw Captive screw Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide 4-15 OL-14325-01...
Hold the pull handle and pull the blanking panel toward you until it completely disengages from the Step 3 chassis. Caution Never operate the Cisco SFS 3504 Server Switch with a vacant AC power-fan module slot. Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide 4-16 OL-14325-01...
C H A P T E R Cisco SFS 3504 Server Switch LEDs This chapter describes the LEDs on the Cisco SFS 3504 Server Switch and its components and includes the following sections: InfiniBand Switch LEDs, page 5-1 • Fibre Channel Gateway LEDs, page 5-4 •...
Slow blink Slow blink LED test following power on condition (4 seconds) (4 seconds) Blink Identify the IB switch card by causing it to blink by using the (1/2 second) Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Management Ethernet Port LED Indicators LEDs Indicators Link Label TX/RX Label Don’t care No signal and port if off Link is present but no traffic or activity Flickering TX/RX traffic active Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Location of Fibre Channel Gateway and Fibre Channel Port LEDs Fault Link SFS3500 Series FC Gateway Fault Link Fibre Channel card status LEDs Fibre Channel port status LEDs (1 green and 1 yellow) (1 green and 1 yellow) Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Blinking Set to beaconing (1/2 second) Blinking Active communication (exchange of Fibre Channel frames) (1/8 second) Blinking Blinking Set to beaconing and active communication (1/8 second) (1/2 second) Port failure Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Slow blink Slow blink LED test following power on condition: Both LEDs blink slowly (1/4 Hz.) Fast Blink Identify the gateway card by causing it to blink by using the Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Ethernet transmit and receive activity. This LED is labelled TX and is green. Table 5-8 Ethernet Activity LED Speed Indicator LED State Indicators No Tx or Rx traffic Green (blinks with Tx or Rx traffic active traffic: on/off) Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Valid AC input voltage detected. FAN OK Green Signals the fan status State Decode Fan failure detected (in either of two fans in a module). Fans fully operational (both fans in a module). Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 73
AC Power-Fan Module Status LED Indicators (continued) LED Label Color Indicators OUTPUT FAIL Yellow Signals the DC line voltage status State Decode 12 V DC output voltage working. 12 V DC output voltage failure detected. Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 74
Chapter 5 Cisco SFS 3504 Server Switch LEDs AC Power-Fan Module LEDs Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide 5-10 OL-14325-01...
A P P E N D I X Specifications and Compliance This appendix describes the Cisco SFS 3504 Server Switch specifications and compliance certifications. General Specifications Table A-1 General Specifications Specification Detail Operating Temperature Range 0° to 40° C Non-Operating Temperature -40 to 70 °...
InfiniBand Trade Association identification Internet Protocol light-emitting diode logical unit number Peripheral Component Interconnect PCIe PCI Express radio frequency interference storage area network small form-factor pluggable TX/RX transmit and receive Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...
Page 78
Appendix B Acronyms and Abbreviations Cisco SFS 3504 Multifabric Server Switch Hardware Installation Guide OL-14325-01...