Page 8
Audit log configuration ........107 Verifying host syslog prior to configuring the audit log ..108 Configuring an audit log for specific event classes .
Page 9
Local database user accounts ......137 Default accounts ........138 Local account passwords .
Page 11
IP Filter policy ......... . 217 Creating an IP Filter policy.
Page 12
Chapter 9 Installing and Maintaining Firmware Firmware download process overview ..... . .255 Upgrading and downgrading firmware ....257 Considerations for FICON CUP environments .
Page 13
Limitations and restrictions of Virtual Fabrics ....288 Restrictions on XISLs ....... . .289 Restrictions on moving ports .
Page 14
Zone creation and maintenance ......316 Displaying existing zones ......316 Creating a zone .
Page 15
General rules for TI zones....... . .356 Traffic Isolation Zone violation handling for trunk ports ..357 Supported configurations for Traffic Isolation Zoning .
Page 26
Figure 36 Illegal ETIZ configuration: two paths from one port to two devices on the same remote domain 351 Figure 37 Illegal ETIZ configuration: two paths from one port ..... . 352 Figure 38 Traffic Isolation Zoning over FCR.
Page 31
Table 78 VCs assigned to QoS priority for frame prioritization in CS_CTL auto mode . . 521 Table 79 Trunking over long-distance for the Backbones and blades ....541 Table 80 F_Port masterless trunking considerations .
Page 32
Fabric OS Administrator’s Guide 53-1002745-02...
• Chapter 11, “Administering Advanced Zoning,” provides procedures for use of the Brocade Advanced Zoning feature. • Chapter 12, “Traffic Isolation Zoning,” provides concepts and procedures for use of Traffic Isolation Zones within a fabric. • Chapter 13, “Bottleneck Detection,” describes how you can detect and configure alert thresholds for latency and congestion bottlenecks in the fabric.
• Updated the Note in “In-flight encryption and compression overview” on page 393. • “Encryption and compression restrictions” on page 394, clarified the restriction about the number of ports supported. • Corrected the “Example of enabling encryption and compression on an E_Port” on page 407 so that you activate authentication after setting up the DH-CHAP secret.
variable Variables are printed in italics. In the help pages, values are underlined or enclosed in angled brackets < >. Repeat the previous element, for example “member[;member...]” value Fixed values following arguments are printed in plain font. For example, show WWN Boolean.
Corporation Referenced Trademarks and Products Microsoft Corporation Windows, Windows NT, Internet Explorer Mozilla Corporation Mozilla, Firefox Netscape Communications Corporation Netscape Red Hat, Inc. Red Hat, Red Hat Network, Maximum RPM, Linux Undercover Sun Microsystems, Inc. Sun, Solaris Additional information This section lists additional Brocade and industry-specific documentation that you might find helpful.
Page 39
1. General Information • Switch model • Switch operating system version • Error numbers and messages received • supportSave command output • Detailed description of the problem, including the switch or fabric behavior immediately following the problem, and specific questions •...
Document feedback Quality is our first concern at Brocade and we have made every effort to ensure the accuracy and completeness of this document. However, if you find an error or an omission, or you think that a topic needs further development, we want to hear from you. Forward your feedback to: documentation@brocade.com Provide the title and version number of the document and as much detail as possible about your comment, including the topic heading and page number and your suggestions for improvement.
Section Standard Features This section describes standard Fabric OS features, and includes the following chapters: • Chapter 1, “Understanding Fibre Channel Services” • Chapter 2, “Performing Basic Configuration Tasks” • Chapter 3, “Performing Advanced Configuration Tasks” • Chapter 4, “Routing Traffic” •...
Page 42
Fabric OS Administrator’s Guide 53-1002745-02...
Management server Management server — The management server provides a single point for managing the fabric. This is the only service that users can configure. See “Management server” below for more details Alias server — The alias server keeps a group of nodes registered as one name to handle multicast groups.
Management server database Platform services and Virtual Fabrics Each logical switch has a separate platform database. All platform registrations done to a logical switch are valid only in that particular logical switch’s Virtual Fabric. Activating the platform services on a switch activates the platform services on all logical switches in a Virtual Fabric.
Management server database If the list is empty (the default), the management server is accessible to all systems connected in-band to the fabric. For more access security, you can specify WWNs in the ACL so that access to the management server is restricted to only those WWNs listed. NOTE The management server is logical switch-capable.
Management server database Example of adding a member to the management server ACL switch:admin> msconfigure Done Display the access list Add member based on its Port/Node WWN Delete member based on its Port/Node WWN select : (0..3) [1] 2 Port/Node WWN (in hex): [00:00:00:00:00:00:00:00] 20:00:00:20:37:65:ce:aa *WWN is successfully added to the MS ACL.
Management server database 5. At the “select” prompt, enter 1 to display the access list so you can verify that the WWN you entered was deleted from the ACL. 6. After verifying that the WWN was deleted correctly, enter 0 at the “select” prompt to end the session.
Topology discovery Number of Associated Node Names: 1 Associated Node Names: 10:00:00:60:69:20:15:75 Clearing the management server database Use the following procedure to clear the management server database: NOTE The command msPlClearDB is allowed only in AD0 and AD255. 1. Connect to the switch and log in using an account assigned to the admin role. 2.
Topology discovery *MS Topology Discovery enabled locally. *MS Topology Discovery Enable Operation Complete!! Disabling topology discovery Use the following procedure to disable topology discovery: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the appropriate following command based on how you want to disable discovery: •...
Device login Device login A device can be storage, a host, or a switch. When new devices are introduced into the fabric, they must be powered on and, if a host or storage device, connected to a switch. Switch-to-switch logins (using the E_Port) are handled differently than storage and host logins.
Device login Fabric login process A device performs a fabric login (FLOGI) to determine if a fabric is present. If a fabric is detected then it exchanges service parameters with the fabric controller. A successful FLOGI sends back the 24-bit address for the device in the fabric. The device must issue and successfully complete a FLOGI command before communicating with other devices in the fabric.
High availability of daemon processes Duplicate Port World Wide Name According to Fibre Channel standards, the Port World Wide Name (PWWN) of a device cannot overlap with that of another device, thus having duplicate PWWNs within the same fabric is an illegal configuration.
Page 54
High availability of daemon processes TABLE 1 Daemons that are automatically restarted (Continued) Daemon Description webd Webserver daemon used for WebTools (includes httpd as well). weblinkerd Weblinker daemon provides an HTTP interface to manageability applications for switch management and fabric discovery. Fabric OS Administrator’s Guide 53-1002745-02...
Fabric OS command line interface Although many different software and hardware configurations are tested and supported by Brocade Communications Systems, Inc., documenting all possible configurations and scenarios is beyond the scope of this document. In some cases, earlier releases are highlighted to present considerations for interoperating with them.
Fabric OS command line interface • In a Windows environment enter the following parameters: TABLE 2 Terminal port parameters Parameter Value Bits per second 9600 Databits Parity None Stop bits Flow control None • In a UNIX environment, enter the following string at the prompt: tip /dev/ttyb -9600 If ttyb is already in use, use ttya instead and enter the following string at the prompt: tip /dev/ttya -9600...
Fabric OS command line interface Connecting to Fabric OS using Telnet Use the following procedure to connect to the Fabric OS using Telnet: 1. Connect through a serial port to the switch that is appropriate for your fabric: • If Virtual Fabrics is enabled, log in using an admin account assigned the chassis-role permission.
Fabric OS command line interface The commands in the following table provides help files for the indicated specific topics. TABLE 3 Help topic contents Topic name Help contents description diagHelp Diagnostic help information ficonHelp FICON help information fwHelp Fabric Watch help information iscsiHelp iSCSI help information licenseHelp...
Page 60
Fabric OS command line interface Example cliHistory command output from admin login switch:admin> clihistory CLI history Date & Time Message Thu Sep 27 10:14:41 2012 admin, 10.70.12.101, clihistory Thu Sep 27 10:14:48 2012 admin, 10.70.12.101, clihistory --show switch:admin> cliHistory --show Using the “--show”...
Password modification Notes: • SSH login CLI logs are not recorded in the command line history. • The CLI command log will be collected as part of any “supportsave” operation. The command long record of such an operation will be the equivalent of running “cliHistory --showall”.
The switch Ethernet interface Changing the default account passwords at login Use the following procedure to change the default account passwords: 1. Connect to the switch and log in using the default administrative account. 2. At each of the “Enter new password” prompts, either enter a new password or skip the prompt. To skip a single prompt, press Enter.
The switch Ethernet interface NOTE When you change the Ethernet interface settings, open connections such as SSH or Telnet may be dropped. Reconnect using the new Ethernet IP address information or change the Ethernet settings using a console session through the serial port to maintain your session during the change. You must connect through the serial port to set the Ethernet IP address if the Ethernet network interface is not configured already.
The switch Ethernet interface Host Name: ecp1 Gateway IP Address: 10.1.2.3 IPFC address for virtual fabric ID 123: 11.1.2.3/24 IPFC address for virtual fabric ID 45: 13.1.2.4/20 Slot 7 eth0: 11.1.2.4/24 Gateway: 11.1.2.1 Backplane IP address of CP0 : 10.0.0.5 Backplane IP address of CP1 : 10.0.0.6 IPv6 Autoconfiguration Enabled: Yes Local IPv6 Addresses:...
Page 65
The switch Ethernet interface Setting the static addresses for the Ethernet network interface Use the following procedure to set the Ethernet network interface static addresses: 1. Connect to the switch and log in using an account assigned to the admin role. 2.
The switch Ethernet interface DHCP activation Some Brocade switches have DHCP enabled by default. Fabric OS support for DHCP functionality is only provided for Brocade fixed-port switches. These are listed in the Preface. NOTE The Brocade DCX and Brocade DCX-4S Backbones do not support DHCP. The Fabric OS DHCP client supports the following parameters: •...
Page 67
The switch Ethernet interface 5. You can confirm that the change has been made using the ipAddrShow command. Example of enabling DHCP for IPv4 interactively: switch:admin> ipaddrset Ethernet IP Address [10.1.2.3]: Ethernet Subnetmask [255.255.255.0]: Fibre Channel IP Address [220.220.220.2]: Fibre Channel Subnetmask [255.255.0.0]: Gateway IP Address [10.1.2.1]: DHCP [Off]:on switch:admin>...
The switch Ethernet interface DHCP [On]:off switch:admin> Example of disabling DHCP for IPv4 using a single command: switch:admin> ipaddrset –ipv4 -add -dhcp OFF switch:admin> ipaddrshow SWITCH Ethernet IP Address: 10.20.134.219 Ethernet Subnetmask: 255.255.240.0 Gateway IP Address: 10.20.128.1 DHCP: Off switch:admin> IPv6 autoconfiguration IPv6 can assign multiple IP addresses to each network interface.
Date and time settings Date and time settings Switches maintain the current date and time inside a battery-backed real-time clock (RTC) circuit that receives the date and time from the fabric’s principal switch. Date and time are used for logging events. Switch operation does not depend on the date and time; a switch with an incorrect date and time value functions properly.
Page 70
Date and time settings When you set the time zone for a switch, you can perform the following tasks: • Display all of the time zones supported in the firmware. • Set the time zone based on a country and city combination or based on a time zone ID, such as PST.
Date and time settings Setting the time zone interactively Use the following procedure to set the current time zone to PST using interactive mode: 1. Connect to the switch and log in using an account assigned to the admin role and with the chassis-role permission.
Domain IDs Use the following procedure to synchronize the local time with an external source: 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the tsClockServer command. tsclockserver switch:admin> "ntp1;ntp2" In this syntax, ntp1 is the IP address or DNS name of the first NTP server, which the switch must be able to access.
Domain IDs Displaying the domain IDs Use the following procedure to display device domain IDs: 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the fabricShow command. Example output of fabric information, including the domain ID (D_ID) The principal switch is determined by the arrow ( >...
Switch names Setting the domain ID Use the following procedure to set the domain ID: 1. Connect to the switch and log in on an account assigned to the admin role. 2. Enter the switchDisable command to disable the switch. 3.
Chassis names Chassis names Brocade recommends that you customize the chassis name for each platform. Some system logs identify devices by platform names; if you assign meaningful platform names, logs are more useful. All chassis names supported by Fabric OS v7.0.0 allow 31 characters. Chassis names must begin with an alphabetic character and can include alphabetic and numeric characters, and the underscore ( _ ).
Switch activation and deactivation High availability considerations for fabric names Fabric names locally configured or obtained from a remote switch are saved in the configuration database, and then synchronized to the standby CP on dual-CP-based systems. Upgrade and downgrade considerations for fabric names Fabric names are lost during a firmware downgrade.
Switch and Backbone shutdown Powering off a Brocade switch Use the following procedure to gracefully shut down a Brocade switch. 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the sysShutdown command. 3.
Basic connections Basic connections Before connecting a switch to a fabric that contains switches running different firmware versions, you must first set the same port identification (PID) format on all switches. The presence of different PID formats in a fabric causes fabric segmentation. •...
Port Identifiers (PIDs) and PID binding overview Core PID addressing mode Core PID is the default PID format for Brocade platforms. It uses the entire 24-bit address space of the domain, area ID, and AL_PA to determine an object’s address within the fabric. The Core PID is a 24-bit address built from the following three 8-bit fields: •...
Port Identifiers (PIDs) and PID binding overview • Shared area limitations are removed on 48-port and 64-port blades. • Any port on a 48-port or 64-port blade can support up to 256 NPIV devices (in fixed addressing mode, only 128 NPIV devices are supported in non-VF mode and 64 NPIV devices in VF mode on a 48-port blade).
Port Identifiers (PIDs) and PID binding overview WWN-based PID assignment WWN-based PID assignment is disabled by default. When the feature is enabled, bindings are created dynamically; as new devices log in, they automatically enter the WWN-based PID database. The bindings exist until you explicitly unbind the mappings through the CLI or change to a different addressing mode.
Page 83
Port Identifiers (PIDs) and PID binding overview Use the following procedure to enable automatic PID assignment: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the configure command. 3. At the Fabric Parameters prompt, type y. 4.
Ports Ports Ports provide either a physical or virtual network connection point for a device. Brocade devices support a wide variety of ports. Port Types The following is a list of port types that may be part of a Brocade device: •...
Ports The different blades that can be inserted into a chassis are described as follows: • Control processor blades (CPs) contain communication ports for system management, and are used for low-level, platform-wide tasks. • Core blades are used for intra-chassis switching as well as interconnecting two Backbones. •...
Ports Upgrade and Downgrade considerations For an upgrade, unless both CP8 external Ethernet ports are upgraded and rebooted, the bonding feature will not be enabled. On a downgrade, the first physical port named eth0 has to be connected for the device to initialize correctly; the bonding feature will not be available. Supported devices This feature is available on a CP8 blade when it is installed on a Brocade DCX, Brocade DCX-4S, Brocade DCX 8510-8 or Brocade DCX 8510-4.
Ports Port identification by slot and port number The port number is a number assigned to an external port to give it a unique identifier in a switch. To select a specific port in the Backbones, you must identify both the slot number and the port number using the format slot number/port number.
Ports Configuring a device-switch connection To configure an 8G (and 8G only) connection between a device and a switch, use the portCfgFillWord command. This command provides the following configuration options: • Mode Link Init/Fill Word • Mode 0 IDLE/IDLE • Mode 1 ARBF/ARBF •...
Ports 1. Connect to the switch and log in using an account with admin permissions. 2. Enable the portSwapEnable command to enable the feature. 3. Enter the portDisable command on each of the source and destination ports to be swapped. switch:admin>portdisable 1 ecp:admin>portdisable 1/2 4.
Ports Disabling a port Use the following procedure to disable a port: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the appropriate command based on the current state of the port and on whether it is necessary to specify a slot number: •...
Page 91
Ports • When selecting autonegotiation, you can choose the specific link operating modes that are advertised to the link partner. At least one mode must be advertised in common by both sides of the link. • When forcing the link operating mode, both sides of the link must be forced to the same mode. A link will not work reliably if one side is set to autonegotiate and the other side is set to a forced mode.
Ports Example of setting the port mode to 10 Mbps half-duplex operation To force the link for the eth0 interface from autonegotiation to 10 Mbps half-duplex operation, when entering this command through the serial console port: switch:admin> ifmodeset eth0 Auto-negotiate (yes, y, no, n): [yes] n Force 100 Mbps / Full Duplex (yes, y, no, n): [no] n Force 100 Mbps / Half Duplex (yes, y, no, n): [no] n Force 10 Mbps / Full Duplex (yes, y, no, n): [no] n...
Blade terminology and compatibility Setting port speed for a port octet You can use the portCfgOctetSpeedCombo command to configure the speed for a port octet. Be aware that in a Virtual Fabrics environment, this command applies chassis-wide and not just to the logical switch.
Blade terminology and compatibility TABLE 6 Port blade terminology, numbering, and platform support Supported on: Blade Blade ID DCX family DCX 8510 Ports Definition (slotshow) family FC8-16 8-Gbps port blade supporting 1, 2, 4, and 8 Gbps port speeds. Ports are numbered from 0 through 15 from bottom to top. FC8-32 8-Gbps port blade supporting 1, 2, 4, and 8 Gbps port speeds.
Blade terminology and compatibility TABLE 6 Port blade terminology, numbering, and platform support (Continued) Supported on: Blade Blade ID DCX family DCX 8510 Ports Definition (slotshow) family FCOE10-24 74 ‘No An application blade that provides Converged Enhanced Ethernet to bridge 10-GbE a Fibre Channel and Ethernet SAN.
Enabling and disabling blades Port and application blade compatibility Table 6 on page 94 identifies which port and application blades are supported for each Brocade Backbone. NOTE During power up of a Brocade DCX or DCX-4S Backbone, if an FCOE10-24 is detected first before any other AP blade, all other AP and FC8-64 blades are faulted.
Blade swapping Enabling blades Use the following procedure to enable a blade: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the bladeEnable command with the slot number of the port blade you want to enable. ecp:admin>...
Blade swapping • Blade swapping is not supported when swapping to a different model of blade or a different port count. For example, you cannot swap an FC8-32 blade with an FC8-48 port blade. How blades are swapped The bladeSwap command performs the following operations: 1.
Blade swapping The preparation process also includes any special handling of ports associated with logical switches. For example Figure 3 shows the source blade has ports in a logical switch or logical fabric, then the corresponding destination ports must be included in the associated logical switch or logical fabric of the source ports.
Enabling and disabling switches FIGURE 4 Blade swap with Virtual Fabrics after the swap Swapping blades Use the following procedure to swap blades: 1. Connect to the Backbone and log in using an account with admin permissions. 2. Enter the bladeSwap command. If no errors are encountered, the blade swap will complete successfully.
Power management Using switchCfgPersistentDisable Entering switchCfgPersistentDisable with no arguments disables the switch immediately. Example of using switchCfgPersistentDisable command output without arguments switch:admin> switchCfgPersistentDisable Switch's persistent state set to 'disabled' Using switchCfgPersistentDisable - -disable Using the - -disable argument disables the switch immediately. This is the same as entering switchCfgPersistentDisable without any arguments.
Equipment status The power monitor compares the available power with the power required to determine if there will be enough power to operate. If it is predicted to be less power available than required, the power-off list is processed until there is enough power for operation. By default, the processing begins with slot 1 and proceeds to the last slot in the chassis.
Equipment status 4. Use the switchStatusShow command to further check the status of the switch. Verifying High Availability features (Backbones only) High Availability (HA) features provide maximum reliability and nondisruptive management of key hardware and software modules. Use the following procedure to verify High Availability features for a Backbone: 1.
Track and control switch changes Verifying device connectivity Use the following procedure to verify device connectivity: 1. Connect to the switch and log in using an account with admin permissions. 2. Optional: Enter the switchShow command to verify devices, hosts, and storage are connected. 3.
Track and control switch changes switch:admin> trackchangesset 1 Committing configuration...done. 3. View the log using the commands errDump |more to display a page at a time or errShow to view one line at a time. 2008/10/10-08:13:36, [TRCK-1001], 5, FID 128, INFO, ras007, Successful login by user admin.
Track and control switch changes Flash MarginalPorts 0.00%[0] 0.00%[0] FaultyPorts 0.00%[0] 0.00%[0] MissingSFPs 0.00%[0] 0.00%[0] ErrorPorts 0.00%[0] 0.00%[0] Number of ports: 4 Setting the switch status policy threshold values Use the following procedure to set the switch status policy threshold values: 1.
Audit log configuration Bad Fans contributing to DOWN status: (0..2) [2] Bad Fans contributing to MARGINAL status: (0..2) [1] (output truncated) NOTE On the Brocade Backbones, the command output includes parameters related to CP blades. Audit log configuration When managing SANs you may want to audit certain classes of events to ensure that you can view and generate an audit log for what is happening on a switch, particularly for security-related event changes.
Audit log configuration NOTE Only the active CP can generate audit messages because event classes being audited occur only on the active CP. Audit messages cannot originate from other blades in a Backbone. Switch names are logged for switch components and Backbone names for Backbone components. For example, a Backbone name may be FWDL or RAS and a switch component name may be zone, name server, or SNMP.
Duplicate PWWN handling during device login 4. Enter the auditCfg --show command to view the filter configuration and confirm that the correct event classes are being audited, and the correct filter state appears (enabled or disabled). switch:admin> auditcfg --show Audit filter is enabled. 2-SECURITY 4-FIRMWARE 5.
Duplicate PWWN handling during device login TABLE 9 Duplicate PWWN behavior: Second login overrides first login Input port First port login is F_Port First port login is NPIV port FLOGI received New login forces an explicit logout of original New login forces an explicit logout of original login on the previous F_Port.
Routing overview Paths and route selection Paths are possible ways to get from one switch to another. Each inter-switch link (ISL) has a metric cost based on bandwidth. The cumulative cost is based on the sum of all costs of all traversed ISLs. Route selection is the path that is chosen.
Routing overview FSPF makes minimal use of the ISL bandwidth, leaving virtually all of it available for traffic. In a stable fabric, a switch transmits 64 bytes every 20 seconds in each direction. FSPF frames have the highest priority in the fabric. This guarantees that a control frame is not delayed by user data and that FSPF routing decisions occur very quickly during convergence.
Inter-switch links Inter-switch links An inter-switch link (ISL) is a link between two switches, E_Port-to-E_Port. The ports of the two switches automatically come online as E_Ports once the login process finishes successfully. For more information on the login process, refer to Chapter 1, “Understanding Fibre Channel Services”.
Inter-switch links Buffer credits In order to prevent the dropping of frames in the fabric, a device can never send frames without the receiving device being able to receive them, so an end-to-end flow control is used on the switch. Flow control in Fibre Channel uses buffer-to-buffer credits, which are distributed by the switch.
Gateway links Gateway links A gateway merges SANs into a single fabric by establishing point-to-point E_Port connectivity between two Fibre Channel switches that are separated by a network with a protocol such as IP or SONET. Except for link initialization, gateways are transparent to switches; the gateway simply provides E_Port connectivity from one switch to another.
Routing policies Configuring a link through a gateway 1. Connect to the switch at one end of the gateway and log in using an account assigned to the admin role. 2. Enter the portCfgIISLMode command. 3. Repeat steps 1 and 2 for any additional ports that are connected to the gateway. 4.
Routing policies Displaying the current routing policy 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aptPolicy command with no parameters. The current policy is displayed, followed by the supported policies for the switch. Example of the output from the aptPolicy command In the following example, the current policy is exchange-based routing (3) with the additional AP dedicated link policy.
Routing policies Device-based routing Device-based routing optimizes routing path selection and utilization based on the Source ID (SID) and Destination ID (DID) of the path source and destination ports. As a result, every distinct flow in the fabric can take a different path through the fabric. Effectively, device based routing works the same as exchange-based routing but does not use the OXID field.
Page 121
Routing policies CAUTION Setting the routing policy is disruptive to the fabric because it requires that you disable the switch where the routing policy is being changed. Setting the routing policy Use the following procedure to set the routing policy: 1.
Route selection Route selection Selection of specific routes can be dynamic, so that the router can constantly adjust to changing network conditions; or it may be static, so that data packets always follow a predetermined path. Dynamic Load Sharing The exchange-based routing policy depends on the Fabric OS Dynamic Load Sharing (DLS) feature for dynamic routing path selection.
Frame order delivery Frame order delivery The order in which frames are delivered is maintained within a switch and determined by the routing policy in effect. The frame delivery behaviors for each routing policy are: • Port-based routing All frames received on an incoming port destined for a destination domain are guaranteed to exit the switch in the same order in which they were received.
Frame order delivery Using Frame Viewer to understand why frames are dropped When a frame is unable to reach its destination due to timeout, it is discarded. You can use Frame Viewer to find out which flows contained the dropped frames, which in turn can help you determine which applications might be impacted.
Lossless Dynamic Load Sharing on ports The -txport and -rxport options accept the arguments “-1” (for fixed-port switches) or “-1/-1” (for modular switches). These stand for “any back-end port.”. Using this notation you can select specifically those discarded frames that have a back-end port in the TX port or RX port field. NOTE Individual back-end ports cannot be specified, only the quality of being a back-end port can be specified.
Lossless Dynamic Load Sharing on ports You can disable or enable IOD when Lossless DLS is enabled. You can also choose between exchange- or port-based policies with Lossless DLS. Events that cause a rebalance include the following: • Adding an E_Port •...
Lossless Dynamic Load Sharing on ports ICL limitations If ICL ports are connected during a core blade removal, it is equivalent to removing external E_Ports which may cause I/O disruption on the ICL ports that have been removed. If ICL ports are connected during a core blade insertion, it is equivalent to adding external E_Ports which may cause I/O disruption due to reroutes.
Enabling forward error correction (FEC) To avoid this behavior, it is recommended to define your logical switches as follows: • Define logical switches that require Lossless DLS at the blade boundary. • Define logical switches that require Lossless DLS only using supported blades. For example, do not use blades that support IOD, but do not support Lossless DLS.
Page 129
Enabling forward error correction (FEC) Use the following procedure to enable and disable FEC: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the portCfgFec command, specifying the port or range of ports on which FEC is to be enabled.
Frame Redirection Frame Redirection Frame Redirection provides a means to redirect traffic flow between a host and a target that use virtualization and encryption applications, such as the Brocade SAS blade and Brocade Data Migration Manager (DMM), so that those applications can perform without having to reconfigure the host and target.
Frame Redirection Example of creating a frame redirect zone The following example creates a redirect zone, given a host (10:10:10:10:10:10:10:10), target (20:20:20:20:20:20:20:20), virtual initiator (30:30:30:30:30:30:30:30), and virtual target (40:40:40:40:40:40:40:40): switch:admin>zone --rdcreate 10:10:10:10:10:10:10:10 20:20:20:20:20:20:20:20 \ 30:30:30:30:30:30:30:30 40:40:40:40:40:40:40:40 restartable noFCR Deleting a frame redirect zone Use the following procedure to delete a frame redirect zone: 1.
Page 132
Frame Redirection Fabric OS Administrator’s Guide 53-1002745-02...
User accounts overview Fabric OS provides four options for authenticating users: remote RADIUS service, remote LDAP service, remote TACACS+ service, and the local-switch user database. All options allow users to be managed centrally by means of the following methods: • Remote RADIUS service: Users are managed in a remote RADIUS server.
User accounts overview Admin Domain considerations Legacy users with no Admin Domain specified and whose current role is admin will have access to AD0 through AD255 (physical fabric admin); otherwise, they will have access to AD0 only. If some Admin Domains have been defined for the user and all of them are inactive, the user will not be allowed to log in to any switch in the fabric.
User accounts overview The management channel The management channel is the communication established between the management workstation and the switch. Table 14 shows the number of simultaneous login sessions allowed for each role when authenticated locally. The roles are displayed in alphabetic order, which does not reflect their importance.
Local database user accounts The assigned permissions can be no higher than the admin role permission assigned to the class. The admin role permission for the Security class is Observe/Modify. Therefore, the Observe permission is valid. The roleConfig --show command is available to view the permissions assigned to a user-defined role.
Local database user accounts Default accounts Table 15 lists the predefined accounts offered by Fabric OS that are available in the local-switch user database. The password for all default accounts should be changed during the initial installation and configuration of each switch. TABLE 15 Default local user accounts Account name...
Local database user accounts 3. In response to the prompt, enter a password for the account. The password is not displayed when you enter it on the command line. Deleting an account This procedure can be performed on local user accounts. 1.
Local user account database distribution Changing the password for a different account 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the passwd command specifying the name of the account for which the password is being changed.
Password policies Rejecting distributed user databases on the local switch 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fddCfg --localreject PWD command. Password policies The password policies described in this section apply to the local-switch user database only. Configured password policies (and all user account attribute and password state information) are synchronized across CPs and remain unchanged after an HA failover.
Password policies • Punctuation Specifies the minimum number of punctuation characters that must appear in the password. All printable, non-alphanumeric punctuation characters except the colon ( : ) are allowed. The default value is zero. The maximum value must be less than or equal to the MinLength value. •...
Password policies Password expiration policy The password expiration policy forces the expiration of a password after a configurable period of time. The expiration policy can be enforced across all user accounts or on specified users only. A warning that password expiration is approaching is displayed when the user logs in. When a password expires, the user must change the password to complete the authentication process and open a user session.
Page 144
Password policies A failed login attempt counter is maintained for each user on each switch instance. The counters for all user accounts are reset to zero when the account lockout policy is enabled. The counter for an individual account is reset to zero when the account is unlocked after a lockout duration period expires, or when the account user logs in successfully.
The boot PROM password Denial of service implications The account lockout mechanism may be used to create a denial of service condition when a user repeatedly attempts to log in to an account by using an incorrect password. Selected privileged accounts are exempted from the account lockout policy to prevent users from being locked out from a denial of service attack.
The boot PROM password 4. Enter 2. • If no password was previously set, the following message is displayed: Recovery password is NOT set. Please set it now. • If a password was previously set, the following messages is displayed: Send the following string to Customer Support for password recovery: afHTpyLsDo1Pz0Pk5GzhIw== Enter the supplied recovery password.
The boot PROM password • If a password was previously set, the following messages are displayed: Send the following string to Customer Support for password recovery: afHTpyLsDo1Pz0Pk5GzhIw== Enter the supplied recovery password. Recovery Password: 6. Enter the recovery password (string). The recovery string must be between 8 and 40 alphanumeric characters.
The boot PROM password The following options are available: Option Description Start system. Continues the system boot process. Recovery password. Lets you set the recovery string and the boot PROM password. Enter command shell. Provides access to boot parameters. 4. Enter 3. 5.
Remote authentication The passwd command applies only to the boot PROM password when it is entered from the boot interface. 8. Enter the boot PROM password at the prompt, and then re-enter it when prompted. The password must be eight alphanumeric characters (any additional characters are not recorded).
Remote authentication The supported management access channels that integrate with RADIUS, LDAP, and TACACS+ include serial port, Telnet, SSH, Web Tools, and API. All these access channels require the switch IP address or name to connect. RADIUS, LDAP, and TACACS+ servers accept both IPv4 and IPv6 address formats.
Remote authentication Supported LDAP options Table 16 summarizes the various LDAP options and Brocade support for each. TABLE 16 LDAP options Protocol Description Channel type Default port Brocade supported? LDAPv3 LDAP over TCP Unsecured ldap:// LDAPv3 with TLS LDAPv3 over TLS Secured ldap:// extension...
Remote authentication TABLE 17 Authentication configuration options (Continued) aaaConfig options Description Equivalent setting in Fabric OS v5.1.0 and earlier --radius --switchdb --authspec “ldap; local” Authenticates management connections against any LDAP databases first. If LDAP fails for any reason, it then authenticates against the local user database.
Remote authentication RADIUS, LDAP, and TACACS+ support all the defined RBAC roles described in Table 12 page 134. Users must enter their assigned RADIUS, LDAP, or TACACS+ account name and password when logging in to a switch that has been configured with remote authentication. After the remote authentication (RADIUS, LDAP, or TACACS+) server authenticates a user, it responds with the assigned switch role in a Brocade Vendor-Specific Attribute (VSA).
Remote authentication Fabric OS users on the RADIUS server All existing Fabric OS mechanisms for managing local-switch user accounts and passwords remain functional when the switch is configured to use RADIUS. Changes made to the local switch database do not propagate to the RADIUS server, nor do the changes affect any account on the RADIUS server.
Page 155
Remote authentication Brocade-AVPairs2 = "LFRoleList=admin:2,4-8,70,80,128;ChassisRole=admin", Brocade-Passwd-ExpiryDate = "11/10/2011", Brocade-Passwd-WarnPeriod = "30" RADIUS configuration with Admin Domains or Virtual Fabrics When configuring users with Admin Domains or Virtual Fabrics, you must also include the Admin Domain or Virtual Fabric member list. This section describes the way that you configure attribute types for this configuration.
Remote authentication For example, on a Linux FreeRADIUS Server, the user (user-za) with the following settings takes the “zoneAdmin” permissions, with AD member list: 1, 2, 4, 5, 6, 7, 8, 9, 12; the Home Admin Domain will be 1. user-za Auth-Type := Local, User-Password == "password"...
Page 157
Remote authentication Configuring RADIUS service on Linux consists of the following tasks: • Adding the Brocade attributes to the server • Creating the user • Enabling clients Adding the Brocade attributes to the server 1. Create and save the file $PREFIX/etc/raddb/dictionary.brocade with the following information: # dictionary.brocade VENDOR Brocade 1588...
Page 158
Remote authentication swadmin Auth-Type := System Brocade-Auth-Role = "admin", Brocade-AVPairs1 = "HomeLF=70", Brocade-AVPairs2 = "LFRoleList=admin:2,4-8,70,80,128", Brocade-AVPairs3 = "ChassisRole=switchadmin", Brocade-Passwd-ExpiryDate = "11/10/2008", Brocade-Passwd-WarnPeriod = "30" When you use network information service (NIS) for authentication, the only way to enable authentication with the password file is to force the Brocade switch to authenticate using password authentication protocol (PAP);...
Page 159
Remote authentication If CHAP authentication is required, then Windows must be configured to store passwords with reversible encryption. Reverse password encryption is not the default behavior; it must be enabled. NOTE If a user is configured prior to enabling reverse password encryption, then the user’s password is stored and cannot utilize CHAP.
Page 160
Remote authentication e. After returning to the Internet Authentication Service window, add additional policies for all Brocade login types for which you want to use the RADIUS server. After this is done, you can configure the switch. NOTE Windows 2008 RADIUS (NPS) support is also available. RSA RADIUS server Traditional password-based authentication methods are based on one-factor authentication, where you confirm your identity using a memorized password.
Remote authentication Add Brocade-VSA macro and define the attributes as follows: • vid (Vendor-ID): 1588 • type1 (Vendor-Type): 1 • len1 (Vendor-Length): >=2 ####################################################################### # brocade.dct -- Brocade Dictionary # (See readme.dct for more details on the format of this file) ####################################################################### # Use the Radius specification attributes in lieu of the Brocade one: @radius.dct...
Remote authentication ####################################################################### # dictiona.dcm ####################################################################### # Generic Radius @radius.dct # Specific Implementations (vendor specific) @3comsw.dct @aat.dct @acc.dct @accessbd.dct @agere.dct @agns.dct @airespace.dct @alcatel.dct @altiga.dct @annex.dct @aptis.dct @ascend.dct @ascndvsa.dct @axc.dct @bandwagn.dct @brocade.dct <------- FIGURE 12 Example of the dictiona.dcm file d. When selecting items from the Add Return List Attribute, select Brocade-Auth-Role and type the string Admin.
Page 163
Remote authentication • LDAP authentication is used on the local switch only and not for the entire fabric. • You can use the User-Principal-Name and not the Common-Name for AD LDAP authentication. To provide backward compatibility, authentication based on the Common Name is still supported for Active Directory LDAP 2000 and 2003.
Remote authentication 4. Associate the user to the group by adding the user to the group. For instructions on how to create a user refer to www.microsoft.com or Microsoft documentation to create a user in your Active Directory. 5. Add the user’s Administrative Domains or Virtual Fabrics to the CN_list by either editing the adminDescription value or adding the brcdAdVfData attribute to the existing Active Directory schema.
Remote authentication 3. Right click on select Properties. Click the Attribute Editor tab. 4. Double-click the adminDescription attribute. The String Attribute Editor dialog box opens. 5. Perform the appropriate action based on whether you are using Administrative Domains or Virtual Fabrics: •...
Page 166
Remote authentication Two operational modes exist in LDAP authentication: FIPS mode and non-FIPS mode. This section discusses LDAP authentication in non-FIPS mode. For information on LDAP in FIPS mode, refer to Chapter 7, “Configuring Security Policies”. The following restrictions exist when using OpenLDAP in non-FIPS mode: •...
Page 167
Remote authentication include /usr/local/etc/openldap/schema/cosine.schema include /usr/local/etc/openldap/schema/local.schema ############################################### TLSCACertificateFile /root/sachin/ldapcert/cacert.pem TLSCertificateFile /root/sachin/ldapcert/serverCert.pem TLSCertificateKeyFile /root/sachin/ldapcert/serverKey.pem TLSVerifyClient never pidfile /usr/local/var/run/slapd.pid argsfile /usr/local/var/run/slapd.args database suffix "dc=mybrocade,dc=com" rootdn "cn=Manager,dc=mybrocade,dc=com" rootpw {SSHA}HL8uT5hPaWyIdcP6yAheMT8n0GoWubr3 # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended.
Page 168
Remote authentication Assigning a user to a group Before you can assign a user to a group, the memberOf overlay must be added to the slapd.conf file. Refer to “Enabling group membership” on page 166 for details. To create a group and assign a member: 1.
Page 169
Remote authentication Example to add a group member 1. Create or edit a .ldif file with an entry similar to the following. ##########Adding an attr value dn: cn=admin,ou=groups,dc=mybrocade,dc=com changetype: modify add: member member: cn=test1,cn=Users,dc=mybrocade,dc=com 2. Enter the following ldapmodify command, where test1.ldif is the name of the file you edited in step >...
Page 170
Remote authentication DESC 'Brocade specific data for LDAP authentication' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} ) objectclass ( 1.3.6.1.4.1.8412.110 NAME 'user' DESC 'Brocade switch specific person' SUP top AUXILIARY MAY ( brcdAdVfData $ description ) ) 2. Include the schema file in the slapd.conf file. The following example slapd.conf line assumes that local.schema contains the attribute definition provided in step...
Remote authentication objectClass: uidObject cn: Sachin sn: Mishra description: First user brcdAdVfData: HomeLF=30;LFRoleList=admin:1-128;ChassisRole=admin userPassword: pass uid: mishras@mybrocade.com The following command adds the user to the LDAP directory. > ldapadd -D cn=Sachin,dc=mybrocade,dc=com -x -w secret -f test4.ldif TACACS+ service FabricOS can authenticate users with a remote server using the Terminal Access Controller Access-Control System Plus (TACACS+) protocol.
Remote authentication Configuring the TACACS+ server on LINUX FabricOS software supports TACACS+ authentication on a LINUX server running the Open Source TACACS+ LINUX package v4.0.4 from Cisco. To install and configure this software, perform the following steps. 1. Download the TACACS+ software from http://www.cisco.com and install it. Refer to the Cisco documentation for installation instructions.
Page 173
Remote authentication Configuring Admin Domain lists If your network uses Admin Domains, you should create Admin Domain lists for each user to identify the Admin Domains to which the user has access. Assign the following key-value pairs to the brcd-AV--Pair1 and, optionally, brcd-AV-Pair2 attributes to grant the account access to the Admin Domains: •...
Remote authentication Configuring the password expiration date FabricOS lets you configure a password expiration date for each user account and to configure a warning period for notifying the user that the account password is about to expire. To configure these values, set the following attributes: •...
Page 175
Remote authentication Adding an authentication server to the switch configuration 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aaaConfig --add command. At least one authentication server must be configured before you can enable the RADIUS, LDAP, or TACACS+ service.
Remote authentication Displaying the current authentication configuration 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aaaConfig --show command. If a configuration exists, its parameters are displayed. If the RADIUS, LDAP, or TACACS+ service is not configured, only the parameter heading line is displayed.
Secure Copy TABLE 21 Secure protocol support (Continued) Protocol Description Secure Shell (SSH) is a network protocol that allows data to be exchanged over a secure channel between two computers. Encryption provides confidentiality and integrity of data. SSH uses public-key cryptography to authenticate the remote computer and allow the remote computer to authenticate the user, if necessary.
Secure Shell protocol Setting up SCP for configuration uploads and downloads Use the following procedure to configure SCP for configuration uploads and downloads. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the configure command. 3.
Secure Shell protocol SSH public key authentication OpenSSH public key authentication provides password-less logins, known as SSH authentication, that uses public and private key pairs for incoming and outgoing authentication. This feature allows only one allowed-user to be configured to utilize outgoing OpenSSH public key authentication.Any admin user can perform incoming Open SSH public key authentication.
Page 181
Secure Shell protocol Enter login name:auser Password: Public key is imported successfully. 4. Test the setup by logging in to the switch from a remote device, or by running a command remotely using SSH. Configuring outgoing SSH authentication After the allowed-user is configured, the remaining setup steps must be completed by the allowed-user.
Secure Sockets Layer protocol Deleting public keys on the switch Use the following procedure to delete public keys from the switch. 1. Connect to the switch and log in using an account with admin permissions. 2. Use the sshUtil delpubkeys command to delete public keys. You will be prompted to enter the name of the user whose the public keys you want to delete.
Secure Sockets Layer protocol You should upgrade to the Java 1.6.0 plug-in on your management workstation. To find the Java version that is currently running, open the Java console and look at the first line of the window. For more details on levels of browser and Java support, refer to the Web Tools Administrator’s Guide.
Page 184
Secure Sockets Layer protocol 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the secCertUtil genkey command to generate a public/private key pair. The system reports that this process will disable secure protocols, delete any existing CSR, and delete any existing certificates.
Secure Sockets Layer protocol Obtaining certificates Once you have generated a CSR, you will need to follow the instructions on the website of the certificate issuing authority that you want to use; and then obtain the certificate. Fabric OS and HTTPS support the following types of files from the Certificate Authority(CA): •...
Secure Sockets Layer protocol 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the secCertUtil import command. 3. Select a protocol, enter the IP address of the host on which the switch certificate is saved, and enter your login name and password.
Secure Sockets Layer protocol 4. Click the Intermediate or Trusted Root tab and scroll the list to see if the root certificate is listed. Take the appropriate following action based on whether you find the certificate: • If the certificate is listed, you do not need to install it. You can skip the rest of this procedure.
Simple Network Management Protocol Issuer: CN=Brocade, OU=Software, O=Brocade Communications, L=San Jose, ST=California, C=US Serial number: 0 Valid from: Thu Jan 15 16:27:03 PST 2007 until: Sat Feb 14 16:27:03 PST 2007 Certificate fingerprints: MD5: 71:E9:27:44:01:30:48:CC:09:4D:11:80:9D:DE:A5:E3 SHA1: 06:46:C5:A5:C8:6C:93:9C:FE:6A:C0:EC:66:E9:51:C2:DB:E6:4F:A1 Trust this certificate? [no]: Certificate was added to keystore In the example, changeit is the default password and RootCert is an example root certificate name.
Simple Network Management Protocol • SW-EXTTRAP Includes the swSsn (Software Serial Number) as a part of Brocade SW traps. For information on Brocade MIBs, refer to the Fabric OS MIB Reference. SNMP and Virtual Fabrics When an SNMPv3 request arrives with a particular user name, it executes in the home Virtual Fabric.
Telnet protocol SNMP security levels Use the snmpConfig --set seclevel command to set the security level. For more information about using the Brocade SNMP agent, refer to the Fabric OS MIB Reference. SNMP configuration Use the snmpConfig --set command to change either the SNMPv3 or SNMPv1 configuration. You can also change access control, MIB capability, and system group.
Telnet protocol ATTENTION The rule number assigned must precede the default rule number for this protocol. For example, in the defined policy, the Telnet rule number is 2. Therefore, to effectively block Telnet, the rule number to assign must be 1. If you choose not to use 1, you must delete the Telnet rule number 2 after adding this rule.
Listener applications Refer to “Deleting a rule from an IP Filter policy” on page 223 for more information on deleting IP filter rules. 3. To permanently delete the policy, type the ipfilter --save command. ATTENTION If you deleted the rule to permit Telnet, you must add a rule to permit Telnet. Listener applications Brocade switches block Linux subsystem listener applications that are not used to implement supported features and capabilities.
Ports and applications used by switches TABLE 26 Access defaults (Continued) Access default Devices All devices can access the management server. Any device can connect to any FC port in the fabric. Switch access Any switch can join the fabric. All switches in the fabric can be accessed through a serial port.
Page 194
Ports and applications used by switches Fabric OS Administrator’s Guide 53-1002745-02...
ACL policy management Policies with the same state are grouped together in a Policy Set. Each switch has the following two sets: • Active policy set, which contains ACL policies being enforced by the switch. • Defined policy set, which contains a copy of all ACL policies on the switch. When a policy is activated, the defined policy either replaces the policy with the same name in the active set or becomes a new active policy.
ACL policy management Displaying ACL policies You can view the active and defined policy sets at any time. Additionally, in a defined policy set, policies created in the same login session also appear but these policies are automatically deleted if the you log out without saving them. 1.
ACL policy management Example of deleting an ACL policy switch:admin> secpolicydelete "DCC_POLICY_010" About to delete policy Finance_Policy. Are you sure (yes, y, no, n):[no] y Finance_Policy has been deleted. Adding a member to an existing ACL policy As soon as a policy has been activated, the aspect of the fabric managed by that policy is enforced. 1.
FCS policies Example of aborting unsaved changes switch:admin> secpolicyabort Unsaved data has been aborted. All changes since the last time the secPolicySave or secPolicyActivate commands were entered are aborted. FCS policies Fabric configuration server (FCS) policy in base Fabric OS may be performed on a local switch basis and may be performed on any switch in the fabric.
FCS policies Table 30 shows the commands for switch operations for Primary FCS enforcement. TABLE 30 FCS switch operations Allowed on FCS switches Allowed on all switches secPolicyAdd (Allowed on all switches for SCC and DCC secPolicyShow policies as long as it is not fabric-wide) secPolicyCreate (Allowed on all switches for SCC and fddCfg localaccept or fddCfg --localreject...
FCS policies Creating an FCS policy 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands. 2. Enter the secPolicyCreate “FCS_POLICY” command. Example of creating an FCS policy The following example creates an FCS policy that allows a switch with domain ID 2 to become a primary FCS and domain ID 4 to become a backup FCS: switch:admin>...
FCS policies FCS policy distribution The FCS policy can be automatically distributed using the fddCfg --fabwideset command or it can be manually distributed to the switches using the distribute -p command. Each switch that receives the FCS policy must be configured to receive the policy. To configure the switch to accept distribution of the FCS policy, refer to “Database distribution settings”...
Device Connection Control policies Device Connection Control policies Multiple Device Connection Control (DCC) policies can be used to restrict which device ports can connect to which switch ports. The devices can be initiators, targets, or intermediate devices such as SCSI routers and loop hubs. By default, all device ports are allowed to connect to all switch ports;...
Device Connection Control policies Creating a DCC policy DCC policies must follow the naming convention “DCC_POLICY_nnn,” where nnn represents a unique string. The maximum length is 30 characters, including the prefix DCC_POLICY_. Device ports must be specified by port WWN. Switch ports can be identified by the switch WWN, domain ID, or switch name followed by the port or area number.
Device Connection Control policies Deleting a DCC policy 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands. 2. Enter the secPolicyDelete command. Example of deleting stale DCC policies switch:admin>...
SCC Policies Table 34 shows the behavior of a DCC policy created manually with the physical PWWN of a device. The configurations shown in this table are the recommended configurations when an FA-PWWN is logged into the switch. TABLE 34 DCC policy behavior when created manually with PWWN Configuration WWN seen on...
Authentication policy for fabric elements Creating an SCC policy 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands. 2. Enter the secPolicyCreate “SCC_POLICY” command. 3.
Authentication policy for fabric elements Key database on switch Key database on switch Local secret B Local secret A Peer secret A Peer secret B Switch A Switch B FIGURE 13 DH-CHAP authentication If you use DH-CHAP authentication, then a secret key pair must be installed only in connected fabric elements.
Page 209
Authentication policy for fabric elements Virtual Fabrics considerations The switch authentication policy applies to all E_Ports in a logical switch. This includes ISLs and extended ISLs. Authentication of extended ISLs between two base switches is considered peer-chassis authentication. Authentication between two physical entities is required, so the extended ISL which connects the two chassis needs to be authenticated.
Authentication policy for fabric elements Re-authenticating E_Ports Use the authUtil --authinit command to re-initiate the authentication on selected ports. It provides flexibility to initiate authentication for specified E_Ports, a set of E_Ports, or all E_Ports on the switch. This command does not work on loop, NPIV and FICON devices, or on ports configured for in-flight encryption.
Authentication policy for fabric elements and CT frames, except the AUTH_NEGOTIATE ELS frame, are blocked by the switch. During this time, the Fibre Channel driver rejects all other ELS frames. The F_Port does not form until the AUTH_NEGOTIATE is completed. It is the HBA's responsibility to send an Authentication Negotiation ELS frame after receiving the FLOGI accept frame with the FC-SP bit set.
Authentication policy for fabric elements Authentication protocols Use the authUtil command to perform the following tasks: • Display the current authentication parameters. • Select the authentication protocol used between switches. • Select the DH (Diffie-Hellman) group for a switch. Run the authUtil command on the switch you want to view or change. Below are the different options to specify which DH group you want to use.
Authentication policy for fabric elements Secret key pairs for DH-CHAP When you configure the switches at both ends of a link to use DH-CHAP for authentication, you must also define a secret key pair—one for each end of the link. Use the secAuthSecret command to perform the following tasks: •...
Page 214
Authentication policy for fabric elements Setting a secret key pair 1. Log in to the switch using an account with admin permissions, or an account with OM permissions for the Authentication RBAC class of commands. 2. Enter the secAuthSecret --set command. The command enters interactive mode.
Authentication policy for fabric elements FCAP configuration overview Beginning with Fabric OS release 7.0.0, you must configure the switch to use third-party certificates for authentication with the peer switch. To perform authentication with FCAP protocol with certificates issued from third party, the user has to perform following steps: 1.
Page 216
Authentication policy for fabric elements Exporting the CSR for FCAP You will need to export the CSR file created in “Generating the key and CSR for FCAP” section and send to a Certificate Authority (CA). The CA will in turn provide two files as outlined in “FCAP configuration overview”...
IP Filter policy Starting FCAP authentication 1. Log in to the switch using an account with admin permissions, or an account with OM permissions for the Authentication RBAC class of commands. 2. Enter the authUtil --authinit command to start the authentication using the newly imported certificates.
IP Filter policy Virtual Fabrics considerations: Each logical switch cannot have its own different IP Filter policies. IP Filter policies are treated as a chassis-wide configuration and are common for all the logical switches in the chassis. Creating an IP Filter policy You can create an IP Filter policy specifying any name and using type IPv4 or IPv6.
IP Filter policy 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having the OM permissions for the IPfilter RBAC class of commands. 2. Enter the ipFilter save command. –- Activating an IP Filter policy IP Filter policies are not enforced until they are activated.
IP Filter policy Source address For an IPv4 filter policy, the source address has to be a 32-bit IPv4 address in dot decimal notation. The group prefix has to be a CIDR block prefix representation. For example, 208.130.32.0/24 represents a 24-bit IPv4 prefix starting from the most significant bit. The special prefix 0.0.0.0/0 matches any IPv4 address.
Page 221
IP Filter policy TABLE 37 Supported services (Continued) Service name Port number bootps bootpc tftp http kerberos hostnames sunrpc sftp snmp snmp trap https ssmtp exec login shell uucp biff syslog route timed kerberos4 rpcd securerpcd Protocol TCP and UDP protocols are valid protocol selections. Fabric OS v6.2.0 and later do not support configuration to filter other protocols.
IP Filter policy Traffic type and destination IP The traffic type and destination IP elements allow an IP policy rule to specify filter enforcement for IP forwarding. The INPUT traffic type is the default and restricts rules to manage traffic on IP management interfaces, The FORWARD traffic type allows management of bidirectional traffic between the external management interface and the inband management interface.
IP Filter policy IP Filter policy enforcement An active IP Filter policy is a filter applied to the IP packets through the management interface. IPv4 management traffic passes through the active IPv4 filter policy, and IPv6 management traffic passes through the active IPv6 filter policy. The IP Filter policy applies to the incoming (ingress) management traffic only.
Policy database distribution 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having the OM permissions for the IPfilter RBAC class of commands. 2. Enter the ipFilter transabort command. –- IP Filter policy distribution The IP Filter policy is manually distributed by command.
Policy database distribution • Manually distribute an ACL policy database — Run the distribute command to push the local database of the specified policy type to target switches. “ACL policy distribution to other switches” on page 227. • Fabric-wide consistency policy — Use to ensure that switches in the fabric enforce the same policies.
Policy database distribution TABLE 41 Supported policy databases (Continued) Database type Database identifier (ID) FCS policy database IP Filter policy database IPFILTER Password database SCC policy database Use the chassisDistribute command to distribute IP filter policies. To distribute other security policies, use the distribute command.
Policy database distribution ACL policy distribution to other switches This section explains how to manually distribute local ACL policy databases. The distribute command has the following dependencies: • All target switches must be running Fabric OS v6.2.0 or later. • All target switches must accept the database distribution (see “Database distribution settings”...
Policy database distribution TABLE 42 Fabric-wide consistency policy settings Setting Value When a policy is activated Absent null Database is not automatically distributed to other switches in the fabric. Tolerant database_id All updated and new policies of the type specified (SCC, DCC, FCS, or any combination) are distributed to all Fabric v6.2.0 and later switches in the fabric.
Policy database distribution Notes on joining a switch to the fabric When a switch is joined to a fabric with a tolerant SCC, DCC, or FCS fabric-wide consistency policy, the joining switch must have a matching tolerant SCC, DCC, or FCS fabric-wide consistency policy. If the tolerant SCC, DCC, or FCS fabric-wide consistency policies do not match, the switch can join the fabric, but an error message flags the mismatch.
Management interface security Management interface security You can secure an Ethernet management interface between two Brocade switches or Backbones by implementing IP sec and IKE policies to create a tunnel that protects traffic flows. While the tunnel must have a Brocade switch or Backbone at each end, there may be routers, gateways, and firewalls in between the two ends.
Management interface security FIGURE 14 Protected endpoints configuration A possible drawback of end-to-end security is that various applications that require the ability to inspect or modify a transient packet will fail when end-to-end confidentiality is employed. Various QoS solutions, traffic shaping, and firewalling applications will be unable to determine what type of packet is being transmitted and will be unable to make the decisions that they are supposed to make.
Management interface security FIGURE 16 Endpoint-to-gateway tunnel configuration RoadWarrior configuration In endpoint-to-endpoint security, packets are encrypted and decrypted by the host which produces or consumes the traffic. In the gateway-to-gateway example, a router on the network encrypts and decrypts the packets on behalf of the hosts on a protected network. A combination of the two is referred to as a RoadWarrior configuration where a host on the Internet requires access to a network through a security gateway that is protecting the network.
Management interface security these values in negotiations to create IP sec SAs. You must create an SA prior to creating an SA-proposal. You cannot modify an SA once it is created. Use the IP secConfig --flush manual-sa command to remove all SA entries from the kernel SADB and re-create the SA. For more information on the IP secConfig command, refer to the Fabric OS Command Reference.
Management interface security IP sec traffic selector The traffic selector is a traffic filter that defines and identifies the traffic flow between two systems that have IP sec protection. IP addresses, the direction of traffic flow (inbound, outbound) and the upper layer protocol are used to define a filter for traffic (IP datagrams) that is protected using IP sec.
Management interface security The IP secConfig command does not support manipulating pre-shared keys corresponding to the identity of the IKE peer or group of peers. Use the secCertUtil command to import, delete, or display the pre-shared keys in the local switch database. For more information on this procedure, refer to Chapter 6, “Configuring Protocols”.
Page 237
Management interface security Example of creating an IP sec SA policy This example creates an IP sec SA policy named AH01, which uses AH protection with MD5. You would run this command on each switch; on each side of the tunnel so that both sides have the same IP sec SA policy.
Management interface security 10. Verify traffic is protected. a. Initiate a telnet, SSH, or ping session from the two switches. b. Verify that IP traffic is encapsulated. Monitor IP sec SAs created using IKE for above traffic flow • Use the IP secConfig show manual-sa –a command with the operands specified to -–...
Page 239
Management interface security 6. Import the pre-shared key file using the secCertUtil command. The file name should have a .psk extension. For more information on importing the pre-shared key file, refer to “Installing a switch certificate” on page 185. Configure an IKE policy for the remote peer. switch:admin>...
Page 240
Management interface security • Use the IP secConfig –-show policy ike –a command with the specified operands to display IKE policies. • Use the IP secConfig –-flush manual-sa command with the specified operands to flush the created SAs in the kernel SADB. CAUTION Flushing SAs requires IP sec to be disabled and re-enabled.
Configuration settings If your user account has chassis account permissions, you can use any of the following options when uploading or downloading a configuration file: -fid To upload the specified FID configuration. -all To upload all of the system configuration, including the chassis section and all switch sections for all logical switches.
Page 243
Configuration settings [Active Security policies] [cryptoDev] [FICU SAVED FILES] [Banner] [End] [Switch Configuration End : 0] date = Tue Mar 1 21:28:52 2011 [Switch Configuration Begin : 1] SwitchName = switch_2 Fabric ID = 1 [Boot Parameters] [Configuration] [Bottleneck Configuration] [Zoning] [Defined Security policies] [Active Security policies]...
Configuration file backup Before you upload a configuration file, verify that you can reach the FTP server from the switch. Using a Telnet connection, save a backup copy of the configuration file from a logical switch to a host computer. Secure File Transfer Protocol (SFTP) is now an option when uploading a configuration file.
Configuration file restoration Configuration file restoration When you restore a configuration file, you overwrite the existing configuration with a previously saved backup configuration file. CAUTION Make sure that the configuration file you are downloading is compatible with your switch model. Downloading a configuration file from a different switch model or from a different firmware could cause your switch to fail.
Configuration file restoration -all The number of switches or FIDs defined in the downloaded configuration file must match the number of switches or FIDs currently defined on the switch. The switches must be disabled first. If they are not, the configDownload command will download the configuration for as many switches as possible until a non-disabled switch is found.
Configuration file restoration CAUTION Though the switch itself has advanced error checking, the configdownload feature within Fabric OS was not designed for users to edit, and is limited in its ability. Edited files can become corrupted and this corruption can lead to switch failures. Configuration download without disabling a switch You can download configuration files to a switch while the switch is enabled;...
Page 249
Configuration file restoration Example of configDownload without Admin Domains switch:admin> configdownload Protocol (scp, ftp, local) [ftp]: Server Name or IP Address [host]: 10.1.2.3 User Name [user]: UserFoo Path/Filename [<home dir>/config.txt]: Section (all|chassis|FID# [all]): all *** CAUTION *** This command is used to download a backed-up configuration for a specific switch.
Configurations across a fabric Activating configDownload: Switch is disabled configDownload complete: Only zoning parameters are downloaded to ad5. Example of a non-interactive download of all configurations (chassis and switches) configdownload -a -ftp 10.1.2.3,UserFoo,/pub/configurations/config.txt,password Configurations across a fabric To save time when configuring fabric parameters and software features, you can save a configuration file from one switch and download it to other switches of the same model type.
Configuration management for Virtual Fabrics Uploading a configuration file from a switch with Virtual Fabrics enabled The configUpload command with the -vf option specifies that configuration upload will upload the Virtual Fabrics configuration instead of the non-Virtual Fabrics configuration information. You must specify a file name with the configUpload -vf command.
Configuration management for Virtual Fabrics Wait for the configuration file to download on to the switch. You may need to reconnect to the switch. 4. Enter the configDownload command. 5. Respond to the prompts. Wait for the configuration file to download to the switch. 6.
Brocade configuration form Brocade configuration form Use the form in Table 48 as a hard copy reference for your configuration information. In the hardware reference manuals for the Brocade DCX and DCX-4S Backbones, there is a guide for FC port-setting. TABLE 48 Brocade configuration and connection form Brocade configuration settings...
Page 254
Brocade configuration form Fabric OS Administrator’s Guide 53-1002745-02...
Chapter Installing and Maintaining Firmware In this chapter • Firmware download process overview ......255 •...
Page 256
Firmware download process overview You can download Fabric OS to a Backbone, which is a chassis; and to a nonchassis-based system, also referred to as a fixed-port switch. The difference in the download process is that Backbones have two CPs and fixed-port switches have one CP. Use the firmwareDownload command to download the firmware from either an FTP or SSH server by using FTP, SFTP, or SCP to the switch.
Firmware download process overview Upgrading and downgrading firmware Upgrading means installing a newer version of firmware. Downgrading means installing an older version of firmware. In most cases, you will be upgrading firmware; that is, installing a newer firmware version than the one you are currently running.
Preparing for a firmware download TABLE 49 Backbone HA sync states Active CP Fabric OS Standby CP Fabric OS HA sync state Remedy version version v6.2.0 v6.2.0 inSync v6.2.x v6.3.0 inSync v6.3.0 v6.2.x If Ethernet Switch Service Run firmwareDownload -s on the is enabled, no sync.
Preparing for a firmware download 5. Connect to the switch and log in using an account with admin permissions. Enter the supportSave command to retrieve all current core files prior to executing the firmware download. This information helps to troubleshoot the firmware download process if a problem is encountered.
Firmware download on switches Firmware download on switches Brocade fixed-port switches maintain primary and secondary partitions for firmware. The firmwareDownload command defaults to an autocommit option that automatically copies the firmware from one partition to the other. NOTE This section only applies when upgrading from Fabric OS v7.0.x to v7.1.0, downgrading from v7.1.0 to v7.0.x, or going from v7.1.x to v7.1.x.
Page 261
Firmware download on switches Upgrading firmware for Brocade fixed-port switches 1. Take the following appropriate action based on what service you are using: • If you are using FTP, SFTP, or SCP, verify that the FTP or SSH server is running on the host server and that you have a valid user ID and password on that server.
Firmware download on a Backbone Firmware download on a Backbone ATTENTION To successfully download firmware, you must have an active Ethernet connection on each CP. You can download firmware to a Backbone without disrupting the overall fabric if the two CP blades are installed and fully synchronized.
Page 263
Firmware download on a Backbone Upgrading firmware on Backbones (including blades) There is only one chassis management IP address for the Brocade Backbones. NOTE By default, the firmwareDownload command automatically upgrades both the active and the standby CPs and all co-CPs on the CP blades in the Brocade Backbones. It automatically upgrades all AP blades in the Brocade Backbones using auto-leveling.
Page 264
Firmware download on a Backbone If an AP blade is present: At the point of the failover, an autoleveling process is activated. Autoleveling is triggered when the active CP detects a blade that contains a different version of the firmware, regardless of which version is older. Autoleveling downloads firmware to the AP blade, swaps partitions, reboots the blade, and copies the new firmware from the primary partition to the secondary partition.
Firmware download from a USB device Slot 7 (CP1, active): Firmware has been downloaded to the secondary partition of the switch. [5]: Mon Mar 22 04:37:24 2010 Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes. [6]: Mon Mar 22 04:41:59 2010 Slot 7 (CP1, standby): The commit operation has completed successfully.
FIPS support Downloading from the USB device using the relative path 1. Log in to the switch using an account assigned to the admin role. 2. Enter the firmwareDownload -U command. ecp:admin>firmwaredownload –U v7.1.0 Downloading from the USB device using the absolute path 1.
FIPS support NOTE If FIPS mode is enabled, all logins should be handled through SSH or direct serial method, and the transfer protocol should be SCP. Updating the firmware key 1. Log in to the switch as admin. 2. Enter the firmwareKeyUpdate command and respond to the prompts. The firmwareDownload command The ipublic key file needs to be packaged, installed, and run on your switch before you download a signed firmware.
Testing and restoring firmware on switches Power-on firmware checksum test FIPS requires the checksums of the executables and libraries on the filesystem to be validated before Fabric OS modules are launched. This is to make sure these files have not been changed after they are installed.
Page 269
Testing and restoring firmware on switches User Name: userfoo File Name: /home/userfoo/v7.0.0 Password: <hidden> Do Auto-Commit after Reboot [Y]: n Reboot system after download [N]: y Firmware is being downloaded to the switch. This step may take up to 30 minutes.
Testing and restoring firmware on Backbones Testing and restoring firmware on Backbones This procedure enables you to perform a firmware download on each CP and verify that the procedure was successful before committing to the new firmware. The old firmware is saved in the secondary partition of each CP until you enter the firmwareCommit command.
Page 271
Testing and restoring firmware on Backbones 8. Verify the failover. a. Connect to the Backbone on the active CP, which is the former standby CP. b. Enter the haShow command to verify that the HA synchronization is complete. It takes a minute or two for the standby CP, which is the old active CP, to reboot and synchronize with the active CP.
Page 272
Testing and restoring firmware on Backbones ATTENTION Stop! If you have completed step 11, then you have committed the firmware on both CPs and you have completed the firmware download procedure. 12. Restore the firmware on the standby CP. In the current Backbone session for the standby CP, enter the firmwareRestore command. The standby CP reboots and the current Backbone session ends.
Validating a firmware download Validating a firmware download Validate the firmware download by running the following commands: firmwareShow, firmwareDownloadStatus, nsShow, nsAllShow, and fabricShow. All of the connected servers, storage devices, and switches should be present in the output of these commands. If there is a discrepancy, it is possible that a device or switch cannot connect to the fabric and further troubleshooting is necessary.
Page 274
Validating a firmware download Fabric OS Administrator’s Guide 53-1002745-02...
Logical switch overview This chapter describes the logical switch and logical fabric features. For information about device sharing with Virtual Fabrics, refer to “FC-FC routing and Virtual Fabrics” on page 606. For information about supported switches and port types, refer to “Supported platforms for Virtual Fabrics”...
Logical switch overview After you enable Virtual Fabrics, you can create up to seven additional logical switches, depending on the switch model. Figure 18 shows a Virtual Fabrics-enabled switch before and after it is divided into logical switches. Before you create logical switches, the chassis appears as a single switch (default logical switch). After you create logical switches, the chassis appears as multiple independent logical switches.
Logical switch overview A given port is always in one (and only one) logical switch. The following scenarios refer to the chassis after port assignment in Figure • If you assign P2 to logical switch 2, you cannot assign P2 to any other logical switch. •...
Management model for logical switches Management model for logical switches You can use one common IP address for the hardware that is shared by all of the logical switches in the chassis and you can set up individual IPv4 addresses for each Virtual Fabric. For a management host to manage a logical switch using the Internet Protocol over Fibre Channel (IPFC) IP address, it must be physically connected to the Virtual Fabric using a host bus adapter (HBA).
Logical fabric overview Logical fabric and ISLs Figure 23 shows two physical chassis divided into logical switches. In Figure 23, ISLs are used to connect the logical switches with FID 1 and the logical switches with FID 15. The logical switches with FID 8 are each connected to a non-Virtual Fabrics switch.
Logical fabric overview Base switch and extended ISLs Another way to connect logical switches is to use extended ISLs and base switches. When you divide a chassis into logical switches, you can designate one of the switches to be a base switch.
Logical fabric overview Think of the logical switches as being connected with logical ISLs, as shown in Figure 26. In this diagram, the logical ISLs are not connected to ports because they are not physical cables. They are a logical representation of the switch connections that are allowed by the XISL. FIGURE 26 Logical ISLs connecting logical switches To use the XISL, the logical switches must be configured to allow XISL use.
Page 285
Logical fabric overview By default, the physical ISL path is favored over the logical path (over the XISL) because the physical path has a lower cost. This behavior can be changed by configuring the cost of the dedicated physical ISL to match the cost of the logical ISL. ATTENTION If you disable a base switch, all of the logical ISLs are broken and the logical switches cannot communicate with each other unless they are connected by a physical ISL.
Account management and Virtual Fabrics Account management and Virtual Fabrics When user accounts are created, they are assigned a list of logical fabrics to which they can log in and a home logical fabric (home FID). When you connect to a physical chassis, the home FID defines the logical switch to which you are logged in by default.
Supported platforms for Virtual Fabrics Supported port configurations in Brocade Backbones Some of the ports in the Brocade DCX and DCX 8510 Backbone families are not supported on all types of logical switches. Table 50 lists the blades and ports that are supported on each type of logical switch.
Limitations and restrictions of Virtual Fabrics Virtual Fabrics interaction with other Fabric OS features Table 51 lists some Fabric OS features and considerations that apply when using Virtual Fabrics. TABLE 51 Virtual Fabrics interaction with Fabric OS features Fabric OS feature Virtual Fabrics interaction Access Gateway Virtual Fabrics is not supported on a switch if AG mode is enabled.
Limitations and restrictions of Virtual Fabrics TABLE 52 Maximum number of logical switches per chassis (Continued) Platform Maximum number of logical switches Brocade DCX 8510 family Brocade 5300 Brocade 5100 Brocade 6510 Brocade 6520 Brocade 7800 Brocade VA-40FC Refer to “Supported port configurations in Brocade Backbones”...
Enabling Virtual Fabrics mode Enabling Virtual Fabrics mode A fabric is said to be in Virtual Fabrics mode (VF mode) when the Virtual Fabrics feature is enabled. Before you can use the Virtual Fabrics features, such as logical switch and logical fabric, you must enable VF mode.
Configuring logical switches to use basic configuration values Use the following procedure to disable Virtual Fabrics mode: 1. Connect to the physical chassis and log in using an account with the chassis-role permission. 2. Use the fosConfig command to check whether VF mode is disabled: fosconfig --show 3.
Creating a logical switch or base switch 3. Enter n at the prompts to configure system and cfgload attributes. Enter y at the prompt to configure custom attributes. System (yes, y, no, n): [no] n cfgload attributes (yes, y, no, n): [no] n Custom attributes (yes, y, no, n): [no] y 4.
Executing a command in a different logical switch context Example The following example creates a logical switch with FID 4, and then assigns domain ID 14 to it. sw0:FID128:admin> lscfg --create 4 About to create switch with fid=4. Please wait... Logical Switch with FID (4) has been successfully created.
Deleting a logical switch switchMode: Native switchRole: Principal switchDomain: switchId: fffc0e switchWwn: 10:00:00:05:1e:82:3c:2b zoning: switchBeacon: FC Router: Fabric Name: Fab4 Allow XISL Use: ON LS Attributes: [FID: 4, Base Switch: No, Default Switch: No, Address Mode 0] Index Port Address Media Speed State Proto ============================================== 0e1600...
Adding and moving ports on a logical switch Example of deleting the logical switch with FID 7 switch_4:FID4:admin> lscfg --delete 7 All active login sessions for FID 7 have been terminated. Switch successfully deleted. Adding and moving ports on a logical switch This procedure explains how to add and move ports on logical switches.
Displaying logical switch configuration Displaying logical switch configuration Use the following procedure to display the configuration for a logical switch: 1. Connect to the physical chassis and log in using an account with the chassis-role permission. 2. Enter the lsCfg command to display a list of all logical switches and the ports assigned to them: lscfg --show [ -provision ] If the -provision option is specified, all ports on all slots are displayed, regardless of the slot status.
Changing a logical switch to a base switch Checking and logging message: fid = 5. Please enable your switch. sw0:FID128:admin> fosexec --fid 7 -cmd "switchenable" --------------------------------------------------- "switchenable" on FID 7: Changing a logical switch to a base switch Use the following procedure to change a logical switch to a base switch. 1.
Setting up IP addresses for a Virtual Fabric Configure... Fabric parameters (yes, y, no, n): [no] y WWN Based persistent PID (yes, y, no, n): [no] Allow XISL Use (yes, y, no, n): [yes] n WARNING!! Disabling this parameter will cause removal of LISLs to other logical switches.
Configuring a logical switch to use XISLs Configuring a logical switch to use XISLs When you create a logical switch, it is configured to use XISLs by default. Use the following procedure to allow or disallow the logical switch to use XISLs in the base fabric. XISL use is not supported in some cases.
Creating a logical fabric using XISLs Creating a logical fabric using XISLs This procedure describes how to create a logical fabric using multiple chassis and XISLs and refers to the configuration shown in Figure 28 as an example. FIGURE 28 Example of logical fabrics in multiple chassis and XISLs Use the following procedure to create a logical fabric using XISLs: 1.
Page 301
Creating a logical fabric using XISLs 4. Configure the logical switches in each chassis: a. Connect to the physical chassis and log in using an account with the chassis-role permission. b. Create a logical switch and assign it a fabric ID for the logical fabric. This FID must be different from the FID in the base fabric.
Page 302
Creating a logical fabric using XISLs Fabric OS Administrator’s Guide 53-1002745-02...
Zoning overview • QoS zones Assign high or low priority to designated traffic flows. QoS zones are regular zones with additional QoS attributes specified by adding a QOS prefix to the zone name. See “QoS: SID/DID traffic prioritization” on page 519 for more information. •...
Zoning overview Blue Zone Server 2 Server 1 Storage 2 Red Zone Storage 1 RAID Green Zone Storage 3 Server 3 FIGURE 29 Zoning example Approaches to zoning Table 53 lists the various approaches you can take when implementing zoning in a fabric. TABLE 53 Approaches to fabric-based zoning Zoning approach...
Zoning overview TABLE 53 Approaches to fabric-based zoning (Continued) Zoning approach Description Alternative approaches Application Zoning by application typically requires zoning multiple, perhaps incompatible, operating systems into the same zones. This method of zoning creates the possibility that a minor server in the application suite could disrupt a major server (such as a Web server disrupting a data warehouse server).
Zoning overview The types of zone objects used to define a zone can be mixed. For example, a zone defined with the zone objects 2,12; 2,14; 10:00:00:80:33:3f:aa:11 contains the devices connected to domain 2, ports 12 and 14, and a device with the WWN 10:00:00:80:33:3f:aa:11 (either node name or port name) that is connected on the fabric.
Zoning overview The different types of zone configurations are: • Defined Configuration The complete set of all zone objects defined in the fabric. • Effective Configuration A single zone configuration that is currently in effect. The effective configuration is built when you enable a specified zone configuration.
Zoning overview Identifying the enforced zone type Use the following procedure to identify zones and zone types: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the portZoneShow command, using the following syntax: portzoneshow Considerations for zoning architecture Table 54...
Broadcast zones Best practices for zoning The following are recommendations for using zoning: • Always zone using the highest Fabric OS-level switch. Switches with earlier Fabric OS versions do not have the capability to view all the functionality that a newer Fabric OS provides, as functionality is backwards compatible but not forwards compatible.
Broadcast zones Figure 30 illustrates how broadcast zones work with Admin Domains. Figure 30 shows a fabric with five devices and two Admin Domains, AD1 and AD2. Each Admin Domain has two devices and a broadcast zone. "1,1" "3,1" "4,1" "2,1"...
Zone aliases High availability considerations with broadcast zones If a switch has broadcast zone-capable firmware on the active CP (Fabric OS v5.3.x or later) and broadcast zone-incapable firmware on the standby CP (Fabric OS version earlier than v5.3.0), then you cannot create a broadcast zone because the zoning behavior would not be the same across an HA failover.
Zone aliases Creating an alias Use the following procedure to create an alias: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aliCreate command, using the following syntax: alicreate "aliasname", "member[; member...]" 3.
Zone aliases inconsistent. The inconsistency will result in different Effective Zoning configurations for switches in the fabric if a zone merge or HA failover happens. To avoid inconsistency it is recommended to commit the configurations using the 'cfgenable' command. Do you still want to proceed with saving the Defined zoning configuration only? (yes, y, no, n): [no] y Removing members from an alias...
Zone aliases The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory. If a transaction is open on a different switch in the fabric when this command is run, the transaction on the other switch is automatically aborted. A message displays on the other switches to indicate that the transaction was aborted.
Zone creation and maintenance Zone creation and maintenance Fabric OS allows you to create zones to better manage devices. Notes • Broadcast Zone: To create a broadcast zone, use the reserved name “broadcast”. Do not give a regular zone the name of “broadcast”. “Broadcast zones”...
Zone creation and maintenance To create a broadcast zone, use the reserved name “broadcast”. 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory.
Zone creation and maintenance Example Adding members to a zone switch:admin> zoneadd matt, "ze*; bond*; j*" switch:admin> cfgsave switch:admin> cfgshow Defined configuration: zone: matt 30:06:00:07:1e:a2:10:20; 3,2; zeus; bond; jake; jeff; jones zone: sloth bawn; bolt; bond; brain; 10:00:00:00:01:1e:20:20 alias: bawn 3,5;...
Zone creation and maintenance alias: jeff 30:00:00:05:1e:a1:cd:02; 40:00:00:05:1e:a1:cd:04 alias: jones 7,3; 4,5 alias: zeus 4,7; 6,8; 9,2 Effective configuration: No Effective configuration: (No Access) switch:admin> switch:admin> zoneremove matt,"30:06:00:07:1e:a2:10:20; ja*; 3,2" switch:admin> cfgsave switch:admin> cfgshow Defined configuration: zone: matt zeus; bond; jeff; jones zone: sloth bawn;...
Zone creation and maintenance alias: jake 4,7; 8,9; 14,11 alias: jeff 30:00:00:05:1e:a1:cd:02; 40:00:00:05:1e:a1:cd:04 alias: jones 7,3; 4,5 alias: zeus 4,7; 6,8; 9,2 Effective configuration: No Effective configuration: (No Access) switch:admin> switch:admin> zoneobjectreplace 11,2 4,8 switch:admin> cfgsave switch:admin> cfgshow Defined configuration: zone: matt zeus;...
Page 321
Zone creation and maintenance The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory. If a transaction is open on a different switch in the fabric when this command is run, the transaction on the other switch is automatically aborted. A message displays on the other switches to indicate that the transaction was aborted.
Zone creation and maintenance Viewing a zone in the defined configuration Use the following procedure to view a zone in the configuration: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the zoneShow command, using the following syntax: zoneshow[--sort] ["pattern"] [, mode] If no parameters are specified, the entire zone database (both the defined and effective configuration) is displayed.
Zone creation and maintenance 1,1; 1,2 alias: array1 21:00:00:20:37:0c:76:8c; 21:00:00:20:37:0c:71:02 alias: array2 21:00:00:20:37:0c:76:22; 21:00:00:20:37:0c:76:28 Effective configuration: cfg: fabric_cfg zone: Blue_zone 21:00:00:20:37:0c:76:8c 21:00:00:20:37:0c:71:02 Example Adding a new zone ‘red_zone’, deleting “1,1” and adding “6,15” to green_zone switch:admin> cfgshow --transdiffs Defined configuration: cfg: fabric_cfg Blue_zone zone: Blue_zone...
Page 324
Zone creation and maintenance alias: loop1 21:00:00:20:37:0c:76:85; 21:00:00:20:37:0c:71:df 3. Enter the zone --validate command to list all zone members that are not part of the current zone enforcement table. Note that zone configuration names are case-sensitive; blank spaces are ignored. switch:admin>...
Page 325
Zone creation and maintenance If you enter yes, and the cfgSave operation completes successfully then the following RASlog message [ZONE-1062] will be posted. [ZONE-1062], 620/181, FID 128, WARNING, sw0, Defined and Effective zone configurations are inconsistent, ltime:2012/09/03-23:18:30:983609 You can then either re-enable the updated configuration or revert to the older configuration. If there is no impact to the effective configuration with the latest update to the zoning configuration, then the following message will be displayed.
Default zoning mode Default zoning mode The default zoning mode controls device access if zoning is not implemented or if there is no effective zone configuration. The default zoning mode has two options: • All Access—All devices within the fabric can communicate with all other devices. •...
Zone database size switch:admin> cfgsave WARNING!!! The changes you are attempting to save will render the Effective configuration and the Defined configuration inconsistent. The inconsistency will result in different Effective Zoning configurations for switches in the fabric if a zone merge or HA failover happens.
Zone configurations Zone configurations You can store a number of zones in a zone configuration database. The maximum number of items that can be stored in the zone configuration database depends on the following criteria: • Number of switches in the fabric. •...
Zone configurations Adding zones (members) to a zone configuration Use the following procedure to add members to a zone configuration: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgAdd command, using the following syntax: cfgadd "cfgname", "member[;...
Zone configurations Enabling a zone configuration The following procedure ends and commits the current zoning transaction buffer to nonvolatile memory. If a transaction is open on a different switch in the fabric when this procedure is run, the transaction on the other switch is automatically aborted. A message displays on the other switches to indicate that the transaction was aborted.
Zone configurations Deleting a zone configuration Use the following procedure to delete a zone configuration: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgDelete command, using the following syntax: cfgdelete "cfgname" 3.
Zone configurations alias: array1 21:00:00:20:37:0c:76:8c; 21:00:00:20:37:0c:71:02 alias: array2 21:00:00:20:37:0c:76:22; 21:00:00:20:37:0c:76:28 alias: loop1 21:00:00:20:37:0c:76:85; 21:00:00:20:37:0c:71:df Effective configuration: cfg: USA_cfg zone: Blue_zone 21:00:00:20:37:0c:76:8c 21:00:00:20:37:0c:71:02 21:00:00:20:37:0c:76:22 21:00:00:20:37:0c:76:28 zone: Purple_zone 21:00:00:20:37:0c:76:85 21:00:00:20:37:0c:71:df Viewing selected zone configuration information Use the following procedure to view the selected zone configuration information: 1.
Zone object maintenance Clearing all zone configurations Use the following procedure to clear all zone configurations: 1. Connect to the switch and log in using an account with admin permissions. 2. Use cfgClear to clear all zone information in the transaction buffer. ATTENTION Be careful using the cfgClear command because it deletes the defined configuration.
Zone object maintenance 4. Enter the cfgShow command to verify the new zone object is present. switch:admin> cfgshow "Test*" cfg: Test1 Blue_zone cfg: Test_cfg Purple_zone; Blue_zone switch:admin> cfgShow "US_Test1" cfg: US_Test1 Blue_zone 5. If you want the change preserved when the switch reboots, use cfgSave to save it to nonvolatile (flash) memory.
Zone object maintenance You are about to expunge one configuration or member. This action could result in removing many zoning configurations recursively. [Removing the last member of a configuration removes the configuration.] Do you want to expunge the member? (yes, y, no, n): [no] yes 4.
Zone configuration management Zone configuration management You can add, delete, or remove individual elements in an existing zone configuration to create an appropriate configuration for your SAN environment. After the changes have been made, save the configuration to ensure the configuration is permanently saved in the switch and that the configuration is replicated throughout the fabric.
Page 337
Zone merging Adding a new fabric that has no zone configuration information to an existing fabric is very similar to adding a new switch. All switches in the new fabric inherit the zone configuration data. If the existing fabric has an effective zone configuration, then the same configuration becomes the effective configuration for the new switches.
Zone merging • Merging two fabrics Both fabrics have identical zones and configurations enabled, including the default zone mode. The two fabrics will join to make one larger fabric with the same zone configuration across the newly created fabric. If the two fabrics have different zone configurations, they will not be merged. If the two fabrics cannot join, the ISL between the switches will segment.
Zone merging Zone merging scenarios The following tables provide information on merging zones and the expected results. • Table 55 on page 339: Defined and effective configurations • Table 56 on page 340: Different content • Table 57 on page 340: Different names •...
Page 340
Zone merging TABLE 55 Zone merging scenarios: Defined and effective configurations (Continued) Description Switch A Switch B Expected results Switch A and Switch B have different defined: cfg2 defined: cfg1 Clean merge. The new configuration will be a defined configurations. Switch B has an zone2: ali3;...
Zone merging TABLE 58 Zone merging scenarios: TI zones Description Switch A Switch B Expected results Switch A does not have Traffic Isolation defined: cfg1 defined: cfg1 Clean merge. TI zones are not automatically (TI) zones. activated after the merge. effective: cfg1 TI_zone1 Switch B has TI zones.
Concurrent zone transactions TABLE 60 Zone merging scenarios: Mixed Fabric OS versions Description Switch A Switch B Expected results Switch A is running Fabric OS 7.0.0 or effective: cfg1 No effective Fabric segments due to zone conflict. later. configuration. defzone = allaccess Switch B is running a Fabric OS version defzone - noaccess earlier than 7.0.0.
Page 343
Concurrent zone transactions u30:FID128:admin> cfgsave You are about to save the Defined zoning configuration. This action will only save the changes on Defined configuration. Multiple open transactions are pending in this fabric. Only one transaction can be saved. Please abort all unwanted transactions using the cfgtransabort command.
Page 344
Concurrent zone transactions Fabric OS Administrator’s Guide 53-1002745-02...
Traffic Isolation Zoning overview Figure 31 shows a fabric with a TI zone consisting of the following: • N_Ports: “1,7”, “1,8”, “4,5”, and “4,6” • E_Ports: “1,1”, “3,9”, “3,12”, and “4,7” The dotted line indicates the dedicated path between the initiator in Domain 1 to the target in Domain 4.
Traffic Isolation Zoning overview TABLE 61 Traffic behavior when failover is enabled or disabled in TI zones Failover enabled Failover disabled If the dedicated path is not the shortest path or if the If the dedicated path is not the shortest path or if the dedicated path is broken, the TI zone traffic will use a dedicated path is broken, traffic for that TI zone is non-dedicated path instead.
Traffic Isolation Zoning overview • Ensure that there are multiple paths between switches. Disabling failover locks the specified route so that only TI zone traffic can use it. Non-TI zone traffic is excluded from using the dedicated path. • You should enable failover-enabled TI zones before enabling failover-disabled TI zones, to avoid dropped frames.
Traffic Isolation Zoning overview FSPF routing rules and traffic isolation All traffic must use the lowest cost path. FSPF routing rules take precedence over the TI zones, as described in the following situations. If the dedicated ISL is not the lowest cost path ISL, then the following rules apply: •...
Enhanced TI zones Domain 1 Domain 3 = Dedicated Path = Ports in the TI zone Domain 2 Domain 4 FIGURE 34 Dedicated path is not the shortest path NOTE For information about setting or displaying the FSPF cost of a path, see the linkCost and topologyShow commands in the Fabric OS Command Reference.
Enhanced TI zones Illegal configurations with enhanced TI zones When you create TI zones, ensure that all traffic from a port to all destinations on a remote domain have the same path. Do not create separate paths from a local port to two or more ports on the same remote domain.
Traffic Isolation Zoning over FC routers In this example traffic from the Target to Domain 2 is routed correctly. Only one TI zone describes a path to Domain 2. However, both TI zones describe different, valid paths from the Target to Domain 1.
Traffic Isolation Zoning over FC routers Edge fabric 1 Backbone Edge fabric 2 fabric = Dedicated path set up by TI zone in edge fabric 1 = Dedicated path set up by TI zone in edge fabric 2 = Dedicated path set up by TI zone in backbone fabric FIGURE 38 Traffic Isolation Zoning over FCR In addition to setting up TI zones, you must also ensure that the devices are in an LSAN zone so...
Traffic Isolation Zoning over FC routers TI zones within an edge fabric A TI zone within an edge fabric is used to route traffic between a real device and a proxy device through a particular EX_Port. For example, in Figure 39, you can set up a TI zone to ensure that traffic between Host 1 and the proxy target is routed through EX_Port 9.
Traffic Isolation Zoning over FC routers TI zones within a backbone fabric A TI zone within a backbone fabric is used to route traffic within the backbone fabric through a particular ISL. For example, in Figure 40, a TI zone is set up in the backbone fabric to ensure that traffic between EX_Ports “1,1”...
General rules for TI zones Limitations of TI zones over FC routers Be aware of the following when configuring TI zones over FC routers: • A TI zone defined within the backbone fabric does not guarantee that edge fabric traffic will arrive at a particular EX_Port.
General rules for TI zones For example, in Figure 41, the TI zone was configured incorrectly and E_Port “3,9” was erroneously omitted from the zone. The domain 3 switch assumes that traffic coming from E_Port 9 is not part of the TI zone and so that traffic is routed to E_Port 11 instead of E_Port 12, if failover is enabled.
Supported configurations for Traffic Isolation Zoning E-Port Trunks Trunk members in TI zone: 8 Trunk members not in TI zone: 9 10 E-Port Trunks Trunk members in TI zone: 16 Trunk members not in TI zone: 17 18 Supported configurations for Traffic Isolation Zoning The following configuration rules apply to TI zones: •...
Limitations and restrictions of Traffic Isolation Zoning Trunking with TI zones If you implement trunking and TI zones, you should keep the following points in mind: • To include a trunk group in a TI zone, you must include all ports of the trunk in the TI zone. •...
Admin Domain considerations for Traffic Isolation Zoning • To include a trunk group in a TI zone, you must include all ports of the trunk in the TI zone. • If two N_Ports are online and have the same shared area, and one of them is configured in a TI zone, then they both must be configured in that same TI zone.
Virtual Fabrics considerations for Traffic Isolation Zoning Virtual Fabrics considerations for Traffic Isolation Zoning This section describes how TI zones work with Virtual Fabrics. See Chapter 10, “Managing Virtual Fabrics,” for information about the Virtual Fabrics feature, including logical switches and logical fabrics.
Virtual Fabrics considerations for Traffic Isolation Zoning Domain 8 Domain 3 Domain 5 Domain 9 Target Host = Dedicated Path = Ports in the TI zones FIGURE 43 Creating a TI zone in a logical fabric You must also create and activate a TI zone in the base fabric to reserve the XISLs for the dedicated path.
Traffic Isolation Zoning over FC routers with Virtual Fabrics Traffic Isolation Zoning over FC routers with Virtual Fabrics This section describes how you can set up TI zones over FC routers in logical fabrics. Figure 45 shows two physical chassis configured into logical switches. The initiator in FID 1 communicates with the target in FID 3 over the EX_Ports in the base switches.
Creating a TI zone Creating a TI zone You create and modify TI zones using the zone command. Other zoning commands, such as zoneCreate, aliCreate, and cfgCreate, cannot be used to manage TI zones. When you create a TI zone, you can set the state of the zone to activated or deactivated. By default the zone state is set to activated;...
Page 365
Creating a TI zone Example TI zone creation The following examples create a TI zone named “bluezone”, which contains E_Ports 1,1 and 2,4 and N_Ports 1,8 and 2,6. To create a TI zone with failover enabled and in the activated state (default settings): switch:admin>...
Creating a TI zone Creating a TI zone in a base fabric 1. Connect to the switch and log in using an account with admin permissions. 2. Create a “dummy” zone configuration in the base fabric. For example: zone --create "z1", "1,1" cfgcreate "base_config", z1 3.
Modifying TI zones Modifying TI zones Using the zone --add command, you can add ports to an existing TI zone, change the failover option, or both.You can also activate or deactivate the TI zone. Using the zone --remove command, you can remove ports from existing TI zones. If you remove the last member of a TI zone, the TI zone is deleted.
Changing the state of a TI zone Example of modifying a TI zone To add port members to the existing TI zone bluezone: switch:admin> zone --add bluezone -p "3,4; 3,6" To add port members to the existing TI zone in a backbone fabric: switch:admin>...
Deleting a TI zone Deleting a TI zone Use the zone --delete command to delete a TI zone from the defined configuration. This command deletes the entire zone; to only remove port members from a TI zone, use the zone --remove command, as described in “Modifying TI zones”...
Troubleshooting TI zone routing problems Example displaying information about all TI zones in the defined configuration in ascending order switch:admin> zone --show -ascending Defined TI zone configuration: TI Zone Name: bluezone: Port List: 8,3; 8,5; 9,2; 9,3; Configured Status: Deactivated / Failover-Disabled Enabled Status: Activated / Failover-Enabled TI Zone Name: greenzone:...
Setting up TI over FCR (sample procedure) Setting up TI over FCR (sample procedure) The following example shows how to set up TI zones over FCR to provide a dedicated path shown in Figure 47. In this example, three TI zones are created: one in each of the edge fabrics and one in the backbone fabric.
Page 372
Setting up TI over FCR (sample procedure) The Fabric has 3 switches b. Enter the following commands to create and display a TI zone: E1switch:admin> zone --create -t ti TI_Zone1 -p "4,8; 4,5, 1,-1; 6,-1" E1switch:admin> zone --show Defined TI zone configuration: TI Zone Name: TI_Zone1 Port List:...
Page 373
Setting up TI over FCR (sample procedure) Enter the following commands to reactivate your current effective configuration and enforce the TI zones. E2switch:admin> cfgactvshow Effective configuration: cfg: cfg_TI zone: lsan_t_i_TI_Zone1 10:00:00:00:00:00:02:00:00 10:00:00:00:00:00:03:00:00 10:00:00:00:00:00:08:00:00 E2switch:admin> cfgenable cfg_TI You are about to enable a new zoning configuration. This action will replace the old zoning configuration with the current configuration selected.
Page 374
Setting up TI over FCR (sample procedure) Fabric OS Administrator’s Guide 53-1002745-02...
Bottleneck detection overview • If the bottleneck detection feature detects ISL congestion, you can use ingress rate limiting to slow down low priority application traffic, if it is contributing to the congestion. Notes • Bottleneck detection is configured on a per-switch basis, with optional per-port exclusions. •...
Supported configurations for bottleneck detection You can use the bottleneckMon command to specify alerting parameters for the following: • Whether alerts are to be sent when a bottleneck condition is detected • The size of the time window to look at when determining whether to alert •...
Supported configurations for bottleneck detection High availability considerations for bottleneck detection The bottleneck detection configuration is maintained across a failover or reboot; however, bottleneck statistics collected are lost. Upgrade and downgrade considerations for bottleneck detection The bottleneck detection configuration is persistent across firmware upgrades and downgrades. The sub-second latency criterion parameter settings are not preserved on downgrade to firmware versions earlier than Fabric OS 7.0.0.
Credit Loss Credit Loss Fabric OS v7.1 and later supports back-end credit loss detection back-end ports and core blades as well as on the Brocade 5300 and 6520 switches, although the support is slightly different on each device. See below for details on these switches, and the Fabric OS Troubleshooting and Diagnostics Guide for more general information.
Enabling bottleneck detection on a switch The following credit loss recovery methods are supported for Brocade 6520 back-end ports: • For all the credit loss methods described above, a link reset will automatically be performed, assuming that this option was enabled. See “Enabling back-end credit loss detection and recovery”...
Displaying bottleneck detection configuration details 3. Repeat step 1 step 2 on every switch in the fabric. NOTE Best practice is to use the default values for the alerting and sub-second latency criterion parameters. Example of enabling bottleneck detection (Recommended use case) The following example enables bottleneck detection on the switch with alerts using default values for thresholds and time.
Setting bottleneck detection alerts FIGURE 48 Affected seconds for bottleneck detection The -time parameter specifies the time window. For this example, -time equals 12 seconds. The -cthresh and -lthresh parameters specify the thresholds on number of affected seconds that trigger alerts for congestion and latency bottlenecks, respectively. This example uses the default values for these parameters, where -cthresh = 0.8 (80%) and -lthresh = 0.1 (10%).
Changing bottleneck detection parameters Setting a congestion alert only This example enables a congestion alert and shows its values. Example of setting an alert for congestion switch:admin> bottleneckmon --enable -alert=congestion switch:admin> bottleneckmon --status Bottleneck detection - Enabled ============================== Switch-wide sub-second latency bottleneck criterion: ==================================================== Time threshold - 0.800...
Changing bottleneck detection parameters NOTE Entering a --config command changes only those settings specified in the command; all others are left alone. The only exceptions are for the -alert (restores alerts using recorded values) or -noalert (disables all alerts) switches. This means that if you want alerts, you must specify what you want as the -alert value for every bottleneckmon - -config -alert command.
Page 386
Changing bottleneck detection parameters Switch-wide sub-second latency bottleneck criterion: ==================================================== Time threshold - 0.800 Severity threshold - 50.000 Switch-wide alerting parameters: ================================ Alerts - Yes Latency threshold for alert - 0.200 Congestion threshold for alert - 0.700 Averaging time for alert - 200 seconds Quiet time for alert - 150 seconds...
Page 387
Changing bottleneck detection parameters Congestion threshold for alert - 0.700 Averaging time for alert - 200 seconds Quiet time for alert - 150 seconds Per-port overrides for alert parameters: ======================================== Port Alerts? LatencyThresh CongestionThresh Time (s) QTime (s) ================================================================================= 0.750 Example 5: Changing the latency time value for a single port This changes the time value to 250 seconds for port 47 only.
Advanced bottleneck detection settings Switch-wide alerting parameters: ================================ Alerts - Yes Latency threshold for alert - 0.200 Congestion threshold for alert - 0.700 Averaging time for alert - 200 seconds Quiet time for alert - 150 seconds Adjusting the frequency of bottleneck alerts Depending on the circumstances, a problematic switch or port might be triggering alerts more frequently than desired.
Excluding a port from bottleneck detection • You want greater-than-default (sub-second) latency sensitivity on your fabric, so you set sub-second latency criterion parameters at the time you enable bottleneck detection. • You want to reduce the number of alerts you are receiving about known latency bottlenecks in the fabric, so you temporarily decrease the sub-second latency sensitivity on these ports.
Page 390
Excluding a port from bottleneck detection For trunking, if you exclude a slave port from bottleneck detection, the exclusion has no effect as long as the port is a trunk slave. The exclusion takes effect only if the port becomes a trunk master or leaves the trunk.
Disabling bottleneck detection on a switch Disabling bottleneck detection on a switch When you disable bottleneck detection on a switch, all bottleneck configuration details are discarded, including the list of excluded ports and non-default values of alerting parameters. Use the following procedure to disable bottleneck detection: 1.
Chapter In-flight Encryption and Compression In this chapter • In-flight encryption and compression overview ..... . 393 • Configuring encryption and compression ......399 •...
In-flight encryption and compression overview Compression/Encryption FIGURE 49 Encryption and compression on 16 Gbps ISLs The encryption and compression features are designed to work only with E_Ports, EX_Ports, and XISL ports (in VF mode). Encryption and compression are also compatible with the following features: •...
In-flight encryption and compression overview Bandwidth limits Fabric OS supports up to 32 Gbps of data encryption and 32 Gbps of data compression per 16G-capable FC platform. This limits the number of ports that can have these features enabled at any one time.
In-flight encryption and compression overview The port level authentication security feature must be enabled before encryption configuration can be enabled. Pre-shared secret keys should be configured on both ends of the ISL to perform authentication. Once the link has been authenticated, the port (E_Port or EX_Port) will use the IKE protocol to generate and exchange the keys, IV and Salt values.
Page 397
In-flight encryption and compression overview portCfgCompress The portCfgCompress command allows you to enable or disable compression on the specified port. Usage: portCfgCompress action [slot/]port Example Enabling the compression configuration on port 2 switch:admin> portcfgcompress --enable 2 Example Disabling the compression configuration on port 2 switch:admin>...
Configuring encryption and compression Virtual Fabrics considerations The E_Ports and EX_Ports in the user-created logical switch, base switch, or default switch; and EX_Ports on base switches can support encryption and compression. You can configure encryption on XISL ports, but not on LISL ports. However, frames from the LISL ports are implicitly encrypted or compressed as they pass through encryption/compression enabled XISL ports.
Page 400
Configuring encryption and compression Notes • If you need to disable authentication on a port that has encryption or compression configured, you must first disable encryption or compression on the port, and then disable authentication. • If you want to enable authentication across a FC router and an edge fabric switch, you must first bring all EX_Ports online without using authentication.
Configuring encryption and compression Viewing the encryption and compression configuration To determine which ports are available for encryption or compression on each ASIC on the switch, follow these steps: 1. Connect to the switch and log in using an account with admin permissions. 2.
Configuring encryption and compression Changing port speed on encryption/compression enabled ports The port speed values can be displayed through several commands, including portStatsShow, portEncCompShow, and portCfgSpeed. However, the port speed can only be changed using the portCfgSpeed command. If the port speed is configured as AUTO NEG, the speed of the port is taken as 16G for calculation purposes.
Configuring encryption and compression • Because encryption adds more payload to the port in addition to compression, the compression ratio calculation is significantly affected on ports configured for both encryption and compression. This is because the compressed length then also includes the encryption header.
Configuring encryption and compression For additional information about configuring DH-CHAP authentication for E_Ports and EX_Ports, “Authentication policy for fabric elements” on page 207. Configuring encryption NOTE Before performing this procedure, you must authenticate the port as described in “Configuring and enabling authentication”...
Configuring encryption and compression 4. Enable the port with the portEnable command. After enabling the port, the new configuration becomes active. Disabling encryption To disable encryption on a port, follow these steps: 1. Connect to the switch and log in using an account with secure admin permissions, or an account with OM permissions for the EncryptionConfiguration RBAC class of commands.
Encryption and compression examples Encryption and compression examples The following examples show configuring and enabling encryption and compression. In this case, encryption and compression are being applied to the E_Ports at either end of an ISL connecting a port on a blade in an enterprise class platform named ‘myDCX’ to a port on a Brocade 6510 switch named ‘myswitch’.
Encryption and compression examples Example of enabling encryption and compression on an E_Port This example configures and enables encryption and compression on a given port. The commands in this example are shown entered on the Brocade 6510 named ‘myswitch’. The same commands must also be entered on the peer switch.
Page 408
Encryption and compression examples Are you done? (yes, y, no, n): [no] y Saving data to key store... Done. myswitch:admin> secauthsecret --show Name ----------------------------------------------- 10:00:00:05:1e:e5:cb:00 dcx_150 myswitch:admin> Activate authentication After you set up the DH-CHAP secrets, you activate DH-CHAP authentication. myswitch:admin>...
Page 409
Encryption and compression examples Rate Limit EX Port Mirror Port Credit Recovery F_Port Buffers Fault Delay: 0(R_A_TOV) NPIV PP Limit: CSCTL mode: Frame Shooter Port D-Port mode: Compression: Encryption: FEC: myswitch:admin> Enabling compression Finally, you enable compression on the same port. The subsequent portCfgShow command shows both encryption and compression to be enabled on the port.
Encryption and compression examples Examples of disabling encryption and compression This example disables the encryption and compression that were enabled in the previous example. Example Disabling encryption on port 0 myswitch:admin> portdisable 0 myswitch:admin> portcfgencrypt --disable 0 myswitch:admin> portenable 0 Example Disabling compression on port 0: myswitch:admin>...
Working with EX_Ports Working with EX_Ports An EX_Port is a type of E_Port (expansion port) that connects a Fibre Channel router to an edge fabric. From the point of view of a switch in an edge fabric, an EX_Port appears as a normal E_Port; It follows applicable Fibre Channel standards just line an E_Port.
Working with EX_Ports NOTE If trunking is enabled, be aware that the ports creating the bandwidth limitation will form a trunk group, while the rest of the ports will be segmented. Example of enabling encryption and compression on an EX_Port This example configures and enables encryption and compression on an EX_Port.
Page 413
Working with EX_Ports This command is used to set up secret keys for the DH-CHAP authentication. The minimum length of a secret key is 8 characters and maximum 40 characters. Setting up secret keys does not initiate DH-CHAP authentication. If switch is configured to do DH-CHAP, it is performed whenever a port or a switch is enabled.
Page 414
Working with EX_Ports QOS Port Port Auto Disable: Rate Limit EX Port Mirror Port Credit Recovery F_Port Buffers Fault Delay: 0(R_A_TOV) NPIV PP Limit: CSCTL mode: D-Port mode: Compression: Encryption: FEC: myswitch:admin> Example Enabling compression on port 1 of ‘myswitch’ The subsequent portCfgShow command shows both encryption and compression to be enabled on the port.
Page 415
Working with EX_Ports FCR:admin> portcfgexport 1 Port info Admin: enabled State: Pid format: core(N) Operate mode: Brocade Native Edge Fabric ID: Front Domain ID: Front WWN: 50:00:53:31:37:43:ee:14 Principal Switch: Principal WWN: 10:00:00:05:33:13:70:3e Fabric Parameters: Auto Negotiate R_A_TOV: 10000(N) E_D_TOV: 2000(N) Authentication Type: None DH Group:...
Page 416
Working with EX_Ports characters. Setting up secret keys does not initiate DH-CHAP authentication. If switch is configured to do DH-CHAP, it is performed whenever a port or a switch is enabled. Warning: Please use a secure channel for setting secrets. Using an insecure channel is not safe and may compromise secrets.
Page 417
Working with EX_Ports NPIV PP Limit: CSCTL mode: D-Port mode: Compression: Encryption: FEC: Example Enabling compression on the same port. The portCfgShow command shows that both encryption and compression are now enabled on this port. edge:admin> portdisable 1 edge:admin> portcfgcompress --enable 1 edge:admin>...
Page 418
Working with EX_Ports EX_Port commands See the Fabric OS Command Reference for more details on these EX_Port -valid commands. portCfgExPort The portCfgExPort command sets a port to be an EX_Port, and also sets and displays EX_Port configuration parameters (including those for encryption and compression). Usage: portCfgExPort <action>...
NPIV overview Index Port Address Media Speed State Proto ============================================== 010000 Online FC F-Port 20:0c:00:05:1e:05:de:e4 0xa06601 010100 Online FC F-Port 1 N Port + 4 NPIV public 010200 Online FC F-Port 1 N Port + 119 NPIV public 010300 Online FC F-Port 1 N Port + 221 NPIV public On the Brocade DCX and DCX-4S with the FC8-64 blade, the base port is not included in the NPIV device count.
Configuring NPIV TABLE 64 Number of supported NPIV devices (Continued) Platform Virtual Fabrics Logical switch type NPIV support DCX-4S Enabled Logical switch Yes, 255 virtual device limit. DCX-4S Enabled Base switch Maximum limit support takes precedence if user-configured maximum limit is greater. This applies to shared areas on the FC4-48, FC8-48, and FC8-64 port blades.
Viewing NPIV port configuration information Viewing NPIV port configuration information 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the portCfgShow command to view the switch ports information. The following example shows whether a port is configured for NPIV: switch:admin>...
User- and auto-assigned FA-PWWN behavior NOTE For the server to use the FA-PWWN feature, it must be using a Brocade HBA or adapter. Refer to the release notes for the HBA or adapter versions that support this feature. Some configuration of the HBA must be performed to use the FA-PWWN. User- and auto-assigned FA-PWWN behavior An FA-PWWN can be either user-generated or automatically assigned by the fabric.
Configuring FA-PWWNs This section includes an FA-PWWN configuration procedure for each of the following two topologies: • An FA-PWWN for an HBA device that is connected to an Access Gateway switch. • An FA-PWWN for an HBA device that is connected directly to an edge switch. These topologies are shown in Figure Access Gateway Switch...
Configuring FA-PWWNs 3. Enter the fapwwn --show -ag all command: You should see output similar to the following sample. (In this example, long lines of output are shown split across two lines, for better readability.) ----------------------------------------------------------- AG Port Port Device Port WWN ----------------------------------------------------------- 10:00:00:05:1e:65:8a:d5/16 --:--:--:--:--:--:--:--...
Supported switches and configurations for FA-PWWN 3. Enter the fapwwn --show -port all command: You should see output similar to the following sample. ----------------------------------------------------------------------- Port PPWWN VPWWN PID Enable MapType ----------------------------------------------------------------------- 0 --:--:--:--:--:--:--:-- 52:00:10:00:00:0f:50:30 10101 Yes Port/Auto 1 --:--:--:--:--:--:--:-- 11:22:33:44:33:22:11:22 Port/User 52:00:10:00:00:0f:50:44 10 --:--:--:--:--:--:--:-- 52:00:10:00:00:0f:50:45...
Configuration upload and download considerations for FA-PWWN • Access Gateway platforms running Fabric OS v7.0.0 or later: Brocade 300 Brocade 5100 Brocade 6505 Brocade 6510 • Brocade HBAs with driver version 3.0.0.0: Brocade 415 Brocade 425 Brocade 815 Brocade 825 Configuration upload and download considerations for FA-PWWN The configuration upload and download utilities can be used to import and export the FA-PWWN configuration.
Restrictions of FA-PWWN NOTE When creating the DCC policy, use the physical device WWN and not the FA-PWWN. If you use DCC, a policy check is done on the physical PWWN on the servers. In the case of an HBA, the FA-PWWN is assigned to the HBA only after the DCC check is successful.
Page 432
Access Gateway N_Port failover with FA-PWWN Fabric OS Administrator’s Guide 53-1002745-02...
Administrative Domains overview NOTE Do not confuse an Admin Domain number with the domain ID of a switch. They are two different identifiers. The Admin Domain number identifies the Admin Domain and has a range from 0 through 255. The domain ID identifies a switch in the fabric and has a range from 1 through 239. Figure 52 shows a fabric with two Admin Domains: AD1 and AD2.
Administrative Domains overview Admin Domain features Admin Domains allow you to do the following: • Define the scope of an Admin Domain to encompass ports and devices within a switch or a fabric. • Share resources across multiple Admin Domains. For example, you can share array ports and tape drives between multiple departments.
Administrative Domains overview Table 65 lists each Admin Domain user type and describes its administrative access and capabilities. TABLE 65 AD user types User type Description Physical fabric User account with admin permissions and with access to all Admin Domains (AD0 through administrator AD255).
Page 437
Administrative Domains overview For example, if DeviceA is not a member of any user-defined Admin Domain, then it is an implicit member of AD0. If you explicitly add DeviceA to AD0, then DeviceA is both an implicit and an explicit member of AD0. AD0 implicit members AD0 explicit members AD2 members...
Administrative Domains overview FIGURE 54 Fabric with AD0 and AD255 Home Admin Domains and login You are always logged in to an Admin Domain, and you can view and modify only the devices in that Admin Domain. If you have access to more than one Admin Domain, one of them is designated as your home Admin Domain, the one you are automatically logged in to.
Administrative Domains overview • For user-defined accounts, the home Admin Domain defaults to AD0 but an administrator can set the home Admin Domain to any Admin Domain to which the account is given access. • If you are in any Admin Domain context other than AD0, the Admin Domain number is included in the system prompt displayed during your session.
Administrative Domains overview If a device is a member of an Admin Domain, the switch port to which the device is connected becomes an indirect member of that Admin Domain and the domain,index is removed from the AD0 implicit membership list. NOTE If the switch domain ID changes, the domain,index members are invalid (they are not automatically changed).
Administrative Domains overview Figure 55 on page 441 shows an unfiltered view of a fabric with two switches, three devices, and two Admin Domains. The devices are labeled with device WWNs and the switches are labeled with domain IDs and switch WWNs. FIGURE 55 Fabric showing switch and device WWNs Figure 56...
Admin Domain management for physical fabric administrators Admin Domain compatibility, availability, and merging Admin Domains maintain continuity of service for Fabric OS features and operate in mixed-release Fabric OS environments. High availability is supported with some backward compatibility. When an E_Port comes online, the adjacent switches merge their AD databases. The receiving switch accepts an AD database from the neighboring switch only if the local AD database is empty or if the new AD database exactly matches both the defined and effective configurations of the local AD database.
Admin Domain management for physical fabric administrators Setting the default zoning mode for Admin Domains To begin implementing an Admin Domain structure within your SAN, you must first set the default zoning mode to No Access. You must be in AD0 to change the default zoning mode. 1.
Admin Domain management for physical fabric administrators ad --select 255 5. Enter the ad create command using the -d option to specify device and switch port members and the -s option to specify switch members: ad --create ad_id -d "dev_list" -s "switch_list" 6.
Page 445
Admin Domain management for physical fabric administrators Creating a new user account for managing Admin Domains 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the userConfig add command using the -r option to set the role, the -a option to provide access to Admin Domains, and the -h option to specify the home Admin Domain.
Admin Domain management for physical fabric administrators Removing an Admin Domain from a user account When you remove an Admin Domain from an account, all of the currently active sessions for that account are logged out. 1. Connect to the switch and log in using an account with admin permissions. 2.
Admin Domain management for physical fabric administrators Deactivating an Admin Domain If you deactivate an Admin Domain, the members assigned to the Admin Domain can no longer access their hosts or storage unless those members are part of another Admin Domain. You cannot log in to an Admin Domain that has been deactivated.
Admin Domain management for physical fabric administrators • To save the Admin Domain definition, enter ad save. • To save the Admin Domain definition and directly apply the definition to the fabric, enter ad apply. Example of adding two switch ports, designated by domain,index, to AD1 switch:AD255:admin>...
Admin Domain management for physical fabric administrators 4. Enter the appropriate command based on whether you want to save or activate the Admin Domain definition: • To save the Admin Domain definition, enter ad save. • To save the Admin Domain definition and directly apply the definition to the fabric, enter ad apply.
Admin Domain management for physical fabric administrators Deleting all user-defined Admin Domains When you clear the Admin Domain configuration, all user-defined Admin Domains are deleted, the explicit membership list of AD0 is cleared, and all fabric resources (switches, ports, and devices) are returned to the implicit membership list of AD0.
Page 451
Admin Domain management for physical fabric administrators 3. Enter the zone --copy command to copy the zones from all user-defined Admin Domains to AD0. zone --copy source_AD.source_name dest_name In this syntax, source_AD is the name of the user-defined AD from which you are copying the zone, source_name is the name of the zone to be copied, and dest_name is the name to give to the zone after it is copied to AD0.
Admin Domain management for physical fabric administrators FIGURE 57 AD0 and two user-defined Admin Domains, AD1 and AD2 At the conclusion of the procedure, all devices and zones are moved to AD0, and the user-defined Admin Domains are deleted, as shown in Figure FIGURE 58 AD0 with three zones...
Page 453
Admin Domain management for physical fabric administrators 10:00:00:00:02:00:00:00; 10:00:00:00:03:00:00:00 Effective configuration: cfg: AD1_cfg zone: AD1_BlueZone 10:00:00:00:02:00:00:00 10:00:00:00:03:00:00:00 Zone CFG Info for AD_ID: 2 (AD Name: AD2, State: Active) : Defined configuration: cfg: AD2_cfg AD2_GreenZone zone: AD2_GreenZone 10:00:00:00:04:00:00:00; 10:00:00:00:05:00:00:00 Effective configuration: cfg: AD2_cfg zone:...
SAN management with Admin Domains Validating an Admin Domain member list You can validate the device and switch member list. You can list non-existing or offline Admin Domain members. You can also identify misconfigurations of the Admin Domain. The Admin Domain validation process is not applicable for AD0, because AD0 implicitly contains all unassigned online switches and their devices.
SAN management with Admin Domains CLI commands in an AD context The CLI command input arguments are validated against the AD member list; they do not work with input arguments that specify resources that are not members of the current Admin Domain. All commands present filtered output, showing only the members of the current Admin Domain.
SAN management with Admin Domains Displaying an Admin Domain configuration You can display the membership information and zone database information of a specified Admin Domain. Notice the following differences in the information displayed based on the Admin Domain: • AD255: If you do not specify the AD name or number, all information about all existing Admin Domains is displayed.
SAN management with Admin Domains You cannot switch to another Admin Domain context from within the shell created by ad --select. You must first exit the shell, and then issue the ad --select command again. Example of switching to a different Admin Domain context The following example switches to the AD12 context and back.
SAN management with Admin Domains TABLE 67 Admin Domain interaction with Fabric OS features (Continued) Fabric OS feature Admin Domain interaction FDMI FDMI operations are allowed only in AD0 and AD255. FICON Admin Domains support FICON. However, you must perform additional steps because FICON management requires additional physical control of the ports.
SAN management with Admin Domains The AD zone database also has the following characteristics: Each zone database has its own name space. For example, you can define a zone name of test_z1 in more than one Admin Domain. There is no zone database linked to the physical fabric (AD255) and no support for zone database updates.
SAN management with Admin Domains LSAN zone names in AD0 are never converted for backward-compatibility reasons. The auto-converted LSAN zone names might collide with LSAN zone names in AD0 (in the example, if AD0 contains lsan_for_linux_farm_AD005, this causes a name collision). Fabric OS does not detect or report such name clashes.
Licensing overview Table 69 lists the optionally licensed features that are available in Fabric OS 7.1. TABLE 69 Available Brocade licenses License Description • 10 Gigabit FCIP/Fibre Channel Allows 10 Gbps operation of FC ports on the Brocade 6510or (10G license) 6520 switches or the FC ports of FC16-32 or FC16-48 port blades installed on a Brocade DCX 8510 Backbone.
Page 465
Licensing overview TABLE 69 Available Brocade licenses (Continued) License Description • Advanced FICON Acceleration Allows use of specialized data management techniques and automated intelligence to accelerate FICON tape read and write and IBM Global Mirror data replication operations over distance, while maintaining the integrity of command and acknowledgement sequences.
Page 466
Licensing overview TABLE 69 Available Brocade licenses (Continued) License Description Enterprise ICL Allows you to connect more than four chassis in a fabric using ICLs. You can connect up to four Brocade DCX 8510 Backbones via ICLs without this license. If the number of interconnected chassis using ICLs exceeds four, then all of the chassis using ICLs require the Enterprise ICL license.
Page 467
Licensing overview TABLE 69 Available Brocade licenses (Continued) License Description • Integrated Routing Allows any ports in Brocade 5100, 5300, 6510, 6520, and VA-40FC switches, the Brocade Encryption Switch, or the Brocade DCX, DCX-4S, and DCX 8510 family platforms to be configured as an EX_Port supporting FC-FC routing.
Page 468
Licensing overview TABLE 70 License requirements and location name by feature (Continued) Feature License Where license should be installed FCIP High Performance Extension over FCIP/FC NOTE: Local and attached switches. License is needed on both sides of tunnel. FCIP Trunking Advanced Extension Local and attached switches.
Licensing overview TABLE 70 License requirements and location name by feature (Continued) Feature License Where license should be installed Logical switch No license required. Long distance Extended Fabrics Local and attached switches. NOTE: License is needed on both sides of connection. NPIV No license required.
Brocade 7800 Upgrade license TABLE 70 License requirements and location name by feature (Continued) Feature License Where license should be installed Speed 8 Gbps license needed to support 8 Gbps on Local switch the Brocade 300, 5100, 5300, and VA-40FC switches and embedded switches only.
ICL licensing TABLE 71 Base to Upgrade license comparison (Continued) Feature Base model 7800 Upgrade license Number of FCIP Tunnels Tape Pipelining over FCIP Tunnel ICL licensing Brocade ICL links operate between the core blades of the DCX 8510 Backbone family, or between the core blades of the DCX and DCX-4S Backbones.
ICL licensing ICL 8-link license The ICL 8-link license activates half of the ICL bandwidth for each ICL port on the Brocade DCX platform by enabling only half of the ICL links available. This allows you to purchase half the bandwidth of the Brocade DCX ICL ports initially and upgrade with an additional ICL license to use the full ICL bandwidth later.
8G licensing Example switchShow output if no Enterprise ICL license is installed A message such as the following is displayed if a required EICL license is not installed: ------ Online E-Port segmented,10:00:00:05:33:0d:52:00 (No EICL License)(Trunk master) ------ Online E-Port segmented,10:00:00:05:33:0d:52:00 (No EICL License)(Trunk master) Example switchShow output if maximum number of chassis is reached A message such as the following is displayed if the maximum number of supported chassis is reached:...
Slot-based licensing Slot-based licensing Slot-based licensing is used on the Brocade DCX and DCX 8510 Backbone families to support the FX8-24 blade, and on the Brocade DCX 8510 Backbone family to support the 16 Gbps FC port blades (FC16-24 and FC16-48). License capacity is equal to the number of slots. These licenses allow you to select the slots that the license will enable up to the capacity purchased and to increase the capacity without disrupting slots that already have licensed features running.
10G licensing Assigning a license to a slot Use the following procedure to assign a licence to a slot: 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions in the license class of RBAC commands. 2.
10G licensing After applying a 10G license to the Brocade 6510or 6520 chassis or to a 16 Gbps FC blade, you must also configure the port octet (portCfgOctetSpeedCombo command) with the correct port octet speed group and configure each port to operate at 10 Gbps (portCfgSpeed command). It is necessary to configure the port octet because only certain combinations of port speeds are allowed within the port octet.
Temporary licenses • FICON Management Server (CUP) license • Extended Fabrics license • High Performance Extension over FCIP/FC license • Integrated Routing license • Server Application Optimization license • ISL Trunking license Restrictions on upgrading temporary slot-based licenses If the capacity of the permanent license is equal to or greater than the capacity of the temporary license and you use the same slot assignments, then replacing the temporary license with a permanent license is non-disruptive.
Temporary licenses Expired licenses Once a temporary license has expired, you can view it through the licenseShow command. Expired licenses have an output string of “License has expired”. RASlog warning messages are generated every hour for licenses present in the database which have expired or are going to expire in the next five days.
Viewing installed licenses Viewing installed licenses Use the following procedure to view all installed licenses: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the licenseShow command. Activating a license The transaction key is case-sensitive; it must be entered exactly as it appears in the paperpack. To lessen the chance of error, copy and paste the transaction key.
Removing a licensed feature Use the following procedure to add a licensed feature: 1. Connect to the switch and log in using an account with admin permissions. 2. Activate the license using the licenseAdd command. 3. Verify the license was added by entering the licenseShow command. The licensed features currently installed on the switch are listed.
Ports on Demand 4. Enter the licenseShow command to verify the license is disabled. switch:admin> licenseshow bQebzbRdScRfc0iK: Entry Fabric license Fabric Watch license SybbzQQ9edTzcc0X: Fabric license switch:admin> licenseremove "bQebzbRdScRfc0iK" removing license key "bQebzbRdScRfc0iK" Entering the licenseShow command after the licenseRemove command displays the remaining licenses.
Ports on Demand TABLE 72 List of available ports when implementing PODs Platform Available user ports No POD license POD1 or POD2 present Both POD licenses present Brocade 300 0-15 0-23 Brocade 5100 0-23 0-31 0-39 Brocade 5300 0-47 0-63 0-79 Brocade 5410 0-11...
Ports on Demand First Ports on Demand license - additional 16 port upgrade license SdSSc9SyRSTeXTdn: Second Ports on Demand license - additional 16 port upgrade license SdSSc9SyRSTuXTd3: Full Ports on Demand license - additional 32 port upgrade license ATTENTION If you enable or disable an active port, you will disrupt any traffic and potentially lose data flowing on that port.
Ports on Demand For the embedded switch modules, the Dynamic POD feature detects and assigns ports to a POD license only if the server blade is installed with an HBA present. A server blade that does not have a functioning HBA is treated as an inactive link during initial POD port assignment. For the non-server blade switches, the dynamic assignment occurs when an attached Fibre Channel link transitions to the “link active”...
Ports on Demand switch:admin> licenseport --method dynamic The POD method has been changed to dynamic. Please reboot the switch now for this change to take effect. 3. Enter the reboot command to restart the switch. switch:admin> reboot 4. Enter the licensePort --show command to verify the switch started the Dynamic POD feature. switch:admin>...
Ports on Demand Ports assigned to the full POD license: 0, 9, 10, 11, 12, 13, 14, 15, 16, 21, 22, 23 Reserving a port license You can allocate licenses by reserving and releasing POD assignments to specific ports. Disabled ports are not candidates for automatic license assignment by the Dynamic POD feature.
Page 489
Ports on Demand After a port is assigned to the POD set, the port is licensed until it is manually removed from the POD port set. When a port is released from its POD port set (Base, Single, or Double), it creates a vacancy in that port set.
Page 490
Ports on Demand Fabric OS Administrator’s Guide 53-1002745-02...
ICLs for the Brocade DCX 8510 Backbone family NOTE You cannot interconnect a Brocade DCX Backbone family chassis to a Brocade DCX 8510 Backbone family chassis. Refer to the specific hardware reference manuals for additional information about LED status meanings and ICL connections, including instructions on how to cable ICLs. ICLs for the Brocade DCX 8510 Backbone family Each ICL connects the core blades of two Brocade DCX 8510 chassis and provides up to 64 Gbps of throughput within a single cable.
ICLs for the Brocade DCX Backbone family NOTE Brocade recommends that you have a maximum of eight ICLs connected to the same neighboring domain, with a maximum of four ICLs from each core blade. • The ICLs can connect to either core blade in the neighboring chassis. Unlike the copper ICLs, the QSFP ICLs do not need to be cross-connected.
Virtual Fabrics considerations for ICLs FIGURE 60 DCX-4S allowed ICL connections The following ICL connections are not allowed: • ICL0 ports to ICL0 ports • ICL1 ports to ICL1 ports ICL trunking on the Brocade DCX and DCX-4S ICL trunks form automatically but additional licenses may be required for enabling all ICL ports or for larger ICL configurations.
Supported topologies for ICL connections Supported topologies for ICL connections You can connect the Brocade Backbones in a mesh topology and a core-edge topology. A brief description of each follows. (You can also connect two DCX 8510s point-to-point.) The illustrations in this section show sample topologies. Refer to the Brocade SAN Scalability Guidelines for details about maximum topology configurations.
Supported topologies for ICL connections FIGURE 62 Full nine-mesh topology During an ICL break in the triangular topology, the chassis that has the connections of the other two is the main chassis. Any error messages relating to a break in the topology appear in the RASlog of the main chassis.
Advanced Performance Monitoring overview Restrictions for installing monitors • Advanced Performance Monitoring is not supported on VE_Ports and EX_Ports. If you issue commands for any Advanced Performance Monitoring on VE_Ports or EX_Ports, you will receive error messages. • For the Brocade 8000, Advanced Performance Monitoring is supported only on the FC ports and not on the CEE ports.
End-to-end performance monitoring Access Gateway considerations for Advanced Performance Monitoring EE monitors and frame monitors are supported on switches in Access Gateway mode. Top Talker monitors are not supported on these switches. EE monitors must be installed on F_Ports. Frame monitors can be installed on F_Ports or N_Ports. Refer to the Access Gateway Administrator’s Guide for additional information.
End-to-end performance monitoring Virtual Fabrics considerations: If Virtual Fabrics is enabled, the Brocade DCX, DCX-4S, DCX 8510 and 5300 models allow up to 256 end-to-end monitors on one logical switch. The Brocade 5100, 6510, 6520, and VA-40FC allow up to 341 end-to-end monitors on one logical switch. Supported port configurations for EE monitors You can configure EE monitors on F_Ports and, depending on the switch model, on E_Ports.
End-to-end performance monitoring This monitor (Monitor 1) counts the frames that have an SID of 0x011200 and a DID of 0x021e00. For Monitor 1, RX_COUNT is the number of words from Host A to Dev B, and TX_COUNT is the number of words from Dev B to Host A.
End-to-end performance monitoring The perfSetPortEEMask command sets a mask for the domain ID, area ID, and AL_PA of the SIDs and DIDs for frames transmitted from and received by the port. Figure 65 shows the mask positions in the command. A mask (“ff”) is set on slot 1, port 2 to compare the AL_PA fields on the SID and DID in all frames (transmitted and received) on port 2.
Frame monitoring perfmonitorshow --class monitor_class [slotnumber/]portnumber [interval] Example of displaying an end-to-end monitor on a port at 10-second intervals switch:admin> perfMonitorShow --class EE 4/5 10 Showing EE monitors 4/5 10: Tx/Rx are # of bytes --------- --------- --------- --------- --------- ========= ========= =========...
Frame monitoring NOTE The Advanced Performance Monitoring license is required to use the fmMonitor command. The monitoring functionality also requires the Fabric Watch license. When you configure actions and alerts through the fmMonitor command, Fabric Watch uses these values and generates alerts based on the configuration.
Frame monitoring The value of the offset must be between 0 and 63, in decimal format. Byte 0 indicates the first byte of the Start of Frame (SOF), byte 4 is the first byte of the frame header, and byte 28 is the first byte of the payload.
Frame monitoring Adding frame monitors to a port If the switch does not have enough resources to add a frame monitor to a port, then other frame monitors on that port may have to be deleted to free resources. 1. Connect to the switch and log in using an account with admin permissions. 2.
Frame monitoring Example The following example displays the existing frame types and associated bit patterns on the switch. switch:admin> fmmonitor --show FRAME_TYPE PATTERN ---------------------------------------- scsi 12,0xFF,0x08; scsiread 12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x08,0x28; scsiwrite 12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x08,0x28,0x0A,0x2A; scsirw 12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x08,0x28,0x0A,0x2A; scsi2reserve 12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x16,0x56; scsi3reserve 12,0xFF,0x08;4,0xFF,0x06;40,0xFF,0x5F;41,0xFF,0x01 12,0xFF,0x05; abts 4,0xFF,0x81;40,0xFF,0x81;12,0xFF,0x0;17,0xFF,0x0; baacc 4,0xff,0x84;12,0xff,0x00;17,0xff,00;...
Top Talker monitors Top Talker monitors Top Talker monitors determine the flows (SID and DID pairs) that are the major users of bandwidth (after initial stabilization). Top Talker monitors measure bandwidth usage data in real time and relative to the port on which the monitor is installed. NOTE Initial stabilization is the time taken by a flow to reach the maximum bandwidth.
Top Talker monitors How do Top Talker monitors differ from EE monitors? EE monitors provide counter statistics for traffic flowing between a given SID and DID pair. Top Talker monitors identify all possible SID and DID flow combinations that are possible on a given port and provide a sorted output of the top talking flows.
Top Talker monitors Edge fabric E_Port FC router EX_Port Backbone fabric FIGURE 66 Fabric mode Top Talker monitors on FC router do not monitor any flows Edge fabric E_Port E_Port E_Port FC router EX_Port Backbone fabric FIGURE 67 Fabric mode Top Talker monitors on FC router monitor flows over the E_Port Limitations of Top Talker monitors Be aware of the following when using Top Talker monitors: •...
Top Talker monitors Adding a Top Talker monitor to a port (port mode) 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the perfTTmon add command. perfttmon --add [egress | ingress] [slotnumber/]port The following example monitors the incoming traffic on port 7. perfttmon --add ingress 7 The following example monitors the outgoing traffic on slot 2, port 4 on a Backbone.
Top Talker monitors The output is sorted based on the data rate of each flow. If you do not specify the number of flows to display, then the command displays the top 8 flows or the total number of flows, whichever is less.
Trunk monitoring Deleting all fabric mode Top Talker monitors 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the perfTTmon delete fabricmode command. perfttmon --delete fabricmode All Top Talker monitors are deleted. Trunk monitoring To monitor E_Port (ISL) and F_Port trunks, you can set monitors only on the master port of the trunk.
Performance data collection 1. Connect to the switch and log in using an account with admin permissions. 2. Enter one of the following commands, depending on the action you want to perform: • To save the current EE monitor and frame monitor configuration settings into nonvolatile memory, use the perfCfgSave command.
Ingress Rate Limiting • Ingress Rate Limiting Ingress Rate Limiting restricts the speed of traffic from a particular device to the switch port. Ingress Rate Limiting requires an Adaptive Networking license. See “Ingress Rate Limiting” page 518 for more information about this feature. •...
QoS: SID/DID traffic prioritization Virtual Fabrics considerations If Virtual Fabrics is enabled, the rate limit configuration on a port is on a per-logical switch basis. That is, if a port is configured to have a certain rate limit value, and the port is then moved to a different logical switch, it would have no rate limit applied to it in the new logical switch.
QoS: SID/DID traffic prioritization Table 76 shows a basic comparison between CS-CTL-based and QoS zone-based prioritization. “CS_CTL-based frame prioritization” on page 521 and “QoS zone-based traffic prioritization” on page 523 for detailed information about each type of prioritization scheme. TABLE 76 Comparison between CS_CTL-based and QoS zone-based prioritization CS_CTL-based frame prioritization QoS zone-based traffic prioritization...
CS_CTL-based frame prioritization CS_CTL-based frame prioritization CS_CTL-based frame prioritization allows you to prioritize the frames between a host and target as having high, medium, or low priority, depending on the value of the CS_CTL field in the FC frame header. The CS_CTL field in the FC header can be used to assign a priority to a frame.
Page 522
CS_CTL-based frame prioritization NOTE If a switch is running a firmware version earlier than Fabric OS v6.0.0, the outgoing frames from that switch lose their priority. High-availability considerations for CS_CTL-based frame prioritization If the standby CP is running a Fabric OS version earlier than 6.3.0 and is synchronized with the active CP, then you cannot enable CS_CTL-based frame prioritization on the active CP.
QoS zone-based traffic prioritization Set CSCTL QoS Mode to 1 to enable auto mode, establishing the settings shown in Table 78 page 521. Set CSCTL QoS Mode to 0 to disable auto mode and revert to default settings, shown in Table 77 on page 521.
QoS zone-based traffic prioritization To preserve existing trunk groups, before you install the Adaptive Networking license, manually disable QoS on these ports, as described in “Manually disabling QoS on trunked ports” page 524. Manually disabling QoS on trunked ports NOTE QoS is disabled by default on long-distance 8-Gbps and 16-Gbps ports.
QoS zones The switch automatically sets the priority for the “host,target” pairs specified in the zones according to the priority level (H or L) in the zone name. The flow id allows you to have control over the VC assignment and control over balancing the flows throughout the fabric.
QoS zones NOTE By default, QoS is enabled on 8-Gbps ports, except for long-distance 8-Gbps ports. QoS is disabled by default on all 4-Gbps ports and long-distance 8-Gbps ports. Domain 1 Domain 3 = Low priority = Medium priority = High priority = E_Ports with QoS enabled Domain 2...
QoS zones The following are requirements for establishing QoS over FCRs: • QoS over FC routers is supported in Brocade native mode only. It is not supported in interopmode 2 or interopmode 3. • QoS over FC routers is supported for the following configurations: Edge-to-edge fabric configuration: supported on all platforms.
Setting QoS zone-based traffic prioritization • Traffic prioritization is enforced on the egress ports only, not on the ingress ports. • Traffic prioritization is not supported on 10-Gbps ISLs. • Traffic prioritization is not supported on mirrored ports. • Traffic prioritization is not supported over LSAN zones. The traffic is always medium priority in the ingress edge fabric, the backbone fabric, and the egress edge fabric.
Page 531
Setting QoS zone-based traffic prioritization The portCfgQos command does not affect QoS prioritization. It only enables or disables the link to pass QoS priority traffic. NOTE QoS is enabled by default on all ports (except long-distance ports). If you use the portCfgQos command to enable QoS on a specific port, the port is toggled to apply this configuration, even though the port already has QoS enabled.
Setting QoS zone-based traffic prioritization over FC routers Setting QoS zone-based traffic prioritization over FC routers 1. Connect to the switch in the edge fabric and log in using an account with admin permissions. 2. Create QoS zones in the edge fabric. The QoS zones must have WWN members only, and not D,I members.
Trunking overview Types of trunking Trunking can be between two switches, between a switch and an Access Gateway module, or between a switch and a Brocade adapter. The types of trunking are as follows: • ISL trunking, or E_Port trunking, is configured on an inter-switch link (ISL) between two Fabric OS switches and is applicable only to E_Ports.
Supported configurations for trunking License requirements for trunking All types of trunking require the Trunking license. This license must be installed on each switch that participates in trunking. ATTENTION After you add the Trunking license, to enable trunking functionality, you must disable and then re-enable each port to be used in trunking, or disable and re-enable the switch.
Supported platforms for trunking Trunks operate best when the cable length of each trunked link is roughly equal to the length of the others in the trunk. For optimal performance, no more than 30 meters difference is recommended. Trunks are compatible with both short-wavelength (SWL) and long-wavelength (LWL) fiber-optic cables and transceivers.
Recommendations for trunk groups Recommendations for trunk groups To identify the most useful trunk groups, consider the following recommendations along with the standard guidelines for SAN design: • Evaluate the traffic patterns within the fabric. • Place trunking-capable switches adjacent to each other. This maximizes the number of trunk groups that can form.
Configuring trunk groups Configuring trunk groups After you install the Trunking license, you must re-initialize the ports that are to be used in trunk groups so that they recognize that trunking is enabled. This procedure needs to be performed only once, and is required for all types of trunking.
Displaying trunking information 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the portCfgTrunkPort command to disable trunking on a port. Enter the switchCfgTrunk command to disable trunking on all ports on the switch. Mode 1 enables and mode 0 disables trunking.
EX_Port trunking For additional information on configuring long distance, see “Configuring an extended ISL” page 553. Table 79 summarizes support for Trunking over long-distance for the Backbones and supported blades. TABLE 79 Trunking over long-distance for the Backbones and blades Long-distance mode Distance Number of 2-Gbps ports...
EX_Port trunking Masterless EX_Port trunking EX_Port trunking is masterless except for EX_Ports on Backbones. For the Backbones, Virtual Fabrics must be enabled for masterless EX_Port trunking to take effect. For the fixed-port switches, Virtual Fabrics can be enabled or disabled. If masterless EX_Port trunking is not in effect and the master port goes offline, the entire EX_Port-based trunk re-forms and is taken offline for a short period of time.
F_Port trunking The following is an example of a master EX_Port and a slave EX_Port displayed in switchShow. switch:admin> switchshow Index Slot Port Address Media Speed State ============================================== ee1000 No_Light ee1100 Online EX_Port (Trunk port, master is Slot 2 Port ee1200 Online EX_Port...
F_Port trunking FIGURE 72 Switch in Access Gateway mode without F_Port masterless trunking FIGURE 73 Switch in Access Gateway mode with F_Port masterless trunking NOTE You do not need to map the host to the master port manually, because the Access Gateway will perform a cold failover to the master port.
F_Port trunking Use the following procedure on the edge switch connected to the Access Gateway module to configure F_Port trunking. 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the portCfgShow command to ensure that the ports have trunking enabled. If trunking is not enabled, enter the portCfgTrunkPort port 1 command.
F_Port trunking Enable the trunk on the ports by using the portTrunkArea command. switch:admin> porttrunkarea --enable 3/40-41 -index 296 Trunk index 296 enabled for ports 3/40 and 3/41. 2. On the host side, enable trunking as described in the Brocade Adapters Administrator’s Guide. 3.
Page 547
F_Port trunking TABLE 80 F_Port masterless trunking considerations (Continued) Category Description DCC Policy DCC policy enforcement for the F_Port trunk is based on the Trunk Area; the FDISC requests to a trunk port are accepted only if the WWN of the attached device is part of the DCC policy against the TA.
F_Port trunking TABLE 80 F_Port masterless trunking considerations (Continued) Category Description Trunk Master No more than one trunk master is allowed in a trunk group. The second trunk master will be persistently disabled with the reason "Area has been acquired”. Upgrade There are no limitations on upgrading to Fabric OS v7.0.0 and later if the F_Port is present on the switch.
Displaying F_Port trunking information • If F_Port trunking is enabled on some ports in the default switch, and you disable Virtual Fabrics, all of the F_Port trunking information is lost. • All of the ports in an F_Port trunk must belong to a single trunk group of ports on the platform and must also belong to the same logical switch.
Enabling the DCC policy on a trunk area switch:admin> portdisable 0-2 switch:admin> porttrunkarea --disable 0-2 Trunk index 2 disabled for ports 0, 1, and 2. Enabling the DCC policy on a trunk area After you assign a trunk area, the portTrunkArea command checks whether there are any active DCC policies on the port with the index TA, and then issues a warning to add all the device WWNs to the existing DCC policy with index as TA.
Extended Fabrics device limitations • Optimized switch buffering When Extended Fabrics is installed on gateway switches (with E_Port connectivity from one switch to another), the ISLs (E_Ports) are configured with a large pool of buffer credits. The enhanced switch buffers help ensure that data transfer can occur at near-full bandwidth to use the connection over the extended links efficiently.
Configuring an extended ISL • Dynamic Mode (LD) — LD calculates buffer credits based on the distance measured during port initialization. Brocade switches use a proprietary algorithm to estimate distance across an ISL. The estimated distance is used to determine the buffer credits required in LD (dynamic) extended link mode based on a maximum Fibre Channel payload size of 2,112 bytes.
Configuring an extended ISL portcfglongdistance [slot/]port [distance_level] [vc_translation_link_init] [-distance desired_distance] 6. Repeat step 4 step 5 for the remote extended ISL port. Both the local and remote extended ISL ports must be configured to the same distance_level. When the connection is initiated, the fabric will reconfigure.
Buffer credit management 1. Connect to the switch and log in using an account assigned to the admin role. 2. Disable QoS. switch:admin> portcfgqos --disable [slot/]port If you do not disable QoS, after the second or third Link Reset (LR), ARB fill words display. 3.
Buffer credit management Buffer-to-buffer flow control is flow control between adjacent ports in the I/O path, for example, transmission control over individual network links. A separate, independent pool of credits is used to manage buffer-to-buffer flow control.A sending port uses its available credit supply and waits to have the credits replenished by the port on the opposite end of the link.
Buffer credit management Smaller frame sizes need more buffer credits. Two commands are available to help you determine whether you need to allocate more buffer credits to handle the average frame size. The portBufferShow command calculates the average frames size. The portBufferCalc command uses the average frame size with the speed and link distance to determine the number of buffer credits needed.
Buffer credit management TABLE 82 Fibre Channel data frames Fibre Channel frame fields Field size Start of frame 4 bytes 32 bits Standard frame header 24 bytes 192 bits Data (payload) 0–2,112 bytes 0–16,896 bits 4 bytes 32 bits End of frame 4 bytes 32 bits Total (number bits/frame)
Page 559
Buffer credit management • If QoS is not enabled: (Reserved Buffer for Distance Y) = (X * LinkSpeed / 2) + 6 where X = the distance determined in step 1 (in km). LinkSpeed = the speed of the link determined in step 2. 6 = the number of buffer credits reserved for fabric services, multicast, and broadcast traffic.
Buffer credit management • 8 — the number of reserved buffer credits already allocated to that port. The floor of the resulting number is taken because fractions of a port are not allowed. If you have a distance of 50 km at 1 Gbps, then 484 / (31 – 8) = 21 ports Allocating buffer credits based on average-size frames In cases where the frame size is average, for example 1024 bytes, you must allocate twice the buffer credits or configure twice the distance in the long-distance LS configuration mode.
Buffer credit management Configuring buffers for a single port directly To configure the number of buffers directly, use the -buffers option of the portCfgLongDistance command. Fabric OS uses this value to calculate the total number of buffers according to the following formula: Total Buffers = Configured Buffers + QOS_VC_Credits + Non-data_VC_Credits Seven Virtual Channels (VCs) are required for each QoS port.
Buffer credit management To determine the number of buffers required, perform the following steps: 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the portBufferCalc command and provide values for the distance, port speed, and frame size.
Buffer credit management TABLE 83 Total FC ports, ports per port group, and unreserved buffer credits per port group Switch/blade model Total FC ports (per switch/blade) User port group size Unreserved buffer credits per port group FC8-32 1292/508 FC8-32E 5456 FC8-48 1228/716 FC8-48E...
Buffer credit recovery Buffer credit recovery Buffer credit recovery (CR) allows links to recover after buffer credits are lost when the buffer credit recovery logic is enabled. The buffer credit recovery feature also maintains performance. If a credit is lost, a recover attempt is initiated. During link reset, the frame and credit loss counters are reset without performance degradation.
Buffer credit recovery For an F_Port on a Brocade switch or Access Gateway connected to an adapter, the following conditions must be met: • The Brocade switch or Access Gateway must run Fabric OS v7.1 or later. • Fabric OS must support buffer credit recovery at both ends of the link. •...
Forward error correction on long-distance links The following example enables buffer credit recovery on port 1/20. switch:admin> portcfgcreditrecovery 1/20 -enable Forward error correction on long-distance links Forward error correction (FEC) on user ports is supported for LD and LS long-distance modes. Use the portCfgLongDistance command with the -fecEnable or -fecDisable options to enable or disable FEC, respectively, on a user port.
FC-FC routing overview A Fibre Channel router (FC router) is a switch running the FC-FC routing service. The FC-FC routing service can be simultaneously used as an FC router and as a SAN extension over wide area networks (WANs) using FCIP. You can set up QoS traffic prioritization over FC routers.
FC-FC routing overview • The Backbones have a limit of 128 EX_Ports for each chassis. Refer to the Network OS Administrator’s Guide for supported Network OS platforms. Supported configurations for FC-FC routing FC-FC routing supports the following configurations: • FC router connected to a Fabric OS nonsecured edge fabric. •...
Fibre Channel routing concepts Fibre Channel routing concepts Fibre Channel routing introduces the following concepts: • Fibre Channel router (FC router) A switch running the FC-FC routing service. Refer to “Supported platforms for FC-FC routing” page 570 for a list of platforms that can be FC routers. •...
Fibre Channel routing concepts • Logical SANs (LSANs) An LSAN is defined by zones in two or more edge or backbone fabrics that contain the same devices. You can create LSANs that span fabrics. These LSANs enable Fibre Channel zones to cross physical SAN boundaries without merging the fabrics while maintaining the access controls of zones.
Page 574
Fibre Channel routing concepts • Fabric ID (FID) Every EX_Port and VEX_Port uses the fabric ID (FID) to identify the fabric at the opposite end of the inter-fabric link. The FID for every edge fabric must be unique from the perspective of each backbone fabric.
Fibre Channel routing concepts FC router FC router EX_Port EX_Port Backbone fabric E_Port E_Port Edge SAN 1 Edge SAN 2 = LSAN FIGURE 76 Edge SANs connected through a backbone fabric • Phantom domains A phantom domain is a domain emulated by the Fibre Channel router. The FC router can emulate two types of phantom domains: front phantom domains and translate phantom domains.
Fibre Channel routing concepts Phantom domains A phantom domain is a domain created by the Fibre Channel router. The FC router creates two types of phantom domains: front phantom domains and translate phantom domains. A front phantom domain, or front domain, is a domain that is projected from the FC router to the edge fabric.
Setting up FC-FC routing Identifying and deleting stale xlate domains If a remote edge fabric goes unreachable, the xlate domains created in other edge fabrics for this remote edge fabric are retained and not removed unless there is any disruption in the local edge fabric.
Setting up FC-FC routing 4. Configure IFLs for edge and backbone fabric connection. (Refer to “Inter-fabric link configuration” on page 583.) 5. Modify port cost for EX_Ports, if you want to change from the default settings. (Refer to “FC router port cost configuration” on page 587.) 6.
Backbone fabric IDs RyeSzRScycazfT0G: Integrated Routing license If you are connecting to a Fabric OS or M-EOS fabric and the Integrated Routing license is not installed, you must install it, as described in Chapter 18, “Administering Licensing”. The Integrated Routing license is not required if you are connecting to a Brocade Network OS fabric. 4.
FCIP tunnel configuration ATTENTION In a multi-switch backbone fabric, modification of the FID within the backbone fabric will cause disruption to local traffic. Assigning backbone fabric IDs 1. Log in to the switch or Backbone. 2. Enter the switchDisable command if EX_Ports are online. 3.
Inter-fabric link configuration Refer to the Fibre Channel over IP Administrator’s Guide for instructions on how to configure FCIP tunnels. Inter-fabric link configuration Before configuring an inter-fabric link (IFL), be aware that you cannot configure both IFLs (EX_Ports, VEX_Ports) and ISLs (E_Ports) from a backbone fabric to the same edge fabric. Configuring an inter-fabric link involves disabling ports and cabling them to other fabrics, configuring those ports for their intended uses, and then enabling the ports.
Page 584
Inter-fabric link configuration Hash Algorithm: N/A Edge fabric's primary wwn: N/A Edge fabric's version stamp: N/A This port can now connect to another switch. The following example configures an EX_Port for connecting to a Brocade Network OS fabric. The -m 5 option indicates Network OS connectivity. switch:admin>...
Page 585
Inter-fabric link configuration 8. After identifying such ports, enter the portCfgPersistentEnable command to enable the port, and then the portCfgShow command to verify the port is enabled. switch:admin> portcfgpersistentenable 7/10 switch:admin> portcfgshow 7/10 Area Number: Speed Level: AUTO Trunk Port Long Distance VC Link Init Locked L_Port...
Page 586
Inter-fabric link configuration Edge fabric's primary wwn: N/A Edge fabric's version stamp: N/A portDisableReason: None portCFlags: 0x1 portFlags: 0x1 PRESENT U_PORT EX_PORT portType: 10.0 portState: 2 Offline portPhys: No_Module portScn: port generation number: portId: 014a00 portIfId: 4372080f portWwn: 20:4a:00:60:69:e2:03:86 portWwn of device(s) connected: Distance: normal portSpeed: N4Gbps...
FC router port cost configuration Port cost considerations The router port cost has the following considerations: • Router port sets are defined as follows: 0–7 and FCIP Tunnel 16–23 8–15 and FCIP Tunnel 24–31 • The router port cost does not help distinguish one IFL (or EX_ and VEX_Port link) from another, if all the IFLs are connected to the same port set.
EX_Port frame trunking configuration ------------------------ 1000 1000 1000 7/10 1000 7/13 1000 10/0 1000 You can also use the fcrRouteShow command to display the router port cost. To display the router port cost for a single EX_Port, enter the fcrRouterPortCost command with a port and slot number.
LSAN zone configuration For information about setting up E_Port trunking on an edge fabric, refer to Chapter 22, “Managing Trunking Connections”. LSAN zone configuration An LSAN consists of zones in two or more edge or backbone fabrics that contain the same devices. LSANs provide selective device connectivity between fabrics without forcing you to merge those fabrics.
LSAN zone configuration NOTE The "LSAN_" prefix must appear at the beginning of the zone name. LSAN zones may not be combined with QoS zones. Refer to “QoS zones” on page 525 for more information about the naming convention for QoS zones. To enable device sharing across multiple fabrics, you must create LSAN zones on the edge fabrics (and optionally on the backbone fabric as well), using normal zoning operations to create zones with names that begin with the special prefix “LSAN_”, and adding host and target port WWNs from...
Page 592
LSAN zone configuration 3. Enter the zoneCreate command to create the LSAN lsan_zone_fabric75, which includes the host. switch:admin> zonecreate "lsan_zone_fabric75", "10:00:00:00:c9:2b:c9:0c" 4. Enter the zoneAdd command to add Target A to the LSAN. FID75Domain5:admin> zoneadd "lsan_zone_fabric75", "50:05:07:61:00:5b:62:ed" 5. Enter the cfgAdd or cfgCreate and cfgEnable commands to add and enable the LSAN configuration.
LSAN zone configuration This action will replace the old zoning configuration with the current configuration selected. Do you want to enable 'zone_cfg' configuration (yes, y, no, n): [no] y zone config "zone_cfg" is in effect Updating flash ... 11. Log in as an admin and connect to the FC router. 12.
LSAN zone configuration Setting the maximum LSAN count You can set the maximum number of LSAN zones, or LSAN count, that can be configured on the edge fabrics. By default, the maximum LSAN count is set to 3000. You can increase the maximum LSAN count to 5000 without disabling the switch.
Page 595
LSAN zone configuration You can specify two types of tags: • Enforce tag – Specifies which LSANs are to be enforced in an FC router. • Speed tag – Specifies which LSANs are to be imported or exported faster than other LSANs. The LSAN tags are persistently saved and support configupload and configdownload.
LSAN zone configuration lsan_f2_f1 (H1, D1) lsan_f2_f3 (H1, D2) The LSAN in the host fabric does not need the tag. 3. In Edge fabric 1, configure the following LSAN: lsan_super_f1_f2 (H1, D1) 4. In Edge fabric 3, configure the following LSAN: lsan_super_f3_f2 (H1, D2) 5.
Page 597
LSAN zone configuration • The tag is from 1 through 8 alphanumeric characters. • You can configure only one Speed tag on an FC router, and up to eight Enforce tags on an FC router. The maximum number of tags (Enforce and Speed) on an FC router is eight. •...
LSAN zone configuration 1. Log in to the FC router as admin. 2. Enter the fcrlsan --remove command to remove an existing LSAN tag. If you remove an Enforce LSAN tag, you must disable the switch first. Example of removing an Enforce LSAN tag sw0:admin>...
LSAN zone configuration With LSAN zone binding, each FC router in the backbone fabric stores only the LSAN zone entries of the remote edge fabrics that can access its local edge fabrics. The LSAN zone limit supported in the backbone fabric is not limited by the capability of one FC router. In addition, due to the lower LSAN count, the CPU consumption by the FC router is lower.
Page 600
LSAN zone configuration TABLE 85 LSAN information stored in FC routers, with and without LSAN zone binding WIthout LSAN zone binding With LSAN zone binding FC router 1 FC router 2 FC router 3 FC router 4 FC router 1 FC router 2 FC router 3 FC router 4...
Page 601
LSAN zone configuration FC router matrix definition Depending on the structure of the backbone fabric, you can specify pairs of FC routers that can access each other. For the metaSAN shown in Figure 81, the following FC routers can access each other: •...
Page 602
LSAN zone configuration Setting up LSAN zone binding 1. Log in to the FC router as admin. 2. Enter the following command to add a pair of FC routers that can access each other: FCR:Admin> fcrlsanmatrix --add -fcr wwn1 wwn2 The variables wwn1 and wwn2 are the WWNs of the FC routers.
Proxy PID configuration Proxy PID configuration When an FC router is first configured, the PIDs for the proxy devices are automatically assigned. Proxy PIDs (as well as phantom domain IDs) persist across reboots. The most common situation in which you would set a proxy PID is when you replace a switch. If you replace the switch and want to continue using the old PID assignments, you can configure it to do so;...
Inter-fabric broadcast frames Inter-fabric broadcast frames The FC router can receive and forward broadcast frames between edge fabrics and between the backbone fabric and edge fabrics. Many target devices and HBAs cannot handle broadcast frames. In this case, you can set up broadcast zones to control which devices receive broadcast frames. (Refer to “Broadcast zones”...
Page 605
Resource monitoring You can monitor FC router resources using the fcrResourceShow command. The fcrResourceShow command shows FCR resource limits and usage and includes the following: • LSAN zones and LSAN devices — The information shows the maximum versus the currently used zones and device database entries.
FC-FC routing and Virtual Fabrics 20 | 21 | 22 | 23 | FC-FC routing and Virtual Fabrics If Virtual Fabrics is not enabled, FC-FC routing behavior is unchanged. If Virtual Fabrics is enabled, then in the FC-FC routing context, a base switch is like a backbone switch and a base fabric is like a backbone fabric.
FC-FC routing and Virtual Fabrics • Although the Brocade 6510 and 6520 supports up to four logical switches, if you are using FC-FC routing, they can have a maximum of three logical switches. Logical switch configuration for FC routing Figure 82 shows an example of two chassis partitioned into logical switches.
FC-FC routing and Virtual Fabrics Edge fabric Fabric 128 Edge fabric Fabric 15 Fabric 1 Backbone fabric Fabric 8 FIGURE 83 Logical representation of EX_Ports in a base switch Backbone-to-edge routing with Virtual Fabrics Backbone-to-edge routing is not supported in the base switch, unless you use a legacy FC router. A legacy FC router is an FC router configured on a Brocade 7500 switch.
Upgrade and downgrade considerations for FC-FC routing Physical chassis 1 Physical chassis 2 Logical switch 1 Logical switch 5 (Default logical switch) (Default logical switch) Fabric ID 128 Fabric ID 128 Edge fabric Logical switch 2 Logical switch 6 FID 20 Fabric ID 1 Fabric ID 1 Allows XISL use...
Page 610
Displaying the range of output ports connected to xlate domains 1. Log in to a switch in the edge fabric. 2. Enter the lsDbShow command on the edge fabric. In the lsDbShow output, ports in the range from 129 through 255 are the output ports on the front domain.
Appendix Port Indexing This appendix shows how to use the switchShow command to determine the mapping among the port index, slot/port numbers, and the 24-bit port ID (PID) on any Brocade Backbone. Enter the switchShow command without parameters to show the port index mapping for the entire platform. Enter the switchShow -slot command for port mapping information for the ports on the blade in a specific slot.
Page 612
Port Indexing ------ No_Module ------ No_Module ------ No_Module ------ No_Module ------ No_Module ------ No_Module ------ No_Module ------ No_Module ------ Online FC E-Port 10:00:00:05:1e:39:e4:5a trunkmaster name (Trunk master) ------ Online FC E-Port 10:00:00:05:1e:39:e4:5a trunkmaster name (Trunk master) ------ Online FC E-Port 10:00:00:05:1e:39:e4:5a trunkmaster name (Trunk master) ------ Online...
Page 613
Port Indexing Example of port indexing on an FC8-64 blade on a Brocade DCX-4S Backbone. The Brocade DCX-4S does not need a mapping of ports on port blades because it is a one-to-one mapping. The order is sequential starting at slot 1 port 0 all the way through slot 8 port 255 for the FC8-64 blade.
Page 614
Port Indexing Example of port indexing on an FS8-18 blade on a DCX 8510-8 Backbone This example shows the truncated switchShow output for an FS8-18 encryption blade on the Brocade DCX 8510-8 Backbone. The assignment of port index numbers to PIDs will vary depending on blade type, platform type, and slot number.
Zeroization functions TABLE 86 Zeroization behavior (Continued) Keys Zeroization CLI Description FCSP Challenge secAuthSecret –-remove The secAuthsecret -–create command is used to input Handshake the keys, and the secAuthsecret -–remove command is Authentication Protocol used to remove and zeroize the keys. All the (CHAP) Secret DHCHAP/FCAP authenticated ports are disabled after zeroization.
FIPS mode configuration Power-on self tests A power-on self-test (POST) is invoked by powering on the switch in FIPS mode and does not require any operator intervention. If any KATs fail, the switch goes into a FIPS Error state, which reboots the system to start the test again.
FIPS mode configuration TABLE 87 FIPS mode restrictions (Continued) Features FIPS mode Non-FIPS mode IPsec Usage of AES-XCBC, MD5, and DH group 1 No restrictions are blocked. LDAP CA CA certificate must be available. CA certificate is optional. Common certificate for FCAP and Not supported Supported HTTPS authentication...
Page 619
FIPS mode configuration Setting up LDAP for FIPS mode 1. Log in to the switch using an account with admin or securityadmin permissions, or an account with OM permissions for the RADIUS and switch configuration RBAC classes of commands. 2. Enter the dnsConfig command to configure the DNS on the switch. Example of setting the DNS switch:admin>...
FIPS mode configuration 4. Set up LDAP according to the instructions in “LDAP configuration and Microsoft Active Directory” on page 162, and then perform the following additional Microsoft Active Directory settings a. To support FIPS-compliant TLS cipher suites on the Microsoft Active Directory server, allow the SCHANNEL settings listed in Table TABLE 89...
Preparing a switch for FIPS Exporting an LDAP switch certificate This procedure exports the LDAP CA certificate from the switch to the remote host. 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the PKI RBAC class of commands.
Preparing a switch for FIPS Overview of steps 1. Remove legacy OpenSSH DSA keys. 2. Optional: Configure the RADIUS server or the LDAP server. 3. Optional: Configure any authentication protocols. 4. For LDAP only: Install an SSL certificate on the Microsoft Active Directory server and a CA certificate on the switch for using LDAP authentication.
Page 623
Preparing a switch for FIPS 4. Optional: Set the authentication protocols. a. Enter the authUtil --set -h sha1 command to set the hash type for MD5, which is used in the DH-CHAP and FCAP authentication protocols. b. Enter the authUtil --set -g n command (where n represents the DH group) to set the DH group to 1, 2, 3, or 4.
Preparing a switch for FIPS • System services: No • cfgload attributes: Yes • Enforce secure config Upload/Download: Press Enter to accept the default. • Enforce firmware signature validation: Yes Example switch:admin> configure Not all options will be available on an enabled switch. To disable the switch, use the "switchDisable"...
Preparing a switch for FIPS NOTE Passwords of the default accounts (admin and user) should be changed after every zeroization operation to maintain FIPS 140-2 compliance. 3. Power-cycle the switch. Displaying FIPS configuration 1. Log in to the switch using an account with admin or securityadmin permissions, or a user account with OM permissions for the FCIPCfg RBAC class of commands.
Page 626
Preparing a switch for FIPS Fabric OS Administrator’s Guide 53-1002745-02...
Appendix Hexadecimal Conversion Hexadecimal overview Hexadecimal, also known as hex, is a numeral system with a base of 16, usually written by means of symbols 0–9 and A–F (or a–f). Its primary purpose is to represent the binary code that computers interpret in a format easier for humans to remember.
Page 630
Hexadecimal Conversion Fabric OS Administrator’s Guide 53-1002745-02...
Page 631
Index Numerics configuring F_Port trunking on considerations for Advanced Performance Monitoring 10 Gbps operation on an FC port, enabling F_Port trunking for 10-bit addressing mode F_Port trunking requirements on – 10G license N_Port failover with FA-PWWN 128-bit encryption, in browser shared secrets 16-link ICL license accessing...
Page 632
policy distribution to other switches Admin Domain number and domain ID – policy management Admin Domains policy members about removing policy member access levels resolving conflicting ACL policies ACL policy considerations activating activating ACL policy changes AD list Admin Domains Microsoft Active Directory IP Filter policy OpenLDAP...
Page 633
switch members aptPolicy command switch port members assigning user-defined roles switch WWN assigning users to Admin Domains switching context audit log system-defined configuration TACACS+ service configuring for specific event classes TI zone considerations auditCfg command transaction model auditDump command trunk area AUTH module, Virtual Fabric considerations user-defined AUTH policy...
Page 639
access methods, Web Tools restrictions audit log authentication telnet – authentication policy consistency policies, matching fabric-wide browser security certificates consistency policies, non-matching fabric-wide compression console session on serial port date and time control processor. See: CP. device authentication – converting hexadecimal numbers device-switch connection core blades DHCP...
Page 640
frame redirect zones IP Filter policy LDAP certificates D_Port, described logical switches daemon processes and High Availability private key from switch daemon, tac_plus public key from switch daemons automatically restarted rule from an IP Filter policy date and time TI zones date change license restriction zone configurations date command...
Page 641
compression overview CS_CTL-based frame prioritization rebalancing triggers DHCP See also: Dynamic Load Sharing. F_Port trunking dlsReset command failover in TI zones, considerations dlsSet command in-flight encryption dlsShow command ingress rate limiting dnsConfig command ISL trunking domain ID local switch protection and Admin Domain number NPIV –...
Page 642
edge-to-edge routing restrictions using SSL EE monitors viewing configuration about encryption keys, expiration adding end-to-end (EE) monitoring clearing statistic counters defined end-to-end monitors deleting deleting displaying counters restoring configuration maximum number saving configuration setting a mask for setting a mask supported port configurations for end-to-end performance monitoring effective AD configuration...
Page 643
displaying information and Virtual Fabrics masterless configuring for Brocade adapters supported configurations and platforms considerations Exchange Link Parameters mode. See: ELP mode. for access gateways for Brocade adapters exchange-based routing fabric expired licenses access removing adding Top Talker monitors expiry keys addresses.
Page 644
– command line interface and Virtual Fabrics default roles backbone-to-edge feature interaction with Virtual Fabrics configurations supported interaction with Virtual Fabrics edge-to-edge policies fabric mode Top Talker monitors protocols supported license requirements security protocols supported platforms supported – user accounts routing service –...
Page 645
See also: FC. restrictions Fibre Channel Authentication Protocol. See: FCAP. fipsCfg command Fibre Channel Common Transport (FC-CT) protocol service, Firefox described root certificate installation and verification SSL support Fibre Channel fabrics, and port ID – firmware Fibre Channel Over IP service. See: FCIP. –...
Page 646
port configurations supported configuring port restrictions Fabric OS user setup FL_Port, described user, adding vendor attributes FLOGI See also: RADIUS and Linux. defined FSPF FC-SP bit setting described process number of routes supported rejected path calculation request frame header value traffic isolation routing rules fmMonitor command FSPF-1009 RASLOG message...
Page 647
– TACACS+ indexing ports home LF in-flight compression and port decommissioning Microsoft Active Directory in-flight encryption OpenLDAP configuring RADIUS disabling TACACS+ license host syslog, verifying port decommissioning hosts, accessing restrictions – HTTPS protocol in-flight encryption and compression described on EX_Ports secure protocol overview –...
Page 648
policy rules islShow command policy rules using service names saving policy supported actions supported protocols supported services and port numbers Java IP interface for chassis management installing root certificate in plugin IP sec installing root certificate to plugin algorithms support for SSL Authentication Header protocol supported version configuration on the management interface...
Page 649
in FIPS mode ICL 8-link installing certificates in-flight encryption IPv4 and IPv6 support installation requirements and location non-FIPS mode restrictions Integrated Routing role mapping and OpenLDAP preserving role mapping, and Microsoft Active Directory purchasing keys secure service removing expired LDAP server removing features requirements for SID/DID prioritization adding...
Page 650
blocked basic configuration values chargen changing to a base switch daytime commanding in a different context discard connected devices and echo creating deleting rexec displaying configuration rlogin DLS effect on fabric IDs and rstats management model rusers moving ports time multiple FIDs blocked list number...
Page 651
management server msplMgmtActivate command displaying ACL msplMgmtDeactivate command viewing database mstdDisable command – management server database mstdEnable command Management server, described mstdReadConfig command managing – Admin Domains IP Filter thresholds – trunking connections – user accounts N_Port ID Virtualization. See: NPIV. –...
Page 652
null encryption support for IKE policies passwordless firmware download passwords – boot PROM Backbone with recovery string Backbone without recovery string switch with recovery string – on-demand ports switch without recovery string activating local user accounts available ports – policies for disabling dynamic rules displaying installed licenses...
Page 653
disabling ACL deleting enabling ACL distribution Virtual Fabrics activating IP Filter platforms, FC-FC routing supported adding rule to an IP Filter policy authentication restrictions PLOGI cloning an IP Filter defined creating DCC creating FCS enabling ports creating for IP Filter releasing a port from a set creating SCC reserving a port license...
Page 654
deactivation port area ID decommissioning port area IDs, swapping deleting Top Talker monitor on port decommissioning disabling on port with in-flight encryption/compression disabling dynamic POD port groups for trunking disabling on blades port identifier. See also: PID. displaying license assignments port index displaying the top n bandwidth-using flows –...
Page 655
portDecom command secure HTTPS portDisable command portEnable command SNMPv1 portEncCompShow command SNMPv2 PortFecCap SNMPv3 portLoginShow command SSHv2 portName command SNMP, described – ports on demand SSH, described activating available ports SSL, described disabling dynamic telnet displaying installed licenses protocols dynamic authentication enabling dynamic IP sec...
Page 656
QoS zone-based traffic prioritization RBAC disabling Admin Domain considerations High Availability considerations and Fabric OS limitations and restrictions role permissions setting recommendations for trunk groups ssetting over FC routers recovering a device supported configurations –?? redirecting frames trunking considerations Registered State Change Notification Virtual Fabrics considerations rejecting distributed user databases locally QoS zones...
Page 658
length serial port, console session setting Server Application Optimization. See: SAO. viewing list of sessions, maximum allowed secure copy protocol. See: SCP. setContext command Secure Fabric OS policies setting secure LDAP changing passwords secure protocol chassis configurations HTTPS chassis management IP interface items needed to deploy date default zone mode...
Page 659
security levels supported browsers SNMPv1 supportSave command secure protocol – swapping blades SNMPv2 SW-EXTTRAP secure protocol switch SNMPv3 access secure protocol access methods, Web Tools switch and chassis context enforcement ACL policy distribution v1 support activation and deactivation v3 support adding public key Virtual Fabrics and applications used...
Page 660
switch database distribution setting enabling unique names for logical home Virtual Fabric user-defined accounts homeAD viewing status policy threshold values LINUX based switch authentication mode, setting modifying overview switch authentication policy password expiration, configuring See also: AUTH. user, adding Switch Connection Control. See: SCC. vendor attributes –...
Page 661
setting interactively transaction model for managing Admin Domains – time zone settings transform set, and IP sec time, synchronizing local and external transform set, defined – time-based licenses traps Top Talker monitors adding on all switches in fabric SNMP adding to aport (port mode) trunk area and admin domains and FC-FC routing trunk area, enabling DCC policy on...
Page 662
U_Port, described validating a zone unblocking telnet access validating Admin Domain members universal temporary license VE_Ports defined described described routing policy extending XISL and FX8-24 shelf life verification check unlocking an account verifying unordered frame delivery, restoring device connectivity upgrading firmware High Availability features host syslog upgrading temporary slot-based licenses, restrictions...
Page 663
configDownload restrictions SCC policy considerations configUpload restrictions supported platforms configuration management TACACS+ service – configuring SNMP for TI zone considerations considerations with traffic isolation over FCR for Adv. Perf. Monitoring XISL, allowing on logical switches for WWN-based PID assignment zone alias considerations considerations for ICLs zone database size considerations ContextRoleList...
Page 664
objects optimizing resources zeroization functions for FIPS QoS zones, defined zeroizing for FIPS removing members zone from a configuration access mode, viewing current replacing member accessing saved zone configuration, defined adding a new switch or fabric schemes adding members setting default zoning mode administering security special alias...
Page 665
zoneRemove command zoneShow command zoning – advanced advanced commands defined enforcement on logical ports overview Fabric OS Administrator’s Guide 53-1002745-02...
Page 666
Fabric OS Administrator’s Guide 53-1002745-02...