Page 2
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. USE, COPYING, AND DISTRIBUTION OF ANY DELL SOFTWARE DESCRIBED IN THIS PUBLICATION REQUIRES AN APPLICABLE SOFTWARE LICENSE. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA.
CONTENTS Figures Tables Preface Chapter 1 Introduction Overview.....................12 ScaleIO component requirements............... 12 ScaleIO cluster components............12 Physical server requirements............13 Supported operating systems............13 ScaleIO Gateway server requirements........... 13 Prerequisites....................14 Connectivity...................14 Java....................15 Hardware and operating systems........... 15 ScaleIO packages................15 Supported Internet browsers............
Page 4
Installing the iDRAC Service Module on an ESXi server......66 Install the iSM using SSH.............. 66 Install the iSM using the vSphere CLI..........67 DTK - Hardware Update Bootable ISO............68 Dell OpenManage DRAC Tools (RACADM)........68 Recommended BIOS and firmware settings........71 Troubleshooting the Hardware ISO..........73 Additional resources..............75...
FIGURES vSphere client, with Power off option selected............28 Networking pane - VMkernel NICs tab............... 29 Networking pane - Port groups tab................29 PowerEdge R640 and R740xd PCI slots, integrated NICs and BMC port locations..43 Hardware Configuration and Operating System Installation Guide - ESXi Servers...
Page 6
FIGURES Hardware Configuration and Operating System Installation Guide - ESXi Servers...
TABLES Server physical requirements..................13 Supported operating systems - ScaleIO components..........13 ScaleIO management IP network................37 ScaleIO data IP network for Subnet #1...............38 ScaleIO data IP network for Subnet #2..............38 R740xd server configuration for 25 GbE, 32xSAS, SFP28..........44 R740xd server configuration for 25GbE, 32xSAS, SFP28 (converged)......
Page 8
TABLES Hardware Configuration and Operating System Installation Guide - ESXi Servers...
Preface As part of an effort to improve its product lines, EMC periodically releases revisions of its software and hardware. Therefore, some functions described in this document might not be supported by all versions of the software or hardware currently in use. The product release notes provide the most up-to-date information on product features.
Page 10
Preface Vertical bar indicates alternate selections - the bar means “or” Braces enclose content that the user must specify, such as x or y or Ellipses indicate nonessential information omitted from the example Where to get help EMC support, product, and licensing information can be obtained as follows: Product information For documentation, release notes, software updates, or information about EMC products, go to EMC Online Support at https://support.emc.com.
CHAPTER 1 Introduction This section describes the prerequisites for configuring the hardware and the operating system in a ScaleIO Ready Node environment. It also provides a top-level view of the tasks involved. Overview......................12 ScaleIO component requirements............... 12 Prerequisites.......................14 Supported hardware configurations..............16 Introduction...
Introduction Overview To prepare a server to be part of a ScaleIO Ready Node environment, you are required to perform the following steps. 1. Ensure that the environment meets the prerequisites. 2. Configure the hardware, including the BIOS, BMC (iDRAC) IP address, and ISO installation.
Introduction Physical server requirements Table 1 Server physical requirements Component Requirement Port specifications The following TCP ports are not used by any other application, and are open in the local firewall of the server: MDM: 6611 and 9011 SDS: 7072 (for multiple SDS, ports 7073-7076) ScaleIO Gateway (includes REST Gateway, Installation Manager, and SNMP trap sender): 80 and 443 Light Installation Agent (LIA): 9099...
Introduction Component Requirement You can change the default ports. For more information, see “Changing default ports” in the user documentation. Supported web browsers Internet Explorer 10, or later Firefox, version 42, or later Chrome, version 45, or later Java requirements 1.8 or later, 64-bit Other For a Windows Gateway, the Windows Management Instrumentation service must be...
Introduction Java These Java versions are required: Required version ScaleIO GUI Java 1.8 latest update BMC (iDRAC) management client (Java Java 1.7 latest update console) In your internet browser, ensure that pop-ups are enabled for Java console features. Hardware and operating systems Ensure that all hardware is assembled as described in the ScaleIO Ready Node Hardware Installation Guide .
Introduction Supported hardware configurations The supported hardware configurations are listed in the EMC Simple Support Matrix (ESSM) for your software version. Download the ESSM from https://elabnavigator.emc.com/eln/elnhome. Click Simple Support Matrices Storage and select the desired product and version. > The following points apply, as well: Boot drive - All configurations have two 120 GB BOSS boot drives (configured as RAID 1).
CHAPTER 2 Configuring the Hardware This section describes how to configure the hardware, set BMC (iDRAC) IP addresses, and map the ISO for servers in a ScaleIO Ready Node environment. Set up the BMC (iDRAC) IP address and BIOS...........18 Verify the status of the system hardware, storage controller, and disks....
Configuring the Hardware Set up the BMC (iDRAC) IP address and BIOS Set up the BMC (iDRAC) IP address and set up or validate the BIOS on the ScaleIO Ready Node servers. Before you begin Ensure that you have access to, or have the details for: The KVM console The server BMC (iDRAC) IP address The server BMC (iDRAC) subnet mask...
Page 19
Configuring the Hardware Enable IP IPv4 = Enabled Enable DHCP = Disabled Static IP Address = Static IP address Static Gateway = Gateway IP address Static Subnet Mask = Subnet mask IP address d. From the IPv6 Settings pane, configure the IPv6 parameter values for the BMC (iDRAC) port.
Procedure 1. From a browser, go to http:// <BMC/iDRAC_IP_address> . The DELL Console Login window is displayed. 2. Type the user name and password, then click Login. The Dashboard displays the high-level status of all hardware devices in the System Health pane.
Procedure <iDRAC_IP_address> . 1. From your Internet browser, go to https:// 2. In the DELL Console Login window, type the user name and password, and click Login. 3. From the dashboard, click Launch Virtual Console to start a console session.
Updating the BIOS, firmware and settings ScaleIO Ready Node deployments require specific versions of drivers, BIOS, and firmware that have been qualified by Dell EMC. If the servers do not have the correct versions, you must update them. A variety of factors can influence a mismatch between the required versions and the versions installed on the servers, such as firmware updates post server shipment, or a FRU replacement with a different firmware version than in the warehouse.
Configuring the Hardware Update the hardware using the iDRAC virtual console The iDRAC Virtual KVM console and Virtual CDROM features, provided by the iDRAC Enterprise license, eliminate the need for physical access to the ScaleIO Ready Node servers. The Hardware ISO can be attached as a remote/Virtual CDROM image and configured to perform hardware updates to the ScaleIO Ready Node firmware.
Page 24
12. For each ScaleIO Ready Node server, after the updates are finalized, clear the iDRAC job queue using the iDRAC GUI: <iDRAC_IP_address> . a. From your Internet browser, go to https:// b. In the DELL Console Login window, type these credentials: username: root password: <password> (for Dell-supplied nodes) or password (for EMC- supplied nodes) c.
CHAPTER 3 Installing the Operating System This section describes how to install the operating system. System requirements..................26 Driver support....................26 Rebuild the M.2 cards as a RAID device............. 26 Revert the AMS ESXi ISO Image to a native ESXi Image........27 Map the ISO file on a ScaleIO Ready Node server..........30...
Ensure that correct drivers are installed in your system. The best source for supported storage drivers is the Dell customized ISO for ESXi 6.5 U1. If this cannot be used, consult the VMware support matrix for Dell products on the VMware website.
Installing the Operating System 3. On the System Setup Main Menu screen, select Device Settings AHCI > Controller in Slot 2: BOSS-S1 Configuration Utility. 4. On the BOSS-S1 Configuration Utility screen, select Physical/Virtual Disk Information to verify the disk configuration. 5.
Installing the Operating System username: root password: Scaleio123 5. If a Remember Password popup appears, click Never remember password for this site. 6. If the Help us improve the VMware Host Client dialog appears, clear the Join CEIP check box, and then click OK. The vSphere client main screen appears.
Installing the Operating System 12. Right-click ScaleIO_Internal and select Remove. 13. In the confirmation dialog box, click Remove. 14. In the right pane, in the Vmkernel NICs tab, right-click vmk1 and select Remove. Figure 2 Networking pane - VMkernel NICs tab 15.
5. To verify the ISO file selection, click Virtual Media and view the ISO selection in the drop down list. 6. Reset the server: a. In the main DELL System Setup console window, select Power Reset > System (warm boot) to display the list of keyboard keys with related functionality.
Page 31
Installing the Operating System IP address of VMware vCenter server User name and password for accessing the vCenter with the vSphere Web Client Before deploying ScaleIO systems, you must configure VMware security features. This procedure describes how to do so with the vSphere web client. Note Ensure that the remote connection is established within 1-2 minutes, otherwise, the lockdown mode is enabled on the ESXi servers, preventing you from subsequent...
Page 32
Installing the Operating System c. Using the Search ( ) field, filter the following security parameters: Set Security.AccountLockFailures = The parameter value defines the maximum number of failed login attempts allowed, before locking out the user's account. The zero (0) value disables account locking.
Page 33
Installing the Operating System Note The parameter option of Lockdown Mode must be Disabled. If it is not, Enable to Disable , click Edit change it. To change the parameter option from in the Lockdown Mode pane and change the parameter value to Disable .
Page 34
Installing the Operating System Hardware Configuration and Operating System Installation Guide - ESXi Servers...
CHAPTER 4 Network Architecture and Physical Connectivity This section provides information regarding networking requirements for ScaleIO Ready Node, as well as connecting a ScaleIO Ready Node server to one or more switches. Networking connectivity architecture and cabling best practice (R640, R740XD servers)......................36 Network prerequisites..................36...
Network Architecture and Physical Connectivity Networking connectivity architecture and cabling best practice (R640, R740XD servers) This topic describes connectivity architecture, cabling best practice information, and cable connection examples from typical ScaleIO Ready Node configurations in order to help you plan your network. Note If you are not familiar with ScaleIO system architecture, refer to "System Architecture"...
Network Architecture and Physical Connectivity Note Each node has four 10 GbE ports. Two are used by for internal network communication. The other two ports are used for VMware high availability, DRS, and client network traffic. The switches must have sufficient available network ports to accommodate the following: Data network 10 GbE switches: Two, 10 GbE ports per node, per switch Management network switches: One, 1 GbE and one BMC (iDRAC) port per...
Network Architecture and Physical Connectivity Note If the AMS is not on the same subnet as the SVM management IP addresses, the Discovery server used as a proxy must reside on the same switch network broadcast domain (Physical or VLAN), in order to allow seamless detection and MGMT IP address configuration of the SVM and ESXi IP addresses of all nodes in the system.
Page 39
Network Architecture and Physical Connectivity Note The subnet mask and default gateway of the data networks cannot be modified after initial configuration. If you plan to have 1024 nodes in your ScaleIO system, plan for a subnet allowing 1025 IP addresses for SVM, and 1024 IP addresses for hosts per data network.
Page 40
Network Architecture and Physical Connectivity Hardware Configuration and Operating System Installation Guide - ESXi Servers...
CHAPTER 5 Port Configuration This section describes the port designations for ScaleIO Ready Node servers and provides instructions for configuring the ports. Determine the 2U1N node type................42 PowerEdge R640 and R740xd server NIC and BMC (iDRAC) port locations..42 PowerEdge R640 and R740xd port designations - ESXi........43 Port Configuration...
Procedure 1. From your Internet browser, go to http:// <BMC/iDRAC_IP_address> . 2. In the DELL Console Login window, type the user name and password, and click Login. 3. In the navigation pane, select System Inventory Hardware inventory, and >...
Port Configuration Figure 4 PowerEdge R640 and R740xd PCI slots, integrated NICs and BMC port locations Integrated (iDRAC) NICs iDRAC Integrated (iDRAC) NICs Data cables are connected to two switches, for high availability, via the nodes' 10 G or 25 G ports. Management interfaces are connected to a switch on a separate management network using onboard and BMC (iDRAC) ports.
Port Configuration Table 8 R740xd server configuration for 25GbE, 12xNVMe, SFP28 (continued) Description ScaleIO ScaleIO ScaleIO ScaleIO ScaleIO Not in Use (iDRAC) Data1 Client Client Data2 Managemen Monitoring Network Network t Network Port1 Port2 10 G left and right Speed 1 GB 25 GB 25 GB...
Port Configuration Table 15 R740xd server configuration for 10GbE, 12xNVMe, SFP+ (converged) (continued) Description ScaleIO ScaleIO ScaleIO ScaleIO ScaleIO Not in Use (iDRAC) Data1 Client Client Data2 Managemen Monitoring Network Network t Network Port1 Port2 10 G left and right Speed 1 GB 10 GB...
Port Configuration Table 19 R640 server configuration for 25GbE, 1CPU, SFP28 (converged) (continued) Description ScaleIO ScaleIO ScaleIO ScaleIO ScaleIO Not in Use (iDRAC) Data1 Client Client Data2 Managemen Monitoring Network Network t Network Port1 Port2 VMkernel port vmk2 User User vmk3 vmk0 User...
Port Configuration Table 24 R640 server configuration for 10GbE, 2CPU, SFP+ (continued) Description ScaleIO ScaleIO ScaleIO ScaleIO ScaleIO Not in Use (iDRAC) Data1 Client Client Data2 Managemen Monitoring Network Network t Network Port1 Port2 Physical Mgmt Switch Data Switch 1 Data Switch 1 Data Switch 2 Data...
Page 53
Port Configuration 8. Click OK to approve the selection. 9. In the Configure Management Network window, scroll to IP Configuration, and press Enter. 10. In the IP Configuration popup window, change the selected option to Set static IP address and network configuration, using the Up/Down arrow keys for navigation and the Space bar for selection.
Page 54
Port Configuration Hardware Configuration and Operating System Installation Guide - ESXi Servers...
CHAPTER 6 Preparing Disks to add to a ScaleIO SDS This section describes how to prepare disks so they can be added as ScaleIO SDS devices. Adding disks to a ScaleIO SDS................56 Verify the disk controller type................56 Enable PERCCLI on an ESXi server..............57 Create virtual devices with PERCCLI..............
Procedure <BMC/iDRAC_IP_address> . 1. From your Internet browser, go to http:// 2. In the DELL Console Login window, type the user name and password, and click Login. The Integrated Remote Access Controller screen appears. 3. In the navigation pane, select Storage Controllers.
Root access to the server on which PERCCLI is to be installed Access to the Internet to enable you to download the PERCCLI utility Procedure 1. Download PERCCLI from http://downloads.dell.com/FOLDER03747363M/1/ Vmware-NDS_Perccli_1-17-10.zip. 2. Extract the ZIP file. 3. Copy the installation file, vmware-esx-perccli-1.17.10.vib, to /tmp on the server.
Preparing Disks to add to a ScaleIO SDS Create virtual devices with PERCCLI Create virtual devices (VDs) on disks using the PERCCLI utility. You need to perform this step before adding disks to a ScaleIO SDS server in the ScaleIO Ready Node system.
Page 59
Preparing Disks to add to a ScaleIO SDS Removing all the existing VDs from the node Setting up the controller card boot parameter Procedure 1. Log in to the node. 2. Display the disk information on the node: /opt/lsi/perccli/perccli /c0/eall/sall show Output similar to the following is displayed: The output shows the following: Enclosure ID (EID): Used in a later step when creating VDs...
Preparing Disks to add to a ScaleIO SDS The boot parameter of the controller card is defined. Results You have verified that the disks are in the UGood (unconfigured, but good) state and that the controller card boot parameter is set to on. You can now create virtual devices.
Preparing Disks to add to a ScaleIO SDS Before using PERCCLI, ensure that your working directory is /opt/lsi/perccli/. If it is not, run: cd /opt/lsi/perccli/ Use this procedure both for SSD for storage and SSD to be used to provide caching acceleration.
Page 62
Preparing Disks to add to a ScaleIO SDS 3. Verify that all the devices have a virtual device created on them. In the Drive Information output, the device state displayed as "Onln" in the State column shows that the device has a virtual device created on it. 4.
Page 63
Preparing Disks to add to a ScaleIO SDS Cache = RWTD (for HDDs with DAS Cache) After you finish Continue to deploy the ScaleIO system, as described in the ScaleIO Ready Node VMware Quick Start Guide . Ensure virtual disk creation with PERCCLI...
Page 64
Preparing Disks to add to a ScaleIO SDS Hardware Configuration and Operating System Installation Guide - ESXi Servers...
CHAPTER 7 Reference Material This section contains additional information that may be required for the procedures described in this document. Activate the NVIDIA GPU on the server............. 66 Installing the iDRAC Service Module on an ESXi server........66 DTK - Hardware Update Bootable ISO...............
The iSM is available as a Zip file for installing on systems running the VMware ESXi operating system. You can install the iSM via SSH or using the vSphere CLI. The Zip file follows the naming convention ISM-Dell-Web-3.0.1-<bldno>.VIB- <version>i.zip, where <version >...
Install the iSM using the vSphere CLI Install the iDRAC Service Module (iSM) software on the ESXi host using the vSphere CLI. Procedure 1. Download the ISM-Dell-Web-3.0.1-<bldno>.VIB-<version>i.zip file from http://www.dell.com/support/home/us/en/19/drivers/driversdetails? driverId=G0XDV%20. 2. Unzip the file to a local directory on the system.
Guide. Update the hardware using remote RACADM You can install and execute the Dell RACADM tool from any management system with access to the iDRAC network. The remote RACADM command set is useful in this situation to mount and execute the Hardware ISO to a large number of ScaleIO Ready Node servers.
Page 69
Reference Material Procedure 1. Configure a laptop or server with access to the iDRAC network. 2. Install the Dell DRAC tools, including RACADM: a. In any Internet browser, go to the RACADM Command Line Interface for DRAC page in the Dell Systems Management wiki.
Page 70
Reference Material 10. Repeat steps on page 69 - on page 69 for each ScaleIO Ready Node server needing the BIOS and firmware updates and configuration. All servers can be updated in parallel. 11. Wait for the configuration and firmware updates to complete. The server console screen will indicate when the script is complete.
-r <dracIP> -u root -p <password> update –f <filename.exe> Where: <dracIP> is the iDRAC IP address <password> is the password for the server <filename.exe> is the name of the Dell Windows update packages Note The default password is Scaleio123. Configuration settings The Hardware ISO runs a script that automatically configures the BIOS and iDRAC settings listed in the table below.
Page 72
Reference Material Table 26 Hardware ISO configuration settings (continued) Description Setting Value Hard Disk Boot Order BIOS.BiosBootSettings.HddS Disk.SATAEmbedded.J-1,RAI D.Integrated.1-1 Disk.SATAEmbedded.D-1,RAI D.Integrated.1-1 Server Boot Mode BIOS.BiosBootSettings.Boot Bios Mode SRIOV Global Enablement BIOS.IntegratedDevices.Sriov Enabled GlobalEnable Memory Performance Tuning BIOS.MemSettings.SnoopMo EarlySnoop System Change Tracking BIOS.MiscSettings.InSystem Disabled Characterization...
Reference Material <password> is the password for the server <setting> is the BIOS/iDRAC setting name <value> is the BIOS/iDRAC setting value Note When setting the BIOS configuration, include this command: racadm -r <dracIP> -u root -p <password> jobqueue create BIOS.Setup.1-1 <value>...
Page 74
/bundleapplicationlogs/apply_components.log 3. You can also view the script for the Hardware ISO, which is useful in helping to identify and troubleshoot log entries: less /opt/dell/toolkit/systems/drm_files/apply_bundles.sh Results The script attempts to configure several boot order commands, regardless of the hardware configuration of the ScaleIO Ready Node server. This allows the script to support multiple hardware platform configurations.
Dell Lifecycle Controller (LC) With the launch of the Dell PowerEdge 13th-generation servers in September 2014, Dell has enhanced our embedded management without the need to install a software- based agent within the host operating system. At the heart of the 13th-generation servers' embedded management is the iDRAC8 with Lifecycle Controller (LC) technology.
Page 76
Dell OpenManage Deployment Toolkit (DTK) The Dell OpenManage Deployment Toolkit (DTK) includes a set of utilities, sample scripts, and sample configuration files that you can use to deploy and configure Dell systems. You can use the DTK to build script-based and RPM-based installation for deploying large number of systems on a pre-operating system environment in a reliable way, without changing their current deployment processes.
Page 77
INDEX architecture 36 cabling best practice 36 Management IP address 52 Port designations 43 PowerEdge R640 42 R740xd 42 Hardware Configuration and Operating System Installation Guide - ESXi Servers...
Page 78
Index Hardware Configuration and Operating System Installation Guide - ESXi Servers...