10000 series router sip and spa hardware (82 pages)
Summary of Contents for Cisco Nexus 1000V
Page 1
Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) May 16, 2016 Cisco Systems, Inc. www.cisco.com Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco website at www.cisco.com/go/offices. Text Part Number: OL-31393-01...
Page 2
OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks.
Cisco Support Communities Contacting Cisco or VMware Customer Support Troubleshooting Tools C H A P T E R Commands Ping Traceroute Monitoring Processes and CPUs Identifying the Running Processes and their States Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 4
Recovering the Network Administrator Password Managing Extension Keys Known Extension Problems and Resolutions Resolving a Plug-In Conflict Finding the Extension Key on the Cisco Nexus 1000V Finding the Extension Key Tied to a Specific DVS Verifying Extension Keys Recreating the Cisco Nexus 1000V Installation...
Page 5
C H A P T E R Troubleshooting L3Sec Ports C H A P T E R Information About Ports Information About Interface Characteristics Information About Interface Counters Information About Link Flapping Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 6
C H A P T E R Information About Layer 2 Ethernet Switching 12-1 Port Model 12-1 Viewing Ports from the VEM 12-2 Viewing Ports from the VSM 12-3 Port Types 12-4 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 7
Common NetFlow Problems 15-2 Debugging a Policy Verification Error 15-3 Debugging Statistics Export 15-3 ACLs 16-1 C H A P T E R Information About Access Control Lists 16-1 ACL Configuration Limits 16-1 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 8
20-1 C H A P T E R Information About DHCP Snooping 20-1 Information About Dynamic ARP Inspection 20-2 Information About IP Source Guard 20-2 Guidelines and Limitations for Troubleshooting 20-2 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) viii OL-31593-01...
Page 9
Problems with VM Traffic 22-10 VEM Troubleshooting Commands 22-11 VEM Log Commands 22-12 Error Messages 22-12 Before Contacting Technical Support 23-1 C H A P T E R Cisco Support Communities 23-1 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 10
25-20 VSI Discovery and Configuration Protocol 26-1 C H A P T E R Information About VDP 26-1 Problems with VDP 26-2 VDP Troubleshooting Commands 26-2 VSM Commands 26-2 VEM Commands 26-4 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 11
C H A P T E R Information About vCenter Plug-in 28-1 Prerequisites for VMware vSphere Web Client 28-1 Generating a Log Bundle 28-2 Ethanalyzer 29-1 C H A P T E R Using Ethanalyzer 29-1 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Updated the high availability section. 4.2(1)SV2(1.1) High Availability Added a command output for the new show system internal active-active remote accounting logs command and updated the output for the show system redundancy status command. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) xiii OL-31593-01...
Page 14
4.2(1)SV1(4) Upgrades upgrade problems. VEM health check Added information about the VEM health 4.0(4)SV1(3) Checking Network Connectivity check that shows the cause of a Between the VSM and the VEM connectivity problem. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 15
5.2(1)SV3(1.1) L3Sec the internal control plane communications (Control and Packet traffic) of Cisco Nexus 1000V in a more robust way than in previous releases. It operates only in Layer 3 control mode. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 16
New and Changed Information Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Related Documentation, page xviii • Obtaining Documentation and Submitting a Service Request, page xx • Audience This publication is for experienced network administrators who configure and maintain a Cisco Nexus 1000V. Document Conventions Command descriptions use these conventions: Convention Description boldface font Commands and keywords are in boldface.
Means reader be careful. In this situation, you might do something that could result in equipment Caution damage or loss of data. Related Documentation This section lists the documents used with the Cisco Nexus 1000 and available on Cisco.com at the following URL: http://www.cisco.com/en/US/products/ps9902/tsd_products_support_series_home.html...
Page 19
Cisco Nexus 1000V Password Recovery Guide Cisco NX-OS System Messages Reference Virtual Services Appliance Documentation The Cisco Nexus Virtual Services Appliance (VSA) documentation is available at http://www.cisco.com/en/US/products/ps9902/tsd_products_support_series_home.html Virtual Security Gateway Documentation The Cisco Virtual Security Gateway documentation is available at http://www.cisco.com/en/US/products/ps13095/tsd_products_support_series_home.html...
What’s New in Cisco Product Documentation at: http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html. Subscribe to What’s New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation, as an RSS feed and deliver content directly to your desktop using a reader application. The RSS feeds are a free service.
We recommend that you do the following to ensure the proper operation of your networks: Maintain a consistent Cisco Nexus 1000V release across all network devices. • Refer to the release notes for your Cisco Nexus 1000V release for the latest features, limitations, • and caveats.
• Troubleshooting Basics This section introduces questions to ask when troubleshooting a problem with the Cisco Nexus 1000V or connected devices. Use the answers to these questions to identify the scope of the problem and to plan a course of action.
Use the show interface-brief command to check the status of a virtual Ethernet port or a physical Ethernet port. Verifying Layer 3 Connectivity Answer the following questions to verify Layer 3 connectivity: Have you configured a gateway of last resort? • Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Those problems and corrective actions include the following: • Identify key Cisco Nexus 1000V troubleshooting tools. Obtain and analyze protocol traces using SPAN or Ethanalyzer on the CLI. • Identify or rule out physical port issues.
Server Implementation The syslog facility allows the Cisco Nexus 1000V to send a copy of the message log to a host for more permanent storage. This feature can be useful if the logs need to be examined over a long period of time or when the Cisco Nexus 1000V is not accessible.
# ps -ef |grep syslogd root 23508 1 0 11:01:41 ? 0:00 /usr/sbin/syslogd Test the syslog server by creating an event in the Cisco Nexus 1000V. In this case, port e1/2 was bounced Step 3 and the following was listed on the syslog server. Notice that the IP address of the switch is listed in brackets.
Brief explanation of the steps that you have already taken to isolate and resolve the problem • If you purchased the Cisco Nexus 1000V and support contract from Cisco, contact Cisco for Cisco Nexus 1000V support. Cisco provides Layer 1, Layer 2, and Layer 3 support.
Page 28
Chapter 1 Overview Contacting Cisco or VMware Customer Support Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
C H A P T E R Troubleshooting Tools This chapter describes the troubleshooting tools available for the Cisco Nexus 1000V and includes the following topics: Commands, page 2-1 • Ping, page 2-1 • Traceroute, page 2-2 • Monitoring Processes and CPUs, page 2-2 •...
TTY—Terminal that controls the process. A “-” usually means a daemon is not running on any • particular TTY. Process—Name of the process. • Process states are as follows: D—Uninterruptible sleep (usually I/O). • R—Runnable (on run queue). • S—Sleeping. • • T—Traced or stopped. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
CPU time in average for each process invocation. • 1Sec—CPU utilization in percentage for the last one second. Example 2-2 show processes cpu Command switch# show processes cpu Runtime(ms) Invoked uSecs 1Sec Process ----- ----------- -------- ----- ----- ----------- Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
RADIUS is a protocol used for the exchange of attributes or credentials between a head-end RADIUS server and a client device. These attributes relate to three classes of services: Authentication • Authorization • Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Authentication refers to the authentication of users for access to a specific device. You can use RADIUS to manage user accounts for access to a Cisco Nexus 1000V. When you try to log into a device, the Cisco Nexus 1000V validates you with information from a central RADIUS server.
Page 34
Enables logging for Telnet or SSH • • Disabled by default Example 2-5 terminal monitor Command switch# terminal monitor For more information about configuring syslog, see the Cisco Nexus 1000V System Management Configuration Guide. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
You are logged in to the vSphere client on the ESX server. • You are logged in to the Cisco Nexus 1000V CLI in EXEC mode. • This procedure verifies that your vSphere ESX server uses the VMware Enterprise Plus license. This license includes the Distributed Virtual Switch feature, which allows visibility to the Cisco Nexus 1000V.
If your vSphere ESX server does not have an Enterprise Plus license, you must upgrade your • VMware License to an Enterprise Plus license to have visibility to the Cisco Nexus 1000V. Host is Not Visible from the Distributed Virtual Switch...
Host is Not Visible from the Distributed Virtual Switch Refreshing the vCenter Server Connection You can refresh the connection between the Cisco Nexus 1000V and vCenter Server. From the Cisco Nexus 1000V Connection Configuration mode on the Virtual Supervisor Module (VSM), Step 1 enter the following command sequence:...
Layer 2 domain. Following the installation of the Cisco Nexus 1000V, make certain that you configure a domain ID. Without a domain ID, the VSM cannot connect to the vCenter Server. Follow these guidelines: The domain ID should be a value within the range of 1 to 4095.
Ensure that the VMware VirtualCenter Server service is running. Step 4 Troubleshooting Connections to vCenter Server You can troubleshoot connections between a Cisco Nexus 1000V VSM and a vCenter Server. In a web browser, enter the path: http://<VSM-IP> Step 1 Download the cisco_nexus_1000v_extension.xml file to your desktop.
The actual value of “Cisco_Nexus_1000V_584325821” will vary. It should match the extension key Note from the cisco_nexus_1000v_extension.xml file. Recovering the Network Administrator Password For information about recovering the network administrator password, see the Cisco Nexus 1000V Password Recovery Guide. Managing Extension Keys This section includes the following topics: Known Extension Problems and Resolutions, page 3-7 •...
“Unregistering the Extension Key in the vCenter • Server” section on page 3-12. DETAILED STEPS From the Cisco Nexus 1000V for the VSM whose extension key you want to view, enter the following Step 1 command: show vmware vc extension-key Example: switch# show vmware vc extension-key Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1)
The Summary tab opens with the extension key displayed in the Notes section of the Annotations block. Verifying Extension Keys You can verify that the Cisco Nexus 1000V and vCenter Server are using the same extension key. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1)
Page 43
Chapter 3 Installation Managing Extension Keys DETAILED STEPS Find the extension key used on the Cisco Nexus 1000V using the “Finding the Extension Key on the Step 1 Cisco Nexus 1000V” section on page 3-7. Find the extension key used on the vCenter Server using the “Finding the Extension Key Tied to a...
Recreating the Cisco Nexus 1000V Installation Recreating the Cisco Nexus 1000V Installation You can re-create the complete Cisco Nexus 1000V configuration in the event of a persistent problem that cannot be resolved using any other workaround. Flowchart: Re-creating the Cisco Nexus 1000V Installation...
Log in to the VSM CLI in EXEC mode. • DETAILED STEPS Step 1 From the Cisco Nexus 1000V VSM, use the following commands to remove the DVS from the vCenter Server. config t svs connection vc no vmware dvs...
“Finding the Extension Key on the Cisco Nexus 1000V” section on page 3-7. After unregistering the extension key in vCenter Server, you can start a new installation of the Cisco • Nexus 1000V VSM software. DETAILED STEPS...
Page 47
“Finding the Extension Key on the Cisco Nexus 1000V” section on page 3-7, and then click Invoke Method. The extension key is unregistered in vCenter Server so that you can start a new installation of the Cisco Nexus 1000V VSM software. Step 4 You have completed this procedure.
Chapter 3 Installation Problems with the Cisco Nexus 1000V Installation Management Center Problems with the Cisco Nexus 1000V Installation Management Center The following are possible problems and their solutions. Symptom Problem Recommended Action Port migration fails. The VSM to VEM migration fails in Check if there is any VM running on the •...
• Information About Licenses The name for the Cisco Nexus 1000V license package is NEXUS1000V_LAN_SERVICES_PKG and the version is 3.0. By default, 1024 licenses are installed with the Virtual Supervisor Module (VSM). These default licenses are valid for 60 days. You can purchase permanent licenses that do not expire.
4-5. If there is a license file with the same name, rename your new license file to something else. Do not edit the contents of the license file. If you have already done so, contact your Cisco Customer • Support Account Team.
1 days! The VEMs' VNICS will be brought down if license is allowed to expire. Please contact your Cisco account team or partner to purchase Licenses. To activate your purchased licenses, click on www.cisco.com/go/license. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Example 4-8 on page 4-7. show license usage [license_name] Displays information about the licenses and where they are used. If displayed for a specific license, indicates VEM and socket information. Example 4-1 on page 4-5. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 53
Ethernet scale. Example 4-7 on page 4-7. For detailed information about show command output, see the Cisco Nexus 1000V Command Reference. EXAMPLES Example 4-1 show license usage license_name Command switch# show license usage NEXUS1000V_LAN_SERVICES_PKG...
Page 54
File Name Feature Name Version Count Expiry ------------------------------------------------------------------------------------------ eval.lic NEXUS1000V_LAN_SERVICES_PKG 1.0 17 3-nov-2014 eval0715.lic NEXUS1000V_LAN_SERVICES_PKG 3.0 17 15-jul-2015 show switch edition (purpose: Displays the switch edition, advanced feature status, license expiry and module and veth scale) Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 55
Licenses are Sticky Mod Socket Count License Usage Count License Version License Status --- ------------ ------------------- --------------- -------------- 103 2 2 3.0 licensed 104 2 2 3.0 licensed Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Upgrade Troubleshooting Commands, page 5-16 Information About Upgrades The upgrade for the Cisco Nexus 1000V involves upgrading software on both the VSM and the Virtual Ethernet Module (VEM). An in service software upgrade (ISSU) is available for a stateful upgrade of the Cisco Nexus 1000V image(s) running on the VSM.
Page 58
Make sure that the module removal is complete. the upgrade. Restart the software upgrade using the instructions in Pre-Upgrade check failed. Return code 0x4093000A the Cisco Nexus 1000V Installation and Upgrade Guide. (SRG collection failed) Error message: The standby VSM is not Verify the HA synchronization state.
Page 59
When the correct software images are in the bootflash: repository, restart the software upgrade using the instructions in the Cisco Nexus 1000V Installation and Upgrade Guide. Error message: You might have used an Restart the software upgrade using the correct filenames for incorrect filename when the new software images.
Page 60
• Stop the upgrade and restart one session only using the (0x401E0007) instructions in the Cisco Nexus 1000V Installation and Upgrade Guide. The install command fails The standby VSM fails to Do one of the following: with following error boot with the new image.
• cluster. hosts with the new VEM. availability (HA) Restart the VEM software upgrade using the instructions VMware fault tolerance • in the Cisco Nexus 1000V Installation and Upgrade (FT) Guide. Vmware Distributed • Power Management (DPM) VEM upgrade fails.
Restart the software upgrade using the instructions in unreachable. The other VSM the Cisco Nexus 1000V Installation and Upgrade Guide. has the original pre-upgrade software version installed and is reachable. The upgrade GUI stops and...
The same domain ID and password as that of the primary VSM. • For a detailed procedure, see the Cisco Nexus 1000V Installation and Upgrade Guide. The VSM comes up and forms an HA pair with the newly created standalone VSM. The VSMs have the previous version of the software installed.
Page 64
You have completed this procedure. Return to one of these sections: Step 4 “Recovering a Secondary VSM with Active Primary” section on page 5-7 • • “Recovering a Primary VSM with Active Secondary” section on page 5-12 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 65
Copy the running configuration to the startup configuration. Step 4 copy run start Example: switch#(config)# copy run start [########################################] 100%e switch#(config)# Verify the change in the system and kickstart boot variables. Step 5 show boot Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 66
From the vCenter Server left-hand panel, right-click the VSM and then choose Power > Power On. Step 1 The VSM starts. You have completed this procedure. Return to the “Recovering a Primary VSM with Active Secondary” Step 2 section on page 5-12. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 5-10 OL-31593-01...
Page 67
Example: switch#(config-svs-domain)# copy run start [########################################] 100%e switch#(config-svs-domain)# You have completed this procedure. Return to the “Recovering a Primary VSM with Active Secondary” Step 4 section on page 5-12. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 5-11 OL-31593-01...
The host or cluster of the existing secondary VSM. • For detailed installation procedures, see the Cisco Nexus 1000V Installation and Upgrade Guide. Make sure that the port groups between the host server and VSM are not connected when the new VSM...
Page 69
Connected Connect at Power On • The connection from the VSM to the host server through the management port is dropped and is not restored when you power on the VSM. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 5-13 OL-31593-01...
Page 70
In vCenter Server, select the VSM and then choose Edit > Settings. Step 1 The Virtual Machine Properties dialog box opens. Select the Control port group and check the following Device Settings: Step 2 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 5-14 OL-31593-01...
Page 71
Select the Management port group and check the following Device Setting: Step 3 Connect at Power On • When you power on the VSM, it will connect to the host server through the management port. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 5-15 OL-31593-01...
Displays the boot variables currently in the startup configuration. Example 5-6 on page 5-18. show svs connections Displays the current connections between the VSM and the VMware host server. Example 5-7 on page 5-18. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 5-16 OL-31593-01...
Information About High Availability The purpose of high availability (HA) is to limit the impact of failures—both hardware and software— within a system. The Cisco NX-OS operating system is designed for high availability at the network, system, and service levels.
VSM fails. Network-Level High Availability The Cisco Nexus 1000V HA at the network level includes port channels and Link Aggregation Control Protocol (LACP). A port channel bundles physical links into a channel group to create a single logical link that provides the aggregate bandwidth of up to eight physical links.
Page 79
VSMs. interface. Check the output of the • show system internal redundancy info command and verify if the degraded_mode flag is set to true. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 80
MAC addresses Move the identified VSM(s) out of of the VSM(s) that collide the system to stop role collision. with the working VSM. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Example 6-1show cores Command, page 6-6 show processes [pid pid] Example 6-2show processes log [pid pid] Command, page 6-6 show system internal active-active Example 6-7show system internal active-active remote accounting logs Command, page 6-10 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 82
CWD: /var/sysmgr/work To check redundancy status, use the following commands: Example 6-3 show system redundancy status Command switch# show system redundancy status Redundancy role --------------- administrative: primary <-- Configured redundancy role Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 83
6 rx_set_ver_req_pkts: rx_set_ver_rsp_pkts: rx_heartbeat_req_pkts: 6 rx_heartbeat_rsp_pkts: 442546 <-- Counter should be increasing, as this indicates that communication between VSM is working properly. rx_drops_wrong_domain: 0 rx_drops_wrong_slot: rx_drops_short_pkt: rx_drops_queue_full: rx_drops_inactive_cp: rx_drops_bad_src: rx_drops_not_ready: rx_unknown_pkts: Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 84
Configured to use the real platform manager. Configured to use the real redundancy driver. Redundancy register: this_sup = RDN_ST_AC, other_sup = RDN_ST_SB. EOBC device name: eth0. Remote addresses: MTS - 0x00000201/3 IP - 127.1.1.2 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 85
NOTE: Please run the same command on sup-1 to check for conflicting(if any) sup-1(s) in the same domain. If no collisions are detected, the highlighted output is not displayed. Use the following command to display the accounting logs that are stored on a remote VSM. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 86
The standby VSM console is not accessible externally, but can be accessed from the active VSM through the attach module module-number command. switch# attach module 2 This command attaches to the console of the secondary VSM. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 6-10 OL-31593-01...
The Cisco Nexus 1000V manages a data center defined by a VirtualCenter. Each server in the data center is represented as a module in the Cisco Nexus 1000V and can be managed as if it were a module in a physical Cisco switch.
“Checking the vCenter Server Configuration” section on page 7-10. – ERROR: Datacenter not found • For a list of terms used with the Cisco Nexus 1000V, see the Cisco Nexus 1000V Getting Started Guide. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Verifying the VSM Is Configured Correctly, page 7-7 Checking the vCenter Server Configuration, page 7-10 Checking Network Connectivity Between the VSM and the VEM, page 7-10 Checking the VEM Configuration, page 7-14 Collecting Logs, page 7-16 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
If the plug-in is not found, create one using the following procedure in the Cisco Nexus 1000V Getting Started Guide: Creating a Cisco Nexus 1000V Plug-In on the vCenter Server Following a reboot of the VSM, the system stops functioning in one of the following states and does not recover on its own.
Page 91
VSM. copy source filesystem: filename system:running-config If not, reconfigure the VSM using the • following section in the Cisco Nexus 1000V Getting Started Guide: Setting Up the Software After boot, VSM is stopped at Corrupt boot menu file.
“Unregistering the Extension Key in the vCenter Server” section on page 3-12. Install a new extension key using the “Creating a Cisco Nexus 1000V Plug-In on the vCenter • Server” procedure in the Cisco Nexus 1000V Getting Started Guide. Verify the connection between the VSM and vCenter Server.
Example: switch# show svs domain SVS domain config: Domain id: Control vlan: 3002 Packet vlan: 3003 L2/L3 Control VLAN mode: L2 L2/L3 Control VLAN interface: mgmt0 Status: Config push to VC successful Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 94
BEFORE YOU BEGIN Log in to the CLI in EXEC mode. • Check that the output of the show running-config command shows control and packet VLAN ID • numbers among the VLANs configured, Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 95
Card control VLAN: 168 Card packet VLAN: 168 Control type multicast: No Card Headless Mode : No Processors: 16 Processor Cores: 8 Processor Sockets: 2 Kernel Memory: 25102148 Port link-up delay: 5s Global UUFB: DISABLED Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Checking the vCenter Server Configuration You can verify the configuration on vCenter Server. Confirm that the host is added to the data center and the Cisco Nexus 1000V DVS in that data center. Step 1 Confirm that at least one pnic of the host is added to the DVS, and that pnic is assigned to the Step 2 system-uplink profile.
Page 97
MAC address of the VSM, then there is a problem with connectivity between the server hosting the VSM and the upstream switch. Recheck the VSM configuration and vCenter Server configuration. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 7-11 OL-31593-01...
VLAN. If no such uplink exists, it reports this as an error. You need to specify the -p parameter and rerun the script. You can recover management and control connectivity of a host when a VSM is running on a VEM. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 7-12 OL-31593-01...
Page 99
VLANs in the trunk port profile. If the lists match, all of the expected VLANs are forwarding and the Cisco Nexus 1000V is blocking nonallowed VLANs.
~ # vemcmd show port 48 IfIndex Vlan Bndl SG_ID Pinned_SGID Type Admin State CBL Mode Name . . . 1a030100 PHYS Trunk vmnic1 ~# vemcmd set mtu 9000 ltl 17 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 7-14 OL-31593-01...
Page 101
Verify that the vmnic port that is supposed to carry the control VLAN and packet VLAN is present. vemcmd show bd control_vlan vemcmd show bd packet_vlan Example: ~ # vemcmd show bd 3002 BD 3002, vdc 1, vlan 3002, 2 ports Portlist: Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 7-15 OL-31593-01...
Page 103
13 • show system internal ethpm event-history module 13 • If you need to contact Cisco TAC for assistance in resolving an issue, you will need the output of the Note commands listed in Step VSM and VEM Troubleshooting Commands You can use the commands in this section to troubleshoot problems related to VSM.
Page 104
VM NIC is used. Example 7-16 on page 7-22. vem-connect -i ip_address -v vlan [-pnic Recovers management and control connectivity of vmnicN] a host when a VSM is running on a VEM. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 7-18 OL-31593-01...
Page 106
Command switch# show mac address-table interface Gi3/1 vlan 3002 Legend: * - primary entry age - seconds since last seen n/a - not available vlan mac address type learn ports ------+----------------+--------+-----+----------+-------------------------- Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 7-20 OL-31593-01...
Page 108
VLANs. This might be a normal situation depending on the port profile allowed VLAN list. Compare the output of the vemcmd show port vlans command against the port profile trunk allowed VLANs. If the lists match, all of the expected VLANs are forwarding and the Cisco Nexus 1000V is blocking nonallowed VLANs.
L3Sec This chapter describes how to secure the internal control plane communications (Control and Packet traffic) of Nexus 1000V in a more robust way than in previous releases. It operates only in Layer 3 Control mode. Troubleshooting L3Sec, page 8-1 •...
Some values might not be valid when the interface is down (such as the operation speed). For a complete description of port modes, administrative states, and operational states, see the Cisco Nexus 1000V Interface Configuration Guide.
The port security feature allows you to secure a port by limiting and identifying the MAC addresses that can access the port. Secure MAC addresses can be manually configured or dynamically learned. For detailed information about port security, see the Cisco Nexus 1000V Security Configuration Guide. Is Port Security...
Chapter 9 Ports Port Diagnostic Checklist Port Diagnostic Checklist Use the following checklist to diagnose port interface activity. For more information about port states, see the Cisco Nexus 1000V Interface Configuration Guide. Table 9-1 Port Diagnostic Checklist Checklist Example Verify that the module is active.
Disable and then enable the port. shut no shut Move the connection to a different port on the same module or a different module. Collect the ESX-side NIC configuration. vss-support Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
A control frame is erroneously sent to the device. ESX errors, or link flapping, Use the troubleshooting guidelines in the documentation for your occurs on the upstream switch. ESX or upstream switch. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Open a support case and submit the output of the above commands. For more information see the “Contacting Cisco or VMware Customer Support” section on page 1-7. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Log in to the upstream switch and verify that the packet VLAN is allowed on the port. show running-config interface gigabitEthernet slot/port If the packet VLAN is not allowed on the port, add it to the allowed VLAN list. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Chapter 9 Ports Problems with Ports Port Security Violations For detailed information about port security, see the Cisco Nexus 1000V Security Configuration Guide. Possible Cause Solution The configured maximum Display the secure addresses. number of secured show port -security address vethernet number...
Install the necessary licenses or move the switch to essential mode. svs switch edition essential Port Troubleshooting Commands You can use the commands in this section to troubleshoot problems related to ports. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) OL-31593-01...
Page 122
• Trunk VLAN status • Number of frames sent and received • Transmission errors, including discards, errors, CRCs, and invalid frames Example 9-9 on page 9-13. Example 9-10 on page 9-13. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 9-10 OL-31593-01...
Page 123
-security address interface vethernet Displays information about secure addresses on an interface. Example 9-17 on page 9-16. For detailed information about show command output, see the Cisco Nexus 1000V Command Reference. EXAMPLES Example 9-1 show module Command switch# show mod 3...
Page 124
1 22:43.. Previous state: [PI_FSM_ST_IF_INIT_EVAL] Triggered event: [PI_FSM_EV_IE_ERR_DISABLED_CAP_MISMATCH] Next state: [PI_FSM_ST_IF_DOWN_STATE] Example 9-6 show logging logfile Command switch# show logging logfile . . . 4 06:54:04 switch %PORT_CHANNEL-5-CREATED: port-channel 7 created Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 9-12 OL-31593-01...
Page 125
Auto-Negotiation is turned off Input flow-control is off, output flow-control is off Auto-mdix is turned on Switchport monitor is off 18775 Input Packets 10910 Unicast Packets 862 Multicast Packets 7003 Broadcast Packets 2165184 Bytes Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 9-13 OL-31593-01...
For more information about assigning port profiles to physical or virtual ports, see your VMware documentation. To verify that the profiles are assigned as expected to physical or virtual ports, use the following show commands: show port-profile virtual usage • show running-config interface interface-id • Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 10-1 OL-31593-01...
To verify port profile inheritance, use the following command: show running-config interface interface-id • Inherited port profiles cannot be changed or removed from an interface from the Cisco Nexus 1000V Note CLI. This action can only be done from vCenter Server.
Page 133
Overrun mapping is created. Fix the error in the port profile using the procedures in the Cisco Nexus 1000V Port Profile Configuration Guide. Bring the interface out of quarantine. no shutdown The interface comes back online.
Verify the port profile-to-interface mapping. show port-profile virtual usage Step 5 Verify the interface has come out of quarantine automatically. The interface should no longer appear in the show command output. show port-profile sync-status Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 10-4 OL-31593-01...
Displays the port profile role configuration, port-profile-role-name] including role names, descriptions, assigned users, and assigned groups. Example 10-7 on page 10-8. show running-config port-profile Displays the port profile configuration. [profile-name] Example 10-6 on page 10-8. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 10-5 OL-31593-01...
Page 136
Displays the messages logged about port profile msgs events within the Cisco Nexus 1000V. Example 10-13 on page 10-14. For detailed information about show command output, see the Cisco Nexus 1000V Command Reference. EXAMPLES Example 10-1 show port-profile Command switch# show port-profile port-profile 1...
If a link goes down in a port channel, the upper protocol is not aware of it. To the upper protocol, the link is still there, although the bandwidth is diminished. The MAC address tables are not affected by link failures. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 11-1 OL-31593-01...
Ensure that all interfaces in the port channel have the same destination device for Link Aggregation Control Protocol (LACP) channels. By using the Asymmetric Port Channel (APC) feature in the Cisco Nexus 1000V, ports in an ON mode channel can be connected to two different destination devices.
Use APC when you want to configure a port channel whose members are connected to two different • upstream switches. APC depends on Cisco Discovery Protocol (CDP). Make sure CDP is enabled on the VSM and • upstream switches. Physical ports within an APC get assigned subgroup IDs based on the CDP information received •...
A VLAN is not in the allowed VLAN Add the VLAN to the allowed VLAN list. Use the does not traverse list. switchport trunk allowed vlan add vlan-id command in trunk. the profile used by the interface. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 11-5 OL-31593-01...
Page 150
Chapter 11 Port Channels and Trunking VLAN Traffic Does Not Traverse Trunk Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 11-6 OL-31593-01...
Troubleshooting BPDU Guard, page 12-14 Information About Layer 2 Ethernet Switching The Cisco Nexus1000V is a distributed Layer 2 virtual switch that extends across many virtualized hosts. It consists of two components: The Virtual Supervisor Module (VSM), which is also known as the control plane (CP). The VSM •...
Cisco Nexus1000V. • Virtual Ethernet Ports (VEth)—A vEth port is a port on the Cisco Nexus 1000V. The Cisco Nexus 1000V has a flat space of vEth ports 0..N. The virtual cable plugs into these vEth ports that are moved to the host running the VM.
Each uplink port on the host represents a physical interface. It acts like an lveth port, but because physical ports do not move between hosts, the mapping is 1:1 between an uplink port and a vmnic. Each physical port added to the Cisco Nexus1000V switch appears as a physical Ethernet port, just •...
Po (port channel interfaces)—The physical NICs of an ESX Host can be bundled into a logical • interface. This logical bundle is referred to as a port channel interface. For more information about Layer 2 switching, see the Cisco Nexus 1000V Layer 2 Switching Configuration Guide. Layer 2 Switching Problems This section describes how to troubleshoot Layer 2 problems and lists troubleshooting commands.
Log in to the upstream switch and make sure that the port is configured to allow the VLAN that you are looking for. switch# show running-config interface gigabitEthernet 1/38 Building configuration... Current configuration : 161 bytes interface GigabitEthernet1/38 description Srvr-100:vmnic1 switchport switchport trunk allowed vlan 1,60-69,231-233 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 12-5 OL-31593-01...
[all-ports | brief | id vlan-id name Displays VLAN information as specified. See name | dot1q tag native] Example 12-4 on page 12-9. show vlan summary Displays a summary of VLAN information. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 12-7 OL-31593-01...
Page 158
Example 12-1 show mac address-table Command Note The Cisco Nexus 1000V MAC address table does not display multicast MAC addresses. The “Module” indicates the VEM on which this MAC address is seen. The “N1KV Internal Port” refers to an internal port created on the VEM. This port is used for control and management of the VEM and is not used for forwarding packets.
Page 159
VLAN0118 active VLAN0119 active VLAN0800 active VLAN0801 active VLAN0802 active VLAN0803 active VLAN0804 active VLAN0805 active VLAN0806 active VLAN0807 active VLAN0808 active VLAN0809 active VLAN0810 active VLAN0811 active VLAN0812 active Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 12-9 OL-31593-01...
Access to third-party websites identified in this document is provided solely as a courtesy to customers Note and others. Cisco Systems, Inc. and its affiliates are not in any way responsible or liable for the functioning of any third-party website, or the download, performance, quality, functioning, or support...
When MS NLB VMs have more than one port on the same subnet, a request is flooded, which causes both ports to receive it. The server cannot manage this situation. As a workaround for this situation, enable Unknown Unicast Flood Blocking (UUFB). Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 12-13 OL-31593-01...
Displays the switch edition and license information. Example 12-11 on page 12-15. show run interface name Displays the BPDU guard status on a port profile. Example 12-12 on page 12-15. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 12-14 OL-31593-01...
Page 165
Veth36 Enabled Veth68 Enabled Veth73 Enabled Veth77 Enabled name Example 12-14 show system internal cdm info port-profile Command switch(config-if)# show system internal cdm info port-profile name vm port-profile vm Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 12-15 OL-31593-01...
You can configure a private VLAN port as a SPAN source port. – You can use VLAN-based SPAN (VSPAN) on primary, isolated, and community VLANs or use – SPAN on only one VLAN to separately monitor egress or ingress traffic. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 13-1 OL-31593-01...
Cannot Create a VLAN Symptom Possible Cause Solution Cannot create a Using a reserved VLAN ID VLANs 3968 to 4047 and 4094 are reserved for internal use VLAN. and cannot be changed. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 13-3 OL-31593-01...
The mechanism that restricts Layer 2 communication between two isolated ports in the same switch also restricts Layer 2 communication between two isolated ports in two different switches. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 14-1 OL-31593-01...
• Isolated • Community • For additional information about private VLANs, see the Cisco Nexus 1000V Layer 2 Switching Configuration Guide. Troubleshooting Guidelines Follow these guidelines when troubleshooting private VLAN issues: • Use the show vlan private-vlan command to verify that a private VLAN is configured correctly.
Page 174
1b020000 VIRT 4 Access fedora9.eth0 pvlan community 156 153 If additional information is required for Cisco Technical Support to troubleshoot a private VLAN issue, use the following commands: show system internal private-vlan info • show system internal private-vlan event-history traces •...
• A flow record defines the information that NetFlow gathers, such as packets in the flow and the types of counters gathered per flow. You can define new flow records or use the predefined Cisco Nexus 1000V flow records. For detailed information about configuring NetFlow, see the Cisco Nexus 1000V System Management Configuration Guide.
VEM command: vemcmd show netflow monitor show flow internal pdl detailed • Displays internal flow details. Common NetFlow Problems Common NetFlow configuration problems on the VSM can occur if you attempt to do the following: Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 15-2 OL-31593-01...
Ensure that the UDP port configured on the exporter matches that used by the NetFlow Collector. • View statistics for the exporter and identify any drops by entering the show flow exporter • command. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 15-3 OL-31593-01...
• IPv6—The device applies IPv6 ACLs only to IPv6 traffic • For detailed information about how ACL rules are used to configure network traffic, see the Cisco Nexus 1000V Security Configuration Guide. ACL Configuration Limits The following configuration limits apply to ACLs: •...
The commands listed in this section can be used to display configured ACL policies on the Virtual Ethernet Module (VEM). Use the following command to list the ACLs installed on that server switch(config-if)# module vem 3 execute vemcmd show acl Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 16-2 OL-31593-01...
Save the Telnet or SSH session buffer to a file. Copy the logfile created in bootflash. Troubleshooting ACL Logging This section includes the following topics: Using the CLI to Troubleshoot ACL Logging on a VEM, page 16-4 • Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 16-3 OL-31593-01...
You can use the vemcmd flush aclflows command to detect any new flows that affect the VEM. Clear all the existing flows, and then you can detect new flows that match any expected traffic. Syslog messages are not sent when you do this action. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 16-4 OL-31593-01...
If the ACL rule does not have a log keyword, any flow that matches the ACL is not reported although the ACL statistics continue to advance. You can verify a log keyword. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 16-5...
Page 184
Log in to the VSM and VEM CLI. PROCEDURE Command Description Step 1 Verifies that ACL logging is configured properly. show logging ip access-list status Example: switch# show logging ip access-list status switch # Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 16-6 OL-31593-01...
Page 185
Example: switch# show logging ip access-list status switch # Step 2 Verifies ACL logging on the VEM. vemcmd show acllog config Example: switch# vemcmd show acllog config switch # Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 16-7 OL-31593-01...
Policing —Monitors data rates and burst sizes for a particular class of traffic. QoS policing on a • network determines whether network traffic is within a specified profile (contract). For detailed information about QoS, see the Cisco Nexus 1000V Quality of Service Configuration Guide. QoS Configuration Limits Table 17-1 Table 17-2 list the configuration limits for QoS.
VSM to the connected modules. Example 17-1 on page 17-3 show resource-availability qos-queuing Checks whether the QoS configuration is not exceeding the recommended resource limits. show policy-map interface brief Displays the installed policies: Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 17-2 OL-31593-01...
Step 4 Enter the policy-map command which will execute the command once again with the DPA debug traces output to vemdpalog. Step 5 Enter module vem module-number execute vemdpalog stop command. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 17-4 OL-31593-01...
• Encapsulated remote SPAN (ERSPAN) that can send monitored traffic to an IP destination. • For detailed information about how to configure local SPAN or ERSPAN, see the Cisco Nexus 1000V System Management Configuration Guide. SPAN Session Guidelines The following are SPAN session guidelines: •...
Use the vempkt command to capture packets on the VMKernel NIC LTL and ensure ERSPAN packets are being sent. Use the vemlog debug sfspan d command so that the ERSPAN packets appear in the vempkt capture log. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 18-2 OL-31593-01...
In general, IGMP snooping works as follows: • Ethernet switches, such as Cisco Catalyst 6000 Series switches, parse and intercept all IGMP packets and forward them to a CPU, such as a supervisor module, for protocol processing. Router ports are learned using IGMP queries. The switch returns IGMP queries, it remembers which •...
Make sure that the upstream switch has IGMP configured. • Use the show ip igmp snooping groups command to verify if the Cisco Nexus 1000V switch is • configured correctly and is ready to forward multicast traffic. In the displayed output of the command, look for the letter R under the port heading.
IGMPv3 Report suppression disabled Router port detection using PIM Hellos, IGMP Queries Number of router-ports: 0 Number of groups: 0 show ip igmp snooping groups • switch# show ip igmp snooping groups vlan 1784 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 19-6 OL-31593-01...
Page 204
The multicast group table for 224.1.2.3 shows the interfaces that the VEM forwards to when it receives multicast traffic for group 224.1.2.3. If fedora8 has multicast group 224.1.2.3 on its eth0 interface, LTL 47 should be in the multicast group table for 224.1.2.3. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 19-8 OL-31593-01...
Make sure that the table has the correct information in it. Make sure that the state of the trunk port and the access port is UP/UP. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 19-9 OL-31593-01...
Dynamic ARP inspection (DAI) and IP Source Guard also use information stored in the DHCP snooping binding database. For detailed information about configuring DHCP snooping, see the Cisco Nexus 1000V Security Configuration Guide. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1)
• A maximum of 1000 static DHCP entries per interface can be configured. • For detailed guidelines and limitations used in configuring these features, see the Cisco Nexus 1000V Security Configuration Guide. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 20-2...
On the host connected to the client, enable VEM • packet capture to verify incoming requests and acknowledgements in packets. The Cisco Nexus 1000V is dropping packets. On the VSM, verify DHCP statistics. show ip dhcp snooping statistics module vem mod# execute vemcmd show dhcps stats Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1)
Configuration Guide. If all configurations are correct, make sure to turn on DHCP snooping before DAI or IPSG to make sure the Cisco Nexus 1000V has enough time to add the binding in the snooping database. For more information, see the Cisco Nexus 1000V Security Configuration Guide.
For detailed information about configuring IP Source Guard, see the Cisco Nexus 1000V Security Configuration Guide The IP address that corresponds to the On the VSM, display the binding table.
Displays general information about DHCP snooping. Example 20-2 on page 20-7. show ip dhcp snooping binding Displays the contents of the DHCP snooping binding table. Example 20-3 on page 20-8. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 20-6 OL-31593-01...
Page 213
DHCP snooping is operational on the following VLANs: Insertion of Option 82 is disabled Verification of MAC address is enabled DHCP snooping trust is configured on the following interfaces: Interface Trusted ------------ ------- vEthernet 3 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 20-7 OL-31593-01...
Page 214
Configuration : Disabled Operation State : Inactive Example 20-6 show ip arp inspection interface vethernet Command switch# show ip arp inspection interface vethernet 6 Interface Trust State ------------- ----------- vEthernet 6 Trusted switch# Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 20-8 OL-31593-01...
Page 215
00 00 00 03 00 00 00 01 00 00 00 64 00 00 00 07 contd. Example 20-10 show system internal dhcp mem-stats detail Command VSM-N1k# show system internal dhcp mem-stats detail Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 20-9 OL-31593-01...
Page 216
[16843009] PPF goto setting state 1 4) Event:E_DEBUG, length:23, at 682346 usecs after Mon Oct 8 20:57:11 2012 [16843009] Processed log-mts contd Example 20-12 debug dhcp all Command switch# debug dhcp all Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 20-10 OL-31593-01...
Troubleshooting VEM Commands Displays all the statistics related to broadcast, multcast and unknown unicast traffic: • vemcmd show storm stats Displays the configured storm rate on a Virtual Ethernet Module (VEM): Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 21-1 OL-31593-01...
Debugging Storm Control on a VEM You can debug storm control on a VEM. vemlog clear. Step 1 vemlog start. Step 2 Step 3 vemlog debug sfstormcontrol all. Step 4 vemlog show all. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 21-2 OL-31593-01...
Page 219
The Nexus 1000V manages a data center defined by the vCenter Server. Each server in the Datacenter is represented as a linecard in Nexus 1000V and can be managed as if it were a line card in a physical Cisco switch. The Nexus 1000V implementation has two components: •...
Chapter 22 System General Restrictions for vCenter Server See the Cisco Nexus 1000V Getting Started Guide for a detailed overview of how the Nexus 1000V works with VMware ESX software. General Restrictions for vCenter Server When you are troubleshooting issues related to vCenter Server, make sure that you observe the following...
3-12. From the VC client, register the extension (plug-in) for the VSM. Step 4 For more information see the following procedure in the Cisco Nexus 1000V Getting Started Guide. Creating a Cisco Nexus 1000V Plug-In on the vCenter Server •...
Step 3 For more information, see the “Unregistering the Extension Key in the vCenter Server” procedure on page 3-12. From the VC client, register the extension (plug-in) for the VSM. Step 4 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 22-4 OL-31593-01...
Chapter 22 System Problems Related to VSM and vCenter Server Connectivity For more information see the following procedure in the Cisco Nexus 1000V Getting Started Guide. Creating a Cisco Nexus 1000V Plug-In on the vCenter Server • Manually recreate the old port profiles from your previous configuration.
If you use an MTU other than 1500 (the default) for a physical NIC attached to the Cisco Nexus 1000V, then reboots of the ESX can result in a mismatch with the VMware kernel NIC MTU and failure of the VSM and VEM.
• jumbomtu configured on the interface. For more information about configuring MTU on the interface, see the Cisco Nexus 1000V Interface Configuration Guide. When you configure a system MTU on a system port profile, it takes precedence over an MTU you •...
Enters global configuration mode. config t Example: switch# config t switch(config)# Step 2 Displays the port configuration including the LTL number module vem module_number execute vemcmd show port port-LTL-number needed for Step Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 22-8 OL-31593-01...
Profiles that have the system VLAN configuration allow the VEM to communicate with the VSM. Make sure that the system port-profile is defined with the right system VLANS. Use the show port-profile and show port-profile usage commands to collect basic required information. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 22-9 OL-31593-01...
The VSM may be overloaded. Make sure that you have 1 GB of memory and CPU shares for the VSM VM on the vCenter Server. Problems with VM Traffic When troubleshooting problems with intra-host VM traffic, follow these guidelines: Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 22-10 OL-31593-01...
• Example 22-4 vemcmd help Command [root@ESX-cos1 ~]# vemcmd help show card Show the card's global info show vlan [vlan] Show the VLAN/BD table show bd [bd] Show the VLAN/BD table Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 22-11 OL-31593-01...
This error is displayed when the VSM tries to spec.extensionKey as create a different DVS after changing the switch Cisco_Nexus_1000V_2055343757 already name. exists, cannot create DVS new-switch. A specified parameter was not correct. spec.extensionKey Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 22-12 OL-31593-01...
Page 231
VSM is not aware of DVPortgroup test port 0 is in use. The resource the nics attached to the port groups. vim.dvs.DistributedVirtualPort 0 is in use. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 22-13 OL-31593-01...
• Gathering Information for Technical Support At some point, you may need to contact your customer support representative or Cisco TAC for some additional assistance. This section outlines the steps that the you should perform prior to contacting your next level of support, so you can reduce the amount of time that you spend resolving the issue.
Obtaining a File of Core Memory Information Cisco customer support engineers often use files from your system for analysis. One such file contains memory information and is referred to as a core dump. The file is sent to a TFTP server or to a flash card in slot0: of the local switch.
Back up the startup configuration to a server daily before you make any changes. You can write a short script to be run on the Cisco Nexus 1000V to perform a save and then back up the configuration. The script only needs to contain two commands: copy running-configuration startup-configuration and copy startup-configuration tftp://server/name.
Page 236
Chapter 23 Before Contacting Technical Support Copying Files Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 23-4 OL-31593-01...
If not, replace the username and password on the in the networking configuration on vShield Manager. The NSM feature is not enabled on Verify if the NSM feature is enabled on the Cisco the Cisco Nexus 1000V. Nexus 1000V. show feature If not, enable the NSM feature.
Page 239
Director. no port-profile network name logged in vCloud Director: Delete the bridge domain with the same name if it exists. Network already exists no bridge-domain name Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 24-3 OL-31593-01...
Page 240
Check system logs for a port profile failure vCloud Director fails. A system to create the port profile required message reported by NSM. See the Cisco NX-OS message similar to the following is for the network. System Messages Reference for more logged in vCloud Director: information.
Page 241
Check system logs for a port group property vCloud Director fails. A system Vmware port group property on the failure message reported by NSM. See the Cisco message similar to the following is port profile. NX-OS System Messages Reference for more logged in vCloud Director: information.
Page 242
Check system logs for a port profile description vCloud Director fails. A system description for the port profile failure message reported by NSM. See the Cisco message similar to the following is associated with the network. NX-OS System Messages Reference for more logged in vCloud Director: information.
| grep NSMGR Displays the system logs from the network segmentation manager. For detailed information about show command output, see the Cisco Nexus 1000V Command Reference. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 24-7 OL-31593-01...
MAC frames are sent over the network. You can have multiple VTEPs per VEM that are used as sources for this encapsulated traffic. The encapsulation carries the VXLAN identifier used to Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-1...
VXLANs and routers or services that have traditional VLAN interfaces cannot be used by VXLAN networks. The only way that VXLANs can currently interconnect with traditional VLANs is through VM-based software routers. Starting with Release 5.2(1)SV3(1.15), Cisco Nexus 1000V for VMware vSphere does not support the Note VXLAN Gateway feature.
BGP peering between 16 VSMs to allow VXLAN segments to reach across servers. BGP runs on the VSM and can exchange VXLAN information with the BGP on any other Cisco Nexus 1000V. The Cisco Nexus 1000V can also be used as a route reflector to exchange a VTEP list between VSMs.
Page 248
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 172.23.181.67:5000 (EVI 5000) # RD = <Router-id>:<segment-id> *>l[3]:[5000]:[4]:[192.168.69.3]/88 #Local VTEP 192.168.69.3 0.0.0.0 100 32768 i Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-4 OL-31593-01...
Page 249
VTEP BGP routing table information for VRF default, address family L2VPN EVPN BGP table version is 17, local router ID is 192.168.66.10 Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-5 OL-31593-01...
Page 250
Last written 00:00:59, keepalive timer expiry due 0.819374 Received 4006 messages, 0 notifications, 0 bytes in queue Sent 4008 messages, 0 notifications, 0 bytes in queue Connections established 1, dropped 0 Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-6 OL-31593-01...
VXLAN encapsulation and if the frame does not contain an IP packet. Scalability Maximum Number of VXLANs The Cisco Nexus 1000V supports a total of 4000 and 6144 bridge domains. VSM-DAOX(config-port-prof-srv)# show resource-availability vlan Maximum number of user VLANs supported: 4093...
• Jumbo Frames Jumbo frames are supported by the Cisco Nexus 1000V if there is space on the frame to accommodate the VXLAN encapsulation overhead of at least 50 bytes, and the physical switch/router infrastructure has the capability to transport these jumbo-sized IP packets.
Page 254
0 port_count: 2 action: 4 hwbd: 28 pa_count: 0 Veth2, Veth5 switch(config)# Example 25-6 show system internal seg_bd info port switch# show system internal seg_bd info port if_index = <0x1c000010> Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-10 OL-31593-01...
= 4098 switch# VXLAN Gateway Commands Note Starting with Release 5.2(1)SV3(1.15), Cisco Nexus 1000V for VMware vSphere does not support the VXLAN Gateway feature. To display VXLAN Gateway information that is attached to the VSM: switch# show module vem...
Page 257
Bridge-domain: segment-cisco VTEP Table Version: 2 Note: You can compare the VTEP table version with the echo show vxlan version-table on VEM. Ifindex Module VTEP-IP Address ------------------------------------------------------------------------------ Veth4 10.106.199.116(D) Veth1 10.106.199.117(D) Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-13 OL-31593-01...
Page 258
Use “vemcmd show port vlans” to verify that the VTEPs are in the correct transport VLAN. To verify bridge domain creation on the VEM: ~ # vemcmd show bd bd-name vxlan-home Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-14 OL-31593-01...
Page 259
Portlist: RedHat_VM1_112.eth4 RedHat_VM1_112.eth5 To display the MAC address table that shows the MAC addresses delivered by the VSM: switch# vemcmd show l2 bd-name segment-cisco Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-15 OL-31593-01...
Note: You can compare the download sequence number against the VTEP download sequence number using the vemcmnd show bd bd-name. Displays if the MAC address table displays the remote IP learning in the segment-cisco bridge domain: switch# vemcmd show l2 bd-name segment-cisco Note - Use the module command to check the details of VEM and gateway on the VSM.
You can view the output for all the above logs by using the module vem 4 execute vemlog show all command. VEM Multicast Debugging Use the following command to debug VEM multicast. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-17 OL-31593-01...
"debug dpa_allplatform all" > /tmp/dpafifo To debug the bridge domain configuration, use the following command: echo “debug sfl2agent all” > /tmp/dpafifo To debug the port configuration, use the following command: Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-18 OL-31593-01...
Vempkt has been enhanced to display the VLAN/SegmentID. Use vempkt to trace the packet path through the VEM. Encapsulated: Capture ingress on Seg-VEth LTL – Egress on uplink • Decapsulated: Capture ingress on uplink – Egress on Seg-VEth LTL • Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-19 OL-31593-01...
Displays the remote IP being learned. vemcmd show l2 bd-name bd-name-string Displays the Layer 2 table for one segment bridge domain. vemcmd show arp all Displays the IP-MAC mapping for the outer encapsulated header. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 25-20 OL-31593-01...
• Information About VDP VDP on the Cisco Nexus 1000V is an implementation of the IEEE standard 802.1Qbg/D2.2 (Edge Virtual Bridging). VDP can detect and signal the presence of end hosts and exchange capability with an adjacent VDP-capable bridge. VDP serves as a reliable first-hop protocol and communicates the presence of end-host Virtual Machines (VMs) to adjacent leaf nodes on the Cisco Dynamic Fabric Automation (DFA) architecture.
VM and use this command. A VSI state of 3 means that it is associated. Example 26-1 on page 26-3. show evb Displays configured information in the EVB process. Example 26-2 on page 26-3. Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 26-2 OL-31593-01...
This packet classification is maintained by tagging packets on ingress to the Cisco TrustSec network so that they can be properly identified for the purpose of applying security and other policy criteria along the data path. The tag, also called the security group tag (SGT), allows the network to enforce the access control policy by enabling the endpoint device to act upon the SGT to filter traffic.
Cisco TrustSec Cisco TrustSec Troubleshooting Commands Debugging Commands Command Purpose debug cts authentication Collects and views logs related to Cisco TrustSec authentication. debug cts authorization Collects and views logs related to Cisco TrustSec authorization. debug cts errors Collects and views logs related to Cisco TrustSec errors and warning messages.
Page 273
Cisco Nexus 1000V. See Example 27-1 on page 27-3 vemcmd show cts global Displays if Cisco TrustSec is enabled on the Cisco Nexus 1000V. See Example 27-2 on page 27-3 vemcmd show cts ipsgt Displays the Cisco TrustSec configuration on the Cisco Nexus 1000V.
Chapter 27 Cisco TrustSec Cisco TrustSec Troubleshooting Commands show Commands See the Cisco Nexus 1000V Command Reference for more information on the show commands for Cisco TrustSec. Command Purpose show cts Displays the Cisco TrustSec configuration. show cts sxp Displays the SXP configuration for Cisco TrustSec.
Chapter 27 Cisco TrustSec Problems with Cisco TrustSec Problems with Cisco TrustSec This section includes symptoms, possible causes and solutions for the following problems with Cisco TrustSec. Symptom Possible Causes Verification and Solution The Cisco Nexus 1000V is There is no connection between the...
Web Clients only. The VMware vSphere Web Client enables you to connect to a VMware vCenter Server system to manage a Cisco Nexus 1000V through a browser. The vCenter Plug-in is installed as a new tab called Cisco Nexus 1000v as part of the user interface in the vSphere Web Client.
– vSphere Web Client requires the Adobe Flash Player version 11.1.0 or later to be installed. • Make sure that Cisco Nexus 1000V Release 4.2(1)SV2(1.1) is installed and configured to a vCenter. • Generating a Log Bundle You can collect the diagnostic information for VMware vCenter Server by collecting vSphere log files into a single location.
C H A P T E R Ethanalyzer This chapter describes how to use Ethanalyzer as a Cisco NX-OS protocol analyzer tool and includes the following section: Using Ethanalyzer, page 29-1 • Using Ethanalyzer Ethanalyzer is a Cisco NX-OS protocol analyzer tool based on the Wireshark (formerly Ethereal) open source code.
Page 280
2012-10-01 19:15:23.796608 10.78.110.241 -> 72.163.145.51 SSH Encrypted response packet len=144 2012-10-01 19:15:23.797060 10.78.110.241 -> 72.163.145.51 SSH Encrypted response packet len=144 4 packets captured switch# For more information about Wireshark, see the following URL: http://www.wireshark.org/docs/ Cisco Nexus 1000V Troubleshooting Guide, Release 5.2(1)SV3(1.1) 29-2 OL-31593-01...