Cisco Nexus 7000 Series Configuration Manual
  1. Manuals
  2. Brands
  3. Cisco Manuals
  4. Switch
  5. Nexus 7000 Series
  6. Configuration manual

Cisco Nexus 7000 Series Configuration Manual

Nx-os vxlan
Hide thumbs Also See for Nexus 7000 Series:
Table of Contents

Advertisement

Quick Links

Download this manual See also: Reference Manual, Configuration Manual
Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide
First Published: 2016-11-24
Last Modified: --
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883

Advertisement

Table of Contents
loading

Related Manuals for Cisco Nexus 7000 Series

Summary of Contents for Cisco Nexus 7000 Series

  • Page 1 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide First Published: 2016-11-24 Last Modified: -- Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883...
  • Page 2 © 2016 Cisco Systems, Inc. All rights reserved.

  • Page 3: Table Of Contents

    VXLAN with vPC Overview VXLAN Layer 2 Gateway VXLAN Layer 3 Gateway VXLAN Licensing Information Guidelines and Limitations for VXLAN Considerations for VXLAN Deployment vPC Considerations for VXLAN Deployment Network Considerations for VXLAN Deployments Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 4 Feature History for VXLAN BGP EVPN Configuring ACI WAN Interconnect C H A P T E R 6 VXLAN EVPN - MPLS L3VPN for ACI Fabric Prerequisites for Configuring ACI WAN Interconnect Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 5 LISP Support for Disjointed RLOC Domains C H A P T E R 9 LISP Support for Disjointed RLOC Domains Overview of LISP Support for Disjointed RLOC Domains Prerequisites for LISP Support for Disjointed RLOC Domains Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 6 Guidelines and Limitations for VXLAN BGP EVPN and OTV Interoperation Information About VXLAN BGP EVPN and OTV Interoperation Sample Topologies and Workflow of the VXLAN BGP EVPN and OTV Interoperation Layer 2 Switching Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 7 Display OTV Configuration on the Border Leaf Switch BL1 Display OTV Overlay State Display OTV Adjacencies Display Tier IDs Troubleshooting VXLAN BGP EVPN and OTV Interoperation, and OTV with BDI Feature History for VXLAN BGP EVPN and OTV Interoperation Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 8 Contents Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide viii...

  • Page 9: Chapter

    • Obtaining Documentation and Submitting a Service Request, page 3 Audience This publication is for experienced network administrators who configure and maintain Cisco NX-OS on Cisco Nexus 7000 Series Platform switches. Document Conventions • As part of our constant endeavor to remodel our documents to meet our customers' requirements, Note we have modified the manner in which we document configuration tasks.
  • Page 10 An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line. This document uses the following conventions: Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 11: Related Documentation

    What's New in Cisco Product Documentation. To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the What's New in Cisco Product Documentation RSS feed. RSS feeds are a free service.
  • Page 12 Preface Obtaining Documentation and Submitting a Service Request Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 13: Chapter

    C H A P T E R New and Changed Information This chapter provides release-specific information for each new and changed feature in the Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide. • New and Changed Information, page 5 New and Changed Information The table below summarizes the new and changed features for this document and shows the releases in which each feature is supported.
  • Page 14 EVPN Support for M3 modules is 7.3(0)DX(1) introduced. ACI WAN This feature was introduced. 7.3(1)D1(1) Configuring ACI WAN Interconnect Interconnect Campus Fabric This feature was introduced. 7.3(1)D1(1) Campus Fabric Related Topics http://rtp-aspw-ccms1.cisco.com/astoria/_id_00000013UK9120YZ_CISCO!psosale Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 15: Chapter

    Layer 2 segments from the shared transport network. When deployed as a VXLAN gateway, Cisco Nexus 7000 switches can connect VXLAN and classic VLAN segments to create a common forwarding domain so that tenant devices can reside in both environments.

  • Page 16: Vxlan Flood And Learn

    VTEPs • Gateway functions centralised in VXLAN flood and learn • Cisco Nexus 7000 / 7700 vPC pair with L2 + L3 VXLAN gateway capabilities • vPC provides MAC state synchronization and active-active HSRP forwarding • Redundant VTEPs share Anycast VTEP IP address in underlay •...

  • Page 17: Vxlan Tunnel Endpoint

    RFC 4364 can be employed. It thus allows a seamless stitching together of a VPN that spans both an IP based network overlay and a MPLS VPN. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 18: Vxlan Bgp Evpn Control Plane

    • The inner destination MAC address in the VXLAN header does not belong to the host but to the receiving VTEP that does the routing of the VXLAN payload. This MAC address is distributed via BGP attribute along with EVPN routes. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 19 IP hosts have an associated MAC address, coexistence of both Layer 2 VXLAN and Layer 3 VXLAN Note overlays are supported. Additionally, the Layer 2 VXLAN overlay will also be used to facilitate communication between non-IP based (Layer 2 only) hosts. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 20 Overview VXLAN BGP EVPN Control Plane Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 21: Configuring Vxlan Flood And Learn

    • Since hardware does not have a mechanism to prevent both switches from sending and receiving packets to and from the North, one of the vPC peers is selected as the forwarder by PIM. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 22: Vxlan Layer 2 Gateway

    VNI. • Connectivity of vSwitches to Cisco Nexus 7000 is via a Layer 2 port through a VLAN which is called a hypervisor VLAN. One of the requirements for a VXLAN gateway is that the hypervisor VLAN should be Layer 3 enabled (SVI configured) and be a member of the core VRF.
  • Page 23 • Since the Layer 3 gateway is centralized, there is no need to run control protocols (to advertise the host reachability information). When the Layer 3 gateway receives the packet, it looks at the Layer 3 header Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 24 (Layer 3 gateway is achieved by configuring the BDIs for bridge-domains corresponding to the VNIs), case (1) will not be applicable in the Cisco Nexus 7000, and only case (2) is supported. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 25: Vxlan Licensing Information

    • Number of IPv4 Multicast Groups is 32K due to software limitation. • Maximum number of MAC addresses learned (local MACs and remote MACs) is 64K per F3 ASIC. (F3 MAC table size is 64K). Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 26 • Any Source Multicast (ASM) is supported. Bidirectional PIM is supported on a single, non-vPC leaf switch. • Physical port vPC for Vn-segment Service Instance (VSI) is not supported. • The following Interface NVE counters are supported. ◦ Unicast and Multicast packets and bytes transmitted Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 27: Considerations For Vxlan Deployment

    • VPC peers must have identical configurations as listed below: ◦ Consistent Bridge-domain to VNI mapping. ◦ Consistent NVE binding to the same loopback interface. ◦ Using the same secondary IP address. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 28 The following is an example (best practice) of a VPC configuration: switch# sh ru vpc version 6.1(2)I3(1) feature vpc vpc domain 2 peer-switch peer-keepalive destination 172.29.206.65 source 172.29.206.64 peer-gateway ipv6 nd synchronize Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 29: Network Considerations For Vxlan Deployments

    • ECMP and LACP Hashing Algorithms in the Transport Network As described in a previous section, Cisco Nexus 7000 Series Switches introduce a level of entropy in the source UDP port for ECMP and LACP hashing in the transport network. As a way to augment this...

  • Page 30: Considerations For The Core Vrf

    ◦ Enable and configure IP multicast. ISSU Support The following are the ISSU support details for VXLAN flood and learn deployment: • Cisco Nexus 7000 Series switches running Cisco NX-OS Release 6.2.10 or 6.2.12. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 31: Configuring Vxlan

    Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration. switch# configure terminal switch(config)# feature nv overlay switch(config)# feature vni switch(config)# vni 7000 switch(config)# copy running-config startup-config Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 32: Configuring Vni Service Instances

    Enables bridging to map VLAN to VXLAN VNI. Step 4 member vni number Maps VXLAN to a bridge domain. The following example shows how to map an encapsulation profile to a VNI: switch# configure terminal switch(config)# system bridge-domain 100-500 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 33: Creating An Vtep And Nve Interface

    1 switch(config-if)# source-interface loopback 10 switch(config-if)# member vni 7000 mcast-group 225.1.1.1 switch(config-if)# member vni 8000 mcast-group 226.1.1.1 switch# configure terminal switch(config)# interface loopback 10 switch(config-if)# ip address 10.1.1.1/32 switch(config-if)# vrf member core Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 34: Configuring Vpc Peer-Link

    Step 2 interface var Enters interface configuration mode. Step 3 ip address address Configures the IP address on the interface. Step 4 vrf member core Creates a vrf member core in the interface. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 35: Configuring L3 Interface For Ip Cloud Connectivity

    Configures the IP address on the interface. Step 5 vrf member tenant Configures the VRF member. Step 6 hsrp var Creates an HSRP group and enters HSRP configuration mode. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 36: Disabling Vxlans

    Removes the VXLAN segment ID to which the VLAN no vni is mapped. Step 7 copy running-config (Optional) Saves the change persistently through reboots and startup-config restarts by copying the running configuration to the startup configuration. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 37: Verifying The Vxlan Configuration

    Displays VXLAN parameters, such as VXLAN destination or UDP port. Displays NVE overlay internal detailed information. show nve internal platform interface nve 1 detail show nve vxlan-params Displays VXLAN parameters, such as VXLAN destination or UDP port. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 38: Configuration Examples

    VSI_50_TO_5000 default service instance 2 vni no shutdown encapsulation profile VSI_75_TO_7500 default interface Ethernet10/1 no switchport ip address 10.1.1.1/30 ip pim sparse-mode no shutdown interface loopback10 ip address 10.10.10.1/32 ip pim sparse-mode Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 39 VSI_50_TO_5000 default service instance 2 vni no shutdown encapsulation profile VSI_75_TO_7500 default interface Ethernet10/7 no switchport ip address 10.1.1.2/30 ip pim sparse-mode no shutdown interface loopback10 ip address 10.10.10.2/32 ip pim sparse-mode Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 40: Example Of Verifying Vxlan Flood And Learn Configuration

    Note: MAC table entries displayed are getting read from software. Use the 'hardware-age' keyword to get information related to 'Age' Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 41 Stats: 300/39850 [Packets/Bytes], 0.000 Incoming interface: loopback10, RPF nbr: 10.10.10.1, internal Outgoing interface list: (count: 1) Ethernet10/1, uptime: 12:51:52, pim (10.10.10.2/32, 209.165.1.1/32), uptime: 12:51:34, pim(1) mrib(1) ip(0) Data Created: Yes VXLAN Flags Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 42 5000 209.166.1.1 L2 [50] nve1 7500 192.168.1.1 L2 [75] VTEP-2# show running-config interface nve 1 interface nve1 no shutdown source-interface loopback10 member vni 5000 mcast-group 209.166.1.1 member vni 7500 mcast-group 192.168.1.1 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 43 (*, 209.165.1.1/32), uptime: 19:56:19, nve(1) ip(0) pim(0) Data Created: No VXLAN Flags VXLAN Encap Stats: 8/748 [Packets/Bytes], 0.000 Incoming interface: Ethernet10/7, RPF nbr: 1.1.1.1 Outgoing interface list: (count: 1) nve1, uptime: 19:56:19, nve Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 44 *via 10.10.10.2, Lo10, [0/0], 20:29:39, local *via 10.10.10.2, Lo10, [0/0], 20:29:39, direct 10.50.50.0/24, ubest/mbest: 1/0, attached *via 10.50.50.51, Bdi50, [0/0], 01:22:50, direct 10.50.50.51/32, ubest/mbest: 1/0, attached *via 10.50.50.51, Bdi50, [0/0], 01:22:50, local 10.75.75.0/24, ubest/mbest: 1/0, attached Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 45: Feature History For Vxlan Flood And Learn

    This table lists the release history for this feature. Table 2: Feature History for VXLAN Flood and Learn Feature Name Releases Feature Information VXLAN Flood and Learn 7.2(0)D1(1) This feature was introduced. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 46 Configuring VXLAN Flood and Learn Feature History for VXLAN Flood and Learn Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 47: Chapter 5 Configuring Vxlan Bgp Evpn

    ToRs, a VTEP refers to a ToR or leaf switch enabled with the VTEP function. Note that the VTEP functionality is enabled on all leaf switches in the VXLAN fabric and on border leaf/spine switches. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 48 ToR1 encapsulates the frame in a VXLAN packet, and sends it in the direction of ToR2. The devices in the path between ToR1 to ToR2 are not aware of the original frame and route/switch the packet to ToR2. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 49 The control plane of choice is MP-BGP EVPN. By implementing MP-BGP EVPN with VXLAN, the following is made possible: • End hosts’ information is available to the attached ToR via First Hop Protocols such as ARP/ND/DHCP etc., when a new bare-metal server or VM is attached. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 50: Realizing Layer-2 And Layer-3 Multi-Tenancy

    Traffic between servers in the same tenant network that is confined to the same subnet is bridged. In this case, the VTEPs stamp the layer-2 VNI in the VXLAN header when the communication is between servers that Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 51: Fabric Overlay Control-Plane (Mp-Bgp Evpn)

    When a new end host is attached to a VTEP (aka ToR), the VTEP advertises the MAC and IP address of the end host to a route reflector which in turn advertises it to the other VTEPs through MP-BGP (as Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 52 • This ensures that routed traffic is also optimally forwarded within the fabric without going through any tromboning. • VM Mobility Support • The control plane supports transparent VM mobility and quickly updates reachability information to avoid hair-pinning of east-west traffic. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 53: End Host And Subnet Route Distribution

    For redundancy purposes, an additional route reflector is designated. End Host and Subnet Route Distribution Some pointers about end host MAC and IP route distribution in a VXLAN EVPN fabric are given below: Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 54 IP and MAC address for intra-tenant communication. If an end host moves to a different VTEP, the gateway information remains the same and reachability information is available in the BGP control plane. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 55 After the configurations, end host traffic within a VNI (say 30000) is bridged, and traffic between tenant networks is routed. The routing takes place through a Layer-3 VNI (say 50000) typically having a one-on-one association with a VRF instance. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 56 • VTEP V1 bridges the packets and sends it toward VTEP V2 with a VXLAN header stamped with the Layer 2 VNI 30000. • VTEP V2 receives the packets, and post decapsulation, lookup, bridges them to Host B. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 57 1 A VLAN is configured for each segment - sending segment, VRF segment and receiving segment. 2 BGP and EVPN configurations ensure redistribution of this information across the VXLAN setup. Real time behavior Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 58: Arp Suppression

    ARP-suppression feature is enabled. The ToR will learn about the destination IP/MAC and in turn advertise it over BGP-EVPN to all the other ToRs. In addition, the ToR will reinject the ARP response packet Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 59 ToR switch in the output interface index flood list (OIFL) and excluding overlay Layer-3 ports in the hardware. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 60: Performing End Host Detection, Deletion And Move

    1 VTEP V1 learns Host A's MAC and IP address (MAC_A and IP_A). 2 V1 advertises MAC_A and IP_A to the other VTEPs V2 and V3 through the route reflector. 3 The choice of encapsulation (VXLAN) is also advertised. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 61 1 V3 detects Host A and advertises it with Sequence 1 (updating the previous instance of the sequence, 0). The next hop IP address is reassigned to that of VTEP 3. Figure 16: Host A – Updated Parameters 2 VTEP V1 detects a more recent route and withdraws its advertisement. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 62: Multi-Destination Traffic

    Also, you can use the VXLAN OAM functionality as a workaround. Multi-Destination Traffic Refer to the table below to know the multicast protocol(s) for your Cisco Nexus switches support:: If you are using this Nexus switch: Use this option for BUM traffic...
  • Page 63 Not Supported Supported complex MPLS Not Supported Supported Supported Not Supported Supported (L3VPN) Handoff PIM Bidir Not Supported Supported Supported Not Supported Supported underlay Not Supported Supported Supported Not Supported Not Supported Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 64: Configuring Vxlan Bgp Evpn

    Configuring VXLAN BGP EVPN BGP EVPN and Overlay Configuration The following BGP, EVPN and overlay configurations are required for the Cisco Nexus 7000 Series and 7700 Series switches with F3 and M3 modules: 1 Initial configuration - Install the network virtualization overlay, BGP, and EVPN features on the VTEPs.
  • Page 65 While the system bridge-domain command identifies the bridge domain IDs, the bridge-domain command configures the specified bridge domain(s). Associate a VLAN (or dot1q tag) with the Layer 2 VNI: (config) # encapsulation profile vni cisco dot1q 50 vni 30000 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 66 Note that with the Cisco Nexus 7000 Series switches, a VNI is associated with a bridge-domain (1:1). Refer to the respective configuration guide for more information on bridge-domains. The combination of the router BGP command (configured earlier) and the evpn command ensures that BGP EVPN is configured to advertise ‘MAC address + associated host route (optional)’...
  • Page 67 Use the 'hardware-age' keyword to get information related to 'Age' Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link, E - Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 68 32768 i *>i[2]:[0]:[0]:[48]:[2010.0000.0011]:[0]:[0.0.0.0]/216 10.1.1.56 *>i[2]:[0]:[0]:[48]:[2010.0000.0012]:[0]:[0.0.0.0]/216 10.1.1.74 *>i[2]:[0]:[0]:[48]:[2010.0000.0013]:[0]:[0.0.0.0]/216 10.1.1.56 *>i[2]:[0]:[0]:[48]:[8080.c800.0038]:[0]:[0.0.0.0]/216 10.1.1.74 *>l[2]:[0]:[0]:[48]:[2010.0000.0010]:[32]:[209.165.202.139]/272 10.1.1.54 32768 i *>i[2]:[0]:[0]:[48]:[2010.0000.0011]:[32]:[209.165.202.140]/272 10.1.1.56 *>i[2]:[0]:[0]:[48]:[2010.0000.0012]:[32]:[209.165.202.141]/272 10.1.1.74 *>i[2]:[0]:[0]:[48]:[2010.0000.0013]:[32]:[209.165.202.142]/272 10.1.1.56 *>i[2]:[0]:[0]:[48]:[8080.c800.0038]:[32]:[209.165.202.143]/272 10.1.1.74 Route Distinguisher: 10.1.1.56:3 *>i[5]:[0]:[0]:[24]:[209.165.202.130]:[0.0.0.0]/224 10.1.1.56 Route Distinguisher: 10.1.1.56:32967 *>i[2]:[0]:[0]:[48]:[2010.0000.0011]:[0]:[0.0.0.0]/216 10.1.1.56 *>i[2]:[0]:[0]:[48]:[2010.0000.0013]:[0]:[0.0.0.0]/216 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 69 The following sample output displays MAC - IP address binding for all attached and remote end hosts (learned through the BGP EVPN control plane). switch # show l2route evpn mac-ip all Topology ID Mac Address Prod Host IP Next Hop(s) ----------- -------------- ---- --------------------------------------- -------- Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 70: Feature History For Vxlan Bgp Evpn

    This table lists the release history for this feature. Table 3: Feature History for VXLAN BGP EVPN Feature Name Releases Feature Information VXLAN BGP EVPN 7.2(0)D1(1) This feature was introduced. 7.3(0)DX(1) Support for M3 modules is introduced. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 71: Configuring Aci Wan Interconnect

    VXLAN EVPN - MPLS L3VPN for ACI Fabric Prerequisites for Configuring ACI WAN Interconnect • A Cisco Nexus 7000 Series switch with an F3/M3 line card. Feature History for ACI WAN Interconnect This table lists the release history for this feature.

  • Page 72: Overview Of Vxlan Evpn - Mpls L3Vpn For Aci Fabric

    ACI WAN Interconnect is a multi-platform, multi-OS Data Center Interconnect (DCI) architecture. It connects multi-tenant VXLAN data center fabrics over L3VPN. The Cisco Application Centric Infrastructure (ACI) allows application requirements to define the network. This architecture simplifies, optimizes, and accelerates the entire application deployment life cycle.
  • Page 73 • North-to-South traffic tunneled to Spine will get routed on spine-to-leaf based on /32 lookup. • Routes advertised from DCI to Spine will get reflected to leaves with the DCI VTEP as the next-hop. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 74: Spine - Dci Connectivity

    The MPLS-L3VPN hand-off for ACI fabric can be deployed using one of the following topologies: • Single POD with multiple DCI gateways • Multi-POD with shared DCI gateway • Multi-POD with Separate DCI gateway Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 75 DCI gateway. A set of VRFs are imported and advertised on one DCI pair, while another set of VRFs could be present on another DCI pair. Fabric spines advertise all routes to all DCI pairs, but only configured VRF routes are imported and advertised towards L3VPN PE on the respective DCIs. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 76 In this topology multiple PODs share the same DCI gateway. The DCI pair imports and advertises VRF routes from multiple POD spines. DCI pair has underlay connectivity to multiple PODs over an inter-POD network underlay. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 77: Opflex Dci Auto-Configuration

    OpFlex DCI Auto-Configuration Cisco OpFlex is a southbound protocol in a software-defined network (SDN) designed to facilitate the communications between the SDN Controller and the infrastructure (switches and routers). The goal is to Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 78: Interconnect Policy Provisioning (Ipp)

    VRF from multiple OpFlex frameworks, possibly with different RTs. DCI handles this multiple update scenario by appending the route targets for the POD if the fabric facing local VRF configuration has already been instantiated. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 79: Dci Auto-Configuration Scenario

    ACI VTEPs, BGP peers, and OpFlex proxy on the ACI infra subnet is through the underlay routing to/from this external DCI subnet. The following sections describe the OpFlex configuration steps. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 80 10.10.10.1/24 # VXLAN local TEP interface NVE 1 source-interface loopback0 host-reachability protocol bgp unknown-peer-forwarding enable vxlan udp port 48879 # underlay routing router ospf area 0 interface loopback0 interface e3/1 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 81 $include_client_import_ipv6_bgpRT_8 evpn route-target export $include_client_export_ipv6_bgpRT_8 evpn* router bgp $asn vrf $vrfName address-family ipv4 unicast advertise l2vpn evpn label-allocation-mode per-vrf address-family ipv6 unicast advertise l2vpn evpn label-allocation-mode per-vrf interface nve $nveId Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 82 # DCI Setup infra connectivity to OpFlex (interfaces are fabric facing) interface e3/1.1 no shutdown encapsulation dot1q 4 ip address 10.1.1.1/24 ip ospf network point-to-point ip router ospf 100 area 0.0.0.100 # Add IPP owned per ACI/OpFlex instance configuration Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 83: Show And Debug Command Examples

    Show various event logs of IPP mem-stats Dynamic memory stats Internal IPP pss info work-info Internal IPP worker thread info switch# show ipp internal debug IPP Debug information Debug Flags : Off Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 84 [100] [6495]: comp-mts-rx opc - from sap 19164 cmd ipp_show_internal_event_h ist_cmd 3) Event:E_DEBUG, length:49, at 882139 usecs after Mon May 30 11:33:45 2016 [100] [19410]: nvdb: terminate transaction failed …………………………………… switch# show ipp internal event-history opflex Process opflex logs of IPP Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 85 Total size Library -------------------------------------------------------------------------------- 155244 162128 64567 70336 ld-2.15.so 5712 8504 libc-2.15.so libdl-2.15.so librt-2.15.so 22877 1053813 1533896 libstdc++.so.6.0.16 2294 2872 libvlan_mgr.so.0.0.0 2860 3144 libsviifdb.so.0.0.0 libltlmap.so.0.0.0 libindxobj.so.0.0.0 6472 11072 libast_db.so.0.0.0 2304 4648 libavl.so.0.0.0 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 86 Fabric id Fabric vrf name : coke-11:coke-11 Vrf name : dci_coke-11 Tenant id : 351 Hmm hostid : 351 V4 RT (import/export) : 1:11/1:11 V6 RT (import/export) : 1:11/1:11 Flags : 0x0 …………………… Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 87 Id 266, HostId: 266 flags 0x0 3: Fabric Vrf: coke-100:coke-100, Vrf: dci_coke-100 RT v4:(1:100,1:100) v6:(1:100,1:100) Id 346, HostId: 346 flags 0x0 ……………… switch# show tech-support ipp `show running-config ipp` !Command: show running-config ipp Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 88 285 …………. switch# debug ipp ? All debugs CLI command processing debugs event IPP events HA related debugs IPP HMM api debug opflex IPP opflex debugs periodic IPP events periodic Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 89: Chapter 7 Campus Fabric

    This feature enables a LISP-based Control Plane for VXLAN Fabric. This feature is supported only on the M3 module. The Cisco Nexus 7700 Series with M3 Module acts as a fabric border which connects traditional Layer 3 networks or different fabric domains to the local fabric domain, and translates reachability and policy information from one domain to another.
  • Page 90 Campus Fabric Overview of Campus Fabric Cisco Nexus 7700 is positioned as a fabric border node in the Campus Fabric architecture. Figure 22: Campus Fabric Architecture The key elements of the Campus fabric architecture are explained below. Campus Fabric : The Campus Fabric is an instance of a "Network Fabric". A Network Fabric describes a network topology where data traffic is passed through interconnecting switches, while providing the abstraction of a single Layer-2 and/or Layer-3 device.
  • Page 91 SGT is encoded in the reserved bits of the VXLAN header. Cisco Catalyst 3000 is positioned as the fabric edge and Cisco Nexus 7700 is positioned as the fabric border in this architecture. LISP is the control plane in the campus fabric architecture and it programs the VXLAN routes.

  • Page 92: Vxlan Encapsulation For Layer-3 Lisp Configuration

    9.9.9.9 ip lisp itr map-resolver 9.9.9.9 ip lisp etr map-server 9.9.9.9 key 3 a97b0defe7b8ff70 ip lisp multicast lisp encapsulation vxlan OVERLAY VRF */ vrf context vrf5000 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 93 111.1.1.1/32 ip pim sparse-mode IGP on the UNDERLAY VRF */ router isis 100 net 49.0001.1111.1111.1111.00 vrf core net 49.0001.1111.1111.1111.00 vrf vrf5000 /* BGP neighbor towards the CORE */ Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 94: Feature History For Campus Fabric

    CLI. On enabling this CLI, the TTL propagation will be disabled from the inner header to the outer header during encapsulation. This is enabled using the lisp disable-ttl-propagate command. Feature History for Campus Fabric This table lists the release history for this feature. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 95 Campus Fabric Feature History for Campus Fabric Table 5: Feature History for Campus Fabric Feature Name Releases Feature Information Campus Fabric 7.3(1)D1(1) This feature was introduced. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 96 Campus Fabric Feature History for Campus Fabric Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 97: Campus Fabric Interconnect - Mpls L3Vpn

    Multiprotocol Label Switching (MPLS) L3VPN. The focus of the feature is the role of the Cisco Nexus 7000/7700 Series border leaf switch which sends end host traffic from the fabric to an end host in a remote fabric over MPLS (through the MPLS core).

  • Page 98: Information About Campus Fabric Interconnect-Mpls L3Vpn

    Information About Campus Fabric Interconnect—MPLS L3VPN Information About Campus Fabric Interconnect—MPLS L3VPN Sample topology and traffic flow between two campus fabrics connected through MPLS L3VPN: Figure 24: Sample topology - Campus Fabric Interconnect — MPLS L3VPN Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 99: Campus Fabric Architecture-Fabric 1

    Campus Fabric Architecture—Fabric 1 Fabric 1 and Fabric 2 are two campus fabrics. PxTR 1 and PxTR 2 are Cisco Nexus 7000/7700 Series switches that perform the role of border switches in Fabric 1. PxTR 2 is the fabric border switch in Fabric 2. MPLS configurations are enabled on the PxTR switches such that Fabric 1 and Fabric 2 are connected through MPLS L3VPN between PxTR 1/PxTR 2 and PxTR 3.

  • Page 100: How To Configure Campus Fabric Interconnect-Mpls L3Vpn

    6000 associate-vrf Step 2 Configure LISP related commands Configure LISP parameters and route distinguisher and route target functions for the vrf6000 VRF: PxTR 1(config)# vrf context vrf6000 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 101: Campus Fabric Interconnect Configuration

    PxTR 1(config)# router bgp 100 router-id 209.165.201.1 address-family ipv4 unicast • The IPv4 address family and router ID configurations are enabled. neighbor 209.165.200.225 remote-as 5000 update-source loopback299 ebgp-multihop 10 address-family vpnv4 unicast Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 102: Verifying Campus Fabric Interconnect-Mpls L3Vpn

    TCP connection: 203.0.113.1.646 - 209.165.201.1.63118 State: Oper; Msgs sent/rcvd: 69/71; Downstream Up time: 00:53:49 LDP discovery sources: Ethernet2/20.1, Src IP addr: 192.0.2.250 Addresses bound to peer LDP Ident: 203.0.113.1 172.16.0.1 192.0.2.250 203.0.113.10 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 103: Verifying Mpls Label Switching Vrf Information

    Feature History for Campus 8.2(1) This feature was introduced. This feature explains how to Fabric Interconnect—MPLS L3VPN enable traffic flow across two campus fabrics through MPLS L3VPN. No new commands were introduced for this feature. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 104 Campus Fabric Interconnect - MPLS L3VPN Feature History for Campus Fabric Interconnect—MPLS L3VPN Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 105: Chapter 9 Lisp Support For Disjointed Rloc Domains

    When some sites within a network connect to one routing domain and other sites connect to another routing domain, a gateway function must be provided to facilitate connectivity between these disjointed routing Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 106: Lisp Map-Server

    • Since an RTR performs functions similar to a Proxy Ingress Tunnel Router (PITR) and Proxy Egress Tunnel Router (PETR), the PITR and PETR features must be enabled on the RTR. Cisco Nexus 7000 Series device is used for the PxTR (a device performing PITR and PETR functions) Note and RTR functions.

  • Page 107: Workflow Of Lisp Support For Disjointed Rloc Domains

    • Two virtual routing and forwarding (VRF) instances are created on the RTRs, one for the underlay (VRF core), and one for the overlay (VRF vrf5000). Note Map-Servers and RTRs can be connected to eight locator scopes or address spaces. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 108: How To Configure Lisp Support For Disjointed Rloc Domains

    Step 2 Create two VRF instances on the RTR, one for the underlay (VRF core), and one for the overlay (VRF vrf5000). Configure LISP parameters for the core VRF (config)# vrf context core Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 109 The configured loopback interface IP address is used for IS-IS communication within the LISP site, and is added to VRF core. Step 4 The configurations are relevant for RTR or PxTR 1. Similarly, configure the RTR or PxTR 2 device too. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 110: Pxtr 1 Or Rtr Configuration-Rtr Locator-Set Inheritance

    The configured loopback interface IP address is used for IS-IS communication within the LISP site, and is added to VRF core. Step 4 The configurations are relevant for RTR or PxTR 1. Similarly, configure the RTR or PxTR 2 device too. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 111: Map-Server Configuration

    Map-Server to consider disjointed RLOCs in its Map-Request handling logic. • A device with IOS XE software is used for the role of Map-Server, and not a Cisco Nexus 7000 Series device. The Map-Server configuration is documented for reference and completeness. For information, see IP Routing: LISP Configuration Guide, Cisco IOS XE Release 3S.

  • Page 112: Eid Space Details In The Map-Server/Map-Resolver (Msmr)

    Allowed locators mismatch: 0 ETR 203.0.113.40, last registered 08:12:10, no proxy-reply, map-notify TTL 1d00h, no merge, hash-function sha1, nonce 0x4CC82237-0x6DCB0FC5 state complete, no security-capability xTR-ID 0x90FA8033-0x867FE73F-0x5F32076D-0xE92E8945 site-ID unspecified sourced by reliable transport Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 113: Verify Lisp Map-Cache Details On Pxtr 1

    This table lists the release history for this feature. Table 7: Feature History for LISP Support for Disjointed RLOC Domains Feature Name Release Feature Information Connecting LISP Disjointed 8.1(1) This feature was introduced. RLOC Domains Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 114 LISP Support for Disjointed RLOC Domains Overview of LISP Support for Disjointed RLOC Domains Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 115: C H A P T E

    Feature History for PBR Support for the VXLAN BGP EVPN Fabric, page 111 Prerequisites for PBR Support for the VXLAN BGP EVPN Fabric • A Cisco Nexus 7000 Series switch with an F3 or M3 line card. • Understand how VXLAN BGP EVPN works.

  • Page 116: Information About Pbr Support For The Vxlan Bgp Evpn Fabric

    When the request reaches the load balancer through the border leaf switch BL and the service leaf switch, it forwards the service request to an appropriate server (Server 2). PBR configurations should be enabled on L1, L2, and the service leaf switch. The workflow: Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 117: Pbr Rules On L1 And L2 (Enabled On Bdi10 And Bdi20)

    50000) received on the PBR interface, send traffic to 10.0.0.1 (or 2001:DB8:0:ABCD::1). How to Configure PBR Support for the VXLAN BGP EVPN Fabric Type the switch# configure terminal command to enter global configuration mode (config)# Note Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 118: Enable Pbr Configurations On L1, L2 And The Service Leaf Switch

    Step 4 The configurations are relevant for BDI10. Similarly, enable PBR configurations for BDI 20 on Step 5 Enable PBR configurations on BDI10 and BDI20 on L2 and service leaf switch. Verifying PBR Support for VXLAN BGP EVPN Fabric Verify PBR configurations on L1: Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 119: Verifying Pbr Route Map Policy Configuration On The Bdi

    This table lists the release history for this feature. Table 8: Feature History for PBR Support for the VXLAN BGP EVPN fabric Feature Name Release Feature Information PBR support for the VXLAN 8.2(1) This feature was introduced. BGP EVPN fabric Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 120 PBR support for the VXLAN BGP EVPN fabric Feature History for PBR Support for the VXLAN BGP EVPN Fabric Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 121: Vxlan Bgp Evpn And Otv Interoperation

    • For a functioning VXLAN BGP EVPN datacenter, configurations should be enabled on the leaf and spine switches. For more information see the "Configuring the VXLAN BGP EVPN" chapter, or Cisco Programmable Fabric with VXLAN BGP EVPN Configuration Guide. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 122: Guidelines And Limitations For Vxlan Bgp Evpn And Otv Interoperation

    VXLAN overlays. However, an OTV overlay and an OTV+VXLAN overlay can be enables on separate physical Join interfaces. • In an OTV with BDI single box solution, the ARP proxy function option is not supported in the Cisco NX-OS 8.2(1) release.

  • Page 123: Sample Topologies And Workflow Of The Vxlan Bgp Evpn And Otv Interoperation

    Sample Topologies and Workflow of the VXLAN BGP EVPN and OTV Interoperation Figure 27: Sample topology 1 - VXLAN BGP EVPN and OTV interoperation Figure 28: Sample topology 2 - VXLAN BGP EVPN and OTV interoperation Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 124: Layer 2 Switching

    Layer 2 traffic is transported between the datacenters through the border leaf switches (in DC-1 and DC-2) and OTV devices (in DC-3 and DC-4) at the site border, over the IP WAN. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 125: Control Plane

    Layer 2 access switch. The access switch forwards the packets to the destination server. Control Plane • BGP EVPN is used for advertising MAC and MAC-IP routes across the VXLAN BGP EVPN fabric in DC-1 and DC-2. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 126: Layer 3 Unicast Routing

    Switching (MPLS) L3VPN or virtual routing and forwarding (VRF) Lite. The IGP and external connectivity documentation is available in the Cisco Programmable Fabric with VXLAN BGP EVPN Configuration Guide. A distributed anycast gateway (or BDI) IP address is used for Layer 3 traffic between Layer 2 virtual networks in the VXLAN fabric.

  • Page 127: Layer 2 Multicast Forwarding And Layer 3 Multicast Routing

    • When a server in the VXLAN BGP EVPN fabric sends multicast traffic to the attached ToR/leaf switch, the leaf switch forwards the multicast traffic within the fabric, as explained in the Cisco Programmable Fabric with VXLAN BGP EVPN Configuration Guide. If there are receivers in the Layer 2 CE pod...
  • Page 128 This is the use case wherein a sender within the VXLAN fabric sends Layer 3 multicast traffic to receivers located within the fabric. For more details, see "Multicast Routing in the VXLAN Underlay" section, "IP Fabric Underlay" chapter in the Cisco Programmable Fabric with VXLAN BGP EVPN Configuration Guide.

  • Page 129: How To Configure Vxlan Bgp Evpn And Otv Interoperation, And Otv With Bdi

    40000 exit interface nve1 source-interface loopback0 tunnel-stitching enable member vni 40000 no suppress-arp mcast-group 239.1.1.65 • The tunnel-stitching enable command is the VXLAN command for connecting VXLAN and OTV tunnels. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 130 198.51.100.20/24 ip address 198.51.100.1/24 secondary anycast-primary ipv6 address 2001:DB8:1::1/64 no ipv6 redirects fabric forwarding mode anycast-gateway vPC Peer 2 (BL2) configuration BL2(config)# interface Bdi3500 no shutdown vrf member cust1 Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 131: Configure Otv With Bdi Configuration On The Border Switches In

    • The otv site-vni command enables the OTV site specific VNI. This VNI should not be extended over any overlay interface and should be operationally up before it can be configured as the OTV site VNI. Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 132: Verifying Vxlan Bgp Evpn And Otv Interoperation, And Otv With Bdi

    • In an OTV with BDI single box solution, the otv suppress-arp-nd option is not supported in the Cisco NX-OS 8.2(1) release. • For OTV overlays, only Generic Routing Encapsulation (GRE) encapsulation is supported for the 8.2(1) release.

  • Page 133: Display Otv Configuration On The Border Leaf Switch Bl1

    Adjacency Server(s) : 10.0.0.1 / [None] In the following example, site VLAN, site VNI, and other site details are displayed. You should remove the site VLAN using the no otv site-vlan command. BL1# show otv site detail Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 134: Display Otv Overlay State

    Ensure that you also enable ARP suppression under the VXLAN overlay at the same time. Alternatively, Note disable the ARP proxy/suppression function under both the overlays. BL1# show otv internal arp-nd status Overlay: Overlay1 Suppress arp-nd: Enabled VNI Suppress ARP: Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 135: Display Otv Adjacencies

    Tier ID: 0x1 Tier Peer ID: 0x1002 App: OTV Data: 0x2 Type: Tier Peer ID Tier ID: 0x1 Tier Peer ID: 0x13fd App: OTV Data: 0x0 BL1# show forwarding distribution tierpeerid nve Tier-Peer-id allocations: Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...

  • Page 136: Troubleshooting Vxlan Bgp Evpn And Otv Interoperation, And Otv With Bdi

    The following command was introduced for the VXLAN overlay: tunnel-stitching enable The following command was introduced for the BDI function: ip address secondary anycast-primary Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide...
  • Page 137 I N D E X creating overview VTEP and NVE interface VXLAN with vPC enabling verifying VXLANs VXLAN configuration VXLANs enabling Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide IN-1...
  • Page 138 Index Cisco Nexus 7000 Series NX-OS VXLAN Configuration Guide IN-2...

Table of Contents