Giter Club home page Giter Club logo

nsxt-evpn-eveng-lab's Introduction

Introduction

VMWare NSX-T 3.0 introduced support for EVPN Type-5 integration which allows efficient multi-tenant L3 exchange between VMWare NSX-T Edge and external gateways. The following graph should visualize life before and after the EVPN Type5 support:

Instead of having 802.1q trunk interface with L3 sub-interface per VRF, we now can have a single routed interface with just a single BGP EVPN session. This greatly reduces configuration overhead on both sides. This article will describe all necessary steps required to test this feature against Arista EOS devices.

Test Setup

Let us assume we already have running EVPN/VXLAN fabric with Symmetric IRB (SIRB) as many customers already deployed EVPN/VXLAN in their data centers and just want to use existing infrastructure to deploy NSX-T. We are going to use one VLAN with SIRB 10.10.100.1/24 to get NSX-T Geneve overlay running on top of it. Strictly speaking we could use IP Fabric underlay to distribute NSX-T TEPs reachability to form Geneve overlay. Once we have Geneve traffic successfully forwarded through EVPN fabric, we will add NSX-T Edge Transport Node. NSX-T Edge is going to implement:

  1. L3 multi-tenancy segmentation. Geneve Segment1 will be configured to be part of VRFA.
  2. Route exchange between VRFA and external Arista Gateway using EVPN Type-5
  3. Stitching two different data planes: Geneve and VXLAN

Everything in that setup except physical EVPN GW is running within EVEng. NOTE: It is also possible to run EVPN GW as vEOS in EVEng. At the end of this exercise we should be able to have connectivity between end systems in Geneve Overlay (VM1 and VM2) and workload connected to external Arista GW (VM3). Multi-tenancy must be honoured: both prefixes 192.168.1.0/24 from overlay and 192.168.200.0/24 must be confined only within VRFA.

Deploy vEOS Arista EVPN/VXLAN fabric and ESXi hosts

Following Github repository has EVEng lab and vEOS configuration: Github NSX-T 3.0 EVPN Type 5 Lab.

pcs-esxi1 and pcs-esxi2 are going to be our Host Transport Nodes. VM1 will be running on pcs-esxi1 and VM2 pn pcs-esxi2. NSX-T Manager VM and NSX-T Edge VM will be running on pcs-esxi3. Deployment of ESXi hosts is outside the scope of this article.

Following screenshot shows allocated resources (RAM, vCPUs) for each node in the topology.

Deploy NSX-T Manager on ESXi3

  1. In the vSphere Client of esxi3, select Create / Register VMand follow the steps in VMware NSX-T Manager Installation
  2. Login to the NSX-T Manager web console: https://:443 and proceed to " System -> Fabric -> Transport Zones". Create new Overlay Transport Zone
  3. Under " System -> Fabric -> Nodes -> Host " Transport Nodes click Add button:
  4. Click NEXT and configure Host Transport Node:

    In our setup both Transport Nodes have a single interface towards the EVPN fabric . That is why we set Uplink Profile to nsx-edge-single-nic-uplink-profile. TEP IP is set to 10.10.100.100 for this node. This is going to be used as outer Src IP Header. Finally uplink is mapped to physical NIC on ESXi. Repeat steps 4-5 for all your Host Transport Nodes.
  5. Go to " Networking -> Segments" and add new Segment attached to the Transport Zone TZ1:
  6. Go to ESXi1 and ESXi2 and start one VM on each of them. Connect vNIC of VM to S2 segment:
  7. Configure IP on VM1 to 192.168.1.10/24 and VM2 to 192.168.1.20/24. Ping VM2 from VM1. At this stage you should be done with the Overlay config.

Deploy NSX-T Edge VM

  1. In the vSphere Client of esxi3, select Create / Register VMand follow the steps in VMWare NSX-T Edge Installation
  2. NSX-T Edge OVA comes with 4 interfaces. We are going to use first 3 interfaces:
  3. As we deployed Edge VM from OVA you will need to go through following steps to add it to the NSX-T Manager:How to join NSX-T Edge with management plane
  4. Once added successfully you should see your Edge VM under " System -> Fabric -> Nodes -> Edge Transport Nodes"
  5. Under " System -> Fabric -> Nodes -> Edge Clusters" click Add button and add your single Edge node to cluster:
  6. Configure the first Switch (connected to Overlay Transport Zone TZ1) on your Edge Node :
  7. Create new Uplink Profile BGP-PEERING under " System -> Fabric -> Profiles":
  8. Configure the second Switch (connected to default VLAN Transport Zone nsx-vlan-transportzone) on your Edge Node :
  9. Go to " Networking -> Segments" and add new Segment attached to Transport Zone nsx-vxlan-transportzone:
  10. Create new VNI pool under " Networking -> Setting -> VNI Pool":
  11. Create Tier-0 Gateway under " Networking -> Connectivity -> Tier-0″ (Pulldown Add Gateway):
  12. Create VRFA under " Networking -> Connectivity -> Tier-0″ (Pulldown Add VRF):
  13. Configure VRFA:
  14. Configure Route Targets 65000:10 (both import and export) for this VRFA:
  15. Attach interface on the Tier-0 GW towards external Arista GW Gateway under " Networking -> Connectivity -> Tier-0″. 192.168.150.1/24 is local peering IP on Edge, external GW will have 192.168.150.2/24.
  16. Configure interface on external EOS GW:
interface Ethernet1
   description "to NSX-T Edge"
   mtu 9000
   speed forced 10000full
   no switchport
   ip address 192.168.150.2/24

At this point of time you should be able to ping 192.168.150.1 from EOS external GW. 17. Attach Tier-0 VRFA GW to Overlay Segment S2 as Default GW:
18. Configure default route on both VM1 and VM2 to have next-hop of 192.168.1.1

Configure BGP peering

  1. Configure BGP under " Networking -> Connectivity -> Tier-0 Gateways" (NOT VRFA!):
  2. Configure BGP peer:
  3. Add both address families for BGP peering by clicking on blue number under Route Filter (see above):

Configure BGP Route Re-Distribution

  1. In both VRFA and Tier-0 GW configure following:

Configure BGP peering on EOS external Gateway

Relevant EOS configuration (Arista 7280SR2 was used in this setup):

interface Loopback0
   description BGP router-Id
   ip address 192.168.55.1/32
!
interface Loopback300
   vrf VRFA
   ip address 10.10.10.1/32
!
interface Vlan300
   vrf VRFA
   ip address 192.168.200.1/24
!
interface Vxlan1
   vxlan source-interface Loopback0
   vxlan udp-port 4789
   vxlan vrf VRFA vni 200000
!
hardware tcam
   system profile vxlan-routing
!
ip routing
ip routing vrf VRFA
!
ip prefix-list loopback
   seq 10 permit 192.168.55.1/32
!
route-map loopback permit 10
   match ip address prefix-list loopback
!
router bgp 65001
   router-id 192.168.55.1
   neighbor NSXTEDGE peer group
   neighbor NSXTEDGE bfd
   neighbor NSXTEDGE remote-as 65000
   neighbor NSXTEDGE send-community
   neighbor NSXTEDGE maximum-routes 0
   neighbor 192.168.150.1 peer group NSXTEDGE
   redistribute connected route-map loopback
   !
   address-family evpn
      neighbor NSXTEDGE activate
   !
   vrf VRFA
      rd 192.168.55.1:50001
      route-target import evpn 65000:10
      route-target export evpn 65000:10
      redistribute connected

Control Plane Verification

  1. Verify BGP session status in the NSX-T Manager:
  2. Download the Forwarding Table:
  3. Show downloaded Forwarding Table:
  4. Show BGP status on EOS GW:
show bfd peers
VRF name: default
-----------------
DstAddr        MyDisc    YourDisc  Interface     Type    LastUp LastDown  LastDiag    State
---------------- ------------- ------------ ---------------- --------- --------------------
192.168.150.1 3990680118 127459675 Ethernet1(81) normal 05/25/20 13:18 NA No Diagnostic Up

show  bgp  evpn  summary
BGP summary information for VRF default
Router identifier 192.168.55.1, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  192.168.150.1    4  65000           3312      3897    0    0 10:03:49 Estab   1      1

show  bgp  neighbors 192.168.150.1 evpn  received-routes detail
BGP routing table information for VRF default
Router identifier 192.168.55.1, local AS number 65001
BGP routing table entry for ip-prefix 192.168.1.0/24, Route Distinguisher: 10.10.100.200:10
 Paths: 1 available
  65000
    192.168.150.1 from 192.168.150.1 (192.168.150.1)
      Origin INCOMPLETE, metric 0, localpref -, weight 0, valid, external, best
      Extended Community: Route-Target-AS:65000:10 TunnelEncap:tunnelTypeVxlan EvpnRouterMac:02:50:56:00:08:00
      VNI: 200000

show  bgp  neighbors 192.168.150.1 evpn  advertised-routes detail
BGP routing table information for VRF default
Router identifier 192.168.55.1, local AS number 65001
Update wait-install is disabled
BGP routing table entry for ip-prefix 10.10.10.1/32, Route Distinguisher: 192.168.55.1:50001
 Paths: 1 available
  65001
    192.168.55.1 from - (0.0.0.0)
      Origin IGP, metric -, localpref 100, weight 0, valid, local, best, redistributed (Connected)
      Extended Community: Route-Target-AS:65000:10 TunnelEncap:tunnelTypeVxlan EvpnRouterMac:98:5d:82:97:61:bf
      VNI: 200000
BGP routing table entry for ip-prefix 192.168.200.0/24, Route Distinguisher: 192.168.55.1:50001
 Paths: 1 available
  65001
    192.168.55.1 from - (0.0.0.0)
      Origin IGP, metric -, localpref 100, weight 0, valid, local, best, redistributed (Connected)
      Extended Community: Route-Target-AS:65000:10 TunnelEncap:tunnelTypeVxlan EvpnRouterMac:98:5d:82:97:61:bf
      VNI: 200000
  1. Show BGP status on EDGE CLI:
pcs-nsxedge(tier0_sr)> get bgp neighbor summary
BFD States: NC - Not configured, AC - Activating,DC - Disconnected
            AD - Admin down, DW - Down, IN - Init,UP - Up
BGP summary information for VRF default for address-family: ipv4Unicast
Router ID: 192.168.150.1  Local AS: 65000

Neighbor                            AS          State Up/DownTime  BFD InMsgs  OutMsgs InPfx  OutPfx

192.168.150.2                       65001       Estab 1d01h27m     UP  1798    1534    2      3

BFD States: NC - Not configured, AC - Activating,DC - Disconnected
            AD - Admin down, DW - Down, IN - Init,UP - Up
BGP summary information for VRF default for address-family: l2VpnEvpn
Router ID: 192.168.150.1  Local AS: 65000

Neighbor                            AS          State Up/DownTime  BFD InMsgs  OutMsgs InPfx  OutPfx

192.168.150.2                       65001       Estab 1d01h27m     UP  1798    1534    2      3

pcs-nsxedge(tier0_sr)> get bgp evpn
BGP table version is 1, local router ID is 192.168.150.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 10.10.100.200:10
*> [5]:[0]:[24]:[192.168.1.0]
                    192.168.150.1            0         32768 ?
Route Distinguisher: 192.168.55.1:50001
*> [5]:[0]:[24]:[192.168.200.0]
                    192.168.55.1                           0 65001 i
*> [5]:[0]:[32]:[10.10.10.1]
                    192.168.55.1                           0 65001 i

pcs-nsxedge(tier0_sr)> get bfd-sessions
BFD Session
Dest_port                     : 3784
Diag                          : No Diagnostic
Encap                         : vlan
Forwarding                    : last true (current true)
Interface                     : de8f9cfd-1750-409e-af7b-45daf6501b26
Keep-down                     : false
Last_cp_diag                  : No Diagnostic
Last_cp_rmt_diag              : No Diagnostic
Last_cp_rmt_state             : up
Last_cp_state                 : up
Last_fwd_state                : UP
Last_local_down_diag          : No Diagnostic
Last_remote_down_diag         : No Diagnostic
Last_up_time                  : 2020-05-25 13:18:48
Local_address                 : 192.168.150.1
Local_discr                   : 127459675
Min_rx_ttl                    : 255
Multiplier                    : 3
Received_remote_diag          : No Diagnostic
Received_remote_state         : up
Remote_address                : 192.168.150.2
Remote_admin_down             : false
Remote_diag                   : No Diagnostic
Remote_discr                  : 3990680118
Remote_min_rx_interval        : 300
Remote_min_tx_interval        : 300
Remote_multiplier             : 3
Remote_state                  : up
Router                        : 6c04ae47-9d26-4b8b-9c78-16da971dbc80
Router_down                   : false
Rx_cfg_min                    : 500
Rx_interval                   : 500
Service-link                  : false
Session_type                  : LR_PORT
State                         : up
Tx_cfg_min                    : 500
Tx_interval                   : 500

Data Plane Verification

Ping VM1 from VM3:

VM3#ping 192.168.1.10 source 192.168.200.10
PING 192.168.1.10 (192.168.1.10) from 192.168.200.10 : 72(100) bytes of data.
80 bytes from 192.168.1.10: icmp_seq=1 ttl=62 time=119 ms
80 bytes from 192.168.1.10: icmp_seq=2 ttl=62 time=111 ms
80 bytes from 192.168.1.10: icmp_seq=3 ttl=62 time=105 ms
80 bytes from 192.168.1.10: icmp_seq=4 ttl=62 time=98.0 ms
80 bytes from 192.168.1.10: icmp_seq=5 ttl=62 time=91.2 ms

VM3#traceroute 192.168.1.10 source 192.168.200.10
traceroute to 192.168.1.10 (192.168.1.10), 30 hops max, 60 byte packets
 1  192.168.200.1 (192.168.200.1)  0.224 ms  0.212 ms  0.195 ms
 2  192.168.1.10 (192.168.1.10)  73.891 ms !X  76.826 ms !X  79.203 ms !X

Useful Links

EVPN Webinars: https://www.arista.com/en/company/news/webinars

VMWare NSX-T Distributed Firewall Integration with Arista Policy Control Service: https://events.arista.com/virtual-cloud-builders-2020-data-centre-continuous-innovation

nsxt-evpn-eveng-lab's People

Contributors

mpergament avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.