VXLAN GPO Service Redirection: Firewall Insertion with a VLAN Pair Exception

Disclaimer: Lab notes, not a production runbook. Validate everything in your own environment before implementing anything in production.
Scope: Cisco Nexus 9000 only, on 9300-FX3/GX/GX2A/GX2B platforms, NX-OS 10.5(1)F and later.

North-south and inter-VLAN traffic in a VRF goes through a centralized firewall. Intra-VLAN stays on the fabric. VLAN10↔VLAN20 is the one inter-VLAN exception: direct, no firewall.

Use Case

VRF tenant, AS 65000, N VLANs. The CLI uses VLAN 10, 20, and <N>.

Requirement 1: Inter-VLAN and North-South via the Firewall

Traffic between the external 0.0.0.0/0 network and VLAN10 is redirected through the firewall attached to the service leaf North-south (0.0.0.0/0 ↔ VLAN10 as example) redirected through the firewall
  • North-south toward every VLAN, including VLAN10 and VLAN20.
  • East-west between distinct VLANs, except VLAN10↔VLAN20.

Two hosts in the same VLAN share one SGT, so that flow is intra-group and is not redirected.

Requirement 2: VLAN10 ↔ VLAN20 Direct

VLAN10 and VLAN20 exchange traffic directly, bypassing the firewall VLAN10↔VLAN20 only, direct, no firewall

Security Group Tag (SGT) Plan

One SGT per tenant VLAN, plus the firewall and the default route. Intra-VLAN stays on the fabric. Inter-VLAN and north-south go through the firewall, except VLAN10↔VLAN20.

SGT Name Classifies Configured on
10VLAN10-GROUPmatch vlan 10Every leaf with VLAN10 attached
20VLAN20-GROUPmatch vlan 20Every leaf with VLAN20 attached
<N>VLAN<N>-GROUPmatch vlan <N>Every leaf with that VLAN attached
900FIREWALL-INSIDEmatch interface vlan900 (type layer4-7)Service leaf only
500EXTERNAL-NORTHSOUTHmatch external-subnets ... 0.0.0.0/0 route-injectBorder leaf only

VLAN900 is the firewall inside interface (service leaf). VLAN500 is the transit link to the external router (border leaf).

How the Objects Relate

Three contracts: VLAN10↔VLAN20 permit, firewall return permit, everything else between different SGTs redirect.

Security Contracts (by SGT pair)
security contract 10 ↔ 20
security contract 900 ↔ any
security contract any ↔ any
from 10↔20 and 900↔any
policy-map PERMIT-DIRECT
class-map ANY-TRAFFIC
action permit
from any↔any
policy-map REDIRECT-TO-FW
class-map ANY-TRAFFIC
service-chain FW-CHAIN
epbr service FIREWALL-1

Packet Path for a Redirected Flow

Inter-VLAN or north-south packet path through the service leaf and the firewall:

Host A Leaf compute Spine Service Leaf Firewall Service Leaf Spine Leaf compute Host B
Inter-VLAN (or north-south) redirected flow through the firewall

Compute Leaf Configuration (Leaf 1, Leaf 2, Leaf N)

Same baseline on every leaf that instantiates VRF tenant (compute, service, border). Attach a tenant security-group only where that VLAN exists.

Feature Enablement

system routing template-security-groups carves TCAM for SGT/SGACL. Required before feature security-group. See the GPO chapter, Enable GPO.

Note: the routing template needs a reload, and may need copy running-config startup-config plus system flash sda resize extended.
system routing template-security-groups
feature security-group
feature sla sender
feature pbr
feature epbr

feature sla sender is required for the ICMP probe under epbr service FIREWALL-1.

VRF Enforcement

Default deny for traffic that matches no contract. Intra-VLAN needs none (same SGT). Inter-VLAN and north-south hit the 10↔20 permit or any ↔ any redirect.

vrf context tenant
  security enforce tag 999 default deny

Service-Chain

On every leaf role. The firewall IPs live only in epbr service on the service leaf; here you reference the name.

epbr service-chain FW-CHAIN
  load-balance method src-dst-ipprotocol
  10 set service FIREWALL-1 fail-action drop action redirect

src-dst-ipprotocol keeps both directions of a flow on the same firewall node if you add a second service-end-point later.

Class-Map / Policy-Map

class-map type security match-any ANY-TRAFFIC
  match ip

policy-map type security REDIRECT-TO-FW
  class type security ANY-TRAFFIC
    service-chain FW-CHAIN

policy-map type security PERMIT-DIRECT
  class type security ANY-TRAFFIC
    permit

Tenant VLAN Security-Groups

One group per tenant VLAN. Repeat for every extra VLAN <N>:

security-group 10 name VLAN10-GROUP
  match vlan 10

security-group 20 name VLAN20-GROUP
  match vlan 20

security-group <N> name VLAN<N>-GROUP
  match vlan <N>

Security Contracts

  1. 10 ↔ 20 → permit (the bypass)
  2. 900 ↔ any → permit (return from the firewall, no redirect loop)
  3. any ↔ any → redirect
security contract source 10 destination 20 policy PERMIT-DIRECT bidir
security contract source 900 destination any policy PERMIT-DIRECT bidir
security contract source any destination any policy REDIRECT-TO-FW bidir
One group per VLAN. A shared "other VLANs" tag would make VLAN<N>↔VLAN<M> intra-group and skip the firewall.
match ip does not match ARP, so the redirect does not break ARP to the firewall. See the ePBR L2 guide if you ever need to force that traffic through the chain.

Service Leaf Configuration (Leaf S)

The firewall attaches here. Return traffic on Vlan900 is SGT 900, which contract #2 permits out.

Firewall Attachment VLAN

vlan 900
  name SVC-FW-INSIDE

interface Vlan900
  description SVC-FW-INSIDE
  no shutdown
  vrf member tenant
  ip address 10.0.900.1/24
  fabric forwarding mode anycast-gateway

Firewall Security-Group and ePBR Service

10.0.900.10 is the firewall inside IP. loopback98 is the probe source in VRF tenant:

interface loopback98
  description EPBR-PROBE-SOURCE
  vrf member tenant
  ip address 10.0.98.1/32

security-group 900 name FIREWALL-INSIDE type layer4-7
  match interface vlan900

epbr service FIREWALL-1
  vrf tenant
  security-group 900
  probe icmp timeout 1 source-interface loopback98 frequency 2
  service-end-point ip 10.0.900.10

Border Leaf Configuration (BL)

eBGP to the external router and the default route in VRF tenant.

Transit VLAN and Default Route

vlan 500
  name TRANSIT-EXT

interface Vlan500
  description TRANSIT-EXT
  no shutdown
  vrf member tenant
  ip address 172.16.100.1/30
  fabric forwarding mode anycast-gateway

vrf context tenant
  ip route 0.0.0.0/0 172.16.100.2

router bgp 65000
  vrf tenant
    address-family ipv4 unicast
      default-information originate

External / North-South Security-Group

security-group 500 name EXTERNAL-NORTHSOUTH
  match external-subnets vrf tenant ipv4 0.0.0.0/0 route-inject

route-inject puts SGT 500 on the EVPN Type-5 default route, so every leaf learns it. North-south is then just another inter-group flow and hits any ↔ any.

Verification

show security-group
show contracts detail
show contracts source 10 destination 20
show contracts source 900 destination any
show contracts source any destination any
show vrf tenant security
show epbr service-chain FW-CHAIN
show epbr service FIREWALL-1
show hardware access-list tcam region
show policy-map type security

References & Sources

#networking #vxlan #security #datacenter