TLDR; routed port on vPC switch 1 needs to be accessible from vPC switch 2.
Our data center has a pair of Nexus 5596T with L3 card being used as the core. These are vPC pair, HSRP members, all vlan SVIs live here. All vlans are in a VRF. All routes are static at this location, since it's basically a quad 0 to one side and an RFC-1918 to another side. I have the vPC heartbeat on mgmt0 in its own VRF:
vpc domain 1 system-priority 5000 peer-keepalive destination 172.31.255.1 delay restore 150 peer-gatewayThere is an etherchannel group between the two for the peer link:
interface port-channel1 description N5K Peer Link switchport mode trunk spanning-tree port type network vpc peer-linkI had a need to add a routed port to one of the 5k's, which I placed in the VRF as so:
int eth1/6vrf member VRF-nameno switchportip addr 1.1.1.1/30no shutI am capable of pinging the remote device (1.1.1.2) from the VRF on the switch hosting the routed port (5k-1), however from 5k-2 I am not able to ping and 1.1.1.1 doesn't appear in the routing table due to the static routing config. I don't see a clear way to add the route to 5k-2 other than simply pointing it to a local IP on 5k-1 such as:
ip route 1.1.1.0/30 [IP on 5k-1]What am I missing here? Because we are vPC to L2 switches and servers, I might end up with traffic to 1.1.1.2 getting black-holed depending on which switch receives that traffic.
config:
!Command: show running-config!Time: Sat Nov 12 13:34:07 2016version 7.1(3)N1(2)hostname Sac_N5596-1no feature telnetcfs eth distributefeature pimfeature eigrpfeature pbrfeature udldfeature interface-vlanfeature hsrpfeature lacpfeature dhcpfeature vpcfeature lldpfeature vtpfeature fexlogging level feature-mgr 0ssh key rsa 2048no ip domain-lookupip access-list SDWAN 10 permit ip any 10.9.0.0/16ip access-list SNMP-ACL 1 permit ip 10.254.225.140/32 anyclass-map type qos match-all class-iscsi match cos 4class-map type queuing class-iscsi match qos-group 3policy-map type qos policy-qos class class-iscsi set qos-group 3policy-map type queuing policy-queuing class type queuing class-fcoe bandwidth percent 0 class type queuing class-iscsi bandwidth percent 95 class type queuing class-default bandwidth percent 5class-map type network-qos class-iscsi match qos-group 3policy-map type network-qos jumbo class type network-qos class-default mtu 9216 multicast-optimizepolicy-map type network-qos policy-nq class type network-qos class-iscsi mtu 9216 pause no-dropsystem qos service-policy type network-qos jumbovtp mode transparentvtp domain SACntp peer 10.254.1.254 use-vrf internalntp server 63.145.169.3 use-vrf internalntp server 69.36.224.15 use-vrf internalntp source-interface Vlan1vlan 1vlan 30 name UC-DEVICESvlan 100 name TBD_Removed_100vlan 105 name iDRAC_MGMTvlan 110 name User_10vlan 160 name SERVERvlan 170 name Databasevlan 171 name Applicationvlan 172 name Webvlan 200 name iSCSIvlan 205 name TBD_Removed_205vlan 225 name BACKUPSvlan 250 name DMZvlan 350 name LB-DMZvlan 360 name LB-SERVERvlan 370 name LB-DataBasevlan 371 name LB-Applicationvlan 372 name LB-Webvlan 800 name Network_Endpointsvlan 900 name MPLSvlan 998 name Dummy-VLANspanning-tree vlan 1-998 priority 24576route-map SDWAN permit 10 match ip address SDWAN set ip next-hop verify-availability 1.1.1.1service dhcpip dhcp relayvrf context internal ip route 0.0.0.0/0 10.254.1.1 ip route 10.0.0.0/12 10.99.99.1 ip route 10.5.0.0/16 10.254.1.1 ip route 10.16.0.0/16 10.254.1.1 ip route 10.17.0.0/16 10.254.1.1 ip route 10.249.0.0/16 10.99.99.1 ip route 10.254.254.0/24 10.254.1.1 ip route 172.16.60.0/24 10.254.1.20 ip route 192.4.1.0/24 10.99.99.1 ip route 192.168.0.0/16 10.99.99.1 ip route 192.168.5.0/24 10.254.1.1 ip route 192.168.13.0/24 10.254.1.1 ip route 192.168.195.0/24 10.99.99.1 ip route 192.168.250.0/24 10.99.99.1vrf context managementvpc domain 1 system-priority 5000 peer-keepalive destination 172.31.255.2 delay restore 150interface Vlan1 no shutdown vrf member internal no ip redirects ip address 10.254.1.253/24 hsrp version 2 hsrp 1 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.1.2interface Vlan30 description UC Devices VLAN no shutdown vrf member internal no ip redirects ip address 10.254.30.253/24 hsrp version 2 hsrp 30 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.30.1interface Vlan105 description iDRAC/MGMT no shutdown vrf member internal no ip redirects ip address 10.254.5.253/24 hsrp version 2 hsrp 105 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.5.1 ip dhcp relay address 10.254.60.50interface Vlan110 description User Vlan no shutdown vrf member internal no ip redirects ip address 10.254.10.253/24 hsrp version 2 hsrp 110 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.10.1 ip dhcp relay address 10.254.60.50interface Vlan160 description Server VLAN no shutdown vrf member internal no ip redirects ip address 10.254.60.253/24 hsrp version 2 hsrp 160 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.60.1interface Vlan170 description DataBase no shutdown vrf member internal no ip redirects ip address 10.254.170.253/24 hsrp version 2 hsrp 170 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.170.1interface Vlan171 description Application no shutdown vrf member internal no ip redirects ip address 10.254.171.253/24 hsrp version 2 hsrp 171 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.171.1interface Vlan172 description Web no shutdown vrf member internal no ip redirects ip address 10.254.172.253/24 hsrp version 2 hsrp 172 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.172.1interface Vlan200 description iSCSI Traffic no shutdown vrf member internal no ip redirects ip address 10.254.100.253/24 hsrp version 2 hsrp 200 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.100.1interface Vlan225 description Backup Solution vLAN no shutdown vrf member internal no ip redirects ip address 10.254.225.253/24 hsrp version 2 hsrp 225 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.225.1interface Vlan360 description Load Balancer Servers VLAN no shutdown vrf member internal no ip redirects ip address 172.16.60.253/24 hsrp version 2 hsrp 360 preempt delay minimum 240 priority 150 timers 1 3 ip 172.16.60.1interface Vlan370 description Load Balancer DataBase VLAN no shutdown vrf member internal no ip redirects ip address 172.16.170.253/24 hsrp version 2 hsrp 370 preempt delay minimum 240 priority 150 timers 1 3 ip 172.16.170.1interface Vlan371 description Load Balancer Application VLAN no shutdown vrf member internal no ip redirects ip address 172.16.171.253/24 hsrp version 2 hsrp 371 preempt delay minimum 240 priority 150 timers 1 3 ip 172.16.171.1interface Vlan372 description Load Balancer Web VLAN no shutdown vrf member internal no ip redirects ip address 172.16.172.253/24 hsrp version 2 hsrp 372 preempt delay minimum 240 priority 150 timers 1 3 ip 172.16.172.1interface Vlan800 description Network_Endpoints no shutdown vrf member internal no ip redirects ip address 10.254.0.253/24 hsrp version 2 hsrp 800 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.0.1interface Vlan900 description MPLS no shutdown vrf member internal no ip redirects ip address 10.99.99.253/24 hsrp version 2 hsrp 900 preempt delay minimum 240 priority 150 timers 1 3 ip 10.99.99.2interface port-channel1 description Nexus to Nexus switchport mode trunk spanning-tree port type network vpc peer-linkinterface port-channel5 description Trunk to DevStation_3750X switchport mode trunk speed 10000interface port-channel20 description To 3850 Stack switchport mode trunk vpc 20interface Ethernet1/1 description Firewall LAN Handoff - Primaryinterface Ethernet1/2 description Uplink to MPLS router - Primary switchport access vlan 900interface Ethernet1/3 description Firewall DMZ switchport access vlan 250interface Ethernet1/4 description Uplink to SilverPeak switchport access vlan 800 spanning-tree port type edgeinterface Ethernet1/5 description SilverPeak Web Mgmt switchport access vlan 160 spanning-tree port type edgeinterface Ethernet1/6 description SilverPeak LAN0 no switchport vrf member internal ip address 1.1.1.1/30interface Ethernet1/29 description N5K Peer Link switchport mode trunk channel-group 1 mode activeinterface Ethernet1/30 description N5K Peer Link switchport mode trunk channel-group 1 mode activeinterface Ethernet1/31 description N5K Peer Link switchport mode trunk channel-group 1 mode activeinterface Ethernet1/32 description N5K Peer Link switchport mode trunk channel-group 1 mode activeinterface mgmt0 vrf member management ip address 172.31.255.1/30clock timezone PST -8 0clock summer-time PDT 2 Sun Mar 02:00 1 Sun Nov 02:00 60line consoleline vtyboot kickstart bootflash:/n5000-uk9-kickstart.7.1.3.N1.2.binboot system bootflash:/n5000-uk9.7.1.3.N1.2.binNexus 2:
!Command: show running-config!Time: Sat Nov 12 13:34:32 2016version 7.1(3)N1(2)hostname Sac_N5596-2no feature telnetcfs eth distributefeature pimfeature eigrpfeature pbrfeature udldfeature interface-vlanfeature hsrpfeature lacpfeature dhcpfeature vpcfeature lldpfeature vtpfeature fexno ip domain-lookupip access-list SDWAN 10 permit ip any 10.9.0.0/16ip access-list SNMP-ACL 1 permit ip 10.254.225.140/32 anyclass-map type qos match-all class-iscsi match cos 4class-map type queuing class-iscsi match qos-group 3policy-map type qos policy-qos class class-iscsi set qos-group 3policy-map type queuing policy-queuing class type queuing class-fcoe bandwidth percent 0 class type queuing class-iscsi bandwidth percent 95 class type queuing class-default bandwidth percent 5class-map type network-qos class-iscsi match qos-group 3policy-map type network-qos jumbo class type network-qos class-default mtu 9216 multicast-optimizepolicy-map type network-qos policy-nq class type network-qos class-iscsi mtu 9216 pause no-dropsystem qos service-policy type network-qos jumbovtp mode transparentvtp domain SACntp peer 10.254.2.253ntp server 63.145.169.3ntp server 69.36.224.15ntp source-interface Vlan1vlan 1vlan 30 name UC-DEVICESvlan 100 name TBD_Removed_100vlan 105 name iDRAC_MGMTvlan 110 name User_10vlan 160 name SERVERvlan 170 name DATABASEvlan 171 name Applicationvlan 172 name Webvlan 200 name iSCSIvlan 205 name TBD_Removed_205vlan 225 name BACKUPSvlan 250 name DMZvlan 350 name LB-DMZvlan 360 name LB-SERVERvlan 370 name LB-DataBasevlan 371 name LB-Applicationvlan 372 name LB-Webvlan 800 name Network_Endpointsvlan 900 name MPLSvlan 998 name Dummy-VLANspanning-tree vlan 1-998 priority 28672route-map SDWAN permit 10 match ip address SDWANservice dhcpip dhcp relayvrf context internal ip route 0.0.0.0/0 10.254.1.1 ip route 10.0.0.0/12 10.99.99.1 ip route 10.5.0.0/16 10.254.1.1 ip route 10.16.0.0/16 10.254.1.1 ip route 10.17.0.0/16 10.254.1.1 ip route 10.249.0.0/16 10.99.99.1 ip route 10.254.254.0/24 10.254.1.1 ip route 172.16.60.0/24 10.254.1.20 ip route 192.168.0.0/16 10.99.99.1 ip route 192.168.5.0/24 10.254.1.1 ip route 192.168.13.0/24 10.254.1.1 ip route 192.168.195.0/24 10.99.99.1 ip route 192.168.250.0/24 10.99.99.1 ip route 192.254.1.0/24 10.99.99.1vrf context managementvpc domain 1 system-priority 5000 peer-keepalive destination 172.31.255.1 delay restore 150 peer-gatewayinterface Vlan1 no shutdown vrf member internal no ip redirects ip address 10.254.1.254/24 hsrp version 2 hsrp 1 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.1.2interface Vlan30 description UC Devices VLAN no shutdown vrf member internal no ip redirects ip address 10.254.30.254/24 hsrp version 2 hsrp 30 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.30.1interface Vlan105 description iDRAC/MGMT no shutdown vrf member internal no ip redirects ip address 10.254.5.254/24 hsrp version 2 hsrp 105 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.5.1 ip dhcp relay address 10.254.60.50interface Vlan110 description User Vlan no shutdown vrf member internal no ip redirects ip address 10.254.10.254/24 hsrp version 2 hsrp 110 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.10.1 ip dhcp relay address 10.254.60.50interface Vlan160 description Server VLAN no shutdown vrf member internal no ip redirects ip address 10.254.60.254/24 hsrp version 2 hsrp 160 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.60.1interface Vlan170 description DataBase no shutdown vrf member internal no ip redirects ip address 10.254.170.254/24 hsrp version 2 hsrp 170 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.170.1interface Vlan171 description Application no shutdown vrf member internal no ip redirects ip address 10.254.171.254/24 hsrp version 2 hsrp 171 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.171.1interface Vlan172 description Web no shutdown vrf member internal no ip redirects ip address 10.254.172.254/24 hsrp version 2 hsrp 172 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.172.1interface Vlan200 description iSCSI Traffic no shutdown vrf member internal no ip redirects ip address 10.254.100.254/24 hsrp version 2 hsrp 200 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.100.1interface Vlan225 description Backup Solution vLAN no shutdown vrf member internal no ip redirects ip address 10.254.225.254/24 hsrp version 2 hsrp 225 preempt delay minimum 240 priority 130 timers 1 3 ip 10.254.225.1interface Vlan360 description Load Balancer Servers VLAN no shutdown vrf member internal no ip redirects ip address 172.16.60.254/24 hsrp version 2 hsrp 360 preempt delay minimum 240 priority 130 timers 1 3 ip 172.16.60.1interface Vlan370 description Load Balancer DataBase VLAN no shutdown vrf member internal no ip redirects ip address 172.16.170.254/24 hsrp version 2 hsrp 370 preempt delay minimum 240 priority 130 timers 1 3 ip 172.16.170.1interface Vlan371 description Load Balancer Application VLAN no shutdown vrf member internal no ip redirects ip address 172.16.171.254/24 hsrp version 2 hsrp 371 preempt delay minimum 240 priority 130 timers 1 3 ip 172.16.171.1interface Vlan372 description Load Balancer Web VLAN no shutdown vrf member internal no ip redirects ip address 172.16.172.254/24 hsrp version 2 hsrp 372 preempt delay minimum 240 priority 130 timers 1 3 ip 172.16.172.1interface Vlan800 description Network_Endpoints no shutdown vrf member internal no ip redirects ip address 10.254.0.254/24 hsrp version 2 hsrp 800 preempt delay minimum 240 priority 150 timers 1 3 ip 10.254.0.1interface Vlan900 description MPLS no shutdown vrf member internal no ip redirects ip address 10.99.99.254/24 hsrp version 2 hsrp 900 preempt delay minimum 240 priority 130 timers 1 3 ip 10.99.99.2interface port-channel1 description N5K Peer Link switchport mode trunk spanning-tree port type network vpc peer-linkinterface Ethernet1/29 description N5K Peer Link switchport mode trunk channel-group 1 mode activeinterface Ethernet1/30 description N5K Peer Link switchport mode trunk channel-group 1 mode activeinterface Ethernet1/31 description N5K Peer Link switchport mode trunk channel-group 1 mode activeinterface Ethernet1/32 description N5K Peer Link switchport mode trunk channel-group 1 mode activeinterface mgmt0 vrf member management ip address 172.31.255.2/30clock timezone PST -8 0clock summer-time PDT 2 Sun Mar 02:00 1 Sun Nov 02:00 60line consoleline vtyboot kickstart bootflash:/n5000-uk9-kickstart.7.1.3.N1.2.binboot system bootflash:/n5000-uk9.7.1.3.N1.2.bin
