I am presently dealing with a community challenge between Cisco Nexus 93108 (9.3.11) on an LACP port-channel configuration.
The reproduction beneath assist us to breed the problem we had (utilizing 4x Nexus 9000v qcow2 picture model 9.3.9) as you possibly can see my community topology is between 2 datacenters.
- SWC001,SWC002,SWC011 and SWC012 are Nexus 9000v.
- Switch1 , Switch2 Switch3, OP and OP1 are fundamental GNS3 change
Configuration on every is analogous : 1 Area vPC between 2 switches (fairly fundamental 1 peerlink, 1 peerkeepalive, no layer 3, no peer gateway) and
a WAN hyperlink by means of a vPC Po 100 between each datacenter that permit ALL vlans to transit.
vPC configuration is constant and works effectively: SWC001 is instantly hyperlink to SWC011 and SWC002 to SWC012, all the things runs easily and acquired no points.
The factor is, in actuality, there’s an ISP between each Datacenters and the ache is coming…
We solely know from the ISP they use QinQ configuration in their very own community, as a datacenter consumer we do not know which configuration neither units they’re utilizing.
Additionally, earlier than utilizing Nexus, we had previous HP core change and we did not set any specific configuration relating to to QinQ)
After this HP=>Nexus migration, all the things was nice besides the PortChannel 100 standing (so the prolonged LAN between each DCs)
To simulate an ISP in between, i arrange a fundamental GNS3 change configured with QinQ on e0 and e1 (VLAN 1 and Ethertype 0x88A8)
In my instance Po100 is configured with just one bodily interface (eth1/13), and so the present configuration on all eth1/13 on the 4 change is :
model 9.3(9) Bios:model
interface Ethernet1/13
lacp charge quick
switchport mode trunk
spanning-tree port kind edge trunk
spanning-tree bpdufilter allow
channel-group 100 mode lively
interface port-channel100
switchport mode trunk
spanning-tree port kind edge trunk
spanning-tree bpdufilter allow
no lacp suspend-individual
If we’re on a channel-group lively/lively configuration (after enabled function lacp ), PortChannel protocol is LACP but it surely nonetheless goes on Switched and Down
SWC001# sh port-channel abstract interface port-channel 100
Flags: D - Down P - Up in port-channel (members)
I - Particular person H - Scorching-standby (LACP solely)
s - Suspended r - Module-removed
b - BFD Session Wait
S - Switched R - Routed
U - Up (port-channel)
p - Up in delay-lacp mode (member)
M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port- Kind Protocol Member Ports
Channel
--------------------------------------------------------------------------------
100 Po100(SD) Eth LACP Eth1/13(I)
If we set “channel group 100 mode on” on each facet, Port CHannel protocol is none BUT interface is UP and Po Switched/Up
FRHD01SWC001(config-if)# sh port-channel abstract interface port-channel 100
Flags: D - Down P - Up in port-channel (members)
I - Particular person H - Scorching-standby (LACP solely)
s - Suspended r - Module-removed
b - BFD Session Wait
S - Switched R - Routed
U - Up (port-channel)
p - Up in delay-lacp mode (member)
M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port- Kind Protocol Member Ports
Channel
--------------------------------------------------------------------------------
100 Po100(SU) Eth NONE Eth1/13(P)
Now, if we take a look about lacp counters, from each facet there’s LACPDUs despatched, however 0 acquired from every method.
SWC001# present lacp counters
NOTE: Clear lacp counters to get correct statistics
------------------------------------------------------------------------------
LACPDUs Markers/Resp LACPDUs
Port Despatched Recv Recv Despatched Pkts Err
------------------------------------------------------------------------------
port-channel1
Ethernet1/11 28 24 0 0 0
port-channel100
Ethernet1/13 12 0 0 0 0
Lastly, when i take a look at sh vpc temporary , Po100 standing is down whereas consistency is success
vPC Peer-link standing
---------------------------------------------------------------------
id Port Standing Lively vlans
-- ---- ------ -------------------------------------------------
1 Po1 up 1,5,10-11,13-14,22-23,30,41,44,50-51,55,80-81,90,
100,110-114,120-121,130-135,140-141,150-154,
160-174,176,180,201-205,230,256-257,999
vPC standing
----------------------------------------------------------------------------
Id Port Standing Consistency Purpose Lively vlans
-- ------------ ------ ----------- ------ ---------------
100 Po100 down* success success -
We made totally different configuration on GNS3 to check the place the problem might be, lively/lively, lively/passive or on/on , however we really have no thought of the problem. We additionally take a look on the spanning tree half but it surely appears good.
It appears that evidently if something is related between 2x Nexus, it fails, though hyperlink could also be up (however there is not any packet acquired)
Does anybody have already got a difficulty like this ? with/with out QinQ in between ?
EDIT 28-06-2023 – HP core change working configuration, A24 and B24 are LACP interfaces between each DCs.
