After configuring a Virtual PortChannel (vPC) between two parent Cisco Nexus 5000 switches in my previous post, the next step is to configure a Nexus 2000 Fabric Extender (FEX) modules. These are remote line cards which uplinks to a parent Cisco Nexus 5K, 6K, 7K or 9K switches.
N5K-1# configure terminal
N5K-1(config)# feature fex // ENABLE FEX
N5K-1(config)#
N5K-1(config)# interface Eth1/7
N5K-1(config-if)# description FEX101 - RACK 1
N5K-1(config-if)# switchport mode fex-fabric
N5K-1(config-if)# fex associate 101
N5K-1(config-if)# channel-group 101
N5K-1(config-if)# interface port-channel101
N5K-1(config-if)# description FEX101 - RACK 1
N5K-1(config-if)# switchport mode fex-fabric
N5K-1(config-if)# fex associate 101
N5K-1(config-if)# vpc 101
N5K-1(config-if)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
101 -------- Offline N2K-C2248TP-E-1GE FOX2520PABC // STATE WILL CHANGE FROM DISCOVERED TO OFFLINE
--- -------- Discovered N2K-C2248TP-E-1GE FOX2520PDEF
N5K-1(config-if)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
101 SERVER_RACK_1 Image Download N2K-C2248TP-E-1GE FOX2520PABC // FEX 2K WILL DOWNLOAD IMAGE FROM PARENT N5K SWITCH
--- -------- Discovered N2K-C2248TP-E-1GE FOX2520PDEF
N5K-1(config-if)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
101 -------- Offline N2K-C2248TP-E-1GE FOX2520PABC
--- -------- Discovered N2K-C2248TP-E-1GE FOX2520PDEF
N5K-1(config-if)#
N5K-1(config-if)# 2005 Apr 14 19:42:10 N5K-1 %$ VDC-1 %$ %SATCTRL-FEX101-2-SATCTRL: FEX-101 Module 1: Cold boot
N5K-1(config-if)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
101 -------- Online Sequence N2K-C2248TP-E-1GE FOX2520PABC
--- -------- Discovered N2K-C2248TP-E-1GE FOX2520PDEF
N5K-1(config-if)# 2005 Apr 14 19:42:25 N5K-1 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 101 is online
2005 Apr 14 19:42:25 N5K-1 %$ VDC-1 %$ %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-101 On-line
2005 Apr 14 19:42:26 N5K-1 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 101 is online
N5K-1config-if)# show fex // IT TOOK AROUND 10 MINS FOR FEX 2K TO COMPLETE ITS UPGRADE
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
101 FEX0101 Online N2K-C2248TP-E-1GE FOX2520PABC
--- -------- Discovered N2K-C2248TP-E-1GE FOX2520PDEF
N5K-1(config-if)# 2005 Apr 14 19:42:44 N5K-1 %$ VDC-1 %$ %SATCTRL-FEX101-2-SOHMS_DIAG_ERROR: FEX-101 System minor alarm on power supply 2: failed
2005 Apr 14 19:42:47 N5K-1 %$ VDC-1 %$ %SATCTRL-FEX101-2-SOHMS_DIAG_ERROR: FEX-101 Module 1: Runtime diag detected major event: Voltage failure on power supply: 2
2005 Apr 14 19:42:47 N5K-1 %$ VDC-1 %$ %SATCTRL-FEX101-2-SOHMS_DIAG_ERROR: FEX-101 System minor alarm on power supply 2: failed
N5K-1(config-if)# show fex detail
FEX: 101 Description: FEX0101 state: Online
FEX version: 7.3(8)N1(1) [Switch version: 7.3(8)N1(1)] // DOWNLOADED NX-OS FROM PARENT N5K SWITCH
FEX Interim version: 7.3(8)N1(1)
Switch Interim version: 7.3(8)N1(1)
Extender Serial: FOX2520ABC
Extender Model: N2K-C2248TP-E-1GE, Part No: 73-12345-67
Card Id: 149, Mac Addr: 28:af:fd:19:12:34, Num Macs: 64
Module Sw Gen: 12594 [Switch Sw Gen: 21]
Post level: complete
Pinning-mode: static Max-links: 1
Fabric port for control traffic: Eth1/7
FCoE Admin: false
FCoE Oper: true
FCoE FEX AA Configured: false
Fabric interface state:
Po101 - Interface Up. State: Active
Eth1/7 - Interface Up. State: Active
Fex Port State Fabric Port
Eth101/1/1 Down Po101
Eth101/1/2 Down Po101
Eth101/1/3 Down Po101
Eth101/1/4 Down Po101
Eth101/1/5 Down Po101
Eth101/1/6 Down Po101
Eth101/1/7 Down Po101
Eth101/1/8 Down Po101
Eth101/1/9 Down Po101
Eth101/1/10 Down Po101
Eth101/1/11 Down Po101
Eth101/1/12 Down Po101
Eth101/1/13 Down Po101
Eth101/1/14 Down Po101
Eth101/1/15 Down Po101
Eth101/1/16 Down Po101
Eth101/1/17 Down Po101
Eth101/1/18 Down Po101
Eth101/1/19 Down Po101
Eth101/1/20 Down Po101
Eth101/1/21 Down Po101
Eth101/1/22 Down Po101
Eth101/1/23 Down Po101
Eth101/1/24 Down Po101
Eth101/1/25 Down Po101
Eth101/1/26 Down Po101
Eth101/1/27 Down Po101
Eth101/1/28 Down Po101
Eth101/1/29 Down Po101
Eth101/1/30 Down Po101
Eth101/1/31 Down Po101
Eth101/1/32 Down Po101
Eth101/1/33 Down Po101
Eth101/1/34 Down Po101
Eth101/1/35 Down Po101
Eth101/1/36 Down Po101
Eth101/1/37 Down Po101
Eth101/1/38 Down Po101
Eth101/1/39 Down Po101
Eth101/1/40 Down Po101
Eth101/1/41 Down Po101
Eth101/1/42 Down Po101
Eth101/1/43 Down Po101
Eth101/1/44 Down Po101
Eth101/1/45 Down Po101
Eth101/1/46 Down Po101
Eth101/1/47 Down Po101
Eth101/1/48 Down Po101
Logs:
04/14/2005 19:32:34.118822: Module register received
04/14/2005 19:32:34.120820: Image Version Mismatch
04/14/2005 19:32:34.122113: Registration response sent
04/14/2005 19:32:34.122404: Requesting satellite to download image
04/14/2005 19:32:34.968158: Deleting route to FEX
04/14/2005 19:32:34.975847: Module disconnected
04/14/2005 19:32:34.977323: Module Offline
04/14/2005 19:32:48.807084: Deleting route to FEX
04/14/2005 19:32:48.814678: Module disconnected
04/14/2005 19:32:48.816369: Deleting route to FEX
04/14/2005 19:32:48.823789: Module disconnected
04/14/2005 19:32:48.826038: Offlining Module
04/14/2005 19:32:48.826685: Offlining Module
04/14/2005 19:33:34.065205: Deleting route to FEX
04/14/2005 19:33:34.072753: Module disconnected
04/14/2005 19:33:34.073898: Offlining Module
04/14/2005 19:33:34.075512: Deleting route to FEX
04/14/2005 19:33:34.082780: Module disconnected
04/14/2005 19:33:34.085162: Offlining Module
04/14/2005 19:33:34.108909: Deleting route to FEX
04/14/2005 19:33:34.116541: Module disconnected
04/14/2005 19:33:34.118758: Offlining Module
04/14/2005 19:34:06.942831: Deleting route to FEX
04/14/2005 19:34:06.950497: Module disconnected
04/14/2005 19:34:06.952049: Offlining Module
04/14/2005 19:34:06.953282: Deleting route to FEX
04/14/2005 19:34:06.960850: Module disconnected
04/14/2005 19:34:06.963198: Offlining Module
04/14/2005 19:34:06.988662: Deleting route to FEX
04/14/2005 19:34:06.996309: Module disconnected
04/14/2005 19:34:06.998510: Offlining Module
04/14/2005 19:35:30.639182: Module register received
04/14/2005 19:35:30.641271: Image Version Mismatch
04/14/2005 19:35:30.642573: Registration response sent
04/14/2005 19:35:30.642863: Requesting satellite to download image
04/14/2005 19:40:46.980527: Image preload successful.
04/14/2005 19:40:48.410573: Deleting route to FEX
04/14/2005 19:40:48.418185: Module disconnected
04/14/2005 19:40:48.419680: Module Offline
04/14/2005 19:40:48.421660: Deleting route to FEX
04/14/2005 19:40:48.428859: Module disconnected
04/14/2005 19:40:48.431149: Offlining Module
04/14/2005 19:40:48.452809: Deleting route to FEX
04/14/2005 19:40:48.460347: Module disconnected
04/14/2005 19:40:48.462656: Offlining Module
04/14/2005 19:42:18.759564: Module register received
04/14/2005 19:42:18.763150: Registration response sent
04/14/2005 19:42:18.809369: create module inserted event.
04/14/2005 19:42:18.810380: Module Online Sequence
04/14/2005 19:42:25.044678: Module Online
N5K-1(config)# fex 101
N5K-1(config-fex)# ?
description FEX description
diagnostic Diagnostic commands
hardware FEX Card type
no Negate the command
pinning Pinning configurations
port Configure a port
serial Chassis serial number
show Show running config
type FEX Card type
end Go to exec mode
exit Exit from command interpreter
pop Pop mode from stack or restore from name
push Push current mode to stack or save it under name
where Shows the cli context you are in
N5K-1(config-fex)# description RACK_1
N5K-1(config-fex)# pinning max-links 1
Change in Max-links will cause traffic disruption.
N5K-1(config-fex)# serial ?
WORD Serial number (Max Size 20)
N5K-1(config-fex)# serial FOX2520PABC // BEST PRACTICE TO CONFIGURE serial AND type FOR EASY TROUBLESHOOTING; FROM show fex detail
Changing serial will offline fex.
N5K-1config-fex)# type ?
N2148T Fabric Extender 48x1G 4x10G Module
N2224TP Fabric Extender 24x1G 2x10G SFP+ Module
N2232P Fabric Extender 32x10G 8x10G Module
N2232TM Fabric Extender 32x10GBase-T 8x10G SFP+ Module
N2232TM-E Fabric Extender 32x10GBase-T 8x10G SFP+ Module
N2232TP Fabric Extender 32x10GBase-T 8x10G SFP+ Module
N2232TT Fabric Extender 32x10GBase-T 8x10GBase-T Module
N2248PQ Fabric Extender 48x10G SFP+ 4x40G QSFP Module
N2248T Fabric Extender 48x1G 4x10G Module
N2248TP-E Fabric Extender 48x1G 4x10G Module
N2332TQ Fabric Extender 32x10GBase-T 4x40G QSFP Module
N2348TQ Fabric Extender 48x10GBase-T 6x40G QSFP Module
N2348TQ-E Fabric Extender 48x10GBase-T 6x40G QSFP Module
N2348UPQ Fabric Extender 48x10G SFP+ 6x40G QSFP Module
NB22DELL Fabric Extender 16x10G SFP+ 8x10G SFP+ Module
NB22FJ Fabric Extender 16x10G SFP+ 8x10G SFP+ Module
NB22HP Fabric Extender 16x10G SFP+ 8x10G SFP+ Module
NB22IBM Fabric Extender 14x10G SFP+ 8x10G SFP+ Module
N5K-1(config-fex)# type N2248TP-E
N5K-1(config-fex)# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
101 SERVER_RACK_1 Online N2K-C2248TP-E-1GE FOX2520PABC
--- -------- Discovered N2K-C2248TP-E-1GE FOX2520PDEF
You'll see the new ports reflected in the show run output.
N5K-1# show run
interface Ethernet101/1/1
interface Ethernet101/1/2
interface Ethernet101/1/3
<OUTPUT TRUNCATED>
interface Ethernet101/1/46
interface Ethernet101/1/47
interface Ethernet101/1/48
N5K-1# show run interface Ethernet101/1/1
!Command: show running-config interface Ethernet101/1/1
!Time: Thu Apr 14 20:27:52 2005
version 7.3(8)N1(1)
interface Ethernet101/1/1
Perform the same configuration steps on the other parent N5K-2 switch since their configuration is independent from each other. The FEX 2K module will immediately show up as Connected then change to Online status.
N5K-2# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
--- -------- Discovered N2K-C2248TP-E-1GE FOX2519PDEF
--- -------- Connected N2K-C2248TP-E-1GE FOX2520PABC
N5K-2# 2005 Apr 14 19:42:10 N5K-2 %$ VDC-1 %$ %SATCTRL-FEX101-2-SATCTRL: FEX-101Module 1: Cold boot
2005 Apr 14 19:42:44 N5K-2 %$ VDC-1 %$ %SATCTRL-FEX101-2-SOHMS_DIAG_ERROR: FEX-101 System minor alarm on power supply 2: failed
2005 Apr 14 19:42:47 N5K-2 %$ VDC-1 %$ %SATCTRL-FEX101-2-SOHMS_DIAG_ERROR: FEX-101 Module 1: Runtime diag detected major event: Voltage failure on power supply: 2
2005 Apr 14 19:42:47 N5K-2 %$ VDC-1 %$ %SATCTRL-FEX101-2-SOHMS_DIAG_ERROR: FEX-101 System minor alarm on power supply 2: failed
2005 Apr 14 21:16:24 N5K-2 %$ VDC-1 %$ %SATCTRL-FEX101-2-SOHMS_ENV_ERROR: FEX-101 Module 1: Check environment alarms.
2005 Apr 14 21:16:28 N5K-2 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 101 is online
2005 Apr 14 21:16:28 N5K-2 %$ VDC-1 %$ %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-101 On-line
N5K-2# 2005 Apr 14 21:16:31 N5K-2 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 101 is online
N5K-2# show fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
101 FEX0101 Online N2K-C2248TP-E-1GE FOX2520ABC
--- -------- Discovered N2K-C2248TP-E-1GE FOX2520PDEF
No comments:
Post a Comment