The Cisco Nexus 9000 (N9K) switch upgrade procedure was simplified compared to upgrading an older Nexus platform. You can use the Cisco Nexus upgrade online tool to check and follow the upgrade path.
In this scenario, I needed to upgrade a Cisco N9K switch NX-OS: 10.1.x > 10.2(x)M >
10.3(x)M and also upgrade the 10.3.x.M EPLD image in the last step. You just need to upgrade the EPLD image once and it should be the same version of the final target NX-OS code.
Check first if the Nexus switch has enough space in bootflash.
N9K# dir bootflash:
4096 Jun 06 03:30:14 2024 .rpmstore/
4096 Jun 06 03:30:35 2024 .swtam/
1782 Jun 06 03:32:12 2024 20240606_033204_poap_25583_init.log
1782 Jun 06 03:50:51 2024 20240606_035045_poap_25617_init.log
455215 Jul 24 06:29:15 2024 20240724_041734_poap_25508_1.log
2097294 Jul 24 06:17:20 2024 20240724_041734_poap_25508_init.log
1777998029 Jun 06 03:34:10 2024 aci-n9000-dk9.14.1.2g.bin
4096 Jun 06 03:30:37 2024 eem_snapshots/
4096 Jun 06 03:30:34 2024 evt_log_snapshot/
4096 Jun 06 03:42:13 2024 home/
4096 Jun 06 03:36:26 2024 lost+found/
952 Jun 06 03:53:54 2024 lpssutil_lpss_log
1964521472 Jun 06 03:26:35 2024 nxos.10.1.x.bin
0 Jun 06 03:33:22 2024 platform-sdk.cmd
32665 Jul 24 06:28:44 2024 poap_retry_debugs.log
4096 Jun 06 03:31:08 2024 scripts/
4096 Jun 06 03:32:07 2024 virt_strg_pool_bf_vdc_1/
4096 Jun 06 03:31:07 2024 virtual-instance/
59 Jul 24 04:16:26 2024 virtual-instance.conf
4132 Aug 01 06:17:33 2024 vlan.dat
Usage for bootflash://sup-local
4148449280 bytes used
112038109184 bytes free
116186558464 bytes total
Transfer the NX-OS from an FTP/TFTP server to bootflash.
N9K# copy ftp://ftpuser:password@172.27.2.3/nxos64-cs.10.2.x.M.bin bootflash:nxos64-cs.10.2.x.M.bin
Enter vrf (If no input, current vrf 'default' is considered): <
[##################### ] 42.57MB
***** Transfer of file Completed Successfully *****
Copy complete, now saving to disk (please wait)...
Copy complete.
N9K# dir bootflash:
4096 Jun 06 03:30:14 2024 .rpmstore/
4096 Jun 06 03:30:35 2024 .swtam/
1782 Jun 06 03:32:12 2024 20240606_033204_poap_25583_init.log
1782 Jun 06 03:50:51 2024 20240606_035045_poap_25617_init.log
455215 Jul 24 06:29:15 2024 20240724_041734_poap_25508_1.log
2097294 Jul 24 06:17:20 2024 20240724_041734_poap_25508_init.log
1777998029 Jun 06 03:34:10 2024 aci-n9000-dk9.14.1.2g.bin
4096 Jun 06 03:30:37 2024 eem_snapshots/
4096 Jun 06 03:30:34 2024 evt_log_snapshot/
4096 Jun 06 03:42:13 2024 home/
4096 Jun 06 03:36:26 2024 lost+found/
952 Jun 06 03:53:54 2024 lpssutil_lpss_log
1964521472 Jun 06 03:26:35 2024 nxos.10.1.1.bin
2019112448 Aug 11 14:31:28 2024 nxos64-cs.10.2.x.M.bin
0 Jun 06 03:33:22 2024 platform-sdk.cmd
32665 Jul 24 06:28:44 2024 poap_retry_debugs.log
4096 Jun 06 03:31:08 2024 scripts/
4096 Jun 06 03:32:07 2024 virt_strg_pool_bf_vdc_1/
4096 Jun 06 03:31:07 2024 virtual-instance/
59 Jul 24 04:16:26 2024 virtual-instance.conf
4132 Aug 01 06:17:33 2024 vlan.dat
Usage for bootflash://sup-local
6144655360 bytes used
110041903104 bytes free
116186558464 bytes total
Verify the MD5 checksum and compare it to the checksum published in the Cisco download website. It only took a few seconds to complete this process.
N9K# show file bootflash:nxos64-cs.10.2.x.M.bin md5sum
8ec58aa31b351833cb85e78b69868123
Verify the compatibility of the new NX-OS.
N9K# show install all impact nxos
bootflash:nxos64-cs.10.2.x.M.bin
Installer will perform impact only check. Please wait.
Verifying image bootflash:/nxos64-cs.10.2.x.M.bin for boot variable "nxos".
[####################] 100% -- SUCCESS
Verifying image type.
[####################] 100% -- SUCCESS
Preparing "nxos" version info using image bootflash:/nxos64-cs.10.2.x.M.bin.
[####################] 100% -- SUCCESS
Preparing "bios" version info using image bootflash:/nxos64-cs.10.2.x.M.bin.
[####################] 100% -- SUCCESS
Performing module support checks.
[####################] 100% -- SUCCESS
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes disruptive reset default upgrade is not hitless
Images will be upgraded according to following table:
Module Image Running-Version(pri:alt) New-Version Upg-Required
------ ---------- ---------------------------------------- -------------------- ------------
1 nxos 10.1(x) 10.2(x) yes
1 bios v01.09(10/08/2023):v01.09(10/08/2023) v01.09(10/08/2023) no
Use the install command to initiate the NX-OS upgrade. Type 'y' to proceed and the Nexus switch will auto reboot.
N9K# install all nxos bootflash:nxos64-cs.10.2.x.M.bin
Installer will perform compatibility check first. Please wait.
Installer is forced disruptive
Verifying image bootflash:/nxos64-cs.10.2x.M.bin for boot variable "nxos".
[####################] 100% -- SUCCESS
Verifying image type.
[####################] 100% -- SUCCESS
Preparing "nxos" version info using image bootflash:/nxos64-cs.10.2.x.M.bin.
[####################] 100% -- SUCCESS
Preparing "bios" version info using image bootflash:/nxos64-cs.10.2.x.M.bin.
[####################] 100% -- SUCCESS
Performing module support checks.
[####################] 100% -- SUCCESS
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes disruptive reset default upgrade is not hitless
Images will be upgraded according to following table:
Module Image Running-Version(pri:alt) New-Version Upg-Required
------ ---------- ---------------------------------------- -------------------- ------------
1 nxos 10.1(1) 10.2(7) yes
1 bios v01.09(10/08/2023):v01.09(10/08/2023) v01.09(10/08/2023) no
Switch will be reloaded for disruptive upgrade.
Do
you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Setting boot variables.
[####################] 100% -- SUCCESS
Performing configuration copy.
[####################] 100% -- SUCCESS
Module 1: Refreshing compact flash and upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[####################] 100% -- SUCCESS
2024 Aug 12 08:01:51 N9K %$ VDC-1 %$ %VMAN-2-ACTIVATION_STATE: Successfully deactivated virtual service 'guestshell+'
Finishing the upgrade, switch will reboot in 10
seconds.
N9K#
CISCO MODULE
BIOS Ver: 1.09
Switch G10
RC Revision: 02.05.00
Memory Information:
MRC Revision:00.50.00
Total DRAM: 32768 MB
Memory TOLM: 80000000
PCIE BASE: 80000000 Size : 10000000
PCI32 BASE: 90000000 Limit: FBFFFFFF
PCI64 BASE: 80000000000 Limit: 83FFFFFFFFF
UC START: 80000000000 End : 84000000000
ME Operational Firmware Version: 06:3.0.3.214
DIMM Information:
Clock Speed: 1067MHz
Socket: 0x0 Channel: 0x0 Number: 0x0 Presence: Yes Size: 16GB
Socket: 0x0 Channel: 0x0 Number: 0x1 Presence: No
Socket: 0x0 Channel: 0x1 Number: 0x0 Presence: Yes Size: 16GB
Socket: 0x0 Channel: 0x1 Number: 0x1 Presence: No
Detected CISCO IOFPGA
Booting from Primary BIOS
Code Signing Results: 0x00000000
Booting from Upgrade FPGA
IOFPGA Subsystem Vendor ID 0x10ee
FPGA Revision : 0x00000013
FPGA ID : 0x16905123
FPGA Date : 0x20230456
Power Debug Register1 : 0x00000000
Power Debug Register2 : 0x110a000f
Reset Cause Register : 0x00000001
Boot Ctrl Register : 0x0000e0ff
FPGA Update Status : 0x80000020
Detected CISCO MIFPGA
MIFPGA Version : 0x00000123
MIFPGA Date : 0x20240456
MIFPGA Update Status : 0x00000020
MIFPGA ID : 0x21310123
<OUTPUT TRUNCATED>
Security Lock
Booting bootflash:/nxos64-cs.10.2.x.M.bin
Trying diskboot
Filesystem type is ext2fs, partition type 0x83
It took around 5 mins for the NX-OS upgrade to complete.
N9K login: 2024 Aug 12 08:05:37 N9K %$ VDC-1 %$ %USER-1-SYSTEM_MSG: Registering VNTAGC with dchal 1 - vntagc
2024 Aug 12 08:05:37 N9K %$ VDC-1 %$ %USER-1-SYSTEM_MSG: Registering MTM with dchal 1 - mtm
2024 Aug 12 08:05:39 N9K%$ VDC-1 %$ %USER-0-SYSTEM_MSG: NX mode supported by this card - 0x1fc - nf
2024 Aug 12 08:05:47 N9K %$ VDC-1 %$ %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Ascii Replay Started.
2024 Aug 12 08:05:52 N9K %$ VDC-1 %$ %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Ascii Replay Done.
2024 Aug 12 08:05:53 N9K %$ VDC-1 %$ %ASCII-CFG-2-CONF_CONTROL: System ready
2024 Aug 12 08:06:11 N9K %$ VDC-1 %$ %USER-2-SYSTEM_MSG: Thirdparty RPMs installation succeeded - /thirdparty_install.py
2024 Aug 12 08:06:19 N9K %$ VDC-1 %$ %VMAN-2-ACTIVATION_STATE: Successfully activated virtual service 'guestshell+'
2024 Aug 12 08:06:19 N9K %$ VDC-1 %$ %VMAN-2-GUESTSHELL_ENABLED: The guest shell has been enabled. The command 'guestshell' may be used to access it, 'guestshell destroy' to remove it.
Verify the new NX-OS using the show version command.
N9K# show version
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (C) 2002-2024, Cisco and/or its affiliates.
All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under their own
licenses, such as open source. This software is provided "as is," and unless
otherwise stated, there is no warranty, express or implied, including but not
limited to warranties of merchantability and fitness for a particular purpose.
Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or
GNU General Public License (GPL) version 3.0 or the GNU
Lesser General Public License (LGPL) Version 2.1 or
Lesser General Public License (LGPL) Version 2.0.
A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://opensource.org/licenses/gpl-3.0.html and
http://www.opensource.org/licenses/lgpl-2.1.php and
http://www.gnu.org/licenses/old-licenses/library.txt.
Software
BIOS: version 01.09
NXOS: version 10.2(x) [Maintenance Release]
BIOS compile time: 10/08/2023
NXOS image file is: bootflash:///nxos64-cs.10.2.x.M.bin
NXOS compile time: 2/28/2024 12:00:00 [02/17/2024 21:17:45]
Hardware
cisco Nexus9000 C93xx-xx Chassis
Intel(R) Xeon(R) CPU D-1526 @ 1.80GHz with 32813048 kB of memory.
Processor Board ID FDO282217Q1
Device name: N9K
bootflash: 115805708 kB
Kernel uptime is 0 day(s), 0 hour(s), 3 minute(s), 34 second(s)
Last reset at 526211 usecs after Mon Aug 12 08:02:04 2024
Reason: Reset due to upgrade
System version: 10.1(1)
Service:
plugin
Core Plugin, Ethernet Plugin
Active Package(s):
The memory space in bootflash isn't large enough, so you'll need to delete the previous NX-OS image file. Repeat the same steps until the final target NX-OS code. Ensure there's at least one previous NX-OS code for backup.
/bootflash/nxos64-cs.10.3.x.M.bin: Write could not complete, check free space on device
Error during copy
***** Transfer of file aborted *****
Copy
failed. Removing file nxos64-cs.10.3.x.M.bin
N9K# delete bootflash:nxos.10.1.x.bin
Do
you want to delete "/nxos.10.1.x.bin" ? (yes/no/abort) [y] y
Once the final NX-OS is running, you'll upgrade the EPLD image. Use an identical image version with the main NX-OS.
N9K# install all nxos nxos64-cs.10.3.x.M.bin epld n9000-epld.10.3.x.M.img
Installer will perform compatibility check first. Please wait.
Installer is forced disruptive
Verifying image bootflash:/nxos64-cs.10.3.x.M.bin for boot variable "nxos".
[####################] 100% -- SUCCESS
Verifying EPLD/FPGA image bootflash:/n9000-epld.10.3.x.M.img.
[####################] 100% -- SUCCESS
Verifying image type.
[####################] 100% -- SUCCESS
Preparing "nxos" version info using image bootflash:/nxos64-cs.10.3.x.M.bin.
[####################] 100% -- SUCCESS
Preparing "bios" version info using image bootflash:/nxos64-cs.10.3.x.M.bin.
[####################] 100% -- SUCCESS
Performing module support checks.
[####################] 100% -- SUCCESS
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes disruptive reset default upgrade is not hitless
27 yes disruptive none default upgrade is not hitless
Images will be upgraded according to following table:
Module Image Running-Version(pri:alt) New-Version Upg-Required
------ ---------- ---------------------------------------- -------------------- ------------
1 lcn9k 10.3(x) 10.3(x) no
27 nxos 10.3(x) 10.3(x) no
27 bios v01.09(10/08/2023):v01.09(10/08/2023) v01.09(10/08/2023) no
FPGA microcode will be upgraded according to following table:
Module Type EPLD Running-Version New-Version Upg-Required
------ ---- ------------- --------------- ----------- ------------
27 SUP MI FPGA 0x20 0x18 No
27 SUP IO FPGA 0x13 0x13 No
EPLD Upgrade may result in multiple modules going offline.
Additional info for this installation:
--------------------------------------
Service "vpc" in vdc 1: Vpc is enabled, Please make sure both Vpc peer switches have same boot mode using 'show boot mode' and proceed
Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Setting boot variables.
[[####################] 100% -- SUCCESS
Performing configuration copy.
[####################] 100% -- SUCCESS
Module 1: Refreshing compact flash and upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[####################] 100% -- SUCCESS
Module 27: Refreshing compact flash and upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[####################] 100% -- SUCCESS
Performing EPLD/FPGA upgrade bootflash:/n9000-epld.10.3.x.M.img.
[####################] 100% -- SUCCESS
All EPLD/FPGAs are up to date.
Install has been successful.
