I had to upgrade a Cisco 3850 switch due to a high CPU issue via the software install command.
Switch#show processes cpu sorted
Core 0: CPU utilization for five seconds: 77%; one minute: 71%; five minutes: 71%
Core 1: CPU utilization for five seconds: 28%; one minute: 71%; five minutes: 78%
Core 2: CPU utilization for five seconds: 96%; one minute: 72%; five minutes: 67%
Core 3: CPU utilization for five seconds: 99%; one minute: 90%; five minutes: 91%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
5632 547159 38565649 240 73.91 73.47 73.38 1088 fed
9223 2869678 17955697 98 0.97 1.17 1.15 0 iosd
6297 1592710 28366670 1186 0.14 0.15 0.14 0 cpumemd
6278 403412 5609218 837 0.10 0.01 0.03 0 oom_poll.sh
9219 3878616 89615676 25 0.10 0.10 0.10 0 wcm
5633 2355637 33902453 13 0.05 0.14 0.12 0 platform_mgr
5634 918091 19813894 41 0.05 0.06 0.05 0 stack-mgr
6286 605408 34263960 50 0.05 0.09 0.10 0 pdsd
6290 905582 23382817 22 0.05 0.01 0.01 0 netd
1 1650 1056 1562 0.00 0.00 0.00 0 init
2 0 126 0 0.00 0.00 0.00 0 kthreadd
3 9620 1171075 8 0.00 0.00 0.00 0 migration/0
4 0 3 0 0.00 0.00 0.00 0 sirq-high/0
5 1139468 45134130 2 0.00 0.00 0.00 0 sirq-timer/0
6 0 6 0 0.00 0.00 0.00 0 sirq-net-tx/0
7 10 9636 1 0.00 0.00 0.00 0 sirq-net-rx/0
8 1180 2734379 0 0.00 0.00 0.00 0 sirq-block/0
9 0 8 0 0.00 0.00 0.00 0 sirq-block-iopo
10 20 3286 6 0.00 0.00 0.00 0 sirq-tasklet/0
<OUTPUT TRUNCATED>
Once the IOS is copied to the switch's flash memory (via FTP or TFTP), you can issue the software install command from privileged EXEC prompt.
Switch#software install file flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
Preparing install operation ...
[1]: Starting install operation
[1]: Expanding bundle flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
[1]: Copying package files
[1]: Package files copied
[1]: Finished expanding bundle flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
[1]: Verifying and copying expanded package files to flash:
[1]: Verified and copied expanded package files to flash:
[1]: Starting compatibility checks
[1]: Finished compatibility checks
[1]: Starting application pre-installation processing
[1]: Finished application pre-installation processing
[1]: Old files list:
Removed cat3k_caa-base.SPA.03.02.03.SE.pkg
Removed cat3k_caa-drivers.SPA.03.02.03.SE.pkg
Removed cat3k_caa-infra.SPA.03.02.03.SE.pkg
Removed cat3k_caa-iosd-universalk9.SPA.150-1.EX3.pkg
Removed cat3k_caa-platform.SPA.03.02.03.SE.pkg
Removed cat3k_caa-wcm.SPA.10.0.120.0.pkg
[1]: New files list:
Added cat3k_caa-base.SPA.03.06.04.E.pkg
Added cat3k_caa-drivers.SPA.03.06.04.E.pkg
Added cat3k_caa-infra.SPA.03.06.04.E.pkg
Added cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
Added cat3k_caa-platform.SPA.03.06.04.E.pkg
Added cat3k_caa-wcm.SPA.10.2.140.0.pkg
[1]: Creating pending provisioning file
[1]: Finished installing software. New software will load on reboot.
[1]: Committing provisioning file
[1]: Do you want to proceed with reload? [yes/no]: yes
System configuration has been modified. Save? [yes/no]: yes
Building configuration...
Compressed configuration from 10347 bytes to 4871 bytes[OK]
[1]: Reloading // REMOTE ACCESS WAS LOST
<OUTPUT TRUNCATED>
The IOS upgrade took me around 15-20 minutes before I started to get ping replies on the switch.
Switch#show version | include INSTALL
* 1 56 WS-C3850-48P 03.06.04.E cat3k_caa-universalk9 INSTALL
Switch#show processes cpu // CPU LOAD FOR THE 4 CORE SIGNIFICANTLY DROPPED
Core 0: CPU utilization for five seconds: 6%; one minute: 8%; five minutes: 46%
Core 1: CPU utilization for five seconds: 0%; one minute: 2%; five minutes: 36%
Core 2: CPU utilization for five seconds: 3%; one minute: 2%; five minutes: 42%
Core 3: CPU utilization for five seconds: 1%; one minute: 4%; five minutes: 36%
<OUTPUT TRUNCATED>
Issue the software clean command to remove unwanted package files after everything is verified. You can't do a software rollback once the unwanted package files are completely removed.
Switch#software clean
Preparing clean operation ...
[1]: Cleaning up unnecessary package files
[1]: No path specified, will use booted path flash:packages.conf
[1]: Cleaning flash:
[1]: Preparing packages list to delete ...
In use files, will not delete:
cat3k_caa-base.SPA.03.06.04.E.pkg
cat3k_caa-drivers.SPA.03.06.04.E.pkg
cat3k_caa-infra.SPA.03.06.04.E.pkg
cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
cat3k_caa-platform.SPA.03.06.04.E.pkg
cat3k_caa-wcm.SPA.10.2.140.0.pkg
packages.conf
[1]: Files that will be deleted:
cat3k_caa-base.SPA.03.02.03.SE.pkg
cat3k_caa-drivers.SPA.03.02.03.SE.pkg
cat3k_caa-infra.SPA.03.02.03.SE.pkg
cat3k_caa-iosd-universalk9.SPA.150-1.EX3.pkg
cat3k_caa-platform.SPA.03.02.03.SE.pkg
cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
cat3k_caa-wcm.SPA.10.0.120.0.pkg
packages.conf.00-
[1]: Do you want to proceed with the deletion? [yes/no]: yes
[1]: Clean up completed
The same command is used to upgrade a stacked Cisco 3850 switches but using the switch <SWITCH NUMBER> keyword.
Switch#show switch
Switch/Stack Mac Address : c025.5c85.9123 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active c025.5c85.9123 10 V02 Ready
2 Standby c025.5c85.f456 1 V02 Ready
Swtich#software install file flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin switch ?
Valid switch ids: 1,2
Multiple switch ids may be specified using a
comma separated list or '-' separated range
For example:
1,3,4 specifies switch ids 1, 3 and 4
1-3 specifies switch ids 1, 2 and 3
WORD Switch id(s)
Swtich#software install file flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin switch 1-2
Preparing install operation ...
[1]: Copying software from active switch 1 to switch 2
[1 2]: Starting install operation
[1 2]: Expanding bundle flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
[1 2]: Copying package files
[1 2]: Package files copied
[1 2]: Finished expanding bundle flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
[1 2]: Verifying and copying expanded package files to flash:
[1 2]: Verified and copied expanded package files to flash:
[1 2]: Starting compatibility checks
[1 2]: Finished compatibility checks
[1 2]: Starting application pre-installation processing
[1 2]: Finished application pre-installation processing
[1]: Old files list:
Removed cat3k_caa-base.SPA.03.02.02.SE.pkg
Removed cat3k_caa-drivers.SPA.03.02.02.SE.pkg
Removed cat3k_caa-infra.SPA.03.02.02.SE.pkg
Removed cat3k_caa-iosd-universalk9.SPA.150-1.EX2.pkg
Removed cat3k_caa-platform.SPA.03.02.02.SE.pkg
Removed cat3k_caa-wcm.SPA.10.0.111.0.pkg
[2]: Old files list:
Removed cat3k_caa-base.SPA.03.02.02.SE.pkg
Removed cat3k_caa-drivers.SPA.03.02.02.SE.pkg
Removed cat3k_caa-infra.SPA.03.02.02.SE.pkg
Removed cat3k_caa-iosd-universalk9.SPA.150-1.EX2.pkg
Removed cat3k_caa-platform.SPA.03.02.02.SE.pkg
Removed cat3k_caa-wcm.SPA.10.0.111.0.pkg
[1]: New files list:
Added cat3k_caa-base.SPA.03.06.04.E.pkg
Added cat3k_caa-drivers.SPA.03.06.04.E.pkg
Added cat3k_caa-infra.SPA.03.06.04.E.pkg
Added cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
Added cat3k_caa-platform.SPA.03.06.04.E.pkg
Added cat3k_caa-wcm.SPA.10.2.140.0.pkg
[2]: New files list:
Added cat3k_caa-base.SPA.03.06.04.E.pkg
Added cat3k_caa-drivers.SPA.03.06.04.E.pkg
Added cat3k_caa-infra.SPA.03.06.04.E.pkg
Added cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
Added cat3k_caa-platform.SPA.03.06.04.E.pkg
Added cat3k_caa-wcm.SPA.10.2.140.0.pkg
[1 2]: Creating pending provisioning file
[1 2]: Finished installing software. New software will load on reboot.
[1 2]: Do you want to proceed with reload? [yes/no]:yes
[2]: Reloading
Switch#show processes cpu sorted
Core 0: CPU utilization for five seconds: 77%; one minute: 71%; five minutes: 71%
Core 1: CPU utilization for five seconds: 28%; one minute: 71%; five minutes: 78%
Core 2: CPU utilization for five seconds: 96%; one minute: 72%; five minutes: 67%
Core 3: CPU utilization for five seconds: 99%; one minute: 90%; five minutes: 91%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
5632 547159 38565649 240 73.91 73.47 73.38 1088 fed
9223 2869678 17955697 98 0.97 1.17 1.15 0 iosd
6297 1592710 28366670 1186 0.14 0.15 0.14 0 cpumemd
6278 403412 5609218 837 0.10 0.01 0.03 0 oom_poll.sh
9219 3878616 89615676 25 0.10 0.10 0.10 0 wcm
5633 2355637 33902453 13 0.05 0.14 0.12 0 platform_mgr
5634 918091 19813894 41 0.05 0.06 0.05 0 stack-mgr
6286 605408 34263960 50 0.05 0.09 0.10 0 pdsd
6290 905582 23382817 22 0.05 0.01 0.01 0 netd
1 1650 1056 1562 0.00 0.00 0.00 0 init
2 0 126 0 0.00 0.00 0.00 0 kthreadd
3 9620 1171075 8 0.00 0.00 0.00 0 migration/0
4 0 3 0 0.00 0.00 0.00 0 sirq-high/0
5 1139468 45134130 2 0.00 0.00 0.00 0 sirq-timer/0
6 0 6 0 0.00 0.00 0.00 0 sirq-net-tx/0
7 10 9636 1 0.00 0.00 0.00 0 sirq-net-rx/0
8 1180 2734379 0 0.00 0.00 0.00 0 sirq-block/0
9 0 8 0 0.00 0.00 0.00 0 sirq-block-iopo
10 20 3286 6 0.00 0.00 0.00 0 sirq-tasklet/0
<OUTPUT TRUNCATED>
Once the IOS is copied to the switch's flash memory (via FTP or TFTP), you can issue the software install command from privileged EXEC prompt.
Switch#software install file flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
Preparing install operation ...
[1]: Starting install operation
[1]: Expanding bundle flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
[1]: Copying package files
[1]: Package files copied
[1]: Finished expanding bundle flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
[1]: Verifying and copying expanded package files to flash:
[1]: Verified and copied expanded package files to flash:
[1]: Starting compatibility checks
[1]: Finished compatibility checks
[1]: Starting application pre-installation processing
[1]: Finished application pre-installation processing
[1]: Old files list:
Removed cat3k_caa-base.SPA.03.02.03.SE.pkg
Removed cat3k_caa-drivers.SPA.03.02.03.SE.pkg
Removed cat3k_caa-infra.SPA.03.02.03.SE.pkg
Removed cat3k_caa-iosd-universalk9.SPA.150-1.EX3.pkg
Removed cat3k_caa-platform.SPA.03.02.03.SE.pkg
Removed cat3k_caa-wcm.SPA.10.0.120.0.pkg
[1]: New files list:
Added cat3k_caa-base.SPA.03.06.04.E.pkg
Added cat3k_caa-drivers.SPA.03.06.04.E.pkg
Added cat3k_caa-infra.SPA.03.06.04.E.pkg
Added cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
Added cat3k_caa-platform.SPA.03.06.04.E.pkg
Added cat3k_caa-wcm.SPA.10.2.140.0.pkg
[1]: Creating pending provisioning file
[1]: Finished installing software. New software will load on reboot.
[1]: Committing provisioning file
[1]: Do you want to proceed with reload? [yes/no]: yes
System configuration has been modified. Save? [yes/no]: yes
Building configuration...
Compressed configuration from 10347 bytes to 4871 bytes[OK]
[1]: Reloading // REMOTE ACCESS WAS LOST
<OUTPUT TRUNCATED>
The IOS upgrade took me around 15-20 minutes before I started to get ping replies on the switch.
Switch#show version | include INSTALL
* 1 56 WS-C3850-48P 03.06.04.E cat3k_caa-universalk9 INSTALL
Switch#show processes cpu // CPU LOAD FOR THE 4 CORE SIGNIFICANTLY DROPPED
Core 0: CPU utilization for five seconds: 6%; one minute: 8%; five minutes: 46%
Core 1: CPU utilization for five seconds: 0%; one minute: 2%; five minutes: 36%
Core 2: CPU utilization for five seconds: 3%; one minute: 2%; five minutes: 42%
Core 3: CPU utilization for five seconds: 1%; one minute: 4%; five minutes: 36%
<OUTPUT TRUNCATED>
Issue the software clean command to remove unwanted package files after everything is verified. You can't do a software rollback once the unwanted package files are completely removed.
Switch#software clean
Preparing clean operation ...
[1]: Cleaning up unnecessary package files
[1]: No path specified, will use booted path flash:packages.conf
[1]: Cleaning flash:
[1]: Preparing packages list to delete ...
In use files, will not delete:
cat3k_caa-base.SPA.03.06.04.E.pkg
cat3k_caa-drivers.SPA.03.06.04.E.pkg
cat3k_caa-infra.SPA.03.06.04.E.pkg
cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
cat3k_caa-platform.SPA.03.06.04.E.pkg
cat3k_caa-wcm.SPA.10.2.140.0.pkg
packages.conf
[1]: Files that will be deleted:
cat3k_caa-base.SPA.03.02.03.SE.pkg
cat3k_caa-drivers.SPA.03.02.03.SE.pkg
cat3k_caa-infra.SPA.03.02.03.SE.pkg
cat3k_caa-iosd-universalk9.SPA.150-1.EX3.pkg
cat3k_caa-platform.SPA.03.02.03.SE.pkg
cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
cat3k_caa-wcm.SPA.10.0.120.0.pkg
packages.conf.00-
[1]: Do you want to proceed with the deletion? [yes/no]: yes
[1]: Clean up completed
The same command is used to upgrade a stacked Cisco 3850 switches but using the switch <SWITCH NUMBER> keyword.
Switch#show switch
Switch/Stack Mac Address : c025.5c85.9123 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
------------------------------------------------------------
*1 Active c025.5c85.9123 10 V02 Ready
2 Standby c025.5c85.f456 1 V02 Ready
Swtich#software install file flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin switch ?
Valid switch ids: 1,2
Multiple switch ids may be specified using a
comma separated list or '-' separated range
For example:
1,3,4 specifies switch ids 1, 3 and 4
1-3 specifies switch ids 1, 2 and 3
WORD Switch id(s)
Swtich#software install file flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin switch 1-2
Preparing install operation ...
[1]: Copying software from active switch 1 to switch 2
[1 2]: Starting install operation
[1 2]: Expanding bundle flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
[1 2]: Copying package files
[1 2]: Package files copied
[1 2]: Finished expanding bundle flash:cat3k_caa-universalk9.SPA.03.06.04.E.152-2.E4.bin
[1 2]: Verifying and copying expanded package files to flash:
[1 2]: Verified and copied expanded package files to flash:
[1 2]: Starting compatibility checks
[1 2]: Finished compatibility checks
[1 2]: Starting application pre-installation processing
[1 2]: Finished application pre-installation processing
[1]: Old files list:
Removed cat3k_caa-base.SPA.03.02.02.SE.pkg
Removed cat3k_caa-drivers.SPA.03.02.02.SE.pkg
Removed cat3k_caa-infra.SPA.03.02.02.SE.pkg
Removed cat3k_caa-iosd-universalk9.SPA.150-1.EX2.pkg
Removed cat3k_caa-platform.SPA.03.02.02.SE.pkg
Removed cat3k_caa-wcm.SPA.10.0.111.0.pkg
[2]: Old files list:
Removed cat3k_caa-base.SPA.03.02.02.SE.pkg
Removed cat3k_caa-drivers.SPA.03.02.02.SE.pkg
Removed cat3k_caa-infra.SPA.03.02.02.SE.pkg
Removed cat3k_caa-iosd-universalk9.SPA.150-1.EX2.pkg
Removed cat3k_caa-platform.SPA.03.02.02.SE.pkg
Removed cat3k_caa-wcm.SPA.10.0.111.0.pkg
[1]: New files list:
Added cat3k_caa-base.SPA.03.06.04.E.pkg
Added cat3k_caa-drivers.SPA.03.06.04.E.pkg
Added cat3k_caa-infra.SPA.03.06.04.E.pkg
Added cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
Added cat3k_caa-platform.SPA.03.06.04.E.pkg
Added cat3k_caa-wcm.SPA.10.2.140.0.pkg
[2]: New files list:
Added cat3k_caa-base.SPA.03.06.04.E.pkg
Added cat3k_caa-drivers.SPA.03.06.04.E.pkg
Added cat3k_caa-infra.SPA.03.06.04.E.pkg
Added cat3k_caa-iosd-universalk9.SPA.152-2.E4.pkg
Added cat3k_caa-platform.SPA.03.06.04.E.pkg
Added cat3k_caa-wcm.SPA.10.2.140.0.pkg
[1 2]: Creating pending provisioning file
[1 2]: Finished installing software. New software will load on reboot.
[1 2]: Do you want to proceed with reload? [yes/no]:yes
[2]: Reloading