Stacking Cisco Catalyst 3650 Switches

I've stacked a few Cisco 2960S switches last year and now I'll be stacking switches again but this time on a Cisco 3650 switch. The stacking technique and commands are similar to a Cisco Catalyst 3850. There were IOS version mismatch when I initially booted up the switches. So I've quickly transferred and booted the same IOS via its USB port.

The Cisco 3650 uses a StackWise-160 cable and adapter. The StackWise kit also includes a hex key wrench.

Booting...Initializing RAM +++++++@@@@@@@@...++++++++++++++++++++++++++++++++@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@done.
Memory Test Pass!

Base ethernet MAC Address: 18:8b:9d:0a:d1:23

Interface GE 0 link down***ERROR: PHY link is down
Initializing Flash...

flashfs[7]: 0 files, 1 directories
flashfs[7]: 0 orphaned files, 0 orphaned directories
flashfs[7]: Total bytes: 6784000
flashfs[7]: Bytes used: 1024
flashfs[7]: Bytes available: 6782976
flashfs[7]: flashfs fsck took 1 seconds....done Initializing Flash.
Getting rest of image
Reading full image into memory....done
Reading full base package into memory...: done = 79122052
Bundle Image
--------------------------------------
Kernel Address    : 0x6042d350
Kernel Size       : 0x402ecf/4206287
Initramfs Address : 0x60830220
Initramfs Size    : 0xdb9c62/14392418
Compression Format: .mzip

Bootable image at @ ram:0x6042d350
Bootable image segment 0 address range [0x81100000, 0x82110000] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@boot_system: 377
Loading Linux kernel with entry point 0x81653a10 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

All packages are Digitally Signed
Starting System Services
Sep 28 06:05:26 %STACKMGR-1-STACK_LINK_CHANGE: Stack port 1 on switch 1 is up
#######################################################################################################################


              Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706


Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.03.05SE RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 30-Oct-14 13:12 by prod_rel_team

Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.


FIPS: Flash Key Check : Begin
FIPS: Flash Key Check : End, Not Found,FIPS Mode Not Enabled

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C3650-48PS (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID FDO1817QDEF
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
250456K bytes of Crash Files at crashinfo:.
250456K bytes of Crash Files at crashinfo-2:.
250456K bytes of Crash Files at crashinfo-3:.
1609272K bytes of Flash at flash:.
1609272K bytes of Flash at flash-2:.
1609272K bytes of Flash at flash-3:.
0K bytes of Dummy USB Flash at usbflash0:.
0K bytes of Dummy USB Flash at usbflash0-2:.
0K bytes of Dummy USB Flash at usbflash0-3:.
0K bytes of  at webui:.

Base Ethernet MAC Address          : 18:8b:9d:0a:d1:23
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO18268ABC
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1817QDEF


Press RETURN to get started!

Compressed configuration from 7132 bytes to 2169 bytes[OK]
Switch01>enable
Switch01#show switch
Switch/Stack Mac Address : 188b.9d0a.d123 - Local Mac Address
Mac persistency wait time: Indefinite
                                             H/W   Current
Switch#   Role    Mac Address     Priority Version  State
------------------------------------------------------------
*1       Active   188b.9d0a.d123     1      V02     Ready              
 2       Member   80e0.1de5.6456     1      0       V-Mismatch   // IOS VERSION MISMATCH     
 3       Member   84b8.028f.c789     1      0       V-Mismatch   

Switch01#show version
Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.03.05SE RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 30-Oct-14 13:12 by prod_rel_team


Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.


ROM: IOS-XE ROMMON
BOOTLDR: CAT3K_CAA Boot Loader (CAT3K_CAA-HBOOT-M) Version 1.18, RELEASE SOFTWARE (P)
         
Switch01 uptime is 7 minutes
Uptime for this control processor is 15 minutes
System returned to ROM by reload
System image file is "flash:packages.conf"
Last reload reason: reload


This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.
         
License Level: Ipbase
License Type: Permanent
Next reload license Level: Ipbase

cisco WS-C3650-48PS (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID FDO1817QDEF
1 Virtual Ethernet interface
156 Gigabit Ethernet interfaces
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
250456K bytes of Crash Files at crashinfo:.
250456K bytes of Crash Files at crashinfo-2:.
250456K bytes of Crash Files at crashinfo-3:.
1609272K bytes of Flash at flash:.
1609272K bytes of Flash at flash-2:.
1609272K bytes of Flash at flash-3:.
0K bytes of Dummy USB Flash at usbflash0:.
0K bytes of Dummy USB Flash at usbflash0-2:.
0K bytes of Dummy USB Flash at usbflash0-3:.
0K bytes of  at webui:.

Base Ethernet MAC Address          : 18:8b:9d:0a:d1:23
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO18268ABC
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1817QDEF


Switch Ports Model              SW Version        SW Image              Mode  
------ ----- -----              ----------        ----------            ----  
*    1 52    WS-C3650-48PS      03.03.05SE        cat3k_caa-universalk9 INSTALL


Configuration register is 0x102

Switch01#show flash
-#- --length-- ---------date/time--------- path
  2   79122052 Jun 25 2015 01:07:42.0000000000 +00:00 cat3k_caa-base.SPA.03.03.05SE.pkg
  3    6521532 Jun 25 2015 01:07:43.0000000000 +00:00 cat3k_caa-drivers.SPA.03.03.05SE.pkg
  4   34530288 Jun 25 2015 01:07:42.0000000000 +00:00 cat3k_caa-infra.SPA.03.03.05SE.pkg
  5   34846028 Jun 25 2015 01:07:43.0000000000 +00:00 cat3k_caa-iosd-universalk9.SPA.150-1.EZ5.pkg
  6   25170832 Jun 25 2015 01:07:42.0000000000 +00:00 cat3k_caa-platform.SPA.03.03.05SE.pkg 
  7   77456192 Jun 25 2015 01:07:43.0000000000 +00:00 cat3k_caa-wcm.SPA.10.1.150.0.pkg
  8       1247 Jun 25 2015 01:07:54.0000000000 +00:00 packages.conf
  9    2097152 Sep 28 2015 06:12:05.0000000000 +00:00 nvram_config
1277083648 bytes available (262492160 bytes used)

Switch01#session ?
  standby  Connect to the standby
  switch   Connect to diag-shell

Switch01#session switch ?
  <1-3>  Enter switch #
  <cr>

Switch01#session switch 2    // TO CONSOLE TO SW02
Switch01(diag)>
Exec mode commands
  codecov   request cflow command
  default   Set a command to its defaults
  enable    Enable privilege level
  license   Configure license.
  no        Negate a command or set its defaults
  request   Request commands
  session   Open a connection
  wireless  Configure wireless parameters
Switch01(diag)>enable
Password:
Enable password is not set
Switch01(diag)>
Exec mode commands
  codecov   request cflow command
  default   Set a command to its defaults
  enable    Enable privilege level
  license   Configure license.
  no        Negate a command or set its defaults
  request   Request commands
  session   Open a connection
  wireless  Configure wireless parameters
Switch01(diag)>show
% Unrecognized command      // CAN'T ISSUE COMMANDS SINCE THERE'S VERSION MISMATCH

Switch01#dir usbflash0:
Directory of usbflash0:/

  112  -rwx           0  Aug 20 2013 07:39:20 +00:00  system
  113  -rwx    62682268  Mar 29 2012 12:04:58 +00:00  c2900-universalk9-mz.SPA.150-1.M4.bin
  114  -rwx    21890692  May 26 2012 13:15:44 +00:00  c870-advipservicesk9-mz.124-24.T4.bin
  115  -rwx     4968160   Dec 9 2014 11:44:34 +00:00  TeamViewerQS_en-idch93gk2g.exe
  116  -rwx   310347344   Feb 4 2015 12:52:10 +00:00  cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin
  117  -rwx   125231421  Feb 12 2015 17:00:24 +00:00  lms5.1.bin
  118  -rwx   257650008  Mar 12 2015 14:01:06 +00:00  cat3k_caa-universalk9ldpe.SPA.03.03.05.SE.150-1.EZ5.bin
  119  -rwx    38172672  May 21 2015 10:01:48 +00:00  asa916-4-smp-k8.bin
  120  -rwx   257651868  Sep 28 2015 14:08:58 +00:00  cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin

2013200384 bytes total (934510592 bytes free)
Switch01#
Switch01#copy usbflash0:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin
Destination filename [cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin]?
Copy in progress...CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC

<OUTPUT TRUNCATED>

CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
257650008 bytes copied in 43.970 secs (5859677 bytes/sec)
Switch01#
Switch01#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Switch01(config)#boot system flash:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin
Switch01(config)#do show run | inc boot
boot-start-marker
boot system switch all flash:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.binn
boot-end-marker
diagnostic bootup level minimal
Switch01(config)#end
Switch01#write memory
Building configuration...
Compressed configuration from 5425 bytes to 1982 bytes[OK]
Switch02#reload
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]

<Mon Sep 28 06:34:06 2015> Message from sysmgr: Reason Code:[3] Reset Reason:Reset/Reload requested by [stack-manager]. [Reload command]
umount: /proc/fs/nfsd: not mounted


Switch02#copy usbflash0:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin
Destination filename [cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin]?
Copy in progress...CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC

<OUTPUT TRUNCATED>

CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
257650008 bytes copied in 43.970 secs (5859677 bytes/sec)
Switch02#
Switch02#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Switch02(config)#boot system flash:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin
Switch02(config)#do show run | inc boot
boot-start-marker
boot system switch all flash:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.binn
boot-end-marker
diagnostic bootup level minimal
Switch02(config)#end
Switch02#write memory
Building configuration...
Compressed configuration from 5425 bytes to 1982 bytes[OK]
Switch#reload
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]

<Mon Sep 28 07:17:05 2015> Message from sysmgr: Reason Code:[3] Reset Reason:Reset/Reload requested by [stack-manager]. [Reload command]
umount: /proc/fs/nfsd: not mounted

<OUTPUT TRUNCATED>


Switch02 became the active switch in the stack since its MAC address is higher than Switch01. Switch03 didn't join the stack since it has IOS version mismatch.


Sep 28 07:36:23 %STACKMGR-1-STACK_LINK_CHANGE: Stack port 1 on switch 1 is up
#######################################################################################################################

Switch02 console is now available


Press RETURN to get started.


Switch02>enable
Switch02#show switch
Switch/Stack Mac Address : 80e0.1de5.6456 - Local Mac Address
Mac persistency wait time: Indefinite
                                             H/W   Current
Switch02#   Role    Mac Address     Priority Version  State
------------------------------------------------------------
 1       Member   188b.9d0a.d123     1      V02     Ready           
*2       Active   80e0.1de5.6456     1      V02     Ready  
 3       Member   84b8.028f.c789     1      0       V-Mismatch

Switch02#show version
Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.03.05.SE RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 30-Oct-14 13:12 by prod_rel_team



Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.



ROM: IOS-XE ROMMON
BOOTLDR: CAT3K_CAA Boot Loader (CAT3K_CAA-HBOOT-M) Version 1.18, RELEASE SOFTWARE (P)
         
Switch02 uptime is 1 minute
Uptime for this control processor is 9 minutes
System returned to ROM by reload
System image file is "flash:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin"
Last reload reason: Reload command



This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.
         
License Level: Ipbase
License Type: Permanent
Next reload license Level: Ipbase

cisco WS-C3650-48PS (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID FDO1817QJKL
1 Virtual Ethernet interface
156 Gigabit Ethernet interfaces
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
250456K bytes of Crash Files at crashinfo:.
250456K bytes of Crash Files at crashinfo-1:.
250456K bytes of Crash Files at crashinfo-3:.
1609272K bytes of Flash at flash:.
1609272K bytes of Flash at flash-1:.
1609272K bytes of Flash at flash-3:.
0K bytes of Dummy USB Flash at usbflash0:.
0K bytes of Dummy USB Flash at usbflash0-1:.
0K bytes of Dummy USB Flash at usbflash0-3:.
0K bytes of  at webui:.

Base Ethernet MAC Address          : 80:e0:1d:e5:64:56
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO18268GHI
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1817QJKL


Switch Ports Model              SW Version        SW Image              Mode  
------ ----- -----              ----------        ----------            ----  
     1 52    WS-C3650-48PS      03.03.05.SE       cat3k_caa-universalk9 BUNDLE
*    2 52    WS-C3650-48PS      03.03.05.SE       cat3k_caa-universalk9 BUNDLE


Switch 01
--------
Switch01 uptime                    : 7 minutes
Base Ethernet MAC Address          : 18:8b:9d:0a:d1:23
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO18268ABC
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1817QDEF

Configuration register is 0x102


Switch02#software ?
  auto-upgrade  Initiate auto upgrade for switches running incompatible
                software
  clean         Clean unused package files from local media
  commit        Commit the provisioned software and cancel the automatic
                rollback timer
  expand        Expand a software bundle to local storage, default location is
                where the bundle currently resides
  install       Install software
  rollback      Rollback the committed software

Switch02#software auto-upgrade  // TRIED TO UPGRADE SW03 STACK MEMBER
% The active switch must be running installed software to perform
  the 'software auto-upgrade' operation


Switch03#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Switch03(config)#boot system flash:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin
Switch03(config)#end
*Sep 28 07:47:05.293: %SYS-5-CONFIG_I: Configured from console by console
Switch03#reload

<OUTPUT TRUNCATED>


CONSOLE TO SW01

#Sep 28 07:59:47 %STACKMGR-1-STACK_LINK_CHANGE: Stack port 1 on switch 1 is up


              Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706



Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.03.05.SE RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 30-Oct-14 13:12 by prod_rel_team

Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.


FIPS: Flash Key Check : Begin
FIPS: Flash Key Check : End, Not Found,FIPS Mode Not Enabled

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C3650-48PS (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID FDO1817Q1MR
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
250456K bytes of Crash Files at crashinfo:.
250456K bytes of Crash Files at crashinfo-2:.
250456K bytes of Crash Files at crashinfo-3:.
1609272K bytes of Flash at flash:.
1609272K bytes of Flash at flash-2:.
1609272K bytes of Flash at flash-3:.
0K bytes of Dummy USB Flash at usbflash0:.
0K bytes of Dummy USB Flash at usbflash0-2:.
0K bytes of Dummy USB Flash at usbflash0-3:.
0K bytes of  at webui:.

Base Ethernet MAC Address          : 18:8b:9d:0a:d1:23 
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO18268ABC
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1817QDEF

The System is using a non-recommended Boot mode.
Not all features may be available in this boot mode.
Please check the product configuration guide.


Switch 02  
---------
Switch uptime                      : 4 minutes
Base Ethernet MAC Address          : 80:e0:1d:e5:64:56
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO18268GHI
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1817QJKL

Switch 03  
---------
Switch uptime                      : 3 minutes
Base Ethernet MAC Address          : 84:b8:02:8f:c7:89
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO1826TMNO
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1825QPQR


Press RETURN to get started!


Switch01>% Generating 1024 bit RSA keys, keys will be non-exportable...
[OK] (elapsed time was 1 seconds)
Compressed configuration from 7213 bytes to 2239 bytes[OK]



Switch01#show switch
Switch/Stack Mac Address : 188b.9d0a.d680 - Local Mac Address
Mac persistency wait time: Indefinite
                                             H/W   Current
Switch01#   Role    Mac Address     Priority Version  State
------------------------------------------------------------
*1       Active   188b.9d0a.d123     1      V02     Ready     // BOOTED FIRST           
 2       Standby  80e0.1de5.6456     1      V02     Ready     // BOOTED SECOND         
 3       Member   84b8.028f.c789    1      V02     Ready     // BOOTED LAST         



Switch01#show version
Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.03.05.SE RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 30-Oct-14 13:12 by prod_rel_team


Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.


ROM: IOS-XE ROMMON
BOOTLDR: CAT3K_CAA Boot Loader (CAT3K_CAA-HBOOT-M) Version 1.18, RELEASE SOFTWARE (P)
         
Switch01 uptime is 5 minutes
Uptime for this control processor is 10 minutes
System returned to ROM by reload
System image file is "flash:cat3k_caa-universalk9.SPA.03.03.05.SE.150-1.EZ5.bin"
Last reload reason: reload


This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.
         
License Level: Ipbase
License Type: Permanent
Next reload license Level: Ipbase

cisco WS-C3650-48PS (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID FDO1817Q1MR
1 Virtual Ethernet interface
156 Gigabit Ethernet interfaces
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
250456K bytes of Crash Files at crashinfo:.
250456K bytes of Crash Files at crashinfo-2:.
250456K bytes of Crash Files at crashinfo-3:.
1609272K bytes of Flash at flash:.
1609272K bytes of Flash at flash-2:.
1609272K bytes of Flash at flash-3:.
0K bytes of Dummy USB Flash at usbflash0:.
0K bytes of Dummy USB Flash at usbflash0-2:.
0K bytes of Dummy USB Flash at usbflash0-3:.
0K bytes of  at webui:.

Base Ethernet MAC Address          : 18:8b:9d:0a:d1:23
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO18268ABC
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1817QDEF


Switch Ports Model              SW Version        SW Image              Mode  
------ ----- -----              ----------        ----------            ----  
*    1 52    WS-C3650-48PS      03.03.05.SE       cat3k_caa-universalk9 BUNDLE  
     2 52    WS-C3650-48PS      03.03.05.SE       cat3k_caa-universalk9 BUNDLE  
     3 52    WS-C3650-48PS      03.03.05.SE       cat3k_caa-universalk9 BUNDLE  


Switch 02
---------
Switch uptime                      : 9 minutes
Base Ethernet MAC Address          : 80:e0:1d:e5:64:56
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO18268GHI
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1817QJKL

Switch 03
---------
Switch uptime                      : 8 minutes
Base Ethernet MAC Address          : 84:b8:02:8f:c7:89
Motherboard Assembly Number        : 73-15901-04
Motherboard Serial Number          : FDO1826TMNO
Model Revision Number              : H0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-48PS
System Serial Number               : FDO1825QPQR

Configuration register is 0x102


Switch#switch ?
  <1-9>  Switch Number

Switch#switch 1 ?
  priority  Set the priority of the specified switch
  renumber  Renumber the specified switch number
  stack     Stack port enable or disable

Switch#switch 1 priority ?
  <1-15>  Switch Priority

Switch#switch 1 priority 15   // HIGHER PRIORITY PREFERRED
WARNING: Changing the switchpriority may result in a configuration change for that switch. Do you want to continue?[y/n]y
Switch#switch 2 priority 12
WARNING: Changing the switchpriority may result in a configuration change for that switch. Do you want to continue?[y/n]y
Switch#switch 3 priority 10
WARNING: Changing the switchpriority may result in a configuration change for that switch. Do you want to continue?[y/n]y
Switch#
Switch#show switch
Switch/Stack Mac Address : 188b.9d0a.d680 - Local Mac Address
Mac persistency wait time: Indefinite
                                             H/W   Current
Switch#   Role    Mac Address     Priority Version  State
------------------------------------------------------------
*1       Active   188b.9d0a.d123     15     V02     Ready              
 2       Standby  80e0.1de5.6456     12     V02     Ready              
 3       Member   84b8.028f.c789     10     V02     Ready 

Switch#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Switch(config)#switch ?
  <1-9>  Switch Number

Switch(config)#switch 1 ?
  provision  Configure Switch provision / offline config

Switch(config)#switch 1 provision ?
  ws-c3650-24pd  Catalyst 3650 Series Switch with 24 GE POE and 2GE+2TenGE
                 Interfaces
  ws-c3650-24ps  Catalyst 3650 Series Switch with 24 GE POE and 4 GE Interfaces
  ws-c3650-24td  Catalyst 3650 Series Switch with 24 GE and 2GE+2TenGE
                 Interfaces
  ws-c3650-24ts  Catalyst 3650 Series Switch with 24 GE and 4 GE Interfaces
  ws-c3650-48pd  Catalyst 3650 Series Switch with 48 GE POE and 2GE+2TenGE
                 Interfaces
  ws-c3650-48pq  Catalyst 3650 Series Switch with 48 GE POE and 4 TenGE
                 Interfaces
  ws-c3650-48ps  Catalyst 3650 Series Switch with 48 GE POE and 4 GE Interfaces
  ws-c3650-48td  Catalyst 3650 Series Switch with 48 GE and 2GE+2TenGE
                 Interfaces
  ws-c3650-48tq  Catalyst 3650 Series Switch with 48 GE and 4 TenGE Interfaces
  ws-c3650-48ts  Catalyst 3650 Series Switch with 48 GE and 4 GE Interfaces
  ws-c3850-12s   Catalyst 3850 Series Switch with 12 1GE SFP Interfaces
  ws-c3850-24p   Catalyst 3850 Series Switch with 24 GE POE Interfaces
  ws-c3850-24s   Catalyst 3850 Series Switch with 24 1GE SFP Interfaces
  ws-c3850-24t   Catalyst 3850 Series Switch with 24 GE Interfaces
  ws-c3850-24u   Catalyst 3850 Series Switch with 24 GE UPOE Interfaces
  ws-c3850-48p   Catalyst 3850 Series Switch with 48 GE POE Interfaces
  ws-c3850-48t   Catalyst 3850 Series Switch with 48 GE Interfaces
  ws-c3850-48u   Catalyst 3850 Series Switch with 48 GE UPOE Interfaces

Switch01(config)#switch 1 provision ws-c3650-48ps   // OPTIONAL COMMAND; ENABLE OFFLINE PORT CONFIGURATION/PROVISION PORTS ON ADDITIONAL SWITCH MEMBER (BEFORE STACKING)
Switch01(config)#switch 2 provision ws-c3650-48ps
Switch01(config)#switch 3 provision ws-c3650-48ps

Switch01#show switch detail
Switch/Stack Mac Address : 188b.9d0a.d680 - Local Mac Address
Mac persistency wait time: Indefinite
                                             H/W   Current
Switch#   Role    Mac Address     Priority Version  State
------------------------------------------------------------
*1       Active   188b.9d0a.d123    15     V02     Ready              
 2       Standby  80e0.1de5.6456     12     V02     Ready              
 3       Member   84b8.028f.c789     10     V02     Ready              



         Stack Port Status             Neighbors    
Switch#  Port 1     Port 2           Port 1   Port 2
--------------------------------------------------------
  1         OK         OK               2        3
  2         OK         OK               3        1
  3         OK         OK               1        2


Switch01#show interface status

Port      Name               Status       Vlan       Duplex  Speed Type
Gi1/0/1                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/2                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/3                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/4                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/5                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/6                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/7                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/8                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/9                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/10                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/11                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi1/0/12                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/13                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/14                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/15                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/16                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/17                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/18                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/19                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/20                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/21                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/22                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/23                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi1/0/24                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/25                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/26                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/27                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/28                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/29                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/30                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/31                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/32                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/33                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/34                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/35                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi1/0/36                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/37                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/38                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/39                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/40                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/41                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/42                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/43                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/44                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/45                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/46                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/0/47                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi1/0/48                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi1/1/1                      notconnect   1            auto   auto unknown
Gi1/1/2                      notconnect   1            auto   auto unknown
Gi1/1/3                      notconnect   1            auto   auto unknown
Gi1/1/4                      notconnect   1            auto   auto unknown
Gi2/0/1                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/2                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/3                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/4                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/5                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/6                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/7                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/8                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/9                      notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi2/0/10                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/11                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/12                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/13                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/14                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/15                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/16                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/17                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/18                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/19                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/20                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/21                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi2/0/22                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/23                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/24                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/25                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/26                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/27                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/28                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/29                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/30                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/31                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/32                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/33                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi2/0/34                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/35                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/36                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/37                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/38                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/39                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/40                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/41                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/42                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/43                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/44                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/45                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi2/0/46                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/47                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/0/48                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi2/1/1                      notconnect   1            auto   auto unknown
Gi2/1/2                      notconnect   1            auto   auto unknown
Gi2/1/3                      notconnect   1            auto   auto unknown
Gi2/1/4                      notconnect   1            auto   auto unknown
Gi3/0/1                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/2                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/3                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/4                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/5                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/6                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/7                      notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi3/0/8                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/9                      notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/10                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/11                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/12                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/13                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/14                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/15                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/16                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/17                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/18                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/19                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi3/0/20                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/21                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/22                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/23                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/24                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/25                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/26                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/27                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/28                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/29                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/30                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/31                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi3/0/32                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/33                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/34                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/35                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/36                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/37                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/38                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/39                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/40                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/41                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/42                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/43                     notconnect   1            auto   auto 10/100/1000BaseTX
         
Port      Name               Status       Vlan       Duplex  Speed Type
Gi3/0/44                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/45                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/46                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/47                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/0/48                     notconnect   1            auto   auto 10/100/1000BaseTX
Gi3/1/1                      notconnect   1            auto   auto unknown
Gi3/1/2                      notconnect   1            auto   auto unknown
Gi3/1/3                      notconnect   1            auto   auto unknown
Gi3/1/4                      notconnect   1            auto   auto unknown

Emerson Liebert GXT4 UPS Web Card

I was asked to setup an Emerson Liebert GXT4 UPS IntelliSlot (IS) Web Card for SNMP and monitor its performance in Solarwinds NPM. The IS web card slot is located on top of the UPS fan and has two onboard ports: an Ethernet port for LAN/IP connectivity and an RS232/serial port for local console administration. We use Liebert for the initial username and password.

EMERSON NETWORK POWER IS WEB CARD

Initializing...

Initializing network...
Interface callback: FNS_IF_EVENT_INTERFACE_UP for eth0
Network initialization completed successfully!
Initial DNS resolution complete!

Interface callback: FNS_IF_EVENT_ADDRESS_READY for fe80::202:99ff:fe16:5123.

Booting...

****************************************************************
IntelliSlot Web Card Boot Loader

(C)Copyright 2000-2014 Emerson Network Power

Build Label .............: IS-WEBCARD_HID9_5.300.2_082246

Software Version Number .: v5.300.2
Source Control Number ...: 00082246
Product ID ..............: 00000002
Hardware ID .............: 00000009
Header Revision Number ..: 00000001
Firmware Checksum .......:   0x075C
****************************************************************


****************************************************************
IntelliSlot Web Card

(C)Copyright 2000-2014 Emerson Network Power

Build Label .............: IS-WEBCARD_HID9_5.300.2_082246

Software Version Number .: v5.300.2
Source Control Number ...: 00082246
Product ID ..............: 00000002
Hardware ID .............: 00000009
Header Revision Number ..: 00000001
Firmware Checksum .......:   0x5F13
****************************************************************

Calculating Boot Checksum
Boot Checksum:          0x075C
Saved Boot Checksum:    0x075C
Checksum Valid

Calculating Application Checksum
Application Checksum:          0x5F13
Saved Application Checksum:    0x5F13
Checksum Valid

MAC Address:          00-02-99-16-51-23
Serial Number:        417771G109C2015MAR200123
Manufacture Date:     MAR 20 2015

Initializing...

Initializing network...
Interface callback: FNS_IF_EVENT_INTERFACE_UP for eth0
Network initialization completed successfully!
Initial DNS resolution complete!

Interface callback: FNS_IF_EVENT_ADDRESS_READY for fe80::202:99ff:fe16:5123.
SNMP Research SNMP Agent Resident Module Version 15.3.1.15
Copyright 1989, 1990, 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002 SNMP

Research, Inc.

Web Server Ready!

SNTP task started!

Interface callback: FNS_IF_EVENT_ADDRESS_READY for 202.78.x.x
DhcpCallback: DHCP_LEASE_IS_BOUND, IP 202.78.x.x
Received DHCPv4 DNS Address: 8.8.8.8
IPv4 DHCP Address returned

IP Address: 202.78.x.x
Netmask: 255.255.255.224
Gateway: 202.78.y.y
Broadcast Address: 0.0.0.0
DHCP/BootP Server: 202.78.y.y

IPv4 DNS Server: "8.8.8.8"

Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230

System Information Menu
------------------------
1: Name           Uninitialized
2: Contact        Uninitialized
3: Location       Uninitialized
4: Description    Uninitialized


<ESC>: Cancel menu level

Please select a key ?> 1

Enter name (Max 255 chars) ?> ups-test


Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230
GO TO MAIN MENU AND DO 'EXIT AND SAVE' TO SAVE YOUR CHANGES!

System Information Menu
------------------------
1: Name           ups-test
2: Contact        Uninitialized
3: Location       Uninitialized
4: Description    Uninitialized


<ESC>: Cancel menu level

Please select a key ?> 3

Enter location (Max 64 chars) ?> Warehouse


Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230
GO TO MAIN MENU AND DO 'EXIT AND SAVE' TO SAVE YOUR CHANGES!

System Information Menu
------------------------
1: Name           ups-test
2: Contact        Uninitialized
3: Location       Warehouse
4: Description    Uninitialized


<ESC>: Cancel menu level


Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230

IP Network Settings Menu
-------------------------
1: Boot/IP Settings
2: Domain Name Server (DNS) Settings
3: Management Protocol
4: Web Server
5: Telnet Server
6: Time (SNTP)
7: Change Administrator User Name/Password
8: Change General User Name/Password
9: Reset WEB Authentication to Factory Defaults

<ESC>: Cancel menu level

Please select a key ?> 7

Enter Administrator User Name, press enter for [Liebert]: (Max 32 chars) ?> admin

Enter New Password: (Max 32 chars) ?> ******

Verify Password: ?> ******
New Settings will take effect when saved


Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230
GO TO MAIN MENU AND DO 'EXIT AND SAVE' TO SAVE YOUR CHANGES!

Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230

IP Network Settings Menu
-------------------------
1: Boot/IP Settings
2: Domain Name Server (DNS) Settings
3: Management Protocol
4: Web Server
5: Telnet Server
6: Time (SNTP)
7: Change Administrator User Name/Password
8: Change General User Name/Password
9: Reset WEB Authentication to Factory Defaults

<ESC>: Cancel menu level

Please select a key ?> 1


GXT4-3000RT230

Boot/IP Settings Menu
----------------------
1: Speed/Duplex       Auto
2: IPv4 Settings
3: IPv6 Settings
4: Ping Settings


<ESC>: Cancel menu level

Please select a key ?> 2

Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230

IPv4 Settings Menu
-------------------
1: IPv4 Mode          DHCP
   IP Address         202.78.x.x
   Netmask            255.255.255.224
   Gateway            202.78.y.y


<ESC>: Cancel menu level

Please select a key ?> 1
 Valid Selections:
 ------------------
 1.  Static
 2.  BootP
 3.  DHCP
 4.  Unconfigured
Select IPv4 Mode: ( <Esc> - Cancel) ?> 1

<ESC>: Cancel menu level


Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230
GO TO MAIN MENU AND DO 'EXIT AND SAVE' TO SAVE YOUR CHANGES!

IPv4 Settings Menu
-------------------
1: IPv4 Mode          Static
2: IP Address         0.0.0.0
3: Netmask            0.0.0.0
4: Gateway            0.0.0.0


<ESC>: Cancel menu level

Please select a key ?> 2
Enter IP address [xxx.xxx.xxx.xxx] ?> 202.78.x.x


Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230
GO TO MAIN MENU AND DO 'EXIT AND SAVE' TO SAVE YOUR CHANGES!

IPv4 Settings Menu
-------------------
1: IPv4 Mode          Static
2: IP Address         202.78.x.x
3: Netmask            0.0.0.0
4: Gateway            0.0.0.0


<ESC>: Cancel menu level

Please select a key ?> 3
Enter Netmask address [xxx.xxx.xxx.xxx] ?> 255.255.255.224


Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230
GO TO MAIN MENU AND DO 'EXIT AND SAVE' TO SAVE YOUR CHANGES!

IPv4 Settings Menu
-------------------
1: IPv4 Mode          Static
2: IP Address         202.78.30.21
3: Netmask            255.255.255.224
4: Gateway            0.0.0.0


<ESC>: Cancel menu level

Please select a key ?> 4
Enter Default Gateway[xxx.xxx.xxx.xxx] ?> 202.78.x.x


Emerson Network Power IntelliSlot Web Card

GXT4-3000RT230

Main Menu
----------
1: System Information
2: IP Network Settings
3: Messaging
4: Factory Settings
5: Firmware Updates

q: Quit and abort changes
x: Exit and save


Please select a key ?> x
Exiting and saving...


Configuration saved successfully

Cisco Catalyst 3650 IOS Recovery via USB

I was preparing a new Cisco Catalyst 3650 switch and it's my first time to get a switch: prompt from a newly opened switch. I tried to initialize the it's flash but it didn't work. The IOS recovery via USB on a Cisco 3650 switch is very similar to a Cisco 3850. Below is the procedure that I've performed.


Booting...Initializing RAM +++++++@@@@@@@@...++++++++++++++++++++++++++++++++@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@done.
Memory Test Pass!

Base ethernet MAC Address: f4:0f:1b:84:f1:23

Interface GE 0 link down***ERROR: PHY link is down
Initializing Flash...

flashfs[7]: 0 files, 1 directories
flashfs[7]: 0 orphaned files, 0 orphaned directories
flashfs[7]: Total bytes: 6784000
flashfs[7]: Bytes used: 1024
flashfs[7]: Bytes available: 6782976
flashfs[7]: flashfs fsck took 1 seconds....done Initializing Flash.
flash:packages.conf: no such file or directory
flash:packages.conf: no such file or directory

Error loading "flash:packages.conf"

Interrupt within 5 seconds to abort boot process.
Boot process failed...

The system is unable to boot automatically. The
BOOT environment variable needs to be set to a
bootable image.


switch: flash_init
Initializing Flash...
...The flash is already initialized.

switch: load_helper
Unknown cmd: load_helper

switch: dir flash:
Directory of flash:/

30977  drwx  4096       .
    2  drwx  4096       ..
30978  -rw-  79121160   cat3k_caa-base.SPA.03.03.03SE.pkg
30979  -rw-  6484668    cat3k_caa-drivers.SPA.03.03.03SE.pkg
30980  -rw-  34522096   cat3k_caa-infra.SPA.03.03.03SE.pkg
30981  -rw-  34821452   cat3k_caa-iosd-universalk9.SPA.150-1.EZ3.pkg
30982  -rw-  25131920   cat3k_caa-platform.SPA.03.03.03SE.pkg
30983  -rw-  77312832   cat3k_caa-wcm.SPA.10.1.130.0.pkg

1361833984 bytes available (286060544 bytes used)

switch: boot flash:cat3k_caa-iosd-universalk9.SPA.150-1.EZ3.pkg
flash:cat3k_caa-iosd-universalk9.SPA.150-1.EZ3.pkg: bad/non-bootable Nova bundle file

Error loading "flash:cat3k_caa-iosd-universalk9.SPA.150-1.EZ3.pkg"

Interrupt within 5 seconds to abort boot process.
Boot process failed...

switch: dir usbflash0:
Directory of usbflash0:/

    2  -rw-  62682268   c2900-universalk9-mz.SPA.150-1.M4.bin
    3  -rw-  21890692   c870-advipservicesk9-mz.124-24.T4.bin
    4  -rw-  4968160    TeamViewerQS_en-idch93gk2g.exe
    5  -rw-  310347344  cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin
    6  -rw-  125231421  lms5.1.bin
    7  -rw-  257650008  cat3k_caa-universalk9ldpe.SPA.03.03.05.SE.150-1.EZ5.bin
    8  -rw-  38172672   asa916-4-smp-k8.bin

1200291840 bytes available (821035008 bytes used)

switch: boot usbflash0:cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin    // BOOT IOS FROM USB
Reading full image into memory........................................................................................................................................
....................................................................................................................................................
...............done
Nova Bundle Image
--------------------------------------
Kernel Address    : 0x6258928c
Kernel Size       : 0x3f9602/4167170
Initramfs Address : 0x62982890
Initramfs Size    : 0xe04704/14698244
Compression Format: .mzip

Bootable image at @ ram:0x6258928c
Bootable image segment 0 address range [0x81100000, 0x820a0000] is in range [0x80180000, 0x90000000].
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
File "usbflash0:cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin" uncompressed and installed, entry point: 0x81672c70
Loading Linux kernel with entry point 0x81672c70 ...
Bootloader: Done loading app on core_mask: 0xf

### Launching Linux Kernel (flags = 0x5)

All packages are Digitally Signed
Starting System Services
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=4,mode=600,ptmxmode=000 0 0


FIPS(NGWC): Flash Key Check : Begin
FIPS(NGWC): Flash Key Check : End, Not Found, FIPS Mode Not Enabled


Front-end Microcode IMG MGR: found 1 microcode images for 1 device.
Image for front-end 0: /tmp/microcode_update/front_end/fe_type_8_1

Front-end Microcode IMG MGR: Preparing to program device microcode...
Front-end Microcode IMG MGR: Preparing to program device[0]...83664 bytes.
Front-end Microcode IMG MGR: Programming device 0...wRrsssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss

<OUTPUT TRUNCATED>

sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss!Front-end Microcode IMG MGR: Microcode programming complete for device 0.
Front-end Microcode IMG MGR: Microcode programming complete in 42 seconds


              Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706



Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.07.00.E RELEASE SOFTWARE (fc4)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Mon 08-Dec-14 00:20 by prod_rel_team

Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

FIPS: Flash Key Check : Begin
FIPS: Flash Key Check : End, Not Found, FIPS Mode Not Enabled

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C3650-24PS (MIPS) processor with 4194304K bytes of physical memory.
Processor board ID FDO1808Q123X
2048K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
250456K bytes of Crash Files at crashinfo:.
1609272K bytes of Flash at flash:.
1974239K bytes of USB Flash at usbflash0:.
0K bytes of  at webui:.

Base Ethernet MAC Address          : f4:0f:1b:84:f1:23
Motherboard Assembly Number        : 73-15128-05
Motherboard Serial Number          : FDO1810Z456
Model Revision Number              : A0
Motherboard Revision Number        : A0
Model Number                       : WS-C3650-24PS
System Serial Number               : FDO1808Q123X

The System is using a non-recommended Boot mode.
Not all features may be available in this boot mode.
Please check the product configuration guide.


         --- System Configuration Dialog ---

Enable secret warning
----------------------------------
In order to access the device manager, an enable secret is required
If you enter the initial configuration dialog, you will be prompted for the enable secret
If you choose not to enter the intial configuration dialog, or if you exit setup without setting the enable secret,
please set an enable secret using the following CLI in configuration mode-
enable secret 0 <cleartext password>
----------------------------------
Would you like to enter the initial configuration dialog? [yes/no]: no

Would you like to terminate autoinstall? [yes]:
Startup-config is ignored. So, HTTP Secure server configuration not done

Press RETURN to get started!

*Sep  3 02:18:18.201: %LINK-5-CHANGED: Interface Vlan1, changed state to administratively down
Switch>enable
*Sep  3 02:18:28.433: %PNP-6-HTTP_CONNECTING: PnP Discovery trying to connect to PnP server https://devicehelper.cisco.com/pnp/HELLO
Switch#copy usbflash0:cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin flash   // COPY IOS TO FLASH MEMORY
Destination filename [cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin]?
Copy in progress...CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC

<OUTPUT TRUNCATED>

CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
310347344 bytes copied in 51.920 secs (5977414 bytes/sec)
Switch#dir   
Directory of flash:/

30978  -rw-    79121160  May 28 2014 08:06:16 +00:00  cat3k_caa-base.SPA.03.03.03SE.pkg
30979  -rw-     6484668  May 28 2014 08:06:17 +00:00  cat3k_caa-drivers.SPA.03.03.03SE.pkg
30980  -rw-    34522096  May 28 2014 08:06:16 +00:00  cat3k_caa-infra.SPA.03.03.03SE.pkg
30981  -rw-    34821452  May 28 2014 08:06:17 +00:00  cat3k_caa-iosd-universalk9.SPA.150-1.EZ3.pkg
30982  -rw-    25131920  May 28 2014 08:06:16 +00:00  cat3k_caa-platform.SPA.03.03.03SE.pkg
30983  -rw-    77312832  May 28 2014 08:06:17 +00:00  cat3k_caa-wcm.SPA.10.1.130.0.pkg
46465  drwx        4096   Sep 3 2015 02:15:31 +00:00  virtual-instance
46467  drwx        4096   Sep 3 2015 02:16:13 +00:00  dc_profile_dir
30984  -rw-     2097152   Sep 3 2015 02:17:02 +00:00  nvram_config
46469  drwx        4096   Sep 3 2015 02:17:10 +00:00  wnweb_store
30985  -rw-   310347344   Sep 3 2015 02:19:02 +00:00  cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin

1621966848 bytes total (1048842240 bytes free)

Switch#configure terminal
Switch(config)#boot system switch all flash:cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin
Switch#end
Switch#show run | inc boot
boot-start-marker
boot system switch all flash:cat3k_caa-universalk9.SPA.03.07.00.E.152-3.E.bin
boot-end-marker
diagnostic bootup level minimal
Switch#reload
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]

*Sep  3 02:23:46.882: %SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload command.
*Sep  3 02:23:47.526: %STACKMGR-1-RELOAD_REQUEST: 1 stack-mgr:  Received reload request for all switches, reason Reload command
*Sep  3 02:23:47.527: %STACKMGR-1-RELOAD: 1 stack-mgr:  Reloading due to reason Reload command
*Sep  3 02:23:48.028: %IOSXE-3-PLATFORM: 1 process sysmgr: Reset/Reload requested by [stack-manager].
<Thu Sep  3 02:23:48 2015> Message from sysmgr: Reason Code:[3] Reset Reason:Reset/Reload requested by [stack-manager]. [Reload command]
umount: /proc/fs/nfsd: not mounted
Unmounting ng3k filesystems...
Warning! - some ng3k filesystems may not have unmounted cleanly...
Please stand by while rebooting the system...
Restarting system.

Booting...Initializing RAM +++++++@@@@@@@@...++++++++++++++++++++++++++++++++
Base ethernet MAC Address: f4:0f:1b:84:f1:23

 <OUTPUT TRUNCATED>

Switch>enable
*Sep  3 02:29:47.115: %SSH-5-DISABLED: SSH 1.99 has been disabled
*Sep  3 02:29:47.116: %SSH-5-ENABLED: SSH 1.99 has been enabled
Switch#show version | inc IOS
Cisco IOS Software, IOS-XE Software, Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 03.07.00.E RELEASE SOFTWARE (fc4)
Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
ROM: IOS-XE ROMMON

Renaming Files in Router Flash via the rename Command

One of our core Cisco 7206VXR router rebooted by itself a few days ago and according to show version output, there was a watchdog timer expired. I did some troubleshooting to dig more info and also raised a TAC case based on the show stacks output. The Cisco TAC engineer advised me to upgrade the IOS to c7200-spservicesk9-mz.124-24.T8.bin

CORE01#show version
Cisco IOS Software, 7200 Software (C7200-SPSERVICESK9-M), Version 12.4(24)T1, RELEASE SOFTWARE (fc3) Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Fri 19-Jun-09 20:27 by prod_rel_team

ROM: System Bootstrap, Version 12.3(4r)T3, RELEASE SOFTWARE (fc1)

CORE01 uptime is 3 days, 20 hours, 14 minutes
System returned to ROM by watchdog timer expired
System restarted at 12:16:05 UTC Mon Aug 17 2015
System image file is "disk2:c7200-spservicesk9-mz.124-24.T1.bin"

CORE01#show stacks ?
  <1-8192>  Process to show stack detail on
  |         Output modifiers
  <cr>

CORE01#show stacks
Minimum process stacks:
 Free/Size   Name
 8328/9000   EEM ED RF
59164/60000  script background loader
58916/60000  EEM Auto Registration Proc
 5396/6000   SASL MAIN
 2400/3000   allegro libretto init
10936/12000  Router Init
 4212/6000   Update prst
 3348/12000  Init
 4244/6000   DIB error message
 5192/6000   RADIUS INITCONFIG
 5236/6000   BGP Accepter
 3396/6000   BGP Open
 2280/3000   Rom Random Update Process
 2764/6000   BFD PP Process
11172/12000  BFD
 8632/12000  BFD events
33860/36000  TCP Command
 6640/12000  Virtual Exec
 3484/6000   TFTP Read Process

Interrupt level stacks:
Level    Called Unused/Size  Name
  1   345573575   4724/9000  Network interfaces
  2           0   9000/9000  DMA/Timer Interrupt
  3      146504   7884/9000  PA Management Int Handler
  4          65   8596/9000  Console Uart
  5           0   9000/9000  OIR/Error Interrupt
  7    83084011   8372/9000  NMI Interrupt Handler

Spurious interrupts: 388

System was restarted by watchdog timer expired

-----------------------------------------------------------------
   Possible software fault. Hardware replacement may not
   correct problem. Upon recurrence, please collect
   crashinfo, "show tech" and contact Cisco Technical Support.
-----------------------------------------------------------------


The TFTP process on a Cisco 7206VXR router is slightly different compared to other router platforms and the command to use is copy tftp disk2. I deleted the current IOS since the router flash isn't large enough and also pointed to the new IOS image. Hopefully, there's no auto reboot again on this router and eventually use the new and stable copy of the IOS. After the TFTP process, I need to rename the IOS file name using the rename command in privileged mode.

CORE01#ping 172.27.25.164     // PING TO TFTP SERVER/PC

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.27.25.164, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 52/55/56 ms

CORE01#copy tftp://172.27.25.164/c7200-spservicesk9-mz.124-24.T8.bin disk2
Destination filename [disk2]?     // FOR SOME REASON THE FILE NAME WAS CHANGED TO disk2
Accessing tftp://172.27.25.164/c7200-spservicesk9-mz.124-24.T8.bin...
Loading c7200-spservicesk9-mz.124-24.T8.bin from 172.27.25.164 (via GigabitEthernet0/2): !
%Error copying tftp://172.27.25.164/c7200-spservicesk9-mz.124-24.T8.bin (Not enough space on device)
CORE01#delete disk2:c7200-spservicesk9-mz.124-24.T1.bin                     
Delete filename [c7200-spservicesk9-mz.124-24.T1.bin]?
Delete disk2:/c7200-spservicesk9-mz.124-24.T1.bin? [confirm]
CORE01#copy tftp://172.27.25.164/c7200-spservicesk9-mz.124-24.T8.bin disk2
Destination filename [disk2]?
Accessing tftp://172.27.25.164/c7200-spservicesk9-mz.124-24.T8.bin...
Loading c7200-spservicesk9-mz.124-24.T8.bin from 172.27.25.164 (via GigabitEthernet0/2): !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[OK - 36906144 bytes]

36906144 bytes copied in 4284.884 secs (8613 bytes/sec)

CORE01#show disk2
-#- --length-- -----date/time------ path
2     36906144 Aug 19 2015 02:49:30 disk2    // T8 IOS
3           83 Jun 26 2012 04:01:08 staging-running-config
CORE01#rename disk2: ?
  WORD  Destination file path

CORE01#rename disk2:disk2 c7200-spservicesk9-mz.124-24.T8.bin
CORE01(conf)#no boot system disk2:c7200-spservicesk9-mz.124-24.T1.bin
CORE01(conf)#boot system disk2:c7200-spservicesk9-mz.124-24.T8.bin

Cisco Aironet 1552 Lightweight Outdoor Mesh AP

I was able to test and setup a few Cisco 1552 lightweight outdoor mesh AP. It's controller-based, heavy and rugged AP. It could connect via wireless radio (hence mesh) to another AP that's in mesh going back to the wired LAN. The short and thick antenna is the 5 GHz and the long and thin antenna is the 2.4 GHz. The power injector is enclosed in a metal chassis and has two ports, one going to the switch and the other to the AP.

Here are the steps that I did for the Cisco 1552 APs to join the WLC and form a mesh wireless network:

1) I added the AP's MAC addresses under Security > AAA > TACACS+ > MAC Filtering.
You add the AP MAC address (usually ends with "c") and not the Base MAC address (usually ends with a "0"). Notice there are two MAC addresses when you do a show mac address-table interface x on a switch.

Switch#show mac address-table interface f0/11
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
 10    2c31.2473.1230    DYNAMIC     Fa0/11
 10    2c31.2473.123c    DYNAMIC     Fa0/11

2) Next is I chose the AP role: one AP as the root AP (RAP), which is the one connected to the wired LAN, and the rest were mesh AP (MAP). The AP started to join the WLC and got it's image file after choosing the AP role and set the country to US in the WLC under Wireless > Country. I've tried setting to different countries such as SG, AU and NZ but none of them seem to work.

APe089.9dff.abc0>
*Jul 27 14:47:41.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 172.27.196.25:5246sh
*Jul 27 14:47:42.035: %LWAPP-3-CLIENTERRORLOG: LWAPP LED Init: incorrect led state 255 ve
*Jul 27 14:47:52.095: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*Jul 27 14:47:52.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.27.196.25 peer_port: 5246
*Jul 27 14:47:52.535: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: 172.27.196.25 peer_port: 5246
*Jul 27 14:47:52.535: %CAPWAP-5-SENDJOIN: sending Join Request to 172.27.196.25
*Jul 27 14:47:52.535: %CAPWAP-3-ERRORLOG: Invalid event 10 & state 5 combination.
*Jul 27 14:47:52.535: %CAPWAP-3-ERRORLOG: CAPWAP SM handler: Failed to process message type 10 state 5.
*Jul 27 14:47:52.535: %CAPWAP-3-ERRORLOG: Failed to handle capwap control message from controller
*Jul 27 14:47:52.535: %CAPWAP-3-ERRORLOG: Failed to process encrypted capwap packet from 172.27.196.25
*Jul 27 14:47:57.535: %CAPWAP-5-SENDJOIN: sending Join Request to 172.27.196.25
*Jul 27 14:48:32.859: %MESH-3-TIMER_EXPIRED: Mesh Lwapp join timer expired
*Jul 27 14:48:32.859: %MESH-3-TIMER_EXPIRED: Mesh Lwapp join failed expired
*Jul 27 14:48:32.859: %MESH-6-LINK_UPDOWN: Mesh station e089.9dff.cxyz link Down
% CDP is not supported on this interface, or for this encapsulation
*Jul 27 14:48:51.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 172.27.196.25:5246
*Jul 27 14:48:52.035: %LWAPP-3-CLIENTERRORLOG: LWAPP LED Init: incorrect led state 255
*Jul 27 14:49:32.459: %MESH-6-CAPWAP_RESTART: Mesh Capwap re-started
% CDP is not supported on this interface, or for this encapsulation
*Jul 27 14:49:37.495: %LWAPP-3-CLIENTERRORLOG: LWAPP LED Init: incorrect led state 255
*Jul 27 14:49:47.551: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*Jul 27 14:49:48.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.27.196.25 peer_port: 5246
*Jul 27 14:49:48.535: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: 172.27.196.25 peer_port: 5246
*Jul 27 14:49:48.535: %CAPWAP-5-SENDJOIN: sending Join Request to 172.27.196.25
*Jul 27 14:49:48.535: %CAPWAP-3-ERRORLOG: Invalid event 10 & state 5 combination.
*Jul 27 14:49:48.535: %CAPWAP-3-ERRORLOG: CAPWAP SM handler: Failed to process message type 10 state 5.
*Jul 27 14:49:48.535: %CAPWAP-3-ERRORLOG: Failed to handle capwap control message from controller
*Jul 27 14:49:48.535: %CAPWAP-3-ERRORLOG: Failed to process encrypted capwap packet from 172.27.196.25
*Jul 27 14:49:53.535: %CAPWAP-5-SENDJOIN: sending Join Request to 172.27.196.25
% CDP is not supported on this interface, or for this encapsulation
*Jul 27 14:50:47.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 172.27.196.25:5246
*Jul 27 14:50:48.035: %LWAPP-3-CLIENTERRORLOG: LWAPP LED Init: incorrect led state 255
*Jul 27 14:50:58.095: %CAPWAP-3-ERRORLOG: Go join a capwap controller  
// AP 1552 JOINED WLC AFTER ADDING MAC ADDRESS TO MAC FILTERING
examining image...!
extracting info (285 bytes)  
Image info:
    Version Suffix: k9w8-.152-4.JB6
    Image Name: c1520-k9w8-mx.152-4.JB6
    Version Directory: c1520-k9w8-mx.152-4.JB6
    Ios Image Size: 123392
    Total Image Size: 8581632
    Image Feature: WIRELESS LAN|LWAPP
    Image Family: C1520
    Wireless Switch Management Version: 7.6.130.0
Extracting files...
c1520-k9w8-mx.152-4.JB6/ (directory) 0 (bytes)
extracting c1520-k9w8-mx.152-4.JB6/file_hashes (3118 bytes)
extracting c1520-k9w8-mx.152-4.JB6/8001.img (186860 bytes)!!!!!!
*Jul 27 14:50:58.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.27.196.25 peer_port: 5246
*Jul 27 14:50:58.531: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: 172.27.196.25 peer_port: 5246
*Jul 27 14:50:58.531: %CAPWAP-5-SENDJOIN: sending Join Request to 172.27.196.25
*Jul 27 14:50:58.535: %CAPWAP-3-ERRORLOG: Invalid event 10 & state 5 combination.
*Jul 27 14:50:58.535: %CAPWAP-3-ERRORLOG: CAPWAP SM handler: Failed to process message type 10 state 5.
*Jul !!!!!!!!
extracting c1520-k9w8-mx.152-4.JB6/final_hash.sig (513 bytes)
extracting c1520-k9w8-mx.152-4.JB6/c1520-k9w8-mx.152-4.JB6 (116954 bytes)!27 14:50:58.535: %CAPWAP-3-ERRORLOG: Failed to handle capwap control message from controller
*Jul 27 14:50:58.535: %CAPWAP-3-ERRORLOG: Failed to process encrypted capwap packet from 172.27.196.25perform archive download capwap:/c1520 tar file
*Jul 27 14:50:58.575: %CAPWAP-6-AP_IMG_DWNLD: Required image not found on AP. Downloading image from Controller.
*Jul 27 14:50:58.575: Loading file /c1520...
!!!!!!!!
extracting c1520-k9w8-mx.152-4.JB6/c1520_avr_3.img (14720 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/c1520_avr_2.img (10624 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/c1520_avr_7.img (33152 bytes)!!!
extracting c1520-k9w8-mx.152-4.JB6/8003.img (1043824 bytes)!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
extracting c1520-k9w8-mx.152-4.JB6/c1520_avr_6.img (14720 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/img_sign_rel_sha2.cert (1371 bytes)
extracting c1520-k9w8-mx.152-4.JB6/J2.bin (8888 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/info (285 bytes)
extracting c1520-k9w8-mx.152-4.JB6/I5.bin (20200 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/H4.bin (1212 bytes)
extracting c1520-k9w8-mx.152-4.JB6/H5.bin (34340 bytes)!!!
extracting c1520-k9w8-mx.152-4.JB6/c1520-k9w8-xx.152-4.JB6 (6954849 bytes)!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
*Jul 27 14:52:02.819: %MESH-3-TIMER_EXPIRED: Mesh Lwapp join timer expired!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
extracting c1520-k9w8-mx.152-4.JB6/I2.bin (2828 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/J5.bin (70296 bytes)!!!!!
extracting c1520-k9w8-mx.152-4.JB6/final_hash (141 bytes)
extracting c1520-k9w8-mx.152-4.JB6/H8.bin (2020 bytes)
extracting c1520-k9w8-mx.152-4.JB6/img_sign_rel.cert (1375 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/c1520_avr_5.img (6784 bytes)
extracting c1520-k9w8-mx.152-4.JB6/c1520_avr_1.img (10368 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/H2.bin (9696 bytes)!
extracting c1520-k9w8-mx.152-4.JB6/c1520_avr_4.img (14464 bytes)!
extracting info.ver (285 bytes)
Deleting current version: flash:/c1520-k9w8-mx.152-4.JA1...
Set booting path to recovery image: ''...done.
New software image installed in flash:/c1520-k9w8-mx.152-4.JB6
Writing out the event log to uflash:/event.log ...

Configuring system to use new image...done.
archive download: takes 91 seconds

*Jul 27 14:52:30.127: image upgrade successfully, system is now reloading
*Jul 27 14:52:30.159: %SYS-5-RELOAD: Reload requested by capwap image download proc. Reload Reason: NEW IMAGE DOWNLOAD.
*Jul 27 14:52:30.159: %LWAPP-5-CHANGED: CAPWAP changed state to DOWN

Write of event.log done

IOS Bootloader - Starting system.

Xmodem file system is available.

flashfs[0]: 32 files, 3 directories

flashfs[0]: 0 orphaned files, 0 orphaned directories

flashfs[0]: Total bytes: 31868928

flashfs[0]: Bytes used: 8655360

flashfs[0]: Bytes available: 23213568

flashfs[0]: flashfs fsck took 17 seconds.

Reading cookie from flash parameter block...done.

Base Ethernet MAC address: e0:89:9d:ff:cf:e0

Loading "flash:/c1520-k9w8-mx.152-4.JB6/c1520-k9w8-mx.152-4.JB6"...##############

File "flash:/c1520-k9w8-mx.152-4.JB6/c1520-k9w8-mx.152-4.JB6" uncompressed and installed, entry point: 0x3000

executing...

IOS Secondary Bootloader - Starting system.

Xmodem file system is available.

flashfs[0]: 32 files, 3 directories

flashfs[0]: 0 orphaned files, 0 orphaned directories

flashfs[0]: Total bytes: 31868928

flashfs[0]: Bytes used: 8655360

flashfs[0]: Bytes available: 23213568

flashfs[0]: flashfs fsck took 5 seconds.

Reading cookie from flash parameter block...done.

Base Ethernet MAC address: e0:89:9d:ff:ab:c0

Boot CMD: 'boot  flash:/c1520-k9w8-mx.152-4.JB6/c1520-k9w8-xx.152-4.JB6;flash:/c1520-k9w8-mx.152-4.JB6/c1520-k9w8-mx.152-4.JB6'

Loading "flash:/c1520-k9w8-mx.152-4.JB6/c1520-k9w8-xx.152-4.JB6"...####################

File "flash:/c1520-k9w8-mx.152-4.JB6/c1520-k9w8-xx.152-4.JB6" uncompressed and installed, entry point: 0x3000

executing...

              Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706

Cisco IOS Software, C1550 Software (C1520-K9W8-M), Version 15.2(4)JB6, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Fri 22-Aug-14 13:14 by prod_rel_team

Initializing flashfs...

flashfs[3]: 32 files, 3 directories
flashfs[3]: 0 orphaned files, 0 orphaned directories
flashfs[3]: Total bytes: 31610880
flashfs[3]: Bytes used: 8655360
flashfs[3]: Bytes available: 22955520
flashfs[3]: flashfs fsck took 4 seconds.
flashfs[3]: Initialization complete.
flashfs[4]: 0 files, 1 directories
flashfs[4]: 0 orphaned files, 0 orphaned directories
flashfs[4]: Total bytes: 5806080
flashfs[4]: Bytes used: 1024
flashfs[4]: Bytes available: 5805056
flashfs[4]: flashfs fsck took 0 seconds.
flashfs[4]: Initialization complete....done Initializing flashfs.

Radio0  present 8364B 8000 A8020000 0 A8030000 30
Rate table has 336 entries (20 legacy/64 11n/252 11ac)

POWER TABLE FILENAME = flash:/c1520-k9w8-mx.152-4.JB6/J2.bin

Radio1  present 8364B 8000 B8020000 0 B8030000 13
POWER TABLE FILENAME = flash:/c1520-k9w8-mx.152-4.JB6/J5.bin

Radio2 not present 0 0 0 0 0 11
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco AIR-CAP1552EU-A-K9 (PowerPC 8349) processor (revision A0) with 98294K/32768K bytes of memory.
Processor board ID FTX1905P07E
PowerPC 8349 CPU at 533MHz, revision number 0x0031
Last reset from power source change
LWAPP image version 7.6.130.0
4 Gigabit Ethernet interfaces
2 802.11 Radios

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: E0:89:9D:FF:AB:C0
Part Number                          : 73-13538-02
PCA Assembly Number                  : 800-31224-01
PCA Revision Number                  : 03
PCB Serial Number                    : FOC19028123
Top Assembly Part Number             : 800-38848-04
Top Assembly Serial Number           : FTX1905P456
Top Revision Number                  : A0
Product/Model Number                 : AIR-CAP1552EU-A-K9 
% Please define a domain-name first.

Press RETURN to get started!

*Mar  1 00:00:06.755: %SOAP_FIPS-2-SELF_TEST_IOS_SUCCESS: IOS crypto FIPS self test passed (11)
*Mar  1 00:00:07.515: %SOAP_FIPS-2-SELF_TEST_HW_SUCCESS: HW crypto FIPS self test passed (1-4)
*Mar  1 00:00:07.515: Registering HW DTLS
*Mar  1 00:00:07.979: m8349_ether_enable: MACCFG1 sync timeout
*Mar  1 00:00:09.763: %LINK-6-UPDOWN: Interface Ethernet4, changed state to up
*Mar  1 00:00:10.631: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 0 (4)
*Mar  1 00:00:10.639: %LINK-6-UPDOWN: Interface GigabitEthernet0, changed state to up
*Mar  1 00:00:10.639: %LINK-6-UPDOWN: Interface GigabitEthernet1, changed state to up


To check if all APs got registered look under Wireless > All APs and check for REG under Operational Status.

Here are some useful show commands on the WLC.

  

You'll see DTLS and SHA2 MIC certificate error logs when you console to the AP 1552. You'll need to accept (tick) the Manufactured Installed Certificate (MIC) on the WLC. Just go to SECURITY > AAA > AP Policies.

*Nov 13 08:18:49.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.27.20.6 peer_port: 5246
*Nov 13 08:18:49.323: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: 172.27.20.6 peer_port: 5246
*Nov 13 08:18:49.323: %CAPWAP-5-SENDJOIN: sending Join Request to 172.27.20.6
*Nov 13 08:18:49.327: %DTLS-5-ALERT: Received WARNING : Close notify alert from 172.27.20.6
*Nov 13 08:18:49.327: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 172.27.20.6:5246
*Nov 13 08:18:49.327: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.