MikroTik Router Basic Configuration

I was able to do basic configuration on a MikroTik router and allowed the LAN to go the Internet using NAT (Source NAT).  Below is the network topology and actual photos of the MikroTik Cloud Core Router used for this lab scenario.

To initially configure the MikroTik router, manually set your PC IP to 192.168.88.2/24 > connect an RJ45 cable between PC LAN port and MikroTik router's ETH (leftmost port).

Open a web browser > HTTP to 192.168.88.1 > type admin under login > leave Password blank > click Login.

 

Create a new user account under WebFig > Users > Add New.

Type the Name > choose a Group (full by default) > type and confirm Password > click Apply > OK.

 

You can delete the default admin account by clicking the dash or minus icon ( - ) which is beside D (disable).

I’m using a cable modem at home which provides a public IP address via its Ethernet port. To configure the MikroTik router's ISP/WAN interface, go to IP > DHCP Client > Add New.

 

Under Interface > choose ether1 > Apply > OK. Notice the ISP public IP address appeared.

 

Take note of the default Add Default Route of yes.

To configure the LAN IP address, go to IP > Addresses > Add New.

 

Type the LAN Address 192.168.1.1/24 (default gateway) > type 192.168.1.0 under Network > choose ether2 under Interface > click Apply > OK.

 

To configure a DHCP Server for the LAN, go to IP > DHCP Server > Add New.

 

Type a Name > choose the LAN Interface > set a Lease Time (HH:MM:SS) > click Apply > OK.

Choose dynamic under Bootp Support.

 

Click DHCP Setup > choose ether2 under DHCP Server Interface > Next.

 

Leave the default DHCP Address Space (192.168.1.0/24) > click Next.

 

Leave the default (192.168.1.1) for Gateway for DHCP Network.

 

Leave the default for Address to Give Out (192.168.1.2-192.168.1.254).

 

Type a Primary (optional Secondary) DNS Servers. It auto filled the ISP DNS Servers if WAN is a DHCP Client.

 

Optionally change the default lease time (10 mins) to 8 hours.

 

Click Leases tab to see DHCP clients (I had a PC connected on ether2)

 

You can do an IP Scan on selected Interface (ether2) under Tools > IP Scan. Notice the MikroTik router was able to fingerprint the host NetBIOS (MACBOOKPRO).

You can also view DHCP leased addresses under IP > Pool > Used Addresses.

To configure a default route, go to under IP > Routes. Notice the ISP DHCP Client automatically configured a default route 0.0.0.0/0 since we selected yes under Add Default Route.

Click on the 0.0.0.0/0 route entry to view more details.

You need to configure NAT in order to allow the private IP address (192.168.1.0/24) to reach the public Internet. To configure NAT (Source NAT), go to IP > Firewall > NAT > Add New.

 

Leave the default srcnat under Chain > select Out.Interface ether1 (ISP)

 

You can view interface status and statistics under Interfaces.

Click a specific interface (ether1) to view more details.

You can perform troubleshooting or diagnostics, go to Tools > Ping > type an IP address (8.8.8.8) under Ping To > click Start.
 

You can also do a Traceroute which perform like an MTR.

You can view NAT translations under IP > Firewall > Connections.

Click on a specific output (line 2) in order to view more details.

To view chassis information, go to System > Health.

You can also view more chassis environment info under System > Resources.

To view Syslogs to to Log.

You can also do Packet Sniffer or capture under Tools > Packet Sniffer > Start.

Click Stop > then click Packets.

Click on a specific output or line (line 2) to view more details.

You can do real-time traffic monitoring on a specific interface under Torch > select an Interface (ether2) > type Src Address (Source Address) > type Dst Address (Destination Address) > click Start (then click Stop).

You can do a factory reset under System > Reset Configuration.

You can do a remote reboot  under System >Reboot.

You can do a quick network setup under Quick Set found on the upper right hand corner of the web GUI.

You can launch the CLI terminal (inband) under Terminal.

You can connect (out-of-band) a console (rollover) cable to the MikroTik CONSOLE port. Set the baud rate to 115200 (8-N-1-N).

Below are some useful CLI show commands.


[admin@MikroTik] > system resource print
             uptime: 2h36s
            version: 6.39.2 (stable)
         build-time: Jun/06/2017 08:01:04
   factory-software: 6.36.4
        free-memory: 1742.1MiB
       total-memory: 1956.2MiB
                cpu: tilegx
          cpu-count: 9
      cpu-frequency: 1200MHz
           cpu-load: 0%
     free-hdd-space: 82.1MiB
    total-hdd-space: 128.0MiB
  architecture-name: tile
         board-name: CCR1009-7G-1C-1S+
           platform: MikroTik


[admin@MikroTik] > interface print
Flags: D - dynamic, X - disabled, R - running, S - slave
 #     NAME                                TYPE       ACTUAL-MTU L2MTU  MAX-L2MTU MAC-ADDRESS     
 0  R  combo1                              ether            1500  1580      10222 6C:3B:6B:E3:C5:21
 1  R  ether1                              ether            1500  1580      10222 6C:3B:6B:E3:C5:22
 2  R  ether2                              ether            1500  1580      10222 6C:3B:6B:E3:C5:23
 3     ether3                              ether            1500  1580      10222 6C:3B:6B:E3:C5:24
 4     ether4                              ether            1500  1580      10222 6C:3B:6B:E3:C5:25
 5     ether5                              ether            1500  1580      10222 6C:3B:6B:E3:C5:26
 6     ether6                              ether            1500  1580      10222 6C:3B:6B:E3:C5:27
 7     ether7                              ether            1500  1580      10222 6C:3B:6B:E3:C5:28
 8     sfp-sfpplus1                        ether            1500  1580      10222 6C:3B:6B:E3:C5:20


[admin@MikroTik] > interface ethernet cable-test ether1
    name: ether1
  status: link-ok


[admin@MikroTik] > ip route print
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADS  0.0.0.0/0                          222.165.112.1             1
 1 ADC  192.168.1.0/24     192.168.1.1     ether2                    0
 2 ADC  192.168.88.0/24    192.168.88.1    combo1                    0
 3 ADC  222.165.x.0/21   222.165.x.x9 ether1                    0


[admin@MikroTik] > ping 8.8.8.8
  SEQ HOST                                     SIZE TTL TIME  STATUS                                                                                                                                                                     
    0 8.8.8.8                                    56  55 8ms 
    1 8.8.8.8                                    56  55 7ms 
    2 8.8.8.8                                    56  55 7ms 
    3 8.8.8.8                                    56  55 8ms 
    4 8.8.8.8                                    56  55 8ms 
    5 8.8.8.8                                    56  55 7ms 
    6 8.8.8.8                                    56  55 7ms 
    sent=7 received=7 packet-loss=0% min-rtt=7ms avg-rtt=7ms max-rtt=8ms


[admin@MikroTik] > tool traceroute 8.8.8.8
 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS                                                                                                                                             
 1 10.47.0.1                          0%    7     8ms     7.6     4.6    11.3  2.2                                                                                                                                                    
 2 10.47.0.1                        16..    7   8.1ms     8.2     6.8    10.3     1.2                                                                                                                                                    
 3 172.20.43.65                       0%    6   7.2ms     7.1     6.3     7.6     0.5                                                                                                                                                    
 4 172.20.9.230                       0%    6   9.4ms     9.5     9.4     9.8     0.1                                                                                                                                                    
 5 203.116.188.85                     0%    6  26.5ms    28.3     8.2    41.5    11.3                                                                                                                                                    
 6 203.117.36.21                      0%    6   9.4ms    11.2       6    23.4     6.2                                                                                                                                                    
 7 203.116.189.181                    0%    6   8.1ms    12.9       8    22.5     6.8                                                                                                                                                    
 8 203.117.34.34                      0%    6   7.6ms    18.8     7.6    44.8    13.3                                                                                                                                                    
 9 72.14.196.189                      0%    6   8.5ms     7.5     5.1     8.9     1.4                                                                                                                                                    
10 108.170.242.65                     0%    6   7.7ms     8.4     7.7     8.7     0.3                                                                                                                                                    
11 108.170.237.229                    0%    6   8.1ms    13.2     7.4    27.7       8                                                                                                                                                    
12 8.8.8.8                            0%    6   7.2ms    12.5     7.2    22.5     7.1
                                                                                                                                 
[admin@MikroTik] > /log print
jan/01/1970 00:00:08 system,error,critical router rebooted without proper shutdown, probably power outage
jan/02/1970 00:00:10 script,info Starting_defconf_script_
jan/02/1970 00:00:10 script,info Defconf_script_finished
jan/02/1970 00:00:10 system,info address added
jan/02/1970 00:02:53 interface,info ether2 link up (speed 1G, full duplex)
jan/02/1970 00:07:58 interface,info ether2 link down
jan/02/1970 00:08:02 interface,info ether2 link up (speed 1G, full duplex)
jan/02/1970 00:08:32 interface,info ether2 link down
jan/02/1970 00:08:36 interface,info ether2 link up (speed 1G, full duplex)
jan/02/1970 00:09:06 interface,info ether2 link down
jan/02/1970 00:09:10 interface,info ether1 link up (speed 1G, full duplex)
jan/02/1970 00:10:21 interface,info ether1 link down
jan/02/1970 00:10:27 interface,info combo1 link up (speed 1G, full duplex)
jan/02/1970 00:13:13 system,error,critical login failure for user admin from 192.168.88.2 via ftp
jan/02/1970 00:13:15 system,info,account user admin logged in from 192.168.88.2 via web
jan/02/1970 00:13:16 system,info,account user admin logged in from 192.168.88.2 via ftp
jan/02/1970 00:13:17 system,info,account user admin logged out from 192.168.88.2 via ftp
jan/02/1970 00:14:39 system,info,account user admin logged in via local
jan/02/1970 00:19:15 system,info,account user admin logged out from 192.168.88.2 via web
jan/02/1970 00:19:15 system,info,account user admin logged out via local
jan/02/1970 00:19:27 system,info,account user admin logged in from 192.168.88.2 via web
jan/02/1970 00:19:27 system,info,account user admin logged in via local
jan/02/1970 00:19:51 system,info,account user admin logged in via local
jan/02/1970 00:20:00 system,info,account user admin logged out via local
jan/02/1970 00:20:01 system,info,account user admin logged in via local
jan/02/1970 00:20:04 system,info,account user admin logged out from 192.168.88.2 via web
jan/02/1970 00:20:04 system,info,account user admin logged out via local
jan/02/1970 00:20:04 system,info,account user admin logged out via local
jan/02/1970 00:20:13 system,info,account user admin logged in from 192.168.88.2 via web
jan/02/1970 00:20:14 system,info,account user admin logged in via local
jan/02/1970 00:20:20 system,info,account user admin logged out from 192.168.88.2 via web
jan/02/1970 00:20:20 system,info,account user admin logged out via local
jan/02/1970 00:20:45 system,info,account user admin logged in from 192.168.88.2 via web
jan/02/1970 00:20:47 system,info,account user admin logged in via local
jan/02/1970 00:20:48 system,info,account user admin logged out via local
jan/02/1970 00:25:00 system,info,account user admin logged in via local
jan/02/1970 00:30:00 system,info,account user admin logged out from 192.168.88.2 via web
jan/02/1970 00:31:01 system,info,account user admin logged in from 192.168.88.2 via web
jan/02/1970 00:33:26 system,info,account user admin logged out from 192.168.88.2 via web
jan/02/1970 00:33:28 system,info,account user admin logged in from 192.168.88.2 via web

<OUTPUT TRUNCATED>


[admin@MikroTik] > /system health print
                    fan-mode: auto
                     use-fan: main
                  active-fan: main
          cpu-overtemp-check: yes
      cpu-overtemp-threshold: 100C
  cpu-overtemp-startup-delay: 1m
                     voltage: 24.1V
                     current: 738mA
                 temperature: 34C
             cpu-temperature: 52C
           power-consumption: 17.8W
                  psu1-state: ok
                  psu2-state: fail
                  fan1-speed: 6466RPM


[admin@MikroTik] > /export
# jul/11/2017 22:36:28 by RouterOS 6.39.2
# software id = 1E6H-GITN
#
/ip pool
add name=dhcp_pool0 ranges=192.168.1.2-192.168.1.254
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=ether2 lease-time=8h name=\
    dhcp1
/ip address
add address=192.168.88.1/24 comment=defconf interface=combo1 network=\
    192.168.88.0
add address=192.168.1.1/24 interface=ether2 network=192.168.1.0
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1
/ip dhcp-server network
add address=192.168.1.0/24 dns-server=8.8.8.8,4.2.2.2 gateway=192.168.1.1
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1
/system clock
set time-zone-name=Asia/Singapore


[admin@MikroTik] > system reset-configuration
Dangerous! Reset anyway? [y/N]:

Upgrading Cisco Nexus Switch and Configuring Virtual PortChannel (vPC)

I had to upgrade a new Cisco Nexus 3K switch. Upgrading the NX-OS is not a prerequisite for vPC. Here's a nice Cisco Nexus guide which I used to quickly get started and a link for the best practice configuration for the NX-OS STP extensions or the spanning-tree port type. There's a nice diagram shared in the Cisco forum showing the best practice where to configure them.

The Cisco Nexus switch platform supports In-Service Software Upgrade (ISSU) but I don't need it since the switch isn't in production yet. I used a compatible USB stick to copy both the kickstart (the Linux Kernel) and NX-OS image. The USB slot is found at the back of the chassis and it's below the CONSOLE port.


switch %$ VDC-1 %$ %IDEHSD-2-MOUNT: USB1: online  // DETECTED A COMPATIBLE USB
switch# dir usb1:

<OUTPUT TRUNCATED>

  32782848     Mar 23 09:18:28 2018  n3000-uk9-kickstart.6.0.2.U2.5.bin
  173920429    Mar 23 09:19:36 2018  n3000-uk9.6.0.2.U2.5.bin

Usage for usb1://sup-local
 1925578752 bytes used
  129105920 bytes free
 2054684672 bytes total

switch# copy usb1: bootflash:
Enter source filename: n3000-uk9-kickstart.6.0.2.U2.5.bin
Copy progress 100% 32782KB

<OUTPUT TRUNCATED>

Copy complete, now saving to disk (please wait)...

switch# copy usb1: bootflash:
Enter source filename: n3000-uk9.6.0.2.U2.5.bin
Copy progress 100% 32782KB

<OUTPUT TRUNCATED>

Copy complete, now saving to disk (please wait)...

switch# dir bootflash:

         84    Feb 11 13:26:58 2018  20180211_132618_poap_3828_init.log
         84    Mar 05 10:52:44 2018  20180305_105203_poap_3831_init.log
        168    Mar 05 10:59:51 2018  20180305_105751_poap_3842_init.log
        168    Mar 19 08:45:17 2018  20180319_083732_poap_3829_init.log
         84    Mar 19 09:01:57 2018  20180319_090050_poap_3829_init.log
       4096    May 28 08:38:57 2016  lost+found/
   32782848    Mar 23 07:46:33 2018  n3000-uk9-kickstart.6.0.2.U2.5.bin
   37791232    May 28 08:23:01 2016  n3000-uk9-kickstart.6.0.2.U4.1.bin
  173920429    Mar 23 07:46:59 2018  n3000-uk9.6.0.2.U2.5.bin
  185119072    May 28 08:23:33 2016  n3000-uk9.6.0.2.U4.1.bin
       4096    Jan 29 08:26:13 2010  vdc_2/
       4096    Jan 29 08:26:13 2010  vdc_3/
       4096    Jan 29 08:26:13 2010  vdc_4/
       4096    Jan 29 08:26:16 2010  virtual-instance-stby-sync/
        568    Mar 21 08:15:24 2018  vlan.dat

Usage for bootflash://sup-local
  560041984 bytes used
 1335128064 bytes free
 1895170048 bytes total


I performed a Fast Reload Upgrade on the Cisco Nexus 3K switches using the install all command.

switch# install ?
  all      Upgrade the system
  license  Install license

switch# install all ?
  <CR>           
  force           Force Disruptive upgrade
  kickstart       Boot-variable name
  non-disruptive  Non-Disruptive image upgrade
  ssi             Boot-variable name
  system          Boot-variable name

switch# install all kickstart ?
  bootflash:  Enter image uri
  ftp:        Enter image uri
  modflash:   Enter image uri
  scp:        Enter image uri
  sftp:       Enter image uri
  tftp:       Enter image uri
  usb1:       Enter image uri
  volatile:   Enter image uri

switch# install all kickstart bootflash:?
  bootflash:///           
  bootflash://module-1/   
  bootflash://sup-1/      
  bootflash://sup-active/ 
  bootflash://sup-local/  

switch# install all kickstart bootflash:///?
  bootflash:///20100129_154217_poap_3932_init.log 
  bootflash:///20160719_173151_poap_3840_init.log 
  bootflash:///20180211_135021_poap_3842_init.log 
  bootflash:///20180319_093045_poap_3840_init.log 
  bootflash:///license_FOC2026R31P_24_1.lic       
  bootflash:///lost+found                         
  bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin 
  bootflash:///n3000-uk9-kickstart.6.0.2.U3.7.bin 
  bootflash:///n3000-uk9.6.0.2.U2.5.bin           
  bootflash:///n3000-uk9.6.0.2.U3.7.bin           
  bootflash:///vdc_2                              
  bootflash:///vdc_3                              
  bootflash:///vdc_4                              
  bootflash:///vlan.dat                           

switch# install all kickstart bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin  ?
  <CR>           
  force           Force Disruptive upgrade
  non-disruptive  Non-Disruptive image upgrade
  ssi             Boot-variable name
  system          Boot-variable name

switch# install all kickstart bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin system ?
  bootflash:  Enter image uri
  ftp:        Enter image uri
  modflash:   Enter image uri
  scp:        Enter image uri
  sftp:       Enter image uri
  tftp:       Enter image uri
  usb1:       Enter image uri
  volatile:   Enter image uri

switch# install all kickstart bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin system bootflash:?
  bootflash:///           
  bootflash://module-1/   
  bootflash://sup-1/      
  bootflash://sup-active/ 
  bootflash://sup-local/  

switch# install all kickstart bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin system bootflash:///?
  bootflash:///20100129_154217_poap_3932_init.log 
  bootflash:///20160719_173151_poap_3840_init.log 
  bootflash:///20180211_135021_poap_3842_init.log 
  bootflash:///20180319_093045_poap_3840_init.log 
  bootflash:///license_FOC2026R31P_24_1.lic       
  bootflash:///lost+found                         
  bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin 
  bootflash:///n3000-uk9-kickstart.6.0.2.U3.7.bin 
  bootflash:///n3000-uk9.6.0.2.U2.5.bin           
  bootflash:///n3000-uk9.6.0.2.U3.7.bin           
  bootflash:///vdc_2                              
  bootflash:///vdc_3                              
  bootflash:///vdc_4                              
  bootflash:///vlan.dat                           

switch# install all kickstart bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin system bootflash:///n3000-uk9.6.0.2.U2.5.bin ?
  <CR>           
  force           Force Disruptive upgrade
  non-disruptive  Non-Disruptive image upgrade
  ssi             Boot-variable name

switch# install all kickstart bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin system bootflash:///n3000-uk9.6.0.2.U2.5.bin

Installer is forced disruptive

Verifying image bootflash:/n3000-uk9-kickstart.6.0.2.U2.5.bin for boot variable "kickstart".
[########################################] 100% -- SUCCESS

Verifying image bootflash:/n3000-uk9.6.0.2.U2.5.bin for boot variable "system".
[########################################] 100% -- SUCCESS

Verifying image type.
[########################################] 100% -- SUCCESS

Extracting "system" version from image bootflash:/n3000-uk9.6.0.2.U2.5.bin.
[########################################] 100% -- SUCCESS

Extracting "kickstart" version from image bootflash:/n3000-uk9-kickstart.6.0.2.U2.5.bin.
[########################################] 100% -- SUCCESS

Extracting "bios" version from image bootflash:/n3000-uk9.6.0.2.U2.5.bin.
[########################################] 100% -- SUCCESS

Performing module support checks.
[########################################] 100% -- SUCCESS

Notifying services about system upgrade.
[########################################] 100% -- SUCCESS

Compatibility check is done:
Module  bootable          Impact  Install-type  Reason
------  --------  --------------  ------------  ------
     1       yes      disruptive         reset  Forced by the user

Images will be upgraded according to following table:
Module             Image         Running-Version             New-Version  Upg-Required
------  ----------------  ----------------------  ----------------------  ------------
     1            system             6.0(2)U3(7)             6.0(2)U2(5)           yes
     1         kickstart             6.0(2)U3(7)             6.0(2)U2(5)           yes
     1              bios      v1.4.0(12/09/2013)      v1.2.0(08/25/2011)            no
     1         power-seq                    v4.4                    v4.4            no

Additional info for this installation:
--------------------------------------

Service "vpc" : vPC feature is enabled. Upgrade will be disruptive!!!

Switch will be reloaded for disruptive upgrade.

Do you want to continue with the installation (y/n)?  [n] y

Time Stamp: Fri Mar 23 07:48:16 2018


Install is in progress, please wait.

Performing runtime checks.
[########################################] 100% -- SUCCESS

Setting boot variables.
[########################################] 100% -- SUCCESS

Performing configuration copy.
[########################################] 100% -- SUCCESS

Converting startup config.
[########################################] 100% -- SUCCESS
Time Stamp: Fri Mar 23 07:51:16 2018


Finishing the upgrade, switch will reboot in 10 seconds.
switch# [89720.651353]  writing reset reason 49,


(c) Copyright 2011, Cisco Systems.

N3000 BIOS v.1.4.0, Mon 12/09/2013, 11:26 AM

GRUB Loading stage2 

Image verification OK

Fastboot Memory at 0c100000 of size 201326592
Usage: init 0123POST INIT Starts at Fri Mar 23 07:52:04 UTC 2018
Starting Nexus 3000 Platform POST.....
  Executing Mod 1 1 SEEPROM Test:...done (0 seconds)
  Executing Mod 1 1 GigE Port Test:.done (8 seconds)
  Executing Mod 1 1 PCIE Test:.................done (0 seconds)
  Mod 1 1 Post Completed Successfully
POST is completed
Reconfiguring links to dev files.
ERROR: Internal disk has unrecoverable error (1); please do "init system"
.r.r.r. done.
Loading System Software Fri Mar 23 07:52:23 UTC 2018

System Software(/bootflash/n3000-uk9.6.0.2.U2.5.bin) Loaded Fri Mar 23 07:52:41 UTC 2018
ethernet switching mode

INIT: Entering runlevel: 3

Mounting other filesystems:  [  OK  ]

Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
Added VLAN with VID == 4042 to IF -:muxif:-
2018 Mar 23 07:52:47 switch %$ VDC-1 %$ %USER-0-SYSTEM_MSG: FAST REBOOT DISABLED - bcm_usd
2018 Mar 23 07:52:48 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: loading cmd files begin  - clis
2018 Mar 23 07:53:00 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: loading cmd files end  - clis
2018 Mar 23 07:53:00 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: init begin  - clis
2018 Mar 23 07:53:03 switch %$ VDC-1 %$ %IDEHSD-2-MOUNT: USB1: online
2018 Mar 23 07:53:30 switch %$ VDC-1 %$ %USER-0-SYSTEM_MSG: Starting bcm_attach - bcm_usd
2018 Mar 23 07:53:34 switch %$ VDC-1 %$ %USER-0-SYSTEM_MSG: Finished bcm_attach... - bcm_usd
2018 Mar 23 07:53:46 switch %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 1 has come online


The NX-OS upgrade and bootup took around 5 minutes.

switch login:<USER>
Password:<PASSWORD>
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2014, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php

switch# show version

Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Documents: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_serie
s_home.html
Copyright (c) 2002-2014, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.

Software
  BIOS:      version 1.4.0
  loader:    version N/A
  kickstart: version 6.0(2)U2(5)
  system:    version 6.0(2)U2(5)
  Power Sequencer Firmware:
             Module 1: version v4.4
  BIOS compile time:       12/09/2013
  kickstart image file is: bootflash:///n3000-uk9-kickstart.6.0.2.U2.5.bin
  kickstart compile time:  5/8/2014 16:00:00 [05/09/2014 00:38:26]
  system image file is:    bootflash:///n3000-uk9.6.0.2.U2.5.bin
  system compile time:     5/8/2014 16:00:00 [05/09/2014 02:30:06]


Hardware
  cisco Nexus 3048 Chassis ("48x1GE + 4x10G Supervisor")
  Intel(R) Celeron(R) CPU        P450 with 3785120 kB of memory.
  Processor Board ID FOC20261234

  Device name: switch
  bootflash:    2007040 kB

Kernel uptime is 0 day(s), 0 hour(s), 2 minute(s), 16 second(s)

Last reset at 651345 usecs after  Fri Mar 23 07:51:26 2018

  Reason: Disruptive upgrade
  System version: 6.0(2)U3(7)
  Service:

plugin
  Core Plugin, Ethernet Plugin



After the NX-OS upgrade, I've configured a virtual PortChannel (vPC) which is a Nexus virtualization feature that allows a downstream device (server, router, switch, firewall, etc) to have separate uplinks appear as a single logical interface (port-channel). I've connected a direct cable between the Nexus switch's MGMT0 interface and started configuring vPC first on Nexus Switch 2.

switch2# configure terminal
switch2(config)# feature vpc
switch2(config)# vpc domain 1
switch2(config-vpc-domain)# peer-keepalive destination 10.10.12.253 source 10.10.12.254
Note:
 --------:: Management VRF will be used as the default VRF ::--------
switch2(config-vpc-domain)#  peer-switch    2018 Mar 23 07:54:39 switch2 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_SEND_FAIL: In domain 0, VPC peer keep-alive send has failed
2018 Mar 23 07:54:39 switch2 %$ VDC-1 %$ %STP-2-VPC_PEERSWITCH_CONFIG_DISABLED: vPC peer-switch configuration is disabled. Please make sure to change spanning tree "bridge" priority as per the recommended guidelines.
switch2(config-vpc-domain)# 2018 Mar 23 07:54:40 switch2 %$ VDC-1 %$ %STP-2-VPC_PEERSWITCH_CONFIG_ENABLED: vPC peer-switch configuration is enabled. Please make sure to configure spanning tree "bridge" priority as per recommended guidelines to make vPC peer-switch operational.
switch2(config-vpc-domain)# vrf context management
switch2(config-vrf)#   ip route 0.0.0.0/0 10.10.12.1
switch2(config-vrf)# 2018 Mar 23 07:54:59 switch2 %$ VDC-1 %$ %STP-2-BRIDGE_ASSURANCE_BLOCK: Bridge Assurance blocking port Po12 MST: 0.
switch2(config-vrf)# interface mgmt0
switch2(config-if)# vrf member management
switch2(config-if)# ip address 10.10.21.254/24
switch2(config-if)# 2018 Mar 23 07:55:07 switch2 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_SEND_FAIL: In domain 1, VPC peer keep-alive send has failed


I've configured the last ports on both Nexus switches: interface 47 and 48 as a Layer 2 port-channel (LACP EtherChannel) interface.

switch2(config-if)# show run interface port-channel12
!Command: show running-config interface port-channel12
!Time: Fri Mar 23 07:55:17 2018

version 6.0(2)U2(5)

interface port-channel12
  switchport mode trunk
  spanning-tree port type network


switch2(config-if)# show run interface e1/47
!Command: show running-config interface Ethernet1/47
!Time: Fri Mar 23 07:55:31 2018

version 6.0(2)U2(5)

interface Ethernet1/47
  switchport mode trunk
  channel-group 12 mode active

switch2(config-if)# show run interface e1/47 [J8
!Command: show running-config interface Ethernet1/48
!Time: Fri Mar 23 07:55:34 2018

version 6.0(2)U2(5)

interface Ethernet1/48
  switchport mode trunk
  channel-group 12 mode active

switch2(config-if)# 2018 Mar 23 07:55:39 switch2 %$ VDC-1 %$ last message repeated 16 times
switch2(config-if)# interface port-channel12
switch2(config-if)# vpc peer-link
Please note that spanning tree port type is changed to "network" port type on vPC peer-link.
This will enable spanning tree Bridge Assuranc e on vPC peer-link provided the STP Bridge Assurance(which is enabled by default) is not disabled.
switch2(config-if)# 2018 Mar 23 07:55:53 switch2 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_SEND_FAIL: In domain 1, VPC peer keep-alive send has failed
2018 Mar 23 07:55:57 switch2 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_SEND_FAIL: In domain 1, VPC peer keep-alive send has failed
switch2(config-if)# 2018 Mar 23 07:56:01 switch2 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_SEND_FAIL: In domain 1, VPC peer keep-alive send has failed
switch2(config-if)# end
switch2# 2018 Mar 23 07:56:03 switch2 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_SEND_FAIL: In domain 1, VPC peer keep-alive send has failed

switch2# show run vpc

!Command: show running-config vpc
!Time: Fri Mar 23 07:57:13 2018

version 6.0(2)U2(5)
feature vpc

vpc domain 1
  peer-switch
  peer-keepalive destination 10.10.12.253 source 10.10.12.254

interface port-channel12
  vpc peer-link


switch2# show vpc

Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1  
Peer status                       : peer link is down            
vPC keep-alive status             : Suspended (Destination IP not reachable)
Configuration consistency status  : failed 
Per-vlan consistency status       : success                      
Configuration inconsistency reason: Consistency Check Not Performed
Type-2 consistency status         : failed 
Type-2 inconsistency reason       : QoSMgr type-1 configuration incompatible
vPC role                          : none established             
Number of vPCs configured         : 0  
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Disabled (due to peer configuration)
Auto-recovery status              : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans   
--   ----   ------ --------------------------------------------------
1    Po12   up     -     


Next, I've configured Nexus Switch 1 and gave a higher role priority number.

switch1(config)# feature vpc
switch1(config)# vpc domain 1
switch1(config-vpc-domain)# 2018 Mar 23 08:16:34 switch1 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_SEND_FAIL: In domain 0, VPC peer keep-alive send has failed
2018 Mar 23 08:16:34 switch1 %$ VDC-1 %$ %STP-2-VPC_PEERSWITCH_CONFIG_DISABLED: vPC peer-switch configuration is disabled. Please make sure to change spanning tree "bridge" priority as per the recommended guidelines.
switch1(config-vpc-domain)# 2018 Mar 23 08:16:53
switch1 %$ VDC-1 %$ %STP-2-BRIDGE_ASSURANCE_BLOCK: Bridge Assurance blocking port Po12 MST: 0.
switch1(config-vpc-domain)# role priority 1000
Warning:
 !!:: vPCs will be flapped on current primary vPC switch while attempting role change ::!!
Note:
 --------:: Change will take effect after user has re-initd the vPC peer-link  ::--------
switch1(config-vpc-domain)# peer-keepalive destination 10.10.12.254 source 10.10.12.253
Note:
 --------:: Management VRF will be used as the default VRF ::--------
switch1(config-vpc-domain)# peer-switch   
switch1(config-vpc-domain)# 2018 Mar 23 08:17:10 switch1 %$ VDC-1 %$ %STP-2-VPC_PEERSWITCH_CONFIG_ENABLED: vPC peer-switch configuration is enabled. Please make sure to configure spanning tree "bridge" priority as per recommended guidelines to make vPC peer-switch operational.
switch1(config-vpc-domain)# vrf context management
switch1(config-vrf)# ip route 0.0.0.0/0 10.10.12.1
switch1(config-vrf)# interface mgmt0
switch1(config-if)# vrf member management
switch1(config-if)# ip address 10.10.12.253/24
switch1(config-if)# 2018 Mar 23 08:17:42 switch1 %$ VDC-1 %$ %VPC-2-PEER_KEEP_ALIVE_SEND_FAIL: In domain 1, VPC peer keep-alive send has failed
switch1(config-if)# end

switch1# show vpc

Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1  
Peer status                       : peer link not configured     
vPC keep-alive status             : peer is alive                
Configuration consistency status  : failed 
Per-vlan consistency status       : failed                       
Configuration inconsistency reason: vPC peer-link does not exist 
Type-2 consistency status         : failed 
Type-2 inconsistency reason       : vPC peer-link does not exist 
vPC role                          : none established             
Number of vPCs configured         : 0  
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Disabled (due to peer configuration)
Auto-recovery status              : Disabled


switch1# show run interface e1/47
!Command: show running-config interface Ethernet1/47
!Time: Fri Mar 23 08:18:16 2018

version 6.0(2)U2(5)

interface Ethernet1/47
  switchport mode trunk
  channel-group 12 mode active

switch1# show run interface e1/48
!Command: show running-config interface Ethernet1/48
!Time: Fri Mar 23 08:18:19 2018

version 6.0(2)U2(5)

interface Ethernet1/48
  switchport mode trunk
  channel-group 12 mode active

switch1# show run interface port-channel12
!Time: Fri Mar 23 08:18:20 2018

version 6.0(2)U2(5)

interface port-channel12
  switchport mode trunk
  spanning-tree port type network

switch1# configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
switch1(config)# interface port-channel12
switch1(config-if)# vpc peer-link
Please note that spanning tree port type is changed to "network" port type on vPC peer-link. This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance (which is enabled by default) is not disabled.

switch1# show vpc

Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1  
Peer status                       : peer link is down            
vPC keep-alive status             : peer is alive                
Configuration consistency status  : success
Per-vlan consistency status       : success                      
Type-2 consistency status         : success
vPC role                          : none established             
Number of vPCs configured         : 0  
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans   
--   ----   ------ --------------------------------------------------
1    Po12   up     -               


switch1# show run vpc

!Command: show running-config vpc
!Time: Fri Mar 23 08:19:31 2018

version 6.0(2)U2(5)
feature vpc

vpc domain 1
  peer-switch
  role priority 1000
  peer-keepalive destination 10.10.12.254 source 10.10.12.253

interface port-channel12
  vpc peer-link


switch1# show vpc role

vPC Role status
----------------------------------------------------
vPC role                        : primary                      
Dual Active Detection Status    : 0
vPC system-mac                  : 00:23:04:ee:be:01            
vPC system-priority             : 32667
vPC local system-mac            : 00:3a:7d:b3:27:01            
vPC local role-priority         : 1000


switch1# show vpc peer-keepalive

vPC keep-alive status             : peer is alive                
--Peer is alive for             : (179) seconds, (418) msec
--Send status                   : Success
--Last send at                  : 2018.03.23 08:20:41 403 ms
--Sent on interface             : mgmt0
--Receive status                : Success
--Last receive at               : 2018.03.23 08:20:41 404 ms
--Received on interface         : mgmt0
--Last update from peer         : (0) seconds, (249) msec

vPC Keep-alive parameters
--Destination                   : 10.10.12.254
--Keepalive interval            : 1000 msec
--Keepalive timeout             : 5 seconds
--Keepalive hold timeout        : 3 seconds
--Keepalive vrf                 : management
--Keepalive udp port            : 3200
--Keepalive tos                 : 192


It took around 3-5 minutes for the vPC peer-link adjacency to establish.

switch1# show vpc

Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 1  
Peer status                       : peer adjacency formed ok     
vPC keep-alive status             : peer is alive                
Configuration consistency status  : success
Per-vlan consistency status       : success                      
Type-2 consistency status         : success
vPC role                          : primary                      
Number of vPCs configured         : 0  
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans   
--   ----   ------ --------------------------------------------------
1    Po12   up     1,100

Configuring VTP Version 3, MST and EtherChannel on a Cisco Switch

The key benefits for running VTP Version 3 on a Cisco switch environment are:

• Encrypted VTP password
• Backward compatibility with VTP version 2
• Protection mechanism from overwriting a wrong VLAN database with higher revision number 
• Supports IEEE 802.1Q Extended VLANs 1006 - 4094
• Supports Private VLAN (PVLAN) propagation
• Supports Multiple Spanning Tree (MST) propagation
• Supports Remote SPAN (RSPAN)  

Switch#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Switch(config)#hostname SW01
SW01(config)#vtp version ?
  <1-3>  Set the administrative domain VTP version number

SW01config)#vtp version 3      // NEED TO CONFIGURE VTP DOMAIN FIRST
Cannot set the version to 3 because domain name is not configured
SW01(config)#vtp domain ?
  WORD  The ascii name for the VTP administrative domain.

SW01(config)#vtp domain vtp domain CORE
Changing VTP domain name from NULL to CORE

SW01(config)#vtp mode server ?
  client       Set the device to client mode.
  off          Set the device to off mode.
  server       Set the device to server mode.
  transparent  Set the device to transparent mode.

SW01(config)#vtp mode server ?
  mst      Set the mode for MST VTP instance.
  unknown  Set the mode for unknown VTP instances.
  vlan     Set the mode for VLAN VTP instance.
  <cr>

SW01(config)#vtp mode server mst     // ONLY SUPPORTED IN VTPv3
Device MST VTP mode cannot be changed in VTP version 1

SW01(config)#end
SW01#vtp primary ?
  force  Do not check for conflicting devices
  mst    MST feature
  vlan   Vlan feature
  <cr>

SW01#vtp primary     // NOT STORED IN NVRAM; NEED TO RE-CONFIGURE AGAIN WHEN SWITCH REBOOTS; SECONDARY VTP SERVER/SWITCH DOESN'T AUTOMATICALLY BECOME VTP PRIMARY SERVER
System can be made Primary Server only in VTP version 3

SW01#vtp primary mst
System can be made Primary Server only in VTP version 3

SW01#show vtp status
VTP Version capable             : 1 to 3
VTP version running             : 1
VTP Domain Name                 : CORE
VTP Pruning Mode                : Disabled
VTP Traps Generation            : Disabled
Device ID                       : 003c.109f.4f80
Configuration last modified by 0.0.0.0 at 0-0-00 00:00:00
Local updater ID is 0.0.0.0 (no valid interface found)

Feature VLAN:
--------------
VTP Operating Mode                : Server
Maximum VLANs supported locally   : 1005
Number of existing VLANs          : 5
Configuration Revision            : 0
MD5 digest                        : 0x15 0x89 0x9A 0xAE 0xF4 0x42 0x44 0x7D
                                    0xCA 0x15 0x45 0x7A 0x3B 0xA4 0x2E 0x64

SW01#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
SW01(config)#spanning-tree mode ?
  mst         Multiple spanning tree mode
  pvst        Per-Vlan spanning tree mode
  rapid-pvst  Per-Vlan rapid spanning tree mode

SW01(config)#spanning-tree mode mst
SW01(config)#vtp version 3
SW01(config)#
*Feb  7 03:22:33.292: %SW_VLAN-6-OLD_CONFIG_FILE_READ: Old version 2 VLAN configuration file detected and read OK.  Version 3 files will be written in the future.

SW01(config)#do show vtp status
VTP Version capable             : 1 to 3
VTP version running             : 3
VTP Domain Name                 : CORE
VTP Pruning Mode                : Disabled
VTP Traps Generation            : Disabled
Device ID                       : 003c.109f.4f80

Feature VLAN:
--------------
VTP Operating Mode                : Server
Number of existing VLANs          : 5
Number of existing extended VLANs : 0
Maximum VLANs supported locally   : 4096
Configuration Revision            : 0
Primary ID                        : 0000.0000.0000
Primary Description               :
MD5 digest                        :


Feature MST:
--------------
VTP Operating Mode                : Transparent


Feature UNKNOWN:
--------------
VTP Operating Mode                : Transparent


SW01(config)#vtp mode server mst
Setting device to VTP Server mode for MST.
SW01(config)#end

SW01#vtp primary force       // ENTER IN PRIVILEGE MODE; ONLY ONE PRIMARY SERVER IS ALLOWED; PRIMARY SERVER STATUS LOST WHEN SWITCH REBOOTS
This system is becoming primary server for feature vlan
SW01#
*Feb  7 03:23:34.402: %SW_VLAN-4-VTP_PRIMARY_SERVER_CHG: 003c.109f.4abc has become the primary server for the VLAN VTP feature

SW01#vtp primary mst
This system is becoming primary server for feature mst
No conflicting VTP3 devices found.
Do you want to continue? [confirm]      // PRESS ENTER
SW01#
*Feb  7 03:24:06.016: %SW_VLAN-4-VTP_PRIMARY_SERVER_CHG: 003c.109f.4f80 has become the primary server for the MST VTP feature

SW01(config)#spanning-tree ?
  backbonefast  Enable BackboneFast Feature
  etherchannel  Spanning tree etherchannel specific configuration
  extend        Spanning Tree 802.1t extensions
  logging       Enable Spanning tree logging
  loopguard     Spanning tree loopguard options
  mode          Spanning tree operating mode
  mst           Multiple spanning tree configuration
  pathcost      Spanning tree pathcost options
  portfast      Spanning tree portfast options
  transmit      STP transmit parameters
  uplinkfast    Enable UplinkFast Feature
  vlan          VLAN Switch Spanning Tree

SW01(config)#spanning-tree mst ?
  WORD           MST instance range, example: 0-3,5,7-9
  configuration  Enter MST configuration submode
  forward-time   Set the forward delay for the spanning tree
  hello-time     Set the hello interval for the spanning tree
  max-age        Set the max age interval for the spanning tree
  max-hops       Set the max hops value for the spanning tree

SW01(config)#spanning-tree mst configuration
SW01(config-mst)#?
  abort         Exit region configuration mode, aborting changes
  exit          Exit region configuration mode, applying changes
  instance      Map vlans to an MST instance
  name          Set configuration name
  no            Negate a command or set its defaults
  private-vlan  Set private-vlan synchronization
  revision      Set configuration revision number
  show          Display region configurations

SW01(config-mst)#name CORE
SW01(config-mst)#revison ?
  <0-65535>  Configuration revision number

SW01(config-mst)#revision 1
SW01(config-mst)#instance ?
  <0-4094>  MST instance id

SW01(config-mst)#instance 1 ?
  vlan  Range of vlans to add to the instance mapping

SW01(config-mst)#instance 1 vlan 2-4094    
SW01(config-mst)#exit
SW01(config)#spanning-tree mst 0-1 ?
  WORD           MST instance range, example: 0-3,5,7-9
  configuration  Enter MST configuration submode
  forward-time   Set the forward delay for the spanning tree
  hello-time     Set the hello interval for the spanning tree
  max-age        Set the max age interval for the spanning tree
  max-hops       Set the max hops value for the spanning tree

SW01(config)#spanning-tree mst 0-1 ?
  priority  Set the bridge priority for the spanning tree
  root      Configure switch as root

SW01(config)#spanning-tree mst 0-1 priority ?
  <0-61440>  bridge priority in increments of 4096

SW01(config)#spanning-tree mst 0-1 priority 4096     // SET TO LOWEST PRIORITY TO BECOME ROOT SWITCH

SW01#show spanning-tree mst configuration
Name      [CORE]
Revision  1     Instances configured 2

Instance  Vlans mapped
--------  ---------------------------------------------------------------------
0         1                      // MGMT VLAN; INSTANCE 0 CREATED BY DEFAULT
1         2-4094           // CUSTOMER VLAN
-------------------------------------------------------------------------------


This is the configuration for SW02.

SW02#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
SW02(config)#spanning-tree mode mst
SW02(config)#spanning-tree extend system-id
SW02(config)#spanning-tree mst 0-1 priority 8192      // SECONDARY ROOT SWITCH
SW02(config)#spanning-tree mst configuration
SW02(config-mst)# name CORE
SW02(config-mst)# revision 1
SW02(config-mst)# instance 1 vlan 2-4094
SW02(config-mst)#exit
SW02(config)#vtp domain CORE
Changing VTP domain name from NULL to CORE
SW02(config)#
*Feb  7 03:28:39.961: %SW_VLAN-6-VTP_DOMAIN_NAME_CHG: VTP domain name changed to CORE.
SW02(config)#vtp version 3
SW02(config)#
*Feb  7 03:28:46.010: %SW_VLAN-6-OLD_CONFIG_FILE_READ: Old version 2 VLAN configuration file detected and read OK.  Version 3
    files will be written in the future.
SW02(config)#vtp mode ?
  client       Set the device to client mode.
  off          Set the device to off mode.
  server       Set the device to server mode.
  transparent  Set the device to transparent mode.

SW02(config)#vtp mode server      // DEFAULT VTP MODE
Device mode already VTP Server for VLANS.


I've configured a Layer 2 EtherChannel trunk so VLANs configured on SW01 will automatically propagate to SW02.

SW02(config)#interface range GigabitEthernet1/0/47-48
SW02(config-if-range)#shutdown     // BEST PRACTICE IS TO SHUTDOWN FIRST THE PORTS TO BE CONFIGURED FOR ETHERCHANNEL
SW02(config-if-range)# description ### Trunk: SW01 ###
SW02(config-if-range)# switchport mode trunk
SW02(config-if-range)# channel-group 1 mode desirable
Creating a port-channel interface Port-channel 1

SW02(config-if-range)# no shutdown
SW02(config-if-range)#
SW02(config-if-range)#
*Feb  7 03:31:05.878: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/47, changed state to down
*Feb  7 03:31:05.878: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/48, changed state to down
SW02(config-if-range)#interface Port-channel1        // LAYER 2 ETHERCHANNEL TRUNK
SW02(config-if)# description ### Trunk: SW01 ###
SW02(config-if)# switchport mode trunk
SW02(config-if)#end

SW02#sh run interface g01/0/47
Building configuration...

Current configuration : 133 bytes
!
interface GigabitEthernet1/0/47
 description ### Trunk: SW01 ###
 switchport mode trunk
 channel-group 1 mode desirable
end

SW02#sh run interface g1/0/48
Building configuration...

Current configuration : 133 bytes
!
interface GigabitEthernet1/0/48
 description ### Trunk: SW01 ###
 switchport mode trunk
 channel-group 1 mode desirable
end

SW02#sh run interface po1
Building configuration...

Current configuration : 93 bytes
!
interface Port-channel1
 description ### Trunk: SW01 ###
 switchport mode trunk
end

SW02#show vtp ?
  counters   VTP statistics
  devices    VTP3 domain device information
  interface  VTP interface status and configuration
  password   VTP password
  status     VTP domain status

SW02#show vtp devices
Retrieving information from the VTP domain. Waiting for 5 seconds.

VTP Feature  Conf Revision Primary Server Device ID      Device Description   
------------ ---- -------- -------------- -------------- ----------------------
VLAN         No   2        003c.109f.4f80=003c.109f.4abc   SW01           
MST            Yes  2        003c.109f.4f80=003c.109f.4abc   SW01           

SW02#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
SW02(config)#vlan 100
VTP VLAN configuration not allowed when device is not the primary server for vlan database.     ONLY VTP PRIMARY SERVER CAN ADD/REMOVE VLANS

SW01(config)#interface range GigabitEthernet1/0/47-48
SW01(config-if-range)#shutdown
SW01(config-if-range)#description ### Trunk: SW02 ###
SW01(config-if-range)#switchport mode trunk
SW01(config-if-range)#channel-group 1 mode desirable
Creating a port-channel interface Port-channel 1

SW01(config-if-range)# no shutdown
SW01(config-if-range)#interface Port-channel1
SW01(config-if)# description ### Trunk: SW02 ###
SW01(config-if)# switchport mode trunk
SW01(config-if)#
*Feb  7 03:33:27.837: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/47, changed state to down
*Feb  7 03:33:27.837: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/48, changed state to down
*Feb  7 03:33:30.540: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/47, changed state to up
*Feb  7 03:33:30.698: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/48, changed state to up
*Feb  7 03:33:34.369: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/47, changed state to up
*Feb  7 03:33:34.499: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/48, changed state to up
*Feb  7 03:33:35.369: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up
SW01(config-if)#
*Feb  7 03:33:36.367: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up

SW01#show run interface g1/0/47
Building configuration...

Current configuration : 133 bytes
!
interface GigabitEthernet1/0/47
 description ### Trunk: SW02 ###
 switchport mode trunk
 channel-group 1 mode desirable
end

SW01#show run interface g1/0/48
Building configuration...

Current configuration : 133 bytes
!
interface GigabitEthernet1/0/48
 description ### Trunk: SW02 ###
 switchport mode trunk
 channel-group 1 mode desirable
end

SW01#show run interface p01
Building configuration...

Current configuration : 93 bytes
!
interface Port-channel1
 description ### Trunk: SW02 ###
 switchport mode trunk
end

SW01#show etherchannel ?
  <1-128>       Channel group number
  detail        Detail information
  load-balance  Load-balance/frame-distribution scheme among ports in
                port-channel
  port          Port information
  port-channel  Port-channel information
  protocol      protocol enabled
  summary       One-line summary per channel-group
  |             Output modifiers
  <cr>

SW01#show etherchannel summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port


Number of channel-groups in use: 1
Number of aggregators:           1

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         PAgP      Gi1/0/47(P) Gi1/0/48(P)

SW01#
*Feb  7 03:34:29.880: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/47, changed state to down       // I REMOVED THE CABLE ON G1/0/47
*Feb  7 03:34:30.879: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/47, changed state to down

SW01#show etherchannel summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port

Number of channel-groups in use: 1
Number of aggregators:           1

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         PAgP      Gi1/0/47(D) Gi1/0/48(P)

SW01#show cdp neighbor
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
                  D - Remote, C - CVTA, M - Two-port Mac Relay

Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
SW02            Gig 1/0/48        177              S I   WS-C3850- Gig 1/0/48

Total cdp entries displayed : 1

SW01#
*Feb  7 03:34:56.207: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/47, changed state to up    // PUT BACK CABLE ON G1/0/47
*Feb  7 03:35:00.050: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/47, changed state to up

SW01#show etherchannel summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port


Number of channel-groups in use: 1
Number of aggregators:           1

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         PAgP      Gi1/0/47(P) Gi1/0/48(P)

SW01#show cdp neighbor
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
                  D - Remote, C - CVTA, M - Two-port Mac Relay

Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
SW02            Gig 1/0/47        177              S I   WS-C3850- Gig 1/0/47
SW02            Gig 1/0/48        154              S I   WS-C3850- Gig 1/0/48

Total cdp entries displayed : 2