Thursday, March 31, 2016

Solaris10 - link aggregation



Prerequisite task:-
On Control domain, record the following information.

1. Get the virtual switch information.
# ldm list-services | more
VCC
VSW
    NAME             LDOM             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    primary-vsw0     primary          00:14:4f:fa:33:81 aggr1     0    switch@0              1               1                         1500         on
    primary-vsw1     primary          00:14:4f:f8:c7:39 aggr2     1    switch@1              1               1                         1500         on


2. Get interface binding info for each domain.
# ldm list-bindings | more
VSW
    NAME             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    primary-vsw0     00:14:4f:fa:33:81 aggr1     0    switch@0              1               1                         1500         on
        PEER                        MAC               PVID VID                  MTU   LINKPROP   INTERVNETLINK
        vnet1@mmerp-mw-v06           00:14:4f:fa:53:68 1                         1500
        vnet2@mmerp-mw-v07           00:14:4f:f9:cb:06 1                         1500
        vnet3@mmerp-mw-v08           00:14:4f:fb:36:ea 1                         1500
    NAME             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    primary-vsw1     00:14:4f:f8:c7:39 aggr2     1    switch@1              1               1                         1500         on
        PEER                        MAC               PVID VID                  MTU   LINKPROP   INTERVNETLINK
        vnet1@mmerp-db-v06           00:14:4f:f9:57:32 1                         1500
        vnet2@mmerp-db-v07           00:14:4f:f9:f7:11 1                         1500
        vnet3@mmerp-db-v08           00:14:4f:f9:2a:74 1                         1500


3. Get the aggregation info.
# dladm  show-aggr
key: 1 (0x0001) policy: L4      address: 0:21:28:f1:de:29 (auto)
           device       address                 speed           duplex  link    state
           igb1         0:21:28:f1:de:29          1000  Mbps    full    up      attached
key: 2 (0x0002) policy: L4      address: 0:21:28:f1:de:2b (auto)
           device       address                 speed           duplex  link    state
           igb3         0:21:28:f1:de:2b          1000  Mbps    full    up      attached

Now, perform the following tasks.

1. Login to each server. luacticate newBE and init 0 the servers.
# luactivate newBE; init 0
2. Login to control domain and stop all LDOMs.
# ldm stop myldm
# ldm list - verify
3. Activate BE on control domain and reboot
# luactivate newBE; init 6
4. Once server comes up, verify patch level.
# uname -a
5. Get interface binding info for each LDOM.
# ldm list-services | grep -i vsw
# ldm list-bindings | more


VSW
    NAME             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFA
ULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
primary-vswaggr1  aggr1 switch@1
        vnet1@mmerp-db-v12
 vnet1@mmerp-mw-v11
 vnet1@pkds-mw-v13
 vnet1@pkds-mw-v12
 vnet1@trpn-mw-v08
 vnet1@depr-mw-v09
 vnet1@pkds-mw-v15
 vnet1@depr-mw-v08
 vnet1@mmerp-mw-v15


VSW
NAME             MAC               NET-DEV   ID   DEVICE      LINKPROP 
primary-vsw0     00:14:4f:fa:33:81 aggr1     0    switch@0              1
        vnet1@mmerp-mw-v06
        vnet2@mmerp-mw-v07
        vnet3@mmerp-mw-v08
primary-vsw1     00:14:4f:f8:c7:39 aggr2     1    switch@1              1
        vnet1@mmerp-db-v06
        vnet2@mmerp-db-v07
        vnet3@mmerp-db-v08

6. Remove existing VNETs from all LDOMs
# ldm remove-vnet vnet0 <myldom>
7. Remove the vsw info from LDOM configuration
# ldm remove-vsw primary-vsw0
# ldm remove-vsw primary-vsw1
# ldm remove-vswitch primary-vsw1
8. Remove all existing link aggregations
# dladm  show-aggr
# dladm show-link
aggr1           type: non-vlan  mtu: 1500       aggregation: key 1
aggr2           type: non-vlan  mtu: 1500       aggregation: key 2
Note the value after key on the output above.
# dladm delete-aggr 1
# dladm delete-aggr 2

9. Create new virtual switch using these two interfaces below.
igb1 -> primary-vsw0
igb2 -> primary-vsw1
# ldm add-vsw net-dev=igb1 inter-vnet-link=off linkprop=phys-state primary-vsw0 primary
# ldm add-vsw net-dev=igb2 inter-vnet-link=off linkprop=phys-state primary-vsw1 primary

Now, assign the vnet to the LDOM. To spread between LDOM, half of the LDOM will be on vsw0 and half on vsw1 as primary interface and secondary interface.


pkds-mw-v12  (vnet0:  primary-vsw0    vnet1: primary-vsw1)  --done
pkds-mw-v13  (vnet0:  primary-vsw0    vnet1: primary-vsw1)
pkds-mw-v15  (vnet0:  primary-vsw0    vnet1: primary-vsw1)
mmerp-mw-v15  (vnet0:  primary-vsw0    vnet1: primary-vsw1)
pkds-mw-v13      active     -t----  5003    8     32G       12%   12%  1m
pkds-mw-v15      active     -t----  5000    8     8G        12%   12%  25s
mmerp-mw-v15      active     -t----  5005    16    16G      6.3%  6.3%  14s
# ldm add-vnet linkprop=phys-state vnet0 primary-vsw0 pkds-mw-v13
# ldm add-vnet linkprop=phys-state vnet1 primary-vsw1 pkds-mw-v13
# ldm add-vnet linkprop=phys-state vnet0 primary-vsw0 pkds-mw-v15
# ldm add-vnet linkprop=phys-state vnet1 primary-vsw1 pkds-mw-v15
# ldm add-vnet linkprop=phys-state vnet0 primary-vsw0 mmerp-mw-v15
# ldm add-vnet linkprop=phys-state vnet1 primary-vsw1 mmerp-mw-v15


mmerp-mw-v11  (vnet0:  primary-vsw1    vnet1: primary-vsw0)  -- done  -- not working
trpn-mw-v08  (vnet0:  primary-vsw1    vnet1: primary-vsw0)  - worked
mmerp-db-v12  (vnet0:  primary-vsw1    vnet1: primary-vsw0)   -- card -reader., working ..
depr-mw-v08  (vnet0:  primary-vsw1    vnet1: primary-vsw0)
depr-mw-v09       (vnet0:  primary-vsw1    vnet1: primary-vsw0)
# ldm add-vnet linkprop=phys-state vnet0 primary-vsw1 mmerp-db-v12
# ldm add-vnet linkprop=phys-state vnet1 primary-vsw0 mmerp-db-v12
# ldm add-vnet linkprop=phys-state vnet0 primary-vsw1 depr-mw-v08
# ldm add-vnet linkprop=phys-state vnet1 primary-vsw0 depr-mw-v08
# ldm add-vnet linkprop=phys-state vnet0 primary-vsw1 depr-mw-v09
# ldm add-vnet linkprop=phys-state vnet1 primary-vsw0 depr-mw-v09
# ldm start mmerp-db-v12
LDom mmerp-db-v12 started

primary-vsw0
 vnet1@mmerp-mw-v06
        vnet2@mmerp-mw-v07
        vnet3@mmerp-mw-v08
# ldm add-vnet linkprop=phys-state vnet0 primary-vsw0 <ldm>
# ldm add-vnet linkprop=phys-state vnet1 primary-vsw1 <ldm>

primary-vsw1
        vnet1@mmerp-db-v06
        vnet2@mmerp-db-v07
        vnet3@mmerp-db-v08
# ldm add-vnet linkprop=phys-state vnet0 primary-vsw1 <ldm>
# ldm add-vnet linkprop=phys-state vnet1 primary-vsw0 <ldm>

10. Once interface is assigned to the LDOM, reboot your LDOM
# ldm start <LDOM>
# telnet 0 <port>
{ok} boot -s
11. On single user mode, modify the interface info
# vi /etc/hostname.interface0
hostname  netmask + broadcast + group ipmp up
# vi /etc/hostname.interface1
group ipmp up
12. Boot your system.
# press CTRL + D or init 6

13. Once server comes up, verify the patch level
# uname -a
14. Verify the multipathing info
# if_mpadm -d interface
# ifconfig -r interface
-d -> Detach or off-line the interface
-r -> Reattach



mmerp-db-v06     netmask + broadcast + group ipmp up
group ipmp up

No comments:

Post a Comment