Kernel Zone Storage Live Migration with ZOSS

So we have some Kernel Zones on shared storage that need to migrate to another shared storage. Let's do that without any service interruption.

Current status is:

root@global:~ # zonecfg -z kz1 info
...
device 0:
        match not specified
        storage: lu:luname.naa.6006016027802e00084ff53eefdae511
        id: 1
        bootpri not specified
device 1:
        match not specified
        storage: lu:luname.naa.6006016027802e00965dc37eefdae511
        id: 0
        bootpri: 1
...
root@global:~ # fcinfo lu -v
OS Device Name: /dev/rdsk/c0t600000E00D2B0000002B021000940000d0s2
        HBA Port WWN: 2100000e1ed...
                Remote Port WWN: 500000e0dac...
                        LUN: 0
                Remote Port WWN: 500000e0dac...
                        LUN: 0
        HBA Port WWN: 2100000e1ed...
                Remote Port WWN: 500000e0dac...
                        LUN: 0
                Remote Port WWN: 500000e0dac...
                        LUN: 0
        Vendor: FUJITSU
        Product: ETERNUS_DXH
        Device Type: Disk Device
...
OS Device Name: /dev/rdsk/c0t600000E00D2B0000002B021000980000d0s2
        HBA Port WWN: 2100000e1ed...
                Remote Port WWN: 500000e0dac...
                        LUN: 150
                Remote Port WWN: 500000e0dac...
                        LUN: 150
        HBA Port WWN: 2100000e1ed...
                Remote Port WWN: 500000e0dac...
                        LUN: 150
                Remote Port WWN: 500000e0dac...
                        LUN: 150
        Vendor: FUJITSU
        Product: ETERNUS_DXH
        Device Type: Disk Device

Looks good, we see the new LUNs, let's convert the OS Device Name to a ZOSS Storage URI.

root@global:~ # suriadm lookup-uri -t lu /dev/dsk/c0t600000E00D2B0000002B021000940000d0
lu:luname.naa.600000e00d2b0000002b021000940000
...
root@global:~ # suriadm lookup-uri -t lu /dev/dsk/c0t600000E00D2B0000002B021000980000d0
lu:luname.naa.600000e00d2b0000002b021000980000

Let's add them to our kz1 zone config and do a live zone reconfiguration.

root@global:~ # zonecfg -z kz1
zonecfg:kz1> add device
zonecfg:kz1:device> set storage=lu:luname.naa.600000e00d2b0000002b021000980000
zonecfg:kz1:device> set bootpri=1
zonecfg:kz1:device> set id=2
zonecfg:kz1:device> end
zonecfg:kz1> add device
zonecfg:kz1:device> set storage=lu:luname.naa.600000e00d2b0000002b021000940000
zonecfg:kz1:device> set id=3
zonecfg:kz1:device> end
zonecfg:kz1> verify
zonecfg:kz1> commit
zonecfg:kz1> exit
root@global:~ # zoneadm -z kz1 apply
zone 'kz1': Checking: Adding device storage=lu:luname.naa.600000e00d2b0000002b021000940000
zone 'kz1': Checking: Adding device storage=lu:luname.naa.600000e00d2b0000002b021000980000
zone 'kz1': Applying the changes
...

Next steps are inside the Kernel Zone now.

root@kz1:~ # echo | format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c1d0 <DGC-VRAID-0532-15.00GB>
          /kz-devices@ff/disk@0
       1. c1d1 <DGC-VRAID-0532-65.00GB>
          /kz-devices@ff/disk@1
       2. c1d2 <FUJITSU-ETERNUS_DXH-1070 cyl 1920 alt 0 hd 256 sec 64>
          /kz-devices@ff/disk@2
       3. c1d3 <FUJITSU-ETERNUS_DXH-1070 cyl 8320 alt 0 hd 256 sec 64>
          /kz-devices@ff/disk@3
Specify disk (enter its number): Specify disk (enter its number):

root@kz1:~ # zpool attach rpool c1d0 c1d2
Make sure to wait until resilver is done before rebooting.
root@kz1:~ # zpool attach tank c1d1 c1d3
# wait some time...
root@kz1:~ # zpool status -v
  pool: rpool
 state: ONLINE
  scan: resilvered 7.53G in 33s with 0 errors on Tue Jan 30 15:09:55 2018

config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c1d0    ONLINE       0     0     0
            c1d2    ONLINE       0     0     0

errors: No known data errors

  pool: tank
 state: ONLINE
  scan: resilvered 28.9G in 7m40s with 0 errors on Tue Jan 30 15:17:26 2018

config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c1d1    ONLINE       0     0     0
            c1d3    ONLINE       0     0     0

errors: No known data errors
root@kz1:~ # zpool detach rpool c1d0
root@kz1:~ # zpool detach tank c1d1
root@kz1:~ # bootadm install-bootloader

And that's almost it. Last thing is to remove the old storage LUNs from the Kernel Zone.

root@global:~ # zonecfg -z kz1
zonecfg:kz1> remove device id=0
zonecfg:kz1> remove device id=1
zonecfg:kz1> verify
zonecfg:kz1> commit
zonecfg:kz1> exit
root@global:~ # zoneadm -z kz1 apply
zone 'kz1': Checking: Removing device storage=lu:luname.naa.6006016027802e00084ff53eefdae511
zone 'kz1': Checking: Removing device storage=lu:luname.naa.6006016027802e00965dc37eefdae511
zone 'kz1': Applying the changes

And we're done.

Links

• Replacing Disks in a ZFS Root Pool
• Oracle Solaris: Zones on Shared Storage
• Live Zone Reconfiguration
• Live Zone Reconfiguration for Dataset Resources