Migrating LPARs from one host to another using the IBM VM Recovery Manager HA for Power Systems solution

Introduction

Live Partition Mobility (LPM) operations for multiple logical partitions (LPARs) from a single Hardware Management Console (HMC) interface (GUI or command line) is not possible unless there is a script for the same and the target managed system must be specified if these operations are performed. There is no direct way to restore all LPARs in the environment to its home host from the HMC interface.

This tutorial describes how to perform LPM operations for a list of LPARs or all managed LPARs in a given managed system and how to restore LPARs in the environment to its home host from the IBM® VM Recovery Manager HA for Power Systems solution. This tutorial also describes possible errors during LPM operations and troubleshooting.

Specifying a target-managed system for LPARs is optional in the VM Recovery Manager HA solution for these operations. Supported LPARs include IBM® AIX®, Linux®, and IBM i. VM recovery manager uses the Representational State Transfer (REST) API interface to trigger the LPM operations.

Prerequisites

Complete the following prerequisites to perform LPM operations using the VM Recovery Manager HA solution.

• Install KSYS file sets with version v1.3.0 or later. Follow the link provided in the Resources section for installation steps.

• Configure KSYS. Follow the link provided in the Resources section for configuration steps.

• Complete the configuration required to perform LPM operations. Follow the link provided in the Resources section for required configuration and checklist.

Invocation of LPM operations

LPM operations can be invoked using the ksysmgr utility. LPM operations include: validation, migration, restoration, and recovery of LPARs. You can trigger validation and migration operations for a host or multiple LPARs at a time. Home host must be specified for the restore operation. Recover operation is supported for one LPAR at a time. You can specify target-managed systems for validation and migration operations. For migration, if a target-managed system is not specified, the VM Recovery Manager HA solution chooses the best one from the available target managed systems for each LPAR based on the configured policies. For validation, if the target-managed system is not specified, the VM Recovery Manager HA solution validates with all available managed systems in the host group. These operations support only for running LPARs. A maximum of eight migrations are active or running with HMC at a time.

Validation

This section lists the various commands that you can use for validation.

• Use the following command to invoke validation for the provided LPARs against a given target managed system: ksysmgr lpm vm <vmname1,[vmname2,...]> [to=<hostname|uuid>] action=validate
• Use the following command to invoke validation for the provided LPARs as a list against all other available managed systems in the host group: ksysmgr lpm vm <vmname1, [vmname2,...]> action=validate
• Use the following command to invoke validation for LPARs in a given managed system against a given target-managed system: ksysmgr lpm host <hostname|uuid> [to=<hostname|uuid>] action=validate
• Use the following command to invoke validation for LPARs in a given managed system against all other available managed systems in the host group: ksysmgr lpm host <hostname|uuid> action=validate

Validation example 1

# ksysmgr -t lpm vm yin001,yin002 action=validate
05:26:59 Running Move verification on VM(S), this
      may take few minutes...
05:27:09 lpm verification has started for yin002
05:27:09 lpm verification has completed for yin002
05:27:09 lpm verification has started for yin001
05:27:09 lpm verification has completed for yin001
2 out of 2 VMs have been successfully verified

Check the status while validation is in progress

While the validation is in progress for a VM, VM_status would be VERIFY_IN_PROGRESS.

# ksysmgr query vm yin001
Name:                yin001
UUID:                7928E70F-7B00-4848-ADD6-0C85956F5CF4
State:               VERIFY_ONLY
Host:                Yang-8286-42A-101417V
Priority:            Medium
VM_failure_detection_speed:    fast
HA_monitor:          enable
Homehost:            Yin-8247-42L-211E9DA
VM_status:           VERIFY_IN_PROGRESS
Version_conflict:    No

Validation example 2

# ksysmgr -t lpm vm yin003 action=validate
08:58:18 Running Move verification on VM(S), this may take few minutes...
08:58:29 lpm verification has started for yin003
09:08:53 ERROR: lpm verification has encountered an error for VM yin003
Error VM(s):
    yin003
Please review errors.  The "ksysmgr query system status" command may provide additional    details.

Check the status after validation is completed

# ksysmgr query vm yin003
Name:                yin003
UUID:                3FE30E3F-BA67-42EE-9A64-854005ED4994
State:               READY
Host:                Yang-8286-42A-101417V
Priority:            Medium
VM_failure_detection_speed:    normal
HA_monitor:          enable
Homehost:            Yin-8247-42L-211E9DA
VM_status:           NO_OPERATION_IN_PROGRESS
Version_conflict:    No

LPM Validation Status
LPM validation failed for Hosts:
      Yin-8247-42L-211E9DA

After the validation is completed for a VM, VM_status shows NO_OPERATION_IN_PROGRESS.

Check for errors

If there is an error in validation, you can view it using the following command:

# ksysmgr query system status
 Following error is returned for VM yin003:
 HSCLA27C The operation to get the physical device location for adapter U8286.42A.101417V-V2-C403 on the virtual I/O server partition yangv2 has failed.
 Host_group test_hg is currently in Ready state
 Please review the error(s) and take any corrective actions

You can also use the following command as an alternate method to check the validation errors:

# lsrsrc IBM.VMR_LPAR Name VerifyState
 Resource Persistent Attributes for IBM.VMR_LPAR
 resource 1:
    Name        = "yin001"
    VerifyState = {["38f343b0-95fb-3c8d-ab13-de832fc6e9cf","PASS",""]}
 resource 2:
    Name        = "yin002"
    VerifyState = {["38f343b0-95fb-3c8d-ab13-de832fc6e9cf","PASS",""]}
 resource 3:
    Name        = "yin003"
    VerifyState = {["38f343b0-95fb-3c8d-ab13-de832fc6e9cf","Fail","Errors:
 HSCLA27C The operation to get the physical device location for adapter U8286.42A.101417V-V2-C403 on the virtual I/O server partition yangv2 has failed. ... }

Migration

This section lists the commands that you can use to perform various migration activities.

• Use the following command to move LPARs to a given target-managed system: ksysmgr lpm vm <vmname1, [vmname2,...]> [to=<hostname|uuid>]
• Use the following command to move LPARs to any of the managed system selected by a policy manager in the host group: ksysmgr lpm vm <vmname1, [vmname2,...]>
• Use the following command to move all LPARs of a managed system to a given target-managed system: ksysmgr lpm host <hostname|uuid> [to=<hostname|uuid>]
• Use the following command to move LPARs of a managed system to any of the managed system selected by the VM Recovery Manager HA solution in the host group: ksysmgr lpm host <hostname|uuid>

Migration example 1

# ksysmgr -t lpm vm yin002
    05:48:22 Running move on VM(S), this may take few minutes...
    05:48:32 lpm has started for yin002
    05:54:55 lpm has completed for yin002 on Target host
       Yin-8247-42L-211E9DA
    05:54:56 Waiting for rediscovery.
    1 out of 1 VMs have been successfully moved

Check status while migration is in progress

While validation is in progress for a VM, VM_status would be LPM_OPERATION_IN_PROGRESS.

# ksysmgr query vm yin002
     Name:                yin002
     UUID:                4D8B83C7-578E-478B-B737-ABF317C4DAD9
     State:               RECOVERY_ONLINE
     Host:                Yang-8286-42A-101417V
     Priority:            Medium
     VM_failure_detection_speed:    fast
     HA_monitor:          enable
     Homehost:            Yin-8247-42L-211E9DA
     VM_status:           LPM_OPERATION_IN_PROGRESS
     Version_conflict:    No

Migration example 2

# ksysmgr -t lpm vm yin001
     05:38:06 Running move on VM(S), this may take few minutes...
     05:38:17 ERROR: lpm has encountered an error for VM yin001
     Error VM(s):
              yin001
     Please review errors.  The "ksysmgr query system status" command
     may provide additional details.

Check for errors

If there is an error during migration, you can view it using the following command.

# ksysmgr query system status
     Following error is returned for VM yin001:
     Couldn't find the target Host!!. VM :
     yin001 UUID 7928E70F-7B00-4848-ADD6-0C85956F5CF4
        Host_group test_hg is currently in Ready state
     Please review the error(s) and take any corrective actions

Recover

If VM migration has failed and a VM instance is present on both source and target hosts, use the recover option to recover the VM either on the source or the target host based on the stage where migration failed.

Run the following command to invoke the recover operation: # ksysmgr lpm vm <vmname> action=recover

The recover operation supports one VM at a time.

Restore

Restore is the feature in the VM Recovery Manager HA solution which is used to bring back the managed LPARs to its home host. If you add a host during configuration, the same will be set as the home host by default for the LPARs in that host and will be maintained at the LPAR resource level. Home host for a LPAR can be changed through the ksysmgr utility. If a VM is already in its host, migration will be ignored for this VM.

Run the following command to invoke the restore operation: # ksysmgr restore host <hostname|uuid>

HA state, phase, and phase details of LPM operations

HA state, phase, and phase detail values are used to track the state and progress of the requested operation.

HA states

HA state indicates a specific operation which is in progress in the KSYS system. The same state is active till the requested operation is completed.

Table 1. HA states related to LPM operations

Phases

Phase is the string used to specify the progress and result of the requested operation.

Table 2. Phases related to LPM operations

Phase details

Phase details are used by ksysmgr to track the progress of the last requested operation for the targeted LPARs in the host group.

Table 3. Phase details related to LPM operations

Troubleshooting

If LPAR move has failed while performing LPM from KSYS, you need to check the LPAR state, events in the events.log file and error messages in console from the requested LPM operation to identify the reason.

Events log path: /var/ksys/events.log

ksysmgr log path: /var/ksys/log/ksysmgr.log

Command to check LPAR state: ksysmgr query vm <VM name>

Example to check the LPAR state:

# ksysmgr query vm
Name:            testvm
UUID:            abcdef-BA67-42EE-9A64-ghijklmn
Host:            testhost
State:           LPM_VERIFY_FAILED

In above example State value represents the LPAR state.

Errors can be seen in console at the end of each requested LPM operation or when using the following command:

ksysmgr query system status

Table 4. Failed scenario examples during LPM operations

Summary

This tutorial is helpful to migrate multiple logical partitions from one managed system to another in a KSYS configured environment and track the status of requested operations. It can be more helpful where LPARs must be restored to its home host with a single command. Specifying a target-managed system is optional in the case of validation and migration. After successfully migrating the LPARs, the VM Recovery Manager HA solution makes the necessary changes in its configurations accordingly.