Retaining VM time stamp after IBM VM Recovery Manager DR/HA migration

Introduction

IBM® VM Recovery Manager DR for Power Systems™ provides an easy way to deploy and manage high availability solution for data centers. It enables a virtual machine (VM) restart-based high availability (HA) solution across a group of hosts (servers). VM Recovery Manager HA for Power Systems helps achieve better business continuity without the requirement for one-to-one backup hardware. Host groups allow servers to back up each other in case of unplanned outage events. Additionally, host groups allow for planned, nondisruptive relocation of VMs through Live Partition Mobility (LPM) from one host to another within the group.

An easy-to-use graphical interface can be used to deploy, monitor, and manage high availability for the entire data center. The VM Recovery Manager HA for Power Systems environment provides a deployment wizard that can easily deploy the solution.

This tutorial demonstrates how VM time can be synced with a target VIOS after migrating the VMs from the source site to the backup site using the VM Recovery Manager DR solution.

Note: In this tutorial, syncing of VM time is demonstrated for VM Recovery Manager DR. Same procedure can be followed for VM Recovery Manager HA.

IBM VM Recovery Manager HA for Power Systems provides many flexible HA policies. It includes:

  • Automation that integrates with IBM PowerVM® components, such as Hardware Management Console (HMC) and Virtual I/O Server (VIOS), to enable easy-to-discover-and-deploy HA for the environment.
  • Easily manageable HA for many VMs: You can enable or disable HA monitoring for host, host group, and VMs.
  • Automated restart, relocation, and failover of VMs during outages: Administrators can choose an advisory mode to receive failure-related alerts and perform the failovers manually.
  • Priority-based restart of VMs during recovery.
  • Priority-based capacity adjustment support during a recovery.
  • Control failure detection times for hosts and VMs: Application failure detection is done within the VM and hence those policies are controlled by the VM administrator.
  • Application HA monitoring support: Lightweight framework for AIX and Linux enables custom application monitoring capabilities. Control start and stop sequences of applications within a VM.

The VM Recovery HA for Power Systems solution is supported for VMs virtualized completely through PowerVM.

Additionally, VM Recovery Manager HA for Power Systems uses LPM for planned HA management that includes features such as:

  • Vacating a host for maintenance: All the VMs in the host are redeployed using LPM to other hosts in the host group.
  • Restoring a host after repair: It is possible to bring back all the VMs that were originally part of this host. The Homehost attribute of the VM is used to search your VM list and find the VMs that truly belong to host and move them back to the repaired original host

Figure 1. VM Recovery Manager HA solution architecture VM Recovery Manager HA solution architecture

Figure 1 shows the migration of a VM from one Central Electronics Complex (CEC) to the other CEC using the VM Recovery Manger HA solution. This can be achieved using the VIOS SSP capabilities.

VM Recovery Manager DR for Power Systems

The VM Recovery Manager DR solution is easy to deploy and use and can help clients plan and manage disaster recovery (DR) for their infrastructure. The solution works with components of IBM PowerVM®, such as VIOS and HMC, and also with storage subsystems to automate DR operations. After the setup and verification, administrators can perform DR failovers by running a single command. Some of the capabilities of the DR solution are:

  • Easy setup and deployment of the DR environment in a few steps
  • Administrator-initiated site or subsite (host group) for DR management
  • Storage replication management for DR purposes, including resynch automation
  • DR failover rehearsal for nondisruptive DR failover testing
  • Flexible capacity-based DR site capacity adjustment
  • Virtual local area network (VLAN) and vSwitch customization per site
  • Boot device management for IBM POWER8®, or later systems

Figure 2. VM Recovery Manager DR solution architecture VM Recovery Manager DR solution architecture

Figure 2 shows the VM Recovery Manager DR solution architecture which is based on storage replication technology across sites.

PowerVM Hypervisor and service processor

Each IBM Power System server has a real-time hardware clock which begins running from 0 when the server first starts at the factory. The value in the real-time clock is thus independent of Coordinated Universal Time (UTC). The real-time clock has a battery, and it ticks inexorably forward even when the server is disconnected from electrical power. The real-time clock has sufficient bits of memory to run for many millennia before wrapping back to 0. It is a good practice to set UTC on a server when the server is first deployed. It can be set using the Advanced System Management Interface (ASMI) on the flexible service processor (FSP) only when the server is switched off. The FSP’s ASMI is usually accessed through a HMC.

Setting a server’s UTC does not change the server’s real-time clock. Instead, a delta is saved by which UTC can be computed from the server’s real-time clock. As each logical partition (LPAR) is created, the LPAR is configured with the same delta as the server so that the LPAR’s virtual time-of-day (TOD) clock is consistent with UTC set on the server. The time zone on the operating system running on an LPAR can be configured so that the appropriate local time is displayed on the LPAR. The real-time clock delta of the server and of each LPAR is saved in NVRAM on the FSP so the deltas survive server power loss. An LPAR’s delta does not change unless or until the LPAR’s virtual TOD clock is changed by (for example) using the IBM AIX® date command. The server’s delta can be managed or changed as described in the following sections.

Time Reference Partitions (TRP)

Because the service processor allows a change to the TOD clock only when the server is switched off, PowerVM allows a user-designated Time Reference Partition (TRP) to control the TOD for the PowerVM Hypervisor and the service processor.

Setting up a partition as the TRP can be easily done by HMC user interface.

The TRP is responsible for maintaining the current time for the FSP and the PowerVM Hypervisor. A TRP cannot use partition migration to move to another server.

Once you have established a TRP, whenever the time is changed by the operating system, a notification is sent to the PowerVM Hypervisor to update its time based on the time set by the TRP. In addition, the hypervisor forwards the updated time to the FSP. PowerVM supports multiple partitions to be designated as the TRP. If multiple partitions are designated as TRPs, the TRP that has been active for the longest duration is used as the source for the TOD. If the current TRP becomes inactive, another partition designated as the TRP takes over the responsibility of maintaining the time on the server.

Management of time within a partition

Whenever you start a partition for the first time after switching on the server or anytime you create a new partition, the partition is started based on the current PowerVM Hypervisor time and date. After this first-time partition initialization, the operating system is in control of setting the correct time and date for the partition. Different operating systems have various time zone adjustments that can be applied on a partition based on which geographical location server is deployed. Some operating systems support the Network Time Protocol (NTP) where the partition can receive the current time from a known external source and synchronize the partition’s time and date to the values received from the NTP server. This would allow multiple partitions across multiple servers to stay in sync with regard to TOD. This may be especially important during communication between partitions or in the situation where LPM is used to seamlessly move a partition from one server to another.

Problem description

When the source and target hosts have different hypervisor time, then after migration, VMs activated are not in sync with target VIOS time. Now, the user wants to sync the VM time with the target VIOS time when VMs are migrated.

To sync the VM time with the target VIOS time, enable the Enable Time Reference option at VIOS and disable ETR at the VM level.

Note: If the user wants to retain the same VM time stamp across sites, then sync both source and target VIOS times. So, after migration the VM will acquire the same time stamp.

To enable or disable the Enable Time Reference option:

  • Choose the VIOS instance or the VM to enable or disable Enable Time Reference.
  • In the Properties section, click Properties -> General Properties to view and change the properties of the selected partition.
  • Click the Advanced tab. The Advanced Settings options are displayed.
  • Select or clear the Enable Time Reference check box to select or deselect the option.
  • Click Apply and save the changes.

Note that the Enable Time Reference check box should be selected for both source and target VIOS instances.

Figure 3. Enable Time Reference setting for VIOS Enable Time Reference setting for VIOS

Figure 4. Disable the Enable Time Reference setting for VM Disable the Enable Time Reference setting for VM

Configuration details

After enabling and disabling the Enable Time Reference option, we can migrate the VMs from the source site to the target site. Figure 5 displays the time stamps before migration. Here both source and target VIOS instances are synced but even if both VIOS instances have different time stamps, VM will once activated will be in sync with the target VIOS time.

The following components and resources are used in the DR cluster:

  • Home site: Austin
  • Backup site: India
  • Source VIOS: sockv1
  • Target VIOS: shoev1
  • Managed VMs: sock001, sock002

Time sync procedure after VM Migration

This section shows the time verification before and after migration of VM done with VM Recovery manager with DR.

  1. Verify the time of all VIOS instances and VMs before migration using the date command as shown in Figure 5. The output of the script shows the date and time for all the VIOS instances and VMs on both the sites. As shown in figure, the VIOS time and the VM time are in sync.

    Figure 5. Time stamps on VIOS instances and VM before migration Time stamps on VIOS instances and VM before migration

  2. Perform VM migration from the source site to backup site using the following command:

     ksysmgr move site from=<src_site> to=<backup_site>
    

    VM migration is performed using the command line interface of the VM Recovery Manager DR. In this tutorial, a planned type move is shown for migrating the VM from the source site to the backup site. In this example, we are triggering a planned move for site Austin to site India. Figure 6 displays the output of migration.

    Figure 6. Migration of VM – output from ksysmgr Migration of VM – output from ksysmgr

  3. Verify the time of all VIOS instances and VMs after migration using the date command. After migration, the timing for sock001 and sock002 is the same as that for shoev1. Figure 7 displays the time stamp for the VIOS instances and VMs after migration.

    Figure 7. Checking time stamps of VMs and VIOS instances after migration Checking time stamps of VMs and VIOS instances after migration

  4. Migrate the VM from the backup site to the source site using the following command:

     ksysmgr move site from=<target_site> to=<home_site>
    

    In this example, weare bringing back the VMs from the target site (India) to the source site (Austin). Figure 8 shows the output of migration

    Figure 8. Migration output from target site to home site Migration output from target site to home site

    After migration, the timing for sock001 and sock002 is the same as that of sockv1. Figure 9 displays the time stamp for the VIOS instances and VMs after migration.

    Figure 9. Verifying time stamps for VMs and VIOS instances Verifying time stamps for VMs and VIOS instances

Conclusion

By following the steps mentioned in this tutorial, it is easy to enable and disable the Enable Time R from the HMC. It demonstrated that after setting ETR, if user triggers a move at the host group level or at the site level from VM Recovery Manager DR/HA, then VMs will be activated in sync with the target VIOS time.

Neha Jain
Srikanth Thanneeru
Dishant Doriwala