When was the last time, we tried to create GPFS file system using install-toolkit? How easy was it? In this blog, we are going to discuss the FQDN enhancements made in the IBM Spectrum Scale installation toolkit over the releases. The installation toolkit is a Chef-based automation tool that can be used for quick installation, deployment, and Upgrade of the GPFS file system with multiple protocol support. It is an effective tool for quick protocols deployment and upgrade.
HISTORY:
• Before 5.0.5.0 release, Installer was not designed to support both short name and FQDN during installation, deployment, and Upgrade process.
• Installer never checked the hosts file entries order for its correctness as per the format for at install/deploy earlier then started hitting issues during installation, deployment, and Upgrade process.
• Whenever the nodes were built on short names, config populate was failing as part of the upgrade process in a later/higher release.
• In case, if the cluster originally built using FQDNs for all node names and at some point, the DOMAIN for the datacentres has been changed. This required renaming each node in the cluster. Normally, one would match the new domain name, however if the user decided to lob off the domain name and refer to each node by hostname, relying upon DNS to resolve. This exposes a bug in the Install Toolkit where it has trouble with clusters built on hostnames while the DNS returns FQDNs. To further complicate the issue, DNS was mistakenly resolving to the old DOMAIN name.

case 1. /etc/hosts have entry: 10.18.112.25 node-vm1.in.ibm.com
Now Installer converted node to FQDN during Install/deploy and Cluster created as:
Node Daemon node name IP address Admin node name Designation
———————————————————————————
1 node-vm1.in.ibm.com 10.18.112.25 node-vm1.in.ibm.com quorum-perfmon
Now the DOMAIN for the /etc/hosts or datacentres has been changed and refer to each node by shortname (node-vm1), Installer was not designed before 505 release to handle this scenario which leads to FATAL error during config populate due to node name mismatch.
Case 2. /etc/hosts have entry: 10.18.112.25 node-vm1
Now Installer converted node to FQDN during Install/deploy and Cluster created as:
Node Daemon node name IP address Admin node name Designation
———————————————————————————
1 node-vm1 10.18.112.25 node-vm1 quorum-perfmon
Now the DOMAIN for the /etc/hosts or datacentres has been changed and refer to each node by FQDN (node-vm1.in.ibm.com), Installer was not designed before 505 release to handle this scenario which leads to FATAL error during config populate due to node name mismatch.

Case 3. /etc/hosts have entry: 10.18.112.25 node-vm1 node-vm1.in.ibm.com
Now Installer converted node to FQDN during Install/deploy and Cluster created as:
Node Daemon node name IP address Admin node name Designation
———————————————————————————
1 node-vm1.in.ibm.com 10.18.112.25 node-vm1.in.ibm.com quorum-perfmon
Now the DOMAIN for the /etc/hosts or datacentre has been changed and refer to each node by shortname (node-vm1), Installer was not designed before 505 release to handle this scenario which leads to FATAL error during config populate due to node name mismatch.

Installer Failure Example:
Config populate failure was due to installer was trying to look up by the FQDN but cluster was built using short name.
/etc/hosts have entry: 10.18.112.25 node-vm1
GPFS cluster information:
Node Daemon node name IP address Admin node name Designation
———————————————————————————
1 node-vm1 10.18.112.25 node-vm1 quorum-perfmon
Now the DOMAIN for the /etc/hosts or datacentres has been changed and refer to each node by FQDN (node-vm1.in.ibm.com).
[root@node-vm1 installer] # ./spectrumscale config populate -N node-vm1
[ INFO ] Logging to file: /usr/lpp/mmfs/5.0.5.0/installer/logs/config-populate-26-04-2020_18:09:59.log
[ INFO ] Detected clusterdefinition file present in configuration /usr/lpp/mmfs/5.0.5.0/installer/configuration.
Installer will keep backup of existing clusterdefinition.txt file in /usr/lpp/mmfs/5.0.5.0/installer/configuration path and populate a new one. Do you want to continue [Y/n]: y
[ INFO ] Updating existing configuration. It may take few moments ….
[ INFO ] Getting installer path
[ FATAL ] The admin node (specified by the -N option) does not match any of the node names found in the cluster. This can happen when the cluster has separate admin and daemon interface. Please ensure the admin node’s Admin interface is specified.

ENHANCEMENTS:
• In case if /etc/hosts entries are not in correct order then user will get the warning message.
• Node will get configured based on the etc/hosts entries, if it is short name then installer will configure with short name during “/spectrumscale node add” command only if the hostname is reachable.
• Installer will support both fqdn as well as short name during install/ deploy and upgrade process.
case 1. /etc/hosts have entry: 10.18.112.25 node-vm1.in.ibm.com
Now Installer will not try to convert node to FQDN during Install/deploy and Cluster created as:
Node Daemon node name IP address Admin node name Designation
———————————————————————————
1 node-vm1.in.ibm.com 10.18.112.25 node-vm1.in.ibm.com quorum-perfmon
Now the DOMAIN for the /etc/hosts or datacentres has been changed and refer to each node by shortname (node-vm1), Installer will not try to convert nodes to FQDN but continue to configure with the same name and at the same time it will warn user about the mis-ordered entries in the /etc/hosts as ‘ ‘ during Install/deploy and config populate.
Case 2. /etc/hosts have entry: 10.18.112.25 node-vm1
Now Installer will not try to convert node to FQDN during Install/deploy and Cluster created as:
Node Daemon node name IP address Admin node name Designation
———————————————————————————
1 node-vm1 10.18.112.25 node-vm1 quorum-perfmon
Now the DOMAIN for the /etc/hosts or datacentres has been changed and refer to each node by FQDN (node-vm1.in.ibm.com), Installer will not try to convert nodes to FQDN but continue to configure with the same name and at the same time it will warn user about the mis-ordered entries in the /etc/hosts as ‘ ‘ during Install/deploy and config populate.
Case 3. /etc/hosts have entry: 10.18.112.25 node-vm1 node-vm1.in.ibm.com
Now Installer will not try to convert node to FQDN during Install/deploy and Cluster created as:
Node Daemon node name IP address Admin node name Designation
———————————————————————————
1 node-vm1 10.18.112.25 node-vm1 quorum-perfmon
Now the DOMAIN for the /etc/hosts or datacentres has been changed and refer to each node by FQDN (node-vm1.in.ibm.com), Installer will not try to convert nodes to FQDN but continue to configure with the same name and at the same time it will warn user about the mis-ordered entries in the /etc/hosts as ‘ ‘ during Install/deploy and config populate.
Case 4. /etc/hosts have entry: 10.18.112.25
In this case Installer will not allow the user to continue but FATAL out to format the entries in the /etc/hosts as ‘ ‘ and retry.

Installer Success Example:
Config populate success, installer was not trying to look up by the FQDN but continue with the same shortname where cluster was built using it.
/etc/hosts have entry: 10.18.112.25 node-vm1
GPFS cluster information:
Node Daemon node name IP address Admin node name Designation
———————————————————————————
1 node-vm1 10.18.112.25 node-vm1 quorum-perfmon
Now the DOMAIN for the /etc/hosts or datacentres has been changed and refer to each node by FQDN (node-vm1.in.ibm.com).
[root@node-vm1 installer] # ./spectrumscale config populate -N node-vm1
Configuration successfully updated.

[root@node-vm1 installer]# ./spectrumscale node list

Join The Discussion

Your email address will not be published. Required fields are marked *