IBMÂ® Open Platform (IOP) 4.1 with Apache Hadoop based on Ambari includes a new pack of serviceability tools to assist on the deployment of the product, maintenance and diagnosis. More details about these serviceability tools can be found in this blog post.
As part of the these serviceability tools, the pre-installation check feature aims to facilitate the deployment of the product to customers. It provides a script to verify system requirements and check the environment before IOP and BigInsightsÂ® value-adds services (BigR, BigSQL, BigSheets, and TextAnalytics) installation.
System administrator and IOP users can run the prechecker python script to ensure the environment is prepared to a successful installation. It is intended to be ran before Ambari IOP, and BigInsights value-adds are installed. The script reviews pre-installation requirements and recommendations, such as: supported operating system, jdk version, python version, related running processes, users, directories, required software…
Download serviceability tools
The set of serviceability tools for BigInsightsÂ® 4.1 is available to download on Diagnostic Scripts for IBM BigInsights web site after accepting the license of use.
The prechecker.py is included in the tarball file biginsights-serviceability.tar.gz”.
How to use prechecker.py script?
After the prechecker.py script is downloaded, extracted and ensured permission to execute, it can be ran on one node or several nodes in the cluster depending on the arguments used on its call:
- The call “python prechecker.py” without arguments will run the precheck for the current node only.
- The call “python prechecker.py <HOSTS_FILE>”, where <HOSTS_FILE> is the name of a file that lists the node names to run the precheck on separate lines. Run precheck tool on all the nodes of a cluster is highly recommended to ensure a clean product installation. An example of a <HOSTS_FILE> is a file called “hosts.txt” which includes the following content with the list of hosts on a four nodes cluster:
In this case, to run the precheck on the whole cluster, you would use the command “python prechecker.py hosts.txt”.
In order to run the precheck on multiple hosts, passwordless ssh connections must be setup between the master node where the script is ran and the rest of hosts included in <HOSTS_FILE>. Hence the master node can run the precheck automatically on each host on the list. This is the very first check that the precheck script performs.
Set up passwordless ssh is detailed on Knowledge Center documentation â†’ Preparing your environmentâ†’ Configure your network.
The precheck tool will output results of the current node’s precheck to stdout, but it will give full details regarding precheck results on all of the nodes in a file that it creates (file name is given in stdout).
root@ localhost1.localdomain serviceability]# python prechecker.py ============Running precheck on localhost1.localdomain ============ ==============Check for compatible Operating System============== A failure indicates that this machine is not running on an OS supported by IOP 4.1 redhat 6.5 [OK] ==============Check for existing Ambari/IOP directories============== A failure indicates that there are existing directories related to Ambari or IOP that may cause issues /var/run/knox [ OK ] /var/run/hadoop [ OK ] /var/run/flume [ OK ] /var/run/hadoop-mapreduce [ OK ] /var/run/oozie [ OK ] /var/run/zookeeper [ OK ] /var/run/spark [ OK ] /var/run/solr [ OK ] /var/run/hadoop-yarn [ OK ] /var/run/hive [ OK ] /var/run/hadoop-httpfs [ OK ] /var/run/hadoop-hdfs [ OK ] /var/run/kafka [ OK ] /var/run/hbase [ OK ] /var/run/webhcat [ OK ] ================Check for existing Ambari/IOP users================ A failure indicates that there are existing users related to Ambari or IOP that may cause issues /home/nagios [ OK ] /usr/iop [ OK ] /home/mapred [ OK ] /home/solr [ OK ] /home/kafka [ OK ] /home/hbase [ OK ] /home/hdfs [ OK ] /home/spark [ OK ] /home/ambari-qa [ OK ] /home/oozie [ OK ] /home/flume [ OK ] /home/yarn [ OK ] /home/hcat [ OK ] /home/hadoop [ OK ] /home/knox [ OK ] /home/sqoop [ OK ] /home/hive [ OK ] /home/zookeeper [ OK ] ==============Check for existing Ambari/IOP alternatives============== A failure indicates that there are existing alternatives related to Ambari or IOP that may cause issues /home/hadoop-conf [ OK ] /home/knox-conf [ OK ] /home/hbase-conf [ OK ] /home/sqoop-conf [ OK ] /home/oozie-tomcat-conf [ OK ] /home/zookeeper-conf [ OK ] /home/hive-conf [ OK ] /home/oozie-conf [ OK ] ==============Check that DNS and reverse DNS is configured============== A failure may indicate that there is a DNS issue due to the host name not being in the hosts file DNS configured [ OK ] ==============Check if SELinux is permanently disabled============== A failure indicates that SELinux is not permanently disabled SELinux disabled [ OK ] ========== Checking if the necessary software is already installed ========== A failure indicates that this machine does not have the specified recommended software wget installed [ OK ] rpm installed [ OK ] scp installed [ OK ] curl installed [ OK ] yum installed [ OK ] pdsh installed [ OK ] ====== Checking if there are running processes related to Ambari or IOP ====== A failure indicates that there is an Ambari or IOP related process running that should be stopped Below you will see the name and PID of the processes ================= Checking if there is a JDK installed ================= JDK version 1.7 or greater detected [ OK ] ================= Checking if python is installed ================= Python 2.6 or greater detected [ OK ] ============Running TextAnalytics precheck on localhost1.localdomain============ A port failure indicates that the specified port is in use but needs to be freed for TextAnalytics. An RPM failure indicates there is an installed RPM that should not exist. 32005 is free [ OK ] RPM found when using keyword: text-analytics [ OK ] 32050 is free [ OK ] 32000 is free [ OK ] ============Running BigR precheck on localhost1.localdomain============ An installed R failure indicates that R is not installed properly on current machine. An RPM error means that there is an installed BigR RPM that should not exist R is successfully installed [ OK ] RPM found when using keyword: bigr [ OK ] ============Running BigSheets precheck on localhost1.localdomain============ Checking for existing BigSheets rpms. A failure in the check indicates that there is an existing BigSheets rpm installed. Running and to see existing BigSheets rpms Checking for existing BigSheets directories and files. A failure indicates there is an existing BigSheets directory or file that may cause issues /usr/ibmpacks/current/bigsheets [ OK ] /var/lib/ambari-agent/cache/stacks/BigInsights/4.0/services/BIGSHEETS[ OK ] bigsheets dirs/files in /tmp [ OK ] RPM found when using keyword: IBM-bi* [ OK ] RPM found when using keyword: bigsheets-distrib [ OK ] Checking to make sure necessary ports are available. A failure indicates there is a port needed that is being used port free: pig [ OK ] port free: jetty stop [ OK ] port free: jetty UI [ OK ] port free: derby [ OK ] Checking to make sure that SSH works from master node to all other bigsheets nodes ==== Checking to make sure that passwordless ssh works from master host to all other hosts being checked ==== A failure indicates that passwordless ssh is not correctly set up from master host to specified host ============Running BigSQL precheck on localhost1.localdomain============ Cannot run BigSQL precheck. No BigSQL package found ============Precheck completed!============ Look at precheck results for all hosts at precheckResults.20151212_140704.log
Special gratitude to Anthony Groves for his help on this pre-installation check serviceability tool.