Task

This article will walk you through the steps required to deploy/setup HUE on IBM BigInsights version 4.0 and above.

Introduction

HUE or Hadoop User Experience is a Web interface for analyzing data with Apache Hadoop. With Big Data, you need a tool to navigate through your data, query your data and even search it. This is all tied up together in one place with HUE.

Pre-requisites

To deploy HUE on BigInsights, you need an up and running BigInsights Version 4.x cluster. For the purpose of this article, we can use BigInsights V4 Quickstart edition that is available on IBM website for free. You can download the Quick Start Edition here. It is assumed that your OS is redhat 6.x. If not, you will need to change the package installation commands as per your linux distro.

Install Dependencies

A couple of dependencies are required by HUE to run. So lets start with downloading the required packages. Launch terminal and download the required packages.


[root@rvm /]# yum install ant
[root@rvm /]# yum install python-devel.x86_64
[root@rvm /]# yum install krb5-devel.x86_64
[root@rvm /]# yum install krb5-libs.x86_64
[root@rvm /]# yum install libxml2.x86_64
[root@rvm /]# yum install python-lxml.x86_64
[root@rvm /]# yum install libxslt-devel.x86_64
[root@rvm /]# yum install mysql-devel.x86_64
[root@rvm /]# yum install openssl-devel.x86_64
[root@rvm /]# yum install libgsasl-devel.x86_64
[root@rvm /]# yum install sqlite-devel.x86_64
[root@rvm /]# yum install openldap-devel.x86_64

Download HUE

We will download the latest version of HUE as of today which is version 3.7.1 and extract it. In the terminal, run the following commands:


[root@rvm /]# wget https://dl.dropboxusercontent.com/u/730827/hue/releases/3.7.1/hue-3.7.1.tgz
[root@rvm /]# sudo echo “JAVA_HOME=\”/usr/lib/jvm/java-7-openjdk-1.7.0.75.x86_64/jre\”” >> /etc/environment
[root@rvm /]# tar zxvf hue-3.7.1.tgz

Add User And Group for HUE


[root@rvm /]# groupadd hue
[root@rvm /]# useradd hue -g hue
[root@rvm /]# passwd hue

Now give ownership of extracted hue folder to user hue by executing the following command.


[root@rvm /]# chown hue:hue hue-3.7.1

You will also need to add user hue to sudoers file as a sudoer.

Install HUE

1. As user hue, start the installation as shown below.
[root@rvm /]#sudo make install
make_install
2. By default, HUE installs to ‘/usr/local/hue’ in your Management node’s local filesystem as shown below. Make user hue, the owner of /usr/local/hue folder by executing
sudo chown –R hue:hue /usr/local/hue

Setting up hadoop properties for HUE

1. Configure properties in core-site.xml

i. Enable Webhdfs
Go to Ambari, select HDFS on the left side and then select config as shown.
hdfs_config
Then scroll down and make sure webdfs is check marked as shown below:
web_hdfs
ii. Add the following 2 properties under custom core-site.xml with value “*” as shown below:
core_site

2. Configure properties in oozie-site.xml

Just like above, now select oozie on the left side in Ambari and then select config.
i. Add two properties in oozie for HUE as shown below.
zookeeper_config

3. Configure properties in webcat-site.xml

Now navigate to Hive on left side in Ambari and then select config.
i. Keep scrolling down until you see webcat-site and add two properties in webhcat configuration for HUE as shown below:
webhcat_site

Configure HUE.ini file to point to your Hadoop cluster

– Go to /usr/local/hue/desktop/conf
– Start editing hue.ini using any editor(like vim) after making a backup file.
hue_ini

Note: In this article, the cluster is small-one node, therefore services like Hive Server, Hive Metastore, HBase Master, Zookeepers etc are deployed on one node itself. In case of bigger cluster, put the correct node information for the respective services that we are editing next. The screenshots below are just example to help you configure.

i. Edit Hdfs and webhdfs parameters to point to your cluster. Make the changes as shown. Don’t forget to uncomment these parameters after adding values.

hue_config_ini

ii. Configure YARN parameters and don’t forget to uncomment these parameters as shown:

hue_config_ini_2

iii. Configure Oozie, hive and hbase as show below. Don’t forget to uncomment the parameters.
hue_config_ini_3
hue_config_ini_4hue_config_ini_5

– Save all the changes.

Start HUE

– As hue user, go to /usr/local/hue/build/env folder and start HUE by executing ./supervisor as shown below
start_hue

Testing HUE

In your browser, go to
yourserver:8888/filebrowser
When prompted for userid/password, use user hue and its password that you created earlier to login.
You should see the following screen making sure that HUE is working properly.
demo_start

Conclusion

In this article we have successfully deployed HUE 3.7.1 on top of BigInsights V4.0 using Quick Start edition.This setup would allow an end user to browse/copy/delete HDFS files, fire queries to hive/hbase and even create a dashboard for data analysis. This interface can also be used as a front end for your enterprise search application powered by Solr.

1 comment on"HUE on IBM BigInsights 4.0 to explore Big Data"

  1. […] post was originally published on the IBM blog HUE on IBM BigInsights 4.0 to explore Big Data by Vinayak […]

Join The Discussion

Your email address will not be published. Required fields are marked *