IBM Open Platform (IOP) is a 100% open source platform. It uses Ambari for provisioning and managing Hadoop clusters.

Apache Knox gateway is used in IOP in providing perimeter security. It provides a single point of authentication and access for Hadoop cluster services. It can integrate with Kerberos and provides the following features:

  • Single REST API access point
  • Centralized authentication, authorization and audit for Hadoop REST/HTTP services
  • LDAP/Active Directory authentication, authorization and audit
  • Eliminate SSH edge node risks
  • Hides Network topology


This technical document is intended to show viewers step by step instructions on setting up LDAP with Ambari, Knox, and BigInsights v4.x.

Version Tested

  • IOP v4.0.0.0 (Ambari v1.7, Knox v0.5.0)
  • BigInsights v4.0.0.1 (FP1)
  • OpenLDAP v2.4.39

LDAP / Ambari Setup Steps

  1. Collect the LDAP server information
    • LDAP server FQDN. Example shown in this post is:
    • LDAP server port. Usually default is 389. Ex. 389
    • LDAP server BaseDN. (Currently does not support case sensitive LDAP entries) Ex. dc=ibm,dc=com
    • LDAP server Admin account and its credential. Ex. Account: cn=Manager,dc=ibm,dc=com
  2. Install LDAP client on all Hadoop cluster nodes
    • Run the attached script “”. Usage: $ sudo 389 dc=ibm,dc=com   **Make sure there is no spaces in BaseDN parameter value. (ie. dc=ibm,dc=com)
  3. Test LDAP client install and connectivity to the server
    • Login to Ambari node as root
    • Run connectivity test: Ex. $ sudo openssl s_client -connect
      Note: If connectivity test failed, you should verify if LDAP server’s firewall is opened for port 389 and 636 (Ex. $ sudo netstat -anp|grep 636)
    • Test if a LDAP user can login to Ambari node without an existing local Unix account
      1. $ ssh
      2. You may get an error message indicating that command failed to change home directory for ldap user. This is a correct behavior since ldap user does not exist on the local machine.
      3. If you get the following status, you have successfully logged in using LDAP user id and it’s credential. You are done setting up LDAP client and successfully tested connections. Now you are ready to move on to configure Ambari LDAP.
        ldap user login test
        LDAP User Login Test
  4. Setup Ambari LDAP configuration
    1. Login to Ambari node as user “root”
    2. Either using CLI or direct modify Ambari property file for advanced users. Here is showing via CLI commands.
      • $ sudo ambari-server setup-ldap
      • Answer each question from the prompt. Notice I chose anonymous bind as false so no users can login anonymously.
      • Once you save the settings, the CLI program saves the parameter values into file “/etc/ambari-server/conf/”
        ambari ldap setup
        Ambari LDAP Setup
    3. Restart Ambari process. Ex. $ sudo ambari-server restart
    4. Sync up LDAP users into Ambari metadata store so that Ambari Web UI can allow LDAP users to login.
      ** Be careful of this step. In the example is showing sync all users. You do not want to do this in a production large enterprise environment, you need to narrow it down to a selected targeted users or groups. Ex. $ sudo ambari-server sync-ldap –all (notice there is a double dashes before word “all”)

      ambari ldap sync
      Ambari LDAP Sync Up
  5. LDAP User Test in Ambari
    Now let’s test Ambari Web UI login as a LDAP user. (Screen shot below). Notice that Ambari Admin user needs to assign appropriate permission to each ldap user before s/he can see any screens when they logging in. If not, s/he still can login but will see a blank page.Screen shot #1:Notice all ldap users showed up after sync command. None of those users exist on current cluster nodes nor do you need to create them on cluster servers.

    ambari ldap user list
    Ambari LDAP User List

    Screen shot #2:Ambari administrator must assign permissions to the ldap users.

    ambari assign user permission
    Ambari Assign User Permission

    Screen shot #3:When Ambari Admin assigned LDAP users operator permission. S/He can then login and act as a Hadoop admin to monitoring and maintaining the cluster.

    ambari hadoop admin
    Ambari Hadoop Admin

LDAP / Knox Setup Steps

  1. Login to Ambari Web UI as administrator. (ie. “http://<ambari-server>:8080” default port is 8080. You can change the port in file)
  2. Click on “Knox” from left panel. Select “Configs” tab from right panel.
  3. Extend the section “Advanced topology”, and modify the following entries:
    • main.ldapRealm.userDnTemplate=uid={0},ou=people,dc=ibm,dc=com (modify to match with your ldap server settings)
    • main.ldapRealm.contextFactory.url=ldap://
    knox ldap setup
    Knox LDAP Setup
  4. Click on “Save” button from the upper right corner
  5. Restart “Knox” service via Ambari Web UI

BigInsights v4.x — Testing Knox Application LDAP Login with BigInsights Home Page

    1. URL: https://<biginsights_home_FQDN>:8443/gateway/default/BigInsightsWeb/index.html
    2. Login as a LDAP user ID and its password
BigInsights Home Page with LDAP User Login
BigInsights Home Page with LDAP User Login

Content of

# File name:
# Version: 3.0
# Last Updated Date: 6/2/2015
if [ $# -ne 3 ]
  echo ""
  echo "Usage:   <ldap-server-FQDN> <ldap-server-port> <BASE_DN>"
  echo "Example: 389 dc=ibm,dc=com"
  echo ""
  exit 1

yum install openldap-clients sssd perl-LDAP.noarch nss-pam-ldapd -y

cat > /etc/openldap/ldap.conf << EOFDELIM
TLS_CACERTDIR /etc/openldap/certs

sed -i "s/LDAP_SERVER/$1/" /etc/openldap/ldap.conf
sed -i "s/PORT/$2/" /etc/openldap/ldap.conf
sed -i "s/BASEDN/$3/" /etc/openldap/ldap.conf

cat > /etc/nslcd.conf << EOFDELIM
ssl no
tls_cacertdir /etc/openldap/cacerts

sed -i "s/LDAP_SERVER/$1/" /etc/nslcd.conf
sed -i "s/PORT/$2/" /etc/nslcd.conf
sed -i "s/BASEDN/$3/" /etc/nslcd.conf

cat > /etc/pam_ldap.conf << EOFDELIM
ssl no
tls_cacertdir /etc/openldap/cacerts
pam_password md5

sed -i "s/LDAP_SERVER/$1/" /etc/pam_ldap.conf
sed -i "s/PORT/$2/" /etc/pam_ldap.conf
sed -i "s/BASEDN/$3/" /etc/pam_ldap.conf

cat > /etc/pam.d/system-auth << EOFDELIM
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth        required
auth        sufficient nullok try_first_pass
auth        requisite uid >= 500 quiet
auth        required
auth        sufficient use_first_pass

account     required
account     [default=bad success=ok user_unknown=ignore]
account     sufficient
account     sufficient uid < 500 quiet
account     required

password    requisite try_first_pass retry=3 type=
password    sufficient md5 shadow nullok try_first_pass use_authtok
password    required

password    sufficient use_authtok

session     optional revoke
session     required
session     [success=1 default=ignore] service in crond quiet use_uid
session     required
session     optional
# add if you need ( create home directory automatically if it's none )
session     optional skel=/etc/skel umask=077

cat > /etc/nsswitch.conf << EOFDELIM
passwd:     files ldap
shadow:     files ldap
group:      files ldap

#hosts:     db files nisplus nis dns
hosts:      files dns

# Example - obey only what nisplus tells us...
#services:   nisplus [NOTFOUND=return] files
#networks:   nisplus [NOTFOUND=return] files
#protocols:  nisplus [NOTFOUND=return] files
#rpc:        nisplus [NOTFOUND=return] files
#ethers:     nisplus [NOTFOUND=return] files
#netmasks:   nisplus [NOTFOUND=return] files

bootparams: nisplus [NOTFOUND=return] files

ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files

netgroup:   ldap

publickey:  nisplus

automount:  files ldap
aliases:    files nisplus

cat > /etc/sysconfig/authconfig << EOFDELIM

service nslcd start
chkconfig nslcd on

echo 'Welcome to LDAP World!!'


3 comments on"LDAP Integration for IOP and BigInsights"

  1. Thanks for your notification on script errors! -Linda Liu

  2. […] how to configure LDAP integration with IBM Open Platform on a BigInsights Cluster. Here is the link.In this post, it concentrates on the missing content from the previous post for the Microsoft […]

  3. […] how to configure LDAP integration with IBM Open Platform on a BigInsights Cluster. Here is the link.In this post, it concentrates on the missing content from the previous post for the Microsoft […]

Join The Discussion

Your email address will not be published. Required fields are marked *