How to integrate LDAP with Apache Knox

Apache Knox uses Apache Shiro provider for authentication. The Shiro provider defined in the gateway topology can use either LDAP realm or PAM realm to authenticate a user against directory services.

LDAP Configurations

The following LDAP specific configuration is added to the Shiro provider. These parameters work with both openLDAP and AD.

Basic LDAP Configurations

The example below will work for direct distinguished name (DN) of any username provided by client. If the LDAP DIT is complex where users can belong to different branches then use search base parameters described under Advanced LDAP configurations.

Parameter Value
main.ldapRealm org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm
main.ldapRealm.contextFactory.url ldap:// <ldap_host_name>:389ldaps:// <ldap_host_name>:636
main.ldapRealm.userDnTemplate uid={0},ou=people,dc=hadoop,dc=apache,dc=org (openLDAP)

cn={0},ou=people,dc=hadoop,dc=apache,dc=org (AD)

main.ldapRealm.contextFactory.authenticationMechanism Simple
urls./** authcBasic


ldapRealm: Shiro realm org.apache.shiro.realm.ldap.JndiLdapRealm can also be used for basic configurations but authorization is disabled by default. For advanced configurations and support for authorization, caching, ldap group search, it should be replaced by org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm.

contextFactory.url: This is the LDAP server url and port. If LDAP is used with SSL, the protocol will change to ldaps and ssl port where ldap server is listening.

main.ldapRealm.userDnTemplate: This template expects distinguished name of a user. uid parameter can also include a specific user by replacing {0} with an actual username. User DN parameter above is based on user.ldif configuration defined for default Apache DS demo LDAP that comes with Apache Knox and will have to be changed depending on LDAP DIT that is being used. Active directory expects cn={0} rather than uid={0}.

authenticationMechanism: Apache Knox supports only simple authentication.

urls/**:Same Shiro filter will be used for all paths into the application.

Advanced LDAP Configurations

  1. User search base: A user DN can be created based on LDAP search rather than providing direct user DN. These parameters should replace the userDnTemplate parameter. If both are specified in the topology file Knox will give precedence to search base parameters over userDnTemplate.
Parameter Value
main.ldapRealm.searchBase ou=people,dc=hadoop,dc=apache,dc=org
main.ldapRealm.userSearchAttributeName sAMAccountName
main.ldapRealm.userObjectClass Person

userSearchBase: This is a mandatory field to be used along with the search attribute. Users belonging to this subset of users will be searched. The value for searchBase can be as narrow as possible to avoid performance issue resulting from lot of records returned due to broad search. Number of records returned will follow the individual LDAP limit.

userSearchAttributeName: Attribute from the user DN that will be searched. Any of the attribute that forms a user DN can be used here like uid, email, sAMAccountName etc. This need not be part of bindDN used to search user.

userObjectClass: This is optional parameter to indicate the type of user object. This information can be found in the user record.

  1. Authentication Caching: Knox uses inbuilt Shiro caching mechanism to store authentication information.
Parameter Value
main.cacheManager org.apache.shiro.cache.ehcache.EhCacheManager
main.securityManager.cacheManager $cacheManager
main.ldapRealm.authenticationCachingEnabled True


  1. Group Lookup: This allows Knox to look up group information for the authenticated user. Group lookup is used by Knox service level authorization rules to limit access to services. The minimum parameters required for group lookup are as follows:
Parameter Value
main.ldapRealm.userSearchBase ou=people,dc=hadoop,dc=apache,dc=org
main.ldapRealm.userSearchAttributeName sAMAccountName
main.ldapRealm.userObjectClass Person
main.ldapRealm.contextFactory.systemUsername <system username>
main.ldapRealm.contextFactory.systemPassword <system user password>
main.ldapRealm.groupSearchBase Ou=groups,dc=hadoop,dc=apache,dc=org
main.ldapRealm.groupIdAttribute sAMAccountName
main.ldapRealm.memberAttribute member


userSearchBase: This will replace searchBase when used in conjunction with group lookup.

userSearchAttributeName: also needs to be defined and behavior is same as previous example.

systemUsername: System user DN.

systemPassword: For testing, the system user’s password can be directly provided in the topology file but for production the password needs to be stored in credential store. In that case, value will change to ${ALIAS=ldcSystemPassword}.

groupSearchBase: searchBase used to search for group.

groupIdAttribute: Attribute from the group DN that will be searched.

memberAttribute:  Search will be limited to information provided by this attribute.


PAM configurations

Other LDAP servers like Tivoli AD can use PAM authentication to authenticate user with Knox. To integrate LDAP server with Knox using PAM authentication, make the following changes.

  1. Copy /usr/iop/current/knox-server/conf/knox-pam-ldap-service to /etc/pam.d
  2. Add following parameters instead of any ldapRealm parameters in advanced Knox topology section from Ambari Web UI.
Parameter Value
main.pamRealm org.apache.hadoop.gateway.shirorealm.KnoxPamRealm
main.pamRealm.service knox-pam-ldap-service


Testing Authentication

  1. Before making changes to Knox topology, check if LDAP server host name, port is correct. From the command line, run the followin
    ldapwhoami -h <ldap_host> -p <ldap_port> -x -D ‘<user_DN>’ -w ‘<password>’
  2. If using userDnTemplate, to make sure user DN is correct
    ldapsearch -h <ldap_host> -p <ldap_port> -x -D ‘<user_DN>’ -w ‘<password>’ –b ‘<user_DN>’
  3. If using user search base, to check searchbase, searchAttributeName and objectClass is correct
    ldapsearch -h <ldap_host> -p <ldap_port> -x -D ‘<user_DN>’ -w ‘<password>’ –b ‘<user_search_base>’ ‘(objectClass=person)’ sAMAccountName
  4. Once the LDAP/PAM configurations are added to the Knox topology file, restart Knox server. The following command can be run from command line to get contents of /tmp directory. If user cannot be authenticated, HTTP 401 unauthorized user error will be returned.
    curl –iku <ldap_user>:<ldap_passwd> -x GET https://<knox-gateway url>:<port>/gateway/default/webhdfs/v1/tmp?op=LISTSTATUS
  5. To test authentication caching, use default demoLDAP server. Make sure to Knox topology has demoLDAP details and caching parameters. Restart Knox. After stopping demoLDAP, the above curl command should still show result.
  1. To install openldap client to run the above open ldap commands:
    yum install  openldap-clients –y

2. ldapsearch and ldapwhoami will work with both OpenLDAP/Active Directory.

3. Active Directory can also use dsquery tool to search users from Windows server 2008 command prompt with Run as administrator option.

  •  dsquery user <user search dn> -name <username>
  •  dsquery group <group search dn> -name <groupname>
  •  dsquery ou <domain>

4. To store system password in the credential store, following command can be used.
/usr/iop/current/knox-server/bin/ create-alias ldcSystemPassword –cluster <topology_name> –value <system_password>


Join The Discussion

Your email address will not be published. Required fields are marked *