Getting your hosts installed and setup in a domain using InfoSphere Streams 4.0 is very easy. This article will walk through installing the product to each host in your multi-host environment and setting up the InfoSphere Streams services to automatically restart when the host reboots after a host failure (i.e.  run the domain controller service as a system service).

In this article you will do the installation of all your hosts and then create a domain.

Before you begin:

  • Set up your Apache ZooKeeper server. You will need the ZooKeeper connection string. This walk through will use this connection string: myzookeeper1:9900,myzookeeper2:9900,myzookeeper3:9900. This is an external ZooKeeper with 3 nodes in the ensemble for high availability support.
  • You will need to run some of the commands as user root. If you do not have user root’s password you will have to ask your administrator to set up sudo. This article uses sudo when user root is required.

Installing to each host

Before you do the installation:

  • From the software distribution site, download the main installation package for your specific operating system and architecture. For example, if your target system is a 64-bit RHEL 6 system, you must download the InfoSphere Streams 64-bit RHEL 6 installation package.  Download the package  to the  host that you want to be your main management host. This is where you will typically be running commands and where you will do your development from. This document will refer to this host as the main management host.

Now you are ready to begin.

  • Extract the contents of the main installation package. Run this command from the directory where you stored the main installation package.

tar zxf tar-file-name.tar.gz

  • Change directory to StreamsInstallFiles directory created by the extraction.

cd StreamsInstallFiles

  • Run the dependency checker script to verify your host meets the software and system requirements. Fix any errors and warnings that are reported.

./dependency_checker.sh

  • Install the product by running the main  installation binary. This command will run the installation program in GUI mode if an XWindow System is available on your host. Otherwise this will run in console mode. You can install as a root user or non-root user. Follow the prompts on the panels taking all the defaults. You will be asked to select the edition you purchased.

./InfoSphereStreamsSetup.bin

Note: On the Post Installation panel leave the Open Domain Manager checkbox checked. This will open the Domain Manager which you will use later to complete the setup.

  • Setup the ZooKeeper environment variable. This is so you don’t have to continuously enter in the connection string. You can add this command to your .bashrc file or /etc/profile.d or run this from a command prompt. Replace myzookeeper1:9900,myzookeeper2:9900,myzookeeper3:9900 with your ZooKeeper connection string.

export STREAMS_ZKCONNECT=myzookeeper1:9900,myzookeeper2:9900,myzookeeper3:9900

  • Setup the InfoSphere Streams environment variables. You can add this command to your .bashrc file or /etc/profile.d or run this from a command prompt.

source installation-directory/version/bin/streamsprofile.sh

  •  Set up the main management host so that the InfoSphere Streams services can automatically recover from a host failure. This command is required to run as user root.

sudo -E $STREAMS_INSTALL/bin/streamtool registerdomainhost -d <domain-id>

  • Create the domain host installation package. This  package will be used to setup other hosts in your environment. This package contains a subset of the main installation package. You will need to specify a domain name but the domain does not need to exist. We don’t need to specify --zkconnect parameter because we set STREAMS_ZKCONNECT environment variable. The domain host installation package will be created in the current directory.  The default installation location will be /opt/ibm/InfoSphere_Streams and the owner and group will be derived from the current installation.  You can override these values by specifying the corresponding parameters. For information on all of the parameters see the man pages:  streamtool man mkhostpkg.

 streamtool mkhostpkg -d domain-id

  • You must run the following on each additional host in your environment (excluding the main management host since you already installed the product there). In this example we are going to run this on f0401v25 and f0401v23.

♦ Copy the domain host package to each host.

♦  Extract the domain host package contents.

tar xf tar-file-name.tar

♦  Change directory to StreamsDomainHost directory created by the extraction.

cd StreamsDomainHost

♦  Verify your host meets the software and system requirements. Fix any issues that are reported. Some of the RPM dependencies are shipped in the main installation package.

./dependency_checker.sh

♦  Run the following script. This will install the domain host installation, register the host with the domain, create the system service, and start the system service. You must do this step as user root or using sudo.

sudo ./streamsdomainhostsetup.sh

♦  This host will be available to participate in the domain.

  • The hosts are now installed and the InfoSphere Streams domain controller service is running as a system service.

 

Creating a domain and tagging the hosts

You will continue with the setup by creating a domain with the domain-id used in the above steps. When you create the domain, you will see that the hosts that you did the setup on are now in the domain. You will complete the rest of the setup using the Domain Manager and Streams Console application.

Before you begin:

  • Your administrator will need to setup an authentication mechanism, this will be either LDAP or PAM.
    • If you are using PAM authentication no additional connection information is needed. A highly available solution is to use PAM over LDAP.  If you plan to use other PAM options you should review the requirements in  http://www-01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.cfg.doc/streamtool/doc/setting-up-enterprise-domain-pam.html.
    • If you are using LDAP authentication you will have to gather the following information.
      • LDAP server URL
      • LDAP user DN pattern
      • LDAP user ID
      • LDAP password
      • LDAP user secondary lookup (if required by your environment)
      • LDAP group object (if required by your environment)
      • LDAP group attribute (if required by your environment)
      • LDAP user attribute (if required by your environment)

Now you are ready to create the domain and assign host tags. You will run this setup from the main management host.

  • Setup the InfoSphere Streams environment variables. You can add this command to your .bashrc file or /etc/profile.d or run this from a command prompt.

source installation-directory/version/bin/streamsprofile.sh

  • Open the Domain Manager. If the Domain Manager is not already opened,  enter the following command.

streamtool launch --domainmgr

You can also run the installation-directory/version/DomainManager.sh script. This script wraps the source streamsprofile.sh and the streamtool launch --domainmgr command.

When Domain Manager is first launched you will be prompted to create a domain. Select to create an enterprise PAM or enterprise LDAP domain depending on your authentication mechanism. In this example we are going to select to create an enterprise LDAP domain.

CreateDomain

  • On the Create Enterprise Domain (LDAP) panel complete the prompts and click Create. After the domain is created, the domain will be started. When the domain is starting, you will be prompted to authenticate to the domain. You should enter the domain owner’s LDAP user ID and password, just like you entered it on the create panel.
Create LDAP enterprise domain page 1

 

DMcreateldap2
Create LDAP enterprise domain page 2
  • After the domain is created,  the list of domains that exist in the ZooKeeper connection you are using will be shown in the list. All domains created using this same ZooKeeper connection will be shown in the list.

Next you will set the high availability count on the domain to be at least  2. This will allow for redundant domain services to be started. To edit domain properties click on the Show or edit properties in the center pane. You may have to scroll down to see it. This will display the domain properties in the third pane. For more information about the domain high availability count see http://www-01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.cfg.doc/doc/hacount-domain.html

DM-Editproperties

  • To edit the property, scroll down to the Domain category and find the highAvailabilityCount property. Double click on the property value to open an editor. You should change this value to match the number of hosts you want to run domain management services. In this example we will change this value to 2 and press enter. This will change the property immediately.

DM-changeDomainPropertyHAcoutn

  • The rest of the set up will be completed in the Streams Console. The first box in the third pane has a list of what you can do to complete the setup in the Streams Console. Select the domain just created and click on the Open InfoSphere Streams Console button.

DMListHighlightonly

  • The first thing we have to do is to tag the hosts. Select the RESOURCES control. Then click on the Manage Resources and Tags menu item. As you can see, the hosts that we already set up in the Install to each host section are already added to the domain.

ResourceAndTag

addtags

  • Your domain is now setup and ready to use.
  • To verify the InfoSphere Streams services will be restarted when a host reboots, run the streamtool getdomainstate command with -l (long format) from the main management host. Look for the text highlighted in the example below. In this example all the hosts are running the domain controller as a system service.

getdomainstateexample

Documentation links

For considerations on setting up a multi-host environment see: http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install.doc/doc/ibminfospherestreams-install-options-configuration.html.

For more information on setting up the enterprise domain and setting up the domain controller to run as a Linux system service see: http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.cfg.doc/doc/setting-up-enterprise-domain.html.

For information about system and software dependencies see: http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install.doc/doc/ibminfospherestreams-install-prerequisites.html.

For information about installing the main installation package, see: http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install.doc/doc/ibminfospherestreams-install.html.

 

Additional commands:
The following streamtool commands will help you manage your InfoSphere Streams system service:
  • registerdomainhost – registers a host with a domain, and installs the domain controller service as a system service.
  • stopdomainhost – stops the domain controller system service on a host.
  • startdomainhost – starts the domain controller system service on a host.
  • unregisterdomainhost – removes a resource from the domain, removes the domain controller system service.
  • getdomainhoststatus –  returns the status of the domain controller system service on the host.

For information on streamtool commands, run streamtool man <command> or see: http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.ea.doc/streamtool/doc/streamtool.html.

3 Comments on "Multi-host environment: Installing to each host and setting up a domain"

  1. DanDebrunner April 03, 2015

    Great article! Easy to follow to install Streams using the domain host package.

    Minor point that the ‘streamtool getdomainstate’ can only be executed once the domain is created and running.

    • Thanks Dan. You are correct the ‘streamtool getdomainstate’ can only be run when the domain is created and running. The article was updated to move that step after the domain is created and started.

  2. Naveen_Dronavalli June 02, 2015

    Fantastic! Thank you Laurie.

Join The Discussion