With the advent of Docker container support in IBM Spectrum LSF 10.1, it’s become much easier to build and maintain environments for containerized workloads. This blog will explorer using LSF Application Profiles with static Docker container options and then how to add dynamic Docker container options via scripting. More specifically, you will learn how to dynamically add the user’s secondary LINUX user groups as Docker container run time options.

Prerequisites

To start with you will need the following:

Component Version Edition
IBM Spectrum LSF 10.1.0.3+ Standard Edition or Suite
Docker 17.03+ Community or Enterprise Edition
Docker Engine 1.12+

Verify your Docker Engine version with this command:

$ docker version | grep API

LSF Configuration

This blog assumes you have installed IBM Spectrum LSF and Docker and both are up and running on nodes in your cluster.

Prepare LSF to run jobs in Docker containers:

Follow steps in this URL: https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_docker/lsf_docker_prepare.html

Configure a test LSF Application Profile

  • Create a new LSF Application Profile
  • Next, we define a Docker application profile in the lsb.applications configuration file. The lsb.application file is located in $LSF_ENVDIR/lsbatch/<your_cluster_name>/configdir.

    Begin Application
    NAME = dockerapp1
    CONTAINER = docker[image(ubuntu:latest) options(–rm –net=host –ipc=host ) \
                -v /etc/passwd:/etc/passwd \
                -v /etc/group:/etc/group \
                starter(root)] \
    DESCRIPTION = Test Docker Application Profile 1
    End Application

    Note, the above example assumes you are using only local passwords and groups.

  • Reconfigure the LSF Master batch daemon:
  • To add or modify LSF Application Profiles reconfigure or restart the LSF Master Batch daemon on the LSF Master node as the LSF Administrator.

    $ badmin reconfig

    Additional information is available here: https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_docker/lsf_docker_config.html

    Running a Test Job

    Use the bsub -app option to tell LSF to run a job using a particular application. In our case the container is called “dockerapp1“, so run something like this:

    $ bsub -app dockerapp1 -I echo Hello World
    Job <760> is submitted to default queue <interactive>.
    <<Waiting for dispatch …>>
    <<Starting on gauss02>>
    Hello World
    $

    Note, if the Docker image specified in the Application Profile is not on the LSF compute node, it will be pulled down to the compute node. Please consider setting up on your own Docker Registry so as to pull Docker images from your local Docker Registry.

    Here is an example job that outputs the version of Ubuntu:

    $ bsub -app dockerapp1 -I cat /etc/lsb-release
    Job <761> is submitted to default queue <interactive>.
    <<Waiting for dispatch …>>
    <<Starting on gauss02>>
    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=18.04
    DISTRIB_CODENAME=bionic
    DISTRIB_DESCRIPTION=”Ubuntu 18.04 LTS”
    $

    Create a script to generate the Docker options

    Create the script below and call the script docker_options.sh. You will need to place the script on every LSF node in the same directory path or ideally place the script into a shared directory path that is accessible across all LSF nodes. For this example, we will assume the shared directory path is /shared_directory_path/scripts. Additionally, the script must be executable by the LSF Administrator user, which is typically lsfadmin.

    Refer to LSF documentation for more details: https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_config_ref/lsb.applications.5.html

    /shared_directory_path/scripts/docker_options.sh

    #!/bin/sh
    #
    # Example script to add Docker run options to an LSF job
    #
    # This script outputs the LSF job’s user’s groups.
    #
    # Example output when user has groups of 1100 and 1200:
    # –group-add 1100 –group-add 1200
    #
    # Options should be written to standard output on a single line
    #

    IDCMD=”id $LSFUSER”

    # Add groups
    for GID2 in `$IDCMD -G`
    do
       echo -n ” –group-add $GID2″
    done

    # add newline to standard output
    echo

    Configure a second test LSF Application Profile with the Docker options script

  • Add the LSF Application Profile below to lsb.applications file.
  • Begin Application
    NAME = dockerapp2
    CONTAINER = docker[image(ubuntu:latest) options(–rm –net=host –ipc=host ) \
                -v /etc/passwd:/etc/passwd \
                -v /etc/group:/etc/group \
                -v /shared_directory_path/scripts:/shared_directory_path/scripts \
                @/shared_directory_path/scripts/docker_options.sh \
                starter(root)] \
    DESCRIPTION = Test Docker Application Profile 2
    End Application

    Please change the /shared_directory_path/scripts above to the location where you placed the docker_options.sh script.

  • Reconfigure the LSF Master batch daemon:
  • To add or modify LSF Application Profiles reconfigure or restart the LSF Master Batch daemon on the LSF Master node as the LSF Administrator.

    $ badmin reconfig

    Test the solution

    First, you will need to login as a user that belongs to more than one group and then you can perform the tests below. Here is an example testing the two different application profiles created:

    Verify your user id has multiple groups.

    $ id -G
    1100 1200
    $

    The first application profile should only be aware of your first group.

    $ bsub -app dockerapp1 -I id -G
    Job <762> is submitted to default queue <interactive>.
    <<Waiting for dispatch …>>
    <<Starting on gauss02>>
    1100
    $

    The second application profile should show all your groups.

    $ bsub -app dockerapp2 -I id -G
    Job <763> is submitted to default queue <interactive>.
    <<Waiting for dispatch …>>
    <<Starting on gauss02>>
    1100 1200
    $

    Conclusion

    You have seen how to create a simple LSF Application Profile with static Docker container options and then create another LSF Application Profile to add dynamic Docker container options at run time via a script.

    Join The Discussion

    Your email address will not be published. Required fields are marked *