This workshop walks you through extending your Etherpad web application with a hosted database service on Bluemix. The MySQL service was chosen for this scenario. Arch-Ref-Diagram-Etherpad-Lab2 Contents:

Before you begin

This workshop is written for Ubuntu 14 and Mac OSX development workstations because it’s easy to set up the required tools on those platforms. If your development workstation is running another operating system (such as Windows), you should either obtain an additional server running Ubuntu or run Ubuntu in a virtual machine on your development workstation. Important: Before proceeding with this workshop, you will need to have completed Workshop 1 in this series. In this workshop we will be mixing the use of Bluemix apps along with containers. These are two separate conceptual ideas for executing code inside of Bluemix today. The apps closely map to the Cloud Foundry App model, while containers are simply Docker Containers running on Bluemix. Eventually, we will be able to bind container instances to apps and take advantage of all the native integration points available to apps and services in Bluemix today. However, based on the experimental status of the IBM Containers service, that is not available across all users’ IBM Containers service interfaces. Therefore in this workshop and other workshops, we will be creating an App which we bind our services to for instantiation. We will then reference that service instance directly from the container during the build and execution stages. Our created apps will simply be placeholders for our services in this workshop series. In the future, we will be able to take advantage of the platform integration between apps and containers, leveraging an environment variable in our Docker images and Dockerfiles to retrieve this same information dynamically.

Task 1. Create your MySQL service instance

For this workshop, we’ll be using a partner service to provide us with a managed database-as-a-service, that we can use as our backend repository for the Etherpad data.
  1. Log in to Bluemix. The dashboard opens, as shown:
    Bluemix Dashboard
    Bluemix Dashboard
    The dashboard shows an overview of the active Bluemix space for your organization.
  2. Click the option to CREATE AN APP.
  3. You will then be prompted to whether you are creating a MOBILE or WEB application. Select WEB.
  4. You will then be prompted for the runtime of your application. For this workshop, you can choose any runtime, as we are only using the application to aggregate our services. Future workshops will deploy application contents to the necessary runtimes. We chose SDK for Node.js as the runtime for this walkthrough. Click CONTINUE. Note: As mentioned in Before you begin, the apps will only be placeholders for the services you are creating in this Workshop series, and will have no direct correlation via the Bluemix interface to the containers we deploy. We will be directly linking the containers we run to the services we create via configuration files in the running containers in these exercises.
  5. Give your application a meaningful name such as etherpad-db-app and click FINISH. You can see your new application in the Dashboard view. Click it for details.
  6. Now we will create a new service instance and bind it to the application. Click Overview in the left navigation menu. Click ADD A SERVICE OR API on the Overview page.
  7. Scroll to the Data Management section and click the ClearDB MySQL Database service.
  8. Accept all the defaults and click CREATE.
  9. You may be asked to restage your application. If so, click RESTAGE and wait the few seconds for your app to restage.
  10. Once your page has refreshed and your app has restaged, you now have a managed MySQL datastore ready for your use.
    Managed MySQL datastore ready for your use
    Managed MySQL datastore ready for your use

Task 2. Configuring your application to use the MySQL service

We now have a managed MySQL Database service which we can connect to from our Etherpad application. We will configure our application via a settings file to point to the new persistent datastore instead of using a local, volatile file on the Container filesystem. Tip: On Linux, you will need to begin each IBM Containers Extension (ICE) command. below with sudo ice, followed by the rest of the command. On Mac OSX, you should not use the sudo command, begin each command with ice followed by the rest of the command.
  1. From your application details screen, click Show Credentials under your ClearDB MySQL Database.
    The credentials for accessing the MySQL Database
    The credentials for accessing the MySQL Database
    Make note of name (the generated database name), hostname, port, username, and password. All of these fields have been generated for you automatically on the database service already. You will now use these values to update the settings.json file and redeploy the application.
  2. In a text editor, open the etherpad-lite/settings.json file and find the lines starting with “dbType” : “dirty” (around line 38). We will need to replace these configuration options with the options from the credentials panel in Bluemix.
    Configuration options - before the changes
    Configuration options – before the changes
    1. Comment out or delete the lines highlighted above. Uncomment the lines surrounding the “dbType” : “mysql” block.
    2. Copy over your username field from the Bluemix panel into the user field, replace __username__. Replace the underscores as well, as they are only shown for placeholder text.
    3. Copy over your hostname field from the Bluemix panel into the host field, replacing __hostname__. Replace the underscores as well, as they are only shown for placeholder text.
    4. Copy over your password field from the Bluemix panel into the password field, replacing __password__. Replace the underscores as well, as they are only shown for placeholder text.
    5. Copy over your name field found under the credentials field from the Bluemix panel into the database field, replacing __database__. Replace the underscores as well, as they are only shown for placeholder text.
    Save your newly updated settings.json. You should have a settings.json that resembles the following.
    Configuration options - after the changes
    Configuration options – after the changes
Note: The implementation method here is not as flexible as it could be. We are hard-coding the service properties into the Docker image when we build it. Eventually, we will be able to dynamically bind any service properties to the Docker image when it first boots and will not need to hard code service properties. However, as mentioned previously, the capability to bind containers to apps for this capability is not yet globally available, due to the experimental state of the IBM Containers service. This workshop will be updated when this capability is available in later releases of the IBM Containers service.

Task 3. Build the image for the application

This section is similar to Task 4. Build the image for the application in Workshop 1, however we will be using different image names, since we are leveraging an additional MySQL service. Tip: On Linux you will normally need to start the IBM Containers Extension (ICE) commands below with sudo ice. On Mac OSX, you should not use the sudo command.
  1. Log in to the ICE command line tool:
    ice login
  2. Build your container using the following commands in succession, from the etherpad-lite directory. You will need to use the value of your own Registry URL in the following commands, found in the container launch page that opens after clicking CREATE A CONTAINER button on the Bluemix dashboard. The first command issued will pull the latest official IBM Node.js container from the global IBM repository.
    ice --local pull registry.ng.bluemix.net/ibmnode:latest
    The second command issued will build our application container, against that pulled latest image and with our Dockerfile and settings.json that were extracted from the Workshops GIT repository. The -t parameter provides a tag that will automatically be applied to the built image, so you can reference it locally. The trailing period (.) is important to make sure we specify the current directory (the cloned etherpad-lite directory) is the working directory for Docker to build our image from.
    • Format:
      ice --local build -t YOUR_LOCAL_IMAGE_NAME .
    • Example:
      ice --local build -t etherpad_bluemix_mysql .
    Finally, we will tag our image into a repository with a format that will allow us to publish our built image to Bluemix and run it on the IBM Containers service. This command takes your local image that was tagged previously during the build command and applies a repository-specific tag to it, so it can be correctly placed in the IBM Containers service shared repository. This format is required and you will not be able to successfully push an image to Bluemix if you use another image name format that does not match.
    • Format:
      ice --local tag -f YOUR_LOCAL_IMAGE_NAME registry.ng.bluemix.net/ YOUR_REGISTRY_URL / YOUR_REMOTE_IMAGE_NAME
    • Example:
      ice --local tag -f etherpad_bluemix_mysql registry.ng.bluemix.net/bluemix_residency/etherpad_bluemix_mysql
    Your local and remote image names do not have to be the same, but it does help for clarity when comparing local and remote image lists. Also, you can tag your initial image with the full repository tag during the build step instead, but it helps to use shorter tags when running locally to save time and apply the Bluemix-required format before pushing your image to Bluemix. In order, these commands would appear as follows, replacing the necessary formatted parts with your organization and images information:
    ice --local pull registry.ng.bluemix.net/ibmnode:latest
    ice --local build -t etherpad_bluemix_mysql .
    ice --local tag -f etherpad_bluemix_mysql registry.ng.bluemix.net/bluemix_residency/etherpad_bluemix_mysql

Task 4. Test the application on your workstation

Now that our container is built, we can try testing our Etherpad web application locally.
  1. The following command will start a Docker container and run Etherpad in it. It will also print out a container ID (CID) that you will need for the next step:
    docker run -d -p 9080:9080 etherpad_bluemix_mysql
  2. The next step is to find the URL to type into your browser to see the running Etherpad web application. On Linux operating systems, you'll need to get the IP address of the Docker container itself if you are running in a Linux Desktop environment. In the command below, replace CID with the container ID from the docker run command above:
    docker inspect --format '{{ .NetworkSettings.IPAddress }}' CID
  3. If you are running a Linux Server environment inside a virtual machine, you will need to configure port-forwarding for your virtualization platform to access Etherpad running on the container from your host machine. This will vary from platform to platform, but forwarding port 9080 on your host machine to the internal virtual machine guest is usually all that is needed. An example of a VirtualBox Port Forwarding configuration is shown below.
    Port Forwarding Rules window
    Port Forwarding Rules window
    Similarly, a virtual machine running on VMWare would require a port forwarding rule as well, but the Guest IP value would be the IP address associated with the VMWare 8 network. On Mac OSX, boot2docker is running Docker in a virtual machine, so you need the IP address of boot2docker (usually 192.168.59.103). Run this command to get the IP address of boot2docker:
    boot2dockerip
  4. We're hosting Etherpad on port 9080, so to test your running Etherpad application you can open a URL like this in your browser (replacing 192.168.59.103 with your container's IP address or localhost if you configured port-forwarding): http://192.168.59.103:9080/ If everything is running correctly, you'll see a welcome screen like the one shown below:
    New Pad window
    New Pad window
  5. Go ahead and create a new Pad and play around with it. This previous workshop example used a simple Etherpad setup which leveraged a local file to store the Pad text, so data was not persisted across containers, and your Pads and text were deleted when you shut down the container. Now our data is persisted in a permanent datastore, so our Pads and text will be available across server outages and network issues. Remember the name of the pad that you created here. We'll need to come back to it later.
  6. From the command line, run the following command to see your locally running container instances:
    docker ps
    You should see similar output to the following:
    CONTAINER ID IMAGE COMMAND CREATED
    7dad5190094a regist...mysql:latest /src/etherpad-lite/b 18 seconds ago
  7. You can now stop this container and restart it, without losing your pad data. From the command line, run the following commands to stop and then delete your running container instance, replacing the container ID below (starting with 7dad) with the container ID of your specific container:
    docker stop 7dad5190094a
    docker rm 7dad5190094a
  8. Now execute the docker run command again to start a new container instance, but accessing your previously created Pad via the same URL or by entering the same Pad name in the textbox on the Etherpad landing page. Note: You may need to update the IP address. Re-run the docker inspect command from above with the new CID to verify whether or not it has changed.
    docker run -d -p 9080:9080 etherpad_bluemix_mysql
You will see your changes persisted across the lifetime of multiple container instances! Before moving on, you should stop (and optionally delete) the running local Docker container instance as described at the end of Stopping or removing a running container instance on Bluemix in Workshop 1.

Task 5. Run the application in Bluemix

Once we have the application running locally, we can publish it to a container running in Bluemix and run it on the Internet, similar to our previous work in Task 6. Run the application in Bluemix in Workshop 1. Here are the commands to run the container in Bluemix. You will need to replace some values with your environment specific values, so read the formatting of each command before executing them below, in order.
ice --local push registry.ng.bluemix.net/bluemix_residency/etherpad_bluemix_mysql
ice images
ice run --name etherpad_mysql_01 registry.ng.bluemix.net/bluemix_residency/etherpad_bluemix_mysql:latest
The ice --local push command publishes the container image to Bluemix and places it in your organization's registry, scoped by the Registry URL you configured when you created your IBM Containers service instance.
  • Format:
    ice --local push registry.ng.bluemix.net/YOUR_REGISTRY_URL/YOUR_REMOTE_IMAGE_NAME
  • Example:
    ice --local push registry.ng.bluemix.net/bluemix_residency/etherpad_bluemix_mysql
The ice images command will print out the published image URL for you, along with some other data associated with the rest of the images you have pushed up to Bluemix:
ice images
Note the etherpad_bluemix_mysql image that we pushed up to Bluemix with the previous command and the difference between its image name and the other two images provided by IBM. As an end user, you can only pull from the base global directory, but you can push to your Organization-scoped repository by using the format in the previous command.
Pushing to your Organization-scoped repository
Pushing to your Organization-scoped repository
Now we will create a running container instance on Bluemix by issuing the run command below:
ice run -p 9080 --name etherpad__mysql_01 registry.ng.bluemix.net/bluemix_residency/etherpad_bluemix_mysql:latest
You will need to change the container name parameter from etherpad__mysql_01 or whatever you choose each time you create a new container. This container name does not have to be scoped with a complete URL like the image name does, as it is only for ease of identification of running containers. It can be a simple string and just has to be unique across the containers you have running inside your IBM Containers service instance. You will also need to replace registry.ng.bluemix.net/bluemix_residency/etherpad_bluemix_mysql:latest with your own image URL. The :latest tag at the end of the URL specifies to run the latest version of that image, if multiple exist. Specifying the -p parameter will expose the desired port on the running container inside the Container Service. This parameter needs to be specified for each port you wish to be accessible on your container.
  • Format: ice run -p PORT_TO_EXPOSE --name registry.ng.bluemix.net/ YOUR_REGISTRY_URL / YOUR_REMOTE_IMAGE_NAME :latest
  • Example: ice run -p 9080 --name etherpad_01 registry.ng.bluemix.net/bluemix_residency/etherpad_bluemix_mysql:latest
The ice run command will return a unique container ID (CID); use that CID in commands run for that container. In the following examples, the CID is dd489740-ffff-4c8f-9117-d94cd147f122, so replace that with your own CID. Before you can connect to your web application, you will need to assign it a floating public IP address. The ice ip request command will assign one to you. Replace 129.0.0.0 in the example below with the IP address returned from the ice ip request command, and replace dd489740-ffff-4c8f-9117-d94cd147f122 with your own CID:
ice ip request
ice ip bind 129.0.0.0 dd489740-ffff-4c8f-9117-d94cd147f122
We're hosting Etherpad on port 9080, so to test your running Etherpad application you can open a URL like this in your browser (replacing 129.0.0.0 with the IP address returned from the ice ip request command): http://129.0.0.0:9080/ Your account only gets two of these public IP addresses for free, so unbind the IP addresses when you're not using them. Replace 129.0.0.0 in the example below with the IP address returned from the ice ip request command, and replace dd489740-ffff-4c8f-9117-d94cd147f122 with your own CID:
ice ip unbind 129.0.0.0 dd489740-ffff-4c8f-9117-d94cd147f122
If you want to look up the CIDs for your running containers, and your published image names, you can click the Create A Container view of the Bluemix dashboard, or you can use the following ICE commands to view running containers and published images respectively:
ice ps
ice images
To stop or remove a running container instance on Bluemix, you can issue the following commands respectively, providing the necessary CID of the container you want work on:
sudo ice stop CID
sudo ice rm CID
Observe that a container does not have to be stopped before it can be removed from the IBM Containers service, which is a requirement when running containers locally using Docker.

Task 6. Validate the application data in the MySQL database (Optional)

Based on the way we integrated the MySQL database service into our application, we have all the necessary information to view the database in a workbench to validate that we are actually writing data to the database. This can be done with a number of tools, but the screen capture is from the MySQL Workbench, connecting with the same information collected from the Credentials panel.
MySQL Workbench
MySQL Workbench
There is one table, named store, that holds all the data necessary in a key-value pair relationship. This validates, along with our previous container recreation, that we are indeed leveraging the MySQL service for backing our Etherpad application. A future workshop will detail how we can use Caching services available on Bluemix to leverage an even faster datastore.

What's next?

Continue with the remaining workshops in this series:

1 Comment on "Actionable Architecture: Web application hosting with Database Services (Etherpad Workshop 2)"

  1. Hello,

    I have, me too, hosted an etherpad on BlueMix containers. I have not used the ice commands, but once the container as created, I accessed it through ssh, and then was able to build my configuration. I used nodejs (v0.12), the etherpad application, but then, due to network restrictions on ports (I wanted only 80 for the output and 22 for SSH), I added a NginX reverse-proxy between the etherpad(127.0.0.1) and the web (129.41.XXX.XXX:80).
    It then works very fine. I kept the initial dirtyDB (test configuration), but a “real” DB could have been added.

    However, it is very interesting to see that through 2 different ways we can get the same functional results in BlueMix

Join The Discussion

Your email address will not be published. Required fields are marked *