Kubernetes with OpenShift World Tour: Get hands-on experience and build applications fast! Find a workshop!

Create a new Appsody stack

Appsody is an open source project that simplifies cloud native application development. Appsody’s primary component is a stack, which is a Docker image containing a set of pre-configured technologies and configurations, that is ready to be deployed in a cloud environment.

Stacks enable an application developer to concentrate on writing the application’s code, rather than worrying about the underlying technology components and how to build and deploy into a Kubernetes environment. There are already a number of public stacks available, but stack architects can also build stacks for their own enterprise that represent the set of technologies and versions the company uses, as well as chosen solutions for monitoring, logging, health checking, and more.

This tutorial walks you through the steps to create a brand new Appsody stack. Learn what to do, as a stack architect, to allow application developers to run, build, and deploy applications based on your stack.

Not sure whether you need to create a new stack? Read Customizing Appsody stacks to see whether you need to build a new one, modify an existing one, or use a template.

Prerequisites

Before beginning the steps in this tutorial, you need to understand what Appsody is and how it works. Check out the Appsody documentation and read the Introduction to Appsody blog post to learn the fundamentals.

To build and test stacks on your local workstation, complete the following steps:

In order to deploy the applications to a Kubernetes Service, you also need to have access to a cluster.

The role of a stack in the development process

Developers use stacks to simplify building applications that require a specific set of technologies or development patterns. While there are numerous publicly available stacks to choose from, many enterprises want to build their own set of stacks that uphold their specific requirements and standards for how they want to their developers to build cloud native applications.

Before learning how to create a new Appsody stack, let’s do a quick review of the design requirements for stacks. A stack is designed to support the developer in either a rapid, local development mode or a build-and-deploy mode.

Rapid, local development mode

In this mode, the stack contains everything a developer needs to build a new application on a local machine, with the application always running in a local containerized Docker environment. Introducing containerization from the start of the application development process (as opposed to development solely in the user space of the local machine) decreases the introduction of subtle errors in the containerization process and removes the need for a developer to install the core technology components of their application.

In this mode, the stack is required to have all the dependencies for the specific technologies pre-built into the Docker image, and also to dynamically compliment these with whatever dependencies the developer adds explicitly for his or her code.

Rapid local development mode in Appsody consists of the Appsody CLI (hooked into a local IDE if required) communicating with a local Docker container that is running the application under development. With this mode, application code can be held on the local file system, while being mounted in the Docker container, so that a local change can automatically trigger a restart of the application.

Build-and-deploy mode

In this mode, the stack enables the Appsody CLI to build a self-contained Docker image that includes both the core technologies in the stack plus the application code, along with the combined dependencies of both. You can deploy the resulting image manually or programmatically to any platform that supports Docker images (such as a local or public Kubernetes cluster).

The following image shows how an application developer uses a stack:

Appsody Flow

The above development flow shows the manual deployment to a Kubernetes cluster. In more production-orientated environments, GitOps might trigger the build and deploy steps, and Tekton Pipelines would drive the deployment. Kabanero Collections, which is part of Cloud Pak for Applications, brings together Appsody stacks, GitOps, and Tekton Pipelines to provide an enterprise-ready solution for cloud-native application development and deployment.

Stack structure

Because a single Appsody stack can enable both rapid, local development and build-and-deploy modes, all stacks follow a standard structure. The structure below represents the source structure of a stack:

my-stack
├── README.md
├── stack.yaml
├── image/
|   ├── config/
|   |   └── app-deploy.yaml
|   ├── project/
|   |   ├── [files that provide the technology components of the stack]
|   |   └── Dockerfile
│   ├── Dockerfile-stack
|   └── LICENSE
└── templates/
    ├── my-template-1/
    |       └── [example files as a starter for the application, e.g. "hello world"]
    └── my-template-2/
            └── [example files as a starter for a more complex application]

As a stack architect, you must create the above structure, build it into an actual stack image ready for use by an application developer who bases their new application on your stack. Part of your role as a stack architect is to include one of more sample applications (known as templates) to help the application developer get started.

Hence, when you build a stack, the structure above is processed and generates a Docker image for the stack, along with tar files of each of the templates, which can then all be stored and referenced in a local or public Appsody repo. The Appsody CLI can access the repo to use the stack to initiate local development.

Create a new stack

To create a new stack, you must first construct a scaffold of the above structure. Stacks are classified as being stable, incubating or experimental. You can read more about these classifications in the Appsody documentation. Appsody actually provides you with two different sample stacks. The first (in samples/sample-stack) provides just the stack structure and files for you to fill in. The second (in incubator/starter) provides the same plus a minimal working stack that you can copy, run, and then modify as a basis for a new stack.

We will use this second starter stack in this tutorial. To make things even easier, the Appsody CLI supports an appsody stack create command to create a new stack, as a copy of an existing stack.

Note In general, Appsody always tries to look in the existing repositories first for stacks, and then in the local cache. For normal stack usage, this is exactly what you want. However, when in the process of creating new stacks, by definition, the existing repositories will not yet know about your new stack. So, it’s quicker in this situation to tell Appsody to look in the local cache first. You can do this by setting the following environment variable: export APPSODY_PULL_POLICY=IFNOTPRESENT.

  1. Copy and rename the starter stack by running the appsody stack create command, which creates a subdirectory containing the starter stack.

     $ cd ~
     $ appsody stack create mystack --copy incubator/starter
     $ cd mystack
     $ ls - al
     total 16
     drwxr-xr-x  6 henrynash  staff  192 21 Oct 00:14 .
     drwxr-xr-x  3 henrynash  staff   96 21 Oct 00:14 ..
     -rw-r--r--  1 henrynash  staff  621 21 Oct 00:14 README.md
     drwxr-xr-x  7 henrynash  staff  224 21 Oct 00:14 image
     -rw-r--r--  1 henrynash  staff  297 21 Oct 00:14 stack.yaml
     drwxr-xr-x  3 henrynash  staff   96 21 Oct 00:14 templates
    

    The starter sample stack you copied is more than a scaffold. It is actually a stack you can build and run. It doesn’t do much, by design, but it ensures that your Appsody stack building environment is working before you make changes.

  2. Build your new stack

    Building (or packaging) a stack creates a stack image (which is a Docker image) that the Appsody CLI can use to initiate a project using that stack. Before you start, make sure you have a Docker environment set up. We recommend a local Docker environment.

    There is a Docker file (Dockerfile-stack) within the sample stack structure you copied. The appsody stack package command uses this to build the image.

    To build your new stack in this way, from the stacks/mystack directory enter:

     $ appsody stack package
    

    This runs a Docker build, installs mystack into a local Appsody repository (called dev.local), and runs some basic tests to make sure the file is well formed.

    Once the build is complete, check that it is now available in the local repo:

     $ appsody list dev.local
     REPO             ID       VERSION         TEMPLATES       DESCRIPTION
     dev.local        mystack  0.1.0           *simple         sample stack to help...
    
  3. Get the sample stack working

    So, at this point, you have been carrying out your role as a stack architect to build and install your copy of the sample stack. Now it’s time to try it out as an application developer.

    Create a new directory and initialize it with this new Appsody stack:

     $ mkdir ~/test
     $ cd ~/test
     $ appsody init dev.local/mystack
    

    The above code sets up a application development folder based on the default template in mystack, which you can now inspect:

     $ ls -al
     drwxr-xr-x   6 henrynash  staff   192 14 Sep 19:52 .
     drwxr-xr-x  41 henrynash  staff  1312 14 Sep 19:52 ..
     -rw-r--r--   1 henrynash  staff    27 14 Sep 19:52 .appsody-config.yaml
     drwxr-xr-x   3 henrynash  staff    96 14 Sep 19:52 .vscode
     -rw-r--r--   1 henrynash  staff    26 14 Sep 19:52 hello.sh
     drwxr-xr-x   3 henrynash  staff    96 14 Sep 19:52 tests
    

    The file .appsody-config.yaml points at the stack, while hello.sh is the starter application. The starter application is super simple and just echoes a line of text saying “Hello from Appsody!”

    Use appsody run to run the starter application in a container environment.

    As you can see below, our sample stack successfully executed the simple hello.sh within a container running in our Docker environment.

     $ appsody run
     Running development environment...
     Running command: docker[pull appsody/mystack:0.1]
     Using local cache for image appsody/mystack:0.1
     Running docker command: docker[run --rm -p 8080:8080 --name test20-dev -v /Users/henrynash/codewind-workspace/test20/:/project/userapp -v test20-deps:/project/deps -v /Users/henrynash/.appsody/appsody-controller:/appsody/appsody-controller -t --entrypoint /appsody/appsody-controller appsody/mystack:0.1 --mode=run]
     [Container] Running: /bin/bash /project/userapp/hello.sh
     [Container] Hello from Appsody!
    

Create your custom stack

Now that you have your environment set up and your sample stack working, it’s time to really become a stack architect and modify mystack to create the stack you actually want.

When creating a new stack, you need to consider a number of things:

  • What set of technologies do you need installed in your stack? How will you ensure all the dependencies are also installed?
  • What kind of sample starter application(s) do you want to provide application developers using your stack?
  • How will you ensure that application developers can install any additional dependencies they might need as they build out their application?

In this tutorial, we show you how to create a stack that allows application developers to create and deploy an application based on a Python HTTP server.

So, to answer the questions above: We configure our Python stack to use the popular Python web framework Flask. We use the Python packaging tool pipenv to install the technology and any dependencies. For a starter application, we’ll just have simple “Hello World” application that will respond to a url of /hello.

Let’s look at an overview of the steps you need to carry out, then we will dive in and execute them one by one.

  1. Modify the Docker file for the stack image: Stacks actually contain two Docker files. The first file, Dockerfile-stack, is in the mystack/image directory within the stack source structure. Docker uses the Dockerfile-stack file to build an image of the stack itself that the Appsody CLI uses. As part of this step, you will choose a base image, include the pipenv commands to install Flask, set the Appsody docker environment variables to the appropriate settings, and set any relevant ports that are needed.
  2. Decide on the architecture of how the components in the stack interact with developer’s applications: Consider whether there is a server process that’s in control or whether the developer’s application is in control. Essentially, if you were writing a regular Docker container, what would be the entry point? In this case, the Flask application is in control. To minimize the amount of code the application developer has to write, we’ll architect it so that the developer using the stack only has to provide whatever URL entry points they need for their application. We will make the entry point be server code that we provide as part of the stack.
  3. Write a sample Hello World application in the template directory: As just discussed, Flask will manage the web environment and ensure that a developer using the stack only needs to provide code for any endpoints they are exposing. In these steps, you’ll create a single Python file that contains your /hello endpoint. The application developers using your stack can take this file and expand it to contain all the endpoints in their actual application.
  4. Modify the Docker file for the final application image: Docker uses the second Docker file (Dockerfile), in the mystack/image/project directory within the stack source structure to build the final application image. This application image contains everything from the stack and the developer-written application. This build is carried out by the Appsody CLI build and deploy commands. This Docker file is responsible for ensuring the combined dependencies are installed in this final image.

Now that you’ve gotten the overview of the steps, let’s examine each one closely.

Modify the Docker file for the stack image

You will use this file as a basis for the Docker file you want to change. This file has a number of the Appsody environment variables already configured that match the structure of the sample stack, so you don’t have to start from scratch.

Let’s go ahead and make the changes to mystack/image/Dockerfile-stack in mystack we need for our new stack.

  1. First, you need to select a base image.

    A good base image is the standard Docker Python image, so replace the current RedHat image selection of registry.access.redhat.com/ubi7/ubi, defined by the FROM label at the top of the file, with: FROM python:3.7

  2. Modify the appropriate Appsody environment variables

    As you can see in the sample stack, there are a lot of possible variables. To simply run your new stack with an application, you only need to change a set of three variables that control how appsody run operates.

    Since the sample stack is unopinionated on technology, the example used for an application is just a shell script. So, in the sample stack, these variables look something like:

     ENV APPSODY_RUN="/bin/bash /project/userapp/hello.sh"
     ENV APPSODY_RUN_ON_CHANGE=$APPSODY_RUN
     ENV APPSODY_RUN_KILL=false
    

    To update these for your stack, you need to update the APPSODY_RUN environment variable, since Appsody passes control to this variable when you issue the appsody run command while operating in rapid local development mode. This is typically whatever command you would use to execute your main technology service if you were running it manually. For Flask, it is

     ENV APPSODY_RUN="python -m flask run --host=0.0.0.0 --port=8080"
    

    In rapid local development mode, your application can be restarted when source file changes are detected in the developer’s application directory; this isn’t waiting for a git commit, it is directly watching for files being changed. In case you need to do anything special on a restart, the command to re-start the application is kept in its own environment variable (APPSODY_RUN_ON_CHANGE). For Flask, however, nothing special is needed, so we can keep this the same.

    You can use APPSODY_RUN_KILL to determine whether Appsody needs to forcibly terminate the current running process. Since the sample stack actually just runs a one-shot shell script, it is set to false. For your new stack, you need Appsody to kill the existing Flask process, so set this to true:

     ENV APPSODY_RUN_KILL=true
    

    So our appsody run environment variable should now look like:

     ENV APPSODY_RUN="python -m flask run --host=0.0.0.0 --port=8080"
     ENV APPSODY_RUN_ON_CHANGE=$APPSODY_RUN
     ENV APPSODY_RUN_KILL=true
    

    Finally, use the APPSODY_WATCH_REGEX variable to tell Appsody which files to watch for relevant changes that will rerun your application.

    For a Python stack, we might set this to any file ending in .py.

     ENV APPSODY_WATCH_REGEX="^.*.py$"
    

    We’ll come back and update additional environment variables later in this tutorial. For reference, you can find all of them in the Appsody documentation.

  3. Ensure the Flask module and all dependencies are installed

    Now that the environment variables are set, you need to ensure that the Flask module is installed, along with any dependencies. In our example, we use pipenv to help us build a list of dependencies, and then use pip to install those dependencies into a dependency directory. We then set the Python path to pick up these dependencies.

    Don’t worry if you are not that familiar with dependencies in Python. The key lesson here is that you need to provide some code in the Docker file that will install whatever technology components you need, plus their dependencies.

    The final step is to tell Flask the entry point (which is server code we write in the next step).

    To do all of the above, add the following to Dockerfile-stack, typically towards the end of the file after the WORKDIR is set:

     RUN pip install pipenv
     RUN pipenv install flask
     RUN pipenv lock -r > requirements.txt
     RUN python -m pip install -r requirements.txt -t /project/deps
     ENV PYTHONPATH=/project/deps
     ENV FLASK_APP=/project/server/__init__.py
    

Save the changes you have made to Dockerfile-stack.

Provide the server side of the architecture that will support the developer’s application

Now that your Docker file is fixed, you need to provide that server entry point that is referenced above. For clarity, it is good practice to use a sub-directory to the mystack/project directory to hold your server code. Within that sub-directory, create a file that contains your server. For example, with the current directory set to mystack:

$ mkdir project/server
$ cat <<EOF > project/server/__init__.py
from flask import Flask

app = Flask(__name__)

from userapp import *
EOF

This code starts a Flask application (app) and then imports any Python files the user writes, so that they are part of the server. That’s all you need to do for the server side!

Write a sample Hello World application in the template directory

Now it’s time to create a sample application that only responds to a single URL end point. A stack can have any number of templates (each represented as a directory in templates). These templates can represent different classes of application that a developer can build with a particular stack.

The sample stack you copied has a single template called simple. As you saw when you first ran the sample stack, within that template is a trivial “application” which is a hello.sh script.

For our customized stack, we create a simple Python app instead, in the mystack/templates/simple directory:

$ cat <<EOF > templates/simple/__init__.py
from server import app

@app.route('/hello')
def HelloWorld():
    return 'Hello from Appsody!'
EOF

Build and run your new stack

You now have the minimal support required for your new Python/Flask stack. So, let’s try it out!

  1. In your role as a stack architect, from the stacks/mystack directory, build your sample stack image using the same stack packaging command as before:

     $ appsody stack package
    

    Provided this builds successfully, we are ready to run it.

  2. Now, acting as an application developer using your stack, create a new directory and re-initialize using Appsody. For example:

     $ mkdir ~/myapp
     $ cd ~/myapp
     $ appsody init dev.local/mystack
    

    You should now see that your new Hello World! Python app is installed:

     $ ls -al
     drwxr-xr-x   9 henrynash  staff   288 16 Sep 00:22 .
     drwxr-xr-x  43 henrynash  staff  1376 16 Sep 00:00 ..
     -rw-r--r--   1 henrynash  staff    27 16 Sep 00:00 .appsody-config.yaml
     drwxr-xr-x   3 henrynash  staff    96 16 Sep 00:00 .vscode
     -rw-r--r--   1 henrynash  staff    96 16 Sep 00:00 __init__.py
     -rw-r--r--   1 henrynash  staff    26 16 Sep 00:00 hello.sh
     drwxr-xr-x   3 henrynash  staff    96 16 Sep 00:00 tests
    

    Note The original hello.sh is still there because we haven’t yet deleted it from our stack – and since we have not yet hooked in the TEST and DEBUG options for our flask server, that’s OK, since those facilities still reference it. Once we have added these, we can remove hello.sh.

  3. Now, run your app and stack using appsody run:

     $ appsody run
     Running development environment...
     Using local cache for image appsody/mystack:0.1
     Running docker command: docker[run --rm -p 8080:8080 --name myapp-dev -v /Users/henrynash/codewind-workspace/myapp.   /:/project/userapp -v myapp-deps:/project/deps -v /Users/henrynash/.appsody/appsody-controller:/appsody/appsody-controller -t     --entrypoint /appsody/appsody-controller appsody/mystack:0.1 --mode=run]
     [Container] Running: python -m flask run --host=0.0.0.0 --port=8080
     [Container]  * Serving Flask app "/project/server/__init__.py"
     [Container]  * Environment: production
     [Container]    WARNING: This is a development server. Do not use it in a production deployment.
     [Container]    Use a production WSGI server instead.
     [Container]  * Debug mode: off
     [Container]  * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
    

    See that now Flask is the running application. If you hit the published endpoint, you should get a response:

     $ curl http://0.0.0.0:8080/hello
     Hello from Appsody!
    
  4. You can also check that Appsody will restart the server if you make a change to your hello world application. For instance, if you edit __init__.py in the myapp directory to change the message response for the /hello URL to, say, “Hello again from Appsody!” and save the file. You should see the Flask server automatically restart. If you hit the endpoint again you should see the updated message:

     $ curl http://0.0.0.0:8080/hello
     Hello again from Appsody!
    

Your new stack is now running in rapid local development mode. A developer using your stack could now build out their application, and their changes would immediately be live in their container for testing.

Now that the basics are running, let’s augment the stack with other changes needed for a full solution, namely:

  • Support TEST and DEBUG phases of rapid local development mode
  • Support for inclusion of dependencies from the developer’s application
  • Support for appsody build and deploy
  • Future improvements proposed as a follow on to this tutorial

Add support for DEBUG and TEST phases of rapid local development mode

You have already configured the environment variables for appsody run. Now you need to fill out the equivalent variables for appsody debug and appsody test.

Support for DEBUG phase

Similarly to the three variable for run mode, there are the equivalents for debug mode. In the sample stack they look something like this:

ENV APPSODY_DEBUG="echo -n \"Debugging \"; /bin/bash /project/userapp/hello.sh"
ENV APPSODY_DEBUG_ON_CHANGE=$APPSODY_DEBUG
ENV APPSODY_DEBUG_KILL=false

For our Flask stack, let’s start Flask in debugging mode, which will give us extra diagnostics in case of errors. To do this:

  1. Set the FLASK_ENV environment variable to development.
  2. You don’t need to make any changes for APPSODY_DEBUG_ON_CHANGE (similar to the run case).
  3. Set APPSODY_DEBUG_KILL to true.

So, after these changes, your three debug variables should look like this:

ENV APPSODY_DEBUG="FLASK_ENV=development python -m flask run --host=0.0.0.0 --port=8080"
ENV APPSODY_DEBUG_ON_CHANGE=$APPSODY_DEBUG
ENV APPSODY_DEBUG_KILL=true

You should now rebuild the stack from the stacks/mystack directory:

$ appsody stack package

Now run appsody debug and you should see that Flask starts in debugger mode:

$ cd ~myapp
$ appsody debug
Running debug environment
Using local cache for image appsody/mystack:0.1
Running docker command: docker[run --rm -p 8080:8080 --name myapp-dev -v /Users/henrynash/codewind-workspace/myapp/:/project/userapp -v myapp-deps:/project/deps -v /Users/henrynash/.appsody/appsody-controller:/appsody/appsody-controller -t --entrypoint /appsody/appsody-controller appsody/mystack:0.1 --mode=debug]
[Container] Running: FLASK_ENV=development python -m flask run --host=0.0.0.0 --port=8080
[Container]  * Serving Flask app "/project/server/__init__.py" (lazy loading)
[Container]  * Environment: development
[Container]  * Debug mode: on
[Container]  * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
[Container]  * Restarting with stat
[Container]  * Debugger is active!
[Container]  * Debugger PIN: 131-015-677

If you would like to see the Flask debugger mode in action, you could introduce an error into __init__.py in myapp, and then hit the endpoint http://0.0.0.0:8080/hello from a browser. You should see the debugger information displayed.

Support for TEST phase

The test mode of Appsody allows unit tests written by the developer for their application to be executed. The actual tests are stack-specific and often initiate a unit testing framework (unittest, for instance, in Python). We will use this to write a simple functional tests that hits the /hello endpoint and checks the returned data.

We will do this by creating a file in the mystack/templates/simple/tests directory. For example, from the mystack directory:

$ cat <<EOF > templates/simple/tests/test.py
from server import app
import unittest

class ServerTestCase(unittest.TestCase):

    def setUp(self):
        # create a test client
        self.app = app.test_client()
        self.app.testing = True

    def test_hello_endpoint(self):
        result = self.app.get('/hello')
        assert b'Hello' in result.data

if __name__ == '__main__':
    unittest.main()
EOF

Now we will update APPSODY_TEST variable in Dockerfile-stack to run this unittest:

ENV APPSODY_TEST="python -m unittest discover -s /project/userapp/tests -p *.py"
ENV APPSODY_TEST_ON_CHANGE=$APPSODY_TEST
ENV APPSODY_TEST_KILL=true

If you rebuild the stack again, create a new application directory (deleting the old one), and re-initialize and run appsody test, you will see that the Flask server runs, followed by your test:

$ cd ~
$ rm -r myapp
$ mkdir myapp
$ cd myapp
$ appsody init dev.local/mystack
$ appsody test
Running test environment
Using local cache for image appsody/mystack:0.1
Running docker command: docker[run --rm -p 8080:8080 --name myapp-dev -v /Users/henrynash/codewind-workspace/myapp/:/project/userapp -v myapp-deps:/project/deps -v /Users/henrynash/.appsody/appsody-controller:/appsody/appsody-controller -t --entrypoint /appsody/appsody-controller appsody/mystack:0.1 --mode=test]
[Container] Running command:  python -m unittest discover -s /project/userapp/tests -p *.py
[Container] .
[Container] ----------------------------------------------------------------------
[Container] Ran 1 test in 0.006s
[Container]
[Container] OK

Since we also set the APPSODY_TEST_ON_CHANGE as well, the test reruns every time we change one of the files. You can see how you could operate a test-definition-first approach using such a set up.

Add support for inclusion of dependencies from the developer’s application

Application developers need to be able to add additional dependencies for their code and have those dependencies automatically included when Appsody runs — without them needing to actually modify the stack itself.

The APPSODY_PREP Docker variable is designed to allow stack developers to provide a route for application developers to add their dependencies. Appsody executes this variable before each command that it executes (e.g. APPSODY_RUN, APPSODY_DEBUG, or APPSODY_TEST). You need to write the sequence of commands that will add in any new dependencies, for the developer’s application, into APPSODY_PREP.

Note APPSODY_PREP used to be called APPSODY_INSTALL, which is now deprecated (so you may see this in older stacks).

Like the dependency inclusion code that you added to the Dockerfile-stack earlier, the actual dependency management system you use is stack-specific (for example, maven for Java).

For Python, we will stick with pipenv as before. With pipenv, you can either give it a package on the command line or it will look in the current directory for a Pipfile. The Pipfile lists packages that need to be installed, along with any of their dependencies.

ENV APPSODY_PREP="cd /project/userapp; pipenv lock -r > requirements.txt; python -m pip install --upgrade -r requirements.txt -t /project/deps"

The line above allows the application developer to provide a Pipfile within their application directory, and pipenv adds these dependencies into the set of packages in /project/deps (which we already populated for all the components in the stack itself, see earlier in this tutorial). So, every time Appsody runs, it executes the above checks to see if any new dependencies have been added.

Once you have updated the APPSODY_PREP command, then you need to rebuild the stack, which is now ready to pull in any dependencies defined in the developer’s application.

Let’s test out this capability. As an example, we’ll do some fancy date manipulation in our application code which uses a specialist library, the Python package dateutil (which is not included, by default, with python). First, update your hello world sample code in your myapp directory to use it. For example:

from server import app
from datetime import *
from dateutil.relativedelta import *

@app.route('/hello')
def HelloWorld():
    lastFriday = date.today() + relativedelta(day=31, weekday=FR(-1))
    return 'Hello from Appsody, last Friday of this month is: ' + str(lastFriday)

If you do the above (you don’t need to rebuild the stack), and try appsody run, it will fail (since dateutil has not been installed).

So add a Pipfile in myapp to add in this dependency:

$ cat <<EOF > Pipfile
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[packages]
python-dateutil = "*"
EOF

Now when you execute appsody run, the server should come up fine. If you hit the /hello endpoint now, you should see something like:

$ curl http://0.0.0.0:8080/hello
Hello from Appsody, last Friday of this month is: 2019-09-27

You have successfully enabled developers to include their own packages and any dependencies into their application, without their needing to modify the stack.

Remember that, even though this tutorial uses Python and pipenv, you will want to use whatever package management tooling is appropriate for the technology base of your stack. The principle, however, is the same: to allow the developer to include dependencies, and for these to be pulled in by the stack dynamically without a stack rebuild.

Caching installed dependencies

One subtlety that might not be apparent yet is that Appsody provides the ability to cache any installed dependencies between runs (to accelerate the cycle time between runs). Appsody does this by creating a Docker volume and mounting this volume into the runtime environment created by appsody run/debug/test. Docker volumes are independent of any container instance.

In our sample stack, this is already configured by the existing line in Dockerfile-stack:

ENV APPSODY_DEPS=/project/deps

The Docker commands we added to Dockerfile-stack place any packages it is installing in /project/deps. The actual sequence of events here is:

  1. When the stack is built, the set of packages (and their dependencies) are written into /project/deps, by the commands you added to Dockerfile-stack.
  2. When you run appsody run/debug/test, it asks Docker to create a volume called {project_name}-dev (i.e. for our project it would be called myapp-dev) mapped to whatever is set in APPSODY_DEPS. Docker volume semantics mean that since /project/deps already exists in the stack image, the volume will be initialized with its contents.
  3. Subsequent writes to /project/deps (that is, the addition of any dependencies created in the developer’s application, defined in APPSODY_PREP) will update the volume (so the contents will be visible to subsequent runs), but will not affect the underlying stack directory of /project/deps. As a consequence, future appsody build/deploy commands can reference the dependencies of the stack alone by accessing /project/deps from the stack image (as we will see in the next sections)

Add support of appsody build and appsody deploy

At some point, an application developer will want to move beyond the rapid local development mode and actually build and deploy their completed application (as a single image with both their code and the code from the stack) into a Docker/kubernetes environment. To enable that, a stack needs to also contain:

  • A second Dockerfile that Appsody uses to build this combined image
  • A deployment manifest template that Appsody uses to deploy the image

Support for appsody build

To support building a standalone final image, you need to add a Dockerfile to your stack. Before we look at the contents of this Dockerfile, let’s review how the Appsody build process works.

When you issue the appsody build command, it:

  • Extracts the file system contents from the stack image into a local location (usually ~/.appsody/extract/{project_name}-dev)
  • Adds to the this extract the developer’s application, into a directory matching APPSODY_MOUNTS (in our case /project/userapp)
  • Executes a Docker build on this extracted directory structure, using the Dockerfile in /project

For reference, once the first two steps happen and Appsody is about to use Docker, the extracted structure looks something like this:

$ ls -al ~/.appsody/extract/myapp
total 48
drwxr-xr-x  10 henrynash  staff   320 20 Sep 22:55 .
drwxr-xr-x  19 henrynash  staff   608 20 Sep 22:55 ..
-rw-r--r--   1 henrynash  staff    32 13 Sep 17:38 .dockerignore
-rw-r--r--   1 henrynash  staff   578 20 Sep 10:49 Dockerfile
-rw-r--r--   1 henrynash  staff   150 20 Sep 10:50 Pipfile
-rw-r--r--   1 henrynash  staff  4566 20 Sep 10:50 Pipfile.lock
drwxr-xr-x  15 henrynash  staff   480 20 Sep 10:50 deps
-rw-r--r--   1 henrynash  staff   121 20 Sep 10:50 requirements.txt
drwxr-xr-x   3 henrynash  staff    96 16 Sep 14:54 server
drwxr-xr-x   8 henrynash  staff   256 20 Sep 22:55 userapp

Remember that the deps directory contains the dependencies of the stack itself (excluding those added by the developer’s application) and the contents of the application directory (in our case myapp) is in userapp.

Given the above, we can see what we need to write in our Dockerfile for our final application image.

The sample stack has the basics of one included already (it lives in mystack/image/project in the stack source structure), which looks something like this:

FROM registry.access.redhat.com/ubi7/ubi

WORKDIR /project

COPY . ./

EXPOSE 8080

CMD ["/bin/bash",  "/project/userapp/hello.sh"]

Modify this to:

  • Add in any dependencies required by the developer’s application (basically the same commands that were in APPSODY_PREP in the Dockerfile-stack)
  • Set the Python path and FLASK_APP variable (again similar to what we had in Dockerfile-stack)
  • Pass control to our server application directly. The appsody controller is not included in the final application image

These changes to Dockerfile should make it look like this:

FROM python:3.7

RUN pip install pipenv

WORKDIR /project
COPY . ./

WORKDIR /project/userapp
RUN pipenv lock -r > requirements.txt
RUN python -m pip install --upgrade -r requirements.txt -t /project/deps
WORKDIR /project

ENV PYTHONPATH=/project/deps
ENV FLASK_APP=server/__init__.py

EXPOSE 8080
CMD ["python", "-m", "flask", "run", "--host=0.0.0.0", "--port=8080"]

If you now rebuild the stack with these changes and then, acting as the application developer, execute appsody build in the application directory (myapp), you should end up with a built Docker image called myapp. This is your standalone image containing your application and the stack! So, let’s try it out. Since it is a regular image, you can just run it in your local Docker environment (ensuring you map the port exposed), that is:

$ docker run -p 8080:8080 myapp
Serving Flask app "server/__init__.py"
Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
Debug mode: off
Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)

Like before, if you hit our chosen endpoint http://0.0.0.0:8080/hello, you should get the usual response. You now have a build complete image you can deploy to any Docker or Kubernetes platform.

Support for appsody deploy

To help with the final deploy stage, Appsody allows you, as a stack architect, to provide a deployment manifest that Appsody uses as a template to deploy the final application image.

The default deployment uses the Appsody Operator, which the Appsody CLI installs into your Kubernetes cluster if it is not already present. A detailed description the Appsody Operator is beyond the scope of this tutorial, but read their user guide for more information.

Since we are using an operator, the deployment manifest is, in fact, a Kubernetes Custom Resource (CR) of the kind AppsodyApplication. The sample stack’s template (which you can find in mystack/image/config/app-deploy.yaml) looks something like:

apiVersion: appsody.dev/v1beta1
kind: AppsodyApplication
metadata:
  name: APPSODY_PROJECT_NAME
spec:
  version: 1.0.0
  applicationImage: APPSODY_DOCKER_IMAGE
  stack: APPSODY_STACK
  service:
    type: NodePort
    port: APPSODY_PORT
  expose: true

When you run appsody deploy, the appsody CLI substitutes the various variables in the CR for the actual ones relevant to your stack. The good news is that, as a stack architect, the above sample is fine and will work with our new stack.

Before you deploy, you need to pause and think about where the Appsody Operator is going to pull the application image from — and, more importantly, whether it will have the credentials to do this. If you run both your Kubernetes stack and your Docker environment locally (for example with Docker Desktop for Mac), then there is no issue (since the Docker Desktop Kubernetes cluster has access to local Docker registry).

If, however, you want to deploy to a cloud-based cluster, the cluster won’t have access to your local Docker registry. In this case, you need to do two things:

  • Decide on a suitable accessible Docker registry (docker.io is an obvious choice if you have an account there). Tell the deploy process to first push the image to it and then reference it in the generated version of the deploy CR
  • If there are credentials needed to pull from your chosen registry, then you need to add this to the deploy CR as well

Running a local Docker instance

If you are running locally, simply run appsody deploy in your application directory. This builds the application image, pushes it to your default Docker registry, creates the final deployment CR, and applies this to your Kubernetes cluster. This will result in a mapped port you can access for your application, for example:

$ appsody deploy
...
...
Deployed project running at http://localhost:32351

Using a remote Docker registry

If, however, you are using a remote Docker registry (but one that does not need credentials to pull from), you will simply pass some options to the deploy, giving it the name of the registry and image, and instruct it to push your image there first. For example:

$ appsody deploy -t docker.io/henrynash/myapp --push

Using a remote Docker registry that requires credentials

Finally, if you need to push your image to a registry for which the Appsody Operator needs some credentials, you can provide these by modifying the deploy CR. Typically, you would have already stored a Secret in the cluster that contains the credentials, and you can reference this by adding an pullSecret attribute to the CR, which the Appsody Operator will pick up and use when processing the CR.

Appsody actually stores the modified (i.e. substituted) CR in your application directory during the deploy phase. To enable you to amend this ahead of the CR being applied to your cluster, you can ask Appsody to generate this CR, but not deploy it:

$ appsody deploy --generate-only
$ cat app-deploy.yaml
apiVersion: appsody.dev/v1beta1
kind: AppsodyApplication
metadata:
  name: myapp
spec:
  version: 1.0.0
  applicationImage: myapp
  stack: mystack
  service:
    type: NodePort
    port: 8080

The pullSecret attribute should now be added to the spec, i.e.:

apiVersion: appsody.dev/v1beta1
kind: AppsodyApplication
metadata:
  name: myapp
spec:
  version: 1.0.0
  applicationImage: myapp
  stack: mystack
  service:
    type: NodePort
    port: 8080
  pullSecret: mysecret

Once the above has been added, you can then run the deploy step as before:

$ appsody deploy -t docker.io/henrynash/myapp --push

Whichever kind of deploy you carried out, your application image should be running, and you can confirm this by hitting the endpoint as before:

$ curl http://localhost:32351/hello
Hello from Appsody, last Friday of this month is: 2019-09-27

Congratulations on having developed and tested your new stack!

Future improvements proposed as a follow on to this tutorial

Now that you’ve created a working stack, there are a number of things you can do to harden your stack for production use:

Add some standard endpoints you want all applications to have

One of the power of stacks, is that you can implement features that you want all applications built with your stack to inherit. A common use for this, for stacks that provide HTTP endpoints, is to implement some of the industry standard urls such as /live, /health etc. By doing this in the stack itself, all applications will automatically have these endpoints enabled. To add, say, the /live liveness endpoint, you could modify the project/server/__init__.py file to include a response to that endpoint, i.e.:

from flask import Flask, jsonify

app = Flask(__name__)

from userapp import *

@app.route('/live')
def Liveness():
    state = {"status": "UP"}
    return jsonify(state)

If you make the change above, and then re-package the stack using appsody stack package as before, then execute, say, appsody run in the myapp directory, you should see the new endpoint is enabled.

$ curl http://0.0.0.0:8080/live
{"status":"UP"}

You can see how, as a stack architect, you can ensure all applications created by your developers using your stack will contain the standard endpoints you define.

Fill out the README and other stack identification files

There are a couple of others files that are important for when you look to publish your stack:

  1. README gives application developers information on how to use your stack, for example details of endpoints for monitoring/health you have included, any special debug details etc.

  2. stack.yaml should be included to provide meta data for your stack, such as language used, version, maintainer contact details etc.

  3. image/LICENSE should be included to specify the license under which the stack is provided.

Ensure your application runs with minimal privileges

The stack you built in this tutorial runs everything as root. A good principle is to, wherever possible, run the application with just-sufficient privileges. Your chosen base image may come with a user already created for your technologies, or you can use Docker commands to add users. Then you can take advantage of the Docker USER command to run your application as your chosen user.

Pinning your components versions

When we specified the flask package in the Dockerfiles in this tutorial, we didn’t specify any version — we just accepted the latest. While this isn’t wrong, per se, as a stack architect you need to think about whether such an unbounded install can cause issues.

Specifically, there will be a time lag between the time you build your stack (and the contents of /image/Dockerfile-stack is processed) and when your stack is used to build an application (at which point the /image/project/Dockerfile is processed). Since this time lag could be many months, there is a good chance that any packages and dependencies that you pull in at application build time will be more recent than the ones you tested when you built your stack.

In general, therefore, we recommend that you pin your package versions to a known, working version. For example, for the stack we just built, it would be better to have said: RUN pipenv install flask=="1.1.1".

Conclusion and next steps

As you will have seen from this tutorial, Appsody enables:

  • A Stack Architect to create stacks that define the set of technologies/versions, as well as chosen solutions for monitoring/logging/health, for given classes of applications.
  • An Application Developer to start writing an application using these stacks without having to know anything about how to install or configure the technologies involved, or to know about any particular deployment requirements for those technologies.

As a stack architect, now that you know how to create a new stack, try creating one in the language of your choice. Consider submitting your new stack to the Appsody open source project if there isn’t one for your language/technology.

Henry Nash