For quite some time now, the docker community has been grappling with the complexity of supporting multiple operating systems and architectures. The following are two key issues that have arisen as a result of this diversity:

  • New docker users tend to assume that if they do a docker run of an image, that it will work on their platform, and have no idea how to interpret the failure. Sometimes, the error message is exec format error, which is not intuitive. However, what a user may see when using the wrong type of image also depends on the image that is being run.
  • Projects that do support multiple platforms have resorted to using image names to indicate which platform an image was built for. For example, my-cool-app-ppc64le:latest; or creating a Docker Hub namespace for each architecture, like ppc64le/my-cool-app.

To alleviate these issues, the docker community has come up with what is currently called a manifest list, also nicknamed a multi-arch image, or fat manifest. An individual manifest describes the contents of one image, and a manifest list lets you group multiple images together. After the creation of a manifest list, anyone (from the project owners to their user-base) can use the manifest list name where they once used an individual image name. When a user does a docker pull or docker run, the docker engine does the work of selecting which image to pull based on the operating system and architecture on which it is running. As a result, a project’s users no longer have to worry about finding the name of the image that will work for their platform. The project can have a single name “my-cool-app:latest” to put in blog posts, documentation, and so on. This makes the user experience much more pleasant and intuitive.

The community was quick to create tooling around the manifest list before docker decided on how to best organize the UI. Because of all the work that was done by members of the community, the usage for such a tool began to become clear. There is currently a standalone tool maintained by Phil Estes from IBM which most people have been using to create multi-arch images — but the community also began to ask more and more for the ability to make multi-arch images using docker’s tooling. As a result, the manifest sub-command was born. It is still currently in the PR review process, but can be easily used to create a multi-arch image and push it to any docker registry.

Here is how to use the the new docker cli sub-command to create manifest lists.

A little terminology first:

  • Manifest – Describes a single image.
  • Manifest list – Groups multiple images; can be thought of as a list of pointers; must exist on a docker registry.
  • Reference – A name for an image or manifest list (for example, docker.io/library/my-cool-app:latest).
  • Tag – The portion of the reference after the colon. This is not required. However, you can use it to version a reference. For example, app:v1. latest is the default tag.
  • Digest – the hash (currently sha256) of the image contents of a docker image or manifest list’s individual digests.

When working with multi-arch images, you can use an untagged, tagged, or a digested reference (for example, my-cool-app@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2) for image names.

There are two ways to create a manifest list. You can interactively create, annotate (optional), and push a manifest list from your local host by using the following sequence of commands:

$ docker manifest create new-list-ref-name image-ref [image-ref...]
$ docker manifest annotate new-list-ref-name image-ref --os linux --arch arm
$ docker manifest push new-list-ref-name

This will store some metatdata locally until you do the final push step to the registry, at which point the manifest list will be usable.

The first parameter when working with multi-arch images is always the manifest list name that you are going to give to users. The following image-ref parameters are the pre-built, platform-specific images you have previously pushed to a registry.

Note: This is an important detail to understand. Pushing a multi-arch image to a registry does not push the image layers. It only pushes a list of pointers to accessible images. This is why it is better to think of a multi-arch image as what it really is: a manifest list. Project maintainers might be the only ones who need to understand this distinction. End users of the images do not need to be concerned with the difference since the goal of a manifest list is to be transparent to the user.

If you would like to create a multi-arch image for your app or project, you can simply recompile the docker cli and replace your current cli binary (typically /usr/bin/docker — NOT /usr/bin/dockerd), or use the new cli straight from the build dir of the docker project.

 


 

To get the code:

$ git clone -b manifest-cmd https://github.com/clnperez/cli.git

On an x86 system you can build using the project’s Dockerfile:

$ make -f docker.Makefile binary

To make the binary for all supported platforms, and use the one you want, run:

$ make -f docker.Makefile cross

The binaries (if you ran make cross there will be more than one) will be found in the build/ directory:


$ ls build/
docker  docker-darwin-amd64  docker-linux-amd64  docker-linux-arm  docker-linux-ppc64le  docker-windows-amd64

You can use these binaries simply as:

$ ./build/docker manifest --help

Now, you can follow the steps above (either the interactive or yaml approach) to push your multi-arch images to the registry of your choice.

Note: For insecure registries (such as a local one used for testing), if you are building and using the CI PR, ensure that you add the registry to your local docker config file.


> cat ~/.docker/config.json
{
"insecure-registries" : ["127.0.0.1:5000"]
}

Here is an example of using the interactive approach to creating a multi-arch image:

$ docker manifest create awesome_company/mycool-app:v1 awesome_company/mycool-app-x86:v1 \
awesome_company/mycool-app-s390x:v1 \
awesome_company/mycool-app-ppc64le:v1 \
awesome_company/mycool-app-armhf:v1
$ docker manifest annotate awesome_company/mycool-app:v1 awesome_company/mycool-app-armhf:v1 --os linux --arch arm
$ docker manifest push awesome_company/mycool-app:v1

After you have pushed your manifest list to a registry, you use it just as you would have previously used an image name.

A user on an x86 system:


$ docker run awesome_company/mycool-app:v1 echo hi
hi

This command pulls and runs only the x86 image layers that your manifest list points to, and the user will have no awareness that ‘awesome_company/mycool-app:v1’ was a manifest list instead of an image name.

If you are using a POWER system, or any other platform that you included in your manifest list, the command is exactly the same:


$ docker run awesome_company/mycool-app:v1 echo hi
hi

As you can see, multi-arch images (also knows as manifest lists) greatly improve the end-user experience, with just a little extra work from the project maintainers.

Resources

If you are interested in building the base images for other architectures, watch our presentation from DockerCon 2017 here: FROM ARM to z: Building, Shipping, and Running a Multi-platform Docker Swarm

For POWER development resources, several options tailored to varying focus-areas can be found here: https://developer.ibm.com/linuxonpower/cloud-resources

You can request z VMs using the LinuxOne Community Cloud here: https://linuxone20.cloud.marist.edu/cloud/#/login (requires registration)

5 comments on"Create and use multi-architecture docker images"

  1. […] See https://developer.ibm.com/linuxonpower/2017/07/27/create-multi-architecture-docker-image/ for more information on multi-architecture Docker images. […]

  2. Using a manifest file does not appear to work anymore.
    $ ./build/docker manifest push -f ./annotated-manifests.yaml
    unknown shorthand flag: ‘f’ in -f
    See ‘docker manifest push –help’.
    $ ./build/docker manifest push –help

    Usage: docker manifest push [OPTIONS] MANIFEST_LIST

    Push a manifest list to a repository

    Options:
    –help Print usage
    –insecure Allow push to an insecure registry
    -p, –purge Remove the local manifest list after push

    • Christy Perez October 11, 2017

      Hi Mark. Yes, the maintainer asked that I remove that support and re-add it as a follow-on PR. Sorry about that. I will update the blog post to reflect the current PR state.

  3. […] Create and use multi-architecture docker images […]

  4. […] Create and use multi-architecture docker images […]

Join The Discussion

Your email address will not be published. Required fields are marked *