The Blog

 

In Part 1 of this two-part blog series, we introduced Knctl, a command-line interface (CLI) for interacting with Knative, and we showed you how to deploy pre-built Docker images with Knative. In this part, we explore newer features of Knctl, specifically, deploying applications from source, splitting traffic across revisions, and connecting Knative services to databases.

Deploy from source code

Knative deploys applications based on container images. In part one, we showed how you can use Knctl to deploy an image with a simple command. But what if you are still actively working on the application source code, and therefore do not have an image for it yet?

Knative Build is a project that allows you to turn source code into a container image. There are two options using it to build from source: 1) with a Dockerfile, and 2) with a Buildpack build template.

Before you can deploy, you must set up secrets for Knative Build to connect to an image registry for pushing and pulling images. The following example uses DockerHub and assumes that your user name and password are saved in DOCKER_HUB_USERNAME and DOCKER_HUB_PASSWORD environment variables:

$ knctl basic-auth-secret create -s docker-push --docker-hub \
    -u $DOCKER_HUB_USERNAME -p $DOCKER_HUB_PASSWORD

$ knctl basic-auth-secret create -s docker-pull --docker-hub \
    -u $DOCKER_HUB_USERNAME -p $DOCKER_HUB_PASSWORD --for-pulling

$ knctl service-account create -a docker-hub -s docker-push -s docker-pull

After your service account and secrets are created, you clone the sample Go app, which includes a Dockerfile. Note that the example shows the generic case for DockerHub with a private account requiring both a pull and a push secrets. If you used a public account, you might not need to set up a pull secret (because all images are public).

$ git clone https://github.com/cppforlife/simple-app

$ cd simple-app

Dockerfiles

The Dockerfile included in the sample application at github.com/cppforlife/knctl is fairly straightforward and just builds our Go application with a go build command. See Best practices for writing Dockerfiles if you are unfamiliar with creating Dockerfiles.

$ cat Dockerfile

FROM golang:1.10.1
WORKDIR /go/src/github.com/mchmarny/simple-app/
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -v -o app

FROM scratch
COPY --from=0 /go/src/github.com/mchmarny/simple-app/app .
EXPOSE 8080
ENTRYPOINT ["/app"]

Then you can issue a knctl deploy ... command to build from source and pass options to access the DockerHub secrets:

$ knctl deploy --service hello \
    --directory . \
    --service-account docker-hub \
    --image docker.io/dkalinin/simple-app \
    --env SIMPLE_MSG=custom-built

Name  hello

Waiting for new revision to be created...

Tagging new revision 'hello-00001' as 'latest'

Tagging new revision 'hello-00001' as 'previous'

[2018-12-04T12:29:04-08:00] Uploading source code...

[2018-12-04T12:29:06-08:00] Finished uploading source code...

Watching build logs...

build-step-credential-initializer | {"level":"info","ts":1543955343.2327511,"logger":"fallback-logger","caller":"creds-init/main.go:40","msg":"Credentials initialized."}
build-step-build-and-push | INFO[0000] Downloading base image golang:1.10.1
build-step-build-and-push | ERROR: logging before flag.Parse: E1204 20:29:08.377124       1 metadata.go:142] while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
build-step-build-and-push | ERROR: logging before flag.Parse: E1204 20:29:08.381355       1 metadata.go:159] while reading 'google-dockercfg-url' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg-url
build-step-build-and-push | INFO[0000] Executing 0 build triggers
build-step-build-and-push | INFO[0000] Unpacking rootfs as cmd RUN CGO_ENABLED=0 GOOS=linux go build -v -o app requires it.
build-step-build-and-push | INFO[0016] Taking snapshot of full filesystem...

[...]

build-step-build-and-push | INFO[0029] cmd: /bin/sh
build-step-build-and-push | INFO[0029] args: [-c CGO_ENABLED=0 GOOS=linux go build -v -o app]
build-step-build-and-push | net
build-step-build-and-push | vendor/golang_org/x/net/lex/httplex
build-step-build-and-push | crypto/x509
build-step-build-and-push | vendor/golang_org/x/net/proxy
build-step-build-and-push | net/textproto
build-step-build-and-push | crypto/tls
build-step-build-and-push | net/http/httptrace
build-step-build-and-push | net/http
build-step-build-and-push | github.com/mchmarny/simple-app
build-step-build-and-push | INFO[0034] Taking snapshot of full filesystem...
build-step-build-and-push | INFO[0039] Skipping paths under /builder/home, as it is a whitelisted directory
[...]
build-step-build-and-push | INFO[0072] Skipping paths under /workspace, as it is a whitelisted directory
build-step-build-and-push | INFO[0072] COPY --from=0 /go/src/github.com/mchmarny/simple-app/app .
build-step-build-and-push | INFO[0072] Taking snapshot of files...
build-step-build-and-push | INFO[0072] EXPOSE 8080
build-step-build-and-push | INFO[0072] cmd: EXPOSE
build-step-build-and-push | INFO[0072] Adding exposed port: 8080/tcp
build-step-build-and-push | INFO[0072] ENTRYPOINT ["/app"]
build-step-build-and-push | ERROR: logging before flag.Parse: E1204 20:30:21.112897       1 metadata.go:142] while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
build-step-build-and-push | ERROR: logging before flag.Parse: E1204 20:30:21.116650       1 metadata.go:159] while reading 'google-dockercfg-url' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg-url
build-step-build-and-push | 2018/12/04 20:30:22 pushed blob sha256:515ea77a06a3031d0d58aec1afa1c0e6156fed191e01f3c26669bea1ec8d95c9
build-step-build-and-push | 2018/12/04 20:30:22 pushed blob sha256:5e6d1dd21ad5ce9ae385df20fa1323679ba29a26926467cf11162afe442c71be
build-step-build-and-push | 2018/12/04 20:30:23 index.docker.io/dkalinin/simple-app:latest: digest: sha256:2840ce2ae3da7a9caf53463bdc4cc33d54c6c3ce5b1e7b1e03b4f8c3063688c7 size: 428
nop | Build successful

Waiting for new revision 'hello-00001' to be ready for up to 5m0s (logs below)...

hello-00001 > hello-00001-deployment-7d6b588b8-mwv4t | 2018/12/04 20:30:29 Simple app server started...

Revision 'hello-00001' became ready

Continuing to watch logs for 5s before exiting

Succeeded

With the Knctl deploy command you can view the output during execution. When building from source, you see the Docker build process and see the image that is uploaded to the registry. After the build process is complete and successful, you should be able to access the deployed service and even re-deploy it with the resulting image. Re-deploying adds a new revision of the service if you use the same name.

$ knctl curl --service hello

Running: curl '-H' 'Host: hello.default.example.com' 'http://x.x.x.x:80'

Hello World: custom-built!

Buildpacks

BuildTemplate objects in Knative Build allow you to build images in a variety of ways. The Buildpack template is based on the popular Buildpack process, first created by Heroku and now popularized by Cloud Foundry and other platforms as a service (PaaS).

First you need to add a BuildTemplate object to Kubernetes. The object provides directions on how to actually build and configure container image:

$ kubectl apply -f https://raw.githubusercontent.com/knative/build-templates/39146dffac752f187618ffef9d2d712aa9c8d243/buildpack/buildpack.yaml

Then you can use the buildpack template with Knctl. Issue the deploy command with the --template buildpack flag. You still need to include the secrets for your registry because the resulting container image needs to be saved somewhere for subsequent access:

$ knctl deploy --service hello \
    --directory . \
    --service-account docker-hub \
    --image docker.io/dkalinin/simple-app \
    --env SIMPLE_MSG=custom-built \
    --template buildpack \
    --template-env GOPACKAGENAME=main

Name  hello

Waiting for new revision (after revision 'hello-00001') to be created...

Tagging new revision 'hello-00002' as 'latest'

Tagging older revision 'hello-00001' as 'previous'

[2018-12-04T12:34:11-08:00] Uploading source code...

[2018-12-04T12:34:12-08:00] Finished uploading source code...

Watching build logs...

build-step-credential-initializer | {"level":"info","ts":1543955647.333156,"logger":"fallback-logger","caller":"creds-init/main.go:40","msg":"Credentials initialized."}
build-step-build | -----> Go Buildpack version 1.8.26
build-step-build | -----> Installing godep 80
build-step-build |        Download [https://buildpacks.cloudfoundry.org/dependencies/godep/godep-v80-linux-x64-cflinuxfs2-06cdb761.tgz]
build-step-build | -----> Installing glide 0.13.1
build-step-build |        Download [https://buildpacks.cloudfoundry.org/dependencies/glide/glide-v0.13.1-linux-x64-cflinuxfs2-aab48c6b.tgz]
build-step-build | -----> Installing dep 0.5.0
build-step-build |        Download [https://buildpacks.cloudfoundry.org/dependencies/dep/dep-v0.5.0-linux-x64-cflinuxfs2-52c14116.tgz]
build-step-build | -----> Installing go 1.8.7
build-step-build |        Download [https://buildpacks.cloudfoundry.org/dependencies/go/go1.8.7.linux-amd64-cflinuxfs2-fff10274.tar.gz]
build-step-build |        **WARNING** Installing package '.' (default)
build-step-build | -----> Running: go install -tags cloudfoundry -buildmode pie .
build-step-export | 2018/12/04 20:35:23 mounted blob: sha256:21324a9f04e76c93078f3a782e3198d2dded46e4ec77958ddd64f701aecb69c0
build-step-export | 2018/12/04 20:35:23 mounted blob: sha256:a5733e6358eec8957e81b1eb93d48ef94d649d65c69a6b1ac49f616a34a74ac1
build-step-export | 2018/12/04 20:35:23 mounted blob: sha256:6be38da025345ffb57d1ddfcdc5a2bc052be5b9491825f648b49913d51e41acb
build-step-export | 2018/12/04 20:35:23 mounted blob: sha256:1124eb40dd68654b8ca8f5d9ec7e439988a4be752a58c8f4e06d60ab1589abdb
build-step-export | 2018/12/04 20:35:23 pushed blob sha256:a9b41e0563c89746fc7a78bb3b96af6e34858027ac85b063ae785a22be733645
build-step-export | 2018/12/04 20:35:24 pushed blob sha256:0db6b2994ab377e8b484e8b96b5b65ca9bbd43b299fda85504ab38e0efd0bd7e
build-step-export | 2018/12/04 20:35:24 index.docker.io/dkalinin/test-simple-app:latest: digest: sha256:724c471dec76bcc2af0fe2abcb9bc15ed6fd17af51bdc4dce79f99ecf583fe69 size: 1082
nop | Build successful

Waiting for new revision 'hello-00002' to be ready for up to 5m0s (logs below)...

hello-00002 > hello-00002-deployment-76b4c6bf65-899nr | 2018/12/04 20:36:01 Simple app server started...

Revision 'hello-00002' became ready

Continuing to watch logs for 5s before exiting

Succeeded

Note that while we used the same application that includes a Dockerfile, you could remove this file when using the --template buildpack option. With this build template, the container image is created automatically. The buildpack process attempts to discover what your application type is and create the image accordingly. You can then verify that your service was deployed as we did above with the knctl curl command.

A word of caution: While deploying with buildpacks appears to be easier because you don’t need to create your own Dockerfile, it might be more difficult to track dependencies. Tracking dependencies is especially difficult when your application deviates from the common practices of frameworks used to create your application. We therefore recommend creating a Dockerfile for your applications and functions to guarantee repeatability of the image building process.

Traffic splitting

With traffic splitting, you can control the amount of traffic that Knative delivers to a specific revision of your service, which can be useful for a variety of reasons. One common example is to roll out service updates (and roll them back) in an incremental fashion to reduce risk of breaking users. The so-called blue-green deployment is easy to do with knctl rollout command.

The first thing to do when splitting traffic is to create a service with multiple revisions:

$ knctl deploy --service hello \
    --image gcr.io/knative-samples/helloworld-go \
    --env TARGET=first \
    --managed-route=false

$ knctl deploy --service hello \
    --image gcr.io/knative-samples/helloworld-go \
    --env TARGET=second \
    --managed-route=false

The latest revision is tagged as latest and the previous is tagged as previous. You can then use these tags to split traffic between these revisions. You can also chose to specify your own tags when deploying, for example, v1, v2, and so on. You can add these tags after deployment with the $ knctl revision tag ... command.

$ knctl rollout --route hello -p hello:latest=50% -p hello:previous=50%

Because the example split the traffic evenly across the two revisions, the easiest way to verify that traffic is indeed spread across is to curl the service multiple times. You should see that about 50% of the traffic is routed to the latest and other 50% to the previous revision.

$ while true; do knctl curl --service hello; done

Hello first!
...
Hello first!
...
Hello second!

Connecting to a database

Most cloud applications, services, and functions do not exist in a vacuum. That is, for any meaningful cloud service, you need to connect the service to some other external service. The most common example is to connect to a cloud database in order to store your data and allow it to be searched and queried securely and efficiently. Although you can hard code service dependency connections and credential information directly into source code, it is not typically a good idea.

A common solution is to use environment variables, or Kubernetes secrets, or external configuration files to hold sensitive information. We found a sample Go application that uses a MySQL database in order to store and retrieve blog posts. We made a change to the application to use DATABASE_URI environment variable for its database URI connection string.

Before we deployed this application, we created a MySQL service instance from IBM Cloud service catalog. After creating an instance of that service, we created credentials for the MySQL service instance and downloaded the JSON document that includes the connection information. The following screen capture shows the IBM Cloud user interface. The rest is self-explanatory after you create the service instance.

Image of MySQL instance creation on IBM Cloud

(For our adventurous readers, you can try to install the Kubernetes Service Catalog project and instead provision a MySQL service instance through the svcat command-line interface.)

After downloading the credential JSON, you need a YAML file with a Secret object similar to the one in our example:

yaml
apiVersion: v1
kind: Secret
metadata:
  name: mysql-ibm
type: Opaque
stringData:
  uri: "admin:xxx@(sl-us-south-1-portal.45.dblayer.com:17051)/compose"

Note that for a Go application to successfully connect to our MySQL database, the host-port pair (sl-us-south-1-portal.45.dblayer.com:17051) needs to be in parentheses.

The following example shows how to create the secret in the cluster:

$ kubectl apply -f mysql-ibm-secret.yml

After we created the secret, we deployed the Go MySQL sample application and reference created the secret through --env-secret=DATABASE_URI=secret-name/uri flag, as show in the following example:

$ git clone https://github.com/cppforlife/go-mysql-sample-app

$ cd go-mysql-sample-app

$ knctl deploy --service mysql \
    --directory . \
    --image docker.io/dkalinin/sample-mysql \
    --service-account docker-hub \
    --env-secret DATABASE_URI=mysql-ibm/uri # passing DB URI to allow service to connect to it

To verify that the application can successfully use database, you should open it in your browser. If you can’t set up DNS for your Knative installation, you can try using kwt to configure the DNS on your local machine temporarily:

$ sudo -E kwt net start --dns-map-exec 'knctl dns-map'

# ... once finished
$ kwt net clean-up

What’s next?

The important next steps for the Knctl project are not only socializing with Knative enthusiasts and new users but also connecting with the Knative community to increase adoption and perhaps make it the defacto default CLI.

We demonstrated these Knctl features recently at a “show and tell” meeting with the community. We will work with the community to achieve the next steps in defining the CLI for Knative, hopefully using Knctl as the base.

Conclusion

Now you have seen some examples of how to run Knative service from a local directory, split traffic across several revisions of a Knative service, and connect your service to a MySQL database. Using these features through a few commands helps demonstrate the realization of our goals for Knctl: simplifying Knative use for developers.

As we keep Knctl in sync with upcoming releases of Knative, we continue to welcome your your feedback via our Github project and look forward to your pull requests.

Michael Maximilien (IBM) and Dmitriy Kalinin (Pivotal)