Archive date: 2020-04-27
This content is no longer being updated or maintained. The content is provided “as is.” Given the rapid evolution of technology, some content, steps, or illustrations may have changed.Please note that this tutorial was written to work with Tekton Pipeline Alpha and will not be compatible with Tekton Pipeline’s April 2020 Beta release. You can learn more about the Beta release here.
Tekton Pipelines is an open source project to configure and run continuous integration and continuous delivery (CI/CD) pipelines within a Kubernetes cluster. In this tutorial you learn the following concepts and skills:
- The basic concepts used in the Tekton Pipelines project
- Examples of creating a pipeline to build and deploy a Knative application
- Examples of running the pipeline, checking its status, and troubleshooting problems
Prerequisites
Before you start the tutorial you must set up a Kubernetes environment with Knative and Tekton installed. This tutorial uses the IBM Cloud Kubernetes Service as an environment:
Create a standard Kubernetes cluster in the IBM Cloud Kubernetes Service
Note: The managed Knative add-on requires Kubernetes version 1.16 or later.
Create a private container registry in the IBM Cloud Container Registry
Install Knative in your cluster using the managed Knative add-on
Note: The managed Knative add-on also installs Tekton.
Estimated time
This tutorial takes about an hour, including installing and configuring the prerequisites.
Step 1. Understand the Tekton Pipeline concepts
Tekton provides a set of extensions to Kubernetes, in the form of Custom Resources, for defining pipelines.
The following diagram shows the resources used in this tutorial. The arrows depict references from one resource to another resource.
This tutorial uses the following resources:
- A PipelineResource defines an object that is an input (such as a git repository) or an output (such as a docker image) of the pipeline.
- A PipelineRun defines an execution of a pipeline. It references the Pipeline to run and the PipelineResources to use as inputs and outputs.
- A Pipeline defines the set of Tasks that compose a pipeline.
- A Task defines a set of build steps such as compiling code, running tests, and building and deploying images.
The section of the tutorial that walks through an example goes into more detail about each resource.
Step 2: Create a sample pipeline
This section shows how to build a pipeline in Tekton that addresses the following actions:
- builds a Docker image from source files and pushes it to your private container registry
- deploys the image as a Knative service in your Kubernetes cluster
You should clone this project to your workstation since you will need to edit some of the yaml files before applying them to your cluster.
git clone https://github.com/IBM/tekton-tutorial
Work from the bottom up. First you define the task resources needed to build and deploy the image,
then you define the pipeline resource that references the tasks, and finally you create the PipelineRun
and PipelineResource
resources needed to run the pipeline.
You create a task to build an image and push it to a container registry. See the following Tekton task that builds a docker image and pushes it to a container registry. The complete YAML file is available at tekton/tasks/source-to-image.yaml.
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
name: source-to-image
spec:
inputs:
resources:
- name: git-source
type: git
params:
- name: pathToContext
description: The path to the build context, used by Kaniko - within the workspace
default: .
- name: pathToDockerFile
description: The path to the dockerfile to build (relative to the context)
default: Dockerfile
- name: imageUrl
description: Url of image repository
- name: imageTag
description: Tag to apply to the built image
default: "latest"
steps:
- name: build-and-push
image: gcr.io/kaniko-project/executor
command:
- /kaniko/executor
args:
- --dockerfile=$(inputs.params.pathToDockerFile)
- --destination=$(inputs.params.imageUrl):$(inputs.params.imageTag)
- --context=/workspace/git-source/$(inputs.params.pathToContext)
A Tekton task can have one or more steps. Each step defines an image to run to perform the function of the step. This task has one step that uses the kaniko project to build a docker image from source and push it to a registry.
The task requires an input resource of type git
that defines where the source is located.
The git
source is cloned to a local volume at path /workspace/git-source
where git-source
comes from the name we gave to the resource. (Note that a resource is simply an abstract argument to the task.)
Later in this tutorial, you see how it is bound to a PipelineResources
resource, which defines the actual resource that is used. The task is reusable with different Git repositories.
A Tekton task also can have input parameters. Parameters help to make a Task more reusable.
This task accepts the following parameters:
- a path to the Docker build context inside the git source
- a path to the Dockerfile inside the build context
- the URL of the image repository where the image should be stored
- an image tag to apply to the built image
You might be wondering how the task authenticates to the image repository for permission to push the image. Future steps in this tutorial cover that authentication. Keep reading.
To create the task, apply the file to your cluster:
kubectl apply -f tekton/tasks/source-to-image.yaml
Step 3. Create a task to deploy an image to a Kubernetes cluster
The following Tekton task deploys a docker image to a Kubernetes cluster. The complete YAML file is available at tekton/tasks/deploy-using-kubectl.yaml.
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
name: deploy-using-kubectl
spec:
inputs:
resources:
- name: git-source
type: git
params:
- name: pathToYamlFile
description: The path to the yaml file to deploy within the git source
- name: imageUrl
description: Url of image repository
- name: imageTag
description: Tag of the images to be used.
default: "latest"
steps:
- name: update-yaml
image: alpine
command: ["sed"]
args:
- "-i"
- "-e"
- "s;__IMAGE__;$(inputs.params.imageUrl):$(inputs.params.imageTag);g"
- "/workspace/git-source/$(inputs.params.pathToYamlFile)"
- name: run-kubectl
image: lachlanevenson/k8s-kubectl
command: ["kubectl"]
args:
- "apply"
- "-f"
- "/workspace/git-source/$(inputs.params.pathToYamlFile)"
Creating a task to deploy an image to a Kubernetes cluster has two parts:
First, you run
sed
in an Alpine Linux container to update the YAML file used for deployment with the image that was built by the source-to-image task. The YAML file must have a__IMAGE__
character string at the point where this update must occur.Next, you run
kubectl
using Lachlan Evenson’s populark8s-kubectl
container image to apply the YAML file to the same cluster where the pipeline is running. Like the source-to-image task, this task uses an inputPipelineResource
and parameters to make the task as reusable as possible.
If you are wondering about how the task authenticates to the cluster for permission to apply the resources in the YAML file, keep reading. Future steps in this tutorial cover that authentication.
To create the task, apply the file to your cluster:
kubectl apply -f tekton/tasks/deploy-using-kubectl.yaml
Step 4. Create a pipeline
The following Tekton pipeline runs the two tasks previously described. The complete YAML file is available at tekton/pipeline/build-and-deploy-pipeline.yaml.
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
name: build-and-deploy-pipeline
spec:
resources:
- name: git-source
type: git
params:
- name: pathToContext
description: The path to the build context, used by Kaniko - within the workspace
default: src
- name: pathToYamlFile
description: The path to the yaml file to deploy within the git source
- name: imageUrl
description: Url of image repository
- name: imageTag
description: Tag to apply to the built image
tasks:
- name: source-to-image
taskRef:
name: source-to-image
params:
- name: pathToContext
value: "$(params.pathToContext)"
- name: imageUrl
value: "$(params.imageUrl)"
- name: imageTag
value: "$(params.imageTag)"
resources:
inputs:
- name: git-source
resource: git-source
- name: deploy-to-cluster
taskRef:
name: deploy-using-kubectl
runAfter:
- source-to-image
params:
- name: pathToYamlFile
value: "$(params.pathToYamlFile)"
- name: imageUrl
value: "$(params.imageUrl)"
- name: imageTag
value: "$(params.imageTag)"
resources:
inputs:
- name: git-source
resource: git-source
A Tekton Pipeline resource lists the tasks to run and provides the input and output resources and input parameters required by each task. All resources must be exposed as inputs or outputs of the pipeline. The pipeline cannot bind one to an actual PipelineResource
. However, you can choose whether to expose an input parameter for a task as a pipeline input parameter, to set the value directly, or to accept the default value of the task (for an optional parameter).
For example, this pipeline exposes the pathToContext
parameter from the source-to-image task but does not expose the pathToDockerFile
parameter and allows it to default inside the task.
Dependencies between tasks can be expressed by using the runAfter
key. It specifies that the task must run after the list of one of tasks is completed.
In this example, this pipeline specifies that the deploy-using-kubectl
task must run after the source-to-image
task. Tekton orders the execution of the tasks to satisfy this dependency.
You can also express dependencies between tasks with the from
key. This tutorial doesn’t use this key, but you can read more in the Tekton documentation.
To create the pipeline, apply the file to your cluster:
kubectl apply -f tekton/pipeline/build-and-deploy-pipeline.yaml
Step 5. Create PipelineRun and PipelineResources
You defined reusable pipeline and task resources for building and deploying an image. Now it’s time to look at how to run the pipeline with an actual input resource and parameters.
The following Tekton PipelineRun
resource runs the pipeline you defined in the previous step. The complete YAML file is available at tekton/run/picalc-pipeline-run.yaml.
apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
generateName: picalc-pr-
spec:
pipelineRef:
name: build-and-deploy-pipeline
resources:
- name: git-source
resourceRef:
name: picalc-git
params:
- name: pathToYamlFile
value: "knative/picalc.yaml"
- name: imageUrl
value: <REGISTRY>/<NAMESPACE>/picalc
- name: imageTag
value: "1.0"
serviceAccountName: pipeline-account
Although this file is small, there is a lot going on here:
The
PipelineRun
resource does not have a fixed name. It usesgenerateName
to generate a name each time it is created. Why? A particularPipelineRun
resource executes the pipeline only once. If you want to run the pipeline again, you cannot modify an existingPipelineRun
resource to request it to re-run. Instead, you must create anotherPipelineRun
resource. You could usename
to assign a unique name to yourPipelineRun
each time you create one, but it is much easier to usegenerateName
.The
Pipeline
resource is identified under thepipelineRef
key.The Git resource required by the pipeline is bound to specific
PipelineResources
namedpicalc-git
. You define it in an upcoming step.Parameters exposed by the pipeline are set to specific values.
A service account named
pipeline-account
is specified to provide the credentials needed for the pipeline to run successfully. You define this service account in the next part of the tutorial.
You must edit this file to substitute the values of <REGISTRY>
and <NAMESPACE>
with the information for your private container registry. To find the value for <REGISTRY>
, enter the command ibmcloud cr region
. To find the value of <NAMESPACE>
, enter the command ibmcloud cr namespace-list
.
The following Tekton PipelineResource
for picalc-git
defines the Git source. The complete YAML file is available at tekton/resources/picalc-git.yaml.
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: picalc-git
spec:
type: git
params:
- name: revision
value: master
- name: url
value: https://github.com/IBM/tekton-tutorial
The source code for this example is a go program that calculates an approximation of pi. The source includes a Dockerfile, which runs tests, compiles the code, and builds an image for execution.
You can apply the PipelineResource
file to your cluster now. Do not apply the PipelineRun
file yet because you still need to define the service account for it.
kubectl apply -f tekton/pipeline/tekton/resources/picalc-git.yaml
Step 6. Define a service account
The last step before running the pipeline is to set up a service account so that it can access protected resources. The service account ties together secrets containing credentials for authentication with RBAC-related resources for permission to create and modify specific Kubernetes resources.
First you enable programmatic access to your private container registry by creating an IBM Cloud Identity and Access Management (IAM) API key. The process for creating a user API key is described here.
After you have the API key, you can create the following secret:
kubectl create secret generic ibm-cr-push-secret --type="kubernetes.io/basic-auth" --from-literal=username=iamapikey --from-literal-password=<APIKEY>
kubectl annotate secret ibm-cr-push-secret tekton.dev/docker-0=<REGISTRY>
where
<APIKEY>
is either the API key that you created<REGISTRY>
is the URL of your container registry, such asus.icr.io
Now you can create the service account using the following YAML file. The complete YAML file is available at tekton/pipeline-account.yaml.
apiVersion: v1
kind: ServiceAccount
metadata:
name: pipeline-account
secrets:
- name: ibm-cr-push-secret
---
apiVersion: v1
kind: Secret
metadata:
name: kube-api-secret
annotations:
kubernetes.io/service-account.name: pipeline-account
type: kubernetes.io/service-account-token
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: pipeline-role
rules:
- apiGroups: ["serving.knative.dev"]
resources: ["services"]
verbs: ["get", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: pipeline-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: pipeline-role
subjects:
- kind: ServiceAccount
name: pipeline-account
This YAML creates the following Kubernetes resources:
- A
ServiceAccount
namedpipeline-account
. ThePipelineRun
you saw earlier uses this name to reference the account. The service account references theibm-cr-push-secret
secret so that the pipeline can authenticate to your private container registry when it pushes a container image. - A
Secret
namedkube-api-secret
, which contains an API credential (generated by Kubernetes) for accessing the Kubernetes API. Thiskube-api-secret
allows the pipeline to usekubectl
to talk to your cluster. - A
Role
namedpipeline-role
and aRoleBinding
namedpipeline-role-binding
, which provide the resource-based access control permissions needed for the pipeline to create and modify Knative services.
To create the service account and related resources, apply the file to your cluster:
kubectl apply -f tekton/pipeline-account.yaml
Step 7. Run the pipeline
Now all the pieces are in place to run the pipeline:
kubectl create -f tekton/run/picalc-pipeline-run.yaml
Note that this tutorial step uses create
instead of apply
. As previously mentioned, a PipelineRun
resource can run a pipeline only once, so you need to create a new one each time you want to run the pipeline.
Then kubectl
responds with the generated name of the PipelineRun
resource:
pipelinerun.tekton.dev/picalc-pr-db6p6 created
You can check that status of the pipeline using the kubectl describe
command:
kubectl describe pipelinerun picalc-pr-db6p6
If you enter the command relatively quickly after creating the PipelineRun
, you might see output similar to this example:
Name: picalc-pr-db6p6
Namespace: default
Labels: tekton.dev/pipeline=build-and-deploy-pipeline
Annotations: <none>
API Version: tekton.dev/v1alpha1
Kind: PipelineRun
Metadata:
Creation Timestamp: 2019-04-15T14:29:23Z
Generate Name: picalc-pr-
Generation: 1
Resource Version: 3893390
Self Link: /apis/tekton.dev/v1alpha1/namespaces/default/pipelineruns/picalc-pr-db6p6
UID: dd207211-5f8a-11e9-b66d-8eb09a9ab3eb
Spec:
Status:
Params:
Name: pathToYamlFile
Value: knative/picalc.yaml
Name: imageUrl
Value: us.icr.io/mynamespace/picalc
Name: imageTag
Value: 1.3
Pipeline Ref:
Name: build-and-deploy-pipeline
Resources:
Name: git-source
Resource Ref:
Name: picalc-git
Service Account: pipeline-account
Trigger:
Type: manual
Status:
Conditions:
Last Transition Time: 2019-04-15T14:29:23Z
Message: Not all Tasks in the Pipeline have finished executing
Reason: Running
Status: Unknown
Type: Succeeded
Start Time: 2019-04-15T14:29:23Z
Task Runs:
Picalc - Pr - Db 6 P 6 - Source - To - Image - Kczdb:
Pipeline Task Name: source-to-image
Status:
Conditions:
Last Transition Time: 2019-04-15T14:29:28Z
Reason: Building
Status: Unknown
Type: Succeeded
Pod Name: picalc-pr-db6p6-source-to-image-kczdb-pod-7b4e7c
Start Time: 2019-04-15T14:29:23Z
Steps:
Running:
Started At: 2019-04-15T14:29:26Z
Terminated:
Container ID: containerd://b8f770e2b57d59c2bce76c63713d0b0a33f3fd02a14bad6b96978012060a436a
Exit Code: 0
Finished At: 2019-04-15T14:29:26Z
Reason: Completed
Started At: 2019-04-15T14:29:26Z
Terminated:
Container ID: containerd://a637b1cb5d83b1ad2aa0dbecd962bb70b0452900189f611e404c0c9515262443
Exit Code: 0
Finished At: 2019-04-15T14:29:26Z
Reason: Completed
Started At: 2019-04-15T14:29:26Z
Events: <none>
Note the Not all Tasks in the Pipeline have finished executing
. Rerun the command to check the status. If the pipeline runs successfully, the overall status eventually should look like this example:
Status:
Conditions:
Last Transition Time: 2019-04-15T14:30:46Z
Message: All Tasks have completed executing
Reason: Succeeded
Status: True
Type: Succeeded
Start Time: 2019-04-15T14:29:23Z
Check the status of the deployed Knative service. It should be ready, as shown in the following example:
$ kubectl get ksvc picalc
NAME DOMAIN LATESTCREATED LATESTREADY READY REASON
picalc picalc.default.mycluster6.us-south.containers.appdomain.cloud picalc-00001 picalc-00001 True
You can use the URL in the response to curl the service:
$ curl picalc.default.mycluster6.us-south.containers.appdomain.cloud?iterations=20000000
3.1415926036
If the pipeline did not run successfully, the overall status might look like this example:
Status:
Conditions:
Last Transition Time: 2019-04-15T14:30:46Z
Message: TaskRun picalc-pr-db6p6-deploy-to-cluster-7h8pm has failed
Reason: Failed
Status: False
Type: Succeeded
Start Time: 2019-04-15T14:29:23Z
Then, under the task run status you should find a message like the following example that tells you how to get the logs from the failed build step. Look at the logs to identify the problem.
build step "build-step-deploy-using-kubectl" exited with code 1 (image: "docker.io/library/alpine@sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913"); for logs run: kubectl -n default logs picalc-pr-db6p6-deploy-to-cluster-7h8pm-pod-582c73 -c build-step-deploy-using-kubectl
Tips
Be careful when defining a PipelineResource
as output from one task and input to another task.
For example, this tutorial could have used an image PipelineResource to define an output image from the source-to-image
task and an input image to the deploy-using-kubectl
task.
This approach causes Tekton to create a PersistentVolumeClaim for sharing data between tasks. This Tekton Pipeline functionality is not completely implemented at the time of publishing, so it was not used in the tutorial.
Summary
Tekton provides simple, easy-to-learn features for constructing CI/CD pipelines that run on Kubernetes.
This tutorial covered the basics to get you started building your own pipelines. There are more features available and many more planned for upcoming releases. Try it out for yourself with IBM Cloud Kubernetes Service.
Share our content
-
- Prerequisites
- Estimated time
- Step 1. Understand the Tekton Pipeline concepts
- Step 2: Create a sample pipeline
- Step 3. Create a task to deploy an image to a Kubernetes cluster
- Step 4. Create a pipeline
- Step 5. Create PipelineRun and PipelineResources
- Step 6. Define a service account
- Step 7. Run the pipeline
- Tips
- Summary