Please note that this tutorial was written to work with Tekton Pipeline Alpha and will not be compatible with Tekton Pipeline’s April 2020 Beta release. You can learn more about the Beta release here.
Tekton Pipelines is an open source project to configure and run continuous integration and continuous delivery (CI/CD) pipelines within a Kubernetes cluster. In this tutorial you learn the following concepts and skills:
- The basic concepts used in the Tekton Pipelines project
- Examples of creating a pipeline to build and deploy a Knative application
- Examples of running the pipeline, checking its status, and troubleshooting problems
Before you start the tutorial you must set up a Kubernetes environment with Knative and Tekton installed. This tutorial uses the IBM Cloud Kubernetes Service as an environment:
Note: The managed Knative add-on requires Kubernetes version 1.16 or later.
Note: The managed Knative add-on also installs Tekton.
This tutorial takes about an hour, including installing and configuring the prerequisites.
Step 1. Understand the Tekton Pipeline concepts
Tekton provides a set of extensions to Kubernetes, in the form of Custom Resources, for defining pipelines.
The following diagram shows the resources used in this tutorial. The arrows depict references from one resource to another resource.
This tutorial uses the following resources:
- A PipelineResource defines an object that is an input (such as a git repository) or an output (such as a docker image) of the pipeline.
- A PipelineRun defines an execution of a pipeline. It references the Pipeline to run and the PipelineResources to use as inputs and outputs.
- A Pipeline defines the set of Tasks that compose a pipeline.
- A Task defines a set of build steps such as compiling code, running tests, and building and deploying images.
The section of the tutorial that walks through an example goes into more detail about each resource.
Step 2: Create a sample pipeline
This section shows how to build a pipeline in Tekton that addresses the following actions:
- builds a Docker image from source files and pushes it to your private container registry
- deploys the image as a Knative service in your Kubernetes cluster
You should clone this project to your workstation since you will need to edit some of the yaml files before applying them to your cluster.
git clone https://github.com/IBM/tekton-tutorial
Work from the bottom up. First you define the task resources needed to build and deploy the image,
then you define the pipeline resource that references the tasks, and finally you create the
PipelineResource resources needed to run the pipeline.
You create a task to build an image and push it to a container registry. See the following Tekton task that builds a docker image and pushes it to a container registry. The complete YAML file is available at tekton/tasks/source-to-image.yaml.
apiVersion: tekton.dev/v1alpha1 kind: Task metadata: name: source-to-image spec: inputs: resources: - name: git-source type: git params: - name: pathToContext description: The path to the build context, used by Kaniko - within the workspace default: . - name: pathToDockerFile description: The path to the dockerfile to build (relative to the context) default: Dockerfile - name: imageUrl description: Url of image repository - name: imageTag description: Tag to apply to the built image default: "latest" steps: - name: build-and-push image: gcr.io/kaniko-project/executor command: - /kaniko/executor args: - --dockerfile=$(inputs.params.pathToDockerFile) - --destination=$(inputs.params.imageUrl):$(inputs.params.imageTag) - --context=/workspace/git-source/$(inputs.params.pathToContext)
A Tekton task can have one or more steps. Each step defines an image to run to perform the function of the step. This task has one step that uses the kaniko project to build a docker image from source and push it to a registry.
The task requires an input resource of type
git that defines where the source is located.
git source is cloned to a local volume at path
git-source comes from the name we gave to the resource. (Note that a resource is simply an abstract argument to the task.)
Later in this tutorial, you see how it is bound to a
PipelineResources resource, which defines the actual resource that is used. The task is reusable with different Git repositories.
A Tekton task also can have input parameters. Parameters help to make a Task more reusable.
This task accepts the following parameters:
- a path to the Docker build context inside the git source
- a path to the Dockerfile inside the build context
- the URL of the image repository where the image should be stored
- an image tag to apply to the built image
You might be wondering how the task authenticates to the image repository for permission to push the image. Future steps in this tutorial cover that authentication. Keep reading.
To create the task, apply the file to your cluster:
kubectl apply -f tekton/tasks/source-to-image.yaml
Step 3. Create a task to deploy an image to a Kubernetes cluster
The following Tekton task deploys a docker image to a Kubernetes cluster. The complete YAML file is available at tekton/tasks/deploy-using-kubectl.yaml.
apiVersion: tekton.dev/v1alpha1 kind: Task metadata: name: deploy-using-kubectl spec: inputs: resources: - name: git-source type: git params: - name: pathToYamlFile description: The path to the yaml file to deploy within the git source - name: imageUrl description: Url of image repository - name: imageTag description: Tag of the images to be used. default: "latest" steps: - name: update-yaml image: alpine command: ["sed"] args: - "-i" - "-e" - "s;__IMAGE__;$(inputs.params.imageUrl):$(inputs.params.imageTag);g" - "/workspace/git-source/$(inputs.params.pathToYamlFile)" - name: run-kubectl image: lachlanevenson/k8s-kubectl command: ["kubectl"] args: - "apply" - "-f" - "/workspace/git-source/$(inputs.params.pathToYamlFile)"
Creating a task to deploy an image to a Kubernetes cluster has two parts:
First, you run
sedin an Alpine Linux container to update the YAML file used for deployment with the image that was built by the source-to-image task. The YAML file must have a
__IMAGE__character string at the point where this update must occur.
Next, you run
kubectlusing Lachlan Evenson’s popular
k8s-kubectlcontainer image to apply the YAML file to the same cluster where the pipeline is running. Like the source-to-image task, this task uses an input
PipelineResourceand parameters to make the task as reusable as possible.
If you are wondering about how the task authenticates to the cluster for permission to apply the resources in the YAML file, keep reading. Future steps in this tutorial cover that authentication.
To create the task, apply the file to your cluster:
kubectl apply -f tekton/tasks/deploy-using-kubectl.yaml
Step 4. Create a pipeline
The following Tekton pipeline runs the two tasks previously described. The complete YAML file is available at tekton/pipeline/build-and-deploy-pipeline.yaml.
apiVersion: tekton.dev/v1alpha1 kind: Pipeline metadata: name: build-and-deploy-pipeline spec: resources: - name: git-source type: git params: - name: pathToContext description: The path to the build context, used by Kaniko - within the workspace default: src - name: pathToYamlFile description: The path to the yaml file to deploy within the git source - name: imageUrl description: Url of image repository - name: imageTag description: Tag to apply to the built image tasks: - name: source-to-image taskRef: name: source-to-image params: - name: pathToContext value: "$(params.pathToContext)" - name: imageUrl value: "$(params.imageUrl)" - name: imageTag value: "$(params.imageTag)" resources: inputs: - name: git-source resource: git-source - name: deploy-to-cluster taskRef: name: deploy-using-kubectl runAfter: - source-to-image params: - name: pathToYamlFile value: "$(params.pathToYamlFile)" - name: imageUrl value: "$(params.imageUrl)" - name: imageTag value: "$(params.imageTag)" resources: inputs: - name: git-source resource: git-source
A Tekton Pipeline resource lists the tasks to run and provides the input and output resources and input parameters required by each task. All resources must be exposed as inputs or outputs of the pipeline. The pipeline cannot bind one to an actual
PipelineResource. However, you can choose whether to expose an input parameter for a task as a pipeline input parameter, to set the value directly, or to accept the default value of the task (for an optional parameter).
For example, this pipeline exposes the
pathToContext parameter from the source-to-image task but does not expose the
pathToDockerFile parameter and allows it to default inside the task.
Dependencies between tasks can be expressed by using the
runAfter key. It specifies that the task must run after the list of one of tasks is completed.
In this example, this pipeline specifies that the
deploy-using-kubectl task must run after the
source-to-image task. Tekton orders the execution of the tasks to satisfy this dependency.
You can also express dependencies between tasks with the
from key. This tutorial doesn’t use this key, but you can read more in the Tekton documentation.
To create the pipeline, apply the file to your cluster:
kubectl apply -f tekton/pipeline/build-and-deploy-pipeline.yaml
Step 5. Create PipelineRun and PipelineResources
You defined reusable pipeline and task resources for building and deploying an image. Now it’s time to look at how to run the pipeline with an actual input resource and parameters.
The following Tekton
PipelineRun resource runs the pipeline you defined in the previous step. The complete YAML file is available at tekton/run/picalc-pipeline-run.yaml.
apiVersion: tekton.dev/v1alpha1 kind: PipelineRun metadata: generateName: picalc-pr- spec: pipelineRef: name: build-and-deploy-pipeline resources: - name: git-source resourceRef: name: picalc-git params: - name: pathToYamlFile value: "knative/picalc.yaml" - name: imageUrl value: <REGISTRY>/<NAMESPACE>/picalc - name: imageTag value: "1.0" serviceAccountName: pipeline-account
Although this file is small, there is a lot going on here:
PipelineRunresource does not have a fixed name. It uses
generateNameto generate a name each time it is created. Why? A particular
PipelineRunresource executes the pipeline only once. If you want to run the pipeline again, you cannot modify an existing
PipelineRunresource to request it to re-run. Instead, you must create another
PipelineRunresource. You could use
nameto assign a unique name to your
PipelineRuneach time you create one, but it is much easier to use
Pipelineresource is identified under the
The Git resource required by the pipeline is bound to specific
picalc-git. You define it in an upcoming step.
Parameters exposed by the pipeline are set to specific values.
A service account named
pipeline-accountis specified to provide the credentials needed for the pipeline to run successfully. You define this service account in the next part of the tutorial.
You must edit this file to substitute the values of
<NAMESPACE> with the information for your private container registry. To find the value for
<REGISTRY>, enter the command
ibmcloud cr region. To find the value of
<NAMESPACE>, enter the command
ibmcloud cr namespace-list.
The following Tekton
picalc-git defines the Git source. The complete YAML file is available at tekton/resources/picalc-git.yaml.
apiVersion: tekton.dev/v1alpha1 kind: PipelineResource metadata: name: picalc-git spec: type: git params: - name: revision value: master - name: url value: https://github.com/IBM/tekton-tutorial
You can apply the
PipelineResource file to your cluster now. Do not apply the
PipelineRun file yet because you still need to define the service account for it.
kubectl apply -f tekton/pipeline/tekton/resources/picalc-git.yaml
Step 6. Define a service account
The last step before running the pipeline is to set up a service account so that it can access protected resources. The service account ties together secrets containing credentials for authentication with RBAC-related resources for permission to create and modify specific Kubernetes resources.
First you enable programmatic access to your private container registry by creating an IBM Cloud Identity and Access Management (IAM) API key. The process for creating a user API key is described here.
After you have the API key, you can create the following secret:
kubectl create secret generic ibm-cr-push-secret --type="kubernetes.io/basic-auth" --from-literal=username=iamapikey --from-literal-password=<APIKEY> kubectl annotate secret ibm-cr-push-secret tekton.dev/docker-0=<REGISTRY>
<APIKEY>is either the API key that you created
<REGISTRY>is the URL of your container registry, such as
Now you can create the service account using the following YAML file. The complete YAML file is available at tekton/pipeline-account.yaml.
apiVersion: v1 kind: ServiceAccount metadata: name: pipeline-account secrets: - name: ibm-cr-push-secret --- apiVersion: v1 kind: Secret metadata: name: kube-api-secret annotations: kubernetes.io/service-account.name: pipeline-account type: kubernetes.io/service-account-token --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: pipeline-role rules: - apiGroups: ["serving.knative.dev"] resources: ["services"] verbs: ["get", "create", "update", "patch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: pipeline-role-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: pipeline-role subjects: - kind: ServiceAccount name: pipeline-account
This YAML creates the following Kubernetes resources:
PipelineRunyou saw earlier uses this name to reference the account. The service account references the
ibm-cr-push-secretsecret so that the pipeline can authenticate to your private container registry when it pushes a container image.
kube-api-secret, which contains an API credential (generated by Kubernetes) for accessing the Kubernetes API. This
kube-api-secretallows the pipeline to use
kubectlto talk to your cluster.
pipeline-role-binding, which provide the resource-based access control permissions needed for the pipeline to create and modify Knative services.
To create the service account and related resources, apply the file to your cluster:
kubectl apply -f tekton/pipeline-account.yaml
Step 7. Run the pipeline
Now all the pieces are in place to run the pipeline:
kubectl create -f tekton/run/picalc-pipeline-run.yaml
Note that this tutorial step uses
create instead of
apply. As previously mentioned, a
PipelineRun resource can run a pipeline only once, so you need to create a new one each time you want to run the pipeline.
kubectl responds with the generated name of the
You can check that status of the pipeline using the
kubectl describe command:
kubectl describe pipelinerun picalc-pr-db6p6
If you enter the command relatively quickly after creating the
PipelineRun, you might see output similar to this example:
Name: picalc-pr-db6p6 Namespace: default Labels: tekton.dev/pipeline=build-and-deploy-pipeline Annotations: <none> API Version: tekton.dev/v1alpha1 Kind: PipelineRun Metadata: Creation Timestamp: 2019-04-15T14:29:23Z Generate Name: picalc-pr- Generation: 1 Resource Version: 3893390 Self Link: /apis/tekton.dev/v1alpha1/namespaces/default/pipelineruns/picalc-pr-db6p6 UID: dd207211-5f8a-11e9-b66d-8eb09a9ab3eb Spec: Status: Params: Name: pathToYamlFile Value: knative/picalc.yaml Name: imageUrl Value: us.icr.io/mynamespace/picalc Name: imageTag Value: 1.3 Pipeline Ref: Name: build-and-deploy-pipeline Resources: Name: git-source Resource Ref: Name: picalc-git Service Account: pipeline-account Trigger: Type: manual Status: Conditions: Last Transition Time: 2019-04-15T14:29:23Z Message: Not all Tasks in the Pipeline have finished executing Reason: Running Status: Unknown Type: Succeeded Start Time: 2019-04-15T14:29:23Z Task Runs: Picalc - Pr - Db 6 P 6 - Source - To - Image - Kczdb: Pipeline Task Name: source-to-image Status: Conditions: Last Transition Time: 2019-04-15T14:29:28Z Reason: Building Status: Unknown Type: Succeeded Pod Name: picalc-pr-db6p6-source-to-image-kczdb-pod-7b4e7c Start Time: 2019-04-15T14:29:23Z Steps: Running: Started At: 2019-04-15T14:29:26Z Terminated: Container ID: containerd://b8f770e2b57d59c2bce76c63713d0b0a33f3fd02a14bad6b96978012060a436a Exit Code: 0 Finished At: 2019-04-15T14:29:26Z Reason: Completed Started At: 2019-04-15T14:29:26Z Terminated: Container ID: containerd://a637b1cb5d83b1ad2aa0dbecd962bb70b0452900189f611e404c0c9515262443 Exit Code: 0 Finished At: 2019-04-15T14:29:26Z Reason: Completed Started At: 2019-04-15T14:29:26Z Events: <none>
Not all Tasks in the Pipeline have finished executing. Rerun the command to check the status. If the pipeline runs successfully, the overall status eventually should look like this example:
Status: Conditions: Last Transition Time: 2019-04-15T14:30:46Z Message: All Tasks have completed executing Reason: Succeeded Status: True Type: Succeeded Start Time: 2019-04-15T14:29:23Z
Check the status of the deployed Knative service. It should be ready, as shown in the following example:
$ kubectl get ksvc picalc NAME DOMAIN LATESTCREATED LATESTREADY READY REASON picalc picalc.default.mycluster6.us-south.containers.appdomain.cloud picalc-00001 picalc-00001 True
You can use the URL in the response to curl the service:
$ curl picalc.default.mycluster6.us-south.containers.appdomain.cloud?iterations=20000000 3.1415926036
If the pipeline did not run successfully, the overall status might look like this example:
Status: Conditions: Last Transition Time: 2019-04-15T14:30:46Z Message: TaskRun picalc-pr-db6p6-deploy-to-cluster-7h8pm has failed Reason: Failed Status: False Type: Succeeded Start Time: 2019-04-15T14:29:23Z
Then, under the task run status you should find a message like the following example that tells you how to get the logs from the failed build step. Look at the logs to identify the problem.
build step "build-step-deploy-using-kubectl" exited with code 1 (image: "docker.io/library/alpine@sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913"); for logs run: kubectl -n default logs picalc-pr-db6p6-deploy-to-cluster-7h8pm-pod-582c73 -c build-step-deploy-using-kubectl
Be careful when defining a
PipelineResource as output from one task and input to another task.
For example, this tutorial could have used an image PipelineResource to define an output image from the
source-to-image task and an input image to the
This approach causes Tekton to create a PersistentVolumeClaim for sharing data between tasks. This Tekton Pipeline functionality is not completely implemented at the time of publishing, so it was not used in the tutorial.
Tekton provides simple, easy-to-learn features for constructing CI/CD pipelines that run on Kubernetes.
This tutorial covered the basics to get you started building your own pipelines. There are more features available and many more planned for upcoming releases. Try it out for yourself with IBM Cloud Kubernetes Service.