Introduction
Tekton is a Kubernetes-native continuous integration (CI) and continuous delivery (CD) engine. A Tekton Pipeline orchestrates a set of Tasks to provide a desired goal, such as building an application and deploying it to a cluster. Until now, Tasks were always executed in containers running as Kubernetes pods. Tekton Pipelines 0.19 introduced Custom Tasks, which made it possible to integrate different execution models into the Tekton Pipeline. This article introduces Custom Tasks, explores some of the reasons why you would use them, and walks you through executing them in your Pipeline. If you are not familiar with Tekton concepts, refer to Introduction to Tekton architecture and design for an introduction.
Custom Tasks
Custom Tasks are an experimental alpha feature that you can use to plug alternative execution models into a Tekton Pipeline. They are intended to support cases where execution via a Tekton Task is potentially inefficient.
Consider the following use cases:
Implement advanced conditional checks, such as checking that a pipeline parameter setting follows certain conventions: You could do this using a normal Tekton Task, but the time to create a pod to run the task might outweigh the time to perform the check. If a pipeline needs many of these checks, then this time can add up and cause pipelines to run longer.
Wait for an external event to occur, such as an approval to perform a certain activity: You could do this using a normal Tekton Task, but it keeps a pod active in the cluster, consuming resources.
You can create Custom Tasks to support more complex workflows as well:
Execute another pipeline and wait for it to complete.
Execute a task a fixed number of times or until some condition occurs.
Use a Custom Task in a Pipeline
Because Custom Tasks are experimental, they are not enabled by default. To enable them, you need to edit the feature-flags ConfigMap in the tekton-pipelines namespace and change the setting of enable-custom-tasks from false to true:
kubectl edit cm feature-flags -n tekton-pipelines
A Custom Task is identified by an apiVersion, kind, and an optional name. A Pipeline uses the following syntax to reference a Custom Task:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: demo-custom-task
spec:
tasks:
- name: run-my-custom-task
taskRef:
apiVersion: example.dev/v1alpha1
kind: Example
name: my-custom-task
params:
- name: foo
value: bah
When the Pipeline is executed, Tekton creates a Run object to run the Custom Task (as opposed to the TaskRun object it creates for a normal Task):
apiVersion: tekton.dev/v1alpha1
kind: Run
metadata:
name: run-my-custom-task-xjns6
spec:
ref:
apiVersion: example.dev/v1alpha1
kind: Example
name: my-custom-task
params:
- name: foo
value: bah
The cluster must have a running custom task controller that is watching for and processing Run objects that reference this apiVersion and kind. The apiVersion, kind, and name are intended to reference a custom resource that provides input to the custom task controller about what the user wants it to do. In reality, the custom task controller is not obligated to support a custom resource. It can just use the parameters for input.
The custom task controller communicates the status of its processing by setting the Run object’s status field with a condition of the type Succeeded. While it is working on the request, it can set the condition status to Unknown and optionally provide a reason and message.
status:
conditions:
- message: Working on it
reason: Running
status: "Unknown"
type: Succeeded
If the request fails, the custom task controller sets the condition status to False. This causes the Pipeline execution itself to fail.
status:
conditions:
- message: Failed because bah happens
reason: Failed
status: "False"
type: Succeeded
If the request succeeds, the custom task controller sets the condition status to True. This causes any Pipeline Tasks that are directly dependent on this Task to begin execution:
status:
conditions:
- message: Processing completed successfully
reason: Succeeded
status: "True"
type: Succeeded
When the request succeeds, the controller can also return results as part of the status:
status:
results:
- name: conclusion
value: positive
A Pipeline can reference these results using the normal syntax:
$(tasks.run-my-custom-task.results.conclusion)
An example Custom Task: Task Loop
Let’s look at an implementation of a custom task TaskLoops, which provides for each kind of capability by iterating over a task and creating taskRuns for each iteration. You can configure the number of iterations and the values for each iteration through a parameter.
Install the TaskLoop controller:
kubectl apply -f https://storage.googleapis.com/tekton-releases-nightly/task-loops/latest/release.yaml
With TaskLoop, you can run a Task in a loop with varying parameter values. For example, suppose you have a Task that runs tests based on a parameter name called test-type, shown here:
cat <<EOF | kubectl apply -f -
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: testtask
spec:
params:
- name: test-type
type: string
steps:
- name: run-test
image: ubuntu
script: |
echo ["\$(params.test-type)"]
EOF
If you want to run this Task for multiple test types, you can create a TaskLoop custom resource
that looks like this:
cat <<EOF | kubectl apply -f -
apiVersion: custom.tekton.dev/v1alpha1
kind: TaskLoop
metadata:
name: testloop
spec:
taskRef:
name: testtask
iterateParam: test-type
EOF
A Pipeline would run the TaskLoop as follows:
cat <<EOF | kubectl apply -f -
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: testpipeline
spec:
tasks:
- name: run-my-tests
taskRef:
apiVersion: custom.tekton.dev/v1alpha1
kind: TaskLoop
name: testloop
params:
- name: test-type
value:
- ls
- echo
EOF
A PipelineRun would reference the testpipeline as follows:
cat <<EOF | kubectl apply -f -
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: testpipelinerun
spec:
pipelineRef:
name: testpipeline
EOF
This sample PipelineRun runs two iterations of the same task with different parameter:
kubectl get pr testpipelinerun -o json | jq .status.runs[].status.extraFields.taskRuns[].iteration
1
2
Custom Tasks like TaskLoop provide a great way to experiment with extensions to Tekton functionality without having to modify the Tekton controller itself.
Other custom task controllers available in the experimental repo include:
Summary
Custom Tasks provide an exciting new capability to integrate different execution models into Tekton. Custom Tasks are expected to evolve based on user requirements, so reach out to the Tekton community if you find Custom Tasks helpful or need additional guidance implementing your own.