Add finally tasks to Tekton pipelines
Ensure tasks are executed even if the pipeline fails
Tekton Pipeline 0.14 introduced the
finally clause in the
Pipeline specification. This section is ideal for running anything just before exiting the pipeline, such as cleaning up any acquired resources, sending notifications, rolling back deployments, and more.
As defined on the Tekton website, the
finally section takes a list of one or more tasks that are all executed in parallel after all
tasks are finished executing — regardless of a success or a failure. The following code shows the
finally section of a pipeline.
apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name:build-deploy-cleaup-pipeline spec: params: - name: api-url - name: cloud-region tasks: - name: clone taskRef: name: git-clone - name: build taskRef: name: build runAfter: - clone - name: deploy taskRef: name: deploy runAfter: - build finally: - name: cleanup taskRef: name: cleanup
Also, the execution status of a
finally task affects the overall
pipelineRun status. A
pipelineRun is declared failed if one or more
finally tasks fail.
The community has implemented many features over the past few releases to help design
finally tasks for common use cases.
Use cases for finally
A Tekton Pipeline is implemented as a Kubernetes controller and executes a collection of
pods on your Kubernetes cluster. You can configure a pipeline using a directed acyclic graph (DAG) and a set of
finally tasks based on your continuous integration and continuous delivery (CI/CD) workflow. The most common workflow for CI/CD is
build-test-deploy, which we extend here to include
As the following image shows, a CI/CD pipeline defines a
finally task named
send-notification that references send-to-channel-slack and/or sendmail to notify the team and sends a success or failure notification depending on the execution status of the build process. The build process includes build and test tasks in the
tasks section of the pipeline.
A couple of cleanup scenarios:
A pipeline creates a project or acquires resources and sends the name of the project or resources to the
finallytask can then clean up those project resources regardless of a failure in the
taskssection, as shown in the following image:
A pipeline provisions a new Kubernetes cluster. The pipeline starts executing and stops for some reason. The pipeline has defined
finallytasks to free up the Kubernetes cluster. Even though the
finallysection executes to free up the resources.
Terminate rather than cancel
You are running a pipeline that executes a few Extract, Transform, and Load (ETL) processes in a sequence, as shown in the following image. Because of resource constraints, you want to cancel scheduling any further processes but compelete executing the current running process and analyze the results from that process, as shown in this image:
These are the most common use cases that require the functionality of
finally or an
exit handler in a pipeline. Next, let’s look at the list of the features that we introduced to accomplish these use cases.
Use finally with if
finally tasks are guaranteed to execute before the
pipelineRun exits. However, there are use cases where the pipeline needs the flexibility to not run a
finally task based on a certain condition (for example, the user wants to send a Slack notification only if a build fails). We have implemented two features to make this use case possible: TEP-0045 and TEP-0028 have the details. The overall idea here is to enable specifying
when expressions in a
finally task and provide a means to access the execution status of a
task in a
finally task at runtime so that you can evaluate specified
when expressions. You can access an execution status of a task using a context variable
spec: pipelineSpec: tasks: - name: golang-build taskRef: name: golang-build - name: unit-test taskRef: name: golang-unit runAfter: [ golang-build ] - name: deploy taskRef: name: deploy runAfter: [ unit-test ] finally: - name: notify-build-failure when: - input: $(tasks.golang-build.status) operator: in values: ["Failed"] taskRef: name: send-to-slack-channel
Now, instead of notifying only for a build failure, you want to send a Slack notification for any failure. TEP-0049 implements accessing an aggregate status of the
tasks section in
finally using a context variable
spec: pipelineSpec: ... finally: - name: notify-any-failure when: - input: $(tasks.status) operator: in values: ["Failed"] taskRef: name: send-to-slack-channel
Connect tasks and finally sections
If you are a pipeline author, you can now send the execution results of a task to any
finally task so that the project resources created/acquired by the pipeline are guaranteed to be cleaned up by the
finally task. TEP-0004 explains the design details of this feature.
spec: tasks: - name: acquire taskRef: name: acquire finally: - name: release taskRef: name: release params: - name: leased-resource value: $(tasks.acquire.results.leased-resource)
Gracefully terminate a pipeline
We implemented a solution to enable graceful termination (cancellation) of a pipeline where the current running
tasks are cancelled or terminated and the cleanup is scheduled. When a
pipelineRun is executing, you can update its definition to cancel the
tasks, which deletes all the respective pods and schedules the
apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: etl-processes spec: status: "CancelledRunFinally"
We implemented one more solution where the current running
tasks are not interrupted until they are finished and no new
tasks are scheduled. After the current running
tasks finish, the
finally section is executed to analyze the results from already executed
apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: etl-processes spec: status: "StoppedRunFinally"
Now you know how to design a
finally section of a pipeline to implement various use cases. If you have a use case or need a new feature in Tekton, reach out to the Tekton community over Slack or open a discussion in the tektoncd/pipeline repo.