IBM Developer Blog

Follow the latest happenings with IBM Developer and stay in the know.

Tekton Pipeline v0.16 includes When Expressions, which are concise, simple, and efficient


Pipeline is one of the core building blocks in designing a CI/CD use case on Kubernetes with Tekton. A Tekton Pipeline is a collection of tasks, which run based on how they are arranged. These tasks can be represented as a graph in which each node represents a task and can be arranged in many different ways.

  • Run all tasks (task-A, task-B, and task-C) in parallel. The three tasks start running independently and finish irrespective of the success or failure of other tasks. This kind of arrangement applies to a CI/CD use case, such as running unit tests and integration tests in parallel.

      tasks:
        - name: task-A
          taskRef:
            name: task1
        - name: task-B
          taskRef:
            name: task2
        - name: task-C
          taskRef:
            name: task3
    
  • Design sequence of tasks that run one after each other by using runAfter. For example, PipelineRun instantiates build-image and waits for it to finish. If build-image exits with success, PipelineRun instantiates e2e-tests, otherwise PipelineRun exits with failure without running the rest of the tasks. The same process applies to the next task. This example is the ideal arrangement, but hardly works in real world. It’s very common to run into task failures and require further processing if such failures.

      tasks:
        - name: build-image
          taskRef:
            name: build-image
        - name: e2e-tests
          runAfter: [ build-image ]
          taskRef:
            name: e2e-tests
        - name: deploy
          runAfter: [ e2e-tests ]
          taskRef:
            name: deploy
    
  • Design failure strategies by using finally. PipelineRun runs tasks that are specified under the finally section after all tasks under tasks section are completed regardless success or failure. For example, finally tasks start running after all tasks are completed successfully or one of the tasks exits with failure. finally tasks run in parallel just before they exit the PipelineRun. The PipelineRun exit status contains TaskRuns for all tasks, including the finally tasks.

      tasks:
        - name: build-image
          taskRef:
            name: build-image
        - name: e2e-tests
          runAfter: [ build-image ]
          taskRef:
            name: e2e-tests
        - name: deploy
          runAfter: [ e2e-tests ]
          taskRef:
            name: deploy
      finally:
        - name: cleanup
          taskRef:
            name: cleanup
    

So far, this looks great, but what about running a check before you run a task? You can design PipelineRun to include conditional tasks by using the Condition CRD to run a check before you run that task. The Condition CRD, just like a task, creates a new pod in a Kubernetes cluster to run that check. Depending on the outcome of that Condition CRD, a task is run or declared a failure with a conditionCheckFailed.

  apiVersion: tekton.dev/v1alpha1
  kind: Condition
  metadata:
    name: check-if-dev
  spec:
    params:
      - name: "env"
    check:
      image: alpine
      script: |
        if [ $(params.env) != "dev" ]; then
          exit 1
        fi
  ---

  apiVersion: tekton.dev/v1alpha1
  kind: Condition
  metadata:
    name: check-if-stage
  spec:
    params:
      - name: "env"
    check:
      image: alpine
      script: |
        if [ $(params.env) != "stage" ]; then
          exit 1
        fi
  ---

  apiVersion: tekton.dev/v1beta1
  kind: Pipeline
  metadata:
    name: build-test-deploy-in-all-env
  spec:
    params:
      - name: "env"
    tasks:
      ...
      - name: deploy-dev
        taskRef:
          name: deploy
        conditions:
          - conditionRef: "check-if-dev"
            params:
              - name: "env"
                value: "$(params.env)"
      - name: deploy-stage
        taskRef:
          name: deploy
        conditions:
          - conditionRef: "check-if-stage"
            params:
              - name: "env"
                value: "$(params.env)"
  ---

  apiVersion: tekton.dev/v1beta1
  kind: PipelineRun
  metadata:
    name: build-test-deploy-in-dev
  spec:
    pipelineRef:
      name: build-test-deploy-in-all-env
    params:
      - name: "env"
        value: "dev"
  ---

The Condition CRD played an important factor in implementing a conditional task or implementing a guarded task, but soon became too expensive for simple conditional check. For example, by matching the pipeline parameter and running only if the parameter matches a specific string, such as dev or stage.

The following sample PipelineRun results in a pod creation for each conditionRef. For example, (1) build-test-deploy-in-dev-deploy-dev-zvw84-check-if-dev-0--flppx and (2) build-test-deploy-in-dev-deploy-stage-x5vbv-check-if-stag-xn6bz).

kubectl get pods
NAME                                                              READY   STATUS      RESTARTS   AGE
build-test-deploy-in-dev-deploy-dev-q797c-pod-n8nks               0/1     Completed   0          2m46s
build-test-deploy-in-dev-deploy-dev-zvw84-check-if-dev-0--flppx   0/1     Completed   0          2m55s
build-test-deploy-in-dev-deploy-stage-x5vbv-check-if-stag-xn6bz   0/1     Error       0          2m55s

The task for which the condition resulted in no match. For example, deploy-stage in the following sample Pipeline results in failure:

tkn pr describe build-test-deploy-in-dev
Name:              build-test-deploy-in-dev
Namespace:         default
Pipeline Ref:      build-test-deploy-in-all-env
Service Account:   default
Timeout:           1h0m0s
Labels:
 tekton.dev/pipeline=build-test-deploy-in-all-env

🌡️  Status

STARTED         DURATION     STATUS
2 minutes ago   14 seconds   Succeeded(Completed)

⚓ Params

 NAME    VALUE
 ∙ env   dev

🗂  Taskruns

 NAME                                            TASK NAME      STARTED         DURATION    STATUS
 ∙ build-test-deploy-in-dev-deploy-stage-75bmz   deploy-stage   ---             ---         Failed(ConditionCheckFailed)
 ∙ build-test-deploy-in-dev-deploy-dev-q797c     deploy-dev     2 minutes ago   5 seconds   Succeeded

When Expressions

Tekton Pipeline implemented a feature that is called When Expressions, which is similar to the Match Expressions within Kubernetes in that it takes an input, operator, and values.

  tasks:
    - name: deploy-dev
      when:
        - input: "$(params.env)"
          operator: in
          values: [ "dev" ]
      taskRef:
        name: deploy-dev
  • Input can be a static string or Pipeline param or task results.
  • Operator represents an Input‘s relationship to a set of Values. Two operators in and notin are supported by the initial implementation.
  • Values is a list of string values. The list can contain static strings, and/or Pipeline params, and/or task results.

The When Expression simplifies the pipeline and builds an efficient PipelineRun without creating unnecessary pods to match the pipeline parameters, which can be done by the Pipeline controller. Unlike the Conditions CRD, the task with the When Expressions failure is marked as skipped and results in a Successful PipelineRun.

Let’s rewrite the previous example with the Conditions CRD by using When Expressions:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: build-test-deploy-in-all-env
spec:
  params:
    - name: "env"
  tasks:
    - name: deploy-dev
      taskRef:
        name: deploy
      when:
        - input: "$(params.env)"
          operator: in
          values: [ "dev" ]
    - name: deploy-stage
      taskRef:
        name: deploy
      when:
        - input: "$(params.env)"
          operator: in
          values: [ "stage" ]
---

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: build-test-deploy-in-dev
spec:
  pipelineRef:
    name: build-test-deploy-in-all-env
  params:
    - name: "env"
      value: "dev"
---

Notice, only one pod is created for the task that was run:

kubectl get pods
NAME                                                  READY   STATUS      RESTARTS   AGE
build-test-deploy-in-dev-deploy-dev-6h8vj-pod-9ffx2   0/1     Completed   0          16s

No misleading task failures under the list of TaskRuns:

 tkn pr describe build-test-deploy-in-dev
Name:              build-test-deploy-in-dev
Namespace:         default
Pipeline Ref:      build-test-deploy-in-all-env
Service Account:   default
Timeout:           1h0m0s
Labels:
 tekton.dev/pipeline=build-test-deploy-in-all-env

🌡️  Status

STARTED          DURATION    STATUS
48 seconds ago   6 seconds   Succeeded(Completed)

⚓ Params

 NAME    VALUE
 ∙ env   dev

🗂  Taskruns

 NAME                                          TASK NAME    STARTED          DURATION    STATUS
 ∙ build-test-deploy-in-dev-deploy-dev-6h8vj   deploy-dev   48 seconds ago   6 seconds   Succeeded

Retrieve a list of skipped tasks:

kubectl get pr build-test-deploy-in-dev -o json | jq .status.skippedTasks
[
  {
    "name": "deploy-stage"
  }
]

The When Expression can be used in many different use cases:

  • Guard executing tasks based on the result produced by the parent tasks.
  • Design generic pipelines and use it in different environments, regions, cloud providers, and such. For example, one single pipeline can be designed to deploy an application to dev, qa, stage, and prod.
  • Disable one single task or a list of tasks from the entire pipeline.
  • Perform extra processing in terms of a task or a list of tasks by depending on the input from the PipelineRun.

Summary

To summarize, Tekton Pipeline v0.16 includes When Expressions, which are concise, simple, and efficient. Check the Pipeline repository for detailed usage and more examples of using When Expressions.