IBM Developer Blog

Follow the latest happenings with IBM Developer and stay in the know.

Learn more about Tekton Pipeline’s Beta release version 0.11.0, nicknamed Ragdoll Norby.


Earlier this year, I discussed the new features inside of Tekton version 0.10.0’s Bombay Robbie release. In this blog, I am excited to share that Tekton Pipeline’s Beta release version 0.11.0, nicknamed Ragdoll Norby, is finally here!

The first official Tekton Pipeline Beta release has a lot of interesting features and fixes that I’d like to address, so let’s dive right in!

Introducing the v1beta1 API

Tekton Pipeline introduces a new apiVersion called v1beta1 which can be referred to as apiVersion: tekton.dev/v1beta1 in your Tekton resources. In this blog, I discuss five of the new features v1beta1 implements.

1. Share simple data among tasks

Workspaces are incredibly useful for connecting various tasks and sharing data among those tasks within a pipeline. Workspaces are generally bound to a type of volume, such as persistentVolumeClaim (PVC), emptyDir, configMap, or secret and work great for sharing the file system. To share a file or two, Tekton Pipeline Beta introduces ‘/tekton/results/’ directory. /tekton/results/ is created when a task has one or more files specified under the results section and can be populated within steps using echo -n <task-result> | tee $(results. <task-result>.path). Here’s an example:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: parse-git-commit-id
spec:
  results:
    - name: git-commit-id
      description: The file holds SHA256 GitHub commit Id
  steps:
…

Task results also appear in the TaskRun status:

  kubectl get tr pipelinerun-parse-git-commit-id-bkfj6 -o json
  {
    "apiVersion": "tekton.dev/v1beta1",
    "kind": "TaskRun",
    "status": {
      "startTime": "2020-04-08T01:41:11Z",
      "taskResults": [
      {
        "name": " git-commit-id",
        "value": " 80e7b850362bbd984b071fd3b21e07cf51518e16"
      }
      ]
    }
  }

Now any other task in the pipeline can read this commit ID through params:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: send-git-commit-id-to-slack
spec:
  tasks:
   - name: parse-git-commit-id
      taskRef:
        name: parse-git-commit-id
   - name: compose-slack-message
      taskRef:
        name: compose-slack-message
      params:
        - name: id
           value: “$(tasks. parse-git-commit-id.results. git-commit-id)”
…

Task results like “$(tasks. parse-git-commit-id.results. git-commit-id)” are shared across tasks using dollar notations within the Pipeline following the format: $(tasks.<task-name>.results.<task-result>).

If you’re interested in learning more about all that workspaces can do, see my previous blog Create a serverless pipeline using newly enhanced Tekton features.

2. Enable corporate proxy in a git PipelineResource

A git PipelineResource represents a git repository which is generally integrated into a Task to clone and run operations on the contents of the repo. In many enterprise environments, most of the git repositories are behind a corporate proxy and not publicly accessible. Git has to be configured to send requests through a proxy server either using certain environment variables or using git config. Git PipelineResource has been updated to enable an HTTP/S proxy and no proxy configuration through params. Enterprises can now specify their proxy setting in the PipelineResource schema:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
spec:
  type: git
  params:
    - name: url
      value: https://github.com/tekton/pipeline
    - name: httpProxy
      value: <your HTTP Proxy server>
    - name: httpsProxy
      value: <your HTTPS Proxy server>
    - name: noProxy
      value: <no proxy server>

3. PipelineResource can be marked optional

Parameters with default values are considered optional and parameters without default values are considered required. However, the same approach cannot be followed with input and output resources. This is because pipeline resources are of different types (Git, Image, Pull Request) and unable to have defaults for these resources, meaning all declared resources are considered required. With Tekton Pipeline Beta, PipelineResources has been updated with a new field called optional to mark any such PipelineResource as optional:

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
spec:
 resources:
   - name: workspace
     type: git
     optional: true
 tasks:
   …

This feature makes the following use cases possible:

  • Clone application GitHub repo and build an image if the git PipelineResource is specified.
  • Deploy to a specified cluster, but default to the cluster it is running in.

4. Simplify resource specification

PipelineResources are specified as either inputs or outputs to a task. In v1alpha1, spec.inputs have a resources and params section and spec.outputs have a resources section to specify PipelineResources like git and image. Here’s an example:

spec:
  inputs:
    resources:
      - name: source
         type: git
    params:
      - name: pathToDockerfile
         type: string
         default: /workspace/source/Dockerfile
  outputs:
    resources:
      - name: docker_image
         type: image

Here PipelineResources are referred to as $(inputs.resources.source.path) and $(outputs.resources.docker_image.url). This specification syntax has been simplified such that params is moved out of inputs with the new syntax being spec.params. There’s now a spec.resources with an inputs and outputs section. Now PipelineResources are referred to with $(resources.inputs.source.path) and $(resources.outputs.docker_image.url).

Take a look at the v1beta1 specification:

spec:
  params:
    - name: pathToDockerfile
       type: string
       default: /workspace/source/Dockerfile
  resources:
    inputs:
      - name: source
         type: git
    outputs:
      - name: docker_image
         type: image

This whole simplification effort is meant to make the migration to v1beta1 easier by dropping the resources section.

5. PipelineResources aren’t being promoted to beta

Unfortunately, PipelineResources didn’t make it to beta, which means it’s time to move on and drop resources from Tekton Tasks and Pipeline. v1beta1 is the beta API supported by Tekton Pipeline which contains PipelineRun, Pipeline, TaskRun, Task, and ClusterTask objects. PipelineResources, such as git, image, and pullrequest, are not supported by beta API and will be staying under v1alpha1. However, Tekton Pipeline does preserve compatibility of using PipelineResources with beta APIs. For example:

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: tekton-pipeline-source
spec:
  type: git
  params:
  - name: url
     value: https://github.com/tekton/pipeline.git
…
---

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
spec:
  resources:
  - name: app-source
    resourceRef:
      name: tekton-pipeline-source
…

PipelineResources continue to work with beta APIs but its usage is discouraged, and it’s recommended to use the Tekton Catalog. The Tekton Catalog contains a list of reusable Tasks authored by many different organizations. You can deploy any of these Tasks in your cluster which can then be embedded into any Pipeline. For example, git-clone is embedded into the Pipeline using:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
spec:
  workspaces:
  - name: shared-workspace
     description: “clone app repo into workspace”
  tasks:
  - name: clone-app-source
    taskRef:
      name: git-clone # this task is part of Tekton Catalog
    workspaces:
     - name: output
        workspace: shared-workspace
     params:
     - name: url
        value: https://github.com/tekton/pipeline.git
…

Note that there is no need to specify any resources in PipelineRun:

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
spec:
  pipelineRef:
    name: pipeline
  workspaces:
  - name: shared-workspace
…

Migrate to beta

While the v1alpha1 apiVersion is still supported, the Tekton community encourages the migration to v1beta1. To migrate from v1alpha1 to v1beta1, the following key changes must be applied to the existing YAML:

  • Parameters are moved from spec.inputs.params to spec.params
  • PipelineResources are moved from spec.inputs.resources and spec.outputs.resources to spec.resources.inputs and spec.resources.outputs
  • Replace PipelineResources with the Tekton Catalog
  • While adapting the Tekton Catalog, embrace using workspaces and task results to share data among multiple tasks

If you need help with the migration process, please refer to the migrating guide. I would caution using Tekton Beta Pipeline with v1alpha1 resources and suggest using kubectl replace instead of kubectl apply for v1alpha1 tasks with Tekton Beta.

Use kubectl replace instead of kubectl apply

kubectl apply -f task.yaml works great when creating v1alpha1 task for the first time.

kubectl apply -f task.yaml
task.tekton.dev/exit-0-task created

However, the same kubectl apply fails if it’s executed multiple times. For example, submitting the same v1alpha1 task results in an error.

kubectl apply -f task.yaml
Error from server (BadRequest): error when applying patch:
{"spec":{"inputs":{"params":[{"default":"0","name":"exitCode"}]},"steps":[{"image":"ubuntu","name":"exit","script":"exit $(inputs.params.exitCode)"}]}}
to:
…
for: " task.yaml ": admission webhook "validation.webhook.pipeline.tekton.dev" denied the request: validation failed: expected exactly one, got both: inputs.params, params

I am creating a very simple task here using a single parameter:

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: exit-0-task
spec:
  inputs:
    params:
      - name: exitCode
        default: "0"
  steps:
    - name: exit
      image: ubuntu
      script: exit $(inputs.params.exitCode)

To address this issue, you can use kubectl replace instead of kubectl apply to continue creating v1alpha1 tasks with Tekton Beta. kubectl replace deletes the task before creating it again.

Please note that a list of params is specified under the inputs section with v1alpha1 and referred to as $(inputs.params.exitCode). With Tekton Pipeline Beta, params specification is simplified and moved out of the inputs section. params can be specified along with steps and can be referenced with $(params.exitCode). This new structure is back ported and available with v1alpha1 to facilitate an easier migration to v1beta1. To continue using kubectl apply -f task.yaml, you will have to adapt to this new structure:

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: exit-0-task
spec:
  params:
    - name: exitCode
      default: "0"
  steps:
    - name: exit
      image: ubuntu
      script: exit $(params.exitCode)

It should be pointed out that I did not cover all of the new features and updates that Tekton Pipeline Beta has to offer. For a complete list of changes, visit the official release page.

Next steps

The Tekton team is busy working on upcoming implementations, which includes changes to the environment variable. In the meantime, make sure to stop by the new and improved Tekton website. As always, I encourage you to get involved in the Tekton community. There are plenty of projects you can contribute to. Get started today by joining the Tekton Slack workspace!