Introduction to Tekton architecture and design

Tekton is the Kubernetes-native continuous integration (CI) and continuous delivery (CD) engine. This article explains how Tekton works and how you can integrate it into an end-to-end DevOps process.

Building blocks

Tekton’s basic building blocks are called Steps and Tasks. A Step is an individual command to execute and a Task comprises one or more Steps. A Task represents a discrete, self-contained part of a process. An ordered set of Tasks is called a Pipeline. Figure 1 shows a Pipeline example that illustrates how a given Task may require several others to complete before it can proceed. Examples of many standard Tasks can be found in the Tekton Catalog.

Figure 1. A Tekton Pipeline is an ordered set of Tasks. This is an example of how a Pipeline could be constructed.

Tekton is a set of Kubernetes custom resources

Tekton is a cloud-native CI/CD system designed to run on Kubernetes and fit in naturally with existing Kubernetes mechanics. Tekton is implemented as a set of Kubernetes CustomResourceDefinitions (CRDs), which are the standard Kubernetes extension mechanisms.

Tekton introduces several new CRDs including Task, Pipeline, TaskRun, and PipelineRun. A PipelineRun represents a single running instance of a Pipeline. As shown in Figure 2, a PipelineRun is responsible for creating a Pod for each of its Tasks and as many containers within each Pod as it has Steps. (Under the covers, a PipelineRun creates one or more TaskRuns, each of which manages its associated Pod.)

Figure 2. Tekton runs scripts and other commands in containers. Steps map to containers. Tasks map to Pods. PipelineRuns are Pipeline instances, such as a single Travis build or Jenkins job. PipelineRuns manage their associated Pods (by creating TaskRuns). Pods often share a temporary filesystem backed by a PersistentVolume.

I’m often asked to compare and contrast Tekton with other CI/CD systems such as Jenkins and Travis. One short answer is that Tekton is currently optimized for building and deploying cloud-native, microservice-based applications compared to the more general-purpose nature of the others. Jenkins requires a file, generally called /Jenkinsfile, in the root directory of a project that contains its build script. Similarly, Travis looks for a /.travis.yml file. In this way, each project tends towards having its own build logic, often with significant overlap with that in parallel projects.

Tekton’s use of custom resources means that a Pipeline must exist in a namespace before it can be used. It is possible to include a /tekton directory in each project and to fetch and install that before each build. However Tekton nudges its users to centralize and standardize build logic, consolidated into a separate repository and managed under its own lifecycle. This standardization can be beneficial when scaling up to larger numbers of microservices, many of which should share build logic that is identical but for a few substitutable parameters.

Integration with the outside world

Tekton Tasks need to interact with external systems. A very simple Tekton Pipeline is of the following form:

git clone
docker build
docker tag
docker push

This Pipeline clones a Git repository, builds an image from it, and pushes that image to an external registry.

Tekton’s PipelineResource is used to model and interact with external entities such as Git repositories, image registries, pull requests, and cloud storage. PipelineResources are used as inputs to and outputs from Tasks. They also support authentication with the systems that they represent.

Credentials are stored as Kubernetes secrets and then referenced from the Kubernetes service account used to execute a given Task. PipelineResources collect their credentials from this service account. During the initialization phase of a PipelineRun, credentials are converted from their native form into one suitable for the target system. For example, Git secrets are converted and merged into a .gitconfig file and Docker secrets into .docker/config.json.

Building a DevOps pipeline

Tekton gives us the pieces that we need to build a useful DevOps pipeline. Figure 3 shows the high level flows involved in a very simple Pipeline without all the linting, testing, and image scanning involved in a real world implementation.

Figure 3. High level overview of a simple DevOps pipeline

In this example, Pipelines are initiated as a result of Git activity by a developer. See Weaveworks’ excellent What Is GitOps Really? article for more on why this model is increasingly popular.

Figure 3 shows a webhook initiating a Tekton Pipeline. Webhooks are outbound HTTP POST messages with a payload specific to the particular event provider. These payloads contain most of the information necessary to kick off a Tekton PipelineRun. I’ll explain webhooks in a bit more detail below.

The flow in Figure 3 shows the now-familiar process of building and publishing an image. The flow then goes on to show the new image being deployed to a target Kubernetes environment. This could be a dedicated namespace on the cluster where Tekton is running, or on one or more completely separate clusters. Deployment can occur immediately after build. For example, to a shared development or test environment, or it may occur separately as a result of Git commits to a second repository governing the target deployment environment.

Webhooks, triggers, and logs

Figure 3 raises some additional questions:

• How are webhooks established?
• If webhook payloads contain most of the information necessary to kick off a Tekton PipelineRun, where does the rest come from?
• How do we know that a Pipeline has run and where are the logs?

The Tekton Dashboard and Triggers project give some answers. The dashboard provides a web-based user interface for interacting with Tekton and viewing the logs from current and previous PipelineRuns.

Figure 4. Viewing a PipelineRun in the Tekton Dashboard

The Webhooks Extension dashboard adds in support for setting up webhooks and for mapping them to Pipelines with the extra information, such as target namespace, target Docker registry, and service account settings that need to be combined with the webhook payload in order to kick off a new PipelineRun. Finally, the Triggers project introduces a standard way to initiate PipelineRuns from specific events, such as merging a pull request or committing code to a particular branch.

Summary

This article provides a high level overview of Tekton architecture and design, and demonstrates how it can be adopted into a modern DevOps pipeline. Learn more by watching the lightboard video What is Tekton? with Matt Perrins. Then get started by downloading Tekton and trying a tutorial such as Build and deploy a Hello World application on Kubernetes using a Tekton pipeline.