Dive into operators, Part 1: An architectural overview
Extend the functionality of a Kubernetes cluster
Operators are an exciting new way for application developers and system administrators to extend the functionality of a Kubernetes cluster. However, given the complicated internal behavior of Kubernetes, it can be helpful to understand exactly how an operator functions before you install or create one.
To understand operators, you need to take a step back and understand how two fundamental parts of core Kubernetes work: resources and controllers. This article gives a high-level overview of the architecture underlying Kubernetes operators, including the resource/controller watch-loop that core Kubernetes uses to control the system, and how it is replicated by an operator.
Resources and controllers
Resources and controllers are the primary mechanism through which you effect change to what is running on a Kubernetes cluster. Stateful information, such as the specifications of running pods, replica sets, services, and other assets, are contained in resources stored in a central
etcd store. When you want to make something happen, you create or modify resources in this store through an interface like
kubectl. The following example uses a hypothetical
kubectl create example_resource c command only writes data to the store. It doesn’t actually make an
example_resource appear in the cluster, however. That is the job of the controller. There are many controllers running, responsible for many different resources and tasks throughout the cluster. Each controller is set to watch a specific portion of the store that contains data about what that controller is responsible for.
When the state that a controller is watching changes, that controller performs whatever changes to the system are required to arrive at the new state such as starting or stopping pods, adding networking entries, and so on. This behavior varies greatly from controller to controller depending on what behavior that controller is responsible for. It often involves other Kubernetes components, or even potentially systems exterior to the cluster entirely.
After the controller has changed the running state to match the requested state, it returns to watching the store, waiting for something new to change. This watch-change-watch loop is the central process that all of Kubernetes is based on. At their core, operators are a means for developers, operators, and admins to add new watch-loops to their clusters.
Operators consist of two parts: a custom resource definition (CRD) and a controller process. The CRD is added to the store to contain the data needed to represent whatever functionality the operator is providing. The controller is run in one or more pods. Together, they function the same way as the core Kubernetes controllers do. They set a watch loop that waits for user input, and then starts and stops things to reflect that input.
This architecture make Kubernetes operators an extremely powerful tool. You can control and modify the system the same way Kubernetes runs itself. However, the internal logic of the controller watch-loop can be quite complex, especially if the operator interfaces with external systems. The
OperatorSDK tool and the
controller-runtime package are intended to simplify the internal logic to make it easier to write and use operators for the average user. The Kubernetes
controller-runtime is a fundamental package, which provides the API to create the operators. with the
Kubebuilder tools, you can generate the code of the operator (for example, CRDs, reconciler, controller), based on the
Kubebuilder make it easier to write the source code of operators, based on the
Now you have an architecture overview to guide you as you dive into using Kubernetes operators. Read on through the rest of the series for examples and tutorials of using operators. The Pass configuration to Kubernetes operators with kustomize tutorial shows examples of using kustomize and working with the Operator Framework. You see a demonstration of a single configuration and lear how to use kustomize to patch configurations of other forms, such as
json patch and runtime data with variables.
The Design and create operators based on Kubernetes controller-runtime tutorial puts the Operator Framework into practice, demonstrating how to combine the power of OLM and kustomize to manage applications. Part 4, the Design and create operators based on Knative Common Packages tutorial, shows another approach with Knative Common Packages.