Last week we released the YAML Templating Tool, ytt v0.1.0, which brings a new approach to YAML templating. Based on our experiences of managing complex software configurations with YAML, we believe ytt makes YAML templating easier, and in this blog post we intend to tell you why and how.

Helm and similar templating tools, treat YAML templates as text templates without taking advantage of the underlying language structure. As a result, developers have to ensure structural validity of their generated YAML configuration. This makes writing templates tricky. As an example when using Go’s text/template in HELM, patterns like the following emerge (notice indent 4, which is a manual accounting of indentations while injecting YAML elements):

metadata:
{{- with .Values.Labels  }}
  labels:
{{ toYaml . | indent 4  }}
{{- end  }}

ytt however, understands the structure of YAML configurations and uses comments to annotate structures, so that it’s no longer text templating, but YAML structure-aware templating. For example, you would rewrite the example above with ytt like this:

metadata:
  labels: #@ data.labels

Read on if you’re curious or visit get-ytt.io.

Templating tool characteristics

While domain-specific languages like Jsonnet and frameworks like Pulumi have their appeal in the community, and despite frequent backlash on templating YAML, the general Kubernetes community continues to text-template YAML. This can be observed looking at the success of HELM and all the community provided HELM charts, along with community best practices shared by companies like Airbnb.

Looking at various discussions such as here, here, and here, we identified the following set of goals that should improve templating of configuration files:

1. Structure-aware templating instead of text templating
2. Be declarative but also support imperative operations (conditonals, loops)
3. Allow modularization of configuration structures (functions)
4. Support data injection
5. Allow data validation

While text templating YAML configurations only fulfill a subset of these requirements, ytt addresses all the points above.

What does ytt do?

The YAML Templating Tool (ytt) is a new take on YAML templating, and when we say templating what we really mean is data structure building.

At its core, ytt is the idea that we can use YAML comments to annotate YAML structures (such as maps, arrays, and even documents) with necessary instructions and metadata. Some annotations carry declarative information and some are imperative.

Below, we outline the different ways that you can incorporate ytt into your Kubernetes (or other container) plan. Let’s dive in.

1. Use structure-aware templating instead of text templating.

   posts:
     - title: #@ "Some #ytt title"
   

   Show more

   Which results in:

   posts:
    - title: 'Some #ytt title'
   

   Show more

   In the above example, we have a single annotation that is attached to a map item (with key title). Given that ytt understands YAML structure, it sees this as an operation to set the map item value to a string "Some #ytt title". Note that the string value contains # and it will be correctly serialized by the YAML library.

2. Be declarative but also support imperative operations.

   In most templates, conditionals and loops are necessary at some point to achieve correct end results. ytt supports imperative programming by including a Python-like language called Starlark (though it’s been slightly enhanced) to provide a fully featured scripting environment. Starlark is also the language that is used by the Bazel build system.

   #@ titles = ["1st title", "super long title"]
   
   posts:
   #@ for title in titles:
   - title: #@ title
     short_title: #@ title if len(title) < 10 else title[:10]+"..."
   #@ end
   

   Show more

   Which results in:

   posts:
   - title: 1st title
     short_title: 1st title
   - title: super long title
     short_title: super long ...
   

   Show more

3. Allow modularization of configuration structures.

   The example above could be rewritten as:

   #@ titles = ["1st title", "super long title"]
   
   #@ def post(title):
   title: #@ title
   short_title: #@ title if len(title) < 10 else title[:10]+"..."
   #@ end
   
   posts:
   #@ for title in titles:
   - #@ post(title)
   #@ end
   

   Show more

4. Support data injection.

   Given that most templates require some kind of input and typically want to validate it, ytt allows you to set up assertions within your templates. For example:

   #@data/values
   ---
   titles:
   - 1st title
   - super long title
   

   Show more

   #@ load("@ytt:data", "data")
   #@ load("@ytt:assert", "assert")
   
   #@ def post(title):
   #@   if len(title) < 5:
   #@     assert.fail("Title '{}' is too short. Minimum 5 chars.".format(title))
   #@   end
   title: #@ title
   short_title: #@ title if len(title) < 10 else title[:10]+"..."
   #@ end
   
   posts:
   #@ for/end title in data.titles:
   - #@ post(title)
   

   Show more

Syntactic sugar

As a nice addition, a little syntactic sugar is available within ytt for writing conditionals, loops, and functions that are bound to a single YAML structure. For example, take a look at the following:

#@ if True:
post:
  title: some-title
  authors:
  - Dmitriy
  - Nima
#@ end

The above could be rewritten like the following, which avoids closing if with an end:

#@ if/end True:
post:
  title: some-title
  authors:
  - Dmitriy
  - Nima

Comparing ytt to templating in Helm

Let’s have a look at a larger example template taken from one of the published Helm charts and see how it looks when converted to ytt. This example matches the Helm scaffold chart that you get when doing helm create. Notice that for brevity, we have only compared the ingress.yaml templates and a subset of the helper functions between Helm and ytt. The full ytt example can be found on the get-ytt.io interactive playground.

The Kubernetes Ingress YAML with Go text/template in Helm would look like the following (click on “Show More” to see the full snippet):

{{- if .Values.ingress.enabled -}}
{{- $fullName := include "fullname" . -}}
{{- $ingressPath := .Values.ingress.path -}}
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: {{ $fullName }}
  labels:
    app.kubernetes.io/name: {{ include "fullname" . }}
    helm.sh/chart: {{ include "chart" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- with .Values.ingress.annotations }}
  annotations:
{{ toYaml . | indent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.tls }}
  tls:
  {{- range .Values.ingress.tls }}
    - hosts:
      {{- range .hosts }}
        - {{ . | quote }}
      {{- end }}
  {{- end }}
{{- end }}
  rules:
  {{- range .Values.ingress.hosts }}
    - host: {{ . | quote }}
      http:
        paths:
          - path: /
            backend:
              serviceName: {{ $fullName }}
              servicePort: http
  {{- end }}
{{- end }}

Now take the same Ingress YAML, but with the ytt template:

#@ load("@ytt:data", "data")
#@ load("helpers.star", "fullname")

#@ def labels(vars):
app.kubernetes.io/name: #@ fullname(vars)
helm.sh/chart: #@ "{}-{}".format(vars.Chart.Name, vars.Chart.Version).replace("+", "_")
app.kubernetes.io/instance: #@ vars.Release.Name
app.kubernetes.io/managed-b: #@ vars.Release.Service
#@ end

#@ if/end data.values.ingress.enabled:
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: #@ fullname(data.values)
  labels: #@ labels(data.values)
  #@ if/end data.values.ingress.annotations:
  annotations: #@ data.values.ingress.annotations

spec:
  #@ if/end data.values.ingress.tls:
  tls: #@ data.values.ingress.tls

  rules:
  #@ for/end host in data.values.ingress.hosts:
  - host: #@ host
    http:
      paths:
        - path: /
          backend:
            serviceName: #@ fullname(data.values)
            servicePort: #@ data.values.service.externalPort

Let’s look at the required helper function with Go text/template in Helm:

{{- define "fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Chart.Name -}}
{{- .Chart.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Chart.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}

With ytt, however, we would have a pythonic implementation:

def fullname(vars):
  if vars.fullnameOverride:
    return _clean_name(vars.fullnameOverride)
  end
  if vars.nameOverride:
    return _clean_name("{}-{}".format(vars.Chart.name, vars.nameOverride))
  end
  return _clean_name(vars.Chart.name)
end

def _clean_name(name):
  return name[:63].rstrip("-")
end

Summary

ytt can be used to template any YAML configurations such as those needed in Kubernetes, CloudFoundry BOSH manifests, Concourse pipeline definitions, and more. Furthermore, ytt also provides a way to overlay configuration structures via an ‘overlay’ feature similar but better than Kustomize and go-patch tools. For brevity, we did not discuss the ‘overlay’ feature in this post, and leave it to the reader to explore them. We have also written a comparison of ytt-vs-x with x being the existing set of common tools and frameworks for configuration management.

ytt was developed by Dmitriy Kalinin of Pivotal and Nima Kaviani of IBM. Dmitriy and Nima have contributed to Cloud Foundry in the past with Dmitriy serving as the Product Manager for CF BOSH and Nima being a core contributor to Cloud Foundry’s Diego runtime. They are also involved with Kubernetes and Knative communities.

Our hope is for ytt to find an appeal in the community and help with simplifying configuration and deployment of software on top of Kubernetes. We would love it if you try ytt and let us know what you think. The good news is ytt is fully open source for you to try it under get-ytt in GitHub.