Blog Post
ytt: The YAML Templating Tool that simplifies complex configuration management
This blog post introduces ytt v0.1.0, a YAML templating tool that aims to simplify complex configuration management.
Last week we released the YAML Templating Tool, ytt v0.1.0, which brings a new approach to YAML templating. Based on our experiences of managing complex software configurations with YAML, we believe ytt makes YAML templating easier, and in this blog post we intend to tell you why and how.
Helm and similar templating tools, treat YAML templates as text templates without taking advantage of the underlying language structure. As a result, developers have to ensure structural validity of their generated YAML configuration. This makes writing templates tricky. As an example when using Go’s text/template in HELM, patterns like the following emerge (notice indent 4
, which is a manual accounting of indentations while injecting YAML elements):
metadata:
{{- with .Values.Labels }}
labels:
{{ toYaml . | indent 4 }}
{{- end }}
ytt however, understands the structure of YAML configurations and uses comments to annotate structures, so that it’s no longer text templating, but YAML structure-aware templating. For example, you would rewrite the example above with ytt like this:
metadata:
labels: #@ data.labels
Read on if you’re curious or visit get-ytt.io.
Templating tool characteristics
While domain-specific languages like Jsonnet and frameworks like Pulumi have their appeal in the community, and despite frequent backlash on templating YAML, the general Kubernetes community continues to text-template YAML. This can be observed looking at the success of HELM and all the community provided HELM charts, along with community best practices shared by companies like Airbnb.
Looking at various discussions such as here, here, and here, we identified the following set of goals that should improve templating of configuration files:
- Structure-aware templating instead of text templating
- Be declarative but also support imperative operations (conditonals, loops)
- Allow modularization of configuration structures (functions)
- Support data injection
- Allow data validation
While text templating YAML configurations only fulfill a subset of these requirements, ytt addresses all the points above.
What does ytt do?
The YAML Templating Tool (ytt) is a new take on YAML templating, and when we say templating what we really mean is data structure building.
At its core, ytt is the idea that we can use YAML comments to annotate YAML structures (such as maps, arrays, and even documents) with necessary instructions and metadata. Some annotations carry declarative information and some are imperative.
Below, we outline the different ways that you can incorporate ytt into your Kubernetes (or other container) plan. Let’s dive in.
Use structure-aware templating instead of text templating.
posts: - title: #@ "Some #ytt title"
Which results in:
posts: - title: 'Some #ytt title'
In the above example, we have a single annotation that is attached to a map item (with key
title
). Given that ytt understands YAML structure, it sees this as an operation to set the map item value to a string"Some #ytt title"
. Note that the string value contains#
and it will be correctly serialized by the YAML library.Be declarative but also support imperative operations.
In most templates, conditionals and loops are necessary at some point to achieve correct end results. ytt supports imperative programming by including a Python-like language called Starlark (though it’s been slightly enhanced) to provide a fully featured scripting environment. Starlark is also the language that is used by the Bazel build system.
#@ titles = ["1st title", "super long title"] posts: #@ for title in titles: - title: #@ title short_title: #@ title if len(title) < 10 else title[:10]+"..." #@ end
Which results in:
posts: - title: 1st title short_title: 1st title - title: super long title short_title: super long ...
Allow modularization of configuration structures.
The example above could be rewritten as:
#@ titles = ["1st title", "super long title"] #@ def post(title): title: #@ title short_title: #@ title if len(title) < 10 else title[:10]+"..." #@ end posts: #@ for title in titles: - #@ post(title) #@ end
Support data injection.
Given that most templates require some kind of input and typically want to validate it, ytt allows you to set up assertions within your templates. For example:
#@data/values --- titles: - 1st title - super long title
#@ load("@ytt:data", "data") #@ load("@ytt:assert", "assert") #@ def post(title): #@ if len(title) < 5: #@ assert.fail("Title '{}' is too short. Minimum 5 chars.".format(title)) #@ end title: #@ title short_title: #@ title if len(title) < 10 else title[:10]+"..." #@ end posts: #@ for/end title in data.titles: - #@ post(title)
Syntactic sugar
As a nice addition, a little syntactic sugar is available within ytt for writing conditionals, loops, and functions that are bound to a single YAML structure. For example, take a look at the following:
#@ if True:
post:
title: some-title
authors:
- Dmitriy
- Nima
#@ end
The above could be rewritten like the following, which avoids closing if
with an end
:
#@ if/end True:
post:
title: some-title
authors:
- Dmitriy
- Nima
Comparing ytt to templating in Helm
Let’s have a look at a larger example template taken from one of the published Helm charts and see how it looks when converted to ytt. This example matches the Helm scaffold chart that you get when doing helm create
. Notice that for brevity, we have only compared the ingress.yaml
templates and a subset of the helper functions between Helm and ytt. The full ytt example can be found on the get-ytt.io interactive playground.
The Kubernetes Ingress YAML with Go text/template in Helm would look like the following (click on “Show More” to see the full snippet):
{{- if .Values.ingress.enabled -}}
{{- $fullName := include "fullname" . -}}
{{- $ingressPath := .Values.ingress.path -}}
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: {{ $fullName }}
labels:
app.kubernetes.io/name: {{ include "fullname" . }}
helm.sh/chart: {{ include "chart" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- with .Values.ingress.annotations }}
annotations:
{{ toYaml . | indent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ . | quote }}
http:
paths:
- path: /
backend:
serviceName: {{ $fullName }}
servicePort: http
{{- end }}
{{- end }}
Now take the same Ingress YAML, but with the ytt template:
#@ load("@ytt:data", "data")
#@ load("helpers.star", "fullname")
#@ def labels(vars):
app.kubernetes.io/name: #@ fullname(vars)
helm.sh/chart: #@ "{}-{}".format(vars.Chart.Name, vars.Chart.Version).replace("+", "_")
app.kubernetes.io/instance: #@ vars.Release.Name
app.kubernetes.io/managed-b: #@ vars.Release.Service
#@ end
#@ if/end data.values.ingress.enabled:
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: #@ fullname(data.values)
labels: #@ labels(data.values)
#@ if/end data.values.ingress.annotations:
annotations: #@ data.values.ingress.annotations
spec:
#@ if/end data.values.ingress.tls:
tls: #@ data.values.ingress.tls
rules:
#@ for/end host in data.values.ingress.hosts:
- host: #@ host
http:
paths:
- path: /
backend:
serviceName: #@ fullname(data.values)
servicePort: #@ data.values.service.externalPort
Let’s look at the required helper function with Go text/template in Helm:
{{- define "fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Chart.Name -}}
{{- .Chart.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Chart.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}
With ytt, however, we would have a pythonic implementation:
def fullname(vars):
if vars.fullnameOverride:
return _clean_name(vars.fullnameOverride)
end
if vars.nameOverride:
return _clean_name("{}-{}".format(vars.Chart.name, vars.nameOverride))
end
return _clean_name(vars.Chart.name)
end
def _clean_name(name):
return name[:63].rstrip("-")
end
Summary
ytt can be used to template any YAML configurations such as those needed in Kubernetes, CloudFoundry BOSH manifests, Concourse pipeline definitions, and more. Furthermore, ytt also provides a way to overlay configuration structures via an ‘overlay’ feature similar but better than Kustomize and go-patch tools. For brevity, we did not discuss the ‘overlay’ feature in this post, and leave it to the reader to explore them. We have also written a comparison of ytt-vs-x with x
being the existing set of common tools and frameworks for configuration management.
ytt was developed by Dmitriy Kalinin of Pivotal and Nima Kaviani of IBM. Dmitriy and Nima have contributed to Cloud Foundry in the past with Dmitriy serving as the Product Manager for CF BOSH and Nima being a core contributor to Cloud Foundry’s Diego runtime. They are also involved with Kubernetes and Knative communities.
Our hope is for ytt to find an appeal in the community and help with simplifying configuration and deployment of software on top of Kubernetes. We would love it if you try ytt and let us know what you think. The good news is ytt is fully open source for you to try it under get-ytt in GitHub.