Helm is an easy way of deploying to Kubernetes, but helm install
is a bit annoying because it doesn’t save the changes it made to a local repo. That’s where helm template
comes in.
helm install
vs helm template
As a concrete example, let’s talk about installing Gitlab Runner from the Helm chart. This is the part of Gitlab that runs CI builds. Hosting it ourselves is cheaper (and faster) than using gitlab.com. Following the docs, we’re supposed to install it like this:
$ helm repo add gitlab https://charts.gitlab.io/
$ helm repo update
$ helm install gitlab-runner gitlab/gitlab-runner \
--namespace gitlab-runner-ab \
--create-namespace \
--version 0.35.3 \
-f gitlab-runner-values-ab.yaml
This is easy to run, but now we have to remember to run it every time we rebuild the cluster. We can no longer just kubectl apply
a directory to have everything installed.
Secondly, it’s unfortunate that we don’t have a local listing of what changes were made to the cluster. Kubernetes makes it very easy to describe what components a system has, and it’s sad to lose that. Installing charts feels like going back to the dark days of looking at a server, and wondering what was apt-get install
’d, and what configuration files where changed.
Enter helm template
. It takes similar arguments to helm install
, but instead of changing the cluster, it prints out a YAML with the changes. The command is usually mentioned in the context of writing charts, and inspecting their output, but there’s no reason we can’t use it with somebody else’s charts:
$ helm template gitlab-runner gitlab/gitlab-runner \
--namespace gitlab-runner-ab \
--version 0.35.3 \
-f gitlab-runner-values-ab.yaml \
> gitlab-runner.yaml
### We save the output to a file to apply later.
$ git add gitlab-runner.yaml
$ kubectl apply \
--namespace gitlab-runner-ab \
-f gitlab-runner.yaml
The template
command is something we’ll run again whenever we want to change the version, so let’s put it in a Makefile
:
update:
helm repo add gitlab "https://charts.gitlab.io/" || true
helm repo update gitlab
helm template gitlab-runner gitlab/gitlab-runner \
--namespace gitlab-runner-ab \
--version 0.35.3 \
-f gitlab-runner-values-ab.yaml \
> gitlab-runner.yaml
Makefile
Important helm template
flags
Section added 2021-12-18
A few caveats come with using helm template
. The first is that Helm charts have hooks which only run at specific Helm lifecycle events like install, upgrade, or delete. By default, helm template
dumps all of them, so if we’re not careful, we might end up with uninstall batch jobs in our manifest (e.g. the Longhorn chart does exactly this). We need to pass the --no-hooks
flag to disable this behaviour, and then we have to be careful to run any necessary hooks ourselves when appropriate.
The second caveat is that we need an extra flag to output CRDs. The Helm best practices recommend putting CRD declarations in a separate directory instead of interleaving them with other resources in the chart. By default, helm template
doesn’t output these. We need to pass the --include-crds
flag to change this behaviour.
Namespaces
Another problem is that --create-namespace
doesn’t work with helm template
, so the namespace ends up missing. We can create it ourselves with a kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: gitlab-runner-ab
resources:
- namespace.yaml
- gitlab-runner.yaml
kustomization.yaml
apiVersion: v1
kind: Namespace
metadata:
name: gitlab-runner.yaml
namespace.yaml
Conclusion
Putting it all together, we end up with the following directory layout:
gitlab-runner
├── gitlab-runner-values-ab.yaml
├── gitlab-runner.yaml
├── kustomization.yaml
├── Makefile
└── namespace.yaml
We apply this with kubectl apply -k gitlab-runner/
, and we regenerate the manifest from the chart with make -C gitlab-runner/ update
. We can also add this directory to a higher level kustomization, so that installing everything into the cluster is just one command:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gitlab-runner/
- ... other apps to deploy in the cluster ...
kustomization.yaml
Doing all this, we lose Helm’s version tracking and upgrade/rollback support. On the other hand, since we store all the configuration in a repo, we can easily reason about version changes ourselves. I personally find this less stressful than hoping that Helm and the chart work properly.