Helm is an easy way of deploying to Kubernetes, but
helm install is a bit annoying because it doesn’t save the changes it made to a local repo. That’s where
helm template comes in.
helm install vs
As a concrete example, let’s talk about installing Gitlab Runner from the Helm chart. This is the part of Gitlab that runs CI builds. Hosting it ourselves is cheaper (and faster) than using gitlab.com. Following the docs, we’re supposed to install it like this:
$ helm repo add gitlab https://charts.gitlab.io/ $ helm repo update $ helm install gitlab-runner gitlab/gitlab-runner \ --namespace gitlab-runner-ab \ --create-namespace \ --version 0.35.3 \ -f gitlab-runner-values-ab.yaml
This is easy to run, but now we have to remember to run it every time we rebuild the cluster. We can no longer just
kubectl apply a directory to have everything installed.
Secondly, it’s unfortunate that we don’t have a local listing of what changes were made to the cluster. Kubernetes makes it very easy to describe what components a system has, and it’s sad to lose that. Installing charts feels like going back to the dark days of looking at a server, and wondering what was
apt-get install’d, and what configuration files where changed.
helm template. It takes similar arguments to
helm install, but instead of changing the cluster, it prints out a YAML with the changes. The command is usually mentioned in the context of writing charts, and inspecting their output, but there’s no reason we can’t use it with somebody else’s charts:
$ helm template gitlab-runner gitlab/gitlab-runner \ --namespace gitlab-runner-ab \ --version 0.35.3 \ -f gitlab-runner-values-ab.yaml \ > gitlab-runner.yaml ### We save the output to a file to apply later. $ git add gitlab-runner.yaml $ kubectl apply \ --namespace gitlab-runner-ab \ -f gitlab-runner.yaml
template command is something we’ll run again whenever we want to change the version, so let’s put it in a
update: helm repo add gitlab "https://charts.gitlab.io/" || true helm repo update gitlab helm template gitlab-runner gitlab/gitlab-runner \ --namespace gitlab-runner-ab \ --version 0.35.3 \ -f gitlab-runner-values-ab.yaml \ > gitlab-runner.yaml
helm template flags
Section added 2021-12-18
A few caveats come with using
helm template. The first is that Helm charts have hooks which only run at specific Helm lifecycle events like install, upgrade, or delete. By default,
helm template dumps all of them, so if we’re not careful, we might end up with uninstall batch jobs in our manifest (e.g. the Longhorn chart does exactly this). We need to pass the
--no-hooks flag to disable this behaviour, and then we have to be careful to run any necessary hooks ourselves when appropriate.
The second caveat is that we need an extra flag to output CRDs. The Helm best practices recommend putting CRD declarations in a separate directory instead of interleaving them with other resources in the chart. By default,
helm template doesn’t output these. We need to pass the
--include-crds flag to change this behaviour.
Another problem is that
--create-namespace doesn’t work with
helm template, so the namespace ends up missing. We can create it ourselves with a kustomization:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: gitlab-runner-ab resources: - namespace.yaml - gitlab-runner.yaml
apiVersion: v1 kind: Namespace metadata: name: gitlab-runner.yaml
Putting it all together, we end up with the following directory layout:
gitlab-runner ├── gitlab-runner-values-ab.yaml ├── gitlab-runner.yaml ├── kustomization.yaml ├── Makefile └── namespace.yaml
We apply this with
kubectl apply -k gitlab-runner/, and we regenerate the manifest from the chart with
make -C gitlab-runner/ update. We can also add this directory to a higher level kustomization, so that installing everything into the cluster is just one command:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - gitlab-runner/ - ... other apps to deploy in the cluster ...
Doing all this, we lose Helm’s version tracking and upgrade/rollback support. On the other hand, since we store all the configuration in a repo, we can easily reason about version changes ourselves. I personally find this less stressful than hoping that Helm and the chart work properly.