Lookup Kubernetes Resources inside Helm Charts

Illustration split in two panels showing a person adjusting their glasses, with frustration in the first panel and with confidence in the second, symbols and the word "HELM" are between panels. Illustration split in two panels showing a person adjusting their glasses, with frustration in the first panel and with confidence in the second, symbols and the word "HELM" are between panels.

Did you know that you can query Kubernetes API directly from your Helm templates? If you didn’t, let me show this feature to you.

Kubernetes audit: it's a complex framework, and it's tricky to get it right. We are here to help you with that. About Kubernetes audits

Let’s say, we want to allow our Kubernetes users to scale down every deployment in their namespaces to zero at midnight - for example, because we want to reduce our cloud bill by shutting down the development environments.

First, let me start the local Kubernetes instance.

A screenshot of a computer terminal with text showing the process of starting Minikube, preparing Kubernetes, and a warning about kubectl version incompatibility.

I will create two new namespaces and label them with “mkdev.me/park”. I will then create nginx deployments in both of those namespaces.

A computer terminal screen displaying command-line interface interactions, showing executed commands related to Kubernetes namespaces and labeling them, with the prompt 'helm-lookup' visible.

I’ve already prepared a simple Helm chart that we are going to extend. Right now this chart has a static CronJob, called “turn-off”. To make useful this CronJob useful, we need to get the list of all the namespace that have the certain label. To achieve this I can use the lookup function.

A screenshot of a computer screen displaying a terminal with an open text editor, Vim, showing YAML configuration for a Kubernetes cron job with code and file structure visible on the left side.

The lookup function accepts just a few arguments - the api version, the resource type, the resource name and the namespace where the resource is located. The result that the function will return will be a dictionary, which either has the particular resource or, in this case, a single key called “items”, that contains everything that matches the lookup.

To fetch all the namespaces, we need to use “v1”, resource type Namespace and leave two other fields empty. Now we can loop through each namespace with the help of the range function. Lookup function doesn't let us to narrow down the search via labels, so we have to check the labels of the namespace inside the loop.

$namespace variable will contain every field of the namespace and we can access each by using the regular “dot” notation. I am adding an if statement that checks the presence of mkdev.me/park label. If it does exist, will create create a new CronJob.

I will use the namespace name inside the CronJob metadata, so that each CronJob belongs to the namespace that it scales down.

Screenshot of a computer terminal with code open in Vim editor, showing a Kubernetes cron job configuration YAML file.

Now I need to modify the command a bit, so that we can provide the whole script in a multiline YAML field.

Inside the last argument, we need to invoke the lookup function one more time, this time to fetch every Deployment that namespace has - we already have the namespace name in the $namespace.metadata.name:

lookup "v1/apps" "Deployment" "" $namespace.metadata.name

Inside this loop, we will run kubectl:

kubectl scale deploy $deployment.metadata.name -n $namespace.metadata.name --replicas=0

Now let’s do a dry run of our chart. We can see that Helm would create two new CronJobs, each of those scaling down the nginx inside the namespace where the deployment is located.

Let’s install this chart and verify that it does indeed create those two CronJobs.

Screenshot of a computer terminal displaying the output of a 'kubectl get cronjobs' command with entries for 'proj-a-dev' and 'proj-b-dev' namespaces.

Helm lookup function is a curious feature with certain caveats.

One of them is that you have to regularly apply your chart, so that it picks up the latest changes to the API resources that it looks up - this issue can be handled by applying your chart regularly, for example with some tools like ArgoCD.

Another issue is that it in many cases it could be easier to just write a script that does the same job - in our example with the “turn-off” CronJob, we could deploy a simply Python script to do the same.

Then, of course, you would need to know how to code, and then package this script to a container image etc etc, which, in certain situations, might be too much to ask for.

You probably won’t need the Helm lookup function too often, but when you do it might prove really handy and time saving for minor Kubernetes automations.

Here's the same article in video form, so you can listen to it on the go: