Lookup Kubernetes Resources inside Helm Charts

Did you know that you can query Kubernetes API directly from your Helm templates? If you didn’t, let me show this feature to you.

Kubernetes audit: it's a complex framework, and it's tricky to get it right. We are here to help you with that. About Kubernetes audits

Let’s say, we want to allow our Kubernetes users to scale down every deployment in their namespaces to zero at midnight - for example, because we want to reduce our cloud bill by shutting down the development environments.

First, let me start the local Kubernetes instance.

I will create two new namespaces and label them with “mkdev.me/park”. I will then create nginx deployments in both of those namespaces.

I’ve already prepared a simple Helm chart that we are going to extend. Right now this chart has a static CronJob, called “turn-off”. To make useful this CronJob useful, we need to get the list of all the namespace that have the certain label. To achieve this I can use the lookup function.

The lookup function accepts just a few arguments - the api version, the resource type, the resource name and the namespace where the resource is located. The result that the function will return will be a dictionary, which either has the particular resource or, in this case, a single key called “items”, that contains everything that matches the lookup.

To fetch all the namespaces, we need to use “v1”, resource type Namespace and leave two other fields empty. Now we can loop through each namespace with the help of the range function. Lookup function doesn't let us to narrow down the search via labels, so we have to check the labels of the namespace inside the loop.

$namespace variable will contain every field of the namespace and we can access each by using the regular “dot” notation. I am adding an if statement that checks the presence of mkdev.me/park label. If it does exist, will create create a new CronJob.

I will use the namespace name inside the CronJob metadata, so that each CronJob belongs to the namespace that it scales down.

Now I need to modify the command a bit, so that we can provide the whole script in a multiline YAML field.

Inside the last argument, we need to invoke the lookup function one more time, this time to fetch every Deployment that namespace has - we already have the namespace name in the $namespace.metadata.name:

lookup "v1/apps" "Deployment" "" $namespace.metadata.name

Inside this loop, we will run kubectl:

kubectl scale deploy $deployment.metadata.name -n $namespace.metadata.name --replicas=0

Now let’s do a dry run of our chart. We can see that Helm would create two new CronJobs, each of those scaling down the nginx inside the namespace where the deployment is located.

Let’s install this chart and verify that it does indeed create those two CronJobs.

Helm lookup function is a curious feature with certain caveats.

One of them is that you have to regularly apply your chart, so that it picks up the latest changes to the API resources that it looks up - this issue can be handled by applying your chart regularly, for example with some tools like ArgoCD.

Another issue is that it in many cases it could be easier to just write a script that does the same job - in our example with the “turn-off” CronJob, we could deploy a simply Python script to do the same.

Then, of course, you would need to know how to code, and then package this script to a container image etc etc, which, in certain situations, might be too much to ask for.

You probably won’t need the Helm lookup function too often, but when you do it might prove really handy and time saving for minor Kubernetes automations.

Here's the same article in video form, so you can listen to it on the go: