Before You Start, Create Your Tools | ✉️ #51

Illustration for MKDEV Dispatch #51 featuring a smiling person holding a small dog, with a paper airplane thought bubble, on an orange background with the title "Before you start, create your tools."

Hey! 👋

I’ve been reading Tools for builders essay and couldn’t stop thinking how much it applies to cloud (or any) infrastructure work. While working with automating, scaling and securing infrastructure, we don’t write that much customer-facing software. It’s all about architecture, picking the right tools and configuring and integrating them properly. Writing code in most cases is either about tweaking applications to, for example, properly communicate with cloud APIs, or properly handle secrets, errors, retries, traces collection and so on. Or, depending on the level of customer maturity, we create use-case by use-case internal/developer-oriented utilities. From these utilities and custom automations, eventually, internal platform emerges.

But there is another category of software that is an essential part of infrastructure work that is less often talked about due to an ephemeral nature of this software. I am talking, of course, about those beautiful small scripts, usually written, depending on complexity, in Bash or Python. Such software often has a lifespan of few months, doesn’t get much attention, but saves so much time - and improves the quality of time spent working on a particular project.

As much as it’s fun to work on creating those little scripts, we always need to remember the ROI of creating them. There is a famous xkcd 2013 comic on the topic of automating routine tasks:

A chart titled 'How long can you work on making a routine task more efficient before you're spending more time than you save?' calculating time saved over five years.

I used to do such calculations all the time. It might take just a couple of hours to write a Python utility that takes some inputs and performs many boring steps for me, but quiet often just doing those steps manually will take the same couple of hours. I used to just slow the bitter pill of the need to do things manually. But, since many months, I don’t.

You see, even though AI is not incredible at working on complex codebases, it’s insanely useful for writing those scripts. Just throw a sequence of steps that your script needs to perform into Claude (or GPT, but Claude does a way better job) and you get a working script within seconds - yes, today, the script is already working from the first attempt. The whole calculation of whether the ROI of automating some task with the code is good enough stops making sense. It is good enough, because creating any simple automation, including few modifications with subsequent queries to LLM, takes minutes, not hours.

This is one of those areas where productivity gains are insane. I can often clearly see hours of time saved just by describing the task to an advanced LLM, take the resulting script and applying to my problem. And then handing it over to the colleagues so that they can save time too. Building your own tools was never as exciting as now.

The only catch I see, is that with those of us, who came to the DevOps/Infra/SRE world from operations and not from software development, it will be even harder to properly learn software development. There is simply less need to do so - that narrow set of tasks where writing code is really required for the job is almost fully covered by AI today.

And somehow, I ended up writing another intro around AI.

Enjoy your weekend!

What We've Shared

What’s It Like for Women in Tech? Here's a part of the previous DA episode as usual.
DevOps Accents #43: What Drives Job Satisfaction with Fabiano Beselga from ada. Fabiano Beselga is the Head of Technology at ada, music composer and YouTuber. He joins Leo, Pablo and Kirill for a discussion of his career path, work-life balance and product discovery.

And on the website we have articles from both Pablo and Kirill:

Service Weaver, Monolithic or Microservice? Pablo Inigo Sanchez examines the complexities of microservices deployment and highlights Google's Service Weaver, a tool designed to streamline the development of distributed applications, addressing many common challenges.
What is Open Container Initiative (OCI)? Build Spec, Runtime Spec, Image Spec and more. This article delves into container technology, explaining how container runtimes utilize cgroups and namespaces for security and efficiency. It highlights the Open Containers Initiative's role in standardizing these tools to ensure compatibility across systems.

What We've Discovered

The AI summer: A realistic view on the current state of GenAI and LLMs. One quote that stands out:

A lot of these charts are really about what happens when the utopian dreams of AI maximalism meet the messy reality of consumer behaviour and enterprise IT budgets - it takes longer than you think, and it’s complicated (this is also one reason why I think ‘doomers’ are naive).

As always, excellent analytics from Ben Evans.

Host Website with Terraform using CloudFront, S3, and Authentication via AWS Cognito: A simple example of how to protect static website with Cognito, terraform code included.
The Reality of Adding Nines to Your SLOs: Cost and ROI of increasing reliability of your services, visualized.
How to build a Data Pipeline with AWS Glue and Terraform: A step-by-step guide to an ETL project that explores Australian property price, using AWS and terraform.
Terraform at LumApps: A 3-part series on the structure and automation of LumaApps self service infrastructure platform.

A random reminder

If you need to access all our articles, free video courses, infographics or contact us to get help with Argo CD, you can always use the dedicated Argo CD page on our website!

The 52nd mkdev dispatch will arrive on Friday, August 30th. See you next time!