Thoughts on Kamal | ✉️ #30
Hey! 👋
There is a new kid on the block, that promises to simplify containerized applications, and it’s called Kamal. Inspired by Capistrano and Heroku, Kamal lets you deploy your apps in seconds, without spending hours on setting up a Kubernetes cluster. What it does is - SSH into your servers, install Docker, start containers. When you deploy, it repeats this exercise. There is already support for secrets management, “accessory services” and few other essential features to get you up and running quickly.
What I love about Kamal is the general approach. It wraps dozens of Docker commands into a nice declarative manifest, not unlike Docker Compose, but multi-host ready and a bit more “run things in production” focused. I am yet to actually try this tool, but it does seem like a neat utility to get things up and running on top of a good old VPS provider or your own servers.
And, like most of the tools originating from 37Signals, Kamal is focused on solving the problem of 37Signals - which is much better than any tool born without any practical use case to start with. On the other side, Kamal presentations are not being totally honest with you. And I fully understand the irony of a Cloud Native and DevOps consultant telling you that “this very simple to use tool is not good enough”. Let the fact that we always try to convince smaller customers against use Kubernetes convince you that I am trustworthy here.
What Kamal presentations don’t explain to you enough is all the infrastructure that you need to take care of around the things Kamal responsible for. You still need to run a load balancer in front of your servers, and you still need to manage your database, with backups and monitoring. There are still logs to collect, patches to install and much more. Unless you want your servers to be exposed to the internet, you need to somehow let Kamal connect to your private network - assuming you do know how to set up a private network and a VPN solution on top. There doesn’t seem to be anything to help you with horizontal scaling of your instances.
There are many bits and pieces, required for a production application, that Kamal doesn’t handle. And it’s not like 37Signals doesn’t handle those things on their end. They do have an infra team, which most likely takes care of this with a configuration management tool like Chef or Puppet. In this context, Kamal is a great developer-oriented utility. Your infra team can focus on the main platform, and developers can just use Kamal, with many handy little developer-focused features it has.
But it’s not quite what we had with Heroku - the tool, that Kamal author gives as an example of what Kamal is like. Heroku was incredibly developer-friendly. But it also took care of all the layers around this developer-friendly interface, making your base infrastructure disappear, for a premium. Today, cloud providers attempt to make this base layer go away, with serverless offerings like AWS ECS Fargate or GCP Cloud Run, with their own developer-friendly CLIs, like AWS Copilot.
Naturally, the raw price per compute unit is much higher for those offerings. With Kamal, you can relatively quickly deploy things on any cheap VPS. With AWS, it won’t be cheap, but neither it will be horribly expensive for a small startup - especially if you use things like spot capacity and autoscaling. In return for this extra premium, you will get things that 37Signals SRE team is handling handled for you, without hiring your first SRE engineers yourself.
As a result, I am not convinced with Kamal, in it’s current form. It makes things easier, if you are a company with only developers, but it requires at least one developer to have a rough idea of what infrastructure should look like, to be scalable and secure. But not as easy as PaaS platforms and serverless offerings. And it does make things easier for bigger companies, by simplifying access for developers to the SRE-team managed infrastructure. But for this use case, we already have tools like Kuberentes, which might have a bigger entry barier, but you do get much more powerful abstractions in return.
What We've Shared
On our YouTube channel Pablo explains how to upgrade OpenShift, while Kirill's Dockerless course unit is about CRI, CNI and CSI this time.
'DevOps Accents', episode 22: In this episode of DevOps Accents, Pablo and Leo talk to Darko Fabijan, CTO and co-founder of Semaphore, a CI/CD tool for high-performance engineering teams.
And on the website our classic Terraform Lightning Course comes to an end with its fifth and sixth chapters:
Lesson 5: Refactor Terraform code with Modules
Lesson 6: Creating Kubernetes Clusters with Terraform: Learning Provisioners
What We've Discovered
k8s-network-policy-migrator: Now that VPC CNI natively supports NetworkPolicies, many of us can move away from Cilium and similar tools and simplify the networking stack.
Zero Configuration Service Mesh with On-Demand Cluster Discovery: Journey of Netflix from their first self-implemented service mesh setup (totally made sense in 2010!) to Envoy-based mesh. A rare case where a company can justify and argue about having a service mesh in general.
Slack’s Migration to a Cellular Architecture: It seems that at big scale, Envoy is more and more present - while additional extra layers like Istio can be seen much rarely. In Slack's case, once again, Envoy brought some good solutions to drain AZs quickly, regardless of the implementation of each particular microservice.
Traffic 101: Packets Mostly Flow. A high level overview of Slack's network infrastructure, on how they route traffic from edge locations to the primary region and how they deal with outages.
Scaling Kafka to Support PayPal’s Data Growth: Over a trillion messages per day is impressive - and requires not only a solid infra setup, but also lots of custom tooling, which this blog post describes as well in sufficient details.
A random reminder
Did you know that not only do we post YouTube Shorts, but also Instagram Reels? Follow @mkdev.me on Instagram and get DevOps Accents snippets directly into your feed. They have our faces and subs and everything!
The 31st mkdev dispatch will arrive on Friday, November 10th. See you next time!