How GPT-6 Astra Killed My Ruby on Rails App
As long as mkdev.me website existed, it was always a Ruby on Rails app, initially Rails 4, and then being upgraded and adjusted over the years to become a Rails 8 app, that uses most of the things the framework offers - ActiveJob, ActiveStorage, Hotwire + Turbo, among other things.
The app used to be a platform for students, till we eventually sunset that line of business. Since then, mkdev.me became a purely content website, with hundreds of articles, videos, courses and books. I knew since a long time, that there is no big reason for mkdev.me to stay a Rails app - it does nothing that requires a proper majestic web framework and can fit either within any of the numerous CMS systems or, even better, be converted to a static website with the content stored in a Git repository.
The problem with such projects is that they take time and give little in return for this time. What happens is that you end up with the same experience, same website, only powered by a different "something". The whole process is about keeping things as-is for the end user - making sure URLs work, images render and design is the same.
So we never prioritized this move. Instead, we rolled out one quick fix after another - adding Comfortable Media Surfer as a flexible CMS engine, running in parallel to our self-built Admin UI; expanding caching in a way that most most pages are returned directly from CDN without hitting our servers (ok, AWS Lambdas, not truly servers); deprecating features from the old platform one after another; updating dependencies, database versions, all of this things.
Taken separately, each such quick fix or maintenance task doesn't take much time and can be fit between more important activities, like making our customers happy. Of course, if we sum up the time spent on this, we will end up with enough hours to do the bigger move - only that there is never an interrupted stretch of time to do a bigger move in this case. So we never did this, and it remained one of those "very nice to have" projects.
In The Agentic Engineering Myth - 1 year of coding with AI, I wrote about how any blog post about using AI for coding in the most optimal way is going to be outdated in a couple of days since it's published. While preparing that article for publication, Astra got released, offering, on paper, a huge jump in intelligence. So one of the most natural things I tried to do it is to tell it to migrate mkdev.me from Ruby on Rails to Hugo with Sveltia.
I took an existing epic (in Beads) for doing this migration end to end, opened Codex, and, using our Beads Coordinator Loop skill, asked it to work on that Epic till completion.
We are going to publish that skill pretty soon, but in the essence it's our spin on Matt Shumer's Manager Loop, only focused on using Beads epics. In essence, it does this:
- You start new chat in Codex and use this skill like Use $BeadsCoordinatorLoop skill to work on $BeadsEpicID
- That chat becomes coordinator;
- Coordinator picks next ready task from the epic, spins up new chat (not a sub-agent), with the model it thinks fits the task fell, and makes sure the task is done till competition;
Coordinator is always GPT-6 Astra. I default to Max level of reasoning, but it could be it works as-fine with lower levels. Implementation chats can use Astra with varying levels of reasoning or they can also just use GPT 5.6, either Sol or Terra. I found Astra preferring using Astra on medium reasoning level more often that it uses other model family.
Now let's break down what went good, what went bad and what went ugly.
The good
The main outcome of this experiment is that the migration completed successfully.
Astra managed to deploy a couple of temporal AWS resources to be able to pull the latest PostgreSQL dump from RDS - and then cleaned up this resources. It correctly re-created every single public-facing page and blog post as Hugo files, including all the images. We are talking about maybe 700 articles plus all the other public pages.
It could dissect the Rails code in a way that it found all the required CSS and JS and, after preparing that local data with all the files, it made sure that locally running Hugo dev server looks and behaves like exact copy of the Rails app.
Astra prepared new CloudFront distribution with S3 bucket as an origin, it did it via an existing in-repo Terraform setup and it did multiple canary deployments of the new setup, ramping up the traffic to the static website up till 50% and then scaling it back to 0 before proceeding to the next steps.
It also setup the GitHub Actions deployment, and even discovered our separate repository where we configure the creation and permissions of IAM Roles that are trusting GitHub Actions via OIDC. Not only it did the change in that repo, it did it via a separate PR, and it applied the changes (in this case, I give the agent a lot of freedom so that it can show me how far it can go without me).
By the end of the first coordinator run, I had more or less the result I wanted.
The bad
Unfortunately, Astra is not entirely immune to the main problem that I also have with GPT-5.6. From my experience, when Fable or even Opus receive a big and relatively vague task, it tends to just ask me a couple of questions to make sure the implementation goes into the right direction. I don't now if it's the way Claude Code is implemented, or if it's the way the model behaves natively. What I know, is that asking 3-5 questions before jumping to implement anything always result in a way better outcome.
GPT-5.6 Sol prefers not to ask me anything in most cases. I think this is also the reason some people really prefer Codex to Claude Code, because it feels more like "hey agent do the thing - and it just goes and does the thing" experience, while Claude often feels like an extension of your brain, with you still being at the top of decision making. I already wrote in the previous article how this not asking questions results in the model engineering some monstrosities, that require large scale refactoring later on.
From my experience so far (albeit, quite limited still), Astra is still prone to making the same mistake. I was actually quite impressed, how in one other codebase Astra, when asked to deploy the application from my laptop, discovered that my existing local agentic release loop is kind of stable and predictable enough to be converted to a regular CI deployment. I never saw it in Codex before, so Astra asking me any questions at all felt refreshing and surprising in a good way,
In case of killing the Rails app and making it a static website, Astra decided not to ask me anything. It could be because it was working from an existing Beads Epic I handed over to it. Astra created a huge amount of test files with lots of Python things to do triple and quadruple verification of everything. It architected quite a complex routing system, powered by CloudFront Functions with KeyValue Store, and then it hammered it with verification requests with a test harness so sophisticated that each test run would cost $40 in AWS costs. It was trying to do a very smart switch over from SQS-based background jobs to.. nothing, essentially, as there are no background jobs.
It was, to sum it up, an extremely, wonderfully production-ready, proper and scalable and expensive way to do a zero-downtime smooth switch over from Ruby on Rails app to a static website. It maintained a separate .csv file with one row per every page, and it kept updating it with the availability status of each page, many times, on every release.
The ugly
I don't blame Astra for making this migration process a tiny bit more sophisticated than it should have been. It felt like it took the same "character" of GPT models of just doing things, and it 2x, 3x the complexity of how it exposes this character - with more computer use, with more intelligence, with more orchestration of many scripts, workflows, verifications. GPT-5.6 Sol used to just vibe the implementation, and then implementation would be a billion lines of new code and nothing would work. With GPT-6 Astra, everything worked as I expected, but the route there was once again full of unnecessary complexity.
Part of this can be easily blamed on me. I could have given more higher level context about what is the acceptable downtime, what is the criticality of this system and what really matters during this migration. As I wrote before, I like testing new models by being extremely vague, because I want to be able to just say AI to do something, without me caring about how it does it - but with it still doing it right.
The ugly part was that despite all of this impressive migration machinery, Astra completely missed some of the more obvious things in this migration. For example, it just threw away our Posthog integration. None of our website analytics was migrated at all, and I had to add it in a follow up task.
Back to the Good
Still, Astra is, of course, undeniably good, very good. It behaves differently from Fable, so I can't really tell you if it's overall better or not. It is clearly much faster than Fable, and it is also much cheaper. While it does burn tokens like nothing Codex has had before, it still burns them slowly enough for me to default to Astra for all the tasks I throw at it, maybe with a different reasoning level, but never falling back to 5.6 Sol.
The way Astra is using - and is keen on using - computer is nothing but impressive (and it was probably post-trained with a huge focus on computer use). One of the "let me just throw it at AI" tasks I did with Astra was creating a new Obsidian Plugin for syncing a subset of my notes to Basecamp's Docs & Files.
Astra created a plugin just fine from the first attempt, of course - but I am sure that 5.6 Sol and Opus 5 would do it as well - but it also tested it thoroughly by creating a new Vault on my laptop, and then clicking on its own in Obsidian UI to configure this plugin and run some tests against some test project Basecamp. It also went to Obsidian Community website, asked me to sign up, and then proceeded to register the plugin in the marketplace, made and uploaded a screenshot, filled in the description and made the repository public. It also setup an mkdev-hosted oAuth callback endpoint, because Basecamp can't just give you a "Sign in with Basecamp" or an API token, it always needs some kind of callback URL to go to.
Astra is also careful, in some occasions annoyingly careful, like when Fable was first released and it constantly refused to do things or fell back to Opus.
AGI?
We might be at the point where "AGI" term is devaluated the same way Apple devalued "Pro" term by adding it to any product that is more expensive than the baseline version. I remember how couple of years ago everyone was really concerned about o3 model getting to an AGI level and that AGI looks nothing like compared to this AGI. Curiosly enough, OpenAI seems to be using AGI in it's marketing way more often than Antrophic. Maybe we should not focus on this abbreviation too much till one of the models actually takes over the planet in a Skynet style
Is Astra smarter and more capable than most humans? Probably yes, though it's still being directed by humans. Clearly the same Astra that I used to migrate mkdev.me to Hugo can also edit videos, manage marketing channels, solve math problems and do thousands of other things that I personally can't even imagine doing. But I could tell the same about Fable, and before that about any previous frontier model release.
From what we see today, Astra is more or less on the same level as Fable 5.1, but it's much faster and it's much cheaper. Having something as capable as Fable, but way more affordable is an impressive achievement and something for humanity to be truly impressed with - till we get to Fable 6, till we get to GPT-7, till we get to... AGI?