Beware the spaceships
I'm in the middle of migrating our design system and it's fun to see how much AI can do. Just last night I stayed up until 1am to ship another big portion.
Wait why am I coding at 1am if AI's doing all the work? 🤨
It's the spaceship problem. You start with the ending then work your way back to something useful. This kinda sucks.
Spaceship first
Here's what happened.
After the success of my 22,000 line Bootstrap -> Tailwind rewrite last week, I got cocky. If we can merge that monster and very little breaks, what else can we do?
The next thing I tried was a 12,000 line lint+fix run on all our HTML and JavaScript. That was annoying but it worked, we now have a clean linter pass for basically the first time in my career.
Then I got super cocky. Can an LLM transfer a whole section of our app to the new design system? Bootstrap -> Talwind is just styles, but rewriting functional code to a new paradigm?
It worked!
Cleanup second
Tokens go brrr and a few hours later we had a stack of 11 pull requests going page by page. Yes you can make your AI write reviewable stacked pull requests and page-by-page is a good strategy for rewrites.
That's the new github stacked PRs feature btw. I like it a lot.
The rewrite looked good enough to merge, which meant we now had to go through all this code and UI and clean up the details. Crap.
Generating the code took 2 hours. Cleaning it up took 2 engineers 2 days. One to feedback with a good eye for design, another to fix the implementation.
It's always the details
The problem with spaceships is that everything looks great until you pay attention. The little details are off.
Claude loved to look at a ShadCN or BaseUI component as loose inspiration to implement its own. We found a bunch of jank where the bot got confused between The World As It Is and The World As It Ought To Be.
Compare these toasts:
vs the toasts we wanted:
Our toasts come default with the library. The bot's approach was a custom high-effort rebuild of the old design system using the new library 🤦♂️
This was a common issue. LLMs are fantastic at mechanical translation but they struggle with conceptual leaps. I think they're heavily post-trained to ground every approach in existing code otherwise they'd cause too much churn.
Are spaceships the right approach?
That depends. Are you an editor or a creator?
Sometimes it's nice to take an almost-Michelangelo and remove everything that's not a Michelangelo. Other times you don't even know where to begin giving feedback.
But it sure is a fast way to get something working, kick the can, and gather feedback.
Just don't be afraid to hit delete if you got the wrong spaceship.
Cheers,
~Swizec
Filed under: AISoftware EngineeringFrontend
