Bah that would never work for us, we're not a faang
The opposite of folks who follow every FAANG trend and over-engineer their companies to death, are the folks who never take advice from the big boys. The ones who read my recap of Software Engineering at Google and said "Yes of course Google can automate all that work, there's entire teams focusing on that! We gotta grind it out! Ain't nobody got time to make their lives easier!!"
You can automate your work and speed up your team. That's what engineering is all about! Making tools that improve people's lives. Yours included.
First you do the work. Then you notice the patterns. Then you automate them away.
Known as the rule of three when coding, it works for tooling and process. When you find yourself doing a repetitive task for the 3rd time, see if you can make it easier.
You go through life accumulating little snippets of code and keyboard shortcuts that make your machine feel just right. A vanilla computer feels like you lost an arm.
That's called "developer experience" (DX) these days. Large companies do in fact have whole teams devoted to DX.
But most don't. A good team shares the improvements everyone makes. When the company grows, some of you become the DX team because you enjoy making everyone faster.
How to automate a painful process
At work, we use a distributed monolith architecture. The result of a partial migration to micro services.
We have a bunch of services, each in its own repository, all sharing the database as the communication layer. You get every downside of a monolith and every downside of micro services. It's great. 🙃
Production deploys hurt. And they're getting worse as the team grows. More frequent or more painful. Choose your poison.
1. spot the problem
Painful deploys are easy to spot. There's a whole book about it called The Phoenix Project.
You'll see behaviors like:
- Folks avoiding deploys
- Nobody wants to deploy their own code
- Finished code sitting around for weeks
- Deploy meetings where everyone gets together and deploys
- Lots of back-and-forth chatter about what is and isn't ready to go out
- Production bugs from incorrect and partial deploys
Deploys should be a non-event. Something that just happens every day. At least.
We use GitOps with automated AWS CodePipeline deploys driven by git push. The problem is our plethora of microservices that all need to work together.
Other problems may have other symptoms. Pay attention.
2. see the pattern
Going from develop to staging is full of the human element. You need to look at the diff, check the features, and ask every team if it's okay to deploy.
Remember, you don't know what everyone else is doing.
When you're ready, you run a series of commands:
cd ../database git checkout develop git pull origin develop git checkout staging git pull origin staging git merge develop git push origin staging
Get the latest develop database and the latest staging database. Merge. Push.
cd ../microservice1 git checkout develop git pull origin develop git checkout staging git pull origin staging git merge develop git push origin staging
Do the same for the first microservice. Then the second. Then the third. Then you forget the fourth, do the fifth, and oh hey production bug!
Every damn time my friend.
3. make a script
Look, you're mindlessly doing the same process for every deploy. You know who's good at that? Computers.
You can start with a bash script:
#!/bin/bash
git checkout develop
git pull origin develop
git checkout staging
git pull origin staging
git merge develop
git push origin staging
Put that in every repository, call it deploy.sh. Now you can deploy with ./deploy. Saves 20 seconds per service, is always correct.
4. make a better script
Would be nice if you could run one script instead of going into every repository 🤔
You can do this with bash, but I've learned my lesson. Use a language you're familiar with when you need loops, conditions, functions, and user input.
That series of commands can become a Node.js function.
import prompts from "prompts"
import chalk from "chalk"
import { execSync } from "child_process"
async function deployRepository(path) {
const { yes } = await prompts({
type: "confirm",
name: "yes",
message: `Ready to deploy ${path}?`,
initial: true,
})
if (yes) {
const deployCommands = [
"git checkout develop",
"git pull origin develop",
"git checkout staging",
"git pull origin staging",
"git merge develop",
"git push origin staging",
]
for (const command of deployCommands) {
execSync(command, { cwd: path })
}
console.log(chalk.green(`\n${path} deployed\n\n`))
}
}
Prompts lets you ask for user input – "Are you ready to deploy?" in this case. If they answer "yes", you deploy.
Mechanically iterate through the list of commands, use execSync to run them in your desired directory – cwd – then use chalk to print a green success message.
Run deployRepository in a loop over every microservice and you have yourself a huge timesaver!
node deployAll.js
That's a 10min error-prone task whittled down to a quick flick of the wrist. 😍
Later you can make it run every day on its own.
When's it worth automating
As engineers we love to automate tedious work. But it's not always worth it! Use this helpful conversion chart from XKCD:
And remember, automation can be a trap.
Cheers,
~Swizec
Continue reading about You don't need a big team to automate your work
Semantically similar articles hand-picked by GPT-4
- The dangers of spurious automation and how to automate anything
- Don't neglect your upgrades
- Make mistakes easy to fix
- What I learned from Software Engineering at Google
- What does "solve problems" even mean
Learned something new?
Read more Software Engineering Lessons from Production
I write articles with real insight into the career and skills of a modern software engineer. "Raw and honest from the heart!" as one reader described them. Fueled by lessons learned over 20 years of building production code for side-projects, small businesses, and hyper growth startups. Both successful and not.
Subscribe below 👇
Software Engineering Lessons from Production
Join Swizec's Newsletter and get insightful emails 💌 on mindsets, tactics, and technical skills for your career. Real lessons from building production software. No bullshit.
"Man, love your simple writing! Yours is the only newsletter I open and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. 👌"
Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.
Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.
Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help 👉 swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.
Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers 👉 ServerlessHandbook.dev
Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization
Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections
Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog
Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com
By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️