Swizec Teller - a geek with a hatswizec.com

    You don't need a big team to automate your work

    Bah that would never work for us, we're not a faang

    The opposite of folks who follow every FAANG trend and over-engineer their companies to death, are the folks who never take advice from the big boys. The ones who read my recap of Software Engineering at Google and said "Yes of course Google can automate all that work, there's entire teams focusing on that! We gotta grind it out! Ain't nobody got time to make their lives easier!!"

    You can automate your work and speed up your team. That's what engineering is all about! Making tools that improve people's lives. Yours included.

    First you do the work. Then you notice the patterns. Then you automate them away.

    Known as the rule of three when coding, it works for tooling and process. When you find yourself doing a repetitive task for the 3rd time, see if you can make it easier.

    You go through life accumulating little snippets of code and keyboard shortcuts that make your machine feel just right. A vanilla computer feels like you lost an arm.

    That's called "developer experience" (DX) these days. Large companies do in fact have whole teams devoted to DX.

    But most don't. A good team shares the improvements everyone makes. When the company grows, some of you become the DX team because you enjoy making everyone faster.

    How to automate a painful process

    At work, we use a distributed monolith architecture. The result of a partial migration to micro services.

    We have a bunch of services, each in its own repository, all sharing the database as the communication layer. You get every downside of a monolith and every downside of micro services. It's great. ๐Ÿ™ƒ

    Production deploys hurt. And they're getting worse as the team grows. More frequent or more painful. Choose your poison.

    1. spot the problem

    Painful deploys are easy to spot. There's a whole book about it called The Phoenix Project.

    You'll see behaviors like:

    1. Folks avoiding deploys
    2. Nobody wants to deploy their own code
    3. Finished code sitting around for weeks
    4. Deploy meetings where everyone gets together and deploys
    5. Lots of back-and-forth chatter about what is and isn't ready to go out
    6. Production bugs from incorrect and partial deploys

    Deploys should be a non-event. Something that just happens every day. At least.

    We use GitOps with automated AWS CodePipeline deploys driven by git push. The problem is our plethora of microservices that all need to work together.

    Other problems may have other symptoms. Pay attention.

    2. see the pattern

    Going from develop to staging is full of the human element. You need to look at the diff, check the features, and ask every team if it's okay to deploy.

    Remember, you don't know what everyone else is doing.

    When you're ready, you run a series of commands:

    cd ../database
    git checkout develop
    git pull origin develop
    git checkout staging
    git pull origin staging
    git merge develop
    git push origin staging

    Get the latest develop database and the latest staging database. Merge. Push.

    cd ../microservice1
    git checkout develop
    git pull origin develop
    git checkout staging
    git pull origin staging
    git merge develop
    git push origin staging

    Do the same for the first microservice. Then the second. Then the third. Then you forget the fourth, do the fifth, and oh hey production bug!

    Every damn time my friend.

    3. make a script

    Look, you're mindlessly doing the same process for every deploy. You know who's good at that? Computers.

    You can start with a bash script:

    git checkout develop
    git pull origin develop
    git checkout staging
    git pull origin staging
    git merge develop
    git push origin staging

    Put that in every repository, call it deploy.sh. Now you can deploy with ./deploy. Saves 20 seconds per service, is always correct.

    4. make a better script

    Would be nice if you could run one script instead of going into every repository ๐Ÿค”

    You can do this with bash, but I've learned my lesson. Use a language you're familiar with when you need loops, conditions, functions, and user input.

    That series of commands can become a Node.js function.

    import prompts from "prompts"
    import chalk from "chalk"
    import { execSync } from "child_process"
    async function deployRepository(path) {
    const { yes } = await prompts({
    type: "confirm",
    name: "yes",
    message: `Ready to deploy ${path}?`,
    initial: true,
    if (yes) {
    const deployCommands = [
    "git checkout develop",
    "git pull origin develop",
    "git checkout staging",
    "git pull origin staging",
    "git merge develop",
    "git push origin staging",
    for (const command of deployCommands) {
    execSync(command, { cwd: path })
    console.log(chalk.green(`\n${path} deployed\n\n`))

    Prompts lets you ask for user input โ€“ย "Are you ready to deploy?" in this case. If they answer "yes", you deploy.

    Mechanically iterate through the list of commands, use execSync to run them in your desired directory โ€“ cwd โ€“ย then use chalk to print a green success message.

    Run deployRepository in a loop over every microservice and you have yourself a huge timesaver!

    node deployAll.js

    That's a 10min error-prone task whittled down to a quick flick of the wrist. ๐Ÿ˜

    Later you can make it run every day on its own.

    When's it worth automating

    As engineers we love to automate tedious work. But it's not always worth it! Use this helpful conversion chart from XKCD:

    XKCD: Is it worth the time?
    XKCD: Is it worth the time?

    And remember, automation can be a trap.


    Did you enjoy this article?

    Published on July 27th, 2021 in Technical, Learnings, Node.js

    Learned something new?
    Want to become an expert?

    Here's how it works ๐Ÿ‘‡

    Leave your email and I'll send you thoughtfully written emails every week about React, JavaScript, and your career. Lessons learned over 20 years in the industry working with companies ranging from tiny startups to Fortune5 behemoths.

    Join Swizec's Newsletter

    And get thoughtful letters ๐Ÿ’Œ on mindsets, tactics, and technical skills for your career. Real lessons from building production software. No bullshit.

    "Man, love your simple writing! Yours is the only newsletter I open and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. ๐Ÿ‘Œ"

    ~ Ashish Kumar

    Join over 10,000 engineers just like you already improving their careers with my letters, workshops, courses, and talks. โœŒ๏ธ

    Have a burning question that you think I can answer?ย I don't have all of the answers, but I have some! Hit me up on twitter or book a 30min ama for in-depth help.

    Ready to Stop copy pasting D3 examples and create data visualizations of your own? ย Learn how to build scalable dataviz components your whole team can understand with React for Data Visualization

    Curious about Serverless and the modern backend? Check out Serverless Handbook, modern backend for the frontend engineer.

    Ready to learn how it all fits together and build a modern webapp from scratch? Learn how to launch a webapp and make your first ๐Ÿ’ฐ on the side with ServerlessReact.Dev

    Want to brush up on your modern JavaScript syntax?ย Check out my interactive cheatsheet: es6cheatsheet.com

    By the way, just in case no one has told you it yet today: I love and appreciate you for who you areย โค๏ธ

    Created bySwizecwith โค๏ธ