Few of your engineering decisions matter long-term. Software is soft. You can change your mind. But how you structure your components is here to stay.
A 2007 study by MacCormack et al looked at how a commercial piece of software evolved over 15 years. Primarily they wanted to see how architecture impacts evolution. Do past decisions get you stuck?
You can read the paper with my annotations, here. I enjoyed it. Super insightful. I'm not at all biased because it confirms my own observations over the years.
PS: you can read and share this online
Measuring modularity
The authors use a Design Structure Matrix to measure modularity. Files in rows and columns, put a 1 where they depend on each other.
You can multiply this matrix with itself to find which files depend on each other after multiple steps. Lets you find those A->B->C->D dependencies and realize that A depends on D.
They call this "visibility". Because a file can "see" changes in other files. You can compute a file's visibility by summing its row (fan-out – whose changes A can see) or its column (fan-in – who can see changes in A).
Measuring evolution
The authors use version control to go through 15 years of a software's history and look at how components evolve. They looked at major versions only and ignored intermediate releases.
Every new version builds on the previous state and adds new code. This code can be clean new or integrate with previous code in two ways.
The aspects you care about are
- churn – code changes that bring no new functionality or bug fixes
- component survival – old code that sticks around
- augmentation – is new code more modular
The results
As you might expect, the science has proven something obvious is true. But it's cool that you can measure and verify this folk wisdom.
- The software grew 17x in 15 years.
- Initial code was a mess.
- It became more and more modular over time.
- Tightly coupled components are nearly impossible to kill.
- Tight coupling leads to lots of churn.
- Loosely coupled components eventually become stable (stop changing).
- Tightly couple components never stabilize.
That means the core of your system is defined early in its life when there's no time to think about architecture and those decisions reduce your future flexibility. Each new version of the code adds fewer and fewer new components as you slowly become bogged down under the weight of your past decisions and build on top of this immovable core.
You can beautifully see this evolution from a ball of mud to a modular system with an untouchable core when you put DSMs of each version side by side.
But that's a "future me" problem so managers systematically under-invest in modularity and want everything now now now ✌️
~Swizec
PS: MacCormack did a followup study 6 years later that looks into how organization design impacts code structure (conway's law). Looks promising
Continue reading about Empirical evidence for code modularity
Semantically similar articles hand-picked by GPT-4
- Why taming architectural complexity is paramount
- Two types of complexity and their impact
- You can't fix the wrong abstraction
- Finding modules in a big ball of mud
- Why even care about code structure
Learned something new?
Read more Software Engineering Lessons from Production
I write articles with real insight into the career and skills of a modern software engineer. "Raw and honest from the heart!" as one reader described them. Fueled by lessons learned over 20 years of building production code for side-projects, small businesses, and hyper growth startups. Both successful and not.
Subscribe below 👇
Software Engineering Lessons from Production
Join Swizec's Newsletter and get insightful emails 💌 on mindsets, tactics, and technical skills for your career. Real lessons from building production software. No bullshit.
"Man, love your simple writing! Yours is the only newsletter I open and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. 👌"
Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.
Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.
Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help 👉 swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.
Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers 👉 ServerlessHandbook.dev
Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization
Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections
Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog
Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com
By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️