Large programs are never complete. They just continue to evolve forever because adding new functionality to existing systems feels easier than starting over.
One of the first to notice this was Manny Lehman who in 1980 published a prescient paper Programs, Life Cycles, and Laws of Software Evolution. He classifies programs into 3 types, describes the ideal development life cycle (he's wrong), and sets down 5 laws of software evolution that ring true even today. He later expanded these laws.
And he was one of the first to say "To achieve this requires insight, theory, models, methodologies, techniques, tools: a discipline. That is what software engineering is all about."
Favorite bit so far: Lehman describes, in 1980, the difference between programming and software engineering.
— Swizec Teller (@Swizec) September 26, 2024
You'll remember this is a key point in that Software Engineering at Google book from 2020 pic.twitter.com/SFrFBVszMk
Thus the software engineer was born as distinct from programmer. A programmer can write code, an engineer can systematically develop software that evolves over time.
3 types of programs
Omg I figured it out!
— Swizec Teller (@Swizec) September 28, 2024
How do you identify what should be a horizontal library/util/atom vs a vertical module?
It’s the S- vs E- programs from Lehman! 💡
Libs/utils do a thing exactly to a spec that doesn’t change and their input->output map can be fully specced.
Modules…
Lehman classifies programs into 3 types that continue to feel useful today. Reading it laid out like this clarified a lot of vibey thoughts for me.
S-programs map inputs to outputs according to a specification and can be exhaustively verified. S-programs are contained by the spec and don't care about the environment. Their behavior doesn't change. If you want to follow a different spec, you write a different program. These are your utils, atoms, etc.
P-programs are defined by the business rules they follow and aim to fulfill a real world need. You can describe their behavior, but you can't evaluate their correctness without comparing to external criteria. Does it get the job done? A bug-free p-program can still be wrong. These are your molecules, components, modules, etc.
E-programs combine any number of P- and S- programs into a cohesive whole that aims to encode and mechanize a human endeavor or system. These are your software. Constantly changing and adapting as the world changes, needs evolve, and the software itself creates new ideas and possibilities.
Development life-cycle
Lehman published this paper 21 years before agile. As such he is weirdly obsessed with waterfall and big up-front design.
He describes the ideal cycle as:
- Gather requirements
- Design top level specifications
- Recursively go into components
- Design specs for those
- Factor your whole system into nothing but S-programs
- Now programmers can "just implement" the specs
- Integrate
- Deploy
- Maintenance phase begins
This sounds great but as you and I know, steps 5 and 6 never quite happen. By the time you get that deep into writing specs, you may just do the work and write your specification as code.
You could argue that writing code in a high level programming language like JavaScript, Python, Haskell etc is like writing a specification detailed enough that your compiler can "just implement" the code. But they had high level languages in the 80's 🤷
Interestingly, Lehman notes several times that "this is not how it works in practice and the phases are often interleaved and overlapped". We now know that's the only way.
5 laws of software evolution
Lehman notes 5 laws of software evolution. They feel true to my experience. These might not solve any of your problems, but it's nice to know your experience is normal.
- Continuing change means software is never done. As long as you're using it, you're gonna have to make changes to adopt new requirements, changes in environment, or implement new ideas that come from feedback cycles with the software itself. S- programs don't change, P- programs change rarely (after they're "done"), and E-programs change always.
- Increasing complexity means your code keeps getting more complex unless you do something about it. Lehman notes a typical cycle of new feature release followed by cleanup releases. You have to manage complexity.
- The fundamental law means that even though we all make decisions based on local knowledge, the system as a whole behaves statistically and can be measured/understood.
- Conservation of stability means that the rate of output on a software project is constant. Due to growing complexity and maintenance, you have to keep adding people just to keep moving at the same pace.
- Conservation of familiarity means that as software evolves, it is increasingly constrained by past choices. The more users you have, the more disruptive large changes become.
Hope this concrete ways to talk about your gut feels and vibey thoughts. It did for me.
~Swizec
Continue reading about The Laws of Software Evolution
Semantically similar articles hand-picked by GPT-4
- What I learned from Software Engineering at Google
- 25 lessons from 25 years of coding
- You can't stop the business, or why rewrites fail
- My favorite lessons from Pragmatic Programmer
- Programmers are fucking lazy
Learned something new?
Read more Software Engineering Lessons from Production
I write articles with real insight into the career and skills of a modern software engineer. "Raw and honest from the heart!" as one reader described them. Fueled by lessons learned over 20 years of building production code for side-projects, small businesses, and hyper growth startups. Both successful and not.
Subscribe below 👇
Software Engineering Lessons from Production
Join Swizec's Newsletter and get insightful emails 💌 on mindsets, tactics, and technical skills for your career. Real lessons from building production software. No bullshit.
"Man, love your simple writing! Yours is the only newsletter I open and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. 👌"
Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.
Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.
Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help 👉 swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.
Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers 👉 ServerlessHandbook.dev
Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization
Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections
Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog
Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com
By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️