Yesterday's email had a broken link to the paper. Here's the correct link:
What Goes Around Comes Around... And Around... 👈 this PDF has my annotations. The paper is super approachable and you'll learn a lot about different database technologies. Great for any system design interview or conversation.
And if you're in a sharing mood, here's a link to the web version of yesterday's article. Forgot to include it. 👉 https://swizec.com/blog/why-sql-is-forever/
Thanks to everyone who wrote in to let me know I'm bad at copypaste ❤️ Keep reading for some questions from readers.
Is SQL just a better fit for our brains?
Jacek writes in suggesting "maybe SQL is just how our brains think about data"
Hard to say about brains. They’re all so different. I think SQL’s strenght is its permissiveness.
You can really butcher the query and still get a result. And the query optimizer will sigh, roll its eyes, then make it work fine enough 90% of the time. I have for example written awful queries with subqueries of subqueries (couldn’t figure out how to group) on tables with millions of rows … and it still ran in 3 seconds because B-tree indexes are that good.
Guess Jacek is right: SQL fits our brains because the query optimizer can make almost anything work.
Relational good, SQL bad?
Damien writes in arguing that yesterday's article made a great case for relational databases, but not for SQL itself. He suggests that SQL is full of footguns (true) and that you could have relational databases without SQL.
Easy to get A result
I agree with Damien that it's very easy to write SQL that produces the wrong result, but looks correct and doesn't complain. SQL will figure out how to return a result. And that's a threat!
You, the engineer / data scientist, need to have a deep understanding of the underlying data to make sure you're getting sensible results. Super easy to skip a "zero month" in your data and not notice.
OLAP databases help. The data engineer slurping up production data makes sure to fill in the blanks and do all the right joins before squashing data into columnar tables for data scientists to consume.
The syntax is wonky
A forever debate in SQL is whether you're "selecting socks from a drawer" or "from a drawer selecting socks". Damien is in the 2nd camp, I ... eh.
Yes if we started with from, the tooling could help us autocomplete columns and be more useful. But SQL's original intent was to "sound english" and be easy for non-engineers to write.
And it's criminal that you can't use an alias inside the HAVING clause. Life would be a lot easier if you could.
Lack of linting support
This is a big one. SQL is too flexible for its own good. It can check for basic type mismatches (comparing string to integer), but it doesn't understand your data as much as it could.
A user_id is an integer. An article_id is an integer. SQL will let you compare those and not even blink. This leads to queries that work but produce the wrong data. And you won't even notice.
But SQL is here to stay
Despite all these obvious flaws, databases that start without SQL keep evolving their own versions of SQL. ¯\(ツ)/¯
Cheers,
~Swizec
Continue reading about Why SQL is Forever followup
Semantically similar articles hand-picked by GPT-4
- Why SQL is Forever
- Notes on A Relational Model of Data for Large Shared Data Banks
- That time serverless melted my credit card
- Why mentoring matters
- You can't stop the business, or why rewrites fail
Learned something new?
Read more Software Engineering Lessons from Production
I write articles with real insight into the career and skills of a modern software engineer. "Raw and honest from the heart!" as one reader described them. Fueled by lessons learned over 20 years of building production code for side-projects, small businesses, and hyper growth startups. Both successful and not.
Subscribe below 👇
Software Engineering Lessons from Production
Join Swizec's Newsletter and get insightful emails 💌 on mindsets, tactics, and technical skills for your career. Real lessons from building production software. No bullshit.
"Man, love your simple writing! Yours is the only newsletter I open and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. 👌"
Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.
Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.
Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help 👉 swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.
Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers 👉 ServerlessHandbook.dev
Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization
Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections
Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog
Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com
By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️