Swizec Teller - a geek with a hatswizec.com

    Better tooling won't fix your API

    RESTful APIs are like Agile โ€“ย everyone does it differently and if it isn't working, it's your fault for doing it wrong. ๐Ÿคจ

    Swizec Teller in ๐Ÿ‡ธ๐Ÿ‡ฎ avatarSwizec Teller in ๐Ÿ‡ธ๐Ÿ‡ฎ@Swizec
    REST is like Agile.

    Everyone does it differently and if it isnโ€™t working itโ€™s your fault for doing it wrong

    I've written before about How GraphQL blows REST out of the water and about REST API best practices in a GraphQL world. For serious production these days I use RESTful APIs with React Query. Because React Query gives you almost everything you thought you needed GraphQL for.

    And you know what, flip-flopping between REST and GraphQL, using React Query, manually fetching, holding Redux for state management, none of it solved my issues with APIs. They all suck.

    What gives ๐Ÿคจ

    Domain modeling

    The underlying problem is one of domain modeling. How do you translate a fuzzy business domain like accounting or medicine into a rigid system computers understand?

    Swizec Teller in ๐Ÿ‡ธ๐Ÿ‡ฎ avatarSwizec Teller in ๐Ÿ‡ธ๐Ÿ‡ฎ@Swizec
    How would you teach engineers about data modeling?

    hopefully an approach quicker than "Shoot yourself in the foot for 10 years across a dozen apps to build the scars"

    Domain modeling is the fundamental activity of software engineers. Everything else comes secondary.

    A good design makes your code easy. A bad design gives rise to kludge upon kludge and makes it impossible to write good code.

    The edge cases you keep finding are a symptom of a poorly modeled or misunderstood business domain. When you realize this it's often too late to change ๐Ÿ’ฉ.

    Domains are easy to mess up

    10 years ago I learned an important lesson from a friend. A smart engineer learns from others' mistakes ๐Ÿ˜‰

    Their startup built a blog recommendation engine for writers. Helps you link relevant articles to support your points. Wonderful.

    To build a SaaS around this tech, they added user accounts. A User model was born.

    They knew some users are writers and others are advertisers. Because ads make money. A Profile model was born.

    Every User can have a Profile. One profile.

    Until 2 years later a blogger says "Hey I want to buy ads" and an advertiser says "Hey content marketing is cool, can we use your blogging tools?"

    But a user can only be a blogger or an advertiser. ๐Ÿ’ฉ

    Refactoring to support multiple profiles per user took months.

    The original model of their domain wasn't wrong, just incomplete. They missed an obvious-in-retrospect requirement. Or the domain evolved.

    Taxonomies fail at the edges

    A white-brown bear
    A white-brown bear

    Domain modeling likes to fail at the fuzzy edges of taxonomies.

    Engineers and computers want things to be strictly classified. A bear is either white or brown. A URL is either a page or an article. A user is either a blogger or an advertiser.

    But reality is continuous.

    Those hard edges between classes don't exist and white bears can mate with brown to produce whitebrown bears. Is it a new species? We haven't decided yet ๐Ÿคทโ€โ™€๏ธ

    And what's the difference between a web page and an article? They both use the same display machinery to spit out a bunch of HTML.

    The k-means clustering algorithm offers an interesting insight into the problem of taxonomies.

    You give it a bunch of data and say "Find 3 classes". The algorithm groups every datapoint into 1 of 3 classes based on which mean value they resemble most. Yay.

    Ask the same algorithm to find 10 classes with the same data and you now have 10 perfectly strict classes plus a whole new way to fight whether something is this or that.

    Taxonomies always feel arbitrary at the boundaries between groups.

    Who owns what

    Once you have the objects of your domain (like users and pages), it's time to define their relationships. It gets tricky there too.

    In many cases, the relationship is obvious. Users own posts and posts belong to users.

    That's because a user can make many posts but a post can only be made by 1 user ... unless there's editors and collaboration ๐Ÿค”

    Your troubles don't end when you resolve all this and get it saved in the database. How do you design an API for fetching?

    Swizec Teller in ๐Ÿ‡ธ๐Ÿ‡ฎ avatarSwizec Teller in ๐Ÿ‡ธ๐Ÿ‡ฎ@Swizec
    How often does Separation of Concerns just make it harder to change things that are fundamentally intertwined?

    I ran into this when modeling appointments and staff recently. How do you fetch all appointments for a staff member?

    /staff/{uuid}/appointments sounds nice and implies that the staff portion of your codebase needs to know about appointments. That doesn't feel very Separation of Concerns.

    /appointments?staff={uuid} sounds like a filter on the list of appointments. Which feels more Separation of Concerns, but now the staff portion of my codebase is reaching into appointments in a way that feels unobvious.

    Either way is going to feel wrong to someone.

    The RESTful solution

    REST as per Roy Fielding's original thesis gives up on all this and offers a basic solution. You can read about designing clean RESTful APIs in Serverless Handbook chapter 9

    You focus on the objects. Each gets an endpoint that uniquely identifies that object.

    Different HTTP verbs manipulate those objects:

    • GET reads an object from /pages/123
    • POST creates a new object at /pages
    • PATCH updates an object at /pages/123
    • DELETE deletes an object at /pages/123

    If you want objects and their related objects, REST says haha you're on your own kid. Make a new request for each.

    Hope you have good wifi โœŒ๏ธ

    Swizec Teller in ๐Ÿ‡ธ๐Ÿ‡ฎ avatarSwizec Teller in ๐Ÿ‡ธ๐Ÿ‡ฎ@Swizec
    @nloadholtes Funnily in my experience bandwidth in airplanes is usually great, but latency is high and reliability is low.

    This *wrecks* most webapps

    The GraphQL solution

    GraphQL focuses on the technical challenges with REST ... and ignores the fundamental problem of domain modeling. ๐Ÿ’ฉ

    You can read about designing and building GraphQL APIs in Serverless Handbook Chapter 10.

    Basic GraphQL focuses on objects and gives you the tools to query, slice, and dice the data. Collapses related RESTful requests into 1 and improves performance.

    Instead of worrying about /staff/{uuid}/appointments vs /appointments?staff={uuid}, you'd write a query like this:

    query {
    staff(uuid: ...) {
    appointment {
    ...
    }
    }
    }

    "Get me a staff and all their appointments"

    GraphQL then figures out how to combine fetchers for staff and for appointment to make this work. All the complexity of good data modeling washed away by flexibility ๐Ÿ˜

    Except the domain is as complicated as ever.

    Imagine writing a query that touches 10 tables every time you want to see if a staff member can serve a specific type of appointment. That would suck.

    Sure would be nice if the backend took care of that complexity and exposed a nice clean top-level query ...

    The solution you see in reality

    Most APIs I've seen are a mix of clean REST and flexible GraphQL.

    You get a gaggle of baseline endpoints for core operations and a bunch of endpoints that grew out of a specific client need.

    Rarely do teams treat their API as a product of its own or a tool meant for others. They just want this data to get to that UI.

    If your client needs different data, a new endpoint is born. If you need related data, it's added. If you need a new filter, you got it. Soon, anything could fetch everything and nobody knows what's where.

    New endpoints are born because we're afraid to touch the old ones. Any change could break in unexpected ways. Like aggressive type checking blowing up an iOS client when there's a new property. Oops.

    You end up with an API that most closely resembles Remote Procedure Calls. You're making HTTP requests and thinking about it like calling a function.

    It's a people problem

    Learning an API is like learning how another person or organization thinks. How do they see the world?

    Even if it makes no sense to you, as long as it makes sense to them ยฏ\_(ใƒ„)\_/ยฏ

    Cheers,
    ~Swizec

    PS: it rarely makes sense to them either

    Did you enjoy this article?

    Published on November 30th, 2021 in REST, GraphQL, Fullstack, Backend,

    Want to dive into serverless? Not sure where to begin?

    Serverless Handbook was designed for people like you getting into backend programming.

    360 pages, 19 chapters, 6 full projects, hand-drawn diagrams, beautiful chapter art, best-looking cover in tech. โœŒ๏ธ

    Serverless Handbook on your bookshelf
    Serverless Handbook on your bookshelf

    Learn how to choose the right database, write cloud functions, think about scalability, gain the architecture mindsets for robust systems, and more.

    Leave your email to start with a free chapter and email crash course ๐Ÿ‘‡

    Serverless Handbook for Frontend Engineers โ€“ย free chapter

    Dive modern backend. Understand any backend.

    Serverless Handbook taught me high-leveled topics. I don't like recipe courses and these chapters helped me to feel like I'm not a total noob anymore.

    The hand-drawn diagrams and high-leveled descriptions gave me the feeling that I don't have any critical "knowledge gaps" anymore.

    ~ Marek C, engineer

    Start with a free chapter and email crash course โค๏ธ

    Join 15,883+ engineers learning lessons from my "raw and honest from the heart" emails.

    โญ๏ธโญ๏ธโญ๏ธโญ๏ธโœจ
    4.5 stars average rating

    Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.

    Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.

    Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help ๐Ÿ‘‰ swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.

    Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers ๐Ÿ‘‰ ServerlessHandbook.dev

    Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization

    Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections

    Want to brush up on modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com

    Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog

    Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com

    By the way, just in case no one has told you it yet today: I love and appreciate you for who you are โค๏ธ

    Created by Swizec with โค๏ธ