Let me start by proving that this crazy contraption works 👇
LOLCODE goes in 🐱
Taken from Chrome DevTools source maps. That's after Webpack and Babel do their thing. Intermediate output from
Ok so how does this work?
You can see the full source code on GitHub.
You can also watch me build lolcode-babel-macro from scratch in a series of livecode videos. 👇
LOLCODE is an esoteric programming language from 2007. The peak of the lolcat meme when the internet was for cats.
You might remember i can haz cheezburger?
Yeah that was 12 years ago my friends. We're getting old. Some of you might not even remember. A coworker recently said he's "happy that the internet has moved on from such silly nonsense".
Kids these days have TikTok and Snapchat filters so has it really? 🤔
Anyway, Adam Lindsay asked the important question: "What if you could write code like cats speak?". LOLCODE was the answer.
One such interpreter is loljs by Mark Watkinson. I used it as the basis for my compiler.
Aside: an [interpreter](https://en.wikipedia.org/wiki/Interpreter(computing)) executes your code as it's read, a compiler translates it to a different language (or machine code) to be executed later. Important distinction_
The most common type of Babel macro are prefixed ES6 strings. You may have seen them as GraphQL queries or CSS-in-JS. JSX is a Babel plugin. Similar to a macro but different mechanics.
That is a Babel macro.
At compile-time Babel looks for the
Zero run-time overhead 🤘
Macros are extremely powerful and many programming communities have decided they're too powerful. More trouble than they're worth.
You follow a 3 step process:
- Create a
macro.jsfile (naming convention matters)
- Get the wrapper function
- Write your function
- Default export your function wrapped in
Somehow this creates a named export. I'm not sure why or how, but that's how it is. Best not to poke.
You also cannot export anything alongside your macro. Which is a shame and caused me many grief. You can work around it by letting people import other files in your library.
The macro function itself can get tricky. You're dealing with Abstract Syntax Trees (AST). It's up to you to modify as you see fit.
In my case that was finding all
lolcode nodes and replacing them with compiled code.
babel are the same as they are in a normal Babel plugin. Which isn't helpful, if you've never built a Babel plugin 😅
My macro ends up replacing each lolcode's parent node with the compiled string. That worked well.
Most compilers are split into 3 parts:
- The front-end, which uses a Lexer and a Parser to turn your code into an AST
- The middle-end, which performs optimizations and other transformations on your AST
- The back-end, which turns your AST into the final output code
In theory you can swap these parts around.
Best explained with a LOLCODE example 👇
The swappability of compiler parts is what helped me build lolcode-babel-macro.
The first step in compiling code is a tokenizer. Tokenizers take plain strings or files and turn them into lists of tokens. Usually a combination of split-by-space and regex.
The LOLCODE tokenizer turns strings like
O RLY? into
I HAS A into
BIGGR THAN into
BIN_OP etc. You can see the full list here, lines 6 to 86.
You don't want to write your own parser from scratch, trust me.
The full grammar definition for LOLCODE lives here 👉 loljs.jison.
It's a series of lexical definitions 👇
A function call has an
IDENTIFIER, followed by an
arg_list node, and an
arg_list in turn are built out of more tokens and nodes. The rabbit hole goes very deep and I'm happy that somebody else wrote all that for me :)
I remember doing this in my compilers class in college and it was not fun. Easy to make mistakes, hard to verify.
Jison takes your LOLCODE grammar and turns it into a parser.
Here's what the parser code looks like
A convoluted series of hundreds of
new ast.X calls to create an abstract syntax tree based on your grammar and your AST definition.
Best stick to defining the grammar and let parser generators do the rest.
Again thanks to Mark Watkinson, I didn't have to write my own 👉 the LOLCODE AST
Defining your AST can be tedious, but it's not very complex. A function call node looks like this
To generate a
FunctionCall node, you need a
name, and some
args. All coming from your parser.
You return a
FunctionCall, define its
name (identifier), and
args node. All very recursive and following the visitor pattern
The resulting AST is an object a little like this (writing from memory)
Like this 👇
parentPath.node.quasi.quasis in this case. We walk through all
quasis, get their
raw values, and join them into a string.
That's how prefixed ES6 strings work, don't know why.
Take the resulting
parse it, instantiate the
JSify compiler backend, and return the output as a string. We wrap compiled code in a closure with the LOLCODE stdlib, which defines some basic functions.
All your LOLCODE instances share the same stdlib. Assumed to exist in scope via an import.
You can see the full JSify file here.
Keeping with our FunctionCall example from earlier ...
FunctionCall method gets a
node and a flag whether to instantiate a new
IT context. This keeps the same function signature throughout JSify.
IT in LOLCODE is the implicit variable, by the way. Supposed to hold the value of the last expression ... but I had to take some liberties because this is a compiler not an interpreter. You have to explicitly assign values to
IT, but the variable is always there for you.
FunctionCall then checks if this function is in
stdlib and returns the appropriate code. Either a stdlib call or a normal function call.
this.compile for the arguments node.
ArgList node is similar 👉 walk through list of values and recursively call
this.compile for each node. Who knows what expression might lie in there :)
this.compile itself is pretty simple, a switch statement 👇
Checks if it recognizes the node type and calls the appropriate method. If not, it throws an error.
HOWEVER, this opens the door for future hacking 👇
- Great excuse to learn about Babel macros
- Superb way to practice building a small compiler
That last part 😏
Here's how it works 👇
Then get thoughtful letters 💌 on mindsets, tactics, and technical skills for your career.
"Man, love your simple writing! Yours is the only email I open from marketers and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. 👌"
Ready to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz components your whole team can understand with React for Data Visualization
Curious about Serverless and the modern backend? Check out Serverless Handbook, modern backend for the frontend engineer.
Ready to learn how it all fits together and build a modern webapp from scratch? Learn how to launch a webapp and make your first 💰 on the side with ServerlessReact.Dev
By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️