What if you could focus on the fun engineering part of your job, not on coding? Here's what I mean 👉
You break down problems, design solutions, work with product, think about strategy and how software evolves over time, pick the right APIs, design interfaces, work with vendors, think about resilience, and research unknown solutions to fun problems.
<hand wave> turns your ideas into working code.
That's what it feels like to be a tech lead. I'm deeply involved at the design stage, then lightly involved in implementation. Mostly to review code.
Yes that means I'm slowly getting worse at coding and that's okay. Takes a hot minute to swap those pages back into working memory sometimes 😅
So I decided to try an experiment: Can you do this without a team?
Yes! Programming in Markdown!
Results here 👉 https://chat.openai.com/share/a942b81c-7859-44c3-98b4-d22e73818567 code worked for me
The task
I needed to calculate embeddings for every article on swizec.com for a feature idea. Describing how to do this at a high level is easy:
- iterate through directories
- find index.mdx files
- read files
- feed into OpenAI's API
- save embeddings in a db
Writing the code would take a while. Verifying the code worked takes seconds. This makes it a perfect fit for AI – slow for me to generate, quick for me to verify.
How to program in markdown
You talk to the computer like you would a very talented intern. They'll surprise you with how much they know, the intuitive leaps they make, and the boneheaded mistakes they sometimes make.
Figure out how to read this random JSON payload – easy.
Consistently import only libraries that actually exist – nope.
I used ChatGPT with the Code Interpreter plugin because I'm lazy. Using the API should work and I imagine some of the stronger opensource models could make it happen for you.
Start with a broad outline
You start with a broad outline. This helps the AI (or human engineer) anticipate what's coming and construct a framework. It helps you clarify what you need too.
Write a script that finds all index.mdx files in ~/Documents/websites/swizec.com/src/pages/blog even if they're in subdirectories.
Looking back, this one is pretty weak. I'd include more context now.
Describe steps in subheadings
You then provide detail on how to do different parts of the task using subheadings. The AI (or human engineer) can use these when they get stuck.
For example:
## When the script finds a file
- it remembers the file path and filename
- prints the file path + name as a progress indicator
- it parses the file as mdx and remembers the title
- it calls the openai api to get an embedding vector of the whole file contents as input
Make sure to write all the code for making an API call, passing in the file contents, and getting the embedding. Use the official OpenAI SDK to make these requests.
When parsing the markdown, assume it contains a frontmatter with the title. Use a library to parse markdown, do not use regex. Include error handling and skip unparseable files.
Writing out steps in bullet points makes them easier to follow. The additional instructions help guide the AI away from doing silly things.
You won't notice the silly things when you first run the generation. They start happening when you iterate on the prompt and the AI tries changing various things because there's no good way to say what exactly you didn't like.
Provide context from docs
Unlike human engineers, the ChatGPT AI can't go online and look up docs. You'd need to write a custom app for that.
Instead, you can provide context and copypaste examples from docs:
## To compute an embedding
Example code from OpenAI docs to compute an embedding:
\```python
openai.Embedding.create(
model="text-embedding-ada-002",
input="The food was delicious and the waiter..."
)
\```
Example response:
\```json
{
"object": "list",
"data": [
{
"object": "embedding",
"embedding": [
0.0023064255,
-0.009327292,
.... (1536 floats total for ada-002)
-0.0028842222,
],
"index": 0
}
],
"model": "text-embedding-ada-002",
"usage": {
"prompt_tokens": 8,
"total_tokens": 8
}
}
\```
That's all it needs. Example code and example response. Then it stops hallucinating ideas on how to do this.
I can see a future where you have an agent that goes "hey I don't know this" and looks it up online. Would be easy-ish to build 🤔
Review the code and iterate
When you hit the button, you'll get your code:
import os
import frontmatter
import sqlite3
import openai
import json
# define path and API key
path = "~/Documents/websites/swizec.com/src/pages/blog"
openai.api_key = "Your OpenAI Key"
# connect to SQLite database
conn = sqlite3.connect('embedding_vectors.db')
cursor = conn.cursor()
# create table if not exists
cursor.execute('''
CREATE TABLE IF NOT EXISTS vectors (
path TEXT PRIMARY KEY,
filename TEXT,
title TEXT,
vector JSON
)
''')
# walk through the directories
for root, dirs, files in os.walk(os.path.expanduser(path)):
for file in files:
if file == "index.mdx":
# get the full file path
full_path = os.path.join(root, file)
print(f'Processing {full_path}')
# read the file
with open(full_path, 'r') as f:
try:
post = frontmatter.load(f)
title = post.get('title', 'No Title')
except Exception as e:
print(f'Error parsing file {full_path}: {e}')
continue
# get the embedding
try:
response = openai.Embedding.create(
model="text-embedding-ada-002",
input=f.read()
)
embedding = response['data'][0]['embedding']
except Exception as e:
print(f'Error generating embedding for {full_path}: {e}')
continue
# save the embedding
cursor.execute('''
INSERT INTO vectors (path, filename, title, vector)
VALUES (?, ?, ?, ?)
''', (full_path, file, title, json.dumps(embedding)))
# commit changes and close connection
conn.commit()
conn.close()
print('Done.')
In my case the AI figured out many details on its own. Created a database table, iterated directories (I didn't even know how to do this in python), knew how to use the frontmatter library, and knew how to interpret OpenAI's JSON response.
The final prompt you see here – https://chat.openai.com/share/a942b81c-7859-44c3-98b4-d22e73818567 – is my 18th iteration.
The more you iterate on the code, the more you have to add weird little instructions because the AI goes wonky. Feels like it's trying to explore the space of all possible answers in random directions looking for what you'll like.
Adding more instruction solidifies the parts you liked.
Does it replicate?
Yes!
I tried this again for a friend and incorporated lessons like including a better overview and pasting examples from documentation. Worked like a charm on first try.
👉 https://chat.openai.com/share/5dba4c81-dd67-4247-becc-32e90d1bda5e
Unfortunately I don't have the data my friend is trying to analyze and can't test the code. But it looks right.
A wild implication
Here's a wild idea, you could use this technique to interview senior+ engineers. Can you describe a solution to the problem well enough that an AI can write the code?
I think AI could be used to interview senior engineers
— Swizec Teller (@Swizec) July 31, 2023
1. Here is a business problem
2. Describe a solution in enough detail that AI spits out a working script
That way you’re testing engineering skills, not coding. Plus helping others write the code is a big part of the job.
Cheers,
~Swizec
PS: I originally heard the "describe your problem in markdown and generate the code" idea from Swyx
Continue reading about Programming in Markdown
Semantically similar articles hand-picked by GPT-4
- I tried generative AI on lots of data and we're not quite there yet
- How I turned 15 years of writing into a chatbot
- Livecoding #38 - A faux AI that writes JavaScript
- What does "solve problems" even mean
- The programming tutorial SEO industry is dead
Learned something new?
Read more Software Engineering Lessons from Production
I write articles with real insight into the career and skills of a modern software engineer. "Raw and honest from the heart!" as one reader described them. Fueled by lessons learned over 20 years of building production code for side-projects, small businesses, and hyper growth startups. Both successful and not.
Subscribe below 👇
Software Engineering Lessons from Production
Join Swizec's Newsletter and get insightful emails 💌 on mindsets, tactics, and technical skills for your career. Real lessons from building production software. No bullshit.
"Man, love your simple writing! Yours is the only newsletter I open and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. 👌"
Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.
Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.
Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help 👉 swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.
Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers 👉 ServerlessHandbook.dev
Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization
Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections
Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog
Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com
By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️