Serverless Chrome on AWS Lambda, the guide that works in 2019 and beyond

Getting Chrome and Puppeteer to work on AWS Lambda is ... tricky. There's many guides, many blogs out there and they're all broken in subtle little ways you won't realize until it's too late.

Outdated versions of Chrome, build packages too big for an AWS Lambda, or silently breaking with an infinite timeout. When you end up trawling through old Github issues looking for clues you know something's wrong.

That was me.

I'm writing this guide as a love letter to my future self. Never again will I fall into this trap. Maybe it helps you too :)

The objective

We're building a serverless screenshot service. It's the TODOapp of the headless serverless Chrome world. Every other guide uses this example :D

You can see my final repo on Github. Disregard the README. A long time ago I used a starter kit that no longer works. Chrome version too old, hangs on newPage.

We'll take screenshots of Instagram posts. Convenient for embedding in emails. You could use a similar setup for testing or to make social sharing thumbnails. Whatever your heart desires.

I built my screenshot Lambda for TechLetterApp. A tool that converts markdown into newsletters.

You will learn about:

serverless
AWS Lambdas
headless Chrome
Chrome Puppeteer
uploading to S3

Our Lambda is going to take an Instagram URL, take a screenshot, upload the file to S3, and return the URL.

Init new project

Start a new project. We're going raw, no init scripts. Don't need 😇

Yarn or npm will ask you a bunch of questions about your new project. Answer them and you'll end up with a package.json file.

This is also a great opportunity to run git init.

S3 bucket

Before we begin, you'll need an S3 bucket as well. There is surely a way to create these via Serverless, but I don't know it.

Go into your AWS console, search for S3, create a new bucket and give it a name.

Serverless.yml

We start our project with serverless.yml – the Serverless configuration file. Handles all our AWS setup so we can avoid Amazon's confusing dashboard and avoid clicking around. Also means your configuration and server setup becomes part of your code so it's repeatable and always the same.

Version controlled too 👌

I'm assuming you already have Serverless and have set up your AWS credentials.

Create a serverless.yml file and add this stuff 👇

Here's what it means

service gives your service a name. Used when constructing endpoints for your Lambdas and such
provider tells Serverless that we're using AWS and sets some basic config. Which node version we're using, how much memory we want for Lambdas, the default timeout.
provider/iamRoleStatements configures permissions for our service. Our service needs upload access to S3. Permissions are the most common source of weird bugs and failures in my experience. Best copy from somewhere else :)
functions specifies our Lambda functions. We have a screenshot-function handled by the handler method in src/index.js
package specifies service packaging instructions. We're telling Serverless to exclude the local Chromium build just in case it's there.
plugins Serverless comes with a rich ecosystem of plugins. This is where we specify them. We need serverless-plugin-chrome
custom lets us add a bunch of configuration. Usually for our plugins. Here we specify some runtime flags for Chrome and tell serverless-plugin-chrome to make Chrome available to the screenshot-function Lambda

Other guides will show you different ways of running Chrome. Those are outdated and bad. This is the best approach.

Using serverless-plugin-chrome means you're running a remote cloud version of Chrome instead of packaging the entire binary into your Lambda. Makes deploys faster, the Lambda smaller, and your costs lower. I think.

Don't quote me on the costs 😅

Install dependencies

Next we need to install some libraries.

@serverless-chrome/lambda will let us talk to Chrome gm is ImageMagick, we'll use it to resize and optimize images for size puppeteer-core is a minimal version of Puppeteer so we can drive Chrome with JavaScript superagent is a JavaScript library for making requests. We're using it because I copied that part of the code from another guide aws-sdk has a bunch of utilities for talking to AWS. In our case for S3 uploads. url will make it easier for us to parse URLs serverless-plugin-chrome handles all the Chrome stuff for us. We don't have to care 👌

handler.js

Our handler.js file takes care of one thing: Accepting an HTTP request, setting CORS headers, initiating Chrome stuff, calling our code, and responding with an HTTP response.

Looks like this

Check the comments above to understand the code. Easier that way :)

So we've got our handler. Let's create the other files.

getChrome.js

I borrowed getChrome from another guide. Good guide but some details didn't work out for me and lead me straight to a sea of GitHub issues.

getChrome makes sure to launch Chrome and return an instance of the browser we can use.

Ask @serverless-chrome/lambda to run the browser and use superagent to get its webSocket URL. We use it in the next step above to connect with Puppeteer.

Great, that's our browser. Nothing packaged into our Lambda making it fat. Wonderful.

takeScreenshot.js

Now the fun stuff, taking a screenshot. I'm gonna explain the code in comments so it's easier to follow 🤓

In a nutshell, our takeScreenshot function opens a new browser page, navigates to a specific URL, looks for the exact DOM node we want to screenshot, uploads to S3, and returns the result.

I used to try closing the page after screenshotting but that seems to hang sometimes for some websites. Causes problems so it's best to avoid.

Lambdas kill their execution environment, and the browser, after running anyway. Shouldn't be any harm in keeping pages open.

As you can see I wrote this code so it's easy to add other websites. ✌️

uploadScreenshot.js

Last step in the puzzle is uploading our screenshot to S3. You can't host from temporary Lambda files. Gotta put it somewhere permanent.

Again, I'll explain the code inline. Think that's a good approach.

We should now have everything in place to use our Instagram screenshot service.

The very important hack

This is a hack. It's ugly. I found it in GitHub issues. Fixes the "socket hang up" error with Chrome.

Go into your node_modules/@serverless-chrome/lambda/dist/bundle.* files. There's two of them.

In each of those files look for a line like this

Remove the --v=99 part

You have to do this every time you run Yarn or reinstall anything for any reason

Yes it sucks. Yes I should make an upstream PR. But doing it yourself unblocks you and makes your Lambda work right now.

I have no idea why that breaks it.

Another option might be to remove the DEBUG flag in your serverless.yml ... but then how will you debug?

And, deploy :)

Serverless packages your lambda, deploys to AWS, sets everything up for you, and returns a URL.

That URL is your API endpoint. Call it with an Instagram URL and behold your wonderful screenshots.

I wrapped mine in a nice UI. You can try it at TechLetter.App

PS: I'm working on new education materials, workshops, course and such for serverless, lambdas, full stack React, GraphQL, and Gatsby. Hit reply if you're interested or ping me on Twitter.

Published on April 11th, 2019 in Back End, Technical

Did you enjoy this article?

👎👍

Senior Mindset Book

Get promoted, earn a bigger salary, work for top companies

Learn more

Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.

Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.

Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help 👉 swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.

Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers 👉 ServerlessHandbook.dev

Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization

Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections

Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog

Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com

By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️

Senior Mindset Book

Start with a free chapter

Serverless Chrome on AWS Lambda, the guide that works in 2019 and beyond

Serverless Handbook for Frontend Engineers – free chapter

The objective

Init new project

S3 bucket

Serverless.yml

Install dependencies

handler.js

getChrome.js

takeScreenshot.js

uploadScreenshot.js

The very important hack

And, deploy :)

Did you enjoy this article?

Want to dive into serverless? Not sure where to begin?

Serverless Handbook for Frontend Engineers – free chapter

Senior Mindset Book