Getting Chrome and Puppeteer to work on AWS Lambda is … tricky. There’s many guides, many blogs out there and they’re all broken in subtle little ways you won’t realize until it’s too late.
Outdated versions of Chrome, build packages too big for an AWS Lambda, or silently breaking with an infinite timeout. When you end up trawling through old Github issues looking for clues you know something’s wrong.
That was me.
I’m writing this guide as a love letter to my future self. Never again will I fall into this trap. Maybe it helps you too 🙂
We’re building a serverless screenshot service. It’s the TODOapp of the headless serverless Chrome world. Every other guide uses this example 😀
You can see my final repo on Github. Disregard the README. A long time ago I used a starter kit that no longer works. Chrome version too old, hangs on
We’ll take screenshots of Instagram posts. Convenient for embedding in emails. You could use a similar setup for testing or to make social sharing thumbnails. Whatever your heart desires.
I built my screenshot Lambda for TechLetterApp. A tool that converts markdown into newsletters.
You will learn about:
- AWS Lambdas
- headless Chrome
- Chrome Puppeteer
- uploading to S3
Our Lambda is going to take an Instagram URL, take a screenshot, upload the file to S3, and return the URL.
Init new project
Start a new project. We’re going raw, no init scripts. Don’t need 😇
Yarn or npm will ask you a bunch of questions about your new project. Answer them and you’ll end up with a
This is also a great opportunity to run
Before we begin, you’ll need an S3 bucket as well. There is surely a way to create these via Serverless, but I don’t know it.
Go into your AWS console, search for S3, create a new bucket and give it a name.
We start our project with serverless.yml – the Serverless configuration file. Handles all our AWS setup so we can avoid Amazon’s confusing dashboard and avoid clicking around. Also means your configuration and server setup becomes part of your code so it’s repeatable and always the same.
Version controlled too 👌
I’m assuming you already have Serverless and have set up your AWS credentials.
serverless.yml file and add this stuff 👇
Here’s what it means
- service gives your service a name. Used when constructing endpoints for your Lambdas and such
- provider tells Serverless that we’re using AWS and sets some basic config. Which
nodeversion we’re using, how much memory we want for Lambdas, the default timeout.
- provider/iamRoleStatements configures permissions for our service. Our service needs upload access to S3. Permissions are the most common source of weird bugs and failures in my experience. Best copy from somewhere else 🙂
- functions specifies our Lambda functions. We have a
screenshot-functionhandled by the
- package specifies service packaging instructions. We’re telling Serverless to exclude the local Chromium build just in case it’s there.
- plugins Serverless comes with a rich ecosystem of plugins. This is where we specify them. We need
- custom lets us add a bunch of configuration. Usually for our plugins. Here we specify some runtime flags for Chrome and tell
serverless-plugin-chrometo make Chrome available to the
Other guides will show you different ways of running Chrome. Those are outdated and bad. This is the best approach.
serverless-plugin-chrome means you’re running a remote cloud version of Chrome instead of packaging the entire binary into your Lambda. Makes deploys faster, the Lambda smaller, and your costs lower. I think.
Don’t quote me on the costs 😅
Next we need to install some libraries.
@serverless-chrome/lambda will let us talk to Chrome
gm is ImageMagick, we’ll use it to resize and optimize images for size
aws-sdk has a bunch of utilities for talking to AWS. In our case for S3 uploads.
url will make it easier for us to parse URLs
serverless-plugin-chrome handles all the Chrome stuff for us. We don’t have to care 👌
handler.js file takes care of one thing: Accepting an HTTP request, setting CORS headers, initiating Chrome stuff, calling our code, and responding with an HTTP response.
Looks like this
Check the comments above to understand the code. Easier that way 🙂
So we’ve got our
handler. Let’s create the other files.
I borrowed getChrome from another guide. Good guide but some details didn’t work out for me and lead me straight to a sea of GitHub issues.
getChrome makes sure to launch Chrome and return an instance of the browser we can use.
@serverless-chrome/lambda to run the browser and use superagent to get its webSocket URL. We use it in the next step above to connect with Puppeteer.
Great, that’s our browser. Nothing packaged into our Lambda making it fat. Wonderful.
Now the fun stuff, taking a screenshot. I’m gonna explain the code in comments so it’s easier to follow 🤓
In a nutshell, our
takeScreenshot function opens a new browser page, navigates to a specific URL, looks for the exact DOM node we want to screenshot, uploads to S3, and returns the result.
I used to try closing the page after screenshotting but that seems to hang sometimes for some websites. Causes problems so it’s best to avoid.
Lambdas kill their execution environment, and the browser, after running anyway. Shouldn’t be any harm in keeping pages open.
As you can see I wrote this code so it’s easy to add other websites. ✌️
Last step in the puzzle is uploading our screenshot to S3. You can’t host from temporary Lambda files. Gotta put it somewhere permanent.
Again, I’ll explain the code inline. Think that’s a good approach.
We should now have everything in place to use our Instagram screenshot service.
The very important hack
This is a hack. It’s ugly. I found it in GitHub issues. Fixes the “socket hang up” error with Chrome.
Go into your
node_modules/@serverless-chrome/lambda/dist/bundle.* files. There’s two of them.
In each of those files look for a line like this
You have to do this every time you run Yarn or reinstall anything for any reason
Yes it sucks. Yes I should make an upstream PR. But doing it yourself unblocks you and makes your Lambda work right now.
I have no idea why that breaks it.
Another option might be to remove the
DEBUG flag in your serverless.yml … but then how will you debug?
And, deploy 🙂
Serverless packages your lambda, deploys to AWS, sets everything up for you, and returns a URL.
That URL is your API endpoint. Call it with an Instagram URL and behold your wonderful screenshots.
I wrapped mine in a nice UI. You can try it at TechLetter.App
PS: I’m working on new education materials, workshops, course and such for serverless, lambdas, full stack React, GraphQL, and Gatsby. Hit reply if you’re interested or ping me on Twitter.
Learned something new? Want to improve your skills?
Join over 10,000 engineers just like you already improving their skills!
Here's how it works 👇
PS: You should also follow me on twitter 👉 here.
It's where I go to shoot the shit about programming.