Getting Chrome and Puppeteer to work on AWS Lambda is … tricky. There’s many guides, many blogs out there and they’re all broken in subtle little ways you won’t realize until it’s too late.

Outdated versions of Chrome, build packages too big for an AWS Lambda, or silently breaking with an infinite timeout. When you end up trawling through old Github issues looking for clues you know something’s wrong.

That was me.

Click through for source

I’m writing this guide as a love letter to my future self. Never again will I fall into this trap. Maybe it helps you too πŸ™‚

The objective

We’re building a serverless screenshot service. It’s the TODOapp of the headless serverless Chrome world. Every other guide uses this example πŸ˜€

You can see my final repo on Github. Disregard the README. A long time ago I used a starter kit that no longer works. Chrome version too old, hangs on newPage.

We’ll take screenshots of Instagram posts. Convenient for embedding in emails. You could use a similar setup for testing or to make social sharing thumbnails. Whatever your heart desires.

I built my screenshot Lambda for TechLetterApp. A tool that converts markdown into newsletters.

You will learn about:

  • serverless
  • AWS Lambdas
  • headless Chrome
  • Chrome Puppeteer
  • uploading to S3

Our Lambda is going to take an Instagram URL, take a screenshot, upload the file to S3, and return the URL.

Init new project

Start a new project. We’re going raw, no init scripts. Don’t need πŸ˜‡

Click through for source

Yarn or npm will ask you a bunch of questions about your new project. Answer them and you’ll end up with a package.json file.

This is also a great opportunity to run git init.

S3 bucket

Before we begin, you’ll need an S3 bucket as well. There is surely a way to create these via Serverless, but I don’t know it.

Go into your AWS console, search for S3, create a new bucket and give it a name.

Serverless.yml

We start our project with serverless.yml – the Serverless configuration file. Handles all our AWS setup so we can avoid Amazon’s confusing dashboard and avoid clicking around. Also means your configuration and server setup becomes part of your code so it’s repeatable and always the same.

Version controlled too πŸ‘Œ

I’m assuming you already have Serverless and have set up your AWS credentials.

Create a serverless.yml file and add this stuff πŸ‘‡

Click through for source

Here’s what it means

  • service gives your service a name. Used when constructing endpoints for your Lambdas and such
  • provider tells Serverless that we’re using AWS and sets some basic config. Which node version we’re using, how much memory we want for Lambdas, the default timeout.
  • provider/iamRoleStatements configures permissions for our service. Our service needs upload access to S3. Permissions are the most common source of weird bugs and failures in my experience. Best copy from somewhere else πŸ™‚
  • functions specifies our Lambda functions. We have a screenshot-function handled by the handler method in src/index.js
  • package specifies service packaging instructions. We’re telling Serverless to exclude the local Chromium build just in case it’s there.
  • plugins Serverless comes with a rich ecosystem of plugins. This is where we specify them. We need serverless-plugin-chrome
  • custom lets us add a bunch of configuration. Usually for our plugins. Here we specify some runtime flags for Chrome and tell serverless-plugin-chrome to make Chrome available to the screenshot-function Lambda

Other guides will show you different ways of running Chrome. Those are outdated and bad. This is the best approach.

Using serverless-plugin-chrome means you’re running a remote cloud version of Chrome instead of packaging the entire binary into your Lambda. Makes deploys faster, the Lambda smaller, and your costs lower. I think.

Don’t quote me on the costs πŸ˜…

Install dependencies

Next we need to install some libraries.

Click through for source

@serverless-chrome/lambda will let us talk to Chrome
gm is ImageMagick, we’ll use it to resize and optimize images for size
puppeteer-core is a minimal version of Puppeteer so we can drive Chrome with JavaScript
superagent is a JavaScript library for making requests. We’re using it because I copied that part of the code from another guide
aws-sdk has a bunch of utilities for talking to AWS. In our case for S3 uploads.
url will make it easier for us to parse URLs
serverless-plugin-chrome handles all the Chrome stuff for us. We don’t have to care πŸ‘Œ

handler.js

Our handler.js file takes care of one thing: Accepting an HTTP request, setting CORS headers, initiating Chrome stuff, calling our code, and responding with an HTTP response.

Looks like this

Click through for source

Check the comments above to understand the code. Easier that way πŸ™‚

So we’ve got our handler. Let’s create the other files.

getChrome.js

I borrowed getChrome from another guide. Good guide but some details didn’t work out for me and lead me straight to a sea of GitHub issues.

getChrome makes sure to launch Chrome and return an instance of the browser we can use.

Click through for source

Ask @serverless-chrome/lambda to run the browser and use superagent to get its webSocket URL. We use it in the next step above to connect with Puppeteer.

Great, that’s our browser. Nothing packaged into our Lambda making it fat. Wonderful.

takeScreenshot.js

Now the fun stuff, taking a screenshot. I’m gonna explain the code in comments so it’s easier to follow πŸ€“

Click through for source

In a nutshell, our takeScreenshot function opens a new browser page, navigates to a specific URL, looks for the exact DOM node we want to screenshot, uploads to S3, and returns the result.

I used to try closing the page after screenshotting but that seems to hang sometimes for some websites. Causes problems so it’s best to avoid.

Lambdas kill their execution environment, and the browser, after running anyway. Shouldn’t be any harm in keeping pages open.

As you can see I wrote this code so it’s easy to add other websites. ✌️

uploadScreenshot.js

Last step in the puzzle is uploading our screenshot to S3. You can’t host from temporary Lambda files. Gotta put it somewhere permanent.

Again, I’ll explain the code inline. Think that’s a good approach.

Click through for source

We should now have everything in place to use our Instagram screenshot service.

The very important hack

This is a hack. It’s ugly. I found it in GitHub issues. Fixes the “socket hang up” error with Chrome.

Go into your node_modules/@serverless-chrome/lambda/dist/bundle.* files. There’s two of them.

In each of those files look for a line like this

Click through for source

Remove the --v=99 part

Click through for source

You have to do this every time you run Yarn or reinstall anything for any reason

Yes it sucks. Yes I should make an upstream PR. But doing it yourself unblocks you and makes your Lambda work right now.

I have no idea why that breaks it.

Another option might be to remove the DEBUG flag in your serverless.yml … but then how will you debug?

And, deploy πŸ™‚

Click through for source

Serverless packages your lambda, deploys to AWS, sets everything up for you, and returns a URL.

That URL is your API endpoint. Call it with an Instagram URL and behold your wonderful screenshots.

I wrapped mine in a nice UI. You can try it at TechLetter.App

PS: I’m working on new education materials, workshops, course and such for serverless, lambdas, full stack React, GraphQL, and Gatsby. Hit reply if you’re interested or ping me on Twitter.

Learned something new? Want to improve your skills?

Join over 10,000 engineers just like you already improving their skills!

Here's how it works πŸ‘‡

Leave your email and I'll send you an Interactive Modern JavaScript Cheatsheet πŸ“–right away. After that you'll get thoughtfully written emails every week about React, JavaScript, and your career. Lessons learned over my 20 years in the industry working with companies ranging from tiny startups to Fortune5 behemoths.

PS: You should also follow me on twitter πŸ‘‰ here.
It's where I go to shoot the shit about programming.