Skip to content
Swizec Teller - a geek with a hatswizec.com

Serverless Chrome on AWS Lambda, the guide that works in 2019 and beyond

Getting Chrome and Puppeteer to work on AWS Lambda is ... tricky. There's many guides, many blogs out there and they're all broken in subtle little ways you won't realize until it's too late.

Outdated versions of Chrome, build packages too big for an AWS Lambda, or silently breaking with an infinite timeout. When you end up trawling through old Github issues looking for clues you know something's wrong.

That was me.

Click through for source
Click through for source

I'm writing this guide as a love letter to my future self. Never again will I fall into this trap. Maybe it helps you too :)

Serverless Handbook πŸ“– – modern backend for frontend engineers

Learn modern backend development with an in-depth handbook.

Shows you specific serverless recipes, talks about architecture, and teaches you the mindsets required to build robust and scalable systems.

Full of practical examples ❀️

Join over 10,000 engineers just like you already improving their JS careers with my letters, workshops, courses, and talks. ✌️

The objective

We're building a serverless screenshot service. It's the TODOapp of the headless serverless Chrome world. Every other guide uses this example :D

You can see my final repo on Github. Disregard the README. A long time ago I used a starter kit that no longer works. Chrome version too old, hangs on newPage.

We'll take screenshots of Instagram posts. Convenient for embedding in emails. You could use a similar setup for testing or to make social sharing thumbnails. Whatever your heart desires.

FBZeWxj

I built my screenshot Lambda for TechLetterApp. A tool that converts markdown into newsletters.

You will learn about:

  • serverless
  • AWS Lambdas
  • headless Chrome
  • Chrome Puppeteer
  • uploading to S3

Our Lambda is going to take an Instagram URL, take a screenshot, upload the file to S3, and return the URL.

Init new project

Start a new project. We're going raw, no init scripts. Don't need πŸ˜‡

Click through for source
Click through for source

Yarn or npm will ask you a bunch of questions about your new project. Answer them and you'll end up with a package.json file.

This is also a great opportunity to run git init.

S3 bucket

Before we begin, you'll need an S3 bucket as well. There is surely a way to create these via Serverless, but I don't know it.

Go into your AWS console, search for S3, create a new bucket and give it a name.

zvF6SZ3

Serverless.yml

We start our project with serverless.yml – the Serverless configuration file. Handles all our AWS setup so we can avoid Amazon's confusing dashboard and avoid clicking around. Also means your configuration and server setup becomes part of your code so it's repeatable and always the same.

Version controlled too πŸ‘Œ

I'm assuming you already have Serverless and have set up your AWS credentials.

Create a serverless.yml file and add this stuff πŸ‘‡

Click through for source
Click through for source

Here's what it means

  • service gives your service a name. Used when constructing endpoints for your Lambdas and such
  • provider tells Serverless that we're using AWS and sets some basic config. Which node version we're using, how much memory we want for Lambdas, the default timeout.
  • provider/iamRoleStatements configures permissions for our service. Our service needs upload access to S3. Permissions are the most common source of weird bugs and failures in my experience. Best copy from somewhere else :)
  • functions specifies our Lambda functions. We have a screenshot-function handled by the handler method in src/index.js
  • package specifies service packaging instructions. We're telling Serverless to exclude the local Chromium build just in case it's there.
  • plugins Serverless comes with a rich ecosystem of plugins. This is where we specify them. We need serverless-plugin-chrome
  • custom lets us add a bunch of configuration. Usually for our plugins. Here we specify some runtime flags for Chrome and tell serverless-plugin-chrome to make Chrome available to the screenshot-function Lambda

Other guides will show you different ways of running Chrome. Those are outdated and bad. This is the best approach.

Using serverless-plugin-chrome means you're running a remote cloud version of Chrome instead of packaging the entire binary into your Lambda. Makes deploys faster, the Lambda smaller, and your costs lower. I think.

Don't quote me on the costs πŸ˜…

Install dependencies

Next we need to install some libraries.

Click through for source
Click through for source

@serverless-chrome/lambda will let us talk to Chrome gm is ImageMagick, we'll use it to resize and optimize images for size puppeteer-core is a minimal version of Puppeteer so we can drive Chrome with JavaScript superagent is a JavaScript library for making requests. We're using it because I copied that part of the code from another guide aws-sdk has a bunch of utilities for talking to AWS. In our case for S3 uploads. url will make it easier for us to parse URLs serverless-plugin-chrome handles all the Chrome stuff for us. We don't have to care πŸ‘Œ

handler.js

Our handler.js file takes care of one thing: Accepting an HTTP request, setting CORS headers, initiating Chrome stuff, calling our code, and responding with an HTTP response.

Looks like this

Click through for source
Click through for source

Check the comments above to understand the code. Easier that way :)

So we've got our handler. Let's create the other files.

getChrome.js

I borrowed getChrome from another guide. Good guide but some details didn't work out for me and lead me straight to a sea of GitHub issues.

getChrome makes sure to launch Chrome and return an instance of the browser we can use.

Click through for source
Click through for source

Ask @serverless-chrome/lambda to run the browser and use superagent to get its webSocket URL. We use it in the next step above to connect with Puppeteer.

Great, that's our browser. Nothing packaged into our Lambda making it fat. Wonderful.

takeScreenshot.js

Now the fun stuff, taking a screenshot. I'm gonna explain the code in comments so it's easier to follow πŸ€“

Click through for source
Click through for source

In a nutshell, our takeScreenshot function opens a new browser page, navigates to a specific URL, looks for the exact DOM node we want to screenshot, uploads to S3, and returns the result.

I used to try closing the page after screenshotting but that seems to hang sometimes for some websites. Causes problems so it's best to avoid.

Lambdas kill their execution environment, and the browser, after running anyway. Shouldn't be any harm in keeping pages open.

As you can see I wrote this code so it's easy to add other websites. ✌️

uploadScreenshot.js

Last step in the puzzle is uploading our screenshot to S3. You can't host from temporary Lambda files. Gotta put it somewhere permanent.

Again, I'll explain the code inline. Think that's a good approach.

Click through for source
Click through for source

We should now have everything in place to use our Instagram screenshot service.

The very important hack

This is a hack. It's ugly. I found it in GitHub issues. Fixes the "socket hang up" error with Chrome.

Go into your node_modules/@serverless-chrome/lambda/dist/bundle.* files. There's two of them.

In each of those files look for a line like this

Click through for source
Click through for source

Remove the --v=99 part

Click through for source
Click through for source

You have to do this every time you run Yarn or reinstall anything for any reason

Yes it sucks. Yes I should make an upstream PR. But doing it yourself unblocks you and makes your Lambda work right now.

I have no idea why that breaks it.

Another option might be to remove the DEBUG flag in your serverless.yml ... but then how will you debug?

And, deploy :)

Click through for source
Click through for source

Serverless packages your lambda, deploys to AWS, sets everything up for you, and returns a URL.

That URL is your API endpoint. Call it with an Instagram URL and behold your wonderful screenshots.

I wrapped mine in a nice UI. You can try it at TechLetter.App

PS: I'm working on new education materials, workshops, course and such for serverless, lambdas, full stack React, GraphQL, and Gatsby. Hit reply if you're interested or ping me on Twitter.

Did you enjoy this article?

Published on April 11th, 2019 in Back End, Technical

Want to expand your serverless skills?

Leave your email and get the Serverless Handbook - a resource teaching you everything you need to know to dive into modern backend code.

Starts with the very basics, uses practical examples.

Learn how to choose the right database, write cloud functions, think about scalability, gain the architecture mindsets for robust systems, and more.

Serverless Handbook πŸ“– – modern backend for frontend engineers

Learn modern backend development with an in-depth handbook.

Shows you specific serverless recipes, talks about architecture, and teaches you the mindsets required to build robust and scalable systems.

Full of practical examples ❀️

Join over 10,000 engineers just like you already improving their JS careers with my letters, workshops, courses, and talks. ✌️

Have a burning question that you think I can answer?Β I don't have all of the answers, but I have some! Hit me up on twitter or book a 30min ama for in-depth help.

Ready to Stop copy pasting D3 examples and create data visualizations of your own? Β Learn how to build scalable dataviz components your whole team can understand with React for Data Visualization

Curious about Serverless and the modern backend? Check out Serverless Handbook, modern backend for the frontend engineer.

Ready to learn how it all fits together and build a modern webapp from scratch? Learn how to launch a webapp and make your first πŸ’° on the side with ServerlessReact.Dev

Want to brush up on your modern JavaScript syntax?Β Check out my interactive cheatsheet: es6cheatsheet.com

By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❀️