Thursday, April 27, 2017

How to run AWS Lambda on VPC

I'm trying to run an AWS Lambda function inside a VPC because I need it to access ElastiCache. Problem: if you put a Lambda function inside a VPC, it loses Internet access. There are some few documentations online, but they are too complicated and they involve unnecessary steps. Here is the simplest way to run AWS Lambda inside VPC.

1. Create a simple NodeJS function that connects to an external site via HTTP:
'use strict';
const http = require('http');

exports.handler = (event, context, callback) => {
  http.get('', res => {
    callback(null, `Success, with: ${res.statusCode}`);
  }).on('error', err => callback(`Error with: ${err.message}`));

2. Run the above function without a VPC to verify that it's working correctly (i.e., it returns an HTTP 200).

3. In the AWS Console, go to the VPC page and click "Elastic IPs". Then, click the "Allocate new address" button and select the "VPC" scope.

4. Next, go to the VPC Dashboard and click the "Start VPC Wizard" button.

5. Select VPC with Public and Private Subnets option.

6. In the next page, enter your "VPC name", and in the "Elastic IP Allocation ID" field, enter the Elastic IP that you created in Step 3. Click the "Create VPC" button.

7. Finally, go back to the Lambda page and configure your function. Click the "Configuration" tab and go to the "Advanced settings section". Select the VPC that you created in Step 5, and Select the private subnet that you created. This is important; otherwise outgoing Internet connections won't work.

8. Click the "Save and test" button to test your setup.  That's it! For a proper setup, use at least 2 subnets in different availability zones to run your function in high-availability mode.

Some SEO keywords to help other people: aws lambda run vpc nat gateway howto tutorial

Thursday, March 23, 2017

How to run arbitrary binary executables on AWS Lambda 6.10+

After posting the hack yesterday, AWS coincidentally released the Node.js 6.10 platform. The old method from 0.10 now works:

1. Create a bin/ directory inside your project directory.
2. Statically compile your binary as much as possible. Put it inside your bin/ directory. Put any shared libraries that it needs inside the bin/ directory too.
3. chmod +x bin/hello-world-binary
4. Do something like:

'use strict';

const exec = require('child_process').exec;

process.env['PATH'] = process.env['PATH'] + ':' + process.env['LAMBDA_TASK_ROOT'] + '/bin';
process.env['LD_LIBRARY_PATH'] = process.env['LAMBDA_TASK_ROOT'] + '/bin';

exports.handler = (event, context, callback) => {
    const cmd = 'hello-world-binary';
    const child = exec(cmd, (error) => {
        callback(error, 'Process complete!');

    child.stdout.on('data', console.log);
    child.stderr.on('data', console.error);

5. ZIP your project directory and upload to AWS Lambda. Enjoy!

Wednesday, March 22, 2017

How to run arbitrary binary executables on AWS Lambda 4.3+

I've been enjoying AWS Lambda in my work lately. I appreciate how we can create serverless architecture (or nearly serverless, anyway) on the AWS Platform. Yeah, it's fun to set up Linux and/or FreeBSD servers myself, but oftentimes, this "cloud" thingy makes perfect sense in many scenarios.

Anyway, AWS Lambda has various constraints, such as not being able to install packages yourself. These constraints, according to their documentation, allows them to "perform operational and administrative activities on [our] behalf, including provisioning capacity, monitoring fleet health, applying security patches, deploying [our] code, and monitoring and logging [our] Lambda functions."

OK, fine; but how do you deal with code that depends on an external binary? In previous versions of the Lambda platform (Node.js v0.10), you can do this by statically compiling your binary and by putting it inside the bin/ folder inside your project and, by doing something like:

process.env['PATH'] = process.env['PATH'] + ':' + process.env['LAMBDA_TASK_ROOT'] + '/bin';
process.env['LD_LIBRARY_PATH'] = process.env['LAMBDA_TASK_ROOT'] + '/bin';

exec('hello-world-binary', function () {});

With the recent update, however, the above code no longer works. On AWS Lambda platform that uses Node.js v4.3+, we'd need to copy all the binaries to the /tmp folder, chmod +x it, and run it from there.

I couldn't find a cleaner solution, so I just wrapped these in a Node.js package: