Thursday, March 23, 2017

How to run arbitrary binary executables on AWS Lambda 6.10+

After posting the hack yesterday, AWS coincidentally released the Node.js 6.10 platform. The old method from 0.10 now works:

1. Create a bin/ directory inside your project directory.
2. Statically compile your binary as much as possible. Put it inside your bin/ directory. Put any shared libraries that it needs inside the bin/ directory too.
3. chmod +x bin/hello-world-binary
4. Do something like:

'use strict';

const exec = require('child_process').exec;

process.env['PATH'] = process.env['PATH'] + ':' + process.env['LAMBDA_TASK_ROOT'] + '/bin';
process.env['LD_LIBRARY_PATH'] = process.env['LAMBDA_TASK_ROOT'] + '/bin';

exports.handler = (event, context, callback) => {
    const cmd = 'hello-world-binary';
    const child = exec(cmd, (error) => {
        callback(error, 'Process complete!');
    });

    child.stdout.on('data', console.log);
    child.stderr.on('data', console.error);
};

5. ZIP your project directory and upload to AWS Lambda. Enjoy!

Wednesday, March 22, 2017

How to run arbitrary binary executables on AWS Lambda 4.3+

I've been enjoying AWS Lambda in my work lately. I appreciate how we can create serverless architecture (or nearly serverless, anyway) on the AWS Platform. Yeah, it's fun to set up Linux and/or FreeBSD servers myself, but oftentimes, this "cloud" thingy makes perfect sense in many scenarios.

Anyway, AWS Lambda has various constraints, such as not being able to install packages yourself. These constraints, according to their documentation, allows them to "perform operational and administrative activities on [our] behalf, including provisioning capacity, monitoring fleet health, applying security patches, deploying [our] code, and monitoring and logging [our] Lambda functions."

OK, fine; but how do you deal with code that depends on an external binary? In previous versions of the Lambda platform (Node.js v0.10), you can do this by statically compiling your binary and by putting it inside the bin/ folder inside your project and, by doing something like:

process.env['PATH'] = process.env['PATH'] + ':' + process.env['LAMBDA_TASK_ROOT'] + '/bin';
process.env['LD_LIBRARY_PATH'] = process.env['LAMBDA_TASK_ROOT'] + '/bin';

exec('hello-world-binary', function () {});

With the recent update, however, the above code no longer works. On AWS Lambda platform that uses Node.js v4.3+, we'd need to copy all the binaries to the /tmp folder, chmod +x it, and run it from there.

I couldn't find a cleaner solution, so I just wrapped these in a Node.js package:

https://www.npmjs.com/package/lambda_exec

Enjoy!