ashiina / lambda-local Goto Github PK

The following simple Lambda function prints the console log at the top, and the email parameter passed into the handler, but dynamoDB.getItem(..., function(err, data) {...}) never calls its callback function.

// index.js
console.log('Loading function');

// dependencies
var AWS = require('aws-sdk');

AWS.config.update({region: 'eu-west-1'});

// Get reference to AWS clients
var dynamodb = new AWS.DynamoDB();

function getUser(email, fn) {
    dynamodb.getItem({
        TableName: 'Users',
        Key: {
            email: {
                S: email
            }
        }
    }, function(err, data) {
        if (err) {
            return fn(err, null);
        } else {
            if ('Item' in data) {
                fn(null, data.Item.email.S);
            } else {
                fn(null, null); // User not found
            }
        }
    });
}

exports.handler = function(event, context) {
    var email = event.email;

    console.log('event.email: ' + email);
    getUser(email, function(err, email) {
        if (err) {
            console.log('Error: '+err);
            context.fail('Error: '+err);
        } else if (!emailFound) {
            console.log('User not found: '+email);
            context.fail('User not found: '+email);
        } else {
            console.log('User found: '+email);
            context.succeed('User found: '+email);
        }
    });
}

Command line:

$ cat ../events/testemail.js
module.exports = {
    email: "[email protected]"
};
$ cat ~/.aws/credentials
[default]
aws_access_key_id = ABC123DEF456
aws_secret_access_key = ABC123DEF456
$ lambda-local -e ../events/testemail.js -p ~/.aws/credentials -l index.js
Loading function
Logs
------
START RequestId: 8bea705d-497e-ae46-5e5f-8c311a81d2d0
event.email: [email protected]

I've found various people on Stack Overflow reporting problems like this where they aren't waiting for asynchronous functions to complete before calling context.succeed() but that's not the case here.

This code works correctly when used in the AWS Lambda environment.

Has anyone managed to get asynchronous aws-sdk calls to work in this way?

Hello Friends!
The following code will always run into a timeout error as the timeout is initialized but never cleared.

const lambdaLocal = require('lambda-local');
lambdaLocal.execute({
    event: {
      'key': 1,
      'another_key': "Some text"
    },
    lambdaFunc: {
      handler : function (event, context, callback) {
        callback(null, true);
      }
    },
    timeoutMs: 3000,
    callback: function(err, data) {
        if (err) {
            console.log(err);
        } else {
            console.log(data);
        }
    }
});

A relatively simple fix should be to assign the error timeout to a variable and clear it when the context succeeds. Or am I missing something?
I could obviously call process.exit() but that's unfortunately not an option in my program.

It would be great to add also env variables
https://aws.amazon.com/de/about-aws/whats-new/2016/11/aws-lambda-supports-environment-variables/

Example:
$ lambda-local -l function.js -h handler -e event_samples.json -E "firstname:julian,lastname:kleinhans"

or
$ lambda-local -l function.js -h handler -e event_samples.json -env-var="firstname:julian,lastname:kleinhans"

I can contribute this feature in next few weeks if I have some more time

I could be wrong, but it looks like lambda-local does not support the basic JS asynchronous functions (neither callback nor promise).

example with lambda-local 0.0.10:

exports.handler = function(event, context, callback) {
  setTimeout(()=>{
    console.log('Hello world')
    callback(null, 'End now')
  }, 5000)
}

$ node_modules/lambda-local/bin/lambda-local --lambdapath index.js --eventpath event_sample.js --handler handler

// result: 
// Logs
// ------
// START RequestId: 20d8f0c1-c3d2-cfc4-6257-e53621112125

lambda-local.execute is not stateless.

let opts ={
            lambdaPath: "cloud/functions/api",
            verboseLevel: 0,
}
lambda-local.execute(opts); <--- asyncExecute
lambda-local.execute(opts); <--- syncExecute because the first execute injects `callback` into opts.

Thanks for the recent work on verboseLevel :-)

I have a strange behavior here executing handlers on my Windows command line. It seems like callbacks are not executed. I put together the most simplest example I can imagine.

This is my handler:

exports.handler = function(event, context, callback) {
    var f = function() { console.log("x"); setTimeout(f, 500); }
    f();
};

This is the output of the local call:

$>lambda-local -l index.js -h handler -e event-samples/event.js
Logs
------
START RequestId: 43221ee0-9ca6-9d93-c5d6-0a73bf30ff21
x

My expected result would be a timeout or/and a lot of x output. Can you imagine anything I'm doing wrong here?

Hello,

This is a question instead of a reported issue, please please keep reading.

lambda-local is great, and I am expecting to provide CDC test for lambda func via lambda-local. Still find hard way to mock a function. Ex:
lambda function -- require foo -- require bar; how to mock bar.abc function with, say, sinon?

Could you please give me some hint on this? Many thanks.

I tried installing lambda-local from scratch and running it, and couldn't start it due to the following error.

$ node ./bin/lambda-local 
error: uncaughtException: The object must be a winston logger ! 

I realized that the setLogger function is expecting a winston function, but lambda-local is actually passing a winston.Logger object.
https://github.com/ashiina/lambda-local/blob/develop/bin/lambda-local#L11
https://github.com/ashiina/lambda-local/blob/develop/bin/lambda-local#L20

The test code is passing a winston object, so the actual code and test code are different.
https://github.com/ashiina/lambda-local/blob/develop/test/test.js#L12
https://github.com/ashiina/lambda-local/blob/develop/test/test.js#L36

I am not sure why I missed this until now... but lambda-local does not seem to be correctly working from clean install right now, so I am submitting a PR immediately.

I'm trying to locally test a function, but whenever I try to write to files using streams, it creates the file, but it's blank. Works fine in regular NodeJS. But when I try with lambda-local it doesn't work properly.

This is some test code, which works on regular NodeJS:

var fs = require('fs');
var readableStream = fs.createReadStream('file1.txt');
var writableStream = fs.createWriteStream('./tmp/file2.txt');

readableStream.pipe(writableStream);

lambda-local -l lambda.js -h screenshot -e local/event.json

info: START RequestId: dc9d838a-d402-c4a8-1c25-75bb4659ce1a
error:  TypeError: Cannot read property 'indexOf' of undefined
    at Object.<anonymous> (/Users/ninja/projects/redmarker/lambda-js/src/s3.js:9:57)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/Users/ninja/projects/redmarker/lambda-js/lambda.js:5:12)
    at Module._compile (module.js:569:30)

I am able to invoke the function on AWS.

instead of hard coding the profile in the .bash_profile... it would be nice if the param can mimi the aws-lambda function such that it will do a --profile aws_user_profile

that would be great.

Any chance you could run npm publish on the new changes which include the ability to access context.fail? I see you have updated the GitHub to reflect these changes to 0.0.6 but npm is still at version 0.0.5. I have hacked my local files to update but would be good to have it come from NPM. Thanks for all of your efforts on this.

Possibly related to my other problems, aws-sdk doesn't pick up a default region for me. For now, so I don't have to rebuild the compiled command I've hacked lib/utils.js to add

process.env['AWS_REGION'] = 'eu-west-1';

as a workaround.

Hi,
The path to event.js file is not constructed properly in windows.
Please fix it.

I am loving this, but am finding global errors super hard to read.. so before I start looking for a solution for my issue, I thought I would reach out and see if anyone has solved UX issue with human readable errors.

Just a small improvement request: If you pass in an object to context.succeed(), the content is dumped using console.log() which will not render nested objects properly. It would be better to use JSON.stringify. I have modified my local lambda-local as follows:

Context.done = function (err, message) {
        console.log("END");
        console.log("\n");
        console.log("Message");
        console.log("------");
        console.log(typeof message === "object" ?
                JSON.stringify(message, null, "\t") : message);

        process.exit();
};

I'm trying to migrate my project over to heroku and i'm using git to do so.
I cannot for the life of my figure out why it keeps being ignored and results in an 'Cannot find module' error.
awd-sdk is in my node_modules, and it's also specified as a dependency in my package.json.
-I've tried to uninstall/reinstall. I've tried to make sure it was installed globally.
-it's not being ignored in .gitignore, nor can I find it any of the .npmignore of other modules.
-I've cleared/purged my cache
-everything is up to date

anybody have any ideas? I would happy yo provide more info on the issue.

Lambda local doesn't support a new programming model: http://docs.aws.amazon.com/lambda/latest/dg/programming-model.html.

For example: Lambda function, that uses the new model:

exports.handler = function(event, context, callback) {
callback(null, "some success message");
};

Return following error:

Logs

START RequestId: 87968b45-8119-cba1-2d35-72e55c04e7bb

/lambda/app.js:2
callback(null, "some success message");
^
TypeError: undefined is not a function
at Object.exports.handler (/lambda/app.js:2:3)
at /usr/lib/node_modules/lambda-local/bin/lambda-local:54:27
at Object. (/usr/lib/node_modules/lambda-local/bin/lambda-local:56:3)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:935:3

There are no tests against the use of environment variables.
We should have some to increase the quality.

(Apologies for my previous release with a critical bug... That was my responsibility)

Hey,
first of all thanks for such a nice tool.

When trying to access some metadata on a host which is not amazon. it fails to access the host because it is not reachable (this ip is hardcoded in aws-sdk/lib/metadata_service.js)

Error: connect EHOSTUNREACH 169.254.169.254:80 - Local (192.168.178.134:51835)
at Object.exports._errnoException (util.js:856:11)
at exports._exceptionWithHostPort (util.js:879:20)
at connect (net.js:844:14)
at net.js:940:9
at nextTickCallbackWith0Args (node.js:433:9)
at process._tickDomainCallback (node.js:403:13)

I think the following commit's ordering of environment variables has caused an issue with executing AWS API calls:

5764912

The author moved the require for external functions below the setting of AWS_SESSION_TOKEN and custom environment variables. This had the effect of making this token visible to the AWS SDK loaded in the require. Since this token is only valid when using temporary credentials, it causes regular credential requests to fail.

See this AWS Security Blog Post

Loving this tool! Very easy to test my lambda-code locally.

I am writing my Lambda functions using Go, using a Node.js wrapper that spawns of a Go process.

For my test cases in Golang, I'd like to execute these by using lambda-test. For this to work, I was wondering if it is possible to add in an option-flag to turn off the additional 'info' & 'error' logging. So instead of having:

Would it be possible to just return the raw result:

Additionaly, would it be possible to pass in the JSON event-data as an event-string, instead of passing in a file-path. For example:

lambda-local -e '{"type":"qr","width":100,"height":200,"message":"testingtesting"}'

sry, my mistake - pls close this issue

Now that AWS Lambda supports ES6, i've used Lebab to transpile my ES5 code to ES6.

It works fine on AWS Lambda however trying to use lambda-local against it I get 'Unexpected token import' when i'm using Import instead of using 'require'.

Does lambda-local support ES6 yet?

I want to be able to call lambda-local from another node.js program, as a module.
Right now it only supports command line execution.
My immediate need for this comes from not being able to write mocha tests against the lambda-local execution, but I am sure it will be useful in many other situations too.

Perhaps the sdk needs an update... but when running the function and it gets to the success context it has a TypeError.

i tried this, this is what i got:

error: uncaughtException: /usr/local/lib/node_modules/lambda-local/lib/lambdalocal.js:11
const mute = require('mute'),
^^^^^
Use of const in strict mode.

is this a bug? can i solve this somehow? thanks!

In lambda the environment variable LAMBDA_TASK_ROOT is exported to the root of the unpacked archive. But lambda-local does not export it.

Upon testing #24 on a Windows machine the regex for the AWS credentials is breaking with the Windows line endings.

My AWS credentials file looks like this:

[default]
aws_access_key_id = ABCXYZ123
aws_secret_access_key = ABCXYZ123

Note the space either side of the equals. This fails the regex tests in utils.js lines 39 and 47. I'm pretty sure this was generated by aws-cli rather than hand edited, so others may have the same format.

Should be an easy fix to update the regex to ignore whitespace before/after the equals sign.

I have recently started to use lambda-local for unit tests and am a little confused by what the error is in the following output:

Lambda Category handler should return the correct number of categories: 
error: Error
error: ------
Assertion {
  __flags: 
   { ssfi: [Function: proxyGetter],
     lockSsfi: undefined,
     object: 8,
     message: null } }
  √ Lambda Category handler should return the correct number of categories: 8ms
  √ Color lambda should return a list of colors: 4ms

  2 passing (19ms)

Source: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html#nodejs-prog-model-handler-callback

So checking for errors like this is going to break a lot of lambdas that are actually running with no issues on AWS:

if (err === null) {

or

if (err !== null) {

This code also runs successfully:

callback(undefined, 'Hello from Lambda');

This reports error, so it's not just a simple truthy/falsy check:

callback(0, 'Hello from Lambda');

Would it be possible to include an example for Lambda@Edge with CloudFront?

For now the program has been using this to start lambda-local:

#!/usr/bin/env node

But the new nodejs versions are now published with the "nodejs" command instead of only "node".
So for now you had to link node to nodejs...

I guess we should replace node by nodejs in there...

I use a stack where API Gateway invokes Lambda Function via aws_proxy and this works great. The docs of aws says Requests will be proxied to Lambda with request details available in the "context" of your handler function.
So I am wondering if I can invoke it the same way local with this plugin? I did a few tests and generally the invocation works good but didn't get it to run as I needed to.
e.g. apigateway gets a get request to path hello and this invokes the lambda function with a proxy and you get a corresponding answer (normal rest api).

Here is the command I'm trying to run:
lambda-local -l index.js -e event.json -t 10 --envfile .env

And without the --envfileit was running well, provided that I used the dotenv library into my script.
I thought that would be better if I simulate lambda as close as it is in AWS and make use of your env file option.
I know that it works with JSON payload as -E, but I'm not really interested in that.
Any idea why this is happening? Thanks.

Otherwise all is great and thanks for sharing this good library.

What do you think of providing an option to disable these logs?

https://github.com/ashiina/lambda-local/blob/develop/lib/lambdalocal.js#L79-L82

    logger.log('info', 'Logs');
    logger.log('info', '------');
    logger.log('info', 'START RequestId: ' + context.awsRequestId);

I'd like to minimize the number of logs, and these logs do not add any value to my application.

Sorry if I missed something from the docs here, but it is there a way to get context result (message) from the callback, it only seems to return true?

I am building a little express app as wrapper for this so would be useful to get the payload out in this callback.

I am using:

mute: true,
callbackWaitsForEmptyEventLoop: false,
callback: (message) => res.send(message)

Running a script akin to the README:

const lambdaLocal = require('lambda-local');
const path = require('path');

var jsonPayload = {
    'key': 1,
    'another_key': "Some text"
}

lambdaLocal.execute({
    event: jsonPayload,
    lambdaPath: path.join(__dirname, 'lambda.js'),
    timeoutMs: 1000,
    callback: function(err, data) {
        if (err) {
            console.log(err);
        } else {
            console.log(data);
        }
    }
});

the function:

exports.handler = function (event, context) {
    setTimeout(function() {
      context.succeed('This is my serverless function!')
    }, 5000)
}

Will run for 5 seconds, whereas running it with the CLI will generate the appropriate timeout.

This library has been very useful so far. One thing that I found missing is to have an option to display the execution time of the lambda function.

Hi. I was going through the readme and when I got to the part about running lambda local in another node script the MochaJS link takes me to AWS docs, and I didn't explicitly see anything in particular about running lambda-local within another node script. I went through the tests you have written and found this: https://github.com/ashiina/lambda-local/blob/develop/test/test.js#L61, which solves my problem, but I was wondering if the docs were supposed to be linking to something in particular on that page that I missed?

Maybe it would be worth putting an example like the one above in the readme? I wouldn't mind doing this, I just wanted to know if it's something that I missed in the link that is there, though, before I started on that.

Thanks!

Terminal Command:

lambda-local -l build/app/handler.js -h handler -e src/event-samples/sample.js -t 10 -E {\"key\":\"val\"\,\"key2\":\"val2\"}

Output:

Invalid environment variable JSON format. 
Example: {\"key\":\"val\"\,\"key2\":\"val2\"}

version: 4.2.0
OS: MAC

Am I doing something wrong here ?

I want to test a Lambda function that uses aws-sdk. It looks like this should be supported by lambda-local but I'm getting the following error:

croesus$ lambda-local -l index.js -h handler -e ../events/MyLambdaFunctionEvent.js -p ~/.aws/credentials
Loading function
module.js:327
    throw err;
    ^

Error: Cannot find module 'aws-sdk'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/path/MyLambdaFunction/index.js:4:11)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)

I don't have aws-sdk installed locally within my Lambda package. I thought lambda-local would pick up its own packaged copy. Is there an environment variable I'm missing that tells node to look in the globally-installed lambda-local module for aws-sdk? I tried installing aws-sdk directly with npm install -g aws-sdk but that made no difference.

On macOS El Capitan, node 4.4.5, npm 2.15.5

This error seems to be related to #61

Line 51: environment = JSON.parse(program.environment),

program.environment is supposed to be an optional variable, so this line ends up attempting to parse a null variable.

If I run lambda-local -l index.js -h handler -e event-samples/test.js -E [] the issue goes away.

Either the documentation should be updated to make this argument required, or a check should be added in case the string is null or empty.

FULL ERROR MESSAGE:

error: uncaughtException: Unexpected token u in JSON at position 0 date=Wed Feb 08 2017 12:00:07 GMT-0500 (STD), pid=6808, uid=0, gid=0, cwd=/mnt/c/code/Screenshots/create2, execPath=/usr/bin/nodejs, version=v6.9.5, argv=[/usr/bin/nodejs, /usr/local/
bin/lambda-local, -l, index.js, -h, handler, -e, event-samples/test.js, -e, []], rss=21626880, heapTotal=11571200, heapUsed=6396832, loadavg=[0.5185546875, 0.57763671875, 0.5859375], uptime=3924, trace=[column=null, file=null, function=Object.parse, line=null, method=pars
e, native=true, column=28, file=/usr/local/lib/node_modules/lambda-local/bin/lambda-local, function=null, line=51, method=null, native=false, column=3, file=/usr/local/lib/node_modules/lambda-local/bin/lambda-local, function=, line=111, method=null, native=false, column=3
2, file=module.js, function=Module._compile, line=570, method=_compile, native=false, column=10, file=module.js, function=Object.Module._extensions..js, line=579, method=Module._extensions..js, native=false, column=32, file=module.js, function=Module.load, line=487, metho
d=load, native=false, column=12, file=module.js, function=tryModuleLoad, line=446, method=null, native=false, column=3, file=module.js, function=Function.Module._load, line=438, method=Module._load, native=false, column=10, file=module.js, function=Module.runMain, line=60
4, method=runMain, native=false, column=7, file=bootstrap_node.js, function=run, line=394, method=null, native=false], stack=[SyntaxError: Unexpected token u in JSON at position 0, at Object.parse (native), at /usr/local/lib/node_modules/lambda-local/bin/lambda-lo
cal:51:28, at Object. (/usr/local/lib/node_modules/lambda-local/bin/lambda-local:111:3), at Module._compile (module.js:570:32), at Object.Module._extensions..js (module.js:579:10), at Module.load (module.js:487:32), at tryModuleLoad (module.
js:446:12), at Function.Module._load (module.js:438:3), at Module.runMain (module.js:604:10), at run (bootstrap_node.js:394:7)]

This is probably very obvious, but I can't get that to work. I've tried the following:

lambda-local -l sms.js -e event.json -E {\"TWILIO_ACCOUNT_SID\":\"test\",\"TWILIO_AUTH_TOKEN\":\"test\"}

as well as variations of JSON array, to no avail.

Can you update the Readme file and clarify how to handle multiple environment variables/JSON keys?

It seems locally the stack size default is different than in the lambda environment. Being able to set the size of the stack in the command line would fix this issue. You can currently do this when invoking node via the --stack-size=* command line option. I can only assume there's a way to do this with lambda-local that i'm unaware of, but if you could add in the option to set it via a command line argument, that would be greatly appreciated.

Because the context object is a singleton, it is not possible to have concurrent runs of the lambda.execute function: the context is overwritten and shenanigans ensue.

Also, if you have nested calls to lambda.execute (don't ask, it's a corner case), then the 1st callback is overwritten by the 2nd callback, and then the 2nd callback is erroneously called after completion of the 1st call.

When using promises, this caused a livelock. When using callbacks, I ultimately got errors from express because 'headers have already been written' - the closure for the route was referencing the original response object.

I'll raise a PR to fix this.

Currently the context library does not provide the getRemainingTimeInMillis function causing an error:

TypeError: context.getRemainingTimeInMillis is not a function

See here: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html

Eg. lambda-local --debug-brk -l index.js -h handler -e event-samples/s3-put.js

I suggest you change the code sample for events to use Javascript style comments.

# Sample event data 
module.exports = {
    foo: "bar"
};

becomes

/* Sample event data */ 
module.exports = {
    foo: "bar"
};