lucleray / tensorflow-lambda Goto Github PK

View Code? Open in Web Editor NEW

48.0 3.0 9.0 19.91 MB

run tensorflow on lambdas (aws, vercel, ...)

JavaScript 69.87% HCL 13.68% Shell 16.45%

tensorflow nodejs lambda aws aws-lambda tfjs serverless

tensorflow-lambda's Introduction

tensorflow-lambda

Usage

First, install the package:

yarn add tensorflow-lambda

You can then use it like this:

const loadTf = require('tensorflow-lambda')

const tf = await loadTf()

// you get the same `tf` object that would get if you were doing:
// const tf = require('@tensorflow/tfjs')

tf.tensor([1, 2, 3, 4]).print()

Have a look at these examples :

object-detection (deployed with Vercel)

Local usage

When not used in a lambda environment (for example, locally on your computer when you're developing), tensorflow-lambda will require @tensorflow/tfjs-node instead of deflating a pre-compiled version in /tmp.

Therefore, you need to install @tensorflow/tfjs-node to use this package locally:

yarn add @tensorflow/tfjs-node --dev

You can then use the package the same way you would use it in a lambda environment locally.

Have a look at these lines to understand how it detects if it runs in a lambda environement.

How it works ?

The package contains a zipped and compressed version of all the dependencies and binaries needed to run @tensorflow/tfjs-node on AWS Lambda (these dependencies are built with Github Actions).

During cold start, the files are deflated in /tmp and required in your node program.

Motivation

@tensorflow/tfjs works with AWS Lambda but the main problem is that it is slow very slow when used in node. On the other hand, @tensorflow/tfjs-node is fast when used with node but it is >140mo and it does not fit under AWS Lambda's size limit (50mo) and it needs to be pre-compiled for lambda for it to work in a lambda environment.

I was looking for an easy way to use tensorflowjs with lambda and I couldn't find any, so I made this package.

tensorflow-lambda's People

Contributors

Stargazers

Watchers

Forkers

salman0 marearts ajbouh saumilsdk jonathanlehner gradpratik jlarmstrongiv

tensorflow-lambda's Issues

brotli file missing

I made a fork https://github.com/JonathanLehner/tensorflow-lambda to update the tfjs-node version. I cannot get the brotli file to be publish on npm. how to do this?

Error: Cannot find module '/tmp/tfjs-node'

Using now dev 15.7.0
MacOS 10.14.5

Unhandled rejection: { Error: Cannot find module '/tmp/tfjs-node'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:580:15)
    at Function.Module._load (internal/modules/cjs/loader.js:506:25)
    at Module.require (internal/modules/cjs/loader.js:636:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at requireTf (/private/tmp/zeit-fun-36952c187fc17/score/index.js:10513:14)
    at createTfPromise (/private/tmp/zeit-fun-36952c187fc17/score/index.js:10527:12)
    at loadTf (/private/tmp/zeit-fun-36952c187fc17/score/index.js:10552:17)
    at module.exports.585.module.exports (/private/tmp/zeit-fun-36952c187fc17/score/index.js:14701:20)
    at calculateScores (/private/tmp/zeit-fun-36952c187fc17/score/index.js:19343:72)
    at searchSites (/private/tmp/zeit-fun-36952c187fc17/score/index.js:19391:26) code: 'MODULE_NOT_FOUND' }
LambdaError: RequestId: 226328a2-c467-45d9-be5e-da58e19029d4 Process exited before completing request
    at Lambda.<anonymous> (/snapshot/repo/dist/index.js:1:2535360)
    at Generator.next (<anonymous>)
    at fulfilled (/snapshot/repo/dist/index.js:1:2533501)
    at process._tickCallback (internal/process/next_tick.js:68:7)

AWS_LAMBDA_FUNCTION_NAME exists with now dev, so it tries to create tmp/tfjs-node and it fails (permission error). Can we use another variable to check if its local or production env?

It seems your package is broken??

I tried different versions and also tfjs-lambda, but it all gives the error on Zeit now. Cannot use my api anymore!! You shouldn't have fixed the tfjs-converter version to latest, which apparently caused it to break.

START RequestId: d2f72658-c095-47f4-b1a0-87fb5bd73c92 Version: $LATEST
2019-11-16T23:43:59.070Z d2f72658-c095-47f4-b1a0-87fb5bd73c92 TypeError: Cannot read property 'T' of undefined
at OperationMapper.getDtypeParam (/tmp/tfjs-node/node_modules/@tensorflow/tfjs-converter/dist/src/operations/operation_mapper_json.js:203:26)
at /tmp/tfjs-node/node_modules/@tensorflow/tfjs-converter/dist/src/operations/operation_mapper_json.js:150:43
at Array.reduce ()
at OperationMapper.mapNode (/tmp/tfjs-node/node_modules/@tensorflow/tfjs-converter/dist/src/operations/operation_mapper_json.js:112:44)
at /tmp/tfjs-node/node_modules/@tensorflow/tfjs-converter/dist/src/operations/operation_mapper_json.js:57:36
at Array.reduce ()
at OperationMapper.transformGraph (/tmp/tfjs-node/node_modules/@tensorflow/tfjs-converter/dist/src/operations/operation_mapper_json.js:56:29)
at FrozenModel. (/tmp/tfjs-node/node_modules/@tensorflow/tfjs-converter/dist/src/executor/frozen_model_json.js:128:113)
at step (/tmp/tfjs-node/node_modules/@tensorflow/tfjs-converter/dist/src/executor/frozen_model_json.js:32:23)
at Object.next (/tmp/tfjs-node/node_modules/@tensorflow/tfjs-converter/dist/src/executor/frozen_model_json.js:13:53)
2019-11-16T23:43:59.070Z d2f72658-c095-47f4-b1a0-87fb5bd73c92 { Error: The model could not be loaded (bad_request)
at createError (/var/task/api/lib/error.js:25:10)
at BadRequestError (/var/task/api/lib/error.js:41:3)
at loadModel (/var/task/api/index.js:28:11)
at
at process._tickDomainCallback (internal/process/next_tick.js:228:7)
statusCode: 400,
error:
{ code: 'bad_request',
message: 'The model could not be loaded' } }
END RequestId: d2f72658-c095-47f4-b1a0-87fb5bd73c92
REPORT RequestId: d2f72658-c095-47f4-b1a0-87fb5bd73c92

Dependency "iltorb" needs to be pre-compiled

iltorb is a native module, but it's added as a dependency, so it compiles locally. It should be pre-compiled like the rest of the packages.

Thanks for the project, very helpful! :)

Error when calling loadTf() when invoking lambda function locally

I'm trying to test my lambda function locally like this:

serverless invoke local -f myFunction -p event.json -l true

and when loadTf() is called like this:

const loadTf = require('tensorflow-lambda');
await loadTf();

it throws an error:
Cannot find module '/tmp/tfjs-node'

This might be more related to aws-lambda since paths in general seem kind of screwed up when invoking functions locally, but just checking to see if anyone has run into this before and how to handle it?

[email protected] ??

How does this relate to tfjs-lambda as mentioned in the article https://zeit.co/blog/serverless-machine-learning?

Not working with posenet?

Hi!

This project is a great idea and would be ideal for me. Unfortunately it does not work for me when using posenet. I hope this is the right place for my question. I could not find any discussion about this in Stackoverflow.

Here is a minimal example:

`const loadTf = require("tensorflow-lambda");
const posenet = require("@tensorflow-models/posenet");
const Jimp = require("jimp");
const imageScaleFactor = 0.5;
const outputStride = 16;
const flipHorizontal = false;
const tf = (async () => await loadTf())();

const example = async function (event, context, callback) {
const tf = await loadTf(); //(AWS)
const net = await posenet.load();
var jimpSrc = await Jimp.read("./person.jpg");
let input = jimpSrc.bitmap;
const pose = await net.estimateSinglePose(
input,
imageScaleFactor,
flipHorizontal,
outputStride
);
console.log(pose);
};
example();`

When I use this in Amazon AWS , I get the following error:
{
"errorType": "Runtime.ImportModuleError",
"errorMessage": "Error: Cannot find module '@tensorflow/tfjs-core'\nRequire stack:\n- /var/task/node_modules/@tensorflow-models/posenet/dist/mobilenet.js\n- /var/task/node_modules/@tensorflow-models/posenet/dist/index.js\n- /var/task/index.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
"trace": [
"Runtime.ImportModuleError: Error: Cannot find module '@tensorflow/tfjs-core'",
"Require stack:",
"- /var/task/node_modules/@tensorflow-models/posenet/dist/mobilenet.js",
"- /var/task/node_modules/@tensorflow-models/posenet/dist/index.js",
"- /var/task/index.js",
"- /var/runtime/UserFunction.js",
"- /var/runtime/index.js",
" at _loadUserApp (/var/runtime/UserFunction.js:100:13)",
" at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
" at Object. (/var/runtime/index.js:43:30)",
" at Module._compile (internal/modules/cjs/loader.js:1138:30)",
" at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)",
" at Module.load (internal/modules/cjs/loader.js:986:32)",
" at Function.Module._load (internal/modules/cjs/loader.js:879:14)",
" at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)",
" at internal/main/run_main_module.js:17:47"
]
}

Seems to me like the model itself is trying and failing to import tensorflow. Am I doing anything wrong? It would be great to get this to work on AWS lambda!

Updated version

For anyone looking for an updated version, I created tfjs-node-lambda. Contains both tensorflow 1.7.4 and 2.7.0, with automatic nightly updates. Inspired by this repo.