enigmatic-smile / serverless-plugin-optimize Goto Github PK

⛔️ DEPRECATED ⛔️ Bundle with Browserify, transpile and minify with Babel automatically to your NodeJS runtime compatible JavaScript

Home Page: https://www.npmjs.com/package/serverless-plugin-optimize

License: MIT License

JavaScript 100.00%

serverless uglify minify serverless-functions babel serverless-optimizer-plugin browserify aws-lambda nodejs deprecated

serverless-plugin-optimize's People

Contributors

Stargazers

Watchers

serverless-plugin-optimize's Issues

Add support for .babelrc file

It would be nice if the babel configuration was pulled from .babelrc file which is generally the cleanest way to do it and will allow for other babel functionality like plugins etc

Problem with 'knex' module

Hey guys, I tried several different options with bundle/no bundle using 'external' option, etc and I can't make the 'knex' module works using serverless-plugin-optmize.

When function run and uses require('knex') we got the following error:

Cannot find module 'babel-runtime/core-js/object/define-properties'.

Any help ?

very slow to package and deploy service

I have a service with 20 functions in it and when I do sls deploy it optimizes, packages all the functions and deploy.
This is taking more than 30 minutes now to run on my MacOS.. I'm not sure the problem is with this plugin or the serverless framework itself. Anyone experiencing this for large services with more than 15 functions?

My package.json:

{
  "name": "arena-node-api-service",
  "version": "1.0.0",
  "description": "",
  "main": "handler.js",
  "directories": {
    "lib": "lib"
  },
  "scripts": {
    "test": "jest --forceExit"
  },
  "jest": {
    "testEnvironment": "node"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "jest": "^23.6.0",
    "rewire": "^4.0.1",
    "serverless-plugin-optimize": "^3.1.1-rc.1"
  },
  "dependencies": {
    "@iopipe/iopipe": "^1.11.1",
    "algoliasearch": "^3.30.0",
    "aws-sdk": "^2.351.0",
    "cheerio": "^1.0.0-rc.2",
    "elasticsearch": "^15.1.1",
    "firebase-admin": "^6.3.0",
    "jsonwebtoken": "^8.4.0",
    "lambda-api": "^0.8.1",
    "langmap": "0.0.16",
    "lodash": "^4.17.11",
    "mongoose": "5.2.5",
    "mongoose-findorcreate": "^3.0.0",
    "open-graph-scraper": "^3.5.1",
    "pretty-ms": "^4.0.0",
    "redis": "^2.8.0",
    "request": "^2.88.0",
    "request-promise": "^4.2.2",
    "stripe": "^6.17.0",
    "twitter": "^1.7.1",
    "ua-parser-js": "^0.7.19"
  }
}

My serverless.yml:

service: arena-node-api

# You can pin your service to only deploy with a specific Serverless version
# Check out our docs for more details
# frameworkVersion: "=X.X.X"

custom:
  stage: "${opt:stage, self:provider.stage}"
  optimize:
    debug: false
    external: ['mongoose']
  presence-frequency-dev: 10
  presence-frequency-prd: 2

provider:
  name: aws
  stage: dev
  runtime: nodejs8.10
  profile: default
  region: us-west-2
  memorySize: 256
  vpc:
    securityGroupIds:
      - XX-redacted
    subnetIds:
      - XX-redacted
      - XX-redacted
      - XX-redacted
  environment:
      env: ${self:custom.stage}
  apiKeys:
    - admin-apikey-${self:custom.stage}

# you can add packaging information here
package:
  individually: true

functions:
  billing:
    handler: handler_billing_api.handler
    timeout: 12
    events:
      - http:
          path: billing
          method: ANY
          cors:
            origin: '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - Preferred-Language
      - http:
          path: billing/{proxy+}
          method: ANY
          cors:
            origin: '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - Preferred-Language
  oembed:
    handler: handler_oembed.oembed
    timeout: 15
    events:
      - http:
          path: oembed/{proxy+}
          method: ANY
          cors:
            origin: '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - Preferred-Language
  metrics:
    handler: handler_metrics_api.handler
    events:
      - http:
          path: metrics/{proxy+}
          method: ANY
          cors:
            origin: '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - Preferred-Language
  integrations:
    handler: handler_integrations_api.handler
    events:
      - http:
          path: integrations/{proxy+}
          method: ANY
          cors:
            origin: '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - Preferred-Language
  contacts:
    handler: handler_contacts_api.handler
    timeout: 30
    events:
      - http:
          path: contacts/{proxy+}
          method: ANY
          cors:
            origin: '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - Preferred-Language
  content-manager:
    handler: handler_content_manager_api.handler
    timeout: 30
    memorySize: 2536
    optimize:
      external: ['@google-cloud/firestore', 'mongoose']
    events:
      - http:
          path: cm/{proxy+}
          method: ANY
          cors:
            origin: '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - Preferred-Language
  stream:
    handler: handler_stream_api.handler
    timeout: 30
    events:
      - http:
          path: stream/{proxy+}
          method: ANY
          cors:
            origin: '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - Preferred-Language
  admin-api:
    handler: handler_admin_api.handler
    timeout: 30
    optimize:
      external: ['@google-cloud/firestore']
    events:
      - http:
          path: admin/{proxy+}
          method: ANY
          private: true
  live-score-update:
    handler: handler_live_score_update.handler
  billing-job:
    handler: handler_billing_job.handler
    timeout: 30
    reservedConcurrency: 5
  update-intercom:
    handler: handler_update_intercom.handler
    memorySize: 128
    timeout: 300
    reservedConcurrency: 5
    vpc:
      securityGroupIds: []
      subnetIds: []
  track-collect:
    handler: handler_track_collect.handler
    timeout: 300
    events:
      - sqs: arn:aws:sqs:us-west-2:XXXX:events_${self:custom.stage}
  stream-autopost:
      handler: handler_stream_autopost.handler
      memorySize: 512
      optimize:
        external: ['@google-cloud/firestore', 'mongoose']
      timeout: 300
  presence-main-job:
      handler: handler_presence_job.main
      memorySize: 2536
      timeout: 300
      events:
        - schedule: rate(${self:custom.presence-frequency-${self:custom.stage}} minutes)
  presence-update-presence:
      handler: handler_presence_job.update_presence
      memorySize: 512
      timeout: 300
  presence-update-publisher:
      handler: handler_presence_job.update_publisher
      memorySize: 512
      timeout: 300
  presence-check-stale-sessions:
      handler: handler_presence_job.check_stale_sessions
      memorySize: 512
      timeout: 300
      events:
        - schedule: rate(4 hours)
  classify-asset:
      handler: handler_classify_asset.handler
      timeout: 300
      events:
        - s3:
            bucket: arena-cm-upload-${self:custom.stage}
            event: s3:ObjectCreated:*
  sync-fb-likes:
    handler: handler_sync_fb_likes.handler
    memorySize: 512
    timeout: 300
  admin-op:
    handler: handler_admin_api.handler_op
    timeout: 10
    optimize:
      external: ['@google-cloud/firestore']
plugins:
  - serverless-plugin-optimize

serverless cli version: 1.30.1

Feedback on other Serverless setups

We are using this plugin in production but would be useful to get some information how does it handle other setups.

Cheers! 😃

[1.16.x/1.17.x] node_modules is included in the package file

I just updated to serverless v1.17, and the plugin stopped working (the function size was 28mb)
after downgrading to v1.16, the plugin was back (function size 68kb)

No need to exclude aws-sdk by default

Since aws-sdk has recently supported browserify, it might be the time to stop excluding it in this plugin.
More info: aws/aws-sdk-js#696

[TypeScript Plugin] ENOENT: no such file or directory 'node_mnode_modules/...'

When I set external in the serverless.yml options to be the name of two node modules (which exist) I get the following error:

ENOENT: no such file or directory, lstat '/Users/darbio/Repositories/tsz/tsz-serverless-api/.build/node_mnode_modules/nodemailer'

Function:

  wp-shared-email:
    handler: functions/workers/shared/email/handler.index
    optimize:
      external: [
        'nodemailer',
        'email-templates'
      ]
    events:
      - sns:
          topicName: ${self:custom.sns-topic-shared-email}
          displayName: SNS topic for sending email

Error:

192-168-1-101:tsz-serverless-api$ serverless deploy function -f wp-shared-email --aws-profile tsz-dev --stage dev
Serverless: Configuring serverless offline -> localstack
Serverless: Configuring serverless offline -> kinesis consumer
Serverless: Compiling with Typescript...
Serverless: Using local tsconfig.json
Serverless: Typescript compiled.
Serverless: Optimize: starting engines
Serverless: Optimize: tsz-serverless-api-dev-wp-shared-email

  Error --------------------------------------------------

  ENOENT: no such file or directory, lstat '/Users/darbio/Repositories/tsz/tsz-serverless-api/.build/node_mnode_modules/nodemailer'

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           6.9.4
     Serverless Version:     1.26.0

Lambda file structure

I recently encountered the issue around the folder structure that optimize creates and trying to access things on the file system (#32).

I was wondering why it makes that structure at all, rather than your function and files being uploaded directly into the root of each lambda?

Is there a specific reason?

module initialization error

After adding the plugin i get this error in CloudWatch when calling any Lambda function:

module initialization error: Error
at Error (native)
at Object.fs.lstatSync (fs.js:839:18)
at Object.realpathSync (fs.js:1443:21)
at PATH.js:60409:38
at Array.map (native)
at Object.<anonymous> (PATH.js:60408:3)
at Object.291.fs (PATH.js:60539:4)
at s (PATH.js:1:683)
at PATH.js:1:734
at Object.278.PATH.js:57841:16)

This seems to caused by these lines in Uglify.js:


var FILES = exports.FILES = [
    "../lib/utils.js",
    "../lib/ast.js",
    "../lib/parse.js",
    "../lib/transform.js",
    "../lib/scope.js",
    "../lib/output.js",
    "../lib/compress.js",
    "../lib/sourcemap.js",
    "../lib/mozilla-ast.js"
].map(function(file){
    return path.join(path.dirname(fs.realpathSync(__filename)), file);
});

refactor(exclude): aws-sdk should be optional

Error: Cannot find module './drivers/node-mongodb-native/connection'

When using mongoose , as in just doing something like import mongoose from 'mongoose' I get this error: Error: Cannot find module './drivers/node-mongodb-native/connection'. It gets fixed if I remove the plugin so I am pretty sure this is causing it.

Cannot find module 'ejs'

Hi,
I am using ejs module. without plugin everything works fine.
Kindly guide.

Error: Cannot find module 'ejs'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)

feat(serverless-offline): Add plugin support

Optimize and transform handles paths before serverless-offline does its magic.

PR's are welcomed! 👍

Does not optimize nodejs8.10

When I deploy using the nodejs8.10 runtime in my serverless.yml the code is not optimized and my functions get too big.

Support for ES6 and NodeJS 6.10

With AWS Lambda adding support for NodeJS 6.10 uglify needs to be updated to support es6 features.

Currently there is this fork: https://github.com/mishoo/UglifyJS2 that should allow for es6 to be compiled and used with this plugin. I will do some testing to make sure everything works as expected. The other alternative is to compile the code down to es5.

Would this project consider upgrading to uglifyjs2?

Add input excludeStage

Hello, I am wondering if it's possible to add a new input: excludeStage.

For invoke local for instance, we often don't need to optimize the function, or at least it should be optional. It will be good to exclude the optimization processing for some stages.

What do you think?

Warning Messages

Getting warning message multiple time when deploy new code on lambda function.

Warning, the following targets are using a decimal version:

  node: 6.5

We recommend using a string for minor/patch versions to avoid numbers like 6.10
getting parsed as 6.1, which can lead to unexpected behavior.

Serverless version 1.15.3
runtime: nodejs6.10

feat(functions): Add minify flag inside function

This may be useful and easy one.

No works when I attach file from dist folder

I have an external process with a js dependency, includePaths no works with dist folder, I attach my code.
custom: optimize: minify: false includePaths: - 'src/shared/translation/es.json' - 'src/shared/translation/en.json' - 'ormconfig.js.example' - 'dist/database/strategies/snake.naming.js' includeDependencies: always: - 'dist/**'

does not run for google cloud functions

@goncaloneves , I'm trying to use optimize to transpile v8 code down to v6.11 for google cloud functions, however it doesn't seem to run.

Is there a specific filter somewhere in the code

To recreate, just use the google quickstart and add the plugin to the project, and call a local invoke (expected behavior is for sls to optimize first, create a minified version, then execute the optimized code).

Error "Cannot find Module" no stack trace.

Hello,

I had an issue with my was lambda function. I have been following this tutorial - https://medium.com/@robdel12/using-amazon-lambda-to-create-a-contact-form-a41b192fae84

First the issue was the same as in serverless/serverless#948 however after following those instructions and running npm install I now get the following error when testing.

{"errorMessage":"Cannot find module '/var/task/handler'","errorType":"Error","stackTrace":[]}

Regards,

Daniel

make global packages be accessible to the optimized bundle

Hi,

I am using sequelize and it is required that I do
npm install mysql2 --save before the code works.

i tried this config

custom:
  optimize:
    debug: true
    minify: true
    plugins: ['transform-object-rest-spread']
    includePaths: ['.env.example']
    external: ['mysql2']

but i still get this error
You need to install mysql2 package manually
i can see the node_modules folder on the zip file but i suspect it's placed on the wrong folder?

i also tried

require('mysql2');

on my handler but it failed, i think it wasn't able to get the dependencies of mysql2

What could I possibly do to overcome this issue using this plugin?
Will appreciate any help. Thanks!

feat(browserify): Add extensions array

Feature

Add Browserify extensions array, to bundle other extensions besides .js and .json.

Implement

Extensions in Browserify

Fixes

Closes #19 and closes #13.

fix(1.1.0): Zip package not excluding other files

Related with this commit.

feat(deploy): Add individual function support

serverless deploy function needs to be supported.

plugin breaks provider.deploymentBucket

If setting:

plugins:
  - serverless-plugin-optimize
provider:
  name: aws
  deploymentBucket: my-deployment-bucket

deployment fails with:

Serverless: Validating template...

  Error --------------------------------------------------

  The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [ServerlessDeploymentBucket] in the Resources block of the template

Looking at the generated CloudFormation template:

--- cloudformation-template-update-stack.json   2017-08-03 16:26:54.695489014 +0300
+++ cloudformation-template-update-stack.json   2017-08-03 16:04:48.612095055 +0300
@@ -86,11 +86,13 @@
       "Type": "AWS::Lambda::Function",
       "Properties": {
         "Code": {
-          "S3Bucket": "my-deployment-bucket",
-          "S3Key": "serverless/serverless-simple-http-endpoint/dev/1501766814665-2017-08-03T13:26:54.665Z/serverless-simple-http-endpoint.zip"
+          "S3Bucket": {
+            "Ref": "ServerlessDeploymentBucket"
+          },
+          "S3Key": "serverless/serverless-simple-http-endpoint/dev/1501765330724-2017-08-03T13:02:10.724Z/serverless-simple-http-endpoint.zip"
         },
         "FunctionName": "serverless-simple-http-endpoint-dev-currentTime",
-        "Handler": "handler.endpoint",
+        "Handler": "_optimize/serverless-simple-http-endpoint-dev-currentTime/handler.endpoint",
         "MemorySize": 1024,
         "Role": {
           "Fn::GetAtt": [

It seems that the plugin interferes with the S3Bucket definition.

Breaking envkey (TypeError: Cannot create property 'stack' on string 'ENVKEY invalid. Couldn't load vars.')

First of all, thank you for this plugin. It cut down my serverless bundle by about 30x.

However, my function no longer executes on AWS. It executes locally, but not in AWS, and seems to be something to do with envkey, but I'm not sure why it broke.

The error I'm seeing in the logs is:
TypeError: Cannot create property 'stack' on string 'ENVKEY invalid. Couldn't load vars.'

I've verified, however, that the ENVKEY environment variable is set and valid.

I'm importing envkey at the top of my handler file as instructed like:
import 'envkey'

Any ideas? Or config I need to set to make this work?

Add option to disable optimize on invoke local

Suggested by @olconstant in #48.

Probably a new CLI invoke command to run optimize.

fix(options): Null error when arrays are empty

feat(tests): Create tests

For now I will leave the plugin as RC until we have tests done.

Include files in the package root

Cannot include files in the package root (i.e. LAMBDA_TASK_ROOT/lib).

The LD_LIBRARY_PATH environment variable is used to store helper libraries and function code.
The LD_LIBRARY_PATH environment variable contains /lib64, /usr/lib64, LAMBDA_TASK_ROOT, LAMBDA_TASK_ROOT/lib.
https://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html

Access static files (templates)

Hi,

I am trying to deploy a serverless function that requires access to liquid templates.
I managed to include the static files in the bundle through the "includePath" config parameters but I am unable to get a path to them.

This is because the node process calling the handler is not in the same folder as the handler itself. Hence I cannot find a path to the handler (that I can then prefix to the path of my templates) to fs.readFile them... I tried using __dirname, but Browserify changes it while bundling to the path at the time of compilation...

Would you have any idea of how to make it work, please ?

PS: I explored the https://github.com/JohnPostlethwait/stringify way but I cannot define additional transforms to browserify in serverless-plugin-optimize, can I ?

BR,

Hugues

feat(transforms): Add ignore array

Feature

Add ignore array globally or per function, to avoid transpiling and minifying specific modules or local files.

Implement

Run a function locally

Is there a way to run optimize lambda local ?

Including a binary file

Hey,
Thanks for the awesome plugin. Some of my functions require a binary (ffmpeg). How can I set browserify to package the binary for some functions and not others?
Thanks!

Unexpected Token while parsing file (spread operator)

I have tried to disable minification as i read as a solution from another issues here, but it still didnt work.

Unexpected token (273:46) while parsing file at Parser.pp$5.raise

Unable to import module

This worked at one point, but now I'm getting this while running in Lambda:

at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at o (/var/task/_optimize/graphql-api-dev-graphql/src/handlers/graphql.js:1:554)
at /var/task/_optimize/graphql-api-dev-graphql/src/handlers/graphql.js:1:745
at Object.25.. (/var/task/_optimize/graphql-api-dev-graphql/src/handlers/graphql.js:76:851)
at o (/var/task/_optimize/graphql-api-dev-graphql/src/handlers/graphql.js:1:694)
at /var/task/_optimize/graphql-api-dev-graphql/src/handlers/graphql.js:1:745
at Object.33../AmenityService (/var/task/_optimize/graphql-api-dev-graphql/src/handlers/graphql.js:100:191)

Here's line 100 of my function:

"use strict";Object.defineProperty(exports,"__esModule",{value:!0});var AmenityService_1=require("./AmenityService");exports.AmenityService=AmenityService_1.AmenityService;var AuthService_1=require("./AuthService");exports.AuthService=AuthService_1.AuthService;var HotelService_1=require("./HotelService");exports.HotelService=HotelService_1.HotelService;var LocationService_1=require("./LocationService");exports.LocationService=LocationService_1.LocationService;var EmailService_1=require("./EmailService");exports.EmailService=EmailService_1.EmailService;var FbService_1=require("./FbService");exports.FbService=FbService_1.FbService;var PolicyService_1=require("./PolicyService");exports.PolicyService=PolicyService_1.PolicyService;var Services_1=require("./Services");exports.Services=Services_1.Services;var UserService_1=require("./UserService");exports.UserService=UserService_1.UserService;

Any ideas?

'ignore' and 'exclude' don't seem to do anything

They don't seem to be leaving any packages out of the bundle.

Here's what my serverless.yml looks like:

package:
    individually: true
    include:
        - src/**/*.js

functions:
    graphql:
        handler: src/handlers/graphql.handler
        memorySize: 512 # optional, in MB, default is 1024
        events:
            - http:
                  path: graphql
                  method: post
                  cors: true
        optimize:
            debug: true
            exclude: ['aws-sdk', 'envkey']
            ignore: ['envkey']

    ean-update:
        handler: src/handlers/ean-update.handler
        timeout: 300
        events:
            - schedule(1 day)

plugins:
    - serverless-plugin-typescript
    - serverless-offline
    - serverless-plugin-optimize

And my function dir structure on Lambda:

Any ideas?

Cannot find module

When calling my lambda i have :

"errorMessage": "Cannot find module '/var/task/handler'", "errorType": "Error", "stackTrace": [ "Function.Module._load (module.js:276:25)", "Module.require (module.js:353:17)", "require (internal/module.js:12:17)" ] }

Optimizer not firing before packaging when using 'deploy function' in Serverless v1.2.1

When trying to deploy a function using: sls deploy function -f functionName, the optimizer doesn't run until AFTER the function is packaged. This appears to be due to a change in the lifecycle events in v1.2.1 of Serverless.

I was able to fix this by updating the Serverless hooks to:

    /** Serverless hooks */
    this.hooks = {
      'after:deploy:createDeploymentArtifacts': this.afterCreateDeploymentArtifacts.bind(this),
      'before:deploy:createDeploymentArtifacts': this.beforeCreateDeploymentArtifacts.bind(this),
      'before:deploy:function:packageFunction': this.beforeCreateDeploymentArtifacts.bind(this),
      'after:deploy:function:packageFunction': this.afterCreateDeploymentArtifacts.bind(this)
    }

Add support for Azure provider

Serverless 1.8 comes with Azure provider support. While it also requires Node >= 6.5.x, Babel support is less required, but also Azure functions could take the advantage of optimized code.

Currently, the deployment fails with error:

ENOENT: no such file or directory, stat 
'/Users/juha/Projects/azure-test/_optimize/azure-test-dev-httpjs/functions/handler.js'

feat(travis): Add npm publish

This is a must have in the future to allow contributions to be published smoothly.

@jogold Currently I am in a code retreat, and I cant do npm publish. So this last release will only will be available on Nov 28th. 😅

_optimize directory permissions

I’m a newbie who’s rather baffled at a permission issue when deploying with the plug-in installed. I’m getting error: EACCES: permission denied, open ‘/var/task/_optimize…’ upon invocation. The js files are all modded to 644 prior to deploy. However when inspecting the zipped functions in .serverless directory the _optimize dir the dir permissions are set to 640. Running centos7 with NVM alias default set to Node 8.10, but have also tried with 6.10. I'm and running serverless 1.36.3.

Thoughts?

Thanks in advance.

Unexpected character error

Hello,

I'm trying to add this plugin to my codebase, but a error keeps popping up. Have you had this issue before?

Error: SyntaxError: Unexpected character '�' while parsing file: /Users/matiss/personal/property-crawler/list-crawler/node_modules/iconv/build/Release/iconv.node (line: 1, col: 0, pos: 0)

Error
    at new JS_Parse_Error (eval at <anonymous> (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:1545:18)
    at js_error (eval at <anonymous> (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:1553:11)
    at parse_error (eval at <anonymous> (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:1677:9)
    at Object.next_token [as input] (eval at <anonymous> (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:1949:9)
    at next (eval at <anonymous> (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:2078:25)
    at Object.parse (eval at <anonymous> (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:2064:15)
    at addFile (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:71:33)
    at /Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:80:17
    at Array.forEach (native)
    at Object.exports.minify (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglify-js/tools/node.js:78:26)
    at Stream.ready (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglifyify/index.js:71:19)
    at Stream.<anonymous> (/Users/matiss/personal/property-crawler/list-crawler/node_modules/uglifyify/index.js:97:12)
    at _end (/Users/matiss/personal/property-crawler/list-crawler/node_modules/through/index.js:65:9)
    at Stream.stream.end (/Users/matiss/personal/property-crawler/list-crawler/node_modules/through/index.js:74:5)
    at DestroyableTransform.onend (/Users/matiss/personal/property-crawler/list-crawler/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:495:10)
    at DestroyableTransform.g (events.js:291:16)
    at emitNone (events.js:91:20)
    at DestroyableTransform.emit (events.js:185:7)
    at endReadableNT (/Users/matiss/personal/property-crawler/list-crawler/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:865:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickDomainCallback (internal/process/next_tick.js:122:9)
    at module.exports.logError.errorHandlingError (/usr/local/lib/node_modules/serverless/lib/classes/Error.js:82:11)
    at BbPromise.resolve.then.catch.e (/usr/local/lib/node_modules/serverless/bin/serverless:20:15)
    at tryCatcher (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:510:31)
    at Promise._settlePromise (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:687:18)
    at Async._drainQueue (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:138:16)
    at Async._drainQueues (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:148:10)
    at Immediate.Async.drainQueues (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:574:20)
    at tryOnImmediate (timers.js:554:5)
    at processImmediate [as _immediateCallback] (timers.js:533:5)

Here's my package.json dependencies

{
  "dependencies": {
    "aws-sdk": "^2.7.6",
    "babel-preset-es2015": "^6.18.0",
    "crawler": "git://github.com/bda-research/node-crawler.git#1.0.0-beta",
    "q": "^1.4.1",
    "serverless-plugin-optimize": "git://github.com/FidelLimited/serverless-plugin-optimize.git#master"
  },
  "devDependencies": {}
}

__dirname references

after optimization, all references to __dirname pointing to path on local machine.

eg project structure:

/Users/test-user/project/
                                          - bin/
                                                    someFile.bin
                                          - lambda-run-this.js

using default configs of the plug-in
when deploying to lambda
the structure inside zip becomes

_optimize/project-name-lambda-run-this/lambda-run-this.js

in lambda-run-this.js when referencing __dirname + '/bin/someFile.bin' , for some reason it would return /Users/test-user/project/bin/someFile.bin , as in the path on my local machine and not the path on lambda ie /var/task/_optimize/project-name-lambda-run-this/bin/someFile.bin

somehow __dirname gets backed in during the optimisation.

BUG: feeded variables

The optimizer-plugin will never kick in if your yml looks like this

provider:
    ...
    runtime: "${self:custom.NODE_JS_RUNTIME}"
custom:
    ...
    NODE_JS_RUNTIME: nodejs6.10

The reason is that in the index.js constructor the runtime will be recieved as: ${self:custom.NODE_JS_RUNTIME}, wich causes the plugin to stop right away.

I don't know much about the whole thing tho, but I guess this can be setup in another way or?

optimize incompatible with mysql2

Using this serverless plugin is incompatible with the mysql2 node library.

The optimizer fails to load all the files from the mysql2 directory. After several hours of debugging we found the problem stemmed from the way that mysql2 handled require statements.

'auth_switch_request auth_switch_response auth_switch_request_more_data binlog_dump register_slave ssl_request handshake handshake_response query resultset_header column_definition text_row binary_row prepare_statement close_statement prepared_statement_header execute change_user'
  .split(' ')
  .forEach(function(name) {
    var ctor = require('./' + name + '.js');
    module.exports[ctor.name] = ctor;
    // monkey-patch it to include name if debug is on
    if (process.env.NODE_DEBUG) {
      if (ctor.prototype.toPacket) {
        var old = ctor.prototype.toPacket;
        ctor.prototype.toPacket = function() {
          var p = old.call(this);
          p._name = ctor.name;
          return p;
        };
      }
    }
  });

As you can see the require statements are being pieced together inside this for loop and optimize does not pick them up, leaving out the files. When we replaced this code with all the requires and exports written out, the function was optimized fine.

var temp = require('./auth_switch_request.js');
module.exports[temp.name] = temp;
temp = require('./auth_switch_response.js');
module.exports[temp.name] = temp;
temp = require('./auth_switch_request_more_data.js');
module.exports[temp.name] = temp;
temp = require('./register_slave.js');
module.exports[temp.name] = temp;
temp = require('./ssl_request.js');
module.exports[temp.name] = temp;
...

This isn't really a bug with the optimizer necessarily but I wanted to post this here in case anyone else had a similar problem where files were not being included by the optimizer.

If functionality could be implemented to see this type for require inside a loop, that would be very helpful though. :)

Pathing with subdir not working as expected: Unable to import module

Hi,

I'm having trouble using this plugin when it comes to what it does with pathing when it optimizes the code I think.
Using serverless v1.37.0 and serverless-plugin-optimize v4.0.2-rc.1

My code is structured like this:

- serverless.yml
- package.json
- functions/
    - app.js <- handler
    - gdl/
        - a.js
        - b.js
        - c.js

And in my serverless.yaml-file (the relevant sections) looks like this:

package:
  individually: true
  exclude:
    - ./**
    - '!node_modules/**'

functions:
  assistant:
    handler: functions/app.handler

Im getting the following error when I try to trigger the app after deploying it to aws:

Unable to import module '_optimize/assistant-service-test-assistant/functions/app'

I have debugged a bit and if I do a console.log on the path inside app.js it prints the following

app path /var/task/_optimize/assistant-service-test-assistant/functions/functions/app.js

As you can see, it have an extra /functions in the path, so it makes sense that it can't find module.. Someone that can help me with this issue?

Just for fun - If I put the app.js in the gdl-folder I get

app path /var/task/_optimize/assistant-service-test-assistant/functions/gdl/functions/gdl/app.js

which suggests that something strange is happening when the app is optimized.

The debug output:

Serverless: Optimize: debug {
  "functions": [
    {
      "bundle": "<userDir>/dev/gdl/assistant-service/_optimize/assistant-service-test-assistant/functions/app.js",
      "handlerOriginal": "functions/app.handler",
      "handlerOptimize": "_optimize/assistant-service-test-assistant/functions/app.handler",
      "package": {
        "exclude": [
          "**"
        ],
        "include": [
          "_optimize/assistant-service-test-assistant/**"
        ]
      }
    }
  ],
  "options": {
    "debug": true,
    "exclude": [
      "aws-sdk"
    ],
    "external": [],
    "externalPaths": {},
    "extensions": [],
    "global": false,
    "includePaths": [],
    "ignore": [],
    "minify": true,
    "plugins": [],
    "prefix": "_optimize",
    "presets": [
      [
        "<userDir>/dev/gdl/assistant-service/node_modules/@babel/preset-env/lib/index.js",
        {
          "targets": {
            "node": "8.10"
          }
        }
      ]
    ],
    "individually": true
  }
}

Handler path not being rewritten

The handler path when I look inside the aws console is wrong: handler.app (https://d.pr/i/1lwDqI)

I needs to be _optimize/arena-node-api-dev-app/handler.app for it to work.

I have debug: true and this is my output:

Rodrigos-MacBook-Pro:arena-node-api rodrigoreis$ sls1 deploy -f app
Serverless: Optimize: starting engines
Serverless: Optimize: arena-node-api-dev-app
Serverless: Packaging function: app...
Serverless: Excluding development dependencies...
Serverless: Optimize: debug {
  "functions": [
    {
      "bundle": "/Users/rodrigoreis/Documents/arena-api/arena-node-api/_optimize/arena-node-api-dev-app/handler.js",
      "handlerOriginal": "handler.app",
      "handlerOptimize": "_optimize/arena-node-api-dev-app/handler.app",
      "package": {
        "exclude": [
          "**"
        ],
        "include": [
          "_optimize/arena-node-api-dev-app/**"
        ]
      }
    }
  ],
  "options": {
    "debug": true,
    "exclude": [
      "aws-sdk"
    ],
    "external": [
      "mongoose"
    ],
    "externalPaths": {},
    "extensions": [],
    "global": false,
    "includePaths": [],
    "ignore": [],
    "minify": true,
    "plugins": [],
    "prefix": "_optimize",
    "presets": [
      [
        "/Users/rodrigoreis/Documents/arena-api/arena-node-api/node_modules/babel-preset-env/lib/index.js",
        {
          "targets": {
            "node": "8.10"
          }
        }
      ]
    ],
    "individually": true
  }
}

This is my serverless.yml:

service: arena-node-api

# You can pin your service to only deploy with a specific Serverless version
# Check out our docs for more details
# frameworkVersion: "=X.X.X"

custom:
  stage: "${opt:stage, self:provider.stage}"
  optimize:
    debug: true
    external: ['mongoose']

provider:
  name: aws
  stage: dev
  runtime: nodejs8.10
  profile: default
  region: us-west-2
  memorySize: 256
  vpc:
    securityGroupIds:
      - sg-7268ed14
    subnetIds:
      - subnet-28db1d70
      - subnet-44369a0f
      - subnet-f411938d
  environment:
      env: ${self:custom.stage}

# you can add packaging information here
package:
  individually: true

functions:
  app:
    handler: handler.app
    events:
      - http: ANY /
      - http: 'ANY {proxy+}'
    vpc:
      securityGroupIds:
        - sg-7268ed14
      subnetIds:
        - subnet-28db1d70
        - subnet-44369a0f
        - subnet-f411938d
  billing:
      handler: handler.billing
      events:
        - http:
            path: billing
            method: ANY
            cors:
              origin: '*'
              headers:
                - Content-Type
                - X-Amz-Date
                - Authorization
                - X-Api-Key
                - X-Amz-Security-Token
                - X-Amz-User-Agent
                - Preferred-Language
        - http:
            path: billing/{proxy+}
            method: ANY
            cors:
              origin: '*'
              headers:
                - Content-Type
                - X-Amz-Date
                - Authorization
                - X-Api-Key
                - X-Amz-Security-Token
                - X-Amz-User-Agent
                - Preferred-Language
      vpc:
        securityGroupIds:
          - sg-7268ed14
        subnetIds:
          - subnet-28db1d70
          - subnet-44369a0f
          - subnet-f411938d
  live-score-update:
    handler: handler_live_score_update.handler

plugins:
  - serverless-plugin-optimize

fix(package): Breaking change in function package in 1.2.x

Fix

Function optimization serverless deploy function -f myfunction stopped working in commit serverless/serverless@1fd2509.

Implement

Change hooks from deploy:function:deploy to deploy:function:initialize.

enigmatic-smile / serverless-plugin-optimize Goto Github PK

serverless-plugin-optimize's People

Contributors

Stargazers

Watchers

Forkers

serverless-plugin-optimize's Issues

Feature

Implement

Fixes

Feature

Implement

Fix

Implement

Recommend Projects

Recommend Topics

Recommend Org