ajmath / serverless-offline-scheduler Goto Github PK

View Code? Open in Web Editor NEW

97.0 5.0 40.0 886 KB

License: MIT License

JavaScript 100.00%

serverless-offline-scheduler's Introduction

THIS PROJECT IS NOT MAINTAINED

serverless-offline-scheduler

Integrates with serverless-offline to run scheduled lambdas locally. Can also be run independently

Usage

Install module npm i --save-dev serverless-offline-scheduler
Update serverless.yml

plugins:
  - serverless-offline-scheduler

Your scheduled functions with cron syntax can be run with either
- sls schedule
- sls offline start

serverless-offline-scheduler's People

Contributors

Stargazers

Watchers

serverless-offline-scheduler's Issues

Having checkExecSyncError due to not loading Options

I am using serverless-offline-scheduler 0.3.8 in my project.
I encounted an issue if I set Referencing CLI Options in my project'sserverless.yml file using ${opt:some_option}.
like https://serverless.com/framework/docs/providers/aws/guide/variables/#referencing-cli-options
The error message show as below

//---------
Error: Command failed: serverless invoke local --function grower-updates
at checkExecSyncError (child_process.js:621:11)
at Object.execSync (child_process.js:657:15)
at Scheduler._executeFunction (/XXX/node_modules/serverless-offline-scheduler/lib/scheduler.js:87:25)
at Job.job (/XXX/node_modules/serverless-offline-scheduler/lib/scheduler.js:63:29)
at Job.invoke (/XXX/node_modules/node-schedule/lib/schedule.js:173:10)
at /XXX/node_modules/node-schedule/lib/schedule.js:552:11
at Timeout._onTimeout (/XXX/node_modules/node-schedule/lib/schedule.js:510:7)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timers.js:475:7)
//-------

The root cause is when function _executeFunction is executed, the "serverless invoke local" command does not pick up Options variables I set in serverless.yml, also the value of option "--data" is undefined if I does not set any input in scheduler.
I have the PR to solve this issue.
#32.

TypeError: scheduleEvent.replace is not a function

config.yml

cron:
  handler:  cron
  events:
    - schedule:
        name: schedule-cron
        description: "Trigger the every Monday at 08:00 UTC"
        rate: cron(0 8 ? * MON *)

Throws an error

Type Error ----------------------------------------------

TypeError: scheduleEvent.replace is not a function
at Scheduler._convertExpressionToCron (J:\project\node_modules\serverless-offline-scheduler\lib\scheduler.js:177:8)

_convertExpressionToCron(scheduleEvent) {
    const params = scheduleEvent
      .replace("rate(", "")
      .replace("cron(", "")
      .replace(")", "");

    if (scheduleEvent.startsWith("cron(")) {
      return this._convertCronSyntax(params);
    }
    if (scheduleEvent.startsWith("rate(")) {
      return this._convertRateToCron(params);
    }

    this.serverless.cli.log("scheduler: invalid, schedule syntax");
    return null;
  }

_convertExpressionToCron function received scheduleEvent parameter as an array.
[ 'cron(0 8 ? * MON *)' ]

So replace function doesn't work on the array value. This has to be type-checked, iterated, and replace login should be applied.

function replace all previous output of webpack

Good day! If function invokes, it replaces all output of webpack. In this case, I can't use anything from serverless offline. Could somebody provide me information, how I can configure webpack to solve this issue?

stage is ignored

I'm running the scheduler with
sls schedule --stage local
but get an error
Error: Command failed: serverless invoke local --function SchedulerFunction --data {"schedule":"every-minute"}
and before that some errors about it not being able to populate a few variables.
However, those variables shouldn't matter in the local stage.
When I manually run
serverless invoke local --function SchedulerFunction --data {"schedule":"every-minute"}
I get the same error, however if I run
serverless invoke local --stage local --function SchedulerFunction --data {"schedule":"every-minute"}
i.e. with my custom stage passed through, the function runs fine.

http request to localhost is hanging

Trying to send websocket messages to connected clients by making a call to http://localhost:3001/@connections/{connectionId} however all http calls to localhost:3000 or localhost:3001 are hanging. Works if the lambda function is invoked directly, without serverless-offline-scheduler.

Can not see cron running

Hi,

I have a service with Cron event.
I try to add the plugin to the service.
When I run serverless offline I see the log
Serverless: scheduler: scheduling ScheduleTasksExecute/ScheduleTasksExecute with 0/1 * * * ?
But I do not see any logs from the function.
How can I know that the function is running?

Thank you

Python support?

Ideally it would be possible to invoke python handlers

Rulename is the same even for named schedules

Currently the lambda event that's triggered always has "resources":["arn:aws:events:serverless-offline:123456789012:rule/my-schedule"] even when the schedule is named explicitly in the serverless definition.

I'd expect the rule to be named as well to better reflect the real AWS use case.

Recompiling on each execution

We are using serverless-plugin-typescript plugin and we have a Lambda function running once a minute. Each time the function executes, it looks like serverless.yml file is being parsed (since I see my typical warnings about serverless.yml file on each execution) and TypeScript is being recompiled causing some serious delays and resulting with high CPU usage most of the time - by the time one execution completes compiling and executing, the next execution needs to run.

I'm not sure if Serverless framework allows for that but ideally the plugin would reuse already parsed serverless.yml configuration from when severless offline was executed and it would reuse already compiled TypeScript files. This might be related to #34.

Unexpected token export

does it have issue with es6/7 ?

`export const job = async (event, context, callback) => {
^^^^^^

SyntaxError: Unexpected token export
at createScript (vm.js:80:10)`

TypeError when using [email protected]

My Setup

$ serverless --version
Framework Core: 1.60.4
Plugin: 3.2.6
SDK: 2.2.1
Components Core: 1.1.2
Components CLI: 1.4.0

[email protected]
[email protected]

Issue

A TypeError is raised whenever I try serverless offline start:

  Type Error ---------------------------------------------
 
  TypeError: Cannot read property 'location' of undefined
      at Scheduler.run (/Users/cjuega/Developments/mediasuite/mediastream-exporter/node_modules/serverless-offline-scheduler/lib/scheduler.js:43:45)
      at Object.before:offline:start:init [as hook] (/Users/cjuega/Developments/mediasuite/mediastream-exporter/node_modules/serverless-offline-scheduler/index.js:20:57)
      at BbPromise.reduce (/Users/cjuega/n/lib/node_modules/serverless/lib/classes/PluginManager.js:490:55)
      at tryCatcher (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Object.gotValue (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/reduce.js:168:18)
      at Object.gotAccum (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/reduce.js:155:25)
      at Object.tryCatcher (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/Users/cjuega/n/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:15:14)
      at runCallback (timers.js:705:18)
      at tryOnImmediate (timers.js:676:5)
      at processImmediate (timers.js:658:5)
      at process.topLevelDomainCallback (domain.js:120:23)

I did try --location option with similar result.

I had a look at the source code. It looks options property is not being returned when calling this.serverless.pluginManager.getPlugins(). Having a look at serverless-offline's source code too, options was renamed to _options in v6.0.0-alpha.2; and completely removed at v6.0.0-alpha.51.

It looks like serverless-offline no longer uses a location option. Shall it be added to serverless-offline-schedule?

Not working since serverless-offline v5.6.1

I am using this package in conjunction with serverless-webpack. The issue is:

Serverless: scheduler: unable to find source for aaa
Serverless: scheduler: unable to find source for zzz
Serverless: scheduler: unable to find source for yyy
Serverless: scheduler: unable to find source for xxx

I was originally using the solution found in a similar issue #11 and this was working for me. Since updating to [email protected] the solution of setting the --location is no longer working. I'm not sure if this is a serverless-offline, serverless-webpack, or serverless-offline-scheduler issue, but it is only happening since upgrading serverless-offline to v5.6.1 or later.

Duplicate config error

When running sls --config serverless.local.yml offline start locally, I get

Got 2 config paths: [serverless.local.yml, serverless.local.yml].Expected single value

This line https://github.com/ajmath/serverless-offline-scheduler/blob/master/lib/scheduler.js#L101 gives me

{"c":"serverless.local.yml","config":"serverless.local.yml"}

I added a hack locally to make it work

if (opts.config && opts.c && opts.config === opts.c) {
      delete opts.c;
}

Serverless version 1.80.0

Webpack-transpiled code isn't loaded

When using with serverless-webpack v4.1.0, serverless-offline-scheduler v0.3.3 doesn't load the webpack-transpiled code.

Could be related to #8

The Error

When running with $ sls offline start:

/Users/xxx/Code/xxx/my-service/xxx/create.js:1
(function (exports, require, module, __filename, __dirname) { import insertToDb from '../main/insertToDb';
                                                              ^^^^^^
SyntaxError: Unexpected token import

Workaround

Monkey patching scheduler.js#L69-L71 fixed my issue.

Original (version 0.3.3)

Currently, serverless-offline-scheduler loads from /Users/xxx/Code/xxx/my-service/xxx/create.js

serverless-offline-scheduler/lib/scheduler.js

Lines 69 to 71 in 9186272

 const funcPath = path.join( 

 this.serverless.config.servicePath, 

 this.location || "", filename);

Hardcoding the webpack output directory

Forcing serverless-offline-scheduler to load from webpack output directory. In my case: /Users/xxx/Code/xxx/my-service/xxx/.webpack/service/create.js

    const funcPath = path.join(
      this.serverless.config.servicePath,
      '.webpack',
      'service',
      this.location || "", filename);

Is it a bug? Or, have I missed anything?
Thanks!

Skip executions after sleep

We have a Lambda function running on a schedule every minute. Putting my computer to sleep and waking it up after a few hours results with the Lambda function being continuously executed hundreds of times without any break causing my machine to be nearly unusable with high CPU and network usage. Ideally, in situations like this it would skip all these executions except for the last one.

Cannot read property 'rate' of null

Followed installation example, added scheduled event to one function in sls yml, cannot start offline or run sls schedule without getting Cannot read property 'rate' of null error

deprecation warning: Include serverless CLI option definitions

I get this warning when I run serverless schedule

Serverless: Deprecation warning: CLI options definitions were upgraded with "type" property 
(which could be one of "string", "boolean", "multiple"). Below listed plugins do not predefine 
type for introduced options:
             - ServerlessOfflineScheduler for "runSchedulesOnInit"
            Please report this issue in plugin issue tracker.
            Starting with next major release, this will be communicated with a thrown error.
            More Info: https://www.serverless.com/framework/docs/deprecations/#CLI_OPTIONS_SCHEMA

It is important that this be checked as Starting with next major release, this will be communicated with a thrown error.

I may raise a PR for this

Not working with sns

When I used serverless-offline-scheduler I couldn't get a response from sns.publish().promise() it's just waiting and nothing did, but when I changed function event to http it worked normally. It's kinda strange, isn't it?

Support serverless-webpack 2.1.0

After upgrading to [email protected], all of my scheduled jobs stopped running with the following error:

Serverless: scheduler: unable to find source for <JOB_NAME>

Not compatible with serverless-webpack

I believe this is caused by serverless-heaven/serverless-webpack#511 but I wanted to report it here as it also affects the core functionality of this package.

Serverless: scheduler: scheduling payments/payments with */1 * * * *
Serverless: payments
...
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Invoke webpack:validate
Serverless: WARNING: More than one matching handlers found for 'src/jobs/payments'. Using 'src/jobs/payments.js'.

  Serverless Error ---------------------------------------

  The webpack plugin could not find the configuration file at: /app/.webpack/payments/webpack.config.js

  Stack Trace --------------------------------------------

ServerlessError: The webpack plugin could not find the configuration file at: /app/.webpack/payments/webpack.config.js
    at ServerlessWebpack.validate (/app/node_modules/serverless-webpack/lib/validate.js:111:33)
From previous event:
    at Object.webpack:validate:validate [as hook] (/app/node_modules/serverless-webpack/index.js:157:10)
    at BbPromise.reduce (/app/node_modules/serverless/lib/classes/PluginManager.js:464:55)
From previous event:
    at PluginManager.invoke (/app/node_modules/serverless/lib/classes/PluginManager.js:464:22)
    at PluginManager.spawn (/app/node_modules/serverless/lib/classes/PluginManager.js:484:17)
    at ServerlessWebpack.BbPromise.bind.then (/app/node_modules/serverless-webpack/index.js:100:51)
From previous event:
    at Object.before:package:createDeploymentArtifacts [as hook] (/app/node_modules/serverless-webpack/index.js:100:10)
    at BbPromise.reduce (/app/node_modules/serverless/lib/classes/PluginManager.js:464:55)
From previous event:
    at PluginManager.invoke (/app/node_modules/serverless/lib/classes/PluginManager.js:464:22)
    at PluginManager.spawn (/app/node_modules/serverless/lib/classes/PluginManager.js:484:17)
    at AwsInvokeLocal.invokeLocalDocker (/app/node_modules/serverless/lib/plugins/aws/invokeLocal/index.js:357:42)
    at AwsInvokeLocal.invokeLocal (/app/node_modules/serverless/lib/plugins/aws/invokeLocal/index.js:202:17)
From previous event:
    at Object.invoke:local:invoke [as hook] (/app/node_modules/serverless/lib/plugins/aws/invokeLocal/index.js:34:57)
    at BbPromise.reduce (/app/node_modules/serverless/lib/classes/PluginManager.js:464:55)
From previous event:
    at PluginManager.invoke (/app/node_modules/serverless/lib/classes/PluginManager.js:464:22)
    at PluginManager.run (/app/node_modules/serverless/lib/classes/PluginManager.js:496:17)
    at variables.populateService.then (/app/node_modules/serverless/lib/Serverless.js:116:33)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
    at process.topLevelDomainCallback (domain.js:126:23)
From previous event:
    at Serverless.run (/app/node_modules/serverless/lib/Serverless.js:103:74)
    at serverless.init.then (/app/node_modules/serverless/bin/serverless.js:44:32)
    at /app/node_modules/graceful-fs/graceful-fs.js:111:16
    at /app/node_modules/graceful-fs/graceful-fs.js:45:10
    at FSReqWrap.args [as oncomplete] (fs.js:140:20)
From previous event:
    at initializeErrorReporter.then (/app/node_modules/serverless/bin/serverless.js:44:10)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
    at process.topLevelDomainCallback (domain.js:126:23)
From previous event:
    at /app/node_modules/serverless/bin/serverless.js:30:6
    at Object.<anonymous> (/app/node_modules/serverless/bin/serverless.js:71:7)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     OS:                     linux
     Node Version:           10.16.0
     Serverless Version:     1.47.0

Error: Command failed: serverless invoke local --function payments --data undefined
    at checkExecSyncError (child_process.js:629:11)
    at Object.execSync (child_process.js:666:13)
    at Scheduler._executeFunction (/app/node_modules/serverless-offline-scheduler/lib/scheduler.js:76:25)
    at Job.run.schedule.scheduleJob [as job] (/app/node_modules/serverless-offline-scheduler/lib/scheduler.js:63:29)
    at Job.invoke (/app/node_modules/node-schedule/lib/schedule.js:173:10)
    at /app/node_modules/node-schedule/lib/schedule.js:552:11
    at Timeout._onTimeout (/app/node_modules/node-schedule/lib/schedule.js:510:7)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
error Command failed with exit code 1.

Nothing happening

I'm trying out this plugin via:

λ → sls schedule
Serverless: scheduler: scheduling schedule-executors/schedule-executors with */1 * * * *
Serverless: schedule-executors

and

λ → serverless offline

... but I'm not seeing anything happening.

Functionn is defined as:

functions:
  schedule-executors:
    handler: src/functions/scheduleExecutors/index.handler
    events:
      - schedule:
          name: your-scheduled-rate-event-name
          description: 'your scheduled rate event description'
          rate: rate(1 minute)

Hung on first run

Hey there, after either running

serverless offline starat

serverless offline start --runSchedulesOnInit

The serverless offline hungs and upon force exit it returns an error:

Command failed: /usr/bin/node /.../node_modules/.bin/serverless invoke local 
--function cron_ocr_process_pending_submission
--data {"account":"********","region":"serverless-offline","detail":{},"detail-type":"Scheduled 
Event","source":"aws.events","time":"2020-10-28T12:17:08.617Z","id":"some-id","resources":["arn:aws:events:serverless-
offline:1********:rule/cron_ocr_process_pending_submission"],"isOffline":true}

Any idea how to fix this or what is the issue behind it?

Details:

Serverless offline version: ^3.33.0
serverless-offline-scheduler": ^0.4.2

serverless.yml plugins:

plugins:
  - serverless-offline-scheduler
  - serverless-plugin-split-stacks
  - serverless-plugin-browserify
  - serverless-offline
  - serverless-domain-manager
  - serverless-plugin-custom-roles

Does not work with latest serverless-offline - v6.1.4

When updating dependencies to serverless-offline 6.1.4, it doesn't start, the error message is:

Type Error ---------------------------------------------

  TypeError: Cannot read property 'location' of undefined
      at Scheduler.run (/Users/carlos/Cheftonic-tech/analytics-backend/node_modules/serverless-offline-scheduler/lib/scheduler.js:63:45)
      at Object.before:offline:start:init [as hook] (/Users/carlos/Cheftonic-tech/analytics-backend/node_modules/serverless-offline-scheduler/index.js:26:61)
      at /Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/lib/classes/PluginManager.js:490:55
      at tryCatcher (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Object.gotValue (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/reduce.js:168:18)
      at Object.gotAccum (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/reduce.js:155:25)
      at Object.tryCatcher (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/Users/carlos/.nvm/versions/node/v12.13.1/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:439:21)
      at process.topLevelDomainCallback (domain.js:130:23)

	const funcPath = path.join(
	this.serverless.config.servicePath,
	this.location \|\| "", filename);