firstandthird / hapi-agenda Goto Github PK

JavaScript 100.00%

hapi-agenda's Introduction

hapi-agenda

Installation

npm install hapi-agenda --save

Usage

server.register({
  register: require('hapi-agenda'),
  options: {
    mongoUrl: 'localhost:27017/hapi-agenda',
    jobs: __dirname + '/jobs', // path to directory containing job files
    processEvery: '5 seconds', // Defaults to 30 seconds. Lower numbers = higher db calls
    every: { // Runs these jobs
      'say-hello': '10 seconds',
      'here is a task': '30 seconds'
    }
  }
}, function(err) {
  // start server or other stuff
});

Job File:

Job name based on filename

hello-world.js

module.exports = function(data, done) {
  console.log('Hello world');
  done();
};

Job with options and custom name

customjob.js

module.exports = {
  name: 'here is a task',
  job: function(data, done) {
    console.log('hostname', this.info.host);
    done();
  },
  concurrency: 10,
  priority: 'high'
};

hapi-agenda's People

Contributors

Stargazers

Watchers

Forkers

eluttine orion110217 ubiatar

hapi-agenda's Issues

Cannot read property 'name' of undefined

this occurs in mongoskin version included
node_modules\mongoskin\lib\utils.js
line 33

  var skinClassName = 'Skin' + NativeClass.name; <----here
  function SkinClass() {
    var args = __slice.call(arguments);
    this._construct_args = args;
    if(useNativeConstructor && arguments.length > 0) {
      args.unshift(null);

queue log needs name of job and data that was passed in

expose agenda object

create hapi plugin that exposes agenda object. We should pass in mongodb url.

log a queue event when a job is called

I think the best way to handle this is to just put an anon function in the define and have the first thing be the log call and then call the method here: https://github.com/firstandthird/hapi-agenda/blob/master/index.js#L51

More Robust Dashboard that could be hooked into hapi-dashboard

Convert the current json api trigger, or write another parallel handler that would provide administrative or at least html views of the jobs instead of a strict API. You could enable, disable, delete jobs that we're initialized at startup.

JSON api

What about adding an option to expose a json api? should be in this module or separate? @dawnerd ?

limit data for log

When events go into something like good-mongodb, it errors out because we are logging the job class instance and it returns a max call stack size exceeded. Looks like we could just pass back the attrs

log { event: 'log',
  timestamp: 1418862364737,
  tags: [ 'agenda', 'complete' ],
  data:
   { agenda:
      { _name: undefined,
        _processEvery: 30000,
        _defaultConcurrency: 5,
        _maxConcurrency: 20,
        _definitions: [Object],
        _runningJobs: [],
        _jobQueue: [],
        _defaultLockLifetime: 600000,
        _db: [Object],
        _events: [Object],
        _processInterval: [Object],
        _nextScanAt: Wed Dec 17 2014 16:26:26 GMT-0800 (PST) },
     attrs:
      { name: 'email-error-digest',
        data: {},
        type: 'normal',
        priority: 0,
        nextRunAt: null,
        _id: 54921f1cef00780cf583b780,
        lastRunAt: Wed Dec 17 2014 16:26:04 GMT-0800 (PST),
        lastFinishedAt: Wed Dec 17 2014 16:26:04 GMT-0800 (PST),
        lockedAt: null },
     _scheduled: true },
  pid: 62732 }
[RangeError: Maximum call stack size exceeded]

pass in directory of tasks

loop over directory and define a task for each file.
file can either return function or object
if object, use name to name task, job as fn and pass other options as options in the define call
Use the name of the file as the task name unless the module returns an object and the name is defined in there.

Object - some-task.js - new job defined as 'here is a task'

module.exports = {
  name: 'here is a task',
  job: function(data, done) {
  },
  concurrency: 10
}

Fn - some-task.js - new job defined as 'some-task'

module.exports = function(data, done) {
}

any other things we should consider?

pass in option to auto run every

want to be able to pass in an object to the plugin that will auto run agenda.every. It will be used to auto set up maintenance tasks when the server starts up. Could look something like this:

{ 
  plugin: require('haps-agenda'), 
  options: {
    every: {
     '1 hour': 'some job name'
    }
  }
}

Goals

process long running jobs
queue up after reaching a max concurrent jobs
be able to spin up multiple workers
get progress of job
jobs can log process
mongodb backed
be able to retry failed jobs
if app crashes, restart jobs in progress
future jobs - example: run a job at midnight, or run in 24 hours from some event
json API - maybe we just do this in a hapi plugin that uses this lib
job timeouts - job is marked fail after X seconds