Giter Club home page Giter Club logo

operative's Introduction

Operative

Before reading this please ensure you fully understand the concept of Web Workers.

Operative is a small JS utility (~1.8k gzipped) for seamlessly creating Web Worker scripts. Its features include:

  • Seamless API Authoring
  • Producing debuggable Worker Blobs
  • Providing console interface for simple logging
  • Degrading where Worker/Blob support is lacking

Why Operative?

Utilising unabstracted Workers can be cumbersome and awkward. Having to design and implement message-passing contracts and having to setup event listeners yourself is time-consuming and error-prone. Operative takes care of this stuff so you can focus on your code.

Before you get excited:

Even with Operative you are still subject to the constraints of Web Workers, i.e.

  • No DOM/BOM Access.
  • No synchronous communication with parent page.
  • An entirely separate execution context (no shared scope/variables).
  • Limitations on data transfer depending on browser.

And it won't make things uniformly faster or less burdensome on the UI. Operative will fall-back to using iframes in older browsers, which gives you no non-blocking advantage.

Non-blob worker support (i.e. for IE10) requires that you have a same-origin copy of Operative (this means you can't solely rely on CDNs or elsewhere-hosted scripts if you want to support IE10).

Browser Support

Operative 0.4.4 has been explicitly tested in:

  • Chrome 14, 23, 29, 37, 42
  • Firefox 3, 10, 18, 23, 32
  • IE 8, 9, 10 (Windows 7)
  • IE 11 (Windows 10)
  • Opera 25 (Mac)
  • Opera 10.6 (Windows 7)
  • Safari 5.1 (Windows 7)
  • Safari 6, 8 (Mac)
  • Safari (iPad Air, iOS 8)
  • Safari (iPhone 4S, iOS 5.1)

Support for Workers, with varying degrees of support for Transferables and Blobs:

  • FF 17+
  • Chrome 7+
  • Safari 4+
  • Opera 11+
  • IE10+

Note: Operative has not been tested in non-browser envs

Quick install

# bower
bower install operative
# or npm
npm install operative

Or just grab the built JS file from dist/, also available here (0.4.4):

Creating an Operative Module

An Operative module is defined as an object containing properties/methods:

var calculator = operative({
	add: function(a, b, callback) {
		callback( a + b );
	}
});

This would expose an asynchronous API:

calculator.add(1, 2, function(result) {
	result; // => 3
});

The add() function will run within a worker. The value it returns is handled by operative and forwarded, asynchronously to your callback function in the parent page.

Notice that the exposed add method requires its last argument to be a callback. The last argument passed to an operative method must always be a callback. All preceding arguments are passed into the worker itself.

NOTE: It's important to note that the Operative code is not executed in-place. *It's executed within a Worker. You won't be able to access variables surrounding the definition of your operative:

// THIS WILL NOT WORK!

var something = 123;

var myWorker = operative({
	doStuff: function() {
		something += 456;
	}
});

(the something variable won't exist within the Worker)

Instead you can do:

var myWorker = operative({
	something: 123,
	doStuff: function() {
		this.something += 456;
	}
});

Need to iterate 10,000,000,000 times? No problem!

var craziness = operative({

	doCrazy: function(cb) {

		console.time('Craziness');
		for (var i = 0; i < 10000000000; ++i);
		console.timeEnd('Craziness');

		cb('I am done!');
	}

});

craziness.doCrazy(function(result) {
	// Console outputs: Craziness: 14806.419ms
	result; // => "I am done!"
});

Degraded Operative

Operative degrades in this order:

(higher is better/cooler)

  • Full Worker via Blob & Structured-Cloning (Ch13+, FF8+, IE11+, Op11.5+, Sf5.1+)
  • Full Worker via Eval & Structured-Cloning (IE10)
  • Full Worker via Blob & JSON marshalling (???)
  • Full Worker via Eval & JSON marshalling (Sf4)
  • No Worker: Regular JS called via iframe (older browsers)

Operative will degrade in environments with no Worker or Blob support. In such a case the code would execute as regular in-place JavaScript. The calls will still be asynchronous though, not immediate.

If you are looking to support this fully degraded state (honestly, only do it if you have to) then you'll also need to avoid utilising Worker-specific APIs like importScripts.

No Worker-Via-Blob Support

Operative supports browsers with no worker-via-blob support (e.g. IE10, Safari 4.0) via eval, and it requires operative.js or operative.min.js to be its own file and included in the page via a <script> tag. This file must be on the same origin as the parent page.

If you're bundling Operative with other JS, you'll have to have an additional (same-origin!) operative.js and specify it before creating an operative module via operative.setSelfURL('path/to/operative.js') (this'll only generate a request where the aforementioned support is lacking). Due to the usage of eval in these cases it is recommended to debug your operatives in more capable browsers.

Operative API Documentation

  • {Function} operative: A global function which creates a new Operative module with the passed methods/properties. Note: Non-function properties must be basic objects that can be passed to JSON.stringify.
  • Pass an object of methods, e.g. operative({ method: function() {...} ... });
  • Or pass a single function, e.g. operative(function() {}) (in which case a single function is returned)
  • Either signature allows you to pass dependencies as a second param: operative(..., ['dep1.js', 'dep2.js']).
  • {Boolean} operative.hasWorkerSupport: A boolean indicating whether both Blob and Worker support is detected.
  • {Function} operative.setSelfURL: Allows you to set the URL of the operative script. Use this if you want IE10 & Safari 4/5 support and you're not including operative by the conventional <script src="operative.js"></script>.
  • {Function} operative.setBaseURL: Allows you to set the URL that should be prepended to relative dependency URLs.

Creating an Operative:

To create an operative module pass an object of methods/properties:

var myOperative = operative({
	doX: function(a, b, c, callback) {
		// ...
	},
	doY: function(a, b, c, callback) {
		// ...
	}
});

Or a single function to create a singular operative:

var myOperative = operative(function(a, b, c, callback) {
	// Send the result to the parent page:
	callback(...);
});

// Example call:
myOperative(1, 2, 3, function() { /*callback*/ });

Returning results

The most simple way to use operative is to pass in a callback function when calling an operative function and within the operative method call the callback with your result:

var combine = operative(function(foo, bar, callback) {
	callback(foo + bar);
});

combine('foo', 'bar', function() {
	// This callback function will be called with
	// the result from your operative function.
	result; // => 'foobar'
});

Return via Promises

If you don't pass a callback when calling an operative method, operative will assume you want a Promise. Note that operative will reference operative.Promise and will expect it to be a native Promise implementation or compliant polyfill. Operative does not come bundled with a Promise implementation.

var combine = operative(function(foo, bar) {
	// Internally, use a Deferred:
	var deferred = this.deferred();

	// A deferred has two methods: fulfill & reject:

	if (foo !== 'foo') {
		// Error (Rejection)
		deferred.reject('foo should be "foo"!');
	} else {
		// Success (Filfillment)
		deferred.fulfill(foo + bar);
	}
});

// Usage externally:
var promise = combine('foo', 'bar');

promise.then(function(value) {
	// Fulfilled
}, function(err) {
	// Rejected
});

NOTE: Operative will only give you a promise if you don't pass a callback and if operative.Promise is defined. By default operative.Promise will reference window.Promise (native implementation if it exists).

Declaring dependencies

Operative accepts a second argument, an array of JS files to load within the worker ( or in its degraded state, an Iframe ):

// Create interface to call lodash methods inside a worker:
var lodashWorker = operative(function(method, args, cb) {
	cb(
		_[method].apply(_, args)
	);
}, [
	'http://cdnjs.cloudflare.com/ajax/libs/lodash.js/1.3.1/lodash.min.js'
]);

lodashWorker('uniq', [[1, 2, 3, 3, 2, 1, 4, 3, 2]], function(output) {
	output; // => [1, 2, 3, 4]
});

Declared dependencies will be loaded before any of your operative's methods are called. Even if you call one from the outside, that call will be queued until the context (Worker/iFrame) completes loading the dependencies.

Note: Each dependency, if not an absolute URL, will be loaded relative to the calculated base URL, which operative determines like so:

var baseURL = (
	location.protocol + '//' +
	location.hostname +
	(location.port?':'+location.port:'') +
	location.pathname
).replace(/[^\/]+$/, '');

To override at runtime use:

operative.setBaseURL('http://some.com/new/absolute/base/');
// Ensure it ends in a '/'

// To retrieve the current Base URL:
operative.getBaseURL();

Destroying an operative

To terminate the operative (and thus its worker/iframe):

o.terminate();

(terminate is aliased to the now-deprecated destroy)

Testing & Building

Special thanks to BrowserStack for providing free testing!

$ # grab dependencies
$ npm install

$ # install grunt globally if you don't have it...
$ npm install -g grunt-cli

$ # test
$ grunt test

$ # do everything + build dist:
$ grunt

Changelog

  • 0.4.6 (19 Jan 2017)
  • Fix uncloneable native Error obj issue (see #44 & #45)
  • 0.4.5
  • Fix error Uncaught ReferenceError: hasTransferSupport is not defined (see #43)
  • 0.4.4 (27 Apr 2015)
  • Reverted to a global variable to fix undefined errors in bundles
  • 0.4.3 (26 Apr 2015)
  • Fixed self-url setting (see #36)
  • Improved readme
  • 0.4.2 (25 Apr 2015)
  • Added support for CommonJS
  • 0.4.0 (10 Apr 2015)
  • Removed deprecated async() method in favor of callbacks or promises
  • Refactor / Restructure code to make maintenance a bit easier
  • Use mocha_phantomjs to setup mocha testing via grunt
  • 0.4.0-rc1
  • Refactor test suites (use mocha instead of jasmine and fix various flakey specs).
  • Deprecate deferred.fulfil() (worker context promise API) in favour of deferred.resolve() (alias for fulfil still exists).
  • Introduce Transfers API (#23).
  • Fix #18.
  • Retain callbacks (allowing them to be called again and again -- a la Events). See #15.
  • Introduce small benchmarks suite
  • 0.3.2 (7 Jul 2014) AMD Support + Align correctly with ES6 Promise API (PRs 21 and 22 -- thanks Rich!)
  • 0.3.1 (27 Apr 2014) Improved release flow via PR #20.
  • 0.3.0 (21 Sep 2013) API: terminate aliased to destroy (deprecating the latter). See Issue #14.
  • 0.2.1 (30 Jul 2013) Fix worker-via-eval support (Safari 4/5, IE8/9)
  • 0.2.0 (29 Jul 2013) See #10
  • Dependency Loading (initially suggested in #8)
  • Deprecate direct returning in favour of a callback passed to each operative invocation.
  • Fallback to IFrame (to provide safer sandbox for degraded state)
  • 0.1.0 (25 Jul 2013) Support Promises (from Issue #3) if they're provided by a native Promise implementation or compliant polyfill. Also added support for operative(Function) which returns a single function.
  • 0.0.3 (18 Jul 2013) Support for asynchronous returning from within operative methods (via this.async()).
  • 0.0.2 (12 Jul 2013) Improved browser support: IE10 support via eval, degraded JSON-marshalling etc.
  • 0.0.1 (11 Jul 2013) Initial

Contributors

operative's People

Contributors

james-richards-privitar avatar padolsey avatar pgherveou avatar raynos avatar rich-harris avatar rileyjshaw avatar wmluke avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

operative's Issues

Minimized Operative does not work with Firefox

We are currently using an older JS minimization tool called JAWR. When minimizing Operative, it makes Firefox very angry with this:

...
        var loadedMethodName = '__operativeIFrameLoaded' + ++_loadedMethodNameI;
...

Firefox appears to treat that as a somewhat unknown operator, the triple-plus. This fixes it, obviously:

        var loadedMethodName = '__operativeIFrameLoaded' + (++_loadedMethodNameI);

Let me know if you'd like a pull request.

"An object could not be cloned" when promise is rejected with an error

It's considered a best practice to only reject promises with an Error (or derived object). Unfortunately, this breaks operative as errors can't be cloned. This results in an uncaught error within operative's returnResult in the context of the web worker, leaving the primary thread hanging waiting for a response that will never arrive.

I believe that operative should handle this case somehow. If the error can't be cloned via postMessage then we need to do something to tell the main thread that the operation failed -- either transfer the error object using JSON (which can serialize errors, albeit without their functions/prototype) or some other kind of operative-specific error that wraps a structured-cloning-compatible version of the rejection error.

As it stands right now, operative's promise interop is all but useless since it is completely unable to proxy rejections back to the main thread.

Support a web worker pool

It would be nice if I could define a number of web workers to each instance of operative. For example

var lodashWorker = operative(function(method, args, cb) {
    cb(_[method].apply(_, args));
  },
  ['http://cdnjs.cloudflare.com/ajax/libs/lodash.js/1.3.1/lodash.min.js'],
  4 //number of webworkers in pool
);

That way when you call

for(var i=0,l=10;i<l;i++){
  lodashWorker('uniq', [[1, 2, 3, 3, 2, 1, 4, 3, 2]], function(output) {
      output; // => [1, 2, 3, 4]
  });
}

whatever worker is available will run the code

This could be implemented where order is guaranteed or order is indeterminate

BTW Love the way your project is implemented. I appreciate the simplicity, documentation and technical descriptions!

importScripts support

Have you considered a way to allow the worker to importScripts? I have a few ideas for how it might be done.

  1. Add a second argument to the constructor with an array of URLs
operative({
  method1: function() {}
}, ["http://myhost.com/scripts/importlib-0.0.1.js"])
  1. Add a special property to the object passed during construction
operative({
    method1: function() {},
    __scripts__: ["http://myhost.com/scripts/importlib-0.0.1.js"]
})

This way these scripts could be loaded for all methods of the worker. Also when degrading back to running in the UI thread, you could either just ignore these scripts, or only load them if they are not already present on the page.

Thoughts?
Evan

Events

Really need something like:

var worker = operative(function (callback) {
    for (var i = 0; i < 1e9; i++) {
        if (i == 1e9 / 2)  {
             trigger('progress', '50%');
        }
    }

    callback('im done!');
});

worker.on('progress', function (val) { ... });
worker(function (val) { ... });

Accept function directly, without an object proxy

This:

var method = operative(function () {
  // stuff
});

If I want just a single worker, seems redundant to namespace it inside an object.

If this is already supported, I call a MORPH ability on this issue to be transformed into: 'Make better docs' :)

Worker methods to return promises

Best would be probably to feature detect the native promises spec, and if present, return them from methods. People can than choose themselves to include polyfills.

A lot better than inlining it into the code, or adding a dependency...

Operaritve: No operative.js URL available. Please set via operative.setSelfURL(...)

Hey there,

I'm trying to set operative's SelfURL to the version hosted on cdnjs, but I can't get it to work. The following:

var operative = require('operative');
operative.setSelfURL('https://cdnjs.cloudflare.com/ajax/libs/operative/0.4.1/operative.js');

var worker = operative(function (cb) { /* ... */ });
worker(function () {/* ... */});
};

throws an error in IE10:

Operaritve: No operative.js URL available. Please set via operative.setSelfURL(...)

I've tried a few configurations:

// 1
operative.setSelfURL('https://cdnjs.cloudflare.com/ajax/libs/operative/0.4.1/operative.js');

// 2
operative.setBaseURL('https://cdnjs.cloudflare.com/ajax/libs/operative/0.4.1/');

// 3
operative.setBaseURL('https://cdnjs.cloudflare.com/ajax/libs/operative/0.4.1/');
operative.setSelfURL('operative.js');

...but nothing has worked. Any thoughts?

0.3.2 release tag

Hi, any chance you could push a new release tag for those of us who have to use Bower? Copy & pasting this should do the trick:

git checkout master
git tag -a v0.3.2 -m 'version 0.3.2'
git push origin v0.3.2

Thanks

Node.js usage.

Hi, I did some research, looking for an inline Worker+Blob API library. Basically, the three clear options on npmjs are operative, threadless, and paralleljs.

paralleljs seems like something that would be built on top of threadless or operative, so I dropped that out of the picture, looking just for the basic inline worker implementation.

So between operative and threadless, operative has the cleanest API. threadless is more like a C/Java programmer approach, and the required err argument for the callbacks can get annoying.

That being said, threadless shines because it works in both server-side nodejs and client-side browser.

Ultimately I'd like a single API across both environments. threadless does it, but operative's API is so nice.

Would you be willing to adopt the workerjs library that threadless is depending on to make operative work everywhere?

I feel that multi-threading will be necessary in the upcoming years as the web evolves into more of a 3D world, not just simple documents. Libraries like http://famo.us, for example, aren't using workers yet but I think it will become necessity for them do start using them in the new age of 3D web. Three.js is of course using workers already. It makes sense for highly performant applications.

Unclear if the API allows workers to be generated from strings

TL;DR:
Using raw workers, I can generate a Blob from a string. Is there any affordance for this in operative's API? Can I alias the callback to a function that's called in the string?

The reason:
I'm writing a tool that needs to evaluate user-generated code. It's a programming challenge, so when a user thinks they've got the correct result they call verify(result) and see if it passes.

An example of a simple challenge:
Calculate 1+1, then pass the result into the verify function.

verify is defined by me behind the scenes as function (n) { return n == 2; }. After reading the challenge, the user might type something like this into a textarea and hit run.

var result = 1 + 1;
verify(result);

Thoughts so far:
It seems like this should work well with the operative API... I've got a chunk of code to run, and a single callback to send back to the main thread. I'm just not sure how to deal with stringified user input. One possible solution is:

var myOperative = operative(function (__code__, verify) {
  eval(__code__);
});

...but I was hoping to avoid using eval like that. Any thoughts on how best to handle this?

Thanks for all of your hard work on this library!!

[BUG]memory leak after action finished

callback or defferred should be set to null after result returned.

here's my hack, so is the iframe:

switch (data.cmd) {
  case 'console':
    window.console && window.console[data.method].apply(window.console, data.args);
    break;
  case 'deferred_reject_error':
    this.deferreds[data.token].reject(data.error);
    this.deferreds[data.token] = null; // avoid memory leak

    break;
  case 'result':

    var callback = this.callbacks[data.token];
    var deferred = this.deferreds[data.token];

    var deferredAction = data.result && data.result.isDeferred && data.result.action;

    if (deferred && deferredAction) {
      deferred[deferredAction](data.result.args[0]);
      this.deferreds[data.token] = null; // avoid memory leak
    } else if (callback) {
      callback.apply(this, data.result.args);
      this.callbacks[data.token] = null; // avoid memory leak
    } else if (deferred) {
      // Resolve promise even if result was given
      // via callback within the worker:
      deferred.fulfil(data.result.args[0]);
      this.deferreds[data.token] = null; // avoid memory leak
    }

    break;
}

Does not work in Nodejs at all

Simply doesn't work at all.

I try to run op = require('operative');

and then I get this:

ReferenceError: self is not defined
    at /tmp/test/node_modules/operative/dist/operative.min.js:2:3132
    at Object.<anonymous> (/tmp/test/node_modules/operative/dist/operative.min.js:2:8598)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at repl:1:7
    at REPLServer.self.eval (repl.js:110:21)

This is for the latest version on NPM (just installed 5 mins ago).

Support for 'throw' inside worker

Hi! First of all, I've only been using this library for a few days and I already find it awesome, so: thank you for creating it, your work is definitely appreciated!
My issue can probably be considered a request: I'm not certain that this is possible (I believe it should be), since I haven't looked into the source. What I would like to do is the following:

var test = operative(function() {
  throw 'Error'; // Throw an error instead of using deferred.reject
});

test().then(
function() {},
function(e) {
  console.error(e);
});

Is this possible? Is this not a good idea?
Thank you for your time and for your work!

Transferable objects

This is a slightly speculative issue but I thought it worth raising - do you have any plans to support transferable objects?

For anyone not familiar: transferable objects allow you to transfer an ArrayBuffer (and also a CanvasProxy or MessagePort, whatever they are!) to and from a web worker - rather than using the structured cloning algorithm, the reference to a transferable object is 'neutered' once it's transferred - the chunk of memory actually changes owner.

I'm using operative in a project that deals with large amounts of binary data. I've implemented support for transferables in my local copy of the library, and it's blazing fast. If it's useful I can tidy up the code and share it when I get a chance (it's probably not PR-worthy yet).

This is a heavily condensed illustration of how I'm using it:

projectGeometry = operative( function ( arrayBuffer ) {
  var d = this.deferred();

  float32Array = doSomeComplexMathsAndReturnATypedArray( arrayBuffer );
  d.transfer( float32Array.buffer );
}, [ 'poly2tri.min.js' ]);

// `buffer` comes straight from an xhr with responseType: 'arraybuffer'
projectGeometry.transfer( buffer ).then( function ( resultBuffer ) {
  var typedArray = new Float32Array( resultBuffer );
  doSomethingWith( typedArray );
});

An alternative API is to specify the data and the transfer list separately (which is how it works under the hood with postMessage:

w = operative( function ( foo, typedArray, anotherTypedArray ) {
  var d = this.deferred();

  /* some code happens... */

  d.transfer({ aThing: 42, anotherThing: myTypedArray }, [
    // list of objects to transfer from worker
    myTypedArray.buffer
  ]);
});

w.transfer( foo, typedArray, anotherTypedArray, [
  // list of objects to transfer to worker
  typedArray.buffer,
  anotherTypedArray.buffer
]).then( handleResult );

It's fairly easy to detect support for transferables (you just try transferring an array buffer and see if the byteLength falls to zero - article), and falling back to structured cloning works just fine.

0.2.0

  • Consider an operative.setReturnMode() to toggle between promises/direct/cb approaches. Right now it's all a bit of a mess.
  • Implement a way to pass in dependencies ( #8 ). And in the fully degraded state, run the code in an iframe to avoid conflicts and to make it a more worker-like env.
  • Deprecate direct-return in favour of using callback within operative methods. *
  • This will make more sense, for consistency:
var op = operative(function(a, b, cb) {
  cb(a + b);
});

op(1, 2, function(r) {
  r; // => 3
});

NOTE: Non-undefined direct-returning from operative methods will still work but is deprecated and will be removed sometime in the future. So please move over to the new callback style, or, alternatively, promises.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.