Giter Club home page Giter Club logo

async-lock's Introduction

async-lock

Lock on asynchronous code

Build Status

  • ES6 promise supported
  • Multiple keys lock supported
  • Timeout supported
  • Occupation time limit supported
  • Execution time limit supported
  • Pending task limit supported
  • Domain reentrant supported
  • 100% code coverage

Disclaimer

I did not create this package, and I will not add any features to it myself. I was granted the ownership because it was no longer being maintained, and I volunteered to fix a bug.

If you have a new feature you would like to have incorporated, please send me a PR and I will be happy to work with you and get it merged. For any bugs, PRs are most welcome but when possible I will try to get them resolved as soon as possible.

Why do you need locking on single threaded nodejs?

Nodejs is single threaded, and the code execution never gets interrupted inside an event loop, so locking is unnecessary? This is true ONLY IF your critical section can be executed inside a single event loop. However, if you have any async code inside your critical section (it can be simply triggered by any I/O operation, or timer), your critical logic will across multiple event loops, therefore it's not concurrency safe!

Consider the following code

redis.get('key', function(err, value) {
	redis.set('key', value * 2);
});

The above code simply multiply a redis key by 2. However, if two users run concurrently, the execution order may like this

user1: redis.get('key') -> 1
user2: redis.get('key') -> 1
user1: redis.set('key', 1 x 2) -> 2
user2: redis.set('key', 1 x 2) -> 2

Obviously it's not what you expected

With asyncLock, you can easily write your async critical section

lock.acquire('key', function(cb) {
	// Concurrency safe
	redis.get('key', function(err, value) {
		redis.set('key', value * 2, cb);
	});
}, function(err, ret) {
});

Get Started

var AsyncLock = require('async-lock');
var lock = new AsyncLock();

/**
 * @param {String|Array} key 	resource key or keys to lock
 * @param {function} fn 	execute function
 * @param {function} cb 	(optional) callback function, otherwise will return a promise
 * @param {Object} opts 	(optional) options
 */
lock.acquire(key, function(done) {
	// async work
	done(err, ret);
}, function(err, ret) {
	// lock released
}, opts);

// Promise mode
lock.acquire(key, function() {
	// return value or promise
}, opts).then(function() {
	// lock released
});

Error Handling

// Callback mode
lock.acquire(key, function(done) {
	done(new Error('error'));
}, function(err, ret) {
	console.log(err.message) // output: error
});

// Promise mode
lock.acquire(key, function() {
	throw new Error('error');
}).catch(function(err) {
	console.log(err.message) // output: error
});

Acquire multiple keys

lock.acquire([key1, key2], fn, cb);

Domain reentrant lock

Lock is reentrant in the same domain

var domain = require('domain');
var lock = new AsyncLock({domainReentrant : true});

var d = domain.create();
d.run(function() {
	lock.acquire('key', function() {
		//Enter lock
		return lock.acquire('key', function() {
			//Enter same lock twice
		});
	});
});

Options

// Specify timeout - max amount of time an item can remain in the queue before acquiring the lock
var lock = new AsyncLock({timeout: 5000});
lock.acquire(key, fn, function(err, ret) {
	// timed out error will be returned here if lock not acquired in given time
});

// Specify max occupation time - max amount of time allowed between entering the queue and completing execution
var lock = new AsyncLock({maxOccupationTime: 3000});
lock.acquire(key, fn, function(err, ret) {
	// occupation time exceeded error will be returned here if job not completed in given time
});

// Specify max execution time - max amount of time allowed between acquiring the lock and completing execution
var lock = new AsyncLock({maxExecutionTime: 3000});
lock.acquire(key, fn, function(err, ret) {
	// execution time exceeded error will be returned here if job not completed in given time
});

// Set max pending tasks - max number of tasks allowed in the queue at a time
var lock = new AsyncLock({maxPending: 1000});
lock.acquire(key, fn, function(err, ret) {
	// Handle too much pending error
})

// Whether there is any running or pending async function
lock.isBusy();

// Use your own promise library instead of the global Promise variable
var lock = new AsyncLock({Promise: require('bluebird')}); // Bluebird
var lock = new AsyncLock({Promise: require('q')}); // Q

// Add a task to the front of the queue waiting for a given lock
lock.acquire(key, fn1, cb); // runs immediately
lock.acquire(key, fn2, cb); // added to queue
lock.acquire(key, priorityFn, cb, {skipQueue: true}); // jumps queue and runs before fn2

Changelog

See Changelog

Issues

See issue tracker.

License

MIT, see LICENSE

async-lock's People

Contributors

abozaralizadeh avatar azisso avatar bitrivers avatar bmrpatel avatar dmurvihill avatar erikvold avatar jdanford avatar joeschneider32 avatar jtyers avatar lostigeros avatar luke-stead-sonocent avatar mdsummers avatar meatwallace avatar mottymilshtein avatar paulroub avatar philipp91 avatar rain1017 avatar rogierschouten avatar simeonborko avatar taschmidt avatar techieshark avatar thepiz avatar tkrotoff avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

async-lock's Issues

Please remove grunt-env

grunt-env is pretty old and shows vulnerabilities on npm audit. It wasn't like that when it was a dev-dependency.
image

Remove use of Promise.defer()

Running this on Node v10 gives:

(node:28588) Warning: Promise.defer is deprecated and will be removed in a future version. Use new Promise instead.

I see that some Promise usage was cleared up in #2, but I think the code needs to remove all use of Promise.defer() in favour of new Promise(...).

Lock doesn't work in distributed system

@rogierschouten
This works perfectly locally but when it doesn't work on the live, which has 2 kubernetes pods handled by load balancer. It will be no problem with single pod but you know, usually in live environment we have multiple processes running to handle concurrent requests in a distributed way, like in my example or on heroku with multiple dynos.

Is there any solution to overcome this problem?

Thanks in advance and stay safe.

Do not understand the logic of timeouts

Does one failed task fail all queued tasks?
I modified the first test by adding a long task and it was seen that other tasks also failed.

it('Single key test', function (done) {
	var lock = new AsyncLock({ timeout: 10 });

	var taskCount = 8;
	var keyCount = 4;
	var finishedCount = 0;

	var isRunning = {};

	var taskNumbers = [];
	for (var i = 0; i < taskCount; i++) {
		taskNumbers.push(i);
	}

	var key = 1 % keyCount;
	lock.acquire(key, function (cb) {
		console.log('long(key%d) start', key);
		setTimeout(cb, 12);
	}).then(
		() => console.log('long(key%d) done', key),
		err => console.log('long(key%d) error', key, err)
	);

	taskNumbers.forEach(function (number) {
		var key = number % keyCount;
		lock.acquire(key, function (cb) {
			assert(!isRunning[key]);
			assert(lock.isBusy() && lock.isBusy(key));

			var timespan = Math.random() * 1000;
			console.log('task%s(key%s) start, %s ms', number, key, timespan);
			setTimeout(cb.bind(null, null, number), timespan);
		}, function (err, result) {
			if (err) {
				// return done(err);
			}

			console.log('task%s(key%s) done %s', number, key, !err ? '' : err.message || err);

			isRunning[key] = false;
			finishedCount++;
			if (finishedCount === taskCount) {
				lock.acquire(['key1','key2'], () => { done(); });
				// assert(!lock.isBusy());
				// done();
			}
		});
	});
});

And run:

$ ./node_modules/.bin/mocha -f "Single key test"

  AsyncLock Tests
long(key1) start
task0(key0) start, 926.8097332784681 ms
task2(key2) start, 232.97781253882687 ms
task3(key3) start, 631.8339886981712 ms
task1(key1) done async-lock timed out
task4(key0) done async-lock timed out
task5(key1) done async-lock timed out
task6(key2) done async-lock timed out
task7(key3) done async-lock timed out
long(key1) done
task2(key2) done 
task3(key3) done 
task0(key0) done 
    โœ“ Single key test (931ms)


  1 passing (938ms)

Only tasks started before the long task timed out has been complete without error.

example

Please consider this module for use as an example on the project site.

I hate that it requires another module but with a little modification by you that will be gone.

It was so hard to figure out how to put this module into the async() await form that maybe this could be a guide for the next adopter.

-Mike

Tarballs published on npmjs has wrong permissions

Currently files are only readable by owner, this leads to issues in production environments where the ownership of the files are not always the same as the user executing nodejs.

Actual results:

curl -s "https://registry.npmjs.org/async-lock/-/async-lock-1.1.2.tgz" | tar -tvz
-rw-r--r-- 0/0            1140 2018-02-27 03:19 package/package.json
-rw------- 0/0             321 2018-02-27 03:19 package/AUTHORS
-rw------- 0/0            1595 2018-02-27 03:19 package/History.md
-rw------- 0/0              49 2018-02-27 03:19 package/index.js
-rw------- 0/0            1110 2018-02-27 03:19 package/LICENSE
-rw-r--r-- 0/0            4279 2018-02-27 03:19 package/README.md
-rw-r--r-- 0/0            5992 2018-02-27 03:20 package/lib/index.js

Expected results:

curl -s "https://registry.npmjs.org/async-lock/-/async-lock-1.1.2.tgz" | tar -tvz
-rw-r--r-- 0/0            1140 2018-02-27 03:19 package/package.json
-rw-r--r-- 0/0             321 2018-02-27 03:19 package/AUTHORS
-rw-r--r-- 0/0            1595 2018-02-27 03:19 package/History.md
-rw-r--r-- 0/0              49 2018-02-27 03:19 package/index.js
-rw-r--r-- 0/0            1110 2018-02-27 03:19 package/LICENSE
-rw-r--r-- 0/0            4279 2018-02-27 03:19 package/README.md
-rw-r--r-- 0/0            5992 2018-02-27 03:20 package/lib/index.js

Is it ok to make lock.queues public knowledge?

I am working on a detecting if there are pending tasks on a queue, so I checked lock.queues. However we don't make this public knoweldge so it's implementation detail right now. Is it ok to pr to make this public and kind of a contract that we stick to?

Usage with async/await

Hi,

I just wanted to make sure the lock will work as expected if one calls it this way:

import AsyncLock from 'async-lock';

const lock = new AsyncLock();

async function bla () {
  await lock.acquire(bla.name, async () => {
     // some async stuff that should be executed in critical section
  });
}

always time out after long lasting operation

OK, what I did was bullshit, but I tried to hang the fn for a certain time and see, if the lock works, if acquired from another side.

lock.acquire("postman", (id) => {
   var e = new Date().getTime() + (10 * 1000);
   while (new Date().getTime() <= e) { }
}, (err, ret) => {
   if (err) console.log(err);
))

What I see now is, that after the first hang for 10 secs all subsequent requests to lock.acquire end up in err Error: async-lock timed out

I know this case is pathologic, but why is the lock running into the timeout after?

In Electon: DEAD LOCK from unknown source always occurs after frequent use of AsynLock

In my music scheduler application, mutex is required to ensure the music is played as scheduled without conflict which may be caused by scheduler plan update from server. So after each song is over, I lock the generation function of the next music. However, after a few hours (3 ~ 4hours, nearly 100 songs, more than 2000 mutex operation), a deadlock always occurred in production environment.

So I switch to use other mutex library, and no deadlock occurred since then.

The following is the structure of my code.

let globalMusicPlayerAudio = new Audio();
let musicSchedulerLock = new AsyncLock();
globalMusicPlayerAudio.onloadedmetadata = (e) => {
    // show music title only when music is load
    eleMusicName.innerHTML = globalMusicMetaNow.title + ' - ' + globalMusicMetaNow.artist
    // some other log
};
globalMusicPlayerAudio.ontimeupdate = (e) => {
    // calculate current progress and update progress bar
}
globalMusicPlayerAudio.onended = (e) => {
    // state update.
   
    timerNextMusic();
}
globalMusicPlayerAudio.onerror = (e) => {
    deleteLocalFile();
    timerNextMusic();  
}

function timerNexMusic() {
    let taskAfterUnlock = () => {};
    musicSchedulerLock.acquire(['globalMusicUpdateLock'], function (done) {
        try {
            // log state of current music . 

            if (nowIsPlayingAlready) {
                return
            }
            // detect available music . for example, check whether the file exists, whether it is currently playing time, whether the playlist should be switched, etc.
            var findNextMusic = undefined; // this is the result 
            if (findNextMusic == null) {
                // no need to play
            } else {
                taskAfterUnlock = () {
                    globalMusicPlayerAudio.src = findNextMusic.src;
                    globalMusicPlayerAudio.play();
                } 
            }
        } catch(e) {
             // error logger
        } finally {
             done(); 
        }
    }, (err, ret) => {
        if (taskAfterUnlock != null) {
            console.log('Music Timer. Do something after lock release')
            taskAfterUnlock()
        }

        if (globalMusicTimer) {
             clearTimeout(globalMusicTimer)
        }
        globalMusicTimer = setTimeout(timerNexMusic, 60 * 1000);
    })
}

Unhandled promise rejection on lock timeout

Examine the following code:

const AsyncLock = require("async-lock");
const lock = new AsyncLock({ timeout: 5000 });

lock.acquire("test", function(done) {}); // hold the lock open
const promise = lock.acquire("test", function() {}); // try to acquire the busy lock
promise.then(() => console.log("success"));
promise.catch(() => console.log("fail"));

What do you expect the program to output? I was expecting "fail", but the reality was:

$ node test.js
fail
(node:20581) UnhandledPromiseRejectionWarning: Error: async-lock timed out
    at Timeout._onTimeout (/home/vwoo/shared/autocomplete/node_modules/async-lock/lib/index.js:164:17)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)
(node:20581) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:20581) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

It is very strange to me that we have both the "fail" message as well as an unhandled promise rejection error. I'm not sure why and I'll try to take a peek inside the library to understand (just documenting my findings here).

Uncaught Error in callback mode leading to deadlock

Consider the following code:

const AsyncLock = require('async-lock');
const lock = new AsyncLock;

lock.acquire("key", function (done) {
   throw new Error("error");
});

I would expect that acquire() would return rejected Promise but I got Uncaught Error: error, and the lock is not released.

The snippet is just slightly different from the one in the README which returns rejected Promise.

lock.acquire(key, function() {
  throw new Error('error');
})

From your point of view, is my code just improper usage of the library or can we consider it as a bug? If so, I'd be happy to try to prepare PR.

[Feature request] maxPending + skip queue behaviour and ability to manually override lock release

I have two suggestions for the library:

  1. Currently when you have maxPending enabled with a full queue and the next lock acquire uses skip queue, the next lock acquire with skip-queue priority does not get added to the queue because it errors out saying that the queue is already full. I want it to add to queue, but discard the last item in the queue instead.
    I tweaked the example to illustrate how this should behave.
var lock = new AsyncLock({maxPending: 1});
lock.acquire(key, fn1, cb); // runs immediately
lock.acquire(key, fn2, cb); // added to queue (reached maxPending of 1)
lock.acquire(key, priorityFn, cb, {skipQueue: true}); // jumps queue and basically replaces fn2
  1. I would also like to optionally control when to release the lock.
    Please let me know your thoughts on these

Unexpected: batch keys get reversed

Is there a reason why the acquireBatch function reverses the keys here? The problem is Array.prototype.reverse is a destructive function that (unexpectedly) modifies the original array. I had a downstream bug that I just tracked back to this behavior.

At the very least, could the line above be changed to something like [...keys].reverse() or something? And yes, I know I can work around this behavior by doing the same up in my code but this behavior doesn't seem desirable.

Empty the queue

Hello, first of all, thanks for this module.

I was wondering if there is any way to empty the queue of pending locks. At some point, I don't care anymore about what is pending so I would like to cancel everything that is pending in the queue.

Thanks :)

ES5 compliance requested

We use this package in a browser, and we have a problem with the build: the package uses arrow functions that are not ES compliant.

Temporarily we solved the issue by adding the path "node_modules/async-lock/**/*.js" to our tsconfig path (plus allowJs: true) but it's rather a hack than a real solution.

I see the arrow function in 2 lines which could be changed to a simple function without a problem. (I did not check for anything else that is not ES5 compliant.)

We have no problem with Promises and other things that can be polyfilled.

maxOccupationTime not working as I would expect

Hi,

Am I correct that the expected behavior of the maxOccupationTime option is that once the lock has been acquired, the maxOccupationTime timer starts counting for the item that has acquired the lock? What I think I am observing in practice that this maxOccupationTime timer beginning as soon as lock.acquire is called and it enters the queue. This behavior is what I expect from the timeout option. Is my understanding of a lock being acquired incorrect?

My code behaves as follows:

const lock = new AsyncLock({
  maxOccupationTime: 10 * 1000 // 10 seconds
});

for (let index = 0; index < 100; index += 1) {
  lock.acquire("abc", async () => {
    // code that takes about 1 second to execute and returns at the end
  })
}

check if a key is already locked

is it possible to check if the key is already locked and instead of waiting it until it's released, execute another function ?

process not set when acquiring lock

Not sure what has changed in chrome, but we are now getting a process undefined error from this package.

image

updated this line to avoid the error..

and subsequently received the same error on.
image

Multiple keys lock not atomic.

When I started using this library I assumed that the order of keys wouldn't matter when acquiring multiple locks but I was wrong as demonstrated by the example below:

let AsyncLock = require('async-lock');

let lock = new AsyncLock();

lock.acquire(["a", "b"],  (release)=>{
        console.log("1");
        setTimeout(release);
});
lock.acquire(["b", "c"], (release)=>{
        console.log("2");
        setTimeout(release);
});
lock.acquire(["c", "d"], (release)=>{
        console.log("3");
        setTimeout(release);
});

I was expecting it to print 1 -> 2 -> 3 but it prints 1 -> 3 -> 2 because lock.acquire(["b", "c"], only queues "c" after "b" has been acquired. I think that this behavior is counter intuitive and should be explained in the documentation.

Recognise async lock errors

Hi @rogierschouten! Thanks for keeping to work on this library, really appreciate ๐Ÿ™

Correct me if I'm wrong, currently the only way to distinguish between async-lock errors and any error that's thrown by the wrapped async function is by using the error message:

function someAsyncFn() {}

lock.acquire(key, someAsyncFn, opts)
    .catch(function(err) {
        const isAsyncLockTimeoutError = err instanceof Error && err.message.includes('async-lock timed out')
	if (isAsyncLockTimeoutError) {
            // handle async-lock timeout error
        }
    });

IMHO it would be great if async-lock could deal with its own error classes to model anything can go wrong on the library side.

class AsyncLockError extends Error {
    constructor(message) {
        super(message);
    }
}

class AsyncLockTimeoutError extends AsyncLockError {
    constructor(queueKey) {
        super(`Async lock timeout out in queue ${async-lock timed out in queue}`);
        this.name = 'AsyncLockTimeoutError';
    }
}

This mean on the user-land side it would be much cleaner handling any async-lock error:

function someAsyncFn() {}

lock.acquire(key, someAsyncFn, opts)
    // Handle different kind of async-lock errors
    .catch(function(err) {
        if (err instanceof AsyncLockTimeoutError) {
            // do something...
        }
        if (err instanceof AsyncLockMaxOccTimeError) {
            // do something...
        }
    });
function someAsyncFn() {}

lock.acquire(key, someAsyncFn, opts)
    // Catch any kind of async-lock errors
    .catch(function(err) {
        if (err instanceof AsyncLockError) {
            // async-lock error
        } else {
            // Any error thrown by `someAsyncFn`
        }
    });

What do you think about that? I can find some time to work on this if it can help.

using `maxOccupationTime` in typescript not working

Error:(8, 40) TS2345: Argument of type '{ maxOccupationTime: number; }' is not assignable to parameter of type 'AsyncLockOptions'. Object literal may only specify known properties, and 'maxOccupationTime' does not exist in type 'AsyncLockOptions'.

Lock inside promise not actually locking?

I may be doing something obviously wrong, I'm not sure. But I tried putting a lock around some code in a promise that was causing our server issues to be sure it was only done once at a time. However from my logs I see multiple clients ending up in the locked section simultaneously.

This is the function in question: https://github.com/gcp/leela-zero-server/blob/287ae0a852ff03a2cf0d528ee2c7c6981ec33c36/server.js#L182

The bulk of the Promise has the code wrapped in the lock.request(). But I can see extra clients are able to acquire the lock and start an unzip process even before the first unzip has completed.

Does the lock have to be outside the "return new Promise" above it? Do I need to "await" the lock.request() when called inside a Promise?

Any help appreciated. Thanks!

Remove Q dependency

Seems like the Promise global is pretty standard now, it would be nice if that were used as the default instead of Q, and so that this package has no dependencies.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.