Storage.js is a NodeJS library that standardizes common available via npm libraries like pkgcloud and abstracts away differences (especially within callbacks). Just write a simple implementation and leave configuration up to your users.

Table of Contents

• Storage.js
  • Installation
  • Configuration
    • Available storage providers
    • Global init
    • Separate init
    • Default instance
  • Usage
    • Uploading
    • Removing
    • Downloading
  • Hooks
    • Global hooks
    • Local hooks
    • Handling errors
    • Modifying parameters
  • Graceful exits
  • Motivation
  • Testing
  • About us
  • Thanks
  • License

Note: Work in progress.

var Storage = require('storage.js');

Before you start, you have to configure available providers by doing one of the following.

Azure & HP are not returning file url due to development works inside pkgcloud. They will be added soon

More providers will be added upon community requests (feel free to create an issue).

Should be called only once during your script execution as every object passed here overwrites previously set config.

Storage.init({
	cloudProvider: {
		provider: Storage.Providers.AmazonS3,
		container: '',
		key: '',
		keyId: ''
	},
	localProvider: {
		provider: Storage.Providers.LocalSystem,
		container: '/var/www/tmp'
	}
});

Can be called as many times as you find it useful. Simply extends your config with new providers.

Storage.add('cloudProvider', {
	provider: Storage.Providers.AmazonS3
	container: '',
	key: '',
	keyId: ''
});

In order to specify default instance, you should set up a value for default instance by doing the following:

Storage.settings('default instance', 'cloudProvider');

Now, you can use Storage like below:

Storage.get(function (err, client) {

});

Storage.get('cloudProvider', function (err, client) {
	if (err) return;
	client.upload('/path/to/your/file', '/remote/path', function (err, result) {
        // do your job
    });
});

Result object we receive on successful callback looks like the following one:

{
	container: '',
	path: '',
	filename: '',
	url: ''
}

Although that object is the same across all of the providers, please read provider-specific docs for better explanation of properties.

Storage.get('cloudProvider', function (err, client) {
	if (err) return;
	client.remove('/remote/path', function (err) {
		// do your job
	});
});

Storage.get('cloudProvider', function (err, client) {
	if (err) return;
	client.download('/remote/path', '/local/path', function (err) {
		// do your job
	});
});

Following pre/post hooks are available:

• upload,
• download,
• remove.

To register your hook, you can do it either globally (for every provider) or locally (per provider).

Useful if you need a global hook for every provider.

Storage.pre('upload', function (next, localSrc) {
	next();
});

To define a local hook, you can either do it globally or after receiving instance of your client.

Storage.pre('yourProviderName', 'upload', function (next) {
	next();
});
Storage.get('yourProviderName', function (err, client) {
	client.pre('upload', function (next) {
		next();
	});
	// do your upload
});

When defining a hook function, you have an access to a few parameters that we pass to every pre / post method.

The first one is next function that, when invoked, fires next hook or entire method.

It accepts optional Error object that may:

• inside pre - stop firing next hooks / original method and return that error as a result of method call

Storage
	.pre('amazon', 'upload', function (next) {
		next(new Error('Hook!'));
	})
	.pre('upload', function (next) {
		next(new Error('This one is skipped!'));
	});
	.get('amazon', function (err, client) {
		client.upload('file1.txt', 'file2.txt', function (err) {
			console.log(err); //Hook!
		});
	});

• inside post - stop firing next hooks and return that error as a result of method (original result object of method call will be lost)

Storage
	.post('amazon', 'upload', function (next) {
		next(new Error('Hook!'));
	})
	.post('upload', function (next) {
		next(new Error('This one is skipped!'));
	});
	.post('amazon', function (err, client) {
		client.upload('file1.txt', 'file2.txt', function (err) {
			console.log(err); //Hook! (but method was invoked anyway!)
		});
	});

After next function described above, the rest of the arguments is simply a list of original method parameters.

So, it's possible to write a hook like below:

Storage.post('amazon', 'upload', function (next, localSrc, destSrc, callback) {
	next('here we pass modified Src');
	// or next(localSrc, 'here we modify destSrc');
	// and so on...
});

Please note that we pass localSrc parameter to next, so other hooks and original method will used that value instead.

For further information on how hooks work and what are another ways of handling errors and modifying parameters - visit hook.js repository.

In order to clean up resources used by the StorageAPI (for example opened connections to the MongoDB), Storage exposes method, called exit that you can use like below:

process.on('exit', function () {
	Storage.exit(function () {
		console.log('Exited');
	});
});

See examples/mongo.js for further reference.

Please note that in few cases (MongoDB), not calling that method may make it impossible for your script to exit.

If you ever wanted to implement storage integration right in your application - we got you covered. You've probably encountered problems with different libraries, especially if you wanted to integrate two or three providers, just to give your users a better choice. That's why Storage.js was created. Wrapping multiple libraries and creating simple abstraction layer for them allows you to easily add about 5 providers at once!

With our 100% 70% (still improving!) code coverage, we believe it's important. You can run our test suites by a command npm test. To generate code coverage, use gulp coverage.

Author: @grabbou via KeystoneJS

Special thanks to the maintainers of:

• pkgcloud for providing cloud integration with Azure, HP, Rackspace, OpenShift and Amazon,
• gridfs-stream for MongoDB layer,
• hook-js for providing powerful and super easy hook library.

Project licensed under MIT (See LICENSE for further details)