Giter Club home page Giter Club logo

jungle-db's Introduction

jungle-db Complete Check

JungleDB is a simple database abstraction layer for NodeJS (LMDB) and browsers (IndexedDB) supporting advanced features such as transactions with read-isolation and secondary indices.

Quickstart

The easiest option to use jungle-db is to install it from the npm repository.

npm install @nimiq/jungle-db

Or alternatively using yarn add @nimiq/jungle-db.

Usage

Getting started

Depending on your target and preferences, include one of the files in the dist folder into your application.

  • Modern Browsers: indexeddb.js
  • Browser backwards compatibility: indexeddb-babel.js
  • NodeJS LMDB: lmdb.js

In NodeJS, you can use var JDB = require('@nimiq/jungle-db'); to include the LMDB backend.

Note on Edge browser: At the time of writing, Edge does not provide full IndexedDB support. That means that using certain types of indices might fail on Edge. If you observe a DataError in Edge while using JungleDB, it is most likely that you are using one of the features not supported by Edge. One of the unsupported features are binary keys.

Then, create a JungleDB instance and potential object stores as follows:

// Create a JungleDB instance
// The maxDbSize option is only required for LMDB based databases
const db = new JDB.JungleDB('myDatabase', 1);

// Create an object store
db.createObjectStore('myStore');

// Switch to an async context
(async function() {
    // Connect to your database
    await db.connect();

    // Now you can easily put/get/remove objects and use transactions.
    const store = db.getObjectStore('myStore');

    const tx = store.transaction();
    await tx.put('test', 'value');
    
    // Prints value
    console.log(await tx.get('test'));
    // Prints undefined due to read isolation
    console.log(await store.get('test'));
    
    await tx.commit();
    
    // Prints value
    console.log(await store.get('test'));
    await store.remove('test');
})();

If your next application version now includes an index, you can use upgrade conditions:

// Create a JungleDB instance with a new version
const db = new JungleDB('myDatabase', 2);

// Create an object store
// The upgrade condition specifies that the store needs to be physically created
// if the database version is less than 1 (since we created it in version 1).
const st = db.createObjectStore('myStore', { upgradeCondition: version => version < 1 });
st.createIndex('myIndex', 'i', { upgradeCondition: version => version < 2 });

// Switch to an async context
(async function() {
    // Connect to your database
    await db.connect();

    // Now you can easily put/get/remove objects and use transactions.
    const store = db.getObjectStore('myStore');

    const tx = store.transaction();
    await tx.put('test', {'i': 1, 'data': 'value'});
    
    // Prints {'i': 1, 'data': 'value'}
    console.log(await tx.get('test'));
    // Prints undefined due to read isolation
    console.log(await store.get('test'));
    
    await tx.commit();
    
    // Prints {'i': 1, 'data': 'value'}
    console.log(await store.get('test'));
    
    // Prints [{'i': 1, 'data': 'value'}]
    console.log(await store.index('myIndex').values(KeyRange.only(1)));
    console.log(await store.values(Query.eq('myIndex', 1)));
    
    await store.remove('test');
})();

Encoding

JungleDB allows to specify custom encodings for values (primary keys are currently restricted to strings only). The encoding is only applied immediately before writing/after reading from the underlying backend. A custom encoding โ€“ implementing the ICodec interface โ€“ can be passed to the JungleDB.createObjectStore(tableName, options) method in the options argument as follows:

db.createObjectStore('test', {
    codec: {
        encode: value => yourEncodeFunction(value),
        decode: (value, key) => yourDecodeFunction(value, key),
        valueEncoding: JungleDB.JSON_ENCODING // This property is only used for LMDB.
    }    
});

The valueEncoding property defines a backend specific encoding. While the default JSON encoding is sufficient for most cases, it can be used to optimise storage in case only binary data is stored. There is also the possibility to define different backend specific encodings for LMDB using lmdbValueEncoding. Possible backend specific encodings are:

  • JungleDB.JSON_ENCODING for JSON objects
  • JungleDB.NUMBER_ENCODING for numbers
  • JungleDB.STRING_ENCODING for strings
  • JungleDB.BINARY_ENCODING for binary types
  • JungleDB.GENERIC_ENCODING for a generic value (the code automatically determines the encoding and prepends a type byte)

The createIndex(name, keyPath, options) method also supports an optional keyEncoding option to specify the backend specific encoding of the secondary key.

Database Options

There are options specific to some of the backends. Especially the LMDB backend is highly configurable. If an option does not apply for the current backend, it is simply ignored.

LMDB Options

  • maxDbSize: number: The maximum size of the database in bytes (default: 5MB).
  • autoResize: boolean: This flag indicates whether the database should be automatically resized if needed (default: false). If enabled, the DB will be resized by max(minResize, spaceNeeded).
  • minResize: number: The minimum number of bytes the database will be resized (default: 100MB).
  • maxDbs: number: The maximum number of object stores + indices for this JungleDB instance. This value defaults to the correct number of object stores + indices created.

Here is an example to make use of these options.

// Create a JungleDB instance with a new version
const db = new JDB.JungleDB('myDatabase', 2, {
    autoResize: true,
    maxDbSize: 1024*1024*100
});

Documentation

A large fraction of the code is documented using ESDoc and a hosted version of this documentation can be found here.

Installing from Source

  1. Clone this repository git clone https://github.com/nimiq-network/jungle-db.
  2. Run npm install or yarn
  3. Run npm run build or yarn build
  4. Open clients/browser/index.html in your browser to access a simple browser example.

Run Example

Run Browser Example

Open clients/browser/index.html in your browser.

Run NodeJs Example

Start the example by running clients/nodejs/index.js.

cd clients/nodejs/
node index.js

Benchmarks

Run IndexedDB Benchmarks

Open benchmark/indexeddb/index.html in your browser

Run LMDB Benchmarks

Start the example by running benchmark/lmdb/index.js.

cd benchmark/lmdb/
node index.js

Test and Build

Run Testsuite

  • npm test or yarn test runs browser and NodeJS tests.
  • npm run test-indexeddb or yarn test-indexeddb runs the testsuite in your browser only.
  • npm run test-lmdb or yarn test-lmdb runs the LMDB testsuite for NodeJS only.

Run ESLint

npm run lint or yarn lint runs the ESLint javascript linter.

Build

Executing npm run build or yarn build concatenates all sources into dist/{indexeddb,lmdb}.js

Contribute

If you'd like to contribute to the development of JungleDB please follow our Code of Conduct and Contributing Guidelines.

License

This project is under the Apache License 2.0.

jungle-db's People

Contributors

paberr avatar dependabot[bot] avatar riptl avatar jeffesquivels avatar danimoh avatar styppo avatar curdbecker avatar bettelstab avatar mar-v-in avatar sisou avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.