max-mapper / toiletdb Goto Github PK

View Code? Open in Web Editor NEW

215.0 6.0 19.0 29 KB

flushes an object to a JSON file. lets you do simple CRUD with async safely with the backend being a flat JSON file

JavaScript 100.00%

toiletdb's Introduction

toiletdb 🚽

flushes an object to a JSON file. lets you do simple CRUD with async safely with the backend being a flat JSON file

uses require('last-one-wins') to ensure atomicity of CRUD actions

note: if you write Buffers they will be converted to hex strings

API

var toilet = require('toiletdb')
// or require the in-memory version if you want
var db = require('toiletdb/inmemory')

// pass the name of the json file to use
var db = toilet('./data.json')

// open the db
await db.open()

// read data from data.json
var data = await db.read()
// read `key` inside data.json
var value = await db.read(key)

// sets `key` to `val` inside data.json
await db.write(key, val)

// deletes `key` key from data.json
await db.delete(key)

// deletes everything from data.json  
await db.flush()

// synchronous version
db.flushSync()

Custom FS

// pass the name and custom fs
var db = toilet({fs: customFs, name: './data.json'})

// write/read as normal

toiletdb's People

Contributors

Stargazers

Watchers

Forkers

yoshuawuyts retrohacker tehshrike onetwothreeneth thaer899 juliangruber okdistribute garbados jakeburden bladefidz jackub-test

toiletdb's Issues

'Error: EPERM: operation not permitted' running 'dat create'

I just installed DAT and tried to initialize a repository with 'dat create' command. And experienced error like below:

{ Error: EPERM: operation not permitted, rename 'D:\...\dat.json.0.6862932466069354' -> 'D:\...\dat.json'
  errno: -4048,
  code: 'EPERM',
  syscall: 'rename',
  path: 'D:\\...\\dat.json.0.6862932466069354',
  dest: 'D:\\...\\dat.json' }

As I googled on the Internet for similar bug reports and traced down the source code. I confirmed the bug is related to the code in toiletdb.

In the in the write() method, the code is using db.fs.writeFile(), then followed by db.fs.rename(), which can lead to a race condition, if rename() starts before writeFile() finishes, and caused the error I experienced above. After I changed writeFile() to writeFileSync(), the problem is resolved on my machine.

The code after change:

      db.fs.writeFileSync(tmpname, payload, function (err) {
        if (err) {
          return db.fs.unlink(tmpname, function () {
            cb(err)
          })
        }
      });
     db.fs.rename(tmpname, db.name, cb);

The root issue might be in node fs. But changing to writeFileSync makes toiletdb more robust.

Handle nested keys

It'd be neat if toiletdb methods that accepted key parameters handled nested keys. For example:

// db.json = { ports: { http: 80, https: 443 } }
db.read('ports.http', function(err, data) {
  console.log(data)
  > 80
})

or:

// db.json = { ports: { http: 80, https: 443 } }
db.read(['ports', 'http'], function(err, data) {
  console.log(data)
  > 80
})

The second method avoids confusion where '.' may be a part of an attribute's name, ex: data['example.org']

Use ToiletDB to store Vue/React state?

My apologies if this is a dumb question, I'm currently building a simple app and was wondering if there would be anyway to persist a vue components data / a react components state in toiletDB?

package too hilarious, unable to focus on real work

problem statement

jokes are 🚽

solution

destroy all jokes

durability

instead of just writing the file, it might be better to write to a second file (say, filename+'~') then rename that file to the normal filename.

When a toiletdb gets large, it's conceivable that the computer crashes halfway through a write (unlikely, but possible) if you only move the file after the write succeeds, then you'll have durability.

I used this approach in https://github.com/flumedb/atomic-file

cb is not a function

Just an FYI. Was playing with this. Got this error when trying to run:

/Users/klik/Desktop/toiletdb-test/node_modules/toiletdb/index.js:39
       cb(null, state)
       ^

TypeError: cb is not a function
   at /Users/klik/Desktop/toiletdb-test/node_modules/toiletdb/index.js:39:9
   at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:416:3)