implement leveldown backends for all the things! about community HOT 9 CLOSED

Also sophia which I started but shelved since @mmalecki has started one and says he's going to make it LevelDOWN compatible.

Hints for anyone doing this:

• Make it easy on yourself and copy the entirety of leveldown or node lmdb and just mess with it till it fits the data store you're using. Include attribution to the project(s) you copied but these are open source for a reason.
• You'll probably need to do a binding.gyp because it's not really that popular in the wild, lots of examples in leveldown (see binding.gyp, leveldb.gyp and snappy.gyp) and node lmdb, ping me if you need any extra help with this cause I'm unwittingly becoming an expert...
• Use the AbstractLevelDOWN test suite. If you want to be able to plug in to the back of LevelUP then you need to pass these tests. Look at leveldown, node lmdb, memdown, level.js etc. for how this is done.
• Consider using AbstractLevelDOWN as a frontend for your binding, we'll probably be moving to this for LevelDOWN because AbstractLevelDOWN has all the common logic for checking income arguments & callbacks, there's currently duplication inside LevelDOWN because it's doing the same thing.
• Use NAN (or shim / node-addon-layer although it's still a little immature and I have no experience with it yet). Compat across Node versions without help is a nightmare.
• Ask for help! I'm happy to contribute to any projects that want to make a LevelDOWN-compatible binding of anything.

from community.

@maxogden I assume you mean to use db in options to achive this, don't you?

I'll try sparkey but don't expect me to come up with something real since this will be my first C/C++ project.

@rvagg There is already a driver for sophia someone could hock this up to levelup.

from community.

throwing this in here because why not https://github.com/antonmks/Alenka

from community.

@rvagg @maxogden I have started started a binding for BangDb. I am also at the same time thinking of how to map it into Abstract Level Down because I am a LevelUp Lover.

I will definitely need a little bit of help with some issues that i dont have the experience to solve yet

One such issue is figuring out how to make sure the handle to the db is released on abrupt exit from an application. I have tried sticking all cleanup in destructor functions but they dont seem to be called when v8 quits.

I can easily manage this from the nodejs level with a process.on('exit'). However I would love to take care of this @ the C Level.

**Edit - Actually I got it working mostly there is one edge case.

from community.

@miketheprogrammer there's a void node::AtExit(void (*cb)(void* arg), void* arg = 0); you could attach a function pointer to and have it called when Node is going down, should be the same as process.on('exit') but I haven't had reason to use it myself so ymmv.

from community.

@rvagg Im getting closer to having a bang-down interface. Right now trying to implement a bangdb in level down style (i.e. using the level-down code for reference). Then closing the gaps with javascript.

Im close to opening up the code to the public. Just want to make sure i clean it up and standardize it before hand.

Im having some performance concerns.

One question. As far as batch in chainable mode is concerned. BangDb doesnt support batch out of the box however it does support transactions.

Would you recommend using a Map to map keys to values and action types in C and then iterate over the map in a transaction.

Or is it better to store the objects In JS and transfer them all @ once similar to batch in function mode.

Batch in function mode seems to have great performance almost as good as synchronous puts/deletes.

** As well, I may need some help from someone more experienced in setting up the binding.gyp properly. I cant figure out how to link the bang db library file to node. I am currently using a -lbangdb command which works because i have bangdb already added to my system.

from community.

transactions are ideal for batch and it doesn't really matter how you queue up the batch as long as it's atomic (and even that's just a strong preference and not possible in some backends). There's a AbstractChainedBatch in abstract-leveldown you can use to provide most of the implementation of chained-batch, you just hook it in to a non-chained batch() implementation and you're done.

If you can rely on JS to do a lot of the work then do it, it'll end up less painful in the end and you can iterate on performance later on when you know where it's a problem (and if it's a problem). Often doing it in JS can be more efficient anyway because it gets to be optimised whereas the C++ interface to V8 can't be optimised automatically in the same way and you're just relying on the fact that it's compiled by a (hopefully) good compiler.

from community.

quick, someone write hamsterdown! https://github.com/cruppstahl/hamsterdb

from community.

@maxogden hamsterdb looks promising 👍

from community.