Have you done any benchmark on it compared to others? about lokijs HOT 10 CLOSED

Maybe you should consider adding "stamp" also to every entity in a collection.
Everytime you update 1 entity, the stamp goes up 1.
This way 1 user will know if data have been changed since he got it, edited and tried to add it.
on update there could be a extra param u add if you wanted it to check if stamp is the same. (true/false)
By using this you could get a error telling stamp does not match.
This could be practical for in use with workers etc.

from lokijs.

Hi there,

we did benchmark lokijs with similar products, namely PouchDB, NeDB and
TaffyDB. We're yet to compile everything into "official" results but so far
we have significantly faster performance on all operations.
I will try to put something together this coming week if this is something
you need quite urgently.
Joe

On Sun, Oct 12, 2014 at 6:48 PM, vegarringdal [email protected]
wrote:

Have you done any benchmark on it compared to others?

Thx, will give this a try on wakanda server (wakanda.org)

—
Reply to this email directly or view it on GitHub
#6.

from lokijs.

Atm I'm just testing to see if I understand it, testing it on server side on Wakanda in a module.
Is it possible to insert many with the use of a object array?
Is it also possible to query with "contains"?

Looking at find and findOne, would it not be easier to use query word for find, and just use find instead of find one.

from lokijs.

All great suggestions!
We're soon going to release v1.0 which is going to slightly change the API,
and one of the changes is exactly the replacement of findOne with find and
find with query.
The "contains" operator is coming soon (this week), as well as regexp
support for top-level properties.

As for the using the array for insert, it was a simple change so I applied
it directly. If you redo an npm install you should get version 0.1.32 with
array insert support.
Let me know if that works for you.
joe

On Sun, Oct 12, 2014 at 8:26 PM, vegarringdal [email protected]
wrote:

Atm I'm just testing to see if I understand it, testing it on server side
on Wakanda in a module.
Is it possible to insert many with the use of a object array?
Is it also possible to query with "contains"?

Looking at find and findOne, would it not be easier to use query word for
find, and just use find instead of find one.

—
Reply to this email directly or view it on GitHub
#6 (comment).

from lokijs.

Hi,

thanks a lot for the feedback, it is precious.

We are considering revisions, similar to what couchdb has in place, also
because among other things we'd like to implement replication soon enough,
and revisions will help keep data in sync between server instances.
For this reason a simple incremental integer stamp would probably not be
enough (two entities could have the same stamp on two different db
instances but be out of sync), probably a "last-updated" stamp would work,
but i'm not sure about that either, it's quite an important design decision
as it will affect important features.
Hopefully we'll get round to making a decision in the next few days.
You're more than welcome to contribute if you want! In any case i have
taken note of the points you raised (and hopefully already resolved the
insert array issue, don't know if you had time to test..) and will do my
best to stick everything in before v1.0
joe

On Sun, Oct 12, 2014 at 11:28 PM, vegarringdal [email protected]
wrote:

Maybe you should consider adding "stamp" also to every entity in a
collection.
Everytime you update 1 entity, the stamp goes up 1.
This way 1 user will know if data have been changed since he got it,
edited and tried to add it.
on update there could be a extra param u add if you wanted it to check if
stamp is the same. (true/false)
By using this you could get a error telling stamp does not match.
This could be practical for in use with workers etc.

—
Reply to this email directly or view it on GitHub
#6 (comment).

from lokijs.

hmm, I see the issue. (the insert array issue worked fine, thx)
Revisions could be usefull, but it will make it the database grow fast in size, maybe this could a option, not something you force on the user.

Well for the sync of the database you will need a way to lock it.
For the offline copy you will need to set a state of the database.

Just thinking out laud, you prob have a better way of solving this.
But if I write down ides, then you might get a better one. (Ive never worked on syncing a database)

Database:
localState:(attribute)
*attribute that is true or false depending if its a offlinemode or server side.

lockedState: (attribute)
*true/false

exportOfflineCopy: (function)
*function to create a copy oif database, this also sets the localState to true.

syncOfflineCopy(function(localCopy)
*function that check if database if is lockedState === false, and that localCopy is not localState === true, if it is not, then throw error
*If not locked, it sets it to locked, takes a local copy and parses it, it only updates the _localModified ones. and adds the "localNew" that is true.
If localdatabase have a expire date, then it would need to check for this too.

Entity:
id: (attribute)
*the one you have now

_localModified: (attribute, bool)
*Is set to true if modified and localState of database is true, after stamp is adjusted

_localNew:(attribute, bool)
*is set to true if its created in localState === true

_stamp: (attribute, num/int)
*if _localNew === true, it set stamp to 1, and _localModified to true
*if _localCopy === true , and _localModified=== false it goes up 1
*if _localCopy === false it goes up 1
*if _localCopy === true , and _localModified=== true and, it stays where it is.

Issues, should a local copy of database have a expire date/time stamp, to avoid to old data?

Should a local copy be able to add new collections, making sync more complicated.

How do you want to solve stamp issues?
*Throw error and pass back entities that was not OK, and make user verify changes?
This one is tricky, maybe the could be a own collection type,called ConcurrencyCollections.
In the ConcurrencyCollection the entities that threw error, put in a collection of same name, like a image/close of that entity on server
The you should have some function for get a array of the issueOnes from concurrentcyCollection, and a simple function to get your data.
This way user could decide what he wanted to keep, or if he just wanted to keep new data
Maybe a option in the syncOfflineCopy, to keep the newest data, and just add the ones without stamp errors..

this simple suggestion/ ideas got long now, so better stop

from lokijs.

Hi,

lots of food for thought there!
Let me go through your mail in detail and i'll try to form an idea of how
to implement this, hopefully by tomorrow eevning.
I think you make a good point in that there should be some form of opt-in
for revisions and stamps.
Revisions for example should become relevant only in a replicated
environment, otherwise stamps will do.

The two major concerns are db size and performance, in that these two
criteria are the rationale behind lokijs: a really fast in-memory database.
So for as long as the db-size doesn't grow too much and the performance on
inserts, updates and reads is fast then all changes and improvements are
welcome!
Will tackle this issue tomorrow, in the meantime thank you for the
suggestions.

On Mon, Oct 13, 2014 at 7:12 PM, vegarringdal [email protected]
wrote:

hmm, I see the issue. (the insert array issue worked fine, thx)
Revisions could be usefull, but it will make it the database grow fast in
size, maybe this could a option, not something you force on the user.

Well for the sync of the database you will need a way to lock it.
For the offline copy you will need to set a state of the database.

Just thinking out laud, you prob have a better way of solving this.
But if I write down ides, then you might get a better one. (Ive never
worked on syncing a database)

Database:
localState:(attribute)
*attribute that is true or false depending if its a offlinemode or server
side.

lockedState: (attribute)
*true/false

exportOfflineCopy: (function)
*function to create a copy oif database, this also sets the localState to
true.

syncOfflineCopy(function(localCopy)
*function that check if database if is lockedState === false, and that
localCopy is not localState === true, if it is not, then throw error
*If not locked, it sets it to locked, takes a local copy and parses it, it
only updates the _localModified ones. and adds the "localNew" that is true.
If localdatabase have a expire date, then it would need to check for this
too.

Entity:
id: (attribute)
*the one you have now

_localModified: (attribute, bool)
*Is set to true if modified and localState of database is true, after
stamp is adjusted

_localNew:(attribute, bool)
*is set to true if its created in localState === true

_stamp: (attribute, num/int)
*if _localNew === true, it set stamp to 1, and _localModified to true
*if _localCopy === true , and _localModified=== false it goes up 1
*if _localCopy === false it goes up 1
*if _localCopy === true , and _localModified=== true and, it stays where
it is.

Issues, should a local copy of database have a expire date/time stamp, to
avoid to old data?

Should a local copy be able to add new collections, making sync more
complicated.

How do you want to solve stamp issues?
*Throw error and pass back entities that was not OK, and make user verify
changes?
This one is tricky, maybe the could be a own collection type,called
ConcurrencyCollections.
In the ConcurrencyCollection the entities that threw error, put in a
collection of same name, like a image/close of that entity on server
The you should have some function for get a array of the issueOnes from
concurrentcyCollection, and a simple function to get your data.
This way user could decide what he wanted to keep, or if he just wanted to
keep new data
Maybe a option in the syncOfflineCopy, to keep the newest data, and just
add the ones without stamp errors..

this simple suggestion/ ideas got long now, so better stop

—
Reply to this email directly or view it on GitHub
#6 (comment).

from lokijs.

would lamport timestamps help here?

btw: interesting project. Kudos!

from lokijs.

very interesting. It would definitely help in a distributed version of
LokiJS, which is on our roadmap. If you have any suggestions with regards
to a concrete implementation of the algorithm i'm all ears/eyes :)

On Wed, Nov 5, 2014 at 1:33 AM, Geert-Jan Brits [email protected]
wrote:

would lamport timestamps http://en.wikipedia.org/wiki/Lamport_timestamps
help here?

btw: interesting project. Kudos!

—
Reply to this email directly or view it on GitHub
#6 (comment).

from lokijs.

Closing the issue as benchmark are in the source tree, i'm opening a lamport timestamps issue separate to this.

from lokijs.