DISCUSSION: My experience with LokiJS, especially with LokiJS ids and synchronous methods about lokijs HOT 7 CLOSED

Hi Jacques,

thank you very much for the feedback, precious and relevant, it's great to see LokiJS being battle-tested.

Your comments also come in with perfect timing as we've been discussing some of the issues, particularly ids.
I also like very much the idea of a plugin system for persistence.

Just as a note, we never claimed LokiJS to have a MongoDB-compatible API (just "similar"), but we are also seeing this as a recurring request and much wanted feature. I'm quite happy to have at least a subset of the MongoDB api implemented in LokiJS (i like mongodb anyway), some methods may be just facading other loki-native methods, while others may have to be developed from scratch. If there are methods you would absolutely like to see implemented (maybe less common stuff like projections, or writeconcerns implemented as promises or whatever..) please let us know and we'll see how it fits in LokiJS's current roadmap.

I won't reiterate obeliskos's (impeccable) points, id conflict has proved a sore point so it's being addressed as a high priority, and for that reason I opened an issue #30

I also agree to make async an opt-in feature, and default to sync, that also warranted an issue: #31
Main design decision for that was gaining speed on inserts and updates, as meta-data has no urgency, just as a reference.

Finally, about the future of LokiJS: while I started the project to obviate the need for a user-friendly db in a cordova context, LokiJS received attention for both mobile and browser environments. In particular it seems the ability to store subsets of data for a work session from a remote db and sync back (periodically or at the end of the session) seem to be the de-facto sweet spot of LokiJS, so those environments will receive the necessary care to make LokiJS shine as the best product available in the niche.
Funny because i love the idea of LokiJS in node, but i seem to be quite lonely in that camp.

Thanks again, Joe

from lokijs.

Interesting post. We can possibly rename our id column to avoid clashing with existing id columns. You will have to manually apply indexes to your own id columns (or originalId for now). If you apply an index to originalId (for now), you will gain significant performance increases on find() ops against that column.

As for synchronicity, I will defer to Techfort but at least while feature set is being worked out, leaving synchronous makes it much easier to benchmark, test, and debug. Maybe for certain key functions we could support async as option.

Promises may add polyfill dependencies (for now) but in theory I like the idea and would definitely like to support it once it becomes native to node / ecma 6 browsers. I will start looking into level of effort for whenever/if we decide to add support for that.

from lokijs.

@obeliskos How can I create an index on originalId? without digging into LokiJS code since ensureIndex does not seem to take any parameter.

@techfort Look at localforage, it might give you some interesting ideas about plugins and callbacks/promises.

from lokijs.

Jacques, to ensure an index you can either pass it in the collection constructor (i.e. new Collection('coll', ['originalId']) or by calling collection.ensureBinaryIndex(property, force), where force is a boolean forcing index rebuild pretty much at every change in the collection. That said we are going to address the issue.

Quick question for you: should the ids problem and the syncing issues be resolved (they are due to be included in the December release, v1.1.0), would LokiJS suit your use case? If you feel the conversation may go off-topic feel free to drop me a line at [email protected]
Thanks

from lokijs.

The multiple ensure methods is an artifact of our refactoring indexes several months ago. We had reduced the scope of ensureIndex() to just the id index and added a new ensureBinaryIndex for refactored indexes.

Since the old ensureIndex is only used internally, I renamed it to ensureId (which you should not need to use), and renamed ensureBinaryIndex to ensureIndex. This change is in 1.1.0 branch. Once we release 1.1.0 hopefully joe will update the lokijs.org documention to describe all methods.

Since our indexes are normally lazily built (on first query against that column), the force param allows you to built immediately instead. This param may eventually be rolled into an options object.

from lokijs.

From gleaning mongodb documentation, asynchronous methods are provided only for insert/update/delete methods. Any preference on if that is the expected behavior this or should we support it on find(), findOne(), and chained data() calls as well?

As for Promises I am trying to determine the best use cases for this currently can think of two situations where this might be beneficial :

• transactional situations where you want future then action executed only on transaction commit
• a more syntactically elegant way expressing async callbacks

If anyone would expand on their what their expectations of promise behavior might be within loki feel free to elaborate.

While implementations of promises might still be several versions down the road, it might be helpful sooner in considerations of transaction and async callback support.

from lokijs.

Many features related to this were implemented (sync calls, mongo compatibility etc.). Closing this, if there are specific issues not addressed in this discussion please feel free to open new issues!

from lokijs.