Comments (12)
Until more folks express the need for this feature (give it a thumbs up if you do) this can be implemented in userland. Similar to level-exists
but optimized to use an iterator instead of a stream, like @juliangruber's example above.
It might also help to set the keyAsBuffer
option to false (or keyEncoding
to utf8
) depending on which modules you're using - leveldown
for example is typically faster at fetching strings than buffers.
If your values are small (and thus fast to copy from the underlying store) I reckon that db.get(key)
is faster than an iterator though, because it's a single call, while an iterator needs 3 calls (creating the iterator, then nexting, then ending).
For hasMany(keys)
you could perhaps use iterator.seek()
but that'd require some tinkering with options to get the performance right. That API would benefit more from being a builtin feature, because the main cost is in crossing the JS/C++ boundary for every key (rather than just once for all keys).
from community.
Yes, it would undoubtedly perform better. Needs upvotes because new features come with a maintenance cost. Other than that I have no objections; your proposal aligns well with existing features (I like the symmetry with get()
and getMany()
).
from community.
I think currently the most performant way is this:
let exists = false
const it = db.iterator({
values: false,
gte: key,
lte: key,
limit: 1,
})
for await (const _ of it) {
exists = true
}
It's not a nice way though.
from community.
Re: keys vs values, see also Level/abstract-leveldown#380 which proposes key-only and value-only iterators.
from community.
I can answer question 1: in the past, iterators weren't directly exposed (and async iterators weren't a thing) - level was all about streams
from community.
- This package only offers the callback API, if I understand the code correctly. Do you plan to extend it to also include the
Promise
API?- Do you plan to extend it to include an
existsMany(keys)
?
Not at this time but pull requests welcome 👍 Feel free to open an issue there
from community.
@vweevers might be able to tell you more about any ongoing work on this
from community.
I think currently the most performant way is this: ...
Thanks for this alternative idea. I'd think, though, that in most cases this would also fetch the value from the underlying store, because the (const _ of it)
loop would still call it.next()
a single time, which would populate { done: true/false, value: ... }
. I think it's equivalent to
const it = db.iterator({
values: false,
gte: key,
lte: key,
limit: 1,
})
return !it.next().done;
so even though we don't read it.next().value
, it probably would still be fetched by most, if not all, iterator implementation from the underlying store.
from community.
With values: false it should only read keys. You are right that the iteration mechanics can be changed
from community.
Ah yes, sorry, I oversaw the values: false
setting 🙈
from community.
Just looked at https://github.com/juliangruber/level-exists and realize that it really is my db.has
implemented in userland (but using streams). Three questions for @juliangruber:
- Why do you use
createKeyStream
if iterators are more performant? - This package only offers the callback API, if I understand the code correctly. Do you plan to extend it to also include the
Promise
API? - Do you plan to extend it to include an
existsMany(keys)
?
from community.
@vweevers I understand that level-exists
basically implements my proposal in userland. Wouldn't you agree, though, that for some leveldown implementations, having them directly implement has
and hasMany
could result in better performance? It seems to me that going the detour over a db.iterator({ values: false, gte: key, lte: key, limit: 1, })
would still incur some costs as opposed to a very simple db.has
implementation. And, as you said yourself, there isn't really a well performing implementation of hasMany
possible in userland.
Anyway, looking forward to other community members' feedback. Maybe I'm not alone in need of this feature :)
from community.
Related Issues (20)
- Proposal: add map method to abstract-down HOT 4
- Add `db.getMany(keys)` across the board HOT 8
- Refactor encodings HOT 6
- `rocksdb`: to be ported from `leveldown` (after other recent PRs)
- Fix Typings on DefinitelyTyped HOT 7
- Deprecate old modules
- Package level with electron HOT 2
- Redisdown: how to create new `level-` libs? HOT 1
- Replace Sauce Labs with Playwright HOT 3
- Tracking issue: implicit and explicit snapshots
- willing to help revive some databases with abstract-level api HOT 4
- rocks-level implementation HOT 6
- Switch to dependabot HOT 9
- Dump Buffer for TypedArrays (for compactness and efficiency) HOT 6
- Any interest in maintaining 'lmdb'? HOT 3
- Counting entries in a level database HOT 4
- leveldown to remote database HOT 1
- Maintenance round: drop legacy features & runtime environments HOT 1
- Move to GitHub Actions HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from community.