Comments (9)
So currently no snapshot is used whilst reading and you get an inconsistent view (i.e. puts after you start reading reflect in your results)
from community.
actually, sorry, it's NULL
by default so any of the ReadStreams will be over a consistent view of the database created by an implicit snapshot.
from community.
I think this is off by default,
I have a test where a readStream starts in the first tick,
and then later that tick I do some puts into that read's range.
The puts come out in the read stream.
I think when it says "implicit snapshot" it's just referring to an implementation detail.
It's not providing a consistent view.
from community.
OK, well we need tests for snapshots anyway so a good start would be a test for the current implementation to see what it's actually doing. I'll try and get to that soon.
from community.
See: Level/levelup@40dbde6#diff-3
You'll need to update your slow-stream in your devDeps to make this one run properly.
Basically the test adds in a bunch of data, then creates a readStream()
and then goes ahead and overwrites all the original data with new cruft. The stream is piped through a SlowStream
to ensure that it's throttled to be slower than it takes the overwrite to happen.
The test demonstrates that an "implicit snapshot" indeed means that a snapshot is being created to handle the iterator even without being told to do so. You can verify this further with some debug prints on stream 'data'
and the return from the second batch()
which is done with {sync:true}
to be absolutely sure. You can also increase the rate of the SlowStream
to something even higher, say 50ms between 'data'
events and it should still pass. (And yes, SlowStream
does apply back-pressure, I even did some debugging from the C++ layer to ensure that the iterator was being called much slower than it took the batch to happen).
Got any better ideas for a test case on this?
Also on that commit you'll see the basics of some snapshot work on the C++ layer, it should be fairly simple to implement but I need some time to make a nice JS interface and some decent tests for it.
P.S. if you ever get failed tests, then run again and get passed tests, it's probably a database not being deleted properly from the test/
directory. I really need to do a rimraf()
on the dir before it's used again just to make sure..
from community.
I'm +1 on snapshots exposed in levelup
from community.
Still no support for snapshots?
from community.
see #138
marked as "help wanted", it's mostly specced out if someone wants to tackle it but for now I think this should be mostly kept to leveldown rather than being exposed here in levelup
from community.
Tracked in #118.
from community.
Related Issues (20)
- Proposal: add map method to abstract-down HOT 4
- Add `db.getMany(keys)` across the board HOT 8
- Refactor encodings HOT 6
- `rocksdb`: to be ported from `leveldown` (after other recent PRs)
- Fix Typings on DefinitelyTyped HOT 7
- Proposal: Add `db.has(key)` and `db.hasMany(keys)` HOT 12
- Deprecate old modules
- Package level with electron HOT 2
- Redisdown: how to create new `level-` libs? HOT 1
- Replace Sauce Labs with Playwright HOT 3
- Tracking issue: implicit and explicit snapshots
- willing to help revive some databases with abstract-level api HOT 4
- rocks-level implementation HOT 6
- Getting no entry found error in Chrome when using .get("key") HOT 2
- Dump Buffer for TypedArrays (for compactness and efficiency) HOT 6
- Any interest in maintaining 'lmdb'? HOT 3
- Counting entries in a level database HOT 4
- leveldown to remote database HOT 1
- Maintenance round: drop legacy features & runtime environments HOT 1
- Move to GitHub Actions HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from community.