Comments (11)
Is there a database that's faster then leveldb that we should be benchmarking against?
Maybe using leveldb directly with C++ without leveldown. Maybe one of those fast writing databases, Maybe redis.
from community.
That would be good for demonstration purposes, like, "should I use leveldb for this?".
What I have in mind is to benchmark our own features. When we create a suite that can be run by every database out there, we won't be able to benchmark batch vs. writeStream.
from community.
Your idea with benchmarking against leveldb in pure C is interesting though. The last time I did this leveled was about 30% slower then pure C which is very good.
from community.
@juliangruber I started on a benchmark repo. All it does atm is to use npm programatically to download all released versions of LevelUP. If you want I could push it to github and we can start from there.
from community.
I'm also not keen on using it to compare against other engines. The most important thing is to measure levelup against itself, so we can notice if anything has happened to the performance. Once we have that up and running, we can start comparing against other engines.
from community.
I guess there are 2 reasons to have benchmarks. We've focused mainly on improving performance over time and avoid performance regressions; this is obviously the most important type of benchmark. But there's also benchmarks that we may want to share with the public, perhaps even on the README--benchmarks against alternative database types.
See this nice page from LevelDB: http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html
The reason I included SQLite3 was because it's a clear alternative choice and it's worth letting people know how it performs. For the record it performs very badly and I suspect the Node wrapper implementers need to address some issues because I don't think it should perform quite as badly as it does; but this information is valuable for people making a choice.
I'm not really sure what other alternatives there are in npm. Perhaps we should benchmark against a JSON file store some how. Maybe we could grab the npm cache file and make a LevelUP version and test reads/searches? It doesn't look like anyone's done a Kyoto wrapper. There's also BerkelyDB that's a contender.
Apparently the only similar embedded store that outperforms LevelDB for standard reads is OpenLDAP Lightning. Although some work has gone into LevelDB since those benchmarks were done I think; particularly the bloom filter (which I think the Basho guys contributed).
Plus, wouldn't it be fun to compare against external storage systems? Perhaps MySQL, Riak & Redis? Those numbers/graphs would be fun and help people make informed choices.
from community.
Once again @rvagg has convinced me otherwise with his wisdom :) The tests need to be engine agnostic (which we already I have I guess, haven't gone through the current benchmark code yet) and they also have to be run in the same environment.
Obviously we can use LevelUP to store the benchmark results, serve the data on the wire and put a nice interface on top of it with some cute graphs etc.
from community.
Oh, and btw, benchmarks against raw LevelDB shouldn't be too hard now that I have the secret sauce to compile against the LevelDOWN binary & includes. It's just a matter of coming up with the test cases and then they can be turned into C++ versions. I'd like to see those numbers, only 30% slower is awesome! I guess that's where Node's multi-threading comes in to play. I wonder if we can improve on that?
from community.
So we're cool with 2 benchmark suites?
- bench every levelUp feature and compare between levelUp versions
- compare levelUp with levelDB, redis, mysql, couch, and more. Here we can/should have both basic things and more real-life use cases. Like get and put but also "give me all the comments that belong to post X" and "fully replicate the database".
from community.
Yep, sounds good.
On 8 March 2013 19:58, Julian Gruber [email protected] wrote:
So we're cool with 2 benchmark suites?
- compare every levelUp feature between levelUp versions
- compare levelUp with levelDB, redis, mysql, couch, ...
—
Reply to this email directly or view it on GitHubhttps://github.com/rvagg/node-levelup/issues/96#issuecomment-14637461
.
from community.
So we're cool with 2 benchmark suites?
- bench every levelUp feature and compare between levelUp versions
- compare levelUp with levelDB, redis, mysql, couch, and more. Here we can/should have both basic things and more real-life use cases. Like get and put but also "give me all the comments that belong to post X" and "fully replicate the database".
Splitting this up:
- Move
levelup
benchmarks tolevel-bench
🆕 (Level/levelup#652) - Add benchmarks for other stores (done, as
level-bench
takes any) - Add (ability to compare against) C++ benchmarks (Level/bench#1)
from community.
Related Issues (20)
- Proposal: add map method to abstract-down HOT 4
- Add `db.getMany(keys)` across the board HOT 8
- Refactor encodings HOT 6
- `rocksdb`: to be ported from `leveldown` (after other recent PRs)
- Fix Typings on DefinitelyTyped HOT 7
- Proposal: Add `db.has(key)` and `db.hasMany(keys)` HOT 12
- Deprecate old modules
- Package level with electron HOT 2
- Redisdown: how to create new `level-` libs? HOT 1
- Replace Sauce Labs with Playwright HOT 3
- Tracking issue: implicit and explicit snapshots
- willing to help revive some databases with abstract-level api HOT 4
- rocks-level implementation HOT 6
- Switch to dependabot HOT 9
- Dump Buffer for TypedArrays (for compactness and efficiency) HOT 6
- Any interest in maintaining 'lmdb'? HOT 3
- Counting entries in a level database HOT 4
- leveldown to remote database HOT 1
- Maintenance round: drop legacy features & runtime environments HOT 1
- Move to GitHub Actions HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from community.