youzan / zanredisdb Goto Github PK
View Code? Open in Web Editor NEWThis project forked from absolute8511/zanredisdb
Yet another distributed fault-tolerant key-value database Compatible with Redis written in Golang.
License: MIT License
This project forked from absolute8511/zanredisdb
Yet another distributed fault-tolerant key-value database Compatible with Redis written in Golang.
License: MIT License
Some import issues need to merge from rocksdb while fixed.
facebook/rocksdb#7405
tikv/rocksdb#195
tikv/tikv#8243
tikv/rocksdb#208
I'm apply redis-cluster in FeatureStore and model inference. Now I'm looking forward some disk-based KV store and I found ZanKV. Is there any good example of ZanKV applied to real scene? I recommend to create channel on slack or wechat group to share good ideas on this project? what do you think?
any work started in this area?
Searchable and Indexing
Secondary index support on Hash fields
Secondary index support for json kv
Full text search support
some patch can be merged from etcd:
raft: Introduce CommittedEntries pagination
etcd-io/etcd#10063
etcd-io/etcd#9982
etcd-io/etcd#10258
raft: don't allocate slice and sort on every commit
etcd-io/etcd#8689
raft: provide protection against unbounded Raft log growth
etcd-io/etcd#10167
etcd-io/etcd#10199
raft: Avoid scanning raft log in becomeLeader
etcd-io/etcd#9073
如题,请问有啥优化方案吗?多谢
求确认
mysql里有两个表,一个message,一个message_receive, 通过messageId关联,message有3亿条,message_receive有4亿,想处理后导入到zanRedisDB,有啥好建议吗
Support using the k8s operator to deploy a full cluster with the PersistentVolume
Make sure the snapshot restore is atomic and can be fast enough.
related:
tikv/tikv#2609
tikv/tikv#5911
tikv/tikv#8301
tikv/tikv#8408
tikv/tikv#1618
cockroachdb/cockroach#38932
cockroachdb/cockroach#16954
see etcd-io/etcd#8525 and etcd-io/etcd#11308
Support jepsen test to dig possible bugs for the distributed system.
/root/soft/rocksdb/librocksdb.a(db_impl.o): In function rocksdb::DBImpl::DBImpl(rocksdb::DBOptions const&, std::string const&, bool, bool)': /root/soft/rocksdb/./util/compression.h:275: undefined reference to
ZSTD_versionNumber'
PORTABLE=1 USE_SSE=1 USE_PCLMUL=1 WITH_JEMALLOC_FLAG=1 JEMALLOC=1 make static_lib
$DEBUG_LEVEL is 0
GEN util/build_version.cc
$DEBUG_LEVEL is 0
GEN util/build_version.cc
CC cache/clock_cache.o
CC cache/lru_cache.o
CC cache/sharded_cache.o
CC db/builder.o
In file included from db/builder.cc:16:
In file included from ./db/compaction/compaction_iterator.h:13:
In file included from ./db/compaction/compaction.h:11:
In file included from ./db/version_set.h:31:
In file included from ./db/column_family.h:18:
In file included from ./db/table_cache.h:25:
In file included from ./table/table_reader.h:14:
./table/get_context.h:91:3: error: explicitly defaulted default constructor is implicitly deleted [-Werror,-Wdefaulted-function-deleted]
GetContext() = default;
^
./table/get_context.h:168:18: note: default constructor of 'GetContext' is implicitly deleted because field 'tracing_get_id_' of const-qualified
type 'const uint64_t' (aka 'const unsigned long long') would not be initialized
const uint64_t tracing_get_id_;
^
In file included from db/builder.cc:19:
./db/internal_stats.h:112:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
WAL_FILE_BYTES,
^
./include/rocksdb/statistics.h:196:3: note: previous declaration is here
WAL_FILE_BYTES, // Number of bytes written to WAL
^
In file included from db/builder.cc:19:
./db/internal_stats.h:113:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
WAL_FILE_SYNCED,
^
./include/rocksdb/statistics.h:195:3: note: previous declaration is here
WAL_FILE_SYNCED, // Number of times WAL sync is done
^
In file included from db/builder.cc:19:
./db/internal_stats.h:114:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
BYTES_WRITTEN,
^
./include/rocksdb/statistics.h:131:3: note: previous declaration is here
BYTES_WRITTEN,
^
In file included from db/builder.cc:19:
./db/internal_stats.h:115:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
NUMBER_KEYS_WRITTEN,
^
./include/rocksdb/statistics.h:124:3: note: previous declaration is here
NUMBER_KEYS_WRITTEN,
^
In file included from db/builder.cc:19:
./db/internal_stats.h:116:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
WRITE_DONE_BY_OTHER,
^
./include/rocksdb/statistics.h:201:3: note: previous declaration is here
WRITE_DONE_BY_OTHER, // Equivalent to writes done for others
^
In file included from db/builder.cc:19:
./db/internal_stats.h:117:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
WRITE_DONE_BY_SELF,
^
./include/rocksdb/statistics.h:200:3: note: previous declaration is here
WRITE_DONE_BY_SELF,
^
In file included from db/builder.cc:19:
./db/internal_stats.h:118:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
WRITE_WITH_WAL,
^
./include/rocksdb/statistics.h:203:3: note: previous declaration is here
WRITE_WITH_WAL, // Number of Write calls that request WAL
^
8 errors generated.
make: *** [db/builder.o] Error 1
比如一台物理服务器跑了8个zankv(8个分区),就要有8个rsync --daemon
多谢!
如题
Instead of using the default RAM based Raft Storage (MemoryStorage), build one based on Badger or Rocksdb. This should significantly decrease the RAM usage by the Raft write-ahead log.
To keep compatible with old WAL version, we first just replace the MemoryStorage and keep the
old WAL in raft lib.
能是什么原因
如题
Currently, most raft libraries only check the commit index while doing the leader transfer. Some other cases should also be considered to avoid stale/slow read after transfer such as the snapshot status, the applied index, and the pending configure change.
related fix:
tikv/tikv#6539
tikv/tikv#8051
tikv/tikv#3878
cockroachdb/cockroach#63507
etcd-io/etcd#12163
etcd-io/etcd#12134
etcd-io/etcd#10822
请问瓶颈是在pd吗?
please provide some json usage, thank u !
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.