Comments (5)
FYI
fast and efficient parallelized comparison of redis databases
from redis-rdb-cli.
Hi
After migration.
first run info keyspace
to compare datasets.
second open file /path/to/redis-rdb-cli/log/redis-rdb-cli.log
and check if it contains failure
text.
if yes. following is the reason of failures.
# Key with expired time that already expired in migration.
# Rdb file has stored expired time by timestamp. so all nodes need NTP to sync time.
failure[expired]...
# Since redis-rdb-cli 0.9.3 not release yet
# Target redis respond an error and key migration failed.
failure[respond]...
# Connect to target redis failed. so key migration failed.
failure[failed]...
# Keys are not in the same slot in cluster migration. used in `rst` command
# Example: `del {slot1}:user1 {slot2}:user2`
failure[slot]...
# Some incremental commands are not supported in cluster migration. used in `rst` command
# `publish` `swapdb` `move` `flushall` `flushdb` `multi` `exec` `script flush` `script load` `eval` `evalsha`
failure[unsupported]...
from redis-rdb-cli.
Is there a way to compare somehow all the data, like shasum
of RDB/AOF or something similar?
from redis-rdb-cli.
Hi
-
User scenario issues
Compare all data need to scan source and target redis.
But after migration or in migration. user may modify that data and key expiration may not be very accurate.(NTP)
So most of the time the data in the these two redis may not be consistent. -
Implementation issues
Implement a comparison tool need to load source key and the same key in target to memory. if the key is very big. that will consume much memory in comparsion process.
second reason: in different redis. use a different structure to store key value. for example: redis 7 uselistpack
to store small map key. but in redis 6 useziplist
to store small map. so compare small map key they may have differentfield
order even though the two maps are actually equal. so comparision tool need to compare every filed in this small map.
So it is not applicable to implement such a comparison tool in a small usage scenario.
from redis-rdb-cli.
Thank you for the clarifications, so as for now we have just methods you described
info keyspace
redis-rdb-cli/log/redis-rdb-cli.log
- Check some random Redis keys on source and destination
from redis-rdb-cli.
Related Issues (20)
- merge two rdb files with repeated key HOT 2
- Problem installing HOT 1
- password can't have "%" HOT 1
- NOAUTH despite authPassword and/or ACL style on Redis 6.2.6 HOT 4
- Missing required option: '--source <source>' HOT 3
- Redis remote backup import failing HOT 19
- Feature Request: rmt sync from cluster to single redis HOT 1
- java.lang.RuntimeException: java.lang.NumberFormatException: null HOT 6
- expiry is not right HOT 2
- More information about applications HOT 4
- Sync data between KeyDB v6 - Redis v7 HOT 26
- Terminating due to java.lang.OutOfMemoryError: Java heap space HOT 7
- Bloom keys not synced to target redis instance HOT 1
- rst's password maybe can't contain # HOT 2
- Adding any -k parameter returns an empty list HOT 4
- Publish to maven central HOT 5
- Compatibility of redis-rdb-cli with Redis and KeyDB versions HOT 2
- How can I migrate Redis A db0 to redis B db2 HOT 13
- --largest doesn't work HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from redis-rdb-cli.