Check migrated data consistency about redis-rdb-cli HOT 5 CLOSED

FYI
fast and efficient parallelized comparison of redis databases

from redis-rdb-cli.

Hi
After migration.
first run info keyspace to compare datasets.

second open file /path/to/redis-rdb-cli/log/redis-rdb-cli.log and check if it contains failure text.

if yes. following is the reason of failures.

# Key with expired time that already expired in migration. 
# Rdb file has stored expired time by timestamp. so all nodes need NTP to sync time.
failure[expired]...

# Since redis-rdb-cli 0.9.3 not release yet
# Target redis respond an error and key migration failed.
failure[respond]...

# Connect to target redis failed. so key migration failed.
failure[failed]...

# Keys are not in the same slot in cluster migration. used in `rst` command
# Example: `del {slot1}:user1 {slot2}:user2`
failure[slot]...

# Some incremental commands are not supported in cluster migration. used in `rst` command
# `publish` `swapdb` `move` `flushall` `flushdb` `multi` `exec` `script flush` `script load` `eval` `evalsha`
failure[unsupported]...

from redis-rdb-cli.

Is there a way to compare somehow all the data, like shasum of RDB/AOF or something similar?

from redis-rdb-cli.

Hi

1. User scenario issues
   Compare all data need to scan source and target redis.
   But after migration or in migration. user may modify that data and key expiration may not be very accurate.(NTP)
   So most of the time the data in the these two redis may not be consistent.

2. Implementation issues
   Implement a comparison tool need to load source key and the same key in target to memory. if the key is very big. that will consume much memory in comparsion process.
   second reason: in different redis. use a different structure to store key value. for example: redis 7 use listpack to store small map key. but in redis 6 use ziplist to store small map. so compare small map key they may have different field order even though the two maps are actually equal. so comparision tool need to compare every filed in this small map.

So it is not applicable to implement such a comparison tool in a small usage scenario.

from redis-rdb-cli.

Thank you for the clarifications, so as for now we have just methods you described

1. info keyspace
2. redis-rdb-cli/log/redis-rdb-cli.log
3. Check some random Redis keys on source and destination

from redis-rdb-cli.