Comments (8)
Could you check if there are any xadd
operations being performed on the destination Redis instance? These operations might be causing the issue you’re encountering.
from redisshake.
right there are XADD operations being performed. Does it mean that RedisShake can't sync those?
from redisshake.
my bad, I did not read your question carefully. XADD operations are only performed on the source Redis, not destination. I found that you create Redis Stream with "0-1" #808. I wonder if that is causing this issue.
from redisshake.
I am first performing XREAD command with "0-0" and then XADD uses appended ID "0-1" (not set by default)
Sometimes XADD happens before XREAD
from redisshake.
I can't think of a reason for now.
from redisshake.
So I went ahead and deleted the stream from src and dest, restarted RedisShake and error changed:
2024-05-30 02:39:52 INF load config from file: shake.toml
2024-05-30 02:39:52 INF log_level: [info], log_file: [/usr/src/app/RedisShake/bin/data/shake.log]
2024-05-30 02:39:52 INF changed work dir. dir=[/usr/src/app/RedisShake/bin/data]
2024-05-30 02:39:52 INF GOMAXPROCS defaults to the value of runtime.NumCPU [32]
2024-05-30 02:39:52 INF not set pprof port
2024-05-30 02:39:52 INF create SyncStandaloneReader: dev-redis-replicas-0.dev-redis-headless.dev-redis:6379
2024-05-30 02:39:52 INF connecting to master node at dev-redis-ha-node-0.dev-redis-ha-headless.dev-redis-ha.svc.cluster.local:6379
2024-05-30 02:39:52 INF create RedisSentinelWriter: dev-redis-ha.dev-redis-ha:26379
2024-05-30 02:39:52 INF not set status port
2024-05-30 02:39:52 INF start syncing...
2024-05-30 02:39:52 ERR [writer_dev-redis-ha-node-0.dev-redis-ha-headless.dev-redis-ha.svc.cluster.local_6379] receive reply failed. cmd=[xadd nlp:sync:output:4bcce72b-cb24-4b54-949e-867a9f769364:stream MAXLEN 0 0-1 x y], error=[ERR The ID specified in XADD is equal or smaller than the target stream top item]
RedisShake/internal/writer/redis_standalone_writer.go:116 -> (*redisStandaloneWriter).processReply()
runtime/asm_amd64.s:1695 -> goexit()
before: receive reply failed. cmd=[xadd nlp:sync:output:4bcce72b-cb24-4b54-949e-867a9f769364:stream 1709602510226-0
after: receive reply failed. cmd=[xadd nlp:sync:output:4bcce72b-cb24-4b54-949e-867a9f769364:stream MAXLEN 0 0-1 x y]
from redisshake.
Try using the function
to filter all stream class commands. https://tair-opensource.github.io/RedisShake/en/function/best_practices.html#%E8%BF%87%E6%BB%A4%E6%9F%90%E7%B1%BB%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84
from redisshake.
Thank you, that's a good temporary fix
from redisshake.
Related Issues (20)
- 自建Redis与云Redis同步异常 HOT 2
- RedisShake 3.1.11 从4.0.11同步到6.0.14,key数量对不上 HOT 2
- RedisShake 4.x支持断点续传吗 HOT 2
- 使用redis-shake做增量数据同步的时候,有很多warn的信息,我不确认会不会因为这些问题导致我数据不一致 HOT 2
- 同步数据较慢 HOT 3
- 通过nat转换进行同步报 i/o timeout HOT 1
- scan_reader模式不能直接同步增量? HOT 3
- can_reader模式同步过程中报:ERR unexpected EOF HOT 1
- redis6同步支持 HOT 1
- 4.05版本使用sync_reader模式连接阿里云Redis集群版直连地址报错 HOT 15
- RedisShake 4.x 初次单机迁移到集群Key丢失以及scan数量与keyspace数量差异问题 HOT 6
- 最新的4.10版本,设置了scan = false不起作用 HOT 3
- ERR NOAUTH Authentication required [redis_writer] sentinel=true HOT 3
- sync_rdb设为false之后依然拉取全量rdb文件发给slave HOT 4
- 使用codis->的时候少key HOT 1
- scan_reader模式下,如何降低对源端的压力 HOT 3
- scan_reader模式下,无法同步被删除的key HOT 1
- SCAN_READER模式下,设置prefer_replica = true报错
- 同步过程中经常遇到 “ERR write tcp xxxx>xxxx : write:broken pipe” HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from redisshake.