Giter Club home page Giter Club logo

migration's Introduction

TiKV Migration Toolset

TiKV migration toolset is a collection of tools about:

migration's People

Contributors

daimashusheng avatar dependabot[bot] avatar dveeden avatar haojinming avatar kezhenxu94 avatar liangyouze avatar marsishandsome avatar meecreeps avatar mingyang91 avatar peng1999 avatar pingyu avatar renkai avatar rleungx avatar tisonkun avatar zeminzhou avatar zz-jason avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

liangyouze tisonkun zz-jason marsishandsome renkai mingyang91 sabaping peng1999 daimashusheng pingyu purelind rleungx laashub-soa haojinming zeminzhou yongman nchuxyz amoebaprotozoa dveeden guolianwei jjz921024 xiabee zhuguoliang0109 smityz sunxiaojian imsjt

migration's Issues

TiKV-BR: support RawKV API V2 & Cross API version migration

Feature Request

Is your feature request related to a problem? Please describe:

  1. BR for RawKV is an experimental feature, not for production env.
  2. No backup & restore tool for RawKV API V2.
  3. No migration tool between RawKV API V1/V1TTL to V2.

Describe the feature you'd like:

  1. Support backup & restore for RawAPI V2.
  2. Support data migration from API V1/V1TTL to V2.

Describe alternatives you've considered:

N/A.

Teachability, Documentation, Adoption, Migration Strategy:

TBD.

CDC & BR CI failed

Bug Report

1. Describe the bug

CDC & BR CI failed (see here):

  • br-gosec #68
  • br-lint #70
  • br-ut
  • cdc-gosec
  • cdc-lint
  • cdc-ut

2. Minimal reproduce step (Required)

File a PR.

3. What did you see instead (Required)

All CI tasks are successful.

4. What did you expect to see? (Required)

Some are failed.

5. What is your migration tool and TiKV version? (Required)

  • TiKV: N/A
  • TiKV CDC: latest
  • TiKV BR: latest
  • TiKV Online Bulk Load: N/A

cdc cli command fail to detect wrong format of sink-uri

Bug Report

1. Describe the bug

The sink-uri shoudl not have the http:// prefix, but cli fail to detect or correct this automatically.

image

cli create changefeed successfully, but pd client is continuously failing in backgroud.

[WARN] [base_client.go:251] ["[pd] failed to get cluster id"] [url=http://http:] [error="[PD:client:ErrClientGetMember]error:rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup http on 172.16.4.1:53: no such host" target:http: status:TRANSIENT_FAILURE: error:rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup http on 172.16.4.1:53: no such host" target:http: status:TRANSIENT_FAILURE"]

2. Minimal reproduce step (Required)

bin/cdc cli changefeed create --pd http://172.16.5.32:2379 --changefeed-id feed016 --sort-engine unified --sink-uri='tikv://http://172.16.5.32:39244'

3. What did you see instead (Required)

4. What did you expect to see? (Required)

cdc cli command should detect the wrong format and fails, or correct the format automatically.

5. What is your migration tool and TiKV version? (Required)

  • TiKV CDC:

BulkLoad cannot work normally in case of large amount of data, all ingest failed

Bug Report

1. Describe the bug

A large number of failures in the later stage of task execution，all ingest are retry again and again, until failure.

2. Minimal reproduce step (Required)

datum: 500000000 keys, and 1KB per keys.
In the initial stage of the task, each partition can be imported normally and the import speed is within 3-5 seconds. However, in the later stage of the task, each task (partition) needs to be executed for 30-40 minute, and is in the failure and retry stage. Finally, it fails.

3. What did you see instead (Required)

4. What did you expect to see? (Required)

5. What is your migration tool and TiKV version? (Required)

  • TiKV Online Bulk Load:

Upgrade to go1.18 in TiKV-BR

Feature Request

Is your feature request related to a problem? Please describe:
No problem.

Describe the feature you'd like:
Upgrade to go1.18 in TiKV-BR

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Initialize TiKV CDC / BR repo

Motivation

Initialize TiKV CDC / BR repo, to provide CDC feature for TiKV.
Related issue: tikv/tikv#11745
Related RFC: tikv/rfcs#86

Tasks

cdc: Add sorter back to pipeline

Bug Report

1. Describe the bug

The sorter of pipeline was removed in early cdc PR.
And that time we thought that as timestamps of entries in a region are always non-decreasing, we don't need the sorter.

But now I find that actually the processor is not processing only one region, but multiple regions at a time. And now the timestamps are not in order. This will make the resolved-ts logic wrong.
See here:

	keyspanID, resolvedTs := batch[i].keyspanID, batch[i].resolvedTs
	rawKVEntries := b.buffer[keyspanID]

	i := sort.Search(len(rawKVEntries), func(i int) bool {
		return rawKVEntries[i].CRTs > resolvedTs
	})
	if i == 0 {
		continue
	}
	state.metricTotalRows.Add(float64(i))

	err := b.Sink.EmitChangedEvents(ctx, rawKVEntries...)

If the CRTs of entries are not in order, we would not get all the expected entries that earlier than resolvedTs.

Although we can do a full search here, but it would be inefficiency. As the advancing of resolvedTs would be slow, we will scan again and again.

We can implement the sorter using merge sort, to utilize the feature of non-decreasing.

Please also note that, it's possible that we would break the order of entires timestamp in the same region later. See tikv/tikv#12020.

2. Minimal reproduce step (Required)

3. What did you see instead (Required)

4. What did you expect to see? (Required)

5. What is your migration tool and TiKV version? (Required)

  • TiKV:
  • TiKV CDC:
  • TiKV BR:
  • TiKV Online Bulk Load:

Ingest: peer is not leader 

Job aborted due to stage failure: Task 790 in stage 6.0 failed 4 times, most recent failure: Lost task 790.3 in stage 6.0 (TID 926, 10.36.30.17, executor 60): org.tikv.common.exception.GrpcException: message: "peer is not leader for region 8677, leader may Some(id: 8680 store_id: 7)"
not_leader {
  region_id: 8677
  leader {
    id: 8680
    store_id: 7
  }
}

 at org.tikv.common.importer.ImporterStoreClient.multiIngest(ImporterStoreClient.java:152)
 at org.tikv.common.importer.ImporterClient.ingest(ImporterClient.java:225)
 at org.tikv.common.importer.ImporterClient.write(ImporterClient.java:97)
 at org.tikv.bulkload.RawKVBulkLoader.writeAndIngest(RawKVBulkLoader.scala:164)
 at org.tikv.bulkload.RawKVBulkLoader.$anonfun$bulkLoad$5(RawKVBulkLoader.scala:110)
 at org.tikv.bulkload.RawKVBulkLoader.$anonfun$bulkLoad$5$adapted(RawKVBulkLoader.scala:109)
 at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2(RDD.scala:994)
 at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2$adapted(RDD.scala:994)
 at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2139)
 at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
 at org.apache.spark.scheduler.Task.run(Task.scala:127)
 at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:446)
 at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
 at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:449)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)

[TiKV-BR] Checksum mismatch report after backup

Bug Report

1. Describe the bug

checksum mismatch occurs after backup if one tikv-server crash during backup

2. Minimal reproduce step (Required)

  1. Load 2kw kv data to tikv cluster with more than 3 tikv-server nodes.
  2. Run tikv-br backup raw
  3. During backup running, kill one tikv-server process.
  4. Wait for backup finish.

3. What did you see instead (Required)

backup finish with success but checksum report failure.

4. What did you expect to see? (Required)

backup finish with success and checksum succeed.

5. What is your migration tool and TiKV version? (Required)

  • TiKV BR:

BulkLoadExample: More than Int.MaxValue elements error 

Exception in thread "main" java.lang.IllegalArgumentException: More than Int.MaxValue elements.
        at scala.collection.immutable.NumericRange$.check$1(NumericRange.scala:318)
        at scala.collection.immutable.NumericRange$.count(NumericRange.scala:328)
        at scala.collection.immutable.NumericRange.numRangeElements$lzycompute(NumericRange.scala:53)
        at scala.collection.immutable.NumericRange.numRangeElements(NumericRange.scala:52)
        at scala.collection.immutable.NumericRange.length(NumericRange.scala:55)
        at org.apache.spark.rdd.ParallelCollectionRDD$.slice(ParallelCollectionRDD.scala:145)
        at org.apache.spark.rdd.ParallelCollectionRDD.getPartitions(ParallelCollectionRDD.scala:97)
        at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:276)
        at scala.Option.getOrElse(Option.scala:189)
        at org.apache.spark.rdd.RDD.partitions(RDD.scala:272)
        at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
        at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:276)
        at scala.Option.getOrElse(Option.scala:189)
        at org.apache.spark.rdd.RDD.partitions(RDD.scala:272)
        at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
        at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:276)
        at scala.Option.getOrElse(Option.scala:189)
        at org.apache.spark.rdd.RDD.partitions(RDD.scala:272)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:2179)
        at org.apache.spark.rdd.RDD.count(RDD.scala:1227)
        at org.tikv.bulkload.RawKVBulkLoader.getRegionSplitPoints(RawKVBulkLoader.scala:205)
        at org.tikv.bulkload.RawKVBulkLoader.bulkLoad(RawKVBulkLoader.scala:82)
        at org.tikv.bulkload.example.BulkLoadExample$.main(BulkLoadExample.scala:82)
        at org.tikv.bulkload.example.BulkLoadExample.main(BulkLoadExample.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Release action support for tikv-br/tikv-cdc/gc-worker

Feature Request

Is your feature request related to a problem? Please describe:
Release action support for tikv-br/tikv-cdc/gc-worker

Describe the feature you'd like:
Release action support for tikv-br/tikv-cdc/gc-worker

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

br: Return the timestamp of backup to cooperate with TiKV-CDC

Feature Request

Is your feature request related to a problem? Please describe:

TiKV-BR is designed to initialize data before the incremental data replication of TiKV-CDC. Refer to RawKV Cross Cluster Replication (tikv/rfcs#86).

To cooperate with TiKV-CDC, the procedure is expected to be:

  1. TiKV-BR backup from upstream TiKV
    1.1 Get a latest TSO (supposed to be T0).
    1.2 Set BR service safepoint to T0 and TTL as 72 hours (configurable).
    1.3 Invoke backup procedure to TiKV, flush causal_ts at the beginning to make sure that timestamp of following writes must be greater than T0.
    1.4 Return the T0 after backup finished.
  2. TiKV-BR restore to downstream TiKV (It would be nice the store T0 in backup meta to remind the users agin after restore).
  3. TiKV-CDC create changefeed between upstream & downstream with --start-ts T0.

Update 2022.6.29

Set T0 as TSO - safe-interval. Where safe-interval is a configurable duration defaults to 1 minute.

safe-interval is a work around to avoid missing some RawKV writes just in Raft procedure during backup, and assume that all Raw writes should be finished in 1 minute.
The cons of safe-interval is that some entries would be backup or replicated redundantly.
On a system with heavy load, the safe-interval should be configured longer.
The safe-interval would be deprecated after the read-ts of RawKV is implemented. Refer to tikv/rfcs#96.

Describe the feature you'd like:

1.1 ~ 1.4 in above description.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Some unit tests for module Owner are not running

Bug Report

1. Describe the bug

Because of losing func Test(t *testing.T), some unit tests for module Owner are not running.

2. Minimal reproduce step (Required)

cd cdc/cdc/owner
go test

3. What did you see instead (Required)

Some unit tests don't run;

4. What did you expect to see? (Required)

all unit tests run;

5. What is your migration tool and TiKV version? (Required)

  • TiKV CDC:

*: The badges in README.md is incorrect

Bug Report

1. Describe the bug

The badges in README.md is incorrect.

2. Minimal reproduce step (Required)

3. What did you see instead (Required)

4. What did you expect to see? (Required)

5. What is your migration tool and TiKV version? (Required)

  • TiKV:
  • TiKV CDC:
  • TiKV BR:
  • TiKV Online Bulk Load:

[TiKV-BR] checksum mismatch after backup

Bug Report

1. Describe the bug

Backup report checksum failure if one tikv-server crash.

2. Minimal reproduce step (Required)

Load 2kw kv data to tikv cluster with more than 3 tikv-server nodes.
Run tikv-br backup raw
During backup running, kill one tikv-server process.
Wait for backup finish.

3. What did you see instead (Required)

backup finish with success but checksum report failure.

4. What did you expect to see? (Required)

backup finish with success and checksum succeed.

5. What is your migration tool and TiKV version? (Required)

  • TiKV BR:

BR restore failed “ BR:Restore:ErrRestoreTableIDMismatch”

Bug Report

1. Describe the bug

2. Minimal reproduce step (Required)

data format

Key： index_:_Str001_:Str001_:_{int64(20)}
Values: random string

origin_img_v2_e7b4340d-7814-4da9-9274-f1f6f25e9d7g

3. What did you see instead (Required)

[2022/05/26 14:30:47.886 +08:00] [INFO] [client.go:113] ["Restore client closed"]
[2022/05/26 14:30:47.886 +08:00] [INFO] [collector.go:202] ["units canceled"] [cancel-unit=0]
[2022/05/26 14:30:47.886 +08:00] [ERROR] [client.go:752] ["[pd] fetch pending tso requests error"] [dc-location=global] [error="[PD:client:ErrClientGetTSO]context canceled: context canceled"] [stack="[github.com/tikv/pd/client.(*client).handleDispatcher\n\t/root/go_depends/pkg/mod/github.com/tikv/[email protected]/client/client.go:752](http://github.com/tikv/pd/client.(*client).handleDispatcher/n/t/root/go_depends/pkg/mod/github.com/tikv/[email protected]/client/client.go:752)"]
[2022/05/26 14:30:47.886 +08:00] [INFO] [collector.go:68] ["Raw restore failed summary"] [total-ranges=0] [ranges-succeed=0] [ranges-failed=0] [restore-files=404]
[2022/05/26 14:30:47.886 +08:00] [INFO] [client.go:666] ["[pd] exit tso dispatcher"] [dc-location=global]
[2022/05/26 14:30:47.886 +08:00] [ERROR] [restore.go:35] ["failed to restore raw kv"] [error="table id mismatch: [BR:Restore:ErrRestoreTableIDMismatch]restore table ID mismatch"]
 [errorVerbose="[BR:Restore:ErrRestoreTableIDMismatch]restore table ID mismatch\ntable id mismatch\[ngithub.com/tikv/migration/br/pkg/restore.SortRanges\n\t/root/go-projects](http://ngithub.com/tikv/migration/br/pkg/restore.SortRanges/n/t/root/go-projects)
/migration/br/pkg/restore/range.go:60\[ngithub.com/tikv/migration/br/pkg/restore.(*RegionSplitter).Split\n\t/root/go-projects/migration/br/pkg/restore/split.go:96\ngithub.co](http://ngithub.com/tikv/migration/br/pkg/restore.(*RegionSplitter).Split/n/t/root/go-projects/migration/br/pkg/restore/split.go:96/ngithub.co)
m/tikv/migration/br/pkg/restore.SplitRanges\n\t/root/go-projects/migration/br/pkg/restore/util.go:121\[ngithub.com/tikv/migration/br/pkg/task.RunRestoreRaw\n\t/root/go](http://ngithub.com/tikv/migration/br/pkg/task.RunRestoreRaw/n/t/root/go)
-projects/migration/br/pkg/task/restore_raw.go:110\nmain.runRestoreRawCommand\n\t/root/go-projects/migration/br/cmd/br/restore.go:34\nmain.newRawRestoreCommand.func1\n\t/ho
me/huayp/go-projects/migration/br/cmd/br/restore.go:72\[ngithub.com/spf13/cobra.(*Command).execute\n\t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:842\n](http://ngithub.com/spf13/cobra.(*Command).execute/n/t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:842/n)
[github.com/spf13/cobra.(*Command).ExecuteC\n\t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:950\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/huay](http://github.com/spf13/cobra.(*Command).ExecuteC/n/t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:950/ngithub.com/spf13/cobra.(*Command).Execute/n/t/home/huay)
p/go_depends/pkg/mod/[github.com/spf13/[email protected]/command.go:887\nmain.main\n\t/root/go-projects/migration/br/cmd/br/main.go:56\nruntime.main\n\t/usr/local/go/src/runtime/](http://github.com/spf13/[email protected]/command.go:887/nmain.main/n/t/root/go-projects/migration/br/cmd/br/main.go:56/nruntime.main/n/t/usr/local/go/src/runtime/)
proc.go:225\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1371"] [stack="main.runRestoreRawCommand\n\t/root/go-projects/migration/br/cmd/br/restore.go:35\nmain.n
ewRawRestoreCommand.func1\n\t/root/go-projects/migration/br/cmd/br/restore.go:72\[ngithub.com/spf13/cobra.(*Command).execute\n\t/root/go_depends/pkg/mod/github.com/spf](http://ngithub.com/spf13/cobra.(*Command).execute/n/t/root/go_depends/pkg/mod/github.com/spf)
13/[email protected]/command.go:842\[ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/hom](http://ngithub.com/spf13/cobra.(*Command).ExecuteC/n/t/hom)
e/huayp/go_depends/pkg/mod/[github.com/spf13/[email protected]/command.go:950\ngithub.com/spf13/cobra.(*Command).Execute\n\t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:887\nmain.main\n\t/root/go-projects/migration/br/cmd/br/main.go:56\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:225](http://github.com/spf13/[email protected]/command.go:950/ngithub.com/spf13/cobra.(*Command).Execute/n/t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:887/nmain.main/n/t/root/go-projects/migration/br/cmd/br/main.go:56/nruntime.main/n/t/usr/local/go/src/runtime/proc.go:225)"]
[2022/05/26 14:30:47.886 +08:00] [ERROR] [main.go:58] ["br failed"] [error="table id mismatch: [BR:Restore:ErrRestoreTableIDMismatch]restore table ID mismatch"] [errorVerbose="[BR:Restore:ErrRestoreTableIDMismatch]restore table ID mismatch\ntable id mismatch\[ngithub.com/tikv/migration/br/pkg/restore.SortRanges\n\t/root/go-projects/migration/br/pkg/restore/range.go:60\ngithub.com/tikv/migration/br/pkg/restore.(*RegionSplitter).Split\n\t/root/go-projects/migration/br/pkg/restore/split.go:96\ngithub.com/tikv/migration/br/pkg/restore.SplitRanges\n\t/root/go-projects/migration/br/pkg/restore/util.go:121\ngithub.com/tikv/migration/br/pkg/task.RunRestoreRaw\n\t/root/go-projects/migration/br/pkg/task/restore_raw.go:110\nmain.runRestoreRawCommand\n\t/root/go-projects/migration/br/cmd/br/restore.go:34\nmain.newRawRestoreCommand.func1\n\t/root/go-projects/migration/br/cmd/br/restore.go:72\ngithub.com/spf13/cobra.(*Command).execute\n\t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:842\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:950\ngithub.com/spf13/cobra.(*Command).Execute\n\t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:887\nmain.main\n\t/root/go-projects/migration/br/cmd/br/main.go:56\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:225\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1371](http://ngithub.com/tikv/migration/br/pkg/restore.SortRanges/n/t/root/go-projects/migration/br/pkg/restore/range.go:60/ngithub.com/tikv/migration/br/pkg/restore.(*RegionSplitter).Split/n/t/root/go-projects/migration/br/pkg/restore/split.go:96/ngithub.com/tikv/migration/br/pkg/restore.SplitRanges/n/t/root/go-projects/migration/br/pkg/restore/util.go:121/ngithub.com/tikv/migration/br/pkg/task.RunRestoreRaw/n/t/root/go-projects/migration/br/pkg/task/restore_raw.go:110/nmain.runRestoreRawCommand/n/t/root/go-projects/migration/br/cmd/br/restore.go:34/nmain.newRawRestoreCommand.func1/n/t/root/go-projects/migration/br/cmd/br/restore.go:72/ngithub.com/spf13/cobra.(*Command).execute/n/t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:842/ngithub.com/spf13/cobra.(*Command).ExecuteC/n/t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:950/ngithub.com/spf13/cobra.(*Command).Execute/n/t/root/go_depends/pkg/mod/github.com/spf13/[email protected]/command.go:887/nmain.main/n/t/root/go-projects/migration/br/cmd/br/main.go:56/nruntime.main/n/t/usr/local/go/src/runtime/proc.go:225/nruntime.goexit/n/t/usr/local/go/src/runtime/asm_amd64.s:1371)"] [stack="main.main\n\t/root/go-projects/migration/br/cmd/br/main.go:58\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:225"]

4. What did you expect to see? (Required)

5. What is your migration tool and TiKV version? (Required)

  • TiKV: 6.0.1 / Git Commit Hash: 50c2abf87fd144a739ba7ccb70b4100f2c0d92f9
  • TiKV CDC:
  • TiKV BR: tikv-br / 7f90200
  • TiKV Online Bulk Load:

TiKV CDC Development Tasks

TiKV CDC Development Tasks

Related issue: tikv/tikv#11745
Related RFC: tikv/rfcs#86

Tasks

  • Core data structure in ETCD
    • raw changefeed
    • raw task (a part of a changefeed, for one processor)
    • ETCD path planning
  • Owner
    • New changefeed for RawKV
    • Delete table changefeed
  • Scheduler
    • New scheduler for Key Span
    • Delete table scheduler
  • Capture
    • Support rawkv changefeed
    • Delete support of table changefeed
  • Processor
    • New processor for Key Span
  • Sink
    • TiKV Sink
  • Command line
    • cdc cli
  • Others
    • Upgrade dependencies
  • TiUP support
  • Metrics
    • New Grafana board for TiKV-CDC
    • Determine metrics (ref to TiCDC metrics)
  • Tests
    • Integration tests
    • E2E tests

Development Guide

  1. Choose a task you are interested in.
  2. File a ISSUE for the task.
  3. Fix the ISSUE by PR (also refer to CONTRIBUTING guide).
  4. Get the PR merged.

Update TiKV-BR README.md

Bug Report

1. Describe the bug

README.md in migration/br is outdated. Please update it.

2. Minimal reproduce step (Required)

3. What did you see instead (Required)

4. What did you expect to see? (Required)

5. What is your migration tool and TiKV version? (Required)

  • TiKV:
  • TiKV CDC:
  • TiKV BR:
  • TiKV Online Bulk Load:

cdc: unit test clientSuite.TestClientErrNoPendingRegion is not stable

Bug Report

1. Describe the bug

clientSuite.TestClientErrNoPendingRegion is not stable.

(Upstream similar issue pingcap/tiflow#2688).

2. Minimal reproduce step (Required)

make unit_test

3. What did you see instead (Required)

All unit tests are passed.

4. What did you expect to see? (Required)

FAIL: client_test.go:2848: clientSuite.TestClientErrNoPendingRegion
client_test.go:2850:
    s.testClientErrNoPendingRegion(c)
client_test.go:329:
    c.Assert(err, check.IsNil)
... value *errors.Error = &errors.Error{code:0, codeText:"CDC:ErrReachMaxTry", message:"reach maximum try: %d", redactArgsPos:[]int(nil), cause:(*errors.fundamental)(0xc00042fcc8), args:[]interface {}{20}, file:"/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go", line:65} ("[CDC:ErrReachMaxTry]reach maximum try: 20: request id 4 is not larger than 5")
Error: 6/16 03:25:03.851 +00:00] [ERROR] [client.go:1037] ["tikv reported the request cluster ID mismatch error, which is not expected"] [tikvCurrentClusterID=0] [requestClusterID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).handleError\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:1037\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:573\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:04.301 +00:00] [ERROR] [region_worker.go:683] ["The CommitTs must be greater than the resolvedTs"] ["Event Type"=COMMIT] [CommitTs=98] [resolvedTs=100] [regionID=3] [stack="github.com/tikv/migration/cdc/cdc/kv.(*regionWorker).handleEventEntry\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:683\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).processEvent\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:386\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).eventHandler\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:478\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).run.func3\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:601\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:04.952 +00:00] [ERROR] [region_worker.go:664] ["The CommitTs must be greater than the resolvedTs"] ["Event Type"=COMMITTED] [CommitTs=98] [resolvedTs=100] [regionID=3] [stack="github.com/tikv/migration/cdc/cdc/kv.(*regionWorker).handleEventEntry\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:664\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).processEvent\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:386\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).eventHandler\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:478\ngithub.com/tikv/migration/cdc/cdc/kv.(*regionWorker).run.func3\n\t/home/runner/work/migration/migration/cdc/cdc/kv/region_worker.go:601\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:05.256 +00:00] [ERROR] [client.go:1032] ["tikv reported compatibility error, which is not expected"] [rpcCtx="region ID: 3, meta: id:3 region_epoch:<> peers:<id:4 store_id:1 > , peer: id:4 store_id:1 , addr: 127.0.0.1:39921, idx: 0, reqStoreType: TiKvOnly, runStoreType: tikv"] [error="required_version:\"v4.0.7\" "] [stack="github.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).handleError\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:1032\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:573\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:11.561 +00:00] [ERROR] [client.go:1028] ["tikv reported duplicated request to the same region, which is not expected"] [regionID=3] [stack="github.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).handleError\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:1028\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:573\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:20.814 +00:00] [ERROR] [client.go:364] ["check tikv version failed"] [error="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.0.10 is not supported, the minimal compatible version is 5.1.0-alpha"] [errorVerbose="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.0.10 is not supported, the minimal compatible version is 5.1.0-alpha\ngithub.com/pingcap/errors.AddStack\n\t/home/runner/go/pkg/mod/github.com/pingcap/[email protected]/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/home/runner/go/pkg/mod/github.com/pingcap/[email protected]/normalize.go:164\ngithub.com/tikv/migration/cdc/pkg/version.CheckStoreVersion\n\t/home/runner/work/migration/migration/cdc/pkg/version/check.go:181\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:362\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:350\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:746\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:514\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.18.3/x64/src/runtime/asm_amd64.s:1571"] [storeID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:364\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:350\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:746\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:514\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:20.923 +00:00] [ERROR] [client.go:364] ["check tikv version failed"] [error="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.1.0 is not supported, the minimal compatible version is 5.1.0-alpha"] [errorVerbose="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.1.0 is not supported, the minimal compatible version is 5.1.0-alpha\ngithub.com/pingcap/errors.AddStack\n\t/home/runner/go/pkg/mod/github.com/pingcap/[email protected]/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/home/runner/go/pkg/mod/github.com/pingcap/[email protected]/normalize.go:164\ngithub.com/tikv/migration/cdc/pkg/version.CheckStoreVersion\n\t/home/runner/work/migration/migration/cdc/pkg/version/check.go:181\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:362\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:350\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:746\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:514\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.18.3/x64/src/runtime/asm_amd64.s:1571"] [storeID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:364\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:350\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:746\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:514\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:21.025 +00:00] [ERROR] [client.go:364] ["check tikv version failed"] [error="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 2.1.10 is not supported, the minimal compatible version is 5.1.0-alpha"] [errorVerbose="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 2.1.10 is not supported, the minimal compatible version is 5.1.0-alpha\ngithub.com/pingcap/errors.AddStack\n\t/home/runner/go/pkg/mod/github.com/pingcap/[email protected]/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/home/runner/go/pkg/mod/github.com/pingcap/[email protected]/normalize.go:164\ngithub.com/tikv/migration/cdc/pkg/version.CheckStoreVersion\n\t/home/runner/work/migration/migration/cdc/pkg/version/check.go:181\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:362\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:350\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:746\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:514\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.18.3/x64/src/runtime/asm_amd64.s:1571"] [storeID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:364\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:350\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:746\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:514\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:21.127 +00:00] [ERROR] [client.go:364] ["check tikv version failed"] [error="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.0.10 is not supported, the minimal compatible version is 5.1.0-alpha"] [errorVerbose="[CDC:ErrVersionIncompatible]version is incompatible: TiKV 3.0.10 is not supported, the minimal compatible version is 5.1.0-alpha\ngithub.com/pingcap/errors.AddStack\n\t/home/runner/go/pkg/mod/github.com/pingcap/[email protected]/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/home/runner/go/pkg/mod/github.com/pingcap/[email protected]/normalize.go:164\ngithub.com/tikv/migration/cdc/pkg/version.CheckStoreVersion\n\t/home/runner/work/migration/migration/cdc/pkg/version/check.go:181\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:362\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:350\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:746\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:514\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.18.3/x64/src/runtime/asm_amd64.s:1571"] [storeID=1] [stack="github.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:364\ngithub.com/tikv/migration/cdc/pkg/retry.run\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:54\ngithub.com/tikv/migration/cdc/pkg/retry.Do\n\t/home/runner/work/migration/migration/cdc/pkg/retry/retry_with_opt.go:32\ngithub.com/tikv/migration/cdc/cdc/kv.(*CDCClient).newStream\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:350\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).requestRegionToStore\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:746\ngithub.com/tikv/migration/cdc/cdc/kv.(*eventFeedSession).eventFeed.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client.go:514\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75"]
Error: 6/16 03:25:23.035 +00:00] [ERROR] [client_test.go:1714] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestIncompatibleTiKV.func2\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:1714\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 6/16 03:26:28.646 +00:00] [ERROR] [client_test.go:2937] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).testKVClientForceReconnect.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:2937\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 6/16 03:26:29.962 +00:00] [ERROR] [client_test.go:2994] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).testKVClientForceReconnect.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:2994\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 6/16 03:26:34.239 +00:00] [ERROR] [client_test.go:3421] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestPrewriteNotMatchError.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:3421\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 6/16 03:26:35.274 +00:00] [ERROR] [client_test.go:3507] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestPrewriteNotMatchError.func4\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:3507\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 6/16 03:26:44.043 +00:00] [ERROR] [client_test.go:3326] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestRegionWorkerExitWhenIsIdle.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:3326\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 6/16 03:27:02.846 +00:00] [ERROR] [client_test.go:1509] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestStreamRecvWithErrorAndResolvedGoBack.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:1509\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 6/16 03:27:02.922 +00:00] [ERROR] [client_test.go:1509] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestStreamRecvWithErrorAndResolvedGoBack.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:1509\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
Error: 6/16 03:27:09.898 +00:00] [ERROR] [client_test.go:1320] ["mock server error"] [error="rpc error: code = Canceled desc = context canceled"] [stack="github.com/tikv/migration/cdc/cdc/kv.(*clientSuite).TestStreamSendWithError.func3\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:1320\ngithub.com/tikv/migration/cdc/cdc/kv.(*mockChangeDataService).EventFeed.func1\n\t/home/runner/work/migration/migration/cdc/cdc/kv/client_test.go:253"]
OOPS: 48 passed, 1 FAILED

5. What is your migration tool and TiKV version? (Required)

  • TiKV: N/A
  • TiKV CDC: master
  • TiKV BR: N/A
  • TiKV Online Bulk Load: N/A

Audit License check failed

Bug Report

1. Describe the bug

Audit License / check-license (pull_request) is failed. see https://github.com/tikv/migration/runs/5152588535?check_suite_focus=true for details

2. Minimal reproduce step (Required)

submit a pull request to trigger the action

3. What did you see instead (Required)

the check passes

4. What did you expect to see? (Required)

the check failed

5. What is your migration tool and TiKV version? (Required)

  • TiKV: N/A
  • TiKV CDC: master
  • TiKV BR: master
  • TiKV Online Bulk Load: master

Clean TiKV-BR command args

Bug Report

1. Describe the bug

  1. There are many args in tikv-br and some of them are useless or not supported well. Please remove/hide them.
  2. Some help description is not suitable, for example, tikv-br help display br is a tool for TiDB/TiKV cluster..., please check all hekp descriptions.

2. Minimal reproduce step (Required)

tikv-br help
tikv-br backup help
tikv-br restore help
tikv-br backup raw --help
tikv-br restore raw --help

3. What did you see instead (Required)

  1. There are many args in tikv-br and some of them are useless or not supported well. Please remove/hide them.
  2. Some help description is not suitable, for example, tikv-br help display br is a tool for TiDB/TiKV cluster..., please check all hekp descriptions.

4. What did you expect to see? (Required)

all command args should be supported well, else remove them.

5. What is your migration tool and TiKV version? (Required)

  • TiKV BR:

Add ci golang lint and gosec check for gc-worker

Feature Request

Is your feature request related to a problem? Please describe:

Describe the feature you'd like:
Add ci golang lint and gosec check for gc-worker

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

[TiKV-BR] checksum may mismatch when enabling ttl

Bug Report

1. Describe the bug

Some KV data with ttl expired, when doing BR( backup or restore )
The checksum will mismatch.

2. Minimal reproduce step (Required)

  1. enable ttl
  2. set KV with a short ttl duration
  3. BR costs more time than KV ttl.

3. What did you see instead (Required)

Mismatch.

4. What did you expect to see? (Required)

Consistent.

5. What is your migration tool and TiKV version? (Required)

-TiKV: v6.1.0 nightly

BR restore summary is not accurate.

Bug Report

1. Describe the bug

BR summary 修复前

restore summary 显示的 total-ranges = 0, Result = "Nothing to restore"

2. Minimal reproduce step (Required)

run tikv-br restore command

3. What did you see instead (Required)

4. What did you expect to see? (Required)

display the accurate restore file count and result message should be good enough to user.

5. What is your migration tool and TiKV version? (Required)

  • TiKV BR

[TiKV-BR] BR failed to restore when restored V1TTL data, with Error: [BR:KV:ErrKVKeyNotInRegion]key not in region

Bug Report

1. Describe the bug

BR failed to restore when restored V1TTL data, with Error: [BR:KV:ErrKVKeyNotInRegion]key not in region

2. Minimal reproduce step (Required)

  1. Deploy V1+TTL Cluster v6.1.0
  2. Import some data
  3. Backup -dst-api-version V1TTL
  4. Backup -dst-api-version V2
  5. Restore V2 data on the same cluster, expect to fail
  6. Restore V1TTL data on the same cluster, expect to success

3. What did you see instead (Required)

Failed to restore V1TTL data

4. What did you expect to see? (Required)

Should restore V1TTL data successfully.

5. What is your migration tool and TiKV version? (Required)

  • TiKV: ./tikv-server -V
    TiKV
    Release Version: 6.1.0-alpha
    Edition: Community
    Git Commit Hash: 50c2abf87fd144a739ba7ccb70b4100f2c0d92f9
    Git Commit Branch: heads/refs/tags/v6.1.0-alpha
    UTC Build Time: 2022-05-25 14:54:48
    Rust Version: rustc 1.60.0-nightly (1e12aef3f 2022-02-13)
    Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
    Profile: dist_release

  • TiKV BR: ./tikv-br --version
    Release Version: 7f90200
    Git Commit Hash: 7f90200
    Git Branch: pr103
    Go Version: go1.16.5
    UTC Build Time: 2022-05-26 01:59:28
    Race Enabled: false

Seperate RawKV region and TiDB regions

Feature Request

Is your feature request related to a problem? Please describe:

When run tikv-cdc, the resolved-ts would be mixed if there are rawkv data and tidb data in one same region.
PD provides functionality to seperate regions with a specified key.
Please ref:
tikv/pd#3839
https://github.com/pingcap/tiflow/blob/release-5.4/pkg/pdutil/region_label.go

TiKV-CDC should also implement this.

Describe the feature you'd like:

Seperate rawkv and tidb regions.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Use new gc safepoint update interface

Bug Report

1. Describe the bug

Use new gc safepoint update interface

2. Minimal reproduce step (Required)

3. What did you see instead (Required)

4. What did you expect to see? (Required)

Use new gc safepoint update interface

5. What is your migration tool and TiKV version? (Required)

GC-Worker

[BR] no checksum in backup/restore process

Bug Report

1. Describe the bug

Does not do checksum in back/restore process.

2. Minimal reproduce step (Required)

run tikv-br backup raw & tikv-br restore raw

3. What did you see instead (Required)

no checksum is done during back/restore.

4. What did you expect to see? (Required)

do checksum in back/restore process and report checksum result.

5. What is your migration tool and TiKV version? (Required)

  • TiKV BR:

[TiKV-BR] Parameter Identification ERROR and Command Prompt inaccurate when not set --dst-api-version

Bug Report

1. Describe the bug

yNZNKkzDwA

2. Minimal reproduce step (Required)

As the screenshot.

3. What did you see instead (Required)

dst-api-version identifys to dot ".".

4. What did you expect to see? (Required)

dst-api-version only supports: V1, V1TTL, V2

5. What is your migration tool and TiKV version? (Required)

  • TiKV BR: Release Version: d8c0fb1
    Git Commit Hash: d8c0fb1
    Git Branch: main
    Go Version: go1.16.5
    UTC Build Time: 2022-05-25 09:30:18
    Race Enabled: false

[gc-worker] support current pd safepoint update interface

Feature Request

Is your feature request related to a problem? Please describe:

support current pd safepoint update interface

Describe the feature you'd like:

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

cdc: Increase test coverage

Bug Report

1. Describe the bug

The test coverage is low (totally about 59.67% for master). We should increase it.
See https://app.codecov.io/gh/tikv/migration.
image

2. Minimal reproduce step (Required)

3. What did you see instead (Required)

4. What did you expect to see? (Required)

5. What is your migration tool and TiKV version? (Required)

  • TiKV:
  • TiKV CDC:
  • TiKV BR:
  • TiKV Online Bulk Load:

cdc: add GC related procedure

Feature Request

Is your feature request related to a problem? Please describe:

The GC related procedure of TiKV-CDC is expected to be:

  1. On create / resume changefeed, set TiKV-CDC's service safepoint to changefeed's start-ts when it is not zero. If failed (when the service safepoint is lower than min(all service safepoints), the creation / resume of changefeed is failed.

  2. Before setting every checkpoint, set service safepoint to the checkpoint-ts. If failed (would happen when start-ts is zero that the changefeed is synchronizing from the beginning), log a warning and continue, trying to set a larger service safepoint in next round.

  3. TTL of service safepoint defaults to 24 hours (configurable). Which means that a changefeed can be resumed within 24 hours. After that, the changefeed is dead. Synchronize from the beginning, or another BR backup & restore, is required.

Describe the feature you'd like:

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

TiKV BR Development Tasks

TiKV BR Development Tasks

Close: #66
RFC: tikv/rfcs#86

Tasks

Reference

TiKV components related to backup & restore:

  • components/backup: for backup
  • components/sst_importer: for restore
  • split_region interface: for pre-splitting regions before restore

Some merged PR related:

[TiKV-BR] Remove dependence to etcd, pd and tidb

Bug Report

1. Describe the bug

TiKV-BR depends on many tidb and pd packages, which cause some dependence problems. tidb/br has removed the dependence to PD. Please remove the dependence to PD in TiKV-BR and reduce dependence to TiDB also.

2. Minimal reproduce step (Required)

3. What did you see instead (Required)

4. What did you expect to see? (Required)

5. What is your migration tool and TiKV version? (Required)

  • TiKV BR:

Add release action for tikv-br

Feature Request

Is your feature request related to a problem? Please describe:

Describe the feature you'd like:
Add release action for tikv-br

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.