refactor pd job mechanism about pd HOT 5 CLOSED

How To

We have only one final state for every region.

• Pd caches whole region in memory.
• Leader peer for the region will report region status regularly.

For conf change

Leader 1 for region a asks ChangePeer, pd adds 2 to region a directly, so the region meta in etcd contains peer 1, 2 now. The region in TiKV must enter this final state.

If leader 1 asks ChangePeer again, pd still replies adding peer 2 directly until pd finds region a has peer 1, 2 and then does the following Conf change.

For split

Leader 1 for region a [a-c) asks Split, pd allocs the new region ID and peer IDs only.
Region a [a-c) splits -> a [a-b) + b [b-c).
Leader in region a [a-b) reports region info to pd, pd finds it is different from the memory cache, updates the pd first and then update cache.
Leader in region b [b-c) reports new region info to pd, pd finds it is not in cache and not in etcd, so adds it directly to etcd and cache.

Problem: If region a reports status first, but region b delayed, we may meet a gap in key range, because we don't know where to find the data in [b, c). But this gap may be fixed later.

from pd.

/cc @disksing Seems something should be done in ticlient.

from pd.

One region can only do one thing at same time

Do you mean all command? including read-only request such as GetRegion?

Expanding 1, if pd decides how to do for the command, e,g, for region a, adding peer 3 for it, it will execute this job all the time, no matter how much times it retries or the leader asks.

This brings risk for dead lock. we have two 'state', one is in raft node, the other in pd.

1. raft node send request to pd
2. pd decides how to do and send job to raft node
3. raft execute the job and update it's state to pd periodly
4. pd response after it know state updated

the whole process forms a ring, if raft have some trouble and can't finish the job( although I haven't come up with a specific case), then dead lock will occur.

from pd.

b.t.w as long as we have GetRegion, those changes don't affect #102 much.... i.e. if peer found itself inactive for a long time, then it ask pd whether it's still live.

from pd.

Pd now has no job, so no deadlock.

from pd.