Comments (6)
Just my 2c: I don't think there is a generic way to back up logs. We'll also back up to S3, but that's just one of the options. We must support 5 different hyperscalers, all with different APIs. From openraft
POV, the log is continuous and how you cut it for backup is your beer. In our case, once a log segment is filled, it can be backed up. Once openraft
tells us to purge, it's removed locally (unless the backup didn't yet finish).
from openraft.
Hi @drmingdrmer, would you be open to incorporating that logic into openraft? This way, users wouldn't have to replicate the same work.
from openraft.
👋 Thanks for opening this issue!
Get help or engage by:
/help
: to print help messages./assignme
: to assign this issue to you.
from openraft.
In databend, we just export all of the log and the snapshot to a file and backup it. The following is the server side export implementation:
https://github.com/datafuselabs/databend/blob/313b40ed3ff1cfe001bc6e2f806773d9001861d0/src/meta/service/src/store/store_inner.rs#L394
Fix the above url
from openraft.
@drmingdrmer Do you do this in a continuous basis for databend or is this happens after every couple of hours?
from openraft.
@drmingdrmer Do you do this in a continuous basis for databend or is this happens after every couple of hours?
It is a cronjob and is executed by schedule. It is not a incremental backup :(
from openraft.
Related Issues (20)
- Feature: Monoio Runtime HOT 5
- Add sync primitives to `AsyncRuntime` trait HOT 12
- Add `AsyncRuntime::oneshot` HOT 2
- Propose stream-based Replication of Log Entries HOT 2
- Dynamic Cluster Membership HOT 5
- Release 0.9 HOT 2
- Killing the leader of a cluster generates excessive network attempts for replicating log entries HOT 4
- RFC: Client request pipelining HOT 9
- main branch cannot compile, please update dependency rocksdb version HOT 2
- move trait method: from `RaftLogStorage` to `RaftLogReader::read_vote()` HOT 2
- Implement TryAsRef<ForwardToLeader<..>> for RaftError HOT 3
- When quitting, RaftCore outputs unexpected ERROR log HOT 1
- Proposal: Remove `RaftLogReader` Trait Dependency from `RaftLogStorage` HOT 2
- What operations can run at the same time ? HOT 3
- Install Snapshot v1 api HOT 3
- Observe state changes in a Raft node HOT 6
- Split metrics into data metrics and server metrics HOT 5
- About automatic remove HOT 3
- Raft Core Panicking HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openraft.