Add doc: explain threads/tasks that are used in this lib

Openraft

Advanced Raft in 🦀 Rust using Tokio. Please ⭐ on github!

This project intends to improve raft as the next-generation consensus protocol for distributed data storage systems (SQL, NoSQL, KV, Streaming, Graph ... or maybe something more exotic).

Currently, openraft is the consensus engine of meta-service cluster in databend.

🚀 Get started:
- Openraft guide is the best place to get started,
- Openraft docs for more in-depth details,
- Openraft FAQ explains some common questions.
🙌 Questions?
- Why not take a peek at our FAQ? You might find just what you need.
- Wanna chat? Come hang out with us on Discord!
- Or start a new discussion over on GitHub.
- Or join our Feishu group.
- And hey, if you're on WeChat, add us: drmingdrmer. Let's get the conversation started!

Whatever your style, we're here to support you. 🚀 Let's make something awesome together!

Openraft is derived from async-raft with several bugs fixed: Fixed bugs.

Status

The features are almost complete for building an application.
Performance: Supports 70,000 writes/sec for single writer, and 1,000,000 writes/sec for 256 writers. See: Performance
Unit test coverage stands at 92%.
The chaos test has not yet been completed, and further testing is needed to ensure the application's robustness and reliability.

API status

Openraft API is not stable yet. Before 1.0.0, an upgrade may contain incompatible changes. Check our change-log. A commit message starts with a keyword to indicate the modification type of the commit:
- DataChange: on-disk data types changes, which may require manual upgrade.
- Change: if it introduces incompatible changes.
- Feature: if it introduces compatible non-breaking new features.
- Fix: if it just fixes a bug.

Versions

Branch main has been under active development. The main branch is for the release-0.10.
Branch release-0.9: Latest: ( v0.9.0 | Change log ); Upgrade guide: ⬆️ 0.8 to 0.9; release-0.9 Won't accept new features but only bug fixes.
Branch release-0.8: Latest: ( v0.8.8 | Change log ); Upgrade guide: ⬆️ 0.7 to 0.8, ⬆️ 0.8.3 to 0.8.4; release-0.8 Won't accept new features but only bug fixes.
Branch release-0.7: Latest: ( v0.7.6 | Change log ); Upgrade guide: ⬆️ 0.6 to 0.7; release-0.7 Won't accept new features but only bug fixes.
Branch release-0.6: Latest: ( v0.6.8 | Change log ); release-0.6 won't accept new features but only bug fixes.

Roadmap

2022-10-31 Extended joint membership
2023-02-14 Minimize confliction rate when electing; See: Openraft Vote design; Or use standard raft mode with feature flag single-term-leader.
2023-04-26 Goal performance is 1,000,000 put/sec.
Reduce the complexity of vote and pre-vote: get rid of pre-vote RPC;
Support flexible quorum, e.g.: Hierarchical Quorums
Consider introducing read-quorum and write-quorum, improve efficiency with a cluster with an even number of nodes.

Performance

The benchmark is focused on the Openraft framework itself and is run on a minimized store and network. This is NOT a real world application benchmark!!!

clients	put/s	ns/op
256	1,014,000	985
64	730,000	1,369
1	70,000	14,273

For benchmark detail, go to the ./cluster_benchmark folder.

Features

Async and Event-Driven: Operates based on Raft events without reliance on periodic ticks, optimizing message batching for high throughput.
Extensible Storage and Networking: Customizable via RaftLogStorage, RaftStateMachine and RaftNetwork traits, allowing flexibility in choosing storage and network solutions.
Unified Raft API: Offers a single Raft type for creating and interacting with Raft tasks, with a straightforward API.
Cluster Formation: Provides strategies for initial cluster setup as detailed in the cluster formation guide.
Built-In Tracing Instrumentation: The codebase integrates tracing for logging and distributed tracing, with the option to set verbosity levels at compile time.

Functionality:

✅ Leader election: by policy or manually(trigger_elect()).
✅ Non-voter(learner) Role: refer to add_learner().
✅ Log Compaction(snapshot of state machine): by policy or manually(trigger_snapshot()).
✅ Snapshot replication.
✅ Dynamic Membership: using joint membership config change. Refer to dynamic membership
✅ Linearizable read: ensure_linearizable().
⛔️ Wont support: Single-step config change.
✅ Toggle heartbeat / election: enable_heartbeat / enable_elect.
✅ Trigger snapshot / election manually: trigger_snapshot() / trigger_elect().
✅ Purge log by policy or manually: purge_log().

Who use it

Databend - The Next-Gen Cloud [Data+AI] Analytics
CnosDB - A cloud-native open source distributed time series database.
yuyang0/rrqlite - A rust implementation of rqlite.
raymondshe/matchengine-raft - A example to demonstrate how openraft persists snapshots/logs to disk.
Helyim - SeaweedFS implemented in pure Rust.

Contributing

Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.

Contributors

Made with contributors-img.

License

Openraft is licensed under the terms of the MIT License or the Apache License 2.0, at your choosing.

	async fn first_id_in_log(&self) -> Result<Option<LogId>, StorageError>;

	async fn first_known_log_id(&self) -> Result<LogId, StorageError>;

	/// Returns the last log id in log.
	///
	/// The impl should not consider the applied log id in state machine.
	async fn last_id_in_log(&self) -> Result<LogId, StorageError>;

	pub(crate) enum RaftEvent<D: AppData> {
	Replicate {
	/// The new entry which needs to be replicated.
	///
	/// This entry will always be the most recent entry to have been appended to the log, so its
	/// index is the new last_log_index value.
	entry: Arc<Entry<D>>,

	/// The index of the highest log entry which is known to be committed in the cluster.
	committed: LogId,
	},

	pub(self) async fn leader_loop(mut self) -> Result<(), Fatal> {
	loop {
	if !self.core.target_state.is_leader() {
	tracing::info!("id={} state becomes: {:?}", self.core.id, self.core.target_state);

	// implicit drop replication_rx
	// notify to all nodes DO NOT send replication event any more.
	return Ok(());
	}

	let span = tracing::debug_span!("CHrx:LeaderState");
	let _ent = span.enter();

	tokio::select! {
	Some((msg,span)) = self.core.rx_api.recv() => {
	self.handle_msg(msg).instrument(span).await?;
	},

	Some(update) = self.core.rx_compaction.recv() => {
	tracing::info!("leader recv from rx_compaction: {:?}", update);
	self.core.update_snapshot_state(update);
	}

	Some((event, span)) = self.replication_rx.recv() => {
	tracing::info!("leader recv from replication_rx: {:?}", event.summary());
	self.handle_replica_event(event).instrument(span).await?;
	}

	Ok(_) = &mut self.core.rx_shutdown => {
	tracing::info!("leader recv from rx_shudown");
	self.core.set_target_state(State::Shutdown);
	}
	}
	}
	}

datafuselabs / openraft Goto Github PK

openraft's Introduction

Openraft

Advanced Raft in 🦀 Rust using Tokio. Please ⭐ on github!

Status

API status

Versions

Roadmap

Performance

Features

Functionality:

Who use it

Contributing

Contributors

License

openraft's People

Stargazers

Watchers

Forkers

openraft's Issues

The problems are:

Proposed Solution:

Recommend Projects

Recommend Topics

Recommend Org