Comments (2)
There are two major drawbacks of using raft protocol on your own:
- You have to run at least 3 nodes for auto-failover.
- Raft protocol does not recommend to have more than 9 nodes (no big clusters for read-only queries).
Also such approach brings a lot of other inconveniences, like:
- Previously it was possible to have a central place, where you can find all your clusters (and nodes)
- You have to configure every single cluster (and actually every node) uniquely.
I was thinking about implementing more or less the same thing in Patroni with using one of the python raft implementations, but above thoughts stopped me. Now I'm thinking about implementing support of Kubernetes API in addition to Etcd, ZooKeeper and Consul.
from governor.
@oneness totally fine! We're planning on creating a new repo from the golang-custom-raft branch ourselves. But we're happy to have folks forking this project. If someone can create something better than we have then that's awesome and welcome that!
@CyberDem0n yes those are some drawbacks. This solution worked quite well for our particular use case at compose.
A couple things on those compromise though. First, while you need 3 nodes participating in a raft cluster for fail over, not every node must be running governor(see lifeboat in github.com/compose/canoe). How we run the new governor internally, we have 2 nodes running a governor process, and 1 node running lifeboat. Lifeboat is essentially a process which runs as a member of a raft cluster, but doesn't retain state itself beyond the raft log. It acts as a quorum pad. This way we have a 3 node cluster, but not have the overhead of running 3 pg processes.
Also, while you don't have a central store for cluster information, you can query any governor node in a cluster for information on the state of the cluster using the Governor API
Finally, yes. The new governor isn't designed for huge clusters. A thought I have in that though... And shooting from the hip on this totally untested. But what you might be able to do with that is have a 3 node governor cluster. That way you have 3 nodes for a failover scenario. But then you might could query the Governor API to find the leader connection string, then configure a read only pg node as a hot_standby. It would never be able to become the leader, and it wouldn't have a replication slot on the leader to ensure it can keep up. But could be something to test out
from governor.
Related Issues (20)
- database system identifier differs between the primary and standby? HOT 2
- etcd returns 500 internal server error on ubuntu which causes postgres to crash. HOT 1
- 404 error causing the postgres to go down
- Use python-etcd client library HOT 1
- Fatal: requested timeline 8 is not a child of this server's history HOT 3
- Fencing and Quorum Support HOT 3
- Local Docker cluster with Governor on board HOT 1
- Make governor a module
- PostgreSQL + haproxy with multiple IP HOT 3
- not catching ssl timeout exception HOT 1
- rewind ex-leader before joining again HOT 2
- [Errno 32] Broken pipe HOT 1
- non atomic has_lock() and update_lock()
- cannot easily "go build" golang-custom-raft; maybe we should have a new project? HOT 1
- golang-custom-raft: add abilty to execute post-election script HOT 14
- golang-custom-raft: If a PG process is unhealthy - it can kill governor
- golang-custom-raft: maximum_lag_on_failover doesn't work as it should
- help:the connect info in the recover.conf are "None"
- replication slots failing when names contain dashes
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from governor.