Giter Club home page Giter Club logo

goldflake's Introduction

Goldflake

Generator to generate Bigint ID or unique key for 174 years (Approx 200k unique Id/sec/machine on 8k distributed machines at once) inspired by Twitter's Snowflake and Sonyflake

GoDoc Build Status Coverage Status Go Report Card

Goldflake is a distributed unique ID generator inspired by Twitter's Snowflake & Sonyflake

Goal:

To become application & database oriented unique bigint id generator for lifetime without compromising performance.

Why Goldflake id instead of Sonyflake?

  1. Sonyflake generator is good, if you are generating all Ids at application end.
  2. Sonyflake has a limit of 25K id/sec on single node, which seems low in today's scenarios when we are talking about cloud & databases can be created with high system configurations easily.
  3. I have tried & tested Sonyflake bits configuration on Postgres-12 on my machine (Dell XPS, Intel® Core™ i7-9750H Processor 6 Core 12 Threads, 16GB RAM, SSD) & observed duplicate key conflict, which is a very basic configuration for Databases now a days.
  4. Goldflake is usnig 11 bits for sequence instead of 8 (as used by Sonyflake) which gives more unique ID generation on databases as well.

So it has a different bit assignment from Snowflake & Sonyflake. A Goldflake ID is composed of

39 bits for time in units of 10 msec                      // Similar to Sonyflake
13 bits for a machine id or app instance id or shard id   // Balance between Snowflake and Sonyflake (If you can live with 8192 machines)
11 bits for a sequence number                             // To generate 2^11=2048 unique ids per 10 millisecond or 204800 keys/sec on one node

For PostgreSQL -https://github.com/AmreeshTyagi/goldflake-pg

Comparison chart

Twitter Snowflake Sonyflake Goldflake
Lifetime to generate unique id on single machine 69 174 174
No. of distributed machines 1024 65536 8192
No. of unique IDs per 10 milliseconds 40960 256 2048
Max No. of unique IDs per second on single node 4096000 = 4096k 25600 = 25.6k 204800 = 204.8k
Max No. of unique IDs per second on all nodes 4194304000 1677721600 1677721600
Database oriented Yes No Yes
Application oriented Yes Yes Yes
Comment Less number of machines DB will return duplicate soon in case of heavy concurrency 50% application generated IDs & 50% database generated IDs makes it a best fit to use it on app server and db server. I believe 4096 database shards are enough to handle potential large load. Though it depends on solution design as well.

goldflake's People

Contributors

amreeshtyagi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.