t-pwk / flake-idgen Goto Github PK
View Code? Open in Web Editor NEWFlake ID generator yields k-ordered, conflict-free ids in a distributed environment in Node.js
Home Page: https://www.npmjs.org/package/flake-idgen
License: MIT License
Flake ID generator yields k-ordered, conflict-free ids in a distributed environment in Node.js
Home Page: https://www.npmjs.org/package/flake-idgen
License: MIT License
function genFlakeId() {
const flakeIdGen = new FlakeId();
return intformat(flakeIdGen.next(), 'dec');
}
const ids = [];
for (let i = 0; i < 50; i++) {
ids.push(genFlakeId());
}
// ids
[
"6901552045816283136",
"6901552045820477440",
"6901552045820477440",
"6901552045833060352",
"6901552045837254656",
"6901552045841448960",
"6901552045845643264",
"6901552045849837568",
"6901552045854031872",
"6901552045858226176",
"6901552045862420480",
"6901552045875003392",
"6901552045879197696",
"6901552045879197696",
"6901552045879197696",
"6901552045883392000",
"6901552045883392000",
"6901552045883392000",
"6901552045883392000",
"6901552045883392000",
"6901552045887586304",
"6901552045887586304",
"6901552045887586304",
"6901552045887586304",
"6901552045887586304",
"6901552045891780608",
"6901552045891780608",
"6901552045891780608",
"6901552045891780608",
"6901552045891780608",
"6901552045895974912",
"6901552045895974912",
"6901552045895974912",
"6901552045895974912",
"6901552045900169216",
"6901552045900169216",
"6901552045900169216",
"6901552045900169216",
"6901552045900169216",
"6901552045904363520",
"6901552045904363520",
"6901552045904363520",
"6901552045908557824",
"6901552045908557824",
"6901552045908557824",
"6901552045912752128",
"6901552045912752128",
"6901552045912752128",
"6901552045916946432",
"6901552045916946432"
]
I generate id
with flake-idgen
and biguint-format
but when try to save this id to mongodb get next error:
Error: Argument passed in must be a single String of 12 bytes or a string of 24 hex characters at new ObjectID
Is it real fix?
@T-PWK From what I understand, the ms timestamp is 41 bits but in the readme 42 bits are reserved for the timestamp, what is the extra bit?
First, thanks for creating this. It's really cool. I'm considering using it as a cryptographic nonce generator (not the intended use case I know, see SO question here), so I want to make sure it doesn't generate duplicate IDs.
I put together a simple test for it and ran it on my computer. I killed the process half way through to simulate an unexpected server shutdown and then resumed it again. After generating about 200,000,000 total ids in 4 processes running in parallel, I checked for duplicates and found a little under 2,000 (each with exactly 2 copies).
Is there something wrong with my testing methodology? Is this behavior expected?
Hello,
I would like to know if the generated id converted in decimal stay inside the range of the ?
Number.MAX_SAFE_INTEGER ?
Thank you and have a nice day.
I use this, is not the method I use is wrong?
var intformat = require('biguint-format');
var FlakeId = require('flake-idgen');
var flakeIdGen = new FlakeId({epoch: 1300000000000});
var moment = require('moment')
module.exports = function(sequelize, DataTypes) {
return sequelize.define('employment_detail', {
id: {
type: DataTypes.STRING(45),
allowNull: false,
primaryKey: true,
// defaultValue: shortid.generate
defaultValue: intformat(flakeIdGen.next(), 'dec')
}
}
}
I am using flake-idgen in a distributed environment and am seeing duplicate Id errors on a the primary Id field under the circumstances below during K6 load testing (50 Vus):
2+ replicates of Id service
Datacenter = 0
Worker = 1,2,3,n
The issue does not exist when I run a single instance of the Id generation service (datacenter = 0, worker = 0). Based on observation, the Duplicate collisions are being produces by different instances of the service initialized with a different worker Id.
I'll attempt incrementing both datacenter and worker and leave a comment on the results.
Maybe it's a good idea to add a get function for retrieving the datacenter or worker id from a generated id?
Hi,
Your package works fine until 1.3.0 but version 1.4.0 introduce some changes that makes build impossible with Webpack on UglifyJS step.
Error is:
ERROR in assets/bundle.js from UglifyJs
Unexpected character '`' [assets/bundle.js:203079,32]
Extract from devDependencies in our file package.json
:
"babel-core": "6.26.3",
"babel-loader": "7.1.5",
"babel-polyfill": "6.26.0",
"babel-preset-env": "1.7.0",
"babel-preset-react": "6.24.1",
"babel-preset-stage-0": "6.24.1",
"babel-register": "6.26.0",
"copy-webpack-plugin": "5.1.1",
"css-loader": "3.5.2",
"generate-json-webpack-plugin": "0.3.1",
"html-webpack-plugin": "4.2.0",
"imagemin-webpack-plugin": "2.4.2",
"style-loader": "1.1.4",
"uglifyjs-webpack-plugin": "2.2.0",
"url-loader": "4.1.0",
"webpack": "4.42.1",
"webpack-bundle-analyzer": "3.7.0",
"webpack-cli": "3.3.11",
"webpack-dev-server": "3.10.3",
"webpack-merge": "4.2.2"
I tried to use this package in my Sveltekit project and got Buffer is not defined.
Here is my code:
import FlakeId from "flake-idgen";
import intformat from "biguint-format";
const flakeIdGen = new FlakeId();
let id = intformat(flakeIdGen.next(), "dec");
Here is my error log:
ReferenceError: Buffer is not defined
at module.exports.next (flake-id-gen.js:58:13)
at instance (MainComponent.svelte:11:49)
at init (index.mjs:2165:11)
at new MainComponent (MainComponent.svelte:24:23)
at createProxiedComponent (svelte-hooks.js?v=8eec1919:341:9)
at new ProxyComponent (proxy.js?v=8eec1919:242:7)
at new Proxy<MainComponent> (proxy.js?v=8eec1919:349:11)
at create_fragment (+page.svelte:12:97)
at init (index.mjs:2180:37)
at new Page (+page.svelte:141:20)
Not sure if this package will work in a Jamstack project.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.