- ๐ญ Iโm always working on MassTransit
phatboyg / newid Goto Github PK
View Code? Open in Web Editor NEWA sequential id generator that works across nodes with no collisions
License: Apache License 2.0
A sequential id generator that works across nodes with no collisions
License: Apache License 2.0
Hi guys,
I know the Flake ago produces nice looking sequential GUIDs. However, this implementation of the library has the following fatal flaws.
When generating 5M GUIDs using parallel tasks here are the issues:
It roughly takes 5 seconds to generate 5M GUIDs. This rather slow considering other libraries can produce 5M GUIDs in less than 300ms
The issue is collisions. After running 5M GUIDs, we detected on average, the same GUID repeated between 70-300 times per run. This is a big concern. I suggest not to use the Flake algo of GUIDs but rather just replace the last 8 bytes of the GUID to use an Unix Epoch Timestamp in milliseconds. Granted the timestamp portion of the GUID can have collisions but due to the fact the remaining part of a natural GUID (minus the last 8 bytes) provides a much better quasi-uniqueness than the Flake algo.
Hope this helps,
codematrix
I'm using EF Core 8 with Oracle 21c and all my entities have an Id
column of type Guid
.
How can I control / override the value generation for this column during an insert / Add
to use NewId?
In projects that have dependencies on both MassTransit.Abstractions and NewId, the compiler has no way to tell the difference between usages because the class names and namespaces are identical.
@phatboyg have you considered making a breaking change and choosing a new namespace for this standalone library/package?
Edit: I'm aware that having both dependencies is a no-no if you can avoid it, but given that they could be transitive dependencies via NuGet, there are scenarios for which it cannot easily be avoided.
Hi,
Thanks for this useful package.
Could we create a unique ID across all services? Should we config each service instance separately to ensuring about uniqueness ?
As in Doc mentioned, we have sequential Id
with NewID
. In the end, do we get generated sequential Id with ideal performance like int
for cluster index?
Methods should be similar or identical to the ones in the base dotnet Guid type
What is the difference between NewId.NextGuid()
and NewId.NextSequentialGuid()
?
As I understood, NewId.NextGuid()
generates sequential Guid
s.
So my question is when to use which method?
There is a column of time, but the time is not the latest on the top, why is it that order by id ,guid is not generated according to time?
Get NewId running on .net core.
FromSequentialGuid
incorrectly swaps the bytes of the sequence counter, this means that parsing SequentialGuid creates incorrect NewIds. Tests won't detect this because the first NewId generated has the sequence number 0x0000.
This bug was introduced in #20 affecting FromSequentialGuid
and ToNewIdFromSequential
.
Expected: 9bfb0100-e3f8-6a00-06a1-08daed27ef21
But was: 9bfb0001-e3f8-6a00-06a1-08daed27ef21
// Round trip, to SequentialGuid and back with FromSequentialGuid
[Test]
public void Should_parse_sequential_guid_2_as_newid()
{
NewId n = NewId.Next(2)[1];
var nn = n.ToGuid();
var g = n.ToSequentialGuid();
var ng = NewId.FromSequentialGuid(g);
Assert.AreEqual(n, ng);
// Also checks to see if this would throw
Assert.IsTrue(ng.Timestamp != default);
}
static void FromSequentialByteArray(in byte[] bytes, out Int32 a, out Int32 b, out Int32 c, out Int32 d)
{
a = bytes[3] << 24 | bytes[2] << 16 | bytes[1] << 8 | bytes[0];
b = bytes[5] << 24 | bytes[4] << 16 | bytes[7] << 8 | bytes[6];
c = bytes[8] << 24 | bytes[9] << 16 | bytes[10] << 8 | bytes[11];
d = bytes[12] << 24 | bytes[13] << 16 | bytes[14] << 8 | bytes[15];
}
Hi, thank you for sharing your efforts with the community.
We're operating in a multi-language ecosystem with .NET, Java, and PHP and I was wondering if you happen to know of any ports of this to Java or PHP?
Thank you!
Hi,
I really like your library as it solves the problem of sequential id generation for me. However, I'd really like to have greater/less than operators available to NewId so I don't need to use CompareTo methods.
I've just created a PR ( #11 ) that adds it. Could you please consider mergin it to your library?
Thanks!
Is this project somehow related to/intended to work along with MassTransit?
Hey, I am just curious about which type of sequential guid you are using?
Because if you want increase performance in MSSQL you need to use sequential at end, and for MySql you need to use sequential at beginning.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.