Giter Club home page Giter Club logo

shortid's Introduction

Build status Coverage GoReportCard API documentation

Generator of unique non-sequential short Ids

The package shortidenables the generation of short, fully unique, non-sequential and by default URL friendly Ids at a rate of hundreds of thousand per second. It guarantees uniqueness during the time period until 2050!

The package is heavily inspired by the node.js shortid library (see more detail below).

The easiest way to start generating Ids is:

fmt.Printf(shortid.Generate())
fmt.Printf(shortid.Generate())

The recommended one is to initialise and reuse a generator specific to a given worker:

sid, err := shortid.New(1, shortid.DefaultABC, 2342)

// then either:
fmt.Printf(sid.Generate())
fmt.Printf(sid.Generate())

// or:
shortid.SetDefault(sid)
// followed by:
fmt.Printf(shortid.Generate())
fmt.Printf(shortid.Generate())

Id Length

The standard Id length is 9 symbols when generated at a rate of 1 Id per millisecond, occasionally it reaches 11 (at the rate of a few thousand Ids per millisecond) and very-very rarely it can go beyond that during continuous generation at full throttle on high-performant hardware. A test generating 500k Ids at full throttle on conventional hardware generated the following Ids at the head and the tail (length > 9 is expected for this test):

-NDveu-9Q
iNove6iQ9J
NVDve6-9Q
VVDvc6i99J
NVovc6-QQy
VVoveui9QC
...
tFmGc6iQQs
KpTvcui99k
KFTGcuiQ9p
KFmGeu-Q9O
tFTvcu-QQt
tpTveu-99u

Life span

The package guarantees the generation of unique Ids with no collisions for 34 years (1/1/2016-1/1/2050) using the same worker Id within a single (although can be concurrent) application provided application restarts take longer than 1 millisecond. The package supports up to 32 workers all providing unique sequences from each other.

Implementation details

Although heavily inspired by the node.js shortid library this is not just a Go port. This implementation

  • is safe to concurrency (test included);
  • does not require any yearly version/epoch resets (test included);
  • provides stable Id size over a the whole range of operation at the rate of 1ms (test included);
  • guarantees no collisions: due to guaranteed fixed size of Ids between milliseconds and because multiple requests within the same ms lead to longer Ids with the prefix unique to the ms (tests included);
  • supports 32 instead of 16 workers (test included)

The algorithm uses less randomness than the original node.js implementation, which permits to extend the life span as well as reduce and guarantee the length. In general terms, each Id has the following 3 pieces of information encoded: the millisecond since epoch (first 8 symbols, epoch: 1/1/2016), the worker Id (9th symbol), the running concurrent counter within the millisecond (only if required, spanning over all remaining symbols).

The element of randomness per symbol is 1/2 for the worker and the millisecond data and 0 for the counter. The original algorithm of the node.js library uses 1/4 throughout. Here 0 means no randomness, i.e. every value is encoded using a 64-base alphabet directly; 1/2 means one of two matching symbols of the supplied alphabet is used randomly, 1/4 one of four matching symbols. All methods accepting the parameters that govern the randomness are exported and can be used to directly implement an algorithm with e.g. more randomness, but with longer Ids and shorter life spans.

License and copyright

Copyright (c) 2016. Oleg Sklyar and teris.io. MIT license applies. All rights reserved.

Original algorithm: Copyright (c) 2015 Dylan Greene, contributors. The same MIT license applies. Many thanks to Dylan for putting together the original node.js library, which inspired this "port":

Seed computation: based on The Central Randomizer 1.3. Copyright (c) 1997 Paul Houle ([email protected])

shortid's People

Contributors

asellappen avatar chenlujjj avatar shawntoffel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

shortid's Issues

index out of range

Hey Guys, I have a gin project actualy using this library, but sometimes send a panic error when generate a new SortID.

This is the error:
captura de pantalla de 2017-03-25 21-19-53

An this is my generation code:

sid, _ := shortid.New(1, shortid.DefaultABC, 2342)
ssid,e1 := sid.Generate()

What is the reason of this error?

Possible to mock?

Is it possible to mock shortids?

The key feature of shortid is to guarantee its uniqueness, however for testing purpose, is it possible to return a fixed series each time I run my tests? thx.

Add support for Gomodules

Even though this library doesn't use any external dependencies, it's annoying that it doesn't have a go.mod file.

Is it possible to add it?

use base58 alphabet

Does it possible to use only base58 default chars? I wan to avoid symbols that looks identically and also have double click - copy/paste friendly string

Is it Distributed safe?

Hi, thanks for this awesome library. My question is can it be use in a distributed environment?

Returning an error

First of all really good job creating this library! Coming from Node.js I was searching for a shortid alternative a year ago, you finally created it!

When can I expect this library to return an error?
Could this happen randomly, should I panic at this point? Or will this only happen during the first Generate() call?

I'm asking because my current ID generation module does not return an error and I'd prefer to keep my code this way.

Question: Is that safe to remove symbols from DefaultABC?

Is that safe to remove symbols from DefaultABC?
Basically, use the string "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ" instead.
I am using it to generate shorten URL and do not wish to see symbols from the URL.
Thanks a lot for your project.

Edited:

Sorry that I didn't try it before asking this question. I tried to remove those two symbols and return error message panic: alphabet must contain 64 unique characters. I think that means I can NOT remove symbols. It will be nice I can remove symbols from URL. The id does not have to be 100% unique. I will check the id existence before insert and re-generate if it exists.

worker id problem

when I deploy my application in a docker cluster (k8s or docker-compose), I have no method to assign a worker id for different container, I can only get an docker container ID in my app by $HOSTNAME.I want to know what is the best practice to use the lib or generate an shortid by myself?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.