Giter Club home page Giter Club logo

php-snowflake's Introduction

An ID Generator for PHP based on Snowflake Algorithm (Twitter announced).

build passed License Packagist Version Total Downloads

Description

Snowflake & Sonyflake algorithm PHP implementation 中文文档.

file

Snowflake is a network service that generates unique ID numbers at high scale with simple guarantees.

  1. The first bit is an unused sign bit.
  2. The second part consists of a 41-bit timestamp (in milliseconds) representing the offset of the current time relative to a certain reference time.
  3. The third and fourth parts are represented by 5 bits each, indicating the data centerID and workerID. The maximum value for both is 31 (2^5 -1).
  4. The last part consists of 12 bits, which represents the length of the serial number generated per millisecond per working node. A maximum of 4095 IDs can be generated in the same millisecond (2^12 -1).

If you want to generate unique IDs using the snowflake algorithm, you must ensure that sequence numbers generated within the same millisecond on the same node are unique. Based on this requirement, we have created this package which integrates multiple sequence number providers.

  • RandomSequenceResolver (Random Sequence Number, UnSafe)
  • FileLockResolver (Uses PHP file lock fopen/flock, Concurrency Safety)
  • RedisSequenceResolver (Redis psetex and incrby, Concurrency safety)
  • PredisSequenceResolver (redis psetex and incrby, Concurrency Safety)
  • LaravelSequenceResolver (Laravel Cache add lock mechanism)
  • SwooleSequenceResolver (swoole_lock for Concurrency Safety)

Requirement

  1. PHP >= 8.1
  2. Composer

Installation

$ composer require godruoyi/php-snowflake -vvv

# Install `predis/predis` package if you are using PredisSequenceResolver
$ composer require "predis/predis"

# Install `Redis` extensions if you are using RedisSequenceResolver
$ pecl install redis

# Install `Swoole` extensions if you are using SwooleSequenceResolver
$ pecl install swoole

Usage

  1. simple to use.
$snowflake = new \Godruoyi\Snowflake\Snowflake;

$snowflake->id();
// 1537200202186752
  1. Specify the data center ID and machine ID.
$snowflake = new \Godruoyi\Snowflake\Snowflake($datacenterId, $workerId);

$snowflake->id();
  1. Specify start time.
$snowflake = new \Godruoyi\Snowflake\Snowflake;
$snowflake->setStartTimeStamp(strtotime('2019-09-09')*1000); // millisecond

$snowflake->id();

The maximum value of a 41-bit timestamp (in milliseconds) can represent up to 69 years, so the Snowflake algorithm can run safely for 69 years. In order to make the most of it, we recommend setting a start time.

  1. Use Sonyflake
$sonyflake = new \Godruoyi\Snowflake\Sonyflake;

$sonyflake->id();

Advanced

  1. Used in Laravel.

Since the SDK is quite straightforward, we do not offer a specific extension for Laravel. However, you can easily integrate it into your Laravel project by following these steps.

// App\Providers\AppServiceProvider

use Godruoyi\Snowflake\Snowflake;
use Godruoyi\Snowflake\LaravelSequenceResolver;

class AppServiceProvider extends ServiceProvider
{
    /**
     * Register any application services.
     *
     * @return void
     */
    public function register()
    {
        $this->app->singleton('snowflake', function ($app) {
            return (new Snowflake())
                ->setStartTimeStamp(strtotime('2019-10-10')*1000)
                ->setSequenceResolver(new LaravelSequenceResolver($app->get('cache.store')));
        });
    }
}
  1. Custom

To customize the sequence number resolver, you need to implement the Godruoyi\Snowflake\SequenceResolver interface.

class YourSequence implements SequenceResolver
{
    /**
     *  {@inheritdoc}
     */
    public function sequence(int $currentMillisecond)
    {
          // Just test.
        return mt_rand(0, 1);
    }
}

// usage

$snowflake->setSequenceResolver(new YourSequence);
$snowflake->id();

And you also can use the Closure:

$snowflake = new \Godruoyi\Snowflake\Snowflake;
$snowflake->setSequenceResolver(function ($currentMillisecond) {
    static $lastTime;
    static $sequence;

    if ($lastTime == $currentMillisecond) {
        ++$sequence;
    } else {
        $sequence = 0;
    }

    $lastTime = $currentMillisecond;

    return $sequence;
})->id();

License

MIT

php-snowflake's People

Contributors

aaronchenghao avatar cyz-home avatar godruoyi avatar hernandev avatar simivar avatar wuhuaji0 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

php-snowflake's Issues

ID位数

设置不同的开始时间,生成的ID位置可能是18位,可能是19位,而且看某些平台都是1开头的19位,这是在18位上加了一个前导1?

时钟回拨问题

时钟回拨除了抛出异常外,能有其他的解决办法吗?

BUG

datacenter AND workerid will be NULL,Will cause duplicate id to be generated in the for loop

image
image

Add concurrent testing

According to the suggestion in #53, add perform concurrent testing for the following resolvers below.

RedisResolver/FileLockResolver/SwooleResolver

Drop unsupported PHP version & add typehints

As you probably know, PHP no longer actively supports versions below 8.1 (reference). I wonder if you would be open to dropping support for everything lower than 8.1 and releasing a new major version with PHP built-in type-hints added to every method and interface.

As this is a major change it would require a new version release (3.x). With this change, we could bundle other breaking changes if you are looking into doing any of them.

Why set a starting timestamp

Hello!

I'm trying to integrate this into my Laravel application and I had some questions surrounding setting the starting timestamp:

  • First and foremost, why would the starting timestamp be set to anything other than the Unix epoch (1970-01-01 00:00:00 Z)? This would be the most intuitive date and is generally what a timestamp is assumed to be based on.
  • Should I set my own starting timestamp or just leave the default? What are the implication of one over the other?
  • It seems like the default timestamp use is 2019-08-08 08:08:08 which is seemingly random?

Thanks!

parseId解析

这里打印的timestamp,每个几秒,都变成间隔了几个小时,看看是不是bug,还是我理解错误了,并不是时间?

    $temp = $snowflake->parseId($id,true);
    echo "time:",date("Y-m-d H:i:s",($temp["timestamp"])),"\n";

关于维护多个sequence的讨论

假设有users, orders两张数据表,通过snowflake来为他们生成唯一主键id.

如果需要使这两种id的sequence不同怎么处理比较好?

目前想法是实例化两个不同的单例,每个RedisSequenceResolver的前缀设置成不一样的。
不知道有没有更好的方法。

示例:

$this->app->singleton('userIdGenerator', function ($app) {
    return (new Snowflake(0, 0))
        ->setStartTimeStamp(strtotime('2022-07-20')*1000)
        ->setSequenceResolver(
            (new RedisSequenceResolver($app->get('redis')))->setCachePrefix('UserId')
        );
});

$this->app->singleton('orderIdGenerator', function ($app) {
    return (new Snowflake(0, 0))
        ->setStartTimeStamp(strtotime('2022-07-20')*1000)
        ->setSequenceResolver(
            (new RedisSequenceResolver($app->get('redis')))->setCachePrefix('OrderId')
        );
});

能否增加索尼雪花算法

能否增加索尼雪花算法,分别是 1-39-8-16

寿命(174年)长于雪花(69年)
它可以在比Snowflake(2 ^ 10)更多的分布式计算机(2 ^ 16)中工作
它可以在单个机器/线程中每10毫秒最多生成2 ^ 8个ID(比Snowflake慢)

go语言的实现:https://github.com/sony/sonyflake

ID duplication issues when using Snowflake with different SequenceResolvers and cache configurations

Hello everyone!

I'm currently testing Snowflake to implement it in an API I'm working on for generating unique IDs. I have the following script for running some tests and validating that the IDs are not duplicated.

$processes = 10; // Number of processes to create
$iterations = 1000; // Number of iterations per process

$ids = [];

// Function that generates IDs and checks for duplicates
$generateIds = function () use (&$ids, $iterations) {
    for ($i = 0; $i < $iterations; $i++) {
        $snowflake = App::make('snowflake');

        $id = $snowflake->id();

        if (in_array($id, $ids)) {
            echo "Duplicate ID found: $id\n";
        } else {
            $ids[] = $id;
        }
    }
};

// Create processes and execute the ID generation function in parallel
$childProcesses = [];
for ($i = 0; $i < $processes; $i++) {
    $pid = pcntl_fork();

    if ($pid == -1) {
        die("Error creating process.");
    } elseif ($pid == 0) {
        $generateIds();
        exit(0);
    } else {
        $childProcesses[] = $pid;
    }
}

// Wait for all processes to finish
foreach ($childProcesses as $pid) {
    pcntl_waitpid($pid, $status);
}

echo "Concurrency test finished.\n";

I have conducted 3 tests with different configurations in the Snowflake singleton.

Test 1

SequenceResolver: LaravelSequenceResolver
Cache driver: file
Singleton:

$this->app->singleton('snowflake', function (Application $app) {
    return (new Snowflake())
        ->setStartTimeStamp(strtotime('2019-10-10') * 1000)
        ->setSequenceResolver(new LaravelSequenceResolver($app->get('cache.store')));
});

Result: Many "Duplicate ID found: xxx" messages.

Test 2

SequenceResolver: LaravelSequenceResolver
Cache driver: redis
Singleton:

$this->app->singleton('snowflake', function (Application $app) {
    return (new Snowflake())
        ->setStartTimeStamp(strtotime('2019-10-10') * 1000)
        ->setSequenceResolver(new LaravelSequenceResolver($app->get('cache.store')));
});

Result: Many "Duplicate ID found: xxx" messages and some "RedisException read error on connection to redis:6379." messages.

Test 3

SequenceResolver: RedisSequenceResolver
Singleton:

$this->app->singleton('snowflake', function (Application $app) {
    $redis = new \Redis();
    $redis->connect(config('database.redis.cache.host'), (int) config('database.redis.cache.port'));
    $redis->auth(config('database.redis.cache.password'));
    $redis->select((int) config('database.redis.cache.database'));

    return (new Snowflake())
        ->setStartTimeStamp(strtotime('2019-10-10') * 1000)
        ->setSequenceResolver(new RedisSequenceResolver($redis));
});

Result: Works fine and no IDs are repeated.

Instead of a problem, I wanted to share my tests in case they are helpful to someone. In my case, Test 3 is the method that works for me and the one I'll be using. If anyone was able to successfully use the LaravelSequenceResolver as the SequenceResolver, I would like to know how you achieved it.

Greetings.

Add machineId to the Redis/Laravel sequence generator keys

Currently, the key for cache entries in LaravelSequenceResolver and RedisSequenceResolver is just the current time as integer. If the Redis instance is share between machines it would mean that each of the machines would use the same cache entries leading to limitation of 256 unique ids per millisecond per all machines rather than 256 unique ids per machine.

What do you think about prefixing it with the machineId in case of Sonyflake and datacenter and workerId in case of Snowflake?

expected return type is different from actual return type

Hello,

there is a problem in the consistency of the defined return type.
Expected: Int, Given String.

PHPDoc gives the hint, that the return type will be of type int

    /**
     * Get snowflake id.
     *
     * @return int
     */
    public function id()
    {

(source)

Actual return is casted to string

        return (string) ((($currentTime - $this->getStartTimeStamp()) << $timestampLeftMoveLength)
            | ($this->datacenter << $datacenterLeftMoveLength)
            | ($this->workerid << $workerLeftMoveLength)
            | ($sequence));

(source)

生成的id都是一样的

for($i=1;$i<=10;$i++)
{
$snowflake = new \Godruoyi\Snowflake\Snowflake1, 1);
$snowflake->setStartTimeStamp($time);
$snowflake->setSequenceResolver(new RandomSequenceResolver());
return $snowflake->id();
}
生成的id都是一样的

如何使生成的ID长度小一点呢?

首先感谢作者提供这么方便的包!

今天遇到了一个问题,后台生成的ID长度太大了,传递给前端之后,js因为数值太大失去了精度。请问修改什么配置能减小生成ID的长度

how to use it in lumen?

I refer to the documentation ,but it comes to
Call to undefined method Illuminate\Support\Facades\Cache::store()
how to resolve it?

Default value for datacenter and worderId is NULL which causes unpredictable ID formation.

In the SnowFlake constructor the $datacenter and $workerId have a NULL default value.
So, if these values are not provided, the relevant properties are set to NULL ( the function mt_rand(0, 31) is not used ).
The NULL value for these properties seems to be a problem when the code tries to do bitwise shift . The result seems to be unstable.

For example when a execute:

$snowflake = new Snowflake();

for($i=1; $i<50; $i++) {
    $id = $snowflake->id();
    echo "$id \n";
    if ($i % 5 == 0) {
        sleep(2);
    }
}

the datacenter and worker part of the IDs is not the same for all iterations. 🤔
Am I doing something wrong ?

EDIT: My bad! I misunderstood something. Feel free to delete the issue.

setStartTimeStamp not working.

I don't know how but it doesn't continue after setStartTimeStamp() function. Anyone know how to resolve?

$snowflake = new Godruoyi\Snowflake\Snowflake;
echo "seven";
try {
       $snowflake->setStartTimeStamp(strtotime('2019-09-09')*1000);
       echo $snowflake->getStartTimeStamp();
} catch (Exception $exception) {
       echo $exception;
}

echo "four";
$id = $snowflake->id();
echo $id;

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.