Giter Club home page Giter Club logo

aint-queue's Introduction

Aint Queue

Build Status License Php version Version

An async-queue library built on top of swoole, flexible multi-consumer, coroutine supported. 中文说明

queue_status.gif

Feature

  • Default Redis driver
  • Delayed job
  • The custom job retries and times
  • Custom failed callback
  • Job middleware
  • Queue snapshot event
  • Concurrent processing, flexible multi-worker
  • Worker coroutine support
  • Beautiful dashboard

Required

  • PHP 7.2+
  • Swoole 4.4+
  • Redis 3.2+ (redis driver)

Install

composer require littlesqx/aint-queue -vvv

Usage

Config

By default, aint-queue will require config/aint-queue.php as default config. If not exist, /vendor/littlesqx/aint-queue/src/Config/config.php will be the final config file.

<?php

use Littlesqx\AintQueue\Driver\Redis\Queue as RedisQueue;
use Littlesqx\AintQueue\Logger\DefaultLogger;

return [
    // channel_name => [...config]
    'default' => [
        'driver' => [
            'class' => RedisQueue::class,
            'connection' => [
                'host' => '127.0.0.1',
                'port' => 6379,
                'database' => '0',
                // 'password' => 'password',
            ],
        ],
        'logger' => [
            'class' => DefaultLogger::class,
            'options' => [
                'level' => \Monolog\Logger::DEBUG,
            ],
        ],
        'pid_path' => '/var/run/aint-queue',
        'consumer' => [
            'sleep_seconds' => 1,
            'memory_limit' => 96,
            'dynamic_mode' => true,
            'capacity' => 6,
            'flex_interval' => 5 * 60,
            'min_worker_number' => 5,
            'max_worker_number' => 30,
            'max_handle_number' => 0,
        ],
        'job_snapshot' => [
            'interval' => 5 * 60,
            'handler' => [],
        ],
    ],
];

All the options:

name type comment default
channel string The queue unit, every queue pusher and queue listener work for. Multiple channel supported, use --channel option. default
driver.class string Queue driver class, implements QueueInterface. Redis
driver.connection map Queue driver's config.
pid_path string The path of listener master pid file. Noted that permission required. /var/run/aint-queue
consumer.sleep_seconds int Sleep seconds after every empty pop from queue. 1
consumer.memory_limit int Mb. Worker will reload when its memory usage exceeds the limit. 96
consumer.dynamic_mode bool Determine whether worker's number flex dynamically. true
consumer.capacity int The capacity that every consumer can handle in health and in short time, it affects the worker number when dynamic-mode. 5
consumer.flex_interval int every flex_interval seconds monitor process try to flex the worker number. Only work when consumer.dynamic_mode = true. 5
consumer.min_worker_number int Min expansion. 5
consumer.max_worker_number int Max expansion. 30
consumer.max_handle_number int Current consumer's max job-handle time. 0 means no limit. 0
job_snapshot map Every interval seconds, handles will be executed. Handle must implements JobSnapshotterInterface.

Push job

You can use it in your project running via fpm/cli.

<?php

use Littlesqx\AintQueue\Driver\DriverFactory;

$queue = DriverFactory::make($channel, $options);

// push a job
$queue->push(function () {
    echo "Hello aint-queue\n";
});

// push a delay job
$closureJob = function () {
    echo "Hello aint-queue delayed\n";
};
$queue->push($closureJob, 5);

// And class job are allowed.
// 1. Create a class which implements JobInterface, you can see the example in `/example`.
// 2. Noted that job pushed should be un-serialize by queue-listener,
//    it means queue-pusher and queue-listener are required to in the same project.                                          
// 3. You can see more examples in `example` directory.

Manage Queue

We recommend that using Supervisor to monitor and control the listener.

vendor/bin/aint-queue
AintQueue Console Tool

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  help                 Displays help for a command
  list                 Lists commands
 queue
  queue:clear          Clear the queue.
  queue:dashboard      Start http server for dashboard.
  queue:reload-failed  Reload all the failed jobs onto the waiting queue.
  queue:status         Get the execute status of specific queue.
 worker
  worker:listen        Listen the queue.
  worker:reload        Reload worker for the queue.
  worker:run           Run the specific job.
  worker:stop          Stop listening the queue.

Testing

composer test

Contributing

You can contribute in one of three ways:

  1. File bug reports using the issue tracker.
  2. Answer questions or fix bugs on the issue tracker.
  3. Contribute new features or update the wiki.

The code contribution process is not very formal. You just need to make sure that you follow the PSR-2, PSR-12 coding guidelines. Any new code contributions must be accompanied by unit tests where applicable.

License

MIT

aint-queue's People

Contributors

littlesqx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

aint-queue's Issues

其他框架如何跟这个结合?

1.使用 http 或者 RPC 方式.塞入任务,并消费队列
2.其他框架直接将任务塞入 redis.由这个东东读取之后自动消费队列
????
另外,稳定吗?可用于生产吗

不支持参数吗??

就是推送(new)类或者一个方法的时候,不支持传递参数,后期就直接对这个参数进行处理,

supervisor使用问题

我使用supervisor来保证程序正常执行。
配置里面关键的只有一行

[program:aint-queue]
command=sh queue-start.sh

以下是queue-start.sh里面的内容

./vendor/bin/aint-queue worker:listen --channel=default

config就是用的默认配置
目前会出现多个moniter进程的情况,是我的启动脚本有问题吗?

about README

I think "A async-queue library" should be "An async-queue library"

感觉要放弃了

Fatal error: Uncaught TypeError: Return value of "Littlesqx\AintQueue\Console\Command\WorkerListenCommand::execute()" must be of the type int, "NULL" returned. in /Users/xxx/Documents/www/queue/vendor/symfony/console/Command/Command.php:258

aint-queuq: monitor进程很多

随运行时间增长,aint-queue:monitor进程越来越多,占用了大量内存

root      1373     1  0 18:40 ?        00:00:00 aint-queue: monitor#default
root      2952     1  0 18:45 ?        00:00:00 aint-queue: monitor#default
root      4504     1  0 18:50 ?        00:00:00 aint-queue: monitor#default
root      5992     1  0 18:55 ?        00:00:00 aint-queue: monitor#default
root      7442  7441  2 19:00 ?        00:00:02 aint-queue: master#default
root      7456  7442  0 19:00 ?        00:00:00 aint-queue: monitor#default
root      7457  7442  0 19:00 ?        00:00:00 aint-queue: consumer#default
root      7458  7442  0 19:00 ?        00:00:00 aint-queue: consumer#default
root      7459  7442  0 19:00 ?        00:00:00 aint-queue: consumer#default
root      7460  7442  0 19:00 ?        00:00:00 aint-queue: consumer#default
root      7461  7442  0 19:00 ?        00:00:00 aint-queue: consumer#default

我用的是supervisor来执行sh queue-start.sh脚本
queue-start.sh脚本内容如下:

./vendor/bin/aint-queue worker:listen --channel=default

docker 打包顺序有些问题

应该构建镜像的时候进行composer install, 你这边是先install 然后构建镜像,这样就会有一个问题,一般本地环境是没有swoole扩展的,所以第一步就不会成功

yii2高级模板 使用的时候 消费队列报错

Fatal error: Littlesqx\AintQueue\Worker\ConsumerWorker::handle(): The script tried to execute a method or access a property of an incomplete object. Please ensure that the class definition "api\jobs\SimpleJob" of the object you are trying to operate on was loaded before unserialize() gets called or provide an autoloader to load the class definition in var/project/vendor/littlesqx/aint-queue/src/Worker/ConsumerWorker.php on line 124

需要加到swoole首页吗

发现你这个项目很不错,现在swoole首页会放很多应用,需要帮放上去吗 需要联系我qq 393323503

问题请教

大佬,为什么要用到lua?
为什么有用到hexists、zscore等?
$object->getCompressingThreshold()这个方法是哪来的?
大佬,这个是咋写出来的?深刻理解swoole?
好多看不懂的,有难度

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.