Giter Club home page Giter Club logo

laravel-job-status's Introduction

Laravel Job Status

Latest Stable Version Total Downloads Build Status License

Laravel package to add ability to track Job progress, status and result dispatched to Queue.

  • Queue name, attempts, status and created/updated/started/finished timestamp.

  • Progress update, with arbitrary current/max value and percentage auto calculated

  • Handles failed job with exception message

  • Custom input/output

  • Native Eloquent model JobStatus

  • Support all drivers included in Laravel (null/sync/database/beanstalkd/redis/sqs)

  • This package intentionally do not provide any UI for displaying Job progress.

    If you have such need, please refer to laravel-job-status-progress-view

    or make your own implementation using JobStatus model

Requirements

  • PHP >= 7.1
  • Laravel/Lumen >= 5.5

Installation

Installation for Laravel

Installation for Lumen

Usage

In your Job, use Trackable trait and call $this->prepareStatus() in constructor.

<?php
namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Imtigger\LaravelJobStatus\Trackable;

class TrackableJob implements ShouldQueue
{
    use InteractsWithQueue, Queueable, SerializesModels, Trackable;

    public function __construct(array $params)
    {
        $this->prepareStatus();
        $this->params = $params; // Optional
        $this->setInput($this->params); // Optional
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        $max = mt_rand(5, 30);
        $this->setProgressMax($max);

        for ($i = 0; $i <= $max; $i += 1) {
            sleep(1); // Some Long Operations
            $this->setProgressNow($i);
        }

        $this->setOutput(['total' => $max, 'other' => 'parameter']);
    }
}

In your Job dispatcher, call $job->getJobStatusId() to get $jobStatusId:

<?php

class YourController {
    use DispatchesJobs;

    function go() {
        $job = new TrackableJob([]);
        $this->dispatch($job);

        $jobStatusId = $job->getJobStatusId();
    }
}

$jobStatusId can be used elsewhere to retrieve job status, progress and output.

<?php
$jobStatus = JobStatus::find($jobStatusId);

Troubleshooting

Call to undefined method ...->getJobStatusId()

Laravel provide many ways to dispatch Jobs. Not all methods return your Job object, for example:

<?php
YourJob::dispatch(); // Returns PendingDispatch instead of YourJob object, leaving no way to retrive `$job->getJobStatusId();`

If you really need to dispatch job in this way, workarounds needed: Create your own key

  1. Create migration adding extra key to job_statuses table.

  2. In your job, generate your own unique key and pass into prepareStatus();, $this->prepareStatus(['key' => $params['key']]);

  3. Find JobStatus another way: $jobStatus = JobStatus::whereKey($key)->firstOrFail();

Status not updating until transaction commited

On version >= 1.1, dedicated database connection support is added.

Therefore JobStatus updates can be saved instantly even within your application transaction.

Read setup step 6 for instructions.

Documentations

<?php
// Job protected methods (Call from your Job)
$this->prepareStatus();                           // Must be called in constructor before any other methods
$this->setProgressMax(int $v);                    // Update the max number of progress
$this->setProgressNow(int $v);                    // Update the current number progress
$this->setProgressNow(int $v, int $every);        // Update the current number progress, write to database only when $v % $every == 0
$this->incrementProgress(int $offset)             // Increase current number progress by $offset
$this->incrementProgress(int $offset, int $every) // Increase current number progress by $offset, write to database only when $v % $every == 0
$this->setInput(array $v);                        // Store input into database
$this->setOutput(array $v);                       // Store output into database (Typically the run result)

// Job public methods (Call from your Job dispatcher)
$job->getJobStatusId();                       // Return the primary key of JobStatus (To retrieve status later)

// JobStatus object fields
var_dump($jobStatus->job_id);                 // String (Result varies with driver, see note)
var_dump($jobStatus->type);                   // String
var_dump($jobStatus->queue);                  // String
var_dump($jobStatus->status);                 // String [queued|executing|finished|retrying|failed]
var_dump($jobStatus->attempts);               // Integer
var_dump($jobStatus->progress_now);           // Integer
var_dump($jobStatus->progress_max);           // Integer
var_dump($jobStatus->input);                  // Array
var_dump($jobStatus->output);                 // Array, ['message' => $exception->getMessage()] if job failed
var_dump($jobStatus->created_at);             // Carbon object
var_dump($jobStatus->updated_at);             // Carbon object
var_dump($jobStatus->started_at);             // Carbon object
var_dump($jobStatus->finished_at);            // Carbon object

// JobStatus generated fields
var_dump($jobStatus->progress_percentage);    // Double [0-100], useful for displaying progress bar
var_dump($jobStatus->is_ended);               // Boolean, true if status == finished || status == failed
var_dump($jobStatus->is_executing);           // Boolean, true if status == executing
var_dump($jobStatus->is_failed);              // Boolean, true if status == failed
var_dump($jobStatus->is_finished);            // Boolean, true if status == finished
var_dump($jobStatus->is_queued);              // Boolean, true if status == queued
var_dump($jobStatus->is_retrying);            // Boolean, true if status == retrying

Note

$jobStatus->job_id result varys with driver

Driver job_id
null NULL (Job not run at all!)
sync empty string
database integer
beanstalkd integer
redis string(32)
sqs GUID

laravel-job-status's People

Contributors

0x4c6565 avatar aidask avatar alexking avatar bagwaa avatar chrishubert avatar danny50610 avatar duxthefux avatar dverraes avatar imtigger avatar juans avatar lucasgiovanny avatar masbug avatar nickescobedo avatar patbriperso avatar pixellup avatar stephane-monnot avatar tarekadam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

laravel-job-status's Issues

Use Redis as driver

Hi, how can i use Redis as driver with laravel-job-status?
I didn't found an easy way to do.

  • Laravel version: 5.7
  • Laravel-job-status version: 0.2.2

Thanks

Custom job status model

Would be nice to be able to use our own job status model class when we need to add some features to it.

This should not be hard :

  • add a config file to provide the fully qualified class name
  • make migration publishable (#16)
  • use the config class to get the table name in the migration
  • use the config class here and there when we need to create the model

You have a good example of how to provide such configurable model feature in spatie/laravel-model-status package

Failed jobs not handled correctly

It would appear that failing a job via $this->fail() also causes JobProcessed event to be raised, overriding the failed status set when handling the previous JobFailed event.

I will submit a PR to rework the DefaultEventManager class to handle this

Problem with Trackable trait after updating to Laravel v5.8.34

Hello, thank you for the nice work.
Since upgrading to Laravel v5.8.34 I'm getting:

Fatal error: Trait method __sleep has not been applied, because there are collisions with other trait methods on <my job file "use" statement>

Seems like some other trait of these "Dispatchable, InteractsWithQueue, Queueable, SerializesModels" is using this method too..

Thanks

// Edit
Turned out this is SerializesModels trait, which you also are using in Imtigger\LaravelJobStatus\Trackable trait. So the temporary solution is not to use SerializesModels trait in the trackable jobs..

ShouldBeUnique trait is ignored when generating status rows in database

When queueing/monitoring a job with the ShouldBeUnique trait, the dispatch should not result in duplicate status rows in the DB.

I made a quick and dirty modification to Trackable.php based on a comment in the Lumen repo which sorts it out, but there's probably a cleaner way through:

protected function prepareStatus(array $data = [])
    {
         // > New condition
         if ($this instanceof ShouldBeUnique) {

            $uniqueId = method_exists($this, 'uniqueId')
                    ? $this->uniqueId()
                    : ($this->uniqueId ?? '');

            $cache = method_exists($this, 'uniqueVia')
                        ? $this->uniqueVia()
                        : Container::getInstance()->make(Cache::class);

            $uniqueFor = $this->uniqueFor ?? 0;

            $lock = $cache->lock( 'laravel_unique_job:' . get_class($this) . $uniqueId, $uniqueFor);

            // this job is already queued and should be unique. Do not track.
            if (true !== $lock->get()) {
                $this->shouldTrack = false;
            } else {
                // release the lock if we locked it, otherwise it won't queue
                $lock->release();
            }
        }
        // <

        if (!$this->shouldTrack) {
            return;
        }

        /** @var JobStatus */
        $entityClass = app(config('job-status.model'));

        $data = array_merge(['type' => $this->getDisplayName()], $data);
        /** @var JobStatus */
        $status = $entityClass::query()->create($data);

        $this->statusId = $status->getKey();

        return;
    }

php artisan vendor:publish --provider not working?

Hi there,

Been looking to manage the status of jobs and came across this package but unfortunately, I was unable to get it installed with the instructions from the readme.

I've pulled it in with composer, but when I run;

php artisan vendor:publish --provider=\Imtigger\LaravelJobStatus\LaravelJobStatusServiceProvider

It says Publishing complete. but there are no migrations or config file created.

I'm running PHP 7.1, Laravel 5.5.

Instead I just ran php artisan vendor:publish which then gives a list of files to publish, I can choose this one and it works ok. I don't know if this is a change that Laravel has made?

Status stay in retrying

Hi there. I am trying to use this great package at first time.
But I got one issue.
When job get exception, the status not changing to 'failed', but stay 'retrying'.
In DefaultEventManager I have changed exceptionOccurred method functionality to get what I need.

'status' => ($event->job->attempts() == $event->job->maxTries()) ? $this->getEntity()::STATUS_FAILED : $this->getEntity()::STATUS_RETRYING,

Add feature addOutput method

Hello,

I've build an 'addOuput' method so you can add multiple rows to the output. This for adding in run job sub-statuses.

If you want, I can make a pull request.

Kind regards,
Bastiaan

$this->setProgressMax() not woking?

Anybody encounter this?
$this->setProgressMax($max); has no effect on job_statuses table.

Anything necessary missing out ?

use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Trackable;

...
public function __construct($params)
{
    $this->prepareStatus();
}
...
 public function handle()
{
    $this->setProgressMax(100);

    ...

    foreach( ... )
    {
         ...
         $progress_now += 1;
         $this->setProgressNow($progress_now);
    }
}

Checking a job status using ajax

I am sending an ajax request to make a job to delete a folder tree. In FolderTreeJob I have a code look like this

class FolderDeleteJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Trackable;

    private $node;

    public $count = 0;
    public  $jobDone = 0;


    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(StorageFiles $node)
    {
        $this->prepareStatus();
        $this->count = $node->count();
        $this->node = $node;
    }

    /**
     * Execute the job.
     *
     * @return bool|array
     */
    public function handle()
    {

        $this->setProgressMax($this->count);

        $node = $this->node;

        $this->delete($node);

        $this->setOutput(['total' => $this->count]);

    }

    private function delete(StorageFiles $file)
    {
        $this->jobDone++;
        foreach ($file->nodes as $node) {
            $this->delete($node);
        }

//        $file->delete();
        $this->setProgressNow($this->jobDone);
    }
}

Than I am sending an ajax request to chekc a job status (get the progress_now value). My controller code is

$job = JobStatus::find($id)
But the $job is always null, but I have checked db, there is a row with id (for example id =1)

Monitor chained jobs?

I need to chain multiple jobs. First job ImageResize second ShareOnSites, maybe there will be more in the Future.
Till now I added them to the queue like

$job = ImageResize::withChain([
    new ShareOnSites
])->dispatch($image);

But i need to track them.

As far as I see it, for each job, even if they are chained, a new job status is created. So how can I track the second job as soon as the first one finished? I would store in in the image table as job_id column.

Can I get the second job_id in the handle() method of job two, or how would I have to do this?

undefined function config_path()

I'm trying to install on lumen project, when I run command to migrate a error is returned:

In LaravelJobStatusServiceProvider.php line 26:
Call to undefined function Imtigger\LaravelJobStatus\config_path()

Thank you

feature request: wait/resolve/join tasks

It would be nice to have some kind of convenience method that allows you to join (wait on the resolution of) multiple dispatched jobs. This is would be similar to how multi-threading/processing handles queue listeners. Use case would be some parent code that wanted to block waiting for all it's child jobs to complete. In terms of your software I think this would be something as simple as querying the status of an array of job ids every n interval. When they are all resolved (completed/failed) then continue.

Undefined Query

I get this error when the job is dispatched.
Call to undefined method Illuminate\Foundation\Application::query()
Points here as origin of error,
$status = $entityClass::query()->create($data)
Any idea on how to fix it??

Updating Job status within transactions

Having issues using this with transactions

$count = 100;
$this->setProgressMax($count);

DB::beingTransaction();
try {
    for ($i=1; $i <= $count; $i++) {
        $this->setProgressNow($i) // no easy way to do this outside the transaction! :(
        dump($i);
    }
}
catch (\Exception $e) {
    DB::rollBack();
    $this->setProgressNow(0);
    throw $e;
}
DB::commit();

Obviously, with this set up it goes from 0 to 100, because all progress counter is ticking up inside the transaction which doesn't commit until the end of the loop.

What I'm trying to achieve is a way to tick up the progress, whilst the actual work isn't committed until the end, and if an error occurs, set the progress back.

Make job status table configurable

Laravel does support configurable names for the job table name. As we're not using the standard jobs name for some developers it's sometimes kind of irritating the table names do not align and searching for a queue status table. It would be nice making the table name configurable like config/queue.php

job_id not stored into job_statuses

Stored job status in job_statuses table does not store job_id correspond to jobs table id

Expected Behavior

Store job_id for the current job status / progress

Current Behavior

job_id column in job_statuses table is NULL

My Environment

  • Laravel used: v.5.4.28
  • Laravel job status used: v0.1.7

When using scheduler a new entry in job_statuses is created

Hello,
First thanks for this library.

When I'm using $schedule->job(new MyJob(['params'])->days() in Kernel.php and php artisan schedule:run for scheduling it creates a new entry in job_statuses table.

Currently i found a workaround, i'm sending a new param to the Job ('withoutProgress') that act as a flag and didn't execute the function prepareStatus, but to me it's ugly.

Is it possible to make prepareStatus function not executed when it comes from the scheduler ?

I'm on Laravel 5.8 - Php 7.2

Broken? Can't get the model

Is this plugin broken?

Publishing is not working, and can't get the model..
Is says "Publishing complete", but creates no file in the config folder.

Sending Mail job stays queued status

Hello,

I wish to track mail sending using Illuminate\Mail\Mailable. I do get an entry for the job in the job_statuses table, but the status remains queued while the job is executed. I use database as queue driver.

namespace App\Mail;

use Illuminate\Bus\Queueable;
use Illuminate\Mail\Mailable;
use Illuminate\Queue\SerializesModels;
use Imtigger\LaravelJobStatus\Trackable;

class SendMail extends Mailable
{
    use Queueable, SerializesModels, Trackable;

    public function __construct()
    {
        $this->onQueue('default');
        $this->prepareStatus();
    }

    public function build() {
        $this->subject('Test)->view('emails.test_view');
    }
}

Job status not changing from 'executing'

Hi,

I try to use Laravel Job Status for monitoring my queue. However, my jobs do not exit 'executing' status despite having finished or stopped with error message. The output is written to the output column.
I have no idea what logs should I look at to find out the problem.

Abort/cancel queued jobs

Hello, I'd like to know if there is a way to cancel a job which has been queued.
I would delete the entry from the jobs table, but the job_statuses.job_id is NULL until the job is running, although I've included Imtigger\LaravelJobStatus\LaravelJobStatusServiceProvider::class, in the service providers.

Undefined index: command

Laravel version 5.3 & 6.9.
laravel-job-status version: 0.2.2

I see the following occurring in the Laravel log file:

[2020-01-06 13:46:02] production.ERROR: Undefined index: command  
[2020-01-06 13:46:03] production.ERROR: Undefined index: command  
[2020-01-06 13:46:15] production.ERROR: Undefined index: command  
[2020-01-06 13:46:15] production.ERROR: Undefined index: command  

This happens when the queue processes an event with a ShouldQueue listener.

I'm not certain why laravel-job-status is being invoked in this situation (neither the event nor the listener have the Trackable trait.

Nevertheless, I tracked it down to the updateJobStatus() method in LaravelJobStatusServiceProvider.php.

Here's a screenshot of the error occurring whilst debugging in PHPStorm.
Screenshot_2019-12-17_at_11 16 05

I gather that you're working on a rewrite at the moment and can see that the LaravelJobStatusServiceProvider.php code has changed significantly in GitHub from what I'm pulling down with Composer.

TypeError: Argument 1 passed to Imtigger\LaravelJobStatus\LaravelJobStatusServiceProvider::updateJobStatus() must be an instance of Illuminate\Contracts\Queue\Job

I am trying to write a test for my Laravel Job which uses this library.
The thought process is this:

  • When creating a new job instance using new ConvertAudio() the status in the job_statuses table will be queued
    • this is expected since none of the events required by this library (e.g. Illuminate\Queue\Events\JobProcessed) fires when you try to test a job like this
  • I manually fire the JobProcessed event
  • I then check if my code uses the JobStatus correctly after it's filled

I am running into a little incompatibility after firing the JobProcessed event.

Here is my code and the exception:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use App\AudioMedia;
use Imtigger\LaravelJobStatus\Trackable;

class ConvertAudio implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Trackable;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(AudioMedia $model, array $options)
    {
        $this->prepareStatus();
        /* ... */
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        /* ... */
    }
<?php

namespace Tests\Unit;

use Tests\TestCase;
use Illuminate\Foundation\Testing\DatabaseTransactions;
use App\Jobs\ConvertAudio;
use App\AudioMedia;
use Illuminate\Queue\Events\JobProcessed;

class ConvertAudioJobTest extends TestCase
{
    //use DatabaseTransactions;
    /* --- */
    
    public function testJobStatusShouldBeCompleted()
    {
        $model = factory(AudioMedia::class)->create();
        $ffmpegMock = /*...*/
        
        $job = new ConvertAudio($model, [
            ['format' => new \FFMpeg\Format\Audio\Mp3()]
        ]);
        $job->ffmpeg = $ffmpegMock;
        $job->handle();
        
        event(new JobProcessed("foo", $job));
    }
}
TypeError: Argument 1 passed to Imtigger\LaravelJobStatus\LaravelJobStatusServiceProvider::updateJobStatus() must be an instance of Illuminate\Contracts\Queue\Job, instance of App\Jobs\ConvertAudio given, called in /usr/share/nginx/html/vendor/imtigger/laravel-job-status/src/LaravelJobStatusServiceProvider.php on line 35

If I try to implement Illuminate\Contracts\Queue\Job (by extending Illuminate\Queue\Jobs\Job) I get this:

PHP Fatal error:  Illuminate\Queue\Jobs\Job and Illuminate\Bus\Queueable define the same property ($queue) in the composition of App\Jobs\ConvertAudio. However, the definition differs and is considered incompatible. Class was composed in /usr/share/nginx/html/app/Jobs/ConvertAudio.php on line 21

Am I missing something?

Edit:
I am using

  • laravel/framework v5.4.28
  • imtigger/laravel-job-status v0.1.9
  • PHP 7.1.8

MongoDB database

Hi,

I am trying to use this package with a MongoDB database.
I am using jenssegers/mongodb package for all my models. I saw that the JobStatus model provided with the package is using Illuminate\Database\Eloquent\Model which breaks my app. I am getting an error everytime I am dispatching a new job:

2018-08-20 10:04:18] local.ERROR: Call to a member function prepare() on null

Is there any way I can use this together with jenssegers/mongodb?

Thank you.

Associate jobs with user?

Is is possible to associate dispatched jobs with the dispatching user? I would like to display to a logged in user the list of jobs he has dispatched, but not the jobs belonging to other users. I've tried adding a "user_id" column in the jobs_statuses table, but creating a job fails with Field 'user_id' doesn't have a default value...

Job status stays as 'queued' when dispatch fails

When dispatching a job fails because of a missing connection, a record remains in the jobs table with the status of queued. This can be handled manually by wrapping each dispatch in a try...catch block and updating the record manually, but ideally this case should be handled by the library.

Make migration publishable

Currently we cannot publish the migrations. This could be useful for people who need to customize it (e.g. custom JobStatus model).

Too many queries

Thank you for a very useful package, but it has one issue - it's sending too many queries. Every time you want to make an update, it's making one more query to get an object from database.
Below is a code example and the result

public function index()
{
    $this->prepareStatus();

    $max = 20;
    $this->setProgressMax($max);

    for ($i = 0; $i <= $max; $i += 1) {
        $this->setProgressNow($i);
    }

    $this->setOutput(['total' => $max, 'other' => 'parameter']);
}

image

Config file does not get published

When running

php artisan vendor:publish --provider="Imtigger\LaravelJobStatus\LaravelJobStatusServiceProvider"

The config file job-status.php does not get published.

It looks like it's simply an incorrect path in LaravelJobStatusServiceProvider.php (note the database one has a leading / but the config one doesn't):

        $this->publishes([
            __DIR__ . '../database/migrations/' => database_path('migrations'),
        ], 'migrations');

        $this->publishes([
            __DIR__ . '/../config/' => config_path(),
        ], 'config');

Tracking queued mailables

How can I set my mailable classes implemented with queue to be tracked? Currently, the queued job gets created on job_statuses table but neither it has job_id nor do it gets updated though the job gets completed successfully.

Convenience method to increment progress by $step

Great package.

Falls down a little bit when not using a for loop, such as while or foreach

       foreach ($items as $item) {
            sleep(1); // Some Long Operations
            $this->setProgressNow( ???? );
        }

Would be nice to be able to do:

       foreach ($items as $item) {
            sleep(1); // Some Long Operations
            $this->incrementProgress(); // progress_now+1
        }

And allow the increment size to be customized

       foreach($items as $item) {
            sleep(1); // Some Long Operations
            $this->incrementProgress($step); // skip ahead, progress_now + $step
        }

I'd let the progress_now go above the progress_max without error, as that can be useful in various cases.

Thoughts?

[Improvement] Be aware of jobs retries

First, thanks for this great repo that has helped me a lot to build an UI (based on VueJS) to watch my jobs execution.

The most difficult part for me has been to have the right status when a job failed but can be retried later.
I decide to throw an exception each time one of my job failed (I don't know if everyone do that) and I see that Laravel Queue emit a JobExceptionOccurred event in that case.
You have 2 event managers: DefaultEventManager and LegacyEventManager. The first one change the status to 'retrying' when an exception occurs, the second one change the status to 'failed'.

Now, when a job throw an exception and can be retried, Laravel Queue emits only a JobExceptionOccurred but when the job throw an exception and can't be retried, Laravel Queue emits first a JobFailed and second a JobExceptionOccurred. So even with DefaultEventManager the resulting status is 'retrying' but should be 'failed'.

I write on my project a AttemptsAwareEventManager that do some computations when JobExceptionOccurred event is emitted to change status to 'retrying' or 'failed' depending on job retrying conditions.

Are you interested by a pull request for this improvement ?

Beanstalk "Server reported NOT_FOUND" with exceptions

I might be doing something wrong here, but I'm running into an issue where, if the Laravel queue is set to use Beanstalk, job exceptions are not handled correctly.

The crux of the issue appears to be that once the job fails, it disappears from Beanstalk.
So when the DefaultEventManager class tries to call $event->job->attempts() (which gets forwarded to the queue connection), it fails.

Full stacktrace

[2021-01-27 17:30:14] local.ERROR: Server reported NOT_FOUND {"exception":"[object] (Pheanstalk\\Exception\\ServerException(code: 0): Server reported NOT_FOUND at vendor/pda/pheanstalk/src/YamlResponseParser.php:39)
[stacktrace]
#0 vendor/pda/pheanstalk/src/Connection.php(114): Pheanstalk\\YamlResponseParser->parseResponse()
#1 vendor/pda/pheanstalk/src/Pheanstalk.php(369): Pheanstalk\\Connection->dispatchCommand()
#2 vendor/pda/pheanstalk/src/Pheanstalk.php(286): Pheanstalk\\Pheanstalk->dispatch()
#3 vendor/laravel/framework/src/Illuminate/Queue/Jobs/BeanstalkdJob.php(91): Pheanstalk\\Pheanstalk->statsJob()
#4 vendor/imtigger/laravel-job-status/src/EventManagers/DefaultEventManager.php(42): Illuminate\\Queue\\Jobs\\BeanstalkdJob->attempts()
#5 vendor/imtigger/laravel-job-status/src/LaravelJobStatusServiceProvider.php(48): Imtigger\\LaravelJobStatus\\EventManagers\\DefaultEventManager->exceptionOccurred()
#6 vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(347): Imtigger\\LaravelJobStatus\\LaravelJobStatusServiceProvider->Imtigger\\LaravelJobStatus\\{closure}()
#7 vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(196): Illuminate\\Events\\Dispatcher->Illuminate\\Events\\{closure}()
#8 vendor/laravel/framework/src/Illuminate/Queue/Worker.php(510): Illuminate\\Events\\Dispatcher->dispatch()
#9 vendor/laravel/framework/src/Illuminate/Queue/Worker.php(390): Illuminate\\Queue\\Worker->raiseExceptionOccurredJobEvent()
#10 vendor/laravel/framework/src/Illuminate/Queue/Worker.php(358): Illuminate\\Queue\\Worker->handleJobException()
#11 vendor/laravel/framework/src/Illuminate/Queue/Worker.php(300): Illuminate\\Queue\\Worker->process()
#12 vendor/laravel/framework/src/Illuminate/Queue/Worker.php(134): Illuminate\\Queue\\Worker->runJob()
#13 vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(112): Illuminate\\Queue\\Worker->daemon()
#14 vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(96): Illuminate\\Queue\\Console\\WorkCommand->runWorker()
#15 [internal function]: Illuminate\\Queue\\Console\\WorkCommand->handle()
#16 vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(32): call_user_func_array()
#17 vendor/laravel/framework/src/Illuminate/Container/Util.php(36): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#18 vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(90): Illuminate\\Container\\Util::unwrapIfClosure()
#19 vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(34): Illuminate\\Container\\BoundMethod::callBoundMethod()
#20 vendor/laravel/framework/src/Illuminate/Container/Container.php(590): Illuminate\\Container\\BoundMethod::call()
#21 vendor/laravel/framework/src/Illuminate/Console/Command.php(202): Illuminate\\Container\\Container->call()
#22 vendor/symfony/console/Command/Command.php(255): Illuminate\\Console\\Command->execute()
#23 vendor/laravel/framework/src/Illuminate/Console/Command.php(189): Symfony\\Component\\Console\\Command\\Command->run()
#24 vendor/symfony/console/Application.php(1011): Illuminate\\Console\\Command->run()
#25 vendor/symfony/console/Application.php(272): Symfony\\Component\\Console\\Application->doRunCommand()
#26 vendor/symfony/console/Application.php(148): Symfony\\Component\\Console\\Application->doRun()
#27 vendor/laravel/framework/src/Illuminate/Console/Application.php(93): Symfony\\Component\\Console\\Application->run()
#28 vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(131): Illuminate\\Console\\Application->run()
#29 artisan(37): Illuminate\\Foundation\\Console\\Kernel->handle()
#30 {main}
"} 

retry the job

i have using the transaction to catch the exceptions,the job status always transform to finished,
how can i handle or retry?

Job result design question

Hello,

I'm searching for the best approach to fill the job result with errors encountered during job execution. I have large loop which executes all kind of stuff in various classes. Some methods are very deep, so back-porting all result back all the way back to the job loop is hard to do. Perhaps I can fill the job result with an custom exception?

Any help is much appreciated.

Thanks,
Bastiaan

FatalErrorException, laravel worker can't start because of this package.

Symfony\Component\Debug\Exception\FatalErrorException: method_exists(): The script tried to execute a method or access a property of an incomplete object. Please ensure that the class definition "App\Jobs\LogVwoEvent" of the object you are trying to operate on was loaded _before_ unserialize() gets called or provide an autoloader to load the class definition
#0 /vendor/imtigger/laravel-job-status/src/JobStatusUpdater.php(73): null

I have an old job in laravel queue which has dependencies of removed class LogVwoEvent. Because of this laravel worker fails to start, because laravel-job-status can not handle this error. Without this package laravel would discard this job without any issues.

How can I define the queue to use when dispatching the job?

This is written in the Readme.md:

In your Job dispatcher, call $job->getJobStatusId() to get $jobStatusId:

> <?php
> $job = new TrackableJob([]);
> $this->dispatch($job);
> 
> $jobStatusId = $job->getJobStatusId();

How can I dispatch the job to another queue instead of "default"?

I already tried:
$this->dispatch($job)->onQueue("custom_queue");

The laravel log contains these messages:

[2019-02-12 23:58:42] dev.ERROR: Call to a member function onQueue() on integer {"exception":"[object] (Symfony\Component\Debug\Exception\FatalThrowableError(code: 0)
: Call to a member function onQueue() on integer at /home/ansible/apps/laravel/single-project/laravel-ffmpeg-cluster/ffmpeg-cluster/app/Http/Controllers/TranscodeControll
er.php:125)
[stacktrace]
#0 [internal function]: App\Http\Controllers\TranscodeController->encode(Object(Illuminate\Http\Request))
#1 /home/ansible/apps/laravel/single-project/laravel-ffmpeg-cluster/ffmpeg-cluster/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): call_user_func_arra
y(Array, Array)

Thanks in advance.

Use with Laravel-Excel's importing feature

I'm trying to use this with the laravel-excel package which allows chunked importing of CSV data. (https://docs.laravel-excel.com/3.1/imports/queued.html#queuing-chunks)

Laravel version: 5.6.*

The problem I'm running into seems to be in how the job-status package needs the "dispatch" method to happen a certain way, but the laravel-excel package uses a "queue" method that doesn't end up sending out events or SOMETHING which is causing the job status to not be updated. I am having the darndest time trying to track it down.

I apologize, I am ignorant here, and a beginner at this.

Here is some example code:
From the controller where I pick up an API call (this is the excel implementation):
$importer = new $job(array_merge($params, $metaParams));
$importer->queue($bucket . '/' . $uploadFilePath, 'minio');

The job being referenced is:
class BaseBulkImport implements ShouldQueue, WithEvents
{
use Trackable, RegistersEventListeners, InteractsWithQueue, Queueable, SerializesModels;

When I attempt to dispatch the import job as noted in the job-status documentation
$importer = new $job(array_merge($params, $metaParams));
$this->dispatch($importer);

I end up with this problem:
Argument 2 passed to ...::queue() must be of the type string or null, object given, called in /opt/anorak/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php on line 157",

I know it's a long shot, but if you have any pointers, I'd be forever indebted.

Track batch jobs

Hi guys
How can I track a batch job?
For example I dispatch a job via Bus::chain() method and I want to when all job done the status job set to finished.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.