kristianedlund / laravel-external-queue Goto Github PK

I have a distributed architecture where multiple non-Laravel apps will be posting queue messages for my Laravel app to consume. This package looks very helpful in making that happen!

However, you assume the presence of a job attribute in the payload, and then look for a corresponding handler in the config file (at externalqueue.handlers.job).

How about adding a externalqueue.handlers.default or externalqueue.defaultHandler to the config file, which gets used anytime the payload doesn't contain a job attribute? I need to support situations where the queueing app didn't provide a job in the payload.

Hey @kristianedlund, great package!

While trying different scenarios, we noticed failed jobs cannot be tried multiple times because each time their payload is stored in the failed_jobs table, the format is lost and it keeps burying the original message body deeper each time the job fails.

We actually tried another package which is very similar to yours (https://github.com/dusterio/laravel-plain-sqs) and it suffers from the same issue (I am copying and pasting the description here as well). Perhaps you have any ideas?

You can see this behaviour by setting up a job that throws an Exception, which will cause it to go into the failed_jobs table. Retry the job a few times using php artisan queue:retry ID and observe the entire payload being re-serialized into the data property each time.

Here is what the payload looks like for a native Laravel failed job (using AWS SQS as well):
{"job":"Illuminate\\Queue\\CallQueuedHandler@call","data":{"command":"O:39:\"App\\Jobs\\TestSQSJobHandler\":5:{s:16:\"\u0000*\u0000song_id\";i:3;s:10:\"connection\";N;s:5:\"queue\";s:14:\"songsToProcess\";s:5:\"delay\";N;s:6:\"\u0000*\u0000job\";N;}"}}

Here is the payload for a failed job using laravel-external-queue:

First time it fails:
{"job":"App\\Jobs\\TestSQSJobHandlerPlain@handle","data":{"song_id":55,"artist":"Unknown"}}

After trying once via php artisan queue:retry job_id:
{"job":"App\\Jobs\\TestSQSJobHandlerPlain@handle","data":{"job":"App\\Jobs\\TestSQSJobHandlerPlain@handle","data":{"song_id":55,"artist":"Unknown"}}}

After trying twice via php artisan queue:retry job_id:
{"job":"App\\Jobs\\TestSQSJobHandlerPlain@handle","data":{"job":"App\\Jobs\\TestSQSJobHandlerPlain@handle","data":{"job":"App\\Jobs\\TestSQSJobHandlerPlain@handle","data":{"song_id":55,"artist":"Unknown"}}}}

Notice how each time the job is re-tried it re-encodes the data for the previous job, rendering the actual job unusable since the handler expects the data to contain specific JSON data such as song_id and artist.

Thanks!