discogs / pymongo-job-queue Goto Github PK

For some reason the coverage badge isn't working anymore, but the repo is still on codecov: https://codecov.io/github/discogs/pymongo-job-queue

Traceback (most recent call last):
  File "Z:/spdaemon/app.py", line 184, in _runme
    message = self._jobqueue.next().get('data', None)
  File "Z:\Z_VirtualEnv\lib\site-packages\pymjq\jobqueue.py", line 42, in next
    tailable=True)
  File "Z:\Z_VirtualEnv\lib\site-packages\pymongo\collection.py", line 1137, in find
    return Cursor(self, *args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'tailable'
Exception ignored in: <bound method Cursor.__del__ of <pymongo.cursor.Cursor object at 0x0000000003918048>>
Traceback (most recent call last):
  File "Z:\Z_VirtualEnv\lib\site-packages\pymongo\cursor.py", line 214, in __del__
    if self.__id and not self.__killed:
AttributeError: 'Cursor' object has no attribute '_Cursor__id'

This is just in a client code daemon when calling
message = self._jobqueue.next().get('data', None)

Any Ideas ?

Python 3.4.4
Windows 8 x64
Latest MongoDb

need to choose a license.... which one makes the most sense?
http://choosealicense.com/

When you use the job queue, it prints waiting! every 5 seconds if there are no jobs in the queue.
https://github.com/discogs/pymongo-job-queue/blob/master/pymjq/jobqueue.py#L95

It would be awesome to have a silent kwarg on JobQueue initialization so we don't see the waiting! message clogging up the terminal in development.

Maybe something like:

from pymongo import MongoClient
from pymjq import JobQueue
client = MongoClient("localhost", 27017)
db = client.job_queue
jobqueue = JobQueue(db, silent=True)

Need to figure out how to unit test JobQueue.__iter__...
see https://github.com/discogs/pymongo-job-queue/blob/master/pymjq/test.py#L50

Due to the cursor staying open and waiting for new documents to enter the queue, this method never returns unless given a KeyboardInterrupt.

A possible solution might be to use a timeout decorator: https://pypi.python.org/pypi/timeout-decorator

In v2.0.0 it would be great if we introduced channels to the queue. You should be able to assign a worker to listen to a specific channel with full pub/sub capabilities.