Comments (12)
@ouhouhsami you can pass an connection
instance to send_async_mail
to ensure that it uses the same connection
. Something like this should work without requiring you to change your production code.
@job
def send_async_mail(connection=None):
send_mail('foo', 'bar', '[email protected]', ['[email protected]'], fail_silently=False, connection=connection)
def send_custom_mail():
send_async_mail.delay(connection=get_connection())
from django-rq.
Well, thanks for your answer to my first point for asynchrone mail testing.
For the signals test, I'm a bit confused. Here is the python example, using the mock_signal_receiver (https://github.com/dcramer/mock-django/blob/master/mock_django/signals.py) :
# in jobs.py for example
@job
def send_async_signal():
print my_custom_signal.receivers # I got the one created below by mock_signal_receiver
print my_custom_signal.receivers[0][1]().call_count # I have 0, which is right
my_custom_signal.send(sender=Foo)
print my_custom_signal.receivers[0][1]().call_count # I have 1, which is right too, and 'should' be the case at the end of my testcase (see below).
def send_custom_signal():
send_async_signal.delay()
# in tests.py
from django_rq import get_worker
from mock_django import mock_signal_receiver
from jobs import send_custom_signal
class AsyncSendSignalTestCase(unittest.TestCase):
def test_async_signal(self):
with mock_signal_receiver(my_custom_signal) as receiver:
print my_custom_signal.receivers # I have exactly the same receiver as the first print in send_sync_signal()
send_custom_signal()
print receiver.call_count # I get 0, and it's right, because the signal is sent in an asynchronous job that hasn't been yet treated.
get_worker().work(burst=True)
print receiver.call_count # I get 0, and not 1 as I would have expected.
print my_custom_signal.receivers[0][1]().call_count # I also get 0 here (in send_async_signal, the value was 1)
The thing that is done by mock_signal_receiver is just to connect the signal to a receiver.
The problem is that the receiver isn't reached when doing asynchronous jobs, even if, as I wrote in the comments in my example above, my_custom_signal has the "right" reference to the mock_signal_receiver.
Regards,
from django-rq.
I tried to get rid of mock library, but don't really succeed. Here is the code for the test:
# in jobs.py for example
@job
def send_async_signal():
my_custom_signal.send(sender=Foo)
def send_custom_signal():
send_async_signal.delay()
# in tests.py
from django_rq import get_worker
from django.dispatch import receiver
from jobs import send_custom_signal
def count_calls(fn):
def _counting(*args, **kwargs):
_counting.calls += 1
return fn(*args, **kwargs)
_counting.calls = 0
return _counting
@receiver(my_custom_signal)
@count_calls
def my_custom_signal_callback(sender, **kwargs):
print 'well, callback reached'
class AsyncSendSignalTestCase(unittest.TestCase):
def test_async_signal(self):
send_custom_signal()
print my_custom_signal_callback.calls # 0, which is right
get_worker().work(burst=True)
print my_custom_signal_callback.calls # 0, which is wrong, it should return 1
from django-rq.
@ouhouhsami I think it's better to queue the long running function instead of sending an asynchronous signal because it adds another layer of indirection with no obvious benefit (in my opinion).
So instead of doing:
def long_running_function(*args, **kwargs):
# do stuff
my_custom_signal.connect(long_running_function)
@job
def send_async_signal():
my_custom_signal.send(sender=Foo)
def send_custom_signal():
send_async_signal.delay()
class AsyncSendSignalTestCase(unittest.TestCase):
def test_async_signal(self):
send_custom_signal()
print my_custom_signal_callback.calls # 0, which is right
get_worker().work(burst=True)
print my_custom_signal_callback.calls # 0, which is wrong, it should return 1
I would just do:
@job
def long_running_function(*args, **kwargs):
# do stuff
class MyTestCase(unittest.TestCase):
def test_async_job(self):
long_running_function.delay()
get_worker().work(burst=True)
# Check that work gets done
from django-rq.
Thanks for your answer @selwin . The fact is that the things aren't so simple in my real use case.
You are totally right, refactoring my example, but in my case I need this design: the possibility to send signal in an asynchronous function.
I investigated a lot and didn't find any solution to face that problem. I inspected the function ids, and they are the same, I also used weak=False for the signal, it doesn't changed anything. What appears it that the property 'calls' (using my decorator) or 'call_count' (using mock library) are updated 'inside the job function', but when I try to get this attribute inside my test after a get_worker().work(burst=True) , it is reset, even if the functions are the same 'instance' in memory.
Regards,
from django-rq.
Hello,
After all, I reopen this issue, with a test code sample that should achieve my goal, but doesn't pass for thz test_async_signal
function (it uses https://github.com/dcramer/mock-django to mock signal receive)
from django_rq import job
from django.dispatch import Signal
from mock_django import mock_signal_receiver
# set up a Signal
bar = Signal()
# the async function
@job
def async_foo():
bar.send(sender='baz')
def foo():
async_foo.delay()
# the sync function for demonstration purpose
def sync_foo():
bar.send(sender='baz')
class DjangoRQSignalTestCase(unittest.TestCase):
def test_async_signal(self):
with mock_signal_receiver(bar) as receiver:
foo()
get_worker().work(burst=True)
self.assertEquals(receiver.call_count, 1)
def test_sync_signal(self):
with mock_signal_receiver(bar) as receiver:
sync_foo()
self.assertEquals(receiver.call_count, 1)
I'd like your help on this subject.
Regards,
from django-rq.
Hi there,
The code above works, but just not during testing, right? It seems like the mocked received didn't receive the signal, did you try opening an issue at mock-django's repository?
I've never used mock although I've heard good things about it, I haven't delved that deep into signals either so I'm not sure I can be of much help. I'll try to have a look at it this weekend.
In the mean time, I'll reopen this issue to give this a better visibility.
from django-rq.
Thanks for reopening this issue. As you say: the code works, but not for during testing.
I wrote the test_sync_signal function to show that it works without the async process.
So, here is a version without mock-django, but using evil global :
from django.utils import unittest
from django_rq import job
from django.dispatch import Signal
# set up a Signal
bar = Signal()
globvar = 0
# the async function
@job
def async_foo():
global globvar
globvar = 1
def foo():
async_foo.delay()
# the sync function for demonstration purpose
def sync_foo():
global globvar
globvar = 1
class DjangoRQSignalTestCase(unittest.TestCase):
def test_async_signal(self):
foo()
get_worker().work(burst=True)
self.assertEquals(globvar, 1) # this should pass, but globvar is 0
def test_sync_signal(self):
sync_foo()
self.assertEquals(globvar, 1) # this pass
I think this is more or less the same problem as above.
from django-rq.
I played around with it a little bit, no luck. However, when I fix #15 , you'll be able to do this, which works:
global globvar
globvar = 0
def async_foo(async=True):
q = get_queue(async=async)
q.enqueue(sync_foo)
# the sync function for demonstration purpose
def sync_foo():
global globvar
globvar = 1
class DjangoRQSignalTestCase(TestCase):
def test_async_signal(self):
async_foo(async=False)
self.assertEquals(globvar, 1)
def test_sync_signal(self):
sync_foo()
self.assertEquals(globvar, 1) # this pass
from django-rq.
Hi there,
So, I see you have changed the code base adding async param now: a57ee60
The thing I finally have to 'hack' is to allow job to be run synchronously.
My decorators.py is now like this, setting async param to False in a job decorator, run the job synchronously:
from rq.decorators import job as _rq_job
from .queues import get_queue
def job(func_or_queue, connection=None, *args, **kwargs):
"""
The same as RQ's job decorator, but it works automatically works out
the ``connection`` argument from RQ_QUEUES.
And also, it allows simplified ``@job`` syntax to put job into
default queue.
"""
async = kwargs.pop('async', False)
if callable(func_or_queue):
func = func_or_queue
queue = 'default'
else:
func = None
queue = func_or_queue
if isinstance(queue, basestring):
try:
queue = get_queue(queue, async=async)
if connection is None:
connection = queue.connection
except KeyError:
pass
decorator = _rq_job(queue, connection=connection, *args, **kwargs)
if func:
return decorator(func)
return decorator
BTW, I no longer need get_worker().work(burst=True) in my tests, I have just added a SYNC value (True/False) in my django settings, and set the async param to settings.SYNC. The tests pass.
In your point of view, do you see any bad method to do it like this ? More precisely, is async=True in get_queue more or less the same as burst=True mode ?
Regards,
from django-rq.
I originally added the async
argument to Queue
for debugging purposes here rq/rq#112
Testing with async
turned on will still properly execute and test all logic in your code.
It differs slightly in that it completely bypasses the Worker
class - which means that you may not find bugs in RQ's Worker
and django-rq's get_worker
during testing, if there's any.
from django-rq.
I'm closing this issue. Use Queue(async=False)
to run tests.
from django-rq.
Related Issues (20)
- django.core.exceptions.ImproperlyConfigured: Requested setting RQ_SHOW_ADMIN_LINK, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings. HOT 2
- Worker pool implementation
- job_timeout is not killing subprocess
- Separate RQScheduler into its own fake model
- RQ Job Terminated Unexpectedly HOT 2
- Allow Specifying Default Serializer for Django-RQ Queue HOT 2
- How to run django-rq worker via Webhook/API call HOT 1
- Django dumpdata will fail because of unmanaged model HOT 1
- KeyError accessing stats page HOT 2
- Error in job_detail.html at line 226 HOT 2
- Sentinel support broken since 2.9.0 HOT 5
- Add support for floating point intervals in rescheduler command HOT 1
- ValueError: Invalid attribute name/AttributeError: module has no attribute
- keys of command in MULTI calls must be in same slot HOT 1
- TypeError in job_detail.html with Python 12 HOT 3
- Database Errors When Running rqworker-pool HOT 14
- RQ WorkerPool is not loading models at all
- `get_scheduler` should support a custom connection
- Tag 2.10.2 on Git HOT 2
- rqworker-pool and --with-scheduler together
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from django-rq.