ambitioninc / django-db-mutex Goto Github PK
View Code? Open in Web Editor NEWAcquire a mutex via the DB in Django
License: MIT License
Acquire a mutex via the DB in Django
License: MIT License
https://github.com/ambitioninc/django-db-mutex/blob/develop/db_mutex/db_mutex.py#L83
Here the code deletes locks based on a datetime.now() calculation. I believe this means that if all servers aren't properly NTP-time-synced, the locks could be accidentally deleted sooner than expected on servers with time set in the future.
This code should be doing lock deletion based on the SQL server knowledge of time.
https://github.com/ambitioninc/django-db-mutex/blob/develop/db_mutex/db_mutex.py#L112
If another process grabbed a timed-out lock and hasn't released it yet, the simple check for "if not DBMutex.objects.filter(id=self.lock.id).exists()" doesn't verify current ownership of the lock, and so it may not raise the timeout exception when it should.
It would be best if the local object and SQL object contained an owner identifier that was based on a unique machine+process+thread ID.
Then this code could double-check that the lock exists and is still owned by itself as expected.
Alternatively, it might be good-enough to just generate a random 64-bit integer to use for this purpose (would work in 99.9999% of cases).
Would be useful to have a few words about configuration somewhere in the docs.
Namely, that one should add 'db_mutex'
to INSTALLED_APPS
and then run migrations with
python manage.py migrate
.
Could you please provide updated changelog? There is a release notes section in the docs but it ends with v1.1.0 while the latest released version is 3.0.0. Thank you.
Is there a more universal solution that uses timezone.now instead of datetime.utcnow? Right now I'm getting a bunch of runtimewarnings because the datetime isn't timezone aware, but I realize that timezone.now isn't necessarily UTC. If someone has USE_TZ = False
, I'm sure there would be problems during savings time changes.
RuntimeWarning: DateTimeField DBMutex.creation_time received a naive datetime (2015-03-24 15:50:28.141220) while time zone support is active.```
Hi,
Just so you know: 1.0.0 was released on 2017-12-08 but:
Thanks ;)
$ ./manage.py migrate
SystemCheckError: System check identified some issues:
ERRORS:
db_mutex.DBMutex.lock_id: (mysql.E001) MySQL does not allow unique CharFields to have a max_length > 255.
This error is fixed by changing max_length to 255 (instead of 256) on the lock_id field in models.py
On a production environment where Django was deployed via Apache and mod_wsgi in 8 processes, we encountered a "deadlock detected" error during this statement:
Mar 14 12:43:53 ERROR django.request - 500 Internal Server Error: /region/pages/de/14907/edit/
Traceback (most recent call last):
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.DeadlockDetected: deadlock detected
DETAIL: Process 1054249 waits for ShareLock on transaction 18359673; blocked by process 1054250.
Process 1054250 waits for ShareLock on transaction 18359675; blocked by process 1054249.
HINT: See server log for query details.
CONTEXT: while inserting index tuple (0,11) in relation "db_mutex_dbmutex_lock_id_key"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
return handler(request, *args, **kwargs)
File "/usr/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/integreat_cms/cms/views/pages/page_form_view.py", line 324, in post
page_translation_form.instance.page = page_form.save()
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/integreat_cms/cms/forms/custom_model_form.py", line 133, in save
return super().save(commit=commit)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/treebeard/forms.py", line 153, in save
self.instance.move(reference_node, pos=position_type)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/integreat_cms/cms/models/pages/page.py", line 327, in move
super().move(target, pos)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/integreat_cms/cms/models/abstract_tree_node.py", line 270, in move
with db_mutex(self.__class__.__name__):
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/db_mutex/db_mutex.py", line 93, in __enter__
self.start()
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/db_mutex/db_mutex.py", line 107, in start
self.lock = DBMutex.objects.create(lock_id=self.lock_id)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/query.py", line 453, in create
obj.save(force_insert=True, using=self.db)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/base.py", line 739, in save
self.save_base(using=using, force_insert=force_insert,
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/base.py", line 776, in save_base
updated = self._save_table(
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/base.py", line 881, in _save_table
results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/base.py", line 919, in _do_insert
return manager._insert(
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/query.py", line 1270, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
cursor.execute(sql, params)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/cacheops/transaction.py", line 98, in execute
result = self._no_monkey.execute(self, sql, params)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/integreat-cms/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
django.db.utils.OperationalError: deadlock detected
In some cases, I'd want to have a lock that does not immediately throw an exception, but that waits until the resource is released.
Probably this would need another timeout setting to configure how long the thread waits for the lock (and only raises an exception when the lock is not released within that time frame).
In its most basic form, I guess this would just be a busy wait.
I can also think of scenarios where this might be interesting in an asynchronous context, but this would probably require a rewrite of substantial parts of the library, due to the normal database access in Django being synchronized?
Do you agree with this idea, or do you think it's up to the application to wait and retry the corresponding task in case an exception occurs?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.