Giter Club home page Giter Club logo

allianceauth-celeryanalytics's Introduction

celeryanalytics

celery task and worker analytics for Alliance Auth.

Installation

With your venv active,

  1. Pip install

    pip install -U allianceauth-celeryanalytics

  2. Add celeryanalytics to your INSTALLED_APPS in your local.py

  3. From terminal run migrations

    python manage.py migrate celeryanalytics python manage.py collectstatic

  4. Optional if you wish to havve the module cealup old tasks its self run

    python manage.py ca_setup_task

Usage

This module has no permissions. it will start logging all completed and failed tasks on install using the celery signals. To view the UI you need to be superuser, and select Task Queues from the side menu

Task Menu

Toggle sections of the UI on/off here Imgur

Workers

Shows basic info on alll running workers

Imgur

NOTE if you only have a single worker shown, you are probably missing the -n %(program_name)s_%(process_num)02d parameter in your the supervisor config commands.

Active Tasks

Shows tasks that are running in the what workers Imgur

Future Tasks

Shows tasks that are held by workers with a future ETA. These may have been retries with a cool down or tasks scheduled to run in the future.

Imgur

Queue Backlog

Shows tasks split by Queue and Priority that are still pending in the queue

Imgur

Specifics on failed/completed tasks

View the successful/failed tasks in admin of your auth. as below;

Admin Menu

Failures

Details

Cleanup

If you wish to perform a tidy-up of the database you can run the following command from your terminal

python manage.py ca_run_housekeeping

Settings

CA_HOUSEKEEPING_DB_BACKLOG defines how long (in days) records should be kept in your database. Default is 14 days.

CA_RESULT_MAX_LEN if you are using a results fed app you may wish to limit the result spam to database. in your local.py add the setting CA_RESULT_MAX_LEN=1000 set the integer to what ever you want as your max length. Default is -1 or unlimited.

CA_LOG_SUCCESS_TO_DB If you don't want the module to log Successful Tasks to database, set this to False. Default is True

CA_LOG_FAILURE_TO_DB If you don't want the module to log Failed Tasks to database, set this to False. Default is True

Issues

Please remember to report any celeryanalytics related issues using the issues on this repository.

allianceauth-celeryanalytics's People

Contributors

ppfeufer avatar pvyparts avatar soratidus999 avatar transifex-integration[bot] avatar

Stargazers

avatar

Watchers

avatar avatar avatar

Forkers

milleruk staropera ppfeufer erikkalkoken soratidus999 goosefleeteo

allianceauth-celeryanalytics's Issues

Excessive CPU and memory usage

Hello there,

So in alliance auth 3.7.1 and latest version accessing Celery Stats in the UI triggers excessive CPU and RAM usage. My box currently has 64G of ram.

aa-docker-redis-1 | 33:C 28 Oct 2023 19:42:34.914 * RDB: 6900 MB of memory used by copy-on-write aa-docker-redis-1 | 1:M 28 Oct 2023 19:42:36.827 * Background saving terminated with success aa-docker-redis-1 | 1:M 28 Oct 2023 19:43:37.015 * 10000 changes in 60 seconds. Saving... aa-docker-redis-1 | 1:M 28 Oct 2023 19:43:37.633 * Background saving started by pid 34

After a reboot it looks much better
Done loading RDB, keys loaded: 62240, keys expired: 8716.

Missing Requirement for Install

When installing CeleryAnalytics you need to manually pip install django-ninja as it is not defined in the requirements.

Running initial migration for CeleryAnalytics without manually pip install django-ninja results in the following error:

Traceback (most recent call last):
  File "/home/allianceserver/myauth/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/management/base.py", line 98, in wrapped
    res = handle_func(*args, **kwargs)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 91, in handle
    self.check(databases=[database])
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/management/base.py", line 487, in check
    all_issues = checks.run_checks(
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/checks/registry.py", line 88, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/checks/urls.py", line 42, in check_url_namespaces_unique
    all_namespaces = _load_all_namespaces(resolver)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/core/checks/urls.py", line 61, in _load_all_namespaces
    url_patterns = getattr(resolver, "url_patterns", [])
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/utils/functional.py", line 49, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/urls/resolvers.py", line 696, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/utils/functional.py", line 49, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/django/urls/resolvers.py", line 689, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/allianceserver/myauth/myauth/urls.py", line 2, in <module>
    from allianceauth import urls
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/allianceauth/urls.py", line 12, in <module>
    import allianceauth.services.urls
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/allianceauth/services/urls.py", line 17, in <module>
    services = get_hooks('services_hook')
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/allianceauth/hooks.py", line 123, in get_hooks
    register_all_hooks()
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/allianceauth/hooks.py", line 112, in register_all_hooks
    hooks = list(get_app_submodules('auth_hooks'))
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/allianceauth/hooks.py", line 101, in get_app_submodules
    yield name, import_module(f'{name}.{module_name}')
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/celeryanalytics/auth_hooks.py", line 1, in <module>
    from . import urls
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/celeryanalytics/urls.py", line 5, in <module>
    from .api import api
  File "/home/allianceserver/venv/auth/lib/python3.10/site-packages/celeryanalytics/api.py", line 6, in <module>
    from ninja import Field, Form, NinjaAPI
ModuleNotFoundError: No module named 'ninja'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.