Giter Club home page Giter Club logo

django-tasks-scheduler's Introduction

django-tasks-scheduler's People

Contributors

cunla avatar cyber237 avatar dependabot[bot] avatar dirkmueller avatar gabriels1234 avatar mbi avatar rstalbow avatar rwielk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

django-tasks-scheduler's Issues

django-tasks-scheduler should be database agnostic

Describe the bug
pyproject.toml lists psycopg2 = "^2.9.9" as a dependency.

Is this really the case? Does the app not work with other database backends supported by Django?

Also we tend to prefer installing psycopg2-binary as it has fewer system dependencies. As it is we end up installing both psycopg2 and psycopg2-binary ๐Ÿคท

...
psycopg2==2.9.9
    # via django-tasks-scheduler
psycopg2-binary==2.9.9
    # via -r requirements.in
...

To Reproduce
Steps to reproduce the behavior:

  1. install django-tasks-scheduler via any tool that handles dependencies

Expected behavior
Ideally django-tasks-scheduler should be database agnostic

Jobs get scheduled for immediate execution after previous execution has finished

The change in this caused a bigger issue.
We have our scheduler interval configured like this:

SCHEDULER_CONFIG = {
"SCHEDULER_INTERVAL": 1 # In seconds
}
The jobs now run with an interval of 1 second regardless of whether they are enabled or not and ignores the interval defined in the job.

Screenshot from 2023-10-09 18-04-29

Screenshot above shows the job is scheduled to run every hour.

Screenshot from 2023-10-09 18-04-53

Screenshot above shows the job is running every second

I believe the issues is in the callback_save_job where you have added force_schedule. Inside of force_schedule its runs self._schedule_time(). This is the datetime of the very first time it was set to run. It then does an enqueue_at at and as the schedule_time is now in the past it runs it. callback_save_job is run everytime the job is run.

I can try and write a test to replicate this tomorrow

Queue is cleared after first job finishes

Describe the bug
It was hard to track down, but basically, I have a Cron Job that schedules 26 jobs. The first job finishes (as per the worker's logs) and then it stops working. and upon refreshing the RQ Queues, the queue goes from 25 queued jobs to 0.
This also happen when enqueuing via code

To Reproduce
Steps to reproduce the behavior:
Add some code that does:

queue = get_queue()
for i in range(20):
  queue.enqueue(some_func, job_id=f"job_{i}", args=(a,b))

Expected behavior
When queuing more than one job, all the jobs must be performed.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • python 3.11
  • django 4.2.6
  • requirements.txt? django-task-scheduler==1.2.3

Additional context
My project relies on being able to queue jobs programatically (usually more than one at a time). I'm still investigating the root cause, but I've tried using the DjangoQueue and the normal Queue and the results are the same.

Job killed mid running not gracefully causes infinite loop

Describe the bug
A worker is killed mid running a job and the shutdown is not graceful. After this happens you get a 500 error accessing admin/scheduler/queue/ and one of admin/scheduler/{repeatablejob/cronjob/scheduledjob}/ depending on what type of job was killed when running. The exceptions are

RecursionError
maximum recursion depth exceeded while calling a Python object

and
Job repeatable_jobs:{jobname}:{jobid}: error while executing failure callback

This error is due to this change. The self.rqueue.started_job_registry.get_job_ids() calls cleanup which calls is_scheduled thus an infinite loop.

To Reproduce
Steps to reproduce the behavior:

  1. First you need a long running job. For example:
@job()
def test():
    import time
    time.sleep(30)
  1. Setup the job to run via the admin.
  2. When the job is active (by checking the queue admin/scheduler/queue/) kill the worker process (for example by pkill -9 python)
  3. The either:
    • Bring the worker back up and wait for the maintenance tasks to run
    • Run the clean_registries function from a python django shell

Expected behavior
The job gets - Moved to FailedJobRegistry, due to AbandonedJobError. This happens to the job if you revert back to version 1.0.2 but then you have this issue again.

Implementation idea

  1. In the is_scheduled function when running self.rqueue.started_job_registry.get_job_ids() have an option to not run the cleanup inside of get_job_ids. You shouldn't need to run the cleanup from the is_scheduled method.
  2. Revist the fix for [issue](#32 and instead of adding in self.rqueue.started_job_registry.get_job_ids() remove the
self.job_id = None
super(BaseJob, self).save()

v1.2.4: Failed CRON jobs aren't requeued

Describe the bug
When a job scheduled as a CRON Job fails, by raising an error, it isn't re queued. Indications of it not being re-queued are on the django-admin UI and include a - appearing as the next scheduled time, and no entry for next scheduled jobs when clicking into each cron job.

To Reproduce
Steps to reproduce the behavior:

  1. Create a task in any app module that will raise an error. Set up a cron string of every minute.
  2. Create a job that calls that task.
  3. Run the worker and the scheduler.
  4. S

Expected behavior
The job ideally should execute, and fail, and the next scheduled job should not be prevented from being created and/OR running. In current version, the job fails, and none fails.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: Linux Mint/Ubuntu
  • python.311
  • django 4.1

cron job infnite call

when I add a cron task with Cron string 2 */1 * * *
it runs infinitely when the job is finished it executes infinitely without stop
and even in repeated tasks it still executes infinitely

this is my settings

SCHEDULER_CONFIG = {
'EXECUTIONS_IN_PAGE': 20,
'DEFAULT_RESULT_TTL': 500,
'DEFAULT_TIMEOUT': 300, # 5 minutes
'SCHEDULER_INTERVAL': 10, # 10 seconds
}

SCHEDULER_QUEUES = {
'default': {
'URL': 'redis://localhost:6379/0',
'DEFAULT_TIMEOUT': 360,
},
}

can you help me with this problem?
and thank you

Adding Cron Jobs via Code

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
I'd like to control my cron jobs via Code.

Describe the solution you'd like
I'd be great to be able to add Cron Jobs via code, similar to enqueing jobs

Describe alternatives you've considered
n/a

Additional context
If this already exists, I'd like the request to be to update the Doc with an example.

Missing migrations

Describe the bug
There are missing migrations

To Reproduce
Steps to reproduce the behavior:

  1. Checkout project
  2. poetry install
  3. Goto testproject
  4. Run poetry run manage.py makemigrations
  5. Missing migrations get produced

Expected behavior
No migrations are getting produced

Additional context
Will append PR

Let me know if I shall change e.g. when and how makemigration check will get called, make an extra stage, just call it in one tests or whatever.
Thanks for sharing django-tasks-scheduler.

Kind regards
Raiko

Job becomes unscheduled if you access the admin list page when the job is active

Describe the bug
In this method - https://github.com/dsoftwareinc/django-tasks-scheduler/blob/master/scheduler/models/scheduled_job.py#L83
It checks for jobs in scheduled or queued. If it doesn't find the job it sets the job_id to None which means after its finished its current execution its doesn't reschedule.

To Reproduce
Steps to reproduce the behavior:

  1. Go to the admin page admin/scheduler/{repeatablejob/cronjob/scheduledjob}/ and see the green tick against the job to show its scheduled.
  2. Take a look at the queue admin/scheduler/queue/ and wait for the job to move to active (best to test with a job that takes a few seconds to run).
  3. Go to the admin page admin/scheduler/{repeatablejob/cronjob/scheduledjob}/ (or refresh page)
  4. See the red tick against the job to show its no longer scheduled.

Expected behavior
The job stays scheduled.

Implementation idea
Idea 1:
Add an additional check for the job id in self.rqueue.started_job_registry.get_job_ids()
Idea 2:
Remove the

self.job_id = None
self.save()

As I dont see why you would save the database record in this method

When a job fails it becomes unscheduled

Describe the bug
When a job raises an exception it becomes unscheduled.

To Reproduce
Steps to reproduce the behavior:

  1. First you need a job that raises an exception
@job()
def test():
    raise Exception
  1. Setup the job to run via the admin.
  2. After the job has run see that is never runs again (until you set it to run again via the admin or code) and visualise the is scheduled is false.

Cause

In the fix for this there is a new function failure_callback and success_callback. The failure_callback function doesn't reschedule the job. In the previous iterations where it was a function called callback_save_job it did:

    if scheduled_job is not None:
        scheduled_job.unschedule()
        scheduled_job.schedule()

In the success_callback it still reschedules the job. By not rescheduling this is a fundamental breaking change in the package that I dont think it right. I will create an MR to fix

Auto-reload workers page issue with template

Describe the bug
when running locally, and making changes to the code I sometimes encounter an error screen when accessing the Workers Page:

image

The part of the code that might need some error handling:

image

Desktop (please complete the following information):

  • OS: [e.g. iOS] Mac OS.
  • python version: 3.11
  • django version: 4.2.9
  • requirements.txt?....

Thanks!!!

TypeError: EnumType.__call__() missing 1 required positional argument: 'value'

Describe the bug
Getting a 500 error when trying to access a particular job (either in the active or in the failed column)

To Reproduce
Hard to reproduce, I'm still investigating, but probably an unhandled Exception.

Expected behavior
If the error is not critical, fail gracefully, omitting the values that have an issue.

Screenshots
N/A

Desktop (please complete the following information):

  • alpine, python 3.12
    django-tasks-scheduler==1.3.1
    django==5.0.3

Additional context
(Still under investigation)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.