cenkalti / dalga Goto Github PK
View Code? Open in Web Editor NEW⏰ MySQL backed Job Scheduler with a HTTP interface
Home Page: https://pkg.go.dev/github.com/cenkalti/dalga/v3
License: MIT License
⏰ MySQL backed Job Scheduler with a HTTP interface
Home Page: https://pkg.go.dev/github.com/cenkalti/dalga/v3
License: MIT License
There are some tests that should be written regarding how Dalga behaves with multiple instances. From reading the code, I have no doubts that it works well in a multi-instance setup, but there's no such thing as too many tests 😄
It might be cool to have a GUI dashboard for Dalga.
If I get around to this personally, I'd be interested to try out some tech like alpinejs, htmx, tailwindcss, and other low-overhead stuff.
@cenkalti what do you think?
Once the various open PRs land, the documentation should be updated.
Since I will be using Dalga from another Go service, I will end up writing a client package to communicate with Dalga over HTTP. Would you be interested in upstreaming it into this repo?
Hey there, was looking over Dalga and I like that it's straightforward, backed by MySQL, and supports a multi-instance deployment. However, it lacks two things for my use case:
As I said this is a very straightforward project and the source is quite readable. I more or less see what I would need to do to use an ISO8601 interval, and I can make a PR for it. I'm sure retry-customization is not much beyond that.
Before I make any PRs, I wanted to ask: are you open to contributions? Are you interested in having more features in this project?
As a comparison I did quite a bit of work over at Kala which is a similar project, but the way it is structured I don't see how to make it highly-available, which is why I am considering Dalga.
Otherwise, a job may seem stuck in running status forever if remote end does not behave well.
It is not possible to import v2 code with current module setup.
I am interested in adding tests to Dalga to verify its behavior over an extended period of time: firing jobs at the right intervals, retry behavior, etc. I am also interested in testing the interaction of Dalga with another system, including a flow of scheduling jobs, receiving notifications, pausing, resuming, retrying, rescheduling, etc.
To facilitate this, I would like to add github.com/mixer/clock to the project, and use it as the basis for Dalga's awareness of the current time. This means moving some of the time logic out of the SQL and into Go code.
In normal circumstances, the Clock library would use the normal system time, but in unit tests the time can be advanced by a day or a week at a time to simulate the passage of time and ensure that Dalga fires events at the expected points.
This is something that I added to Kala, and tried to add to robfig/cron. I understand it's not a small change but I think it's important for peace of mind when scheduling mission-critical tasks.
I realized that Dalga does not have a pause/resume functionality. Would you be open to adding this feature?
There are some specific semantics that I'm interested in for this. For example, if a job would run weekly and the next run is on the 2nd of the month, pausing on the 1st and resuming on the 3rd should skip that iteration, and the job would run again on the 9th.
This is different than a delay caused by downtime. For instance, if the next run is on the 2nd and the system went down on the 1st and came up on the 3rd, it would immediately process the job and reschedule for the 9th.
(From what I understand this is already how Dalga would handle downtime, I am just providing a counter-example to illustrate what I am looking for in a Pause/Resume functionality.)
Required for #26
I typically run systems with MariaDB instead of MySQL. This usually works out fine because the majority of syntax works across both system types.
In running the Dalga tests, I did encounter a problem with the SKIP LOCKED
clause in Front()
and found that MariaDB does not yet support this syntax.
SKIP LOCKED
are vital to you, would you be open to some sort of config or command-line flag to enable MariaDB compatibility?It must be set by CI on tagged commits.
@tooolbox I have made some changes after your PRs are merged. Do you have any objections about them? Or, do you want to make additional changes before releasing v3.0.0?
It would be good to have a way for Dalga POSTs to be idempotent, meaning they contain some sort of information that allows a receiver to know "this is the Nth iteration of this job" irrespective of retries.
This could be as simple as taking the value of next_sched
as a unix timestamp and including it as header or query param in the POST request. If the receiver keeps records of jobs, and it receives a duplicate next_sched
timestamp header, it can throw it away. The header doesn't have to be called next_sched
, it could be iteration
or something.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.