Comments (16)
Thanks for your fast answer! Curious to see how this will develop.
from gush.
One of the good examples had an idea to allow providing two paths, like so:
run SomeJob,
before_success: MainJob,
before_failure: AlternativeJob,
Though the naming is rather unfortunate because it suggest the MainJob will ran before SomeJob is succeesful, so needs better naming π
from gush.
@ace-subido Marking the whole workflow as failed may be too harsh; @pokonski Indeed, it's not really clear from the DSL π
from gush.
But coming back to the problem from the original issue by @linkyndy, one way would be to introduce a skipped
state in jobs. Similar to how Gitlab CI works. I would be fine with accepting such resolution
from gush.
Hey @linkyndy! Currently it's not possible but there was a suggestion about branching workflows in run time depending on their result, but I haven't had much time tinker with a solution for it.
Right now you could spawn a new Workflow inside A if conditions are met, but it's not a nice solution IMO
from gush.
EDIT: I just re-read OP's shortcircuit scneario, my original comment is irrelevant and doesn't help at all π
from gush.
As an example, let's say we process a new user signup with the above workflow. Job A retrieves the newest user from the database. If the user has been already processed, we'd like to call the workflow complete; the subsequent steps are not necessary anymore, since the user has been already processed.
Hi @linkyndy I've just reimplemented something along those lines, this is how I went about it:
# example_workflow.rb
class ExampleWorkflow < Gush::Workflow
def configure(user_id)
run(FirstJob, params: { user_id: user_id })
run(SecondJob, {
params: { user_id: user_id },
after: FirstJob,
})
end
end
# first_job.rb
class FirstJob < Gush::Job
def perform
user = User.find params[:user_id]
if user.processed?
self.fail!
return
end
# continue code ...
end
end
However this marks the flow.status
as :failed
. But it doesn't run the SecondJob
once I call self.fail
(I found this after snooping around the source).
@pokonski is this how .fail!
could be used? I could put this in the README if you like, would be glad to contribute a PR! π
from gush.
I don't think this can be used, because fail!
will stop the whole workflow π€
from gush.
Yep! That's what I was aiming to achieve πthe subsequent actions would be skipped, I think based on @linkyndy said:
If the user has been already processed, we'd like to call the workflow complete; the subsequent steps are not necessary anymore, since the user has been already processed.
I figured, that's what he/she is also aiming for.
there was a suggestion about branching workflows in run time depending on their result, but I haven't had much time tinker with a solution for it.
What would this look like? Could I take a stab at this, just pseudo-code and how the usage would look like? I'll think up of something later today, some sort of proposal for this.
from gush.
@pokonski so something like this:
class SkippedJob < Gush::Job
def perform
# marks the job as 'skipped', this would also do a 'return'
self.skip!
end
end
We could also do something like skip_remaining!
which skips all of the other jobs too, marking the entire workflow as :skipped
.
class ExampleWorkflow < Gush::Workflow
def configure(user_id)
run SkipRemainingJob
run SecondJob, after: SkipRemainingJob
end
end
class SkipRemainingJob < Gush::Job
def perform
# marks the job as 'skipped' and all other jobs after, marks the workflow itself as 'skipped' too
self.skip_remaining!
end
end
@pokonski What do you think?
from gush.
Yeah this sounds good! One clarification here:
which skips all of the other jobs too
I assume you mean skip all the jobs that were supposed to run after the job skip_remaining!
is called from, right?
from gush.
Yeah, probably cascade throughout job.outgoing
in Gush::Worker#enqueue_outgoing_jobs
and all it's descendants.
from gush.
I don't really see any value in this job.skip!
. If I want to "skip" the current job and go to the next one, I can simply return
.
My initial question was related to cancelling the entire workflow from that moment on, and dealing somehow with the remaining jobs.
from gush.
If that's the case, I'll add more to the PR. Something like: skip_remaining!
, which would tag everything from that point as skipped
from gush.
This particular feature looks very useful for an application we are developing. I would rather be very explicit and name the methods as skip_workflow!
for skipping the whole workflow and skip_descendants!
for skipping the jobs which should run after the current job.
from gush.
Related Issues (20)
- Allow redis 5 HOT 1
- Update Redis gem HOT 1
- Make gush handle wokflows with 1000s of jobs HOT 13
- Many mutex fails after #49000a30fd34ac21cabdc7d HOT 9
- Activejob callbacks not available in the Gush::Job
- Long-running jobs and human job completion HOT 11
- Jobs that eventually succeed will not queue downstream jobs HOT 6
- Gush.configure with namespace issue HOT 3
- Can we run same workflow parallelly with different arguments? HOT 1
- bundle exec viz workflow with parameters HOT 6
- Cannot assign requested address - connect(2) for [::1]:6379 HOT 12
- Overlapping workflows HOT 1
- Update Gem on RubyGems HOT 3
- Project status HOT 8
- Can I create a dynamic workflow based on job's output? HOT 1
- Project Idea - Web UI HOT 1
- cannot create workflow instance HOT 2
- Dead Link to "Parallelizing Operations With Dependencies" by Stephen Toub
- Polling scenario in job HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gush.