Giter Club home page Giter Club logo

Comments (5)

cdrage avatar cdrage commented on July 24, 2024

Doesn't Resque do this though?

Or do a rescue in Go?

from goworker.

rohit4813 avatar rohit4813 commented on July 24, 2024

Hi,
I am experimenting with the goworker library.
I have a requirement of stopping and starting jobs.

Is it possible with the current version? Can anyone tell any workaround for it?

from goworker.

mingan avatar mingan commented on July 24, 2024

@rohit4813 The current implementation listens for few signals and if it receives them, it stops enqueuing new jobs but lets the running jobs finish.

I'm not sure I understand exactly what you're trying to do, but here's our use: We have a scenario where each job is potentially pretty long but has a natural stopping point. For this, we create a channel in the main function that gets written into when a signal is received (basically he same code that is in goworker already). Then we create another channel, this time buffered (capacity = number of workers) and pass that channel to each worker. Workers then select from that channel at natural stopping points. In a separate goroutine (kicked off from the main function), we read from the signals channel and write N times (= number of workers) to the channel.

The whole flow looks like:

  1. The process receives a signal
  2. Both our signals channel and goworker's signals channels are written into
  3. Goworker stops enqueuing new jobs
  4. We copy the event N times
  5. When any worker finishes, it's handled normally
  6. When a worker gets to a checkpoint where it checks the channel, it returns and goworker takes care of it

from goworker.

rohit4813 avatar rohit4813 commented on July 24, 2024

@mingan Thanks for the great explanation.

If I understand correctly, all the workers will read from the workers channel(which gets populated from the signals channel)?

And I have a use case where I want to stop single/multiple worker(s), say which are running for a very long time and if that is the case all the workers will stop on passing the signal to the channel.

I can identify the worker on which the job is running for a very long time. How can I send the signal to this particular worker?

Hope this is not confusing, or am I missing something.

from goworker.

mingan avatar mingan commented on July 24, 2024

@rohit4813 Our use case just creates breakpoints in long-running jobs so that when we need to restart the process, we don't have to wait (tens of) minutes for the whole job to finish.

If you needed to discriminate between workers, I guess you could do that by sending some meaningful value through the channel and then the worker would decide "this msg is meant for me, I'll stop" or "this is meant for the slow one over there, I can keep running". Though, I can't imagine the use case for such behaviour.

from goworker.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.