Comments (2)
Hey @polds!
A couple of comments to explain the behaviour you're seeing:
- The pool created in your sample is a "blocking pool" because
maxCapacity
is set to0
(2nd. argument). This means the buffered channel used to queue tasks submitted to the pool has size 0, so every time a client goroutine does aSubmit()
on the pool, it will block until a worker grabs it from the queue (receives from that channel). - Given that the pool is blocking, when the main goroutine tries to send the 6th task, it'll wait until one of the workers becomes available again. When
p.StopAndWait()
is called from the timeout function, the task queue channel is closed, so the when the main goroutine attempts to continue submitting tasks it panics because of this (panic: send on closed channel
). Notice this panic is thrown in the caller goroutine, not inside one of the tasks. This is why the panic handler is not called (it's only meant to catch panics inside task functions). - The
StopAndWait()
method is intended for a graceful termination of the pool (i.e. stop accepting tasks but finish all running and queued tasks), but looks like you need it to stop immediatelly and discard any queued/pending tasks. If that's the case, you can useStop()
instead, which has that behaviour.
Here's a modified version of your sample that might behave similar to what you expect:
package main
import (
"fmt"
"time"
"github.com/alitto/pond"
)
func h(p interface{}) {
fmt.Printf("Catching panic: %v\n", p)
}
func main() {
// Create a non-blocking channel (maxCapacity = 100)
p := pond.New(5, 100, pond.PanicHandler(h))
quit := make(chan struct{})
// Simulate a timeout event of some sort.
time.AfterFunc(time.Second, func() {
// Stop the pool immediately and discard queued tasks
p.Stop()
quit <- struct{}{}
})
for i := 0; i < 100; i++ {
i := i
// Attempt to submit the task or silently discard it if the pool is stopped
p.TrySubmit(func() {
fmt.Printf("Handling task %d\n", i)
time.Sleep(250 * time.Millisecond)
})
}
<-quit
}
Alternatively, if you need to set a max waiting time for each task to be picked up by a worker you could use SubmitBefore()
instead of Submit()
, which only executes the task if it was grabbed within the specified time and skips it otherwise.
Thanks for reaching out! Please let me know if that works for you.
from pond.
Thanks for the response! I'm looking for a way to expire the entire queue as a whole after a timeout, and not necessarily individual tasks.
Also, thanks for your example code. It unfortunately only ever processes maxWorkers
number of items and blocks until the timeout kicks in regardless of the maxCapacity
value set. It also has the issue if it completes before the timeout it'd wait until the timeout anyways.
What I'm ultimately looking for is a way to throw 1,000s of tasks at a queue, batch them in groups of 50 or whatever and drop whatever doesn't complete within the timeout (to be rescheduled at a later date).
We can close this issue, though. Since my original point was about the panic handler not doing what I thought it should be.
from pond.
Related Issues (20)
- Why use atomic.AddInt32() and sync.Mutex at the same time HOT 2
- Access channel concurrently HOT 3
- Exceptions may occur when closing the pool HOT 7
- does pond support serial queue? HOT 3
- Question: What happens to the pool when a worker panic HOT 3
- goroutine running on other thread; stack unavailable HOT 2
- error handling HOT 4
- Deadlock on zero minimum workers HOT 3
- Unaligned 64-bit atomic operation error when built for linux HOT 2
- pond.New(30, 100, pond.MinWorkers(30)) creates 60 goroutines not 30 HOT 2
- Crash on arm 32-bits (RPi4)
- panic: send on closed channel, when lots of goroutine are running(may be for a long time) HOT 3
- IdleWorkers RunningWorkers HOT 3
- metric problem HOT 3
- run stop panic
- support task priority
- 1 pool for 3 different tasks with the same number of workers on each task HOT 1
- Will requesting to submit an asynchronous task cause blocking? HOT 1
- What is the best way to handle errors? HOT 1
- Setting unlimited maxCapacity, int vs uint HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pond.