Comments (5)
The safest approach would be to start a number of child processes from your root process. You can do this with the standard Process Class. This would also be the least likely to encounter any strange errors.
The simplest approach, and one I recommend testing out before going all out is just to wrap a bunch of TaskClient.Listen()
methods in a Task.Run() and awaiting all of them. Something like this:
await Task.WaitAll(new [] {
Task.Run(() => {
var taskClient = new TaskClient(TaskQueue.Redis(redisConnectionString));
await taskClient.Listen();
}),
Task.Run(() => {
var taskClient = new TaskClient(TaskQueue.Redis(redisConnectionString));
await taskClient.Listen();
})
});
Make sure with the second approach you don't share a TaskClient between threads.
Also it's similar to what we do in the tests, except tasks are executed off the queue directly in multiple threads:
https://github.com/brthor/Gofer.NET/blob/master/Gofer.NET.Tests/GivenATaskSchedulerInAnotherThread.cs#L51
from gofer.net.
I was playing with the code, and went in a completely different direction. I added an override to Listen so you can define how many threads to start. It partially works... for some reason sometimes all the threads run, other times it acts like there is a lock somewhere?
Not a real PR, just a convenient place to see a DIFF: https://github.com/brthor/Gofer.NET/pull/51/files
In testing your example, it looks like it's really the TaskQueue that is the thread block somehow?
If I use your example, but use a single TaskQueue for all TaskClient instances, then it won't run more than one job at a time. I have to give each Task Client it's own TaskQueue.
var tasks = new List<Task>();
for(int i = 0; i < 5; i++)
{
tasks.Add(Task.Run(async () =>
{
var taskClient = new TaskClient(TaskQueue.Redis("10.1.1.1:6379"));
await taskClient.Listen();
}));
}
Task.WaitAll(tasks.ToArray());
from gofer.net.
The redis connection held by the TaskQueue will likely be a bottleneck if it's shared.
from gofer.net.
Regarding your PR #51, having a user-defined number of task listeners in the TaskClient is interesting since it avoids the need to run multiple scheduler threads.
IIRC there are some concerns with the default number of threads available to the standard Task Executor.
I'm going to think on it some more.
from gofer.net.
Yeah, the PR is just some test code. Take your time to think about the right approach :)
For now, your approach is working.
from gofer.net.
Related Issues (20)
- Add Support for overloaded Methods HOT 1
- Improve Documentation
- Re-enable At-Least-Once Processing
- Persist Scheduled Tasks to the Backing Store HOT 2
- Exceptions cannot be passed as as arguments.
- Task execution with argument of type System.Type throws exception HOT 4
- Roadmap to 1.0 Stable Release HOT 1
- Test Gofer.NET with Redis Cluster HOT 1
- TaskClient should run TaskScheduler in its own thread.
- Ensure ScheduledTasks are queued in order of timestamp.
- Add Windows Build
- [FeatureRequest] Add overload with cancellation token to the Listen() method HOT 8
- Can't use connection strings with passwords HOT 3
- Priority tasks
- Simple way to get Queue Depth? HOT 5
- Add a configurable delay if there are no jobs in the queue HOT 2
- Recommended strategy to split producer code and worker code ? HOT 4
- Add TaskQueue.Enqueue(Expression<Func<T>>) to suppress warning CS4014 HOT 2
- Dashboard for monitoring
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gofer.net.