Comments (16)
When you created the queue, did you enable delayed processing?
i.e.
createQueue.Options.EnableDelayedProcessing = true;
from dotnetworkqueue.
Yes
if (!(queueCreator.QueueExists))
{
Logger.Debug($"Queue {queueName} doesn't exist, creating it");
queueCreator.Options.EnableDelayedProcessing = true;
queueCreator.Options.EnableHeartBeat = true;
queueCreator.Options.EnableMessageExpiration = true;
queueCreator.Options.EnableStatus = true;
queueCreator.Options.EnableStatusTable = true;
var result = queueCreator.CreateQueue();
Logger.Debug(result.Status.ToString());
}
from dotnetworkqueue.
That should be all that's required. Which transport are you using ?
from dotnetworkqueue.
I'm using SQLite. I just realized that I probably needed to recreate the queues in the database, so I deleted it and let the program recreate it. I'm testing now.
from dotnetworkqueue.
No luck. I deleted the sqlite db file and let it be recreated from scratch, but I'm not getting the 2 minute delay I expect.
from dotnetworkqueue.
Here's the queue setup:
var container = new QueueContainer<SqLiteMessageQueueInit>();
var schedulerContainer = new SchedulerContainer();
var scheduler = schedulerContainer.CreateTaskScheduler();
taskFactory = schedulerContainer.CreateTaskFactory(scheduler);
taskFactory.Scheduler.Configuration.MaximumThreads = Config.MessageQueue.SchedulerMaxThreads;
taskFactory.Scheduler.Start();
AsyncMessageQueue<T> queue;
using (var queueCreator = queueCreationcontainer.GetQueueCreation<SqLiteMessageQueueCreation>(queueName, connectionString))
{
Logger.Debug($"Checking to see if queue {queueName} exists");
if (!(queueCreator.QueueExists))
{
Logger.Debug($"Queue {queueName} doesn't exist, creating it");
queueCreator.Options.EnableDelayedProcessing = true;
queueCreator.Options.EnableHeartBeat = true;
queueCreator.Options.EnableMessageExpiration = true;
queueCreator.Options.EnableStatus = true;
queueCreator.Options.EnableStatusTable = true;
var result = queueCreator.CreateQueue();
Logger.Debug(result.Status.ToString());
}
queue = new AsyncMessageQueue<T>
{
producer = container.CreateProducer<T>(queueName, connectionString),
consumer = container.CreateConsumerQueueScheduler(queueName, connectionString, taskFactory)
};
queue.producer.Configuration.TimeConfiguration.RefreshTime = TimeSpan.FromMinutes(Config.MessageQueue.RefreshTime);
queue.consumer.Configuration.TimeConfiguration.RefreshTime = TimeSpan.FromMinutes(Config.MessageQueue.RefreshTime);
queue.consumer.Configuration.Worker.TimeToWaitForWorkersToCancel = TimeSpan.FromSeconds(10);
queue.consumer.Configuration.Worker.WorkerCount = Config.MessageQueue.WorkerMaxThreads;
}
and AsyncMessageQueue is
public class AsyncMessageQueue<T> where T : class
{
public IProducerQueue<T> producer { get; set; }
public IConsumerQueueScheduler consumer { get; set; }
public void Start(Action<T> messageHandler)
{
this.consumer.Start<T>((m, n) => messageHandler(m.Body));
}
public void Stop()
{
producer.Dispose();
consumer.Dispose();
}
public void Send(T message, AdditionalMessageData data = null)
{
this.producer.Send(message, data);
}
public async Task SendAsync(T message, AdditionalMessageData data = null)
{
await this.producer.SendAsync(message, data);
}
}
then it would be used like:
queue.Start(async (someObject) => someObjectHandler(someObject));
var data = new AdditionalMessageData();
data.SetDelay(TimeSpan.FromSeconds(120));
queue.SendAsync(someObject, data);
async Task someObjectHandler(SomeObject someObject)
{
doStuff(someObject);
}
from dotnetworkqueue.
To rule out the field not existing, if you open up your db3 file, does the MetaData table contain the column 'QueueProcessTime'?
The samples project didn't contain a sample for delayed processing. I modified it so that it now does. I'm seeing the behavior I expect from Sqlite when using delays of 10 or 30 seconds.
That being said, I don't see anything wrong with your code above. Does it process the record pretty much instantly?
from dotnetworkqueue.
Yes, the field exists in the database, and yes, the record processes within 1-2 seconds of hitting the queue.
from dotnetworkqueue.
For 'data.SetDelay(x)', your using the SQLite extension method, and not the in-memory queue extension method? They set the data in different locations, so that would cause SetDelay to do nothing, if it happens to be the wrong one.
from dotnetworkqueue.
That was it!
I had
using DotNetWorkQueue.Transport.Memory;
instead of
using DotNetWorkQueue.Transport.SQLite.Shared;
from dotnetworkqueue.
Well, that's way too easy to accidentally do.
I'll consider naming these extensions to be transport specific, though it would be a minor breaking change. Or at least re-naming the in-memory one, as it's always present.
from dotnetworkqueue.
While I have your attention, how difficult would it be to add SQL CE 4 as a transport?
from dotnetworkqueue.
Its similar enough to SQLite that it would not be a ton of effort - I'm curious though, what does it give you that SQLite does not?
from dotnetworkqueue.
Sorry it's taken so long to get back. Everything I've found is that SQL CE 4.0 is more performant than SqLite, and I'm wondering if SQL CE would handle multiple threads better.
from dotnetworkqueue.
Ah i see. It might; I'm not sure either.
My plan - though I have no idea when I will get to it, was to add a transport for LiteDB. The idea being, that would give the queue a local data store that no longer depends on unmanaged code. Plus, the LiteDB benchmarks indicate that it does outperform SqLite generally speaking.
from dotnetworkqueue.
I'd like to contribute to that effort, if I can.
from dotnetworkqueue.
Related Issues (20)
- Question: Transactions HOT 3
- Unhanded system exception Newtonsoft.Json.JsonSerializationException: Error resolving type specified in JSON 'DotNetWorkQueue.ValueTypeWrapper`1[[System.DateTime, System.Private.CoreLib]] HOT 3
- Redis Transport - NOSCRIPT Error when Redis Server is restarted HOT 5
- How to clear the queue? HOT 2
- Creates queue tables in another Schema HOT 6
- Simple Producer and Consumer with Redis HOT 2
- [SqlLite, .NET5]: RuntimeBinderException: 'System.MulticastDelegate' does not contain a definition for 'Invoke' HOT 3
- No way to create Named Logs anymore?? HOT 13
- Is there a way to retrieve queue Count? HOT 2
- OpenTelemetry 1.1.0 dependency issues with NET6 HOT 1
- I've had a couple of questions. Please answer, thank you. HOT 1
- Performance of enqueue and dequeue with sqlite HOT 1
- How to create tables beforehand? HOT 5
- Redis - Command timeout on MoveDelayed HOT 3
- Issue with redis on netcore HOT 10
- Issue when consumer run HOT 1
- Access to the Error Queue HOT 3
- A few questions HOT 7
- Messages not re-queuing after expired Heartbeat HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dotnetworkqueue.