Giter Club home page Giter Club logo

Comments (16)

blehnen avatar blehnen commented on June 10, 2024

When you created the queue, did you enable delayed processing?

i.e.

createQueue.Options.EnableDelayedProcessing = true;

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

Yes

if (!(queueCreator.QueueExists))
{
	Logger.Debug($"Queue {queueName} doesn't exist, creating it");
	queueCreator.Options.EnableDelayedProcessing = true;
	queueCreator.Options.EnableHeartBeat = true;
	queueCreator.Options.EnableMessageExpiration = true;
	queueCreator.Options.EnableStatus = true;
	queueCreator.Options.EnableStatusTable = true;
	var result = queueCreator.CreateQueue();
	Logger.Debug(result.Status.ToString());
}

from dotnetworkqueue.

blehnen avatar blehnen commented on June 10, 2024

That should be all that's required. Which transport are you using ?

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

I'm using SQLite. I just realized that I probably needed to recreate the queues in the database, so I deleted it and let the program recreate it. I'm testing now.

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

No luck. I deleted the sqlite db file and let it be recreated from scratch, but I'm not getting the 2 minute delay I expect.

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

Here's the queue setup:

var container = new QueueContainer<SqLiteMessageQueueInit>();
var schedulerContainer = new SchedulerContainer();
var scheduler = schedulerContainer.CreateTaskScheduler();
taskFactory = schedulerContainer.CreateTaskFactory(scheduler);
taskFactory.Scheduler.Configuration.MaximumThreads = Config.MessageQueue.SchedulerMaxThreads;
taskFactory.Scheduler.Start();

AsyncMessageQueue<T> queue;

using (var queueCreator = queueCreationcontainer.GetQueueCreation<SqLiteMessageQueueCreation>(queueName, connectionString))
{
	Logger.Debug($"Checking to see if queue {queueName} exists");
	if (!(queueCreator.QueueExists))
	{
		Logger.Debug($"Queue {queueName} doesn't exist, creating it");
		queueCreator.Options.EnableDelayedProcessing = true;
		queueCreator.Options.EnableHeartBeat = true;
		queueCreator.Options.EnableMessageExpiration = true;
		queueCreator.Options.EnableStatus = true;
		queueCreator.Options.EnableStatusTable = true;
		var result = queueCreator.CreateQueue();
		Logger.Debug(result.Status.ToString());
	}

	queue = new AsyncMessageQueue<T>
	{
		producer = container.CreateProducer<T>(queueName, connectionString),
		consumer = container.CreateConsumerQueueScheduler(queueName, connectionString, taskFactory)
	};

	queue.producer.Configuration.TimeConfiguration.RefreshTime = TimeSpan.FromMinutes(Config.MessageQueue.RefreshTime);
	queue.consumer.Configuration.TimeConfiguration.RefreshTime = TimeSpan.FromMinutes(Config.MessageQueue.RefreshTime);
	queue.consumer.Configuration.Worker.TimeToWaitForWorkersToCancel = TimeSpan.FromSeconds(10);
	queue.consumer.Configuration.Worker.WorkerCount = Config.MessageQueue.WorkerMaxThreads;
}

and AsyncMessageQueue is

public class AsyncMessageQueue<T> where T : class
{
	public IProducerQueue<T> producer { get; set; }
	public IConsumerQueueScheduler consumer { get; set; }

	public void Start(Action<T> messageHandler)
	{
		this.consumer.Start<T>((m, n) => messageHandler(m.Body));
	}

	public void Stop()
	{
		producer.Dispose();
		consumer.Dispose();
	}

	public void Send(T message, AdditionalMessageData data = null)
	{
		this.producer.Send(message, data);
	}

	public async Task SendAsync(T message, AdditionalMessageData data = null)
	{
		await this.producer.SendAsync(message, data);
	}
}

then it would be used like:

queue.Start(async (someObject) => someObjectHandler(someObject));

var data = new AdditionalMessageData();
data.SetDelay(TimeSpan.FromSeconds(120));
queue.SendAsync(someObject, data);

async Task someObjectHandler(SomeObject someObject)
{
    doStuff(someObject);
}

from dotnetworkqueue.

blehnen avatar blehnen commented on June 10, 2024

To rule out the field not existing, if you open up your db3 file, does the MetaData table contain the column 'QueueProcessTime'?

The samples project didn't contain a sample for delayed processing. I modified it so that it now does. I'm seeing the behavior I expect from Sqlite when using delays of 10 or 30 seconds.

That being said, I don't see anything wrong with your code above. Does it process the record pretty much instantly?

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

Yes, the field exists in the database, and yes, the record processes within 1-2 seconds of hitting the queue.

from dotnetworkqueue.

blehnen avatar blehnen commented on June 10, 2024

For 'data.SetDelay(x)', your using the SQLite extension method, and not the in-memory queue extension method? They set the data in different locations, so that would cause SetDelay to do nothing, if it happens to be the wrong one.

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

That was it!

I had
using DotNetWorkQueue.Transport.Memory;

instead of
using DotNetWorkQueue.Transport.SQLite.Shared;

from dotnetworkqueue.

blehnen avatar blehnen commented on June 10, 2024

Well, that's way too easy to accidentally do.

I'll consider naming these extensions to be transport specific, though it would be a minor breaking change. Or at least re-naming the in-memory one, as it's always present.

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

While I have your attention, how difficult would it be to add SQL CE 4 as a transport?

from dotnetworkqueue.

blehnen avatar blehnen commented on June 10, 2024

Its similar enough to SQLite that it would not be a ton of effort - I'm curious though, what does it give you that SQLite does not?

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

Sorry it's taken so long to get back. Everything I've found is that SQL CE 4.0 is more performant than SqLite, and I'm wondering if SQL CE would handle multiple threads better.

from dotnetworkqueue.

blehnen avatar blehnen commented on June 10, 2024

Ah i see. It might; I'm not sure either.

My plan - though I have no idea when I will get to it, was to add a transport for LiteDB. The idea being, that would give the queue a local data store that no longer depends on unmanaged code. Plus, the LiteDB benchmarks indicate that it does outperform SqLite generally speaking.

from dotnetworkqueue.

tmcque9 avatar tmcque9 commented on June 10, 2024

I'd like to contribute to that effort, if I can.

from dotnetworkqueue.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.