glucaci / mongodb-quartz-net Goto Github PK

Hi! I have a problem with using other time zones in my trigger. The timezone doesnt change. I´m using it like this .WithSchedule(cronScheduleBuilder.InTimeZone(TimeZoneInfo.Local)). But I always have one hour less to next fire time than it should.

Hi!

Is it possible to use this together with the Quartz.Extensions.DependencyInjection?

I was thinking of something like this:

                q.UsePersistentStore(ps =>
                {
                    ps.UseMongoDb(connectionString: MongoDbConfig.ConnectionString, prefix: "quartz");
                });

How would the extensionmethod UseMongoDb look like?

Hi, i don't know if this project is still alive.
I have made some tests and i have a problem with misfired jobs.
During startup, misfired jobs are deleted instead of being put in waiting state.
I think there is a mistake in MongoDbStore.DoUpdateOfMisfiredTrigger method.
This condition must be inverted if (operableTrigger.GetNextFireTimeUtc().HasValue), if not trigger has a NextFireTime but will be set as Completed and cleaned up during the startup process.
I've watcher ado.net store implementation here https://github.com/quartznet/quartznet/blob/b33e6f8630a8dddfb0c1b6117237d538a5931dc9/src/Quartz/Impl/AdoJobStore/JobStoreSupport.cs and the condition is if (!trig.GetNextFireTimeUtc().HasValue).

Thanks in advance

Hi Chris,

My server is running on multiple instances so I enabled clustering.
My job is running every minute. It's almost working well when I deploy the server. But sometimes the trigger state of the job is blocked because the state of fired trigger is "executing" in the database and it is not removed after the trigger is fired.
Could you please help me to list some reasons for this issue?

This is my code, I program multi-threads so I'm not sure that the issue is from it.

public async Task Execute(IJobExecutionContext context)
		{
			var stopwatch = new Stopwatch();
			stopwatch.Start();

			_logger.Info($"Automation Job is run at {DateTime.UtcNow}");

			// Run automation event
			var automationEventThread = RunCampaignAutomationEvent();

			// Run reminder service
			var reminderNotificationThread = _reminderService.RunReminderNotification();

			// Push notification for mobile based on Start Time and Reminder Before value.
			var reminderPushNotificationThread = _reminderService.PushNotificationToAssignee();

			await Task.WhenAll(automationEventThread, reminderNotificationThread, reminderPushNotificationThread);

			stopwatch.Stop();

			_logger.Info("Time elapsed: {0}", stopwatch.Elapsed);
		}

The following line is missing in the Trigger constructor:

JobDataMap = trigger.JobDataMap;

Hi,

We have legacy project which uses mongodb as backend.
We upgrade the spi package of your to latest preview.

We use linqprovider v2. But while.using this with quartz, because spi package uses mongodb driver 2.21 it uses linqprovider version 3 by default.

Can you provide a option to change the linq provider in Mongodbjobstore implementation

I am using Quartz.NET. and Quartz.Spi.MongoDbJobStore v 3.1.0
I run into the following issue: imagine Job that have triggers in past. To be more specific: today is 1/20/2022, so lets say trigger
StartTime 2021-12-27 01:00:00.000Z
EndTime 2021-12-27 01:00:00.000Z
NextFireTime 2021-12-27 01:00:00.000Z
This case is possible if for some reason my server was turned off (or down) at the time the trigger should fire.
Once I turn the server on back (after EndTime 2021-12-27 01:00:00.000Z) I want Quartz puck up the missed trigger and fire it right away.
So Quartz.Spi.MongoDbJobStore calls Quartz.Spi.MongoDbJobStore.Models.CalendarIntervalTrigger.GetTrigger() (for Cron triggers there is similar flow)
The method creates new CalendarIntervalTrigger() and then tries to FillTrigger().

As we see StartTime for new CalendarIntervalTrigger is not provided. So by default Quartz thru getter returns SystemTime.UtcNow() - let it be as of today 2022-01-22 03:55:00.000Z

Later when in FillTrigger() we assign EndTime (which is 2021-12-27 01:00:00.000Z) before we assing StartTime (2022-01-22 03:55:00.000Z)

then Quartz throws an exception with message "End time cannot be before start time" because of the logic it has in setter

So in FillTrigger() we need assign first StartTime then EndTime

Versions:
Quartz.Spi.MongoDbJobStore 3.1.0

Describe:
We use Quartz.Spi.MongoDbJobStore 3.1.0 in our background job system. We have our abstraction layer above.
Jobs in our aplications have different jobs settings. Some Jobs need concurrent execution to be disallowed and some have peristend data
However, we can't use attributes [DisallowConcurrentExecutionAttribute] and [PersistJobDataAfterExecution] directly due abstraction.

Currently we override JobDetailImpl ConcurrentExecutionDisallowed property.

But due bug in MongoDbJobStore it doesn't works:

Triger didn't change state from Block to Waitting.

MongoDbJobStore method TriggerFiredInternal returns TriggerFiredBundle JobDetail with parsed settings from MongoDb (JobDetail) without override property.
Сonsequently this job can not excuted again and trigger state willn't change from Block.

Hi,

When i try to add and schedule a job with name already existing, i have this problem :
Quartz throw correctly an exception but in the job collection i found two jobs with the same jobname.
This with sql server doesn't happen. because i think in sql server jobname is a PK.

Thank you!

Team,

We need a support for Azure Cosmos DB with .NET Mongo DB driver.
Ours is a multi-tenancy architecture, so we are going with sharded collections - hence there is a need to have the below properties in all the data models.

1. Type - Type of the document - Here we are trying to combine multiple types of document into a single collection as CosmosDB has a limitation on the number of collections.

2. PK (Partition Key) - TenantID + Type - Used for the logical separation of the data. TenantID will be passed from the application.

Thanks,
Raj.

Hi

Using Quartz.Spi.MongoDbJobStore 3.1.0, .Net5.0, 64 bit.

I'm trying to add some Triggers to already created jobs - (The jobs where created by another service).
The two services share an assembly that creates the JobKey.

When calling:

await scheduler.ScheduleJob(triggerDetail).ConfigureAwait(false);

Im getting the below exception.

System.ArgumentException: 'Job class cannot be null.'
This exception was originally thrown at this call stack:
Quartz.Impl.JobDetailImpl.JobType.set(System.Type)
Quartz.Spi.MongoDbJobStore.Models.JobDetail.GetJobDetail()
Quartz.Spi.MongoDbJobStore.MongoDbJobStore.StoreTriggerInternal(Quartz.Spi.IOperableTrigger, Quartz.IJobDetail, bool, Quartz.Spi.MongoDbJobStore.Models.TriggerState, bool, bool, System.Threading.CancellationToken)
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
System.Runtime.CompilerServices.TaskAwaiter.GetResult()
Quartz.Spi.MongoDbJobStore.MongoDbJobStore.StoreTrigger(Quartz.Spi.IOperableTrigger, bool, System.Threading.CancellationToken)
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)

Tried to get the JobDetails which resulted in the same type of exception.
var jobDetail = await scheduler.GetJobDetail(triggerDetail.JobKey).ConfigureAwait(false);

I tried to add the assembly where the job implementation is present - did not change anything.

System.ArgumentException: 'Job class cannot be null.'
This exception was originally thrown at this call stack:
Quartz.Impl.JobDetailImpl.JobType.set(System.Type)
Quartz.Spi.MongoDbJobStore.Models.JobDetail.GetJobDetail()
Quartz.Spi.MongoDbJobStore.MongoDbJobStore.RetrieveJob(Quartz.JobKey, System.Threading.CancellationToken)
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
...

Not sure if you use MassTransit with Quartz integration for scheduled/delayed messages, and/or if that is a supported use case for this driver, but there seems to be a problem deserializing their message wrapper object from the mongo persisted quartz job.

Error is failed deserialization since there is no default constructor: https://github.com/MassTransit/MassTransit/blob/master/src/MassTransit.QuartzIntegration/ScheduledMessageJob.cs#L31

Here is their example hosting service: https://github.com/MassTransit/MassTransit/tree/master/src/MassTransit.QuartzService

Is this something to be addressed on your end, or should I bring it up with the MassTransit team?

Wondering if this driver has cluster/lock support?

ReplaceTriggerInternal should not remove the trigger and the job.

On large collections (e.g. tens of thousands of triggers), queries which use sort SortBuilder.Ascending(trigger => trigger.NextFireTime), SortBuilder.Descending(trigger => trigger.Priority) will fail unless an index is created.

I can create a PR with the required change.

Hi i have a question on the following exception.

I am running a dockerised .net core API service that schedules different types of jobs.

I keep running into this error: Quartz.JobPersistenceException: Unable to add lock instance for lock TriggerAccess on {MachineName}-{Guid}

Is this a fatal error or something i don't need to be concerned about, i only have a singleton scheduler service that is responsible for adding, removing and executing jobs.

I am using a single database with a single container.

The full exception:
Quartz.JobPersistenceException: Unable to add lock instance for lock TriggerAccess on LAPTOP-CMEYER-fef7cff8-e654-40b8-8dfd-044e3799c6ab ---> System.Exception: Unable to add lock instance for lock TriggerAccess on LAPTOP-CMEYER-fef7cff8-e654-40b8-8dfd-044e3799c6ab at Quartz.Spi.MongoDbJobStore.LockManager.AddLock(LockInstance lockInstance) in C:\Sandbox\Server\Server\src\Scheduler.API\Quartz.Spi.MongoDbJobStore\LockManager.cs:line 75 at Quartz.Spi.MongoDbJobStore.LockManager.AcquireLock(LockType lockType, String instanceId) in C:\Sandbox\Server\Server\src\Scheduler.API\Quartz.Spi.MongoDbJobStore\LockManager.cs:line 54 at Quartz.Spi.MongoDbJobStore.MongoDbJobStore.StoreJobAndTrigger(IJobDetail newJob, IOperableTrigger newTrigger, CancellationToken cancellationToken) in C:\Sandbox\Server\Server\src\Scheduler.API\Quartz.Spi.MongoDbJobStore\MongoDbJobStore.cs:line 201 --- End of inner exception stack trace --- at Quartz.Spi.MongoDbJobStore.MongoDbJobStore.StoreJobAndTrigger(IJobDetail newJob, IOperableTrigger newTrigger, CancellationToken cancellationToken) in C:\Sandbox\Server\Server\src\Scheduler.API\Quartz.Spi.MongoDbJobStore\MongoDbJobStore.cs:line 214 at Quartz.Core.QuartzScheduler.ScheduleJob(IJobDetail jobDetail, ITrigger trigger, CancellationToken cancellationToken) in C:\projects\quartznet\src\Quartz\Core\QuartzScheduler.cs:line 617 at Scheduler.SchedulerService.ScheduleTriggerEvaluationAsync(String groudId, String triggerId, Boolean isActivate, DateTimeOffset date) in C:\Sandbox\Server\Server\src\Scheduler.API\Scheduler\SchedulerService.cs:line 58 at Scheduler.API.Service.SchedulerApi.ScheduleEventAsync(CalendarEventEntity calendarEventEntity) in C:\Sandbox\Server\Server\src\Scheduler.API\Scheduler.API.Service\SchedulerAPI.cs:line 270 at Scheduler.API.Service.SchedulerApi.EventEntityChangeEventAsync(IRedGreenQueueAdapter redq, JObject jo, String publishKey, String subscribeKey, String replyKey, Hashtable headers) in C:\Sandbox\Server\Server\src\Scheduler.API\Scheduler.API.Service\SchedulerAPI.cs:line 89 at DAPI.RedGreenQueue.Adapter.RedGreenQueueAdapter.ProcessSubscriptionMessageAsync(String publishKey, String subscribeKey, String replyKey, String message, Func1 effectiveMessage, Boolean logNoSubscribes, Hashtable headers) [See nested exception: System.Exception: Unable to add lock instance for lock TriggerAccess on LAPTOP-CMEYER-fef7cff8-e654-40b8-8dfd-044e3799c6ab
at Quartz.Spi.MongoDbJobStore.LockManager.AddLock(LockInstance lockInstance) in C:\Sandbox\Server\Server\src\Scheduler.API\Quartz.Spi.MongoDbJobStore\LockManager.cs:line 75
at Quartz.Spi.MongoDbJobStore.LockManager.AcquireLock(LockType lockType, String instanceId) in C:\Sandbox\Server\Server\src\Scheduler.API\Quartz.Spi.MongoDbJobStore\LockManager.cs:line 54
at Quartz.Spi.MongoDbJobStore.MongoDbJobStore.StoreJobAndTrigger(IJobDetail newJob, IOperableTrigger newTrigger, CancellationToken cancellationToken) in C:\Sandbox\Server\Server\src\Scheduler.API\Quartz.Spi.MongoDbJobStore\MongoDbJobStore.cs:line 201]`

I'm migrating to MT3 along with newest MongoDB/Quartz/etc., and am having issues getting your mongodb connector working. I was using https://github.com/Nanoko/quartznet-mongodb previously, but want Mongodb 2.x driver support.

When it tries to resolve the scheduler from the factor I get:

The memberInfo argument must be for class JobKey, but was for class Key`1.\r\nParameter name: 
memberInfo

It seems to be an issue coming from the BsonSerializer, which doesn't make a ton of sense to me as I'm pointing this thing at a fresh DB (was thinking it might be from existing records in the old db at first).

Thoughts?

I'm using the following setup:

var properties = new NameValueCollection
            {
                [StdSchedulerFactory.PropertySchedulerInstanceName] = "MassTransit_QuartzNet",
                [StdSchedulerFactory.PropertySchedulerInstanceId] = $"{Environment.MachineName}-{Guid.NewGuid()}",
                [StdSchedulerFactory.PropertyJobStoreType] = typeof(MongoDbJobStore).AssemblyQualifiedName,
                [
                    $"{StdSchedulerFactory.PropertyJobStorePrefix}.{StdSchedulerFactory.PropertyDataSourceConnectionString}"
                    ] = ConfigurationManager.ConnectionStrings["quartznet-mongodb"].ConnectionString,
                [$"{StdSchedulerFactory.PropertyJobStorePrefix}.collectionPrefix"] = "prefix"
            };
            // I treat the database in the connection string as the one you want to connect to
            // The prefix is optional

            var factory = new StdSchedulerFactory(properties);

with app.config records as follows:

 <connectionStrings>
    <add name="quartznet-mongodb" connectionString="mongodb://quartznet:passwordhere@mongo1/quartznet" />
  </connectionStrings>
  <quartz>
    <add key="quartz.scheduler.instanceName" value="MassTransit_QuartzNet" />
    <add key="quartz.threadPool.type" value="Quartz.Simpl.SimpleThreadPool, Quartz" />
    <add key="quartz.threadPool.threadCount" value="4" />
    <add key="quartz.threadPool.threadPriority" value="2" />
    <add key="quartz.jobStore.misfireThreshold" value="60000" />
    <add key="quartz.jobStore.type" value="Quartz.Spi.MongoDbJobStore.MongoDbJobStore, Quartz.Spi.MongoDbJobStore" />
  </quartz>

Interesting project, but it gives me a hard time while trying to make the configuration correct.

I want to run Quartz.NET scheduler as a Win service and MongoDB to be it's storage engine but I cannot make it work.

This is my Quartz config:

# You can configure your scheduler in either <quartz> configuration section
# or in quartz properties file
# Configuration section has precedence

quartz.scheduler.instanceName = ServerScheduler
#quartz.scheduler.instanceId = 726d6c6e-532e-4fc3-acb2-6e3f4b5a8f66

# configure thread pool info
quartz.threadPool.type = Quartz.Simpl.SimpleThreadPool, Quartz
quartz.threadPool.threadCount = 10
quartz.threadPool.threadPriority = Normal

# job initialization plugin handles our xml reading, without it defaults are used
#quartz.plugin.xml.type = Quartz.Plugin.Xml.XMLSchedulingDataProcessorPlugin, Quartz
#quartz.plugin.xml.fileNames = ~/quartz_jobs.xml

quartz.jobStore.type = Quartz.Spi.MongoDbJobStore.MongoDbJobStore, Quartz
quartz.dataSource.connectionString = "mongodb://192.168.99.100:27017/quartz"
quartz.jobStore.useProperties = true

# export this server to remoting context
quartz.scheduler.exporter.type = Quartz.Simpl.RemotingSchedulerExporter, Quartz
quartz.scheduler.exporter.port = 555
quartz.scheduler.exporter.bindName = QuartzScheduler
quartz.scheduler.exporter.channelType = tcp
quartz.scheduler.exporter.channelName = httpQuartz

but when I start the WinService or run only in console mode, I get the following error log:

2017-07-13 22:04:08,991 [1] DEBUG Quartz.Server.QuartzServerFactory.CreateServer(c:\projects\quartznet\server\Quartz.Server\QuartzServerFactory.cs:24) - Creating new instance of server type 'Quartz.Server.QuartzServer, Quartz.Server, Version=2.5.0.400, Culture=neutral, PublicKeyToken=null'
2017-07-13 22:04:09,013 [1] DEBUG Quartz.Server.QuartzServerFactory.CreateServer(c:\projects\quartznet\server\Quartz.Server\QuartzServerFactory.cs:26) - Instance successfully created
2017-07-13 22:04:09,028 [1] INFO  Quartz.Impl.StdSchedulerFactory.Initialize(c:\projects\quartznet\src\Quartz\Impl\StdSchedulerFactory.cs:231) - Quartz.NET properties loaded from configuration file 'C:\Program Files\Server.Net.Quartz\quartz.config'
2017-07-13 22:04:09,068 [1] ERROR Quartz.Server.QuartzServer.Initialize(c:\projects\quartznet\server\Quartz.Server\QuartzServer.cs:34) - Server initialization failed:Length cannot be less than zero.
Parameter name: length
System.ArgumentOutOfRangeException: Length cannot be less than zero.
Parameter name: length
   at System.String.Substring(Int32 startIndex, Int32 length)
   at Quartz.Util.PropertiesParser.GetPropertyGroups(String prefix) in c:\projects\quartznet\src\Quartz\Util\PropertiesParser.cs:line 582
   at Quartz.Impl.StdSchedulerFactory.Instantiate() in c:\projects\quartznet\src\Quartz\Impl\StdSchedulerFactory.cs:line 535
   at Quartz.Impl.StdSchedulerFactory.GetScheduler() in c:\projects\quartznet\src\Quartz\Impl\StdSchedulerFactory.cs:line 1121
   at Quartz.Server.QuartzServer.Initialize() in c:\projects\quartznet\server\Quartz.Server\QuartzServer.cs:line 34

What am I missing from the config file ?

Hi! I´m developing a Quartz WebApi in .net core 2.2. I´m having trouble the first time i run my service. Some of the jobs are stored in MongoDB. Some aren´t. And then sometimes the jobs don´t trigger. Or if I stop the service and try running the service with one more job scheduled the whole thing gets confused somehow and the job is not stored in MongoDb. I´m using chronSchedule btw.
If i run my service without a persistent jobStore everything works fine.

What am I missing?

Can you give more details about how db trigger removal works? I am encountering some scenarios where triggers shouldn't be removed (on scheduler shutdown/restart), but maybe there's something I'm missing.

Before I actually dive in and start using, could you please tell me, if your library can support the latest mongodb driver 2.4.4?

Thanks!

Hello. I'm getting a NotSupportedException when using this package with a .NET 6 app. The exception is below:

NotSupportedException: BinaryFormatter serialization and deserialization are disabled within this application. See https://aka.ms/binaryformatter for more information.
MongoDB.Driver.Core.Connections.BinaryConnection.SendMessagesAsync(IEnumerable<RequestMessage> messages, MessageEncoderSettings messageEncoderSettings, CancellationToken cancellationToken)

JobPersistenceException: BinaryFormatter serialization and deserialization are disabled within this application. See https://aka.ms/binaryformatter for more information.
Quartz.Spi.MongoDbJobStore.MongoDbJobStore.StoreJobAndTrigger(IJobDetail newJob, IOperableTrigger newTrigger, CancellationToken cancellationToken)

The issue is in the DefaultObjectSerializer class that is used in a couple of places to serialize the job data map and some calendar data. From what I see in the code, it's not possible to opt out of the BinaryFormatter usages by specifying my own serializer for these types since they are registered with the MongoDb driver in JobStoreClassMap.

I don't want to disable this check for my entire project. So, an alternative approach is to use BinaryWriter and BinaryReader to manually serialize/deserialize data, but this would break existing apps since the serialize format would change. Happy to submit a PR if you can offer a suggestion.

Make it work with Quartz 3.0.7 and Mongo DB Driver 2.7.2

Hi!

When trying to schedule many jobs at the same time, the following exception is bursting and consequently taking a long time to finalize the schedule:

MongoDB.Driver.Core.Operations.MongoBulkWriteOperationException em System.Private.CoreLib.dll

Follow the example of building jobs, the code snippet is inserted in a for loop, assigning the respective keys dynamically:

               var triggersAndJobs = new Dictionary<IJobDetail, IReadOnlyCollection<ITrigger>>();

                var jobBuilder = JobBuilder.Create<Scheduler>()
                    .WithIdentity(jobKey, GroupName)
                    .RequestRecovery(true))
                    .Build();

                var triggerCollection = new List<ITrigger>{ TriggerBuilder.Create()
                    .WithIdentity(triggerKey, GroupName)
                    .StartAt(time.ToUniversalTime())
                    .WithSchedule(SimpleScheduleBuilder.Create().WithMisfireHandlingInstructionFireNow())
                    .Build() };

                triggersAndJobs.Add(job, triggerCollection);
            
                await scheduler.ScheduleJobs(triggersAndJobs, true);

In 4.0.0-preview.1, I am facing the following issue:
"Attempt by method 'Quartz.Spi.MongoDbJobStore.Models.JobDetail.GetJobDetail()' to access method 'Quartz.Impl.JobDetailImpl.set_Key(Quartz.JobKey)' failed."

Error in: JobDetail.cs, line #46

since binary formatter is depreciated for security reasons, we would like to have a fix with JSON Serializer for version 3.x.