Since upgrading our services to .NET 8 and upgrading Rebus to version 8.x, second-level retry messages are no longer being dispatched.
Our setup makes use of the Rebus core, Rebus.ServiceProvider and Rebus.TransactionScopes packages. The issue does not appear to be related to the specific transport being used.
Rebus will correctly attempt first-level retries, but fails to dispatch an IFailed message for the 2nd level. An exception (System.Transactions.TransactionAbortedException: The transaction has aborted.
) will be thrown instead, and the message will be sent to the error queue.
info: Repro.ErrorMessageHandler[0]
Handling ErrorMessage
[WRN] Rebus.Retry.Simple.DefaultExceptionLogger (Rebus 1 worker 1): Unhandled exception 1 while handling message with ID "bd502ade-f010-43a7-a84a-fadd439bce46"
System.Exception: Error
at Repro.ErrorMessageHandler.Handle(ErrorMessage message) in C:\Users\BavoLuysterborg\source\repos\Repro\ErrorMessageHandler.cs:line 24
at Rebus.Pipeline.Receive.HandlerInvoker`1.Invoke()
at Rebus.Pipeline.Receive.DispatchIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Sagas.LoadSagaDataStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.ActivateHandlersStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleRoutingSlipsStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.FailedMessageWrapperStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.TransactionScopes.TransactionScopeIncomingStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.DataBus.ClaimCheck.HydrateIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.DefaultRetryStep.Process(IncomingStepContext context, Func`1 next)
info: Repro.ErrorMessageHandler[0]
Handling ErrorMessage
[WRN] Rebus.Retry.Simple.DefaultExceptionLogger (Rebus 1 worker 1): Unhandled exception 2 while handling message with ID "bd502ade-f010-43a7-a84a-fadd439bce46"
System.Exception: Error
at Repro.ErrorMessageHandler.Handle(ErrorMessage message) in C:\Users\BavoLuysterborg\source\repos\Repro\ErrorMessageHandler.cs:line 24
at Rebus.Pipeline.Receive.HandlerInvoker`1.Invoke()
at Rebus.Pipeline.Receive.DispatchIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Sagas.LoadSagaDataStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.ActivateHandlersStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleRoutingSlipsStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.FailedMessageWrapperStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.TransactionScopes.TransactionScopeIncomingStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.DataBus.ClaimCheck.HydrateIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.DefaultRetryStep.Process(IncomingStepContext context, Func`1 next)
[ERR] Rebus.Retry.PoisonQueues.DeadletterQueueErrorHandler (Rebus 1 worker 1): Moving message with ID "bd502ade-f010-43a7-a84a-fadd439bce46" to error queue "error" - error details: "3 unhandled exceptions: 02/02/2024 16:24:18 +01:00: System.Exception: Error
at Repro.ErrorMessageHandler.Handle(ErrorMessage message) in C:\Users\BavoLuysterborg\source\repos\Repro\ErrorMessageHandler.cs:line 24
at Rebus.Pipeline.Receive.HandlerInvoker`1.Invoke()
at Rebus.Pipeline.Receive.DispatchIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Sagas.LoadSagaDataStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.ActivateHandlersStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleRoutingSlipsStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.FailedMessageWrapperStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.TransactionScopes.TransactionScopeIncomingStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.DataBus.ClaimCheck.HydrateIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.DefaultRetryStep.Process(IncomingStepContext context, Func`1 next)
02/02/2024 16:24:19 +01:00: System.Exception: Error
at Repro.ErrorMessageHandler.Handle(ErrorMessage message) in C:\Users\BavoLuysterborg\source\repos\Repro\ErrorMessageHandler.cs:line 24
at Rebus.Pipeline.Receive.HandlerInvoker`1.Invoke()
at Rebus.Pipeline.Receive.DispatchIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Sagas.LoadSagaDataStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.ActivateHandlersStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleRoutingSlipsStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.FailedMessageWrapperStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.TransactionScopes.TransactionScopeIncomingStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.DataBus.ClaimCheck.HydrateIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.DefaultRetryStep.Process(IncomingStepContext context, Func`1 next)
02/02/2024 16:24:19 +01:00: System.Transactions.TransactionAbortedException: The transaction has aborted.
at System.Transactions.TransactionStateAborted.CreateAbortingClone(InternalTransaction tx)
at System.Transactions.DependentTransaction..ctor(IsolationLevel isoLevel, InternalTransaction internalTransaction, Boolean blocking)
at System.Transactions.Transaction.DependentClone(DependentCloneOption cloneOption)
at System.Transactions.TransactionScope.SetCurrent(Transaction newCurrent)
at System.Transactions.TransactionScope.PushScope()
at System.Transactions.TransactionScope.Initialize(Transaction transactionToUse, TimeSpan scopeTimeout, Boolean interopModeSpecified)
at System.Transactions.TransactionScope..ctor(Transaction transactionToUse, TransactionScopeAsyncFlowOption asyncFlowOption)
at Rebus.TransactionScopes.TransactionScopeIncomingStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.DataBus.ClaimCheck.HydrateIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.DefaultRetryStep.DispatchSecondLevelRetry(ITransactionContext transactionContext, StepContext context, Func`1 next)
at Rebus.Retry.Simple.DefaultRetryStep.HandleException(Exception exception, ITransactionContext transactionContext, String messageId, IncomingStepContext context, Func`1 next)"
[WRN] Rebus.Workers.ThreadPoolBased.ThreadPoolWorker (.NET TP Worker): Error in transaction context
System.Transactions.TransactionAbortedException: The transaction has aborted.
at System.Transactions.TransactionStateAborted.BeginCommit(InternalTransaction tx, Boolean asyncCommit, AsyncCallback asyncCallback, Object asyncState)
at System.Transactions.CommittableTransaction.Commit()
at System.Transactions.TransactionScope.InternalDispose()
at System.Transactions.TransactionScope.Dispose()
at Rebus.TransactionScopes.TransactionScopeTransportDecorator.<>c__DisplayClass7_0.<Receive>b__1(ITransactionContext _)
at Rebus.Transport.TransactionContext.Dispose()