Google Group Discussion

So the basic idea is that the user would like to have a "send only" gateway where the site which is doing SendToSites only acts as an HTTP client and doesn't expose an incoming channel.

The implications of supporting that is that it would break Request/Reply semantics over the gateway. Are we willing to introduce this as a configuration option on the gateway and deal with the unpleasant surprises when someone is using Reply but the message never arrives?

My opinion is No but I would like to discuss it here.

Eventually causing the endpoint to run out of memory.

See Particular/NServiceBus#3564 for more details

Task dispatch logic deadlocks when a task was created using Task.Factory.StartNew() and TaskScheduler.Current as argument for the scheduler. If the scheduler threadpool does not have free threads and the task is queued for this pool and the executing thread does not complete this results in a deadlock.

At the moment, in order to provide a custom channel type, you need to config.Gateway().ChannelFactories(...). This entirely overwrites the existing channel factories and the existing http/https channels are internal so the user cannot reference them. If a customer wanted to mix custom custom channels and http/https channels they would need to perform reflection or re-implement the http/https channels themselves.

This feels really heavy handed, especially as we already have ChannelTypeAttribute to allow us to switch channels based on the channel type of the remote site.

Taskforce: @boblangley @mikeminutillo @janovesk

Problem

Currently satellites, including the gateway, have different recoverability requirements from standard endpoints and cannot share FLR, SLR, and StoreFaultsInErrorQueue behaviors.

The approach taken by the Core is for each satellite to implement an internal recoverability behavior to implement its own specific requirements (e.g. Timeouts only use FLR and move failed messages to error queue).

Another aspect is that we don't want users to be notified about FLR/SLR/Fault events that occur in satellites.

The task at hand

In case of the gateway, the requirement is to skip FLR but use SLR and move failed messages to error queue. This rationale behind this approach is the fact that the prevalent failure mode of the gateway is that the channel between sites is down or faulty. In such case immediate message retries would put even more pressure on the faulty channel. Exponential backoff is a much better error handling policy in this situation.

Plan of attack

• Implement the new recoverability behavior - PR
• Creat core PR to extend FaultContext
  • Created alternative approach for core PR
• Get [core PR]Particular/NServiceBus#3694) approved and merged
• Finalize the disussions on Suggested PR
• Document the new config settings
• Decide whether the behavior needs to be configurable using SLR config API or if we can come up with sensible defaults. Figure out what the defaults for retries/timeIncrease should be if wanted.
• Doco PR
  • New behaviour
  • Config API
  • Upgrade guide PR
• Retrospective

Retrospective

Attending: @janovesk @boblangley

What went well?

• Regular handoffs / updates
• Low barrier to refactor others code

What can be improved?

• Smaller doco PRs
• Working over 3 continents can cause design churn. Members of Task Forces with 3 continents should be prepared for more drastic changes when it is their turn to contribute.
• No estimate for size was provided, and it felt like more effort was needed than expected.

What is the problem?

We have recently changed the vocabulary we're using to describe recoverability actions in the core. The terms we now use are:

• Immediate retries,
• Delayed retries,
• Moving a message to error queue.

The Gateway has an API to configure number of retries the Gateway will make when sending message over the channel fails. The term currently used in this API is "retry".

The problem is that this term is not consistent with the vocabulary used by the core. What is more, it does not clearly state that we're configuring number of channel retransmission attempts, which is a different concern than core recoverability.

What can we do about it?

My proposal is to find a proper term to use and rename the APIs accordingly. We have 3 API calls to consider:

• gateway.Retries(5, TimeSpan.FromMinutes(1))
• gateway.CustomRetryPolicy(customRetryPolicy: (message, exception, currentRetry) =>...)
• gateway.DisableRetries()

I think borrowing established terms from related areas of computer science may make it easier for users to understand what is actually being configured. The best term I could find is borrowed from the world of TCP/IP networking and WCF (which seems very relevant in the context of a gateway channel): retransmission. The APIs would then look like this:

• gateway.Retransmissions(5, TimeSpan.FromMinutes(1))
• gateway.RetransmissionsPolicy(retransmissionsPolicy: (message, exception, currentRetransmission) =>...)
• gateway.DisableRetransmissions()

@danielmarbach @andreasohlund I'd love to get your thoughts about this.

Plan of Attack

• Implement the change
• Update doco

While attempting to set up a gateway listening on a wildcard address I found that the gateway currently doesn't listen on multiple addresses. Instead, the last configured receive channel is used.

If you configure a wildcard channel last, then you get an error (because you can't use a wildcard receive channel as the default channel).

The SendToSites extension casts the extended IBus into an UnicastBus which prevents handlers using this extension from being unit tested. We should provide a way for users to test if a message has been sent to a certain site.

Symptoms

When sending a message to multiple sites, only the last registered site will get the message.

Who's affected

v3.0.0 user sending to more than one site.

A V3 endpoint cannot send messages to a Gateway running NSB 5.x and NSB.Gateway 1.0.1.

The V3 client gets 400 Bad Request.

The V5 server shows this:

WARN NServiceBus.Gateway.Channels.Http.HttpChannelReceiver Cannot process HTTP request from [fe80::b061:f586:e59e:6bab%4]:57604. Reason: Invalid CallType 'Submit'. CallTypes supported 'SingleCallSubmit, SingleCallDatabusProperty'.

In V3, the Gateway uses these call types:

public enum CallType
{
    Submit,
    Ack,
    DatabusProperty
}

But in CoreV5/NSB.Gateway 1.0, it seems to have changed drastically:

public enum CallType
{
    /// <summary>
    /// Default request type.
    /// </summary>
    SingleCallSubmit,
    /// <summary>
    /// Request type for Databus properties.
    /// </summary>
    SingleCallDatabusProperty
}

It seems "single call" means that Acks are not required. In V3, it does a Send, then sends any data bus properties, and then ends with an Ack.

Calling on @johnsimons as you touched Gateway code in V3, otherwise @SimonCropp and @andreasohlund may be the only ones with enough historical perspective on this. (Outside Udi of course.)

Related to Desk Case.

Symptoms

When registering more than one receive channel, only the last registered channel is used.

Who's affected

v3.0.0 Users configuring more than one receive channel

Details

Originally raised by @welshdave in #98

Same as Particular/NServiceBus#3418, just for the gateway.

Also as requested #26 (comment)

We do not have any test coverage for this functionality. This was discovered while creating snippets for doco and it was discovered the receive func had the wrong signature, #47 was opened to correct the signature.

We may also want to create samples as well.

Dependabot couldn't reach https://www.myget.org/F/particular/api/v3/index.json as it timed out.

Is https://www.myget.org/F/particular/api/v3/index.json accessible over the internet? If it is then this may be a transitory issue and can be ignored - Dependabot will close it on its next successful update run.

View the update logs.

The core used to support NServiceBus.MultiSite as a profile. Since that's now obsoleted in NSB v5, what's the equivalent profile in NServiceBus.Gateway?

• Fix the obsolete message
• Update doco and remove multisite from it

Raised by @johnsimons
Migrated from by Particular/NServiceBus#1440

The code in the HttpChannelReceiver is creating a HttpListener per channel.

This seems to be quite inefficient because the HttpListener support listening on multiple addresses!

I wonder what the saving would be if we were to just using one HttpListener listening on multiple channel addresses ❓

Cloudflare does not support dots . in header keys which is used on NServiceBus headers. To make it compatible with Cloudflare either the header passing must be changed or should be customizable.

https://eu1.salesforce.com/apex/Case?num=00033052

In the next major version (4.0) we should fix the diverged public API and then collapse the API approval test back to being the same test file for both target frameworks.

Current gateway configuration allows only for a single reply to address. When using the built-in Http channel, this address is also used for hosting the HttpListener.

When the gateway is hosted behind a load balancer, the public address of the gateway can be different from the local address on the machine hosting the actual gateway endpoint.

Having separate reply address config for each channel will decouple the HttpListener from the reply address and allow more flexibility when additional gateway channels (like SQS, ActiveMQ, Azure) are available later on.

http://tech.groups.yahoo.com/group/nservicebus/message/20000

Raised by @johnsimons
Migrated from by Particular/NServiceBus#1380

It is up to the client to send that header.
Therefore the server should only validate it if header is present.

GVT issue as it seems C:\Build\src\packages\NuGetPackager.0.5.5\build\NuGetPackager.targets(6, 5): error MSB4044: The "CreatePackages" task was not given a value for the required parameter "Version".

https://builds.particular.net/viewLog.html?buildId=188806&tab=buildResultsDiv&buildTypeId=NServiceBusGateway_1Build

Different sites can have a different queue infrastructure but potentially also use a different wire format.

It would make sense to be able to choose a serializer per channel to conform to the wireformat used at the remote site:

Example config

<GatewayConfig>
  <Sites>
    <Site
      Key="RemoteSite"
      Address="http://localhost:25899/RemoteSite/"
      ChannelType="Http" />
  </Sites>
  <Channels>
    <Channel
      Address="http://localhost:25899/Headquarters/"
      ChannelType="Http"
      ChannelContentType="text/xml" />
  </Channels>
</GatewayConfig>

In V6 it is possible to register additional serializers to be used for deserialization but if a site/system wanted to migrate to a different wire format then the remote site needs to be update too.

References

• http://docs.particular.net/samples/gateway/
• http://docs.particular.net/nservicebus/serialization/#specifying-additional-deserializers

Raised by @yvesgoeleven
Migrated from by Particular/NServiceBus#1969

Sender does, but receiver does not

Switch completely to code-first API. Keep in mind to update the docs.

Raised by @andreasohlund
Migrated from by Particular/NServiceBus#693

In order to improve our interop story we should make it easier to get message into NSB via http. While this is included today

https://github.com/NServiceBus/NServiceBus/blob/master/Samples/Gateway/WebClient/Index.htm#L17

It's very clunky and needs to be improved.

• POST /Messages/{MessageType} => sends the given message
• We should support protocol translation so that you can submit via eg. json but still have the bus use xml.

The Gateway adds a custom satellite using PushRuntimeSettings.Default as the concurrency setting:

Do we need to make the satellite's queue concurrency settings configurable?

This would allow the gw to work with nginx and cloudflare

See https://github.com/welshdave/NServiceBus.Gateway.Channels.HttpVNext for more details

Raised by @johnsimons
Migrated from by Particular/NServiceBus#1381

When a http request is received, an MD5 is calculated against the body payload, but by default it uses UTF8.

Instead it should look at the Content-Encoding header and then use the correct encoder to retrieve the payload

Connects to Particular/PlatformDevelopment#182

This is the Gateway epic which incorporates the work for making the Gateway async/await enabled. Please see this blog post about the reasoning behind it.

Since its marked as deprecated by MS

#104 (comment)

Also consider enabling the Microsoft.DotNet.Analyzers.Compatibility for the project

When the storage creation requires async initialization the calling code has to either have the ability to use a sync API or needs to do sync over async. Example from RavenDB

            var ravenGatewayDeduplicationConfiguration = new RavenGatewayDeduplicationConfiguration((builder, _) => 
            {
                databaseName = Guid.NewGuid().ToString();
                var documentStore = GetInitializedDocumentStore(databaseName);

                documentStore.Maintenance.Server.Send(new CreateDatabaseOperation(new DatabaseRecord(databaseName)));

                return documentStore;
            })
            {
                EnableClusterWideTransactions = true
            };

CreateStorage should allow async storage creation

Symptoms

An exception can be thrown on startup from GatewayHttpListenerInstaller (such as the exception below, but this is logger-specific) due to logging statements that are misunderstood by certain logger implementations. Specifically, NLog and the Microsoft.Extensions.Logging.Console implementation don't appear to work, but there may be others.

Who's affected

Anyone running the Gateway component with a logger that doesn't support repeated logging tokens, such as NLog.

Root cause

While string.Format("{0}-{0}", "a") will use the same token and result in a-a, using logger.InfoFormat("{0}-{0}", "a") has uneven results depending on the logger implementation being used.

In NLog:

logger.InfoFormat("{0} - {0}", "a", "b"); // Result: "a-a"
logger.InfoFormat("{0} - {0}", "a"); // Result: Exception, wants 2nd param even though it won't use it

Original bug report

Describe the bug

Description

When execute EnableInstaller() In line 49 and 57 of GatewayHttpListenerInstaller.cs the format of the string on calling log.information throws an exception

System.AggregateException: An error occurred while writing to logger(s). (Index (zero based) must be greater than or equal to zero and less than the size of the argument list.)

Expected behavior

The log is write in configured loggers.

Actual behavior

The string in log information is wrong formated

Versions

Nservicebus.Gateway 4.0.0

Steps to reproduce

Use EnableInstallers without admin privileges and debug level sets as "Information".

Relevant log output

System.AggregateException: An error occurred while writing to logger(s). (Index (zero based) must be greater than or equal to zero and less than the size of the argument list.)
 ---> System.FormatException: Index (zero based) must be greater than or equal to zero and less than the size of the argument list.
   at System.Text.ValueStringBuilder.AppendFormatHelper(IFormatProvider provider, String format, ParamsArray args)
   at System.String.FormatHelper(IFormatProvider provider, String format, ParamsArray args)
   at System.String.Format(IFormatProvider provider, String format, Object[] args)
   at Microsoft.Extensions.Logging.LogValuesFormatter.Format(Object[] values)
   at Microsoft.Extensions.Logging.FormattedLogValues.ToString()
   at Microsoft.Extensions.Logging.LoggerExtensions.MessageFormatter(FormattedLogValues state, Exception error)
   at OpenTelemetry.Logs.OpenTelemetryLogger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func`3 formatter)
   at Microsoft.Extensions.Logging.Logger.<Log>g__LoggerLog|13_0[TState](LogLevel logLevel, EventId eventId, ILogger logger, Exception exception, Func`3 formatter, List`1& exceptions, TState& state)
   --- End of inner exception stack trace ---
   at Microsoft.Extensions.Logging.Logger.ThrowLoggingError(List`1 exceptions)
   at Microsoft.Extensions.Logging.Logger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func`3 formatter)
   at Microsoft.Extensions.Logging.LoggerExtensions.Log(ILogger logger, LogLevel logLevel, EventId eventId, Exception exception, String message, Object[] args)
   at Microsoft.Extensions.Logging.LoggerExtensions.Log(ILogger logger, LogLevel logLevel, String message, Object[] args)
   at Microsoft.Extensions.Logging.LoggerExtensions.LogInformation(ILogger logger, String message, Object[] args)
   at NServiceBus.Extensions.Hosting.Logger.InfoFormat(String format, Object[] args) in /_/src/NServiceBus.Extensions.Hosting/Logging/Logger.cs:line 52
   at NServiceBus.Installation.GatewayHttpListenerInstaller.Install(String identity, CancellationToken cancellationToken) in /_/src/NServiceBus.Gateway/Installer/GatewayHttpListenerInstaller.cs:line 57
   at NServiceBus.HostingComponent.RunInstallers(CancellationToken cancellationToken) in /_/src/NServiceBus.Core/Hosting/HostingComponent.cs:line 74
   at NServiceBus.ExternallyManagedContainerHost.Start(IServiceProvider serviceProvider, CancellationToken cancellationToken) in /_/src/NServiceBus.Core/Hosting/ExternallyManagedContainerHost.cs:line 45
   at NServiceBus.Extensions.Hosting.NServiceBusHostedService.StartAsync(CancellationToken cancellationToken) in /_/src/NServiceBus.Extensions.Hosting/NServiceBusHostedService.cs:line 30
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)

Additional Information

Workarounds

Setup the log leven on warning.

Possible solutions

Additional information

Symptoms

Calls to .SendToSites("A","B") silently fails for sites that are not configured.

Who's affected

All users sending to more than one site where one or more of them is not configured.

The gateway is configured with a list of logical site names, each of which has a certain channel specified for it.

It would be nice to be able to monitor connectivity to these sites, though in a geographically distributed system, it is likely for connectivity to fluctuate so we might want to report failure less aggressively than with regular CustomChecks.

transferred from a private repository

To repro run the gateway sample but change endpointConfiguration.EnableFeature<Gateway>() to endpointConfiguration.Gateway()

the endpoint will now blow up with

Autofac.Core.DependencyResolutionException was unhandled
  HResult=-2146233088
  Message=None of the constructors found with 'Autofac.Core.Activators.Reflection.DefaultConstructorFinder' on type 'NServiceBus.Installation.GatewayHttpListenerInstaller' can be invoked with the available services and parameters:
Cannot resolve parameter 'NServiceBus.Gateway.Receiving.IManageReceiveChannels channelManager' of constructor 'Void .ctor(NServiceBus.Gateway.Receiving.IManageReceiveChannels, Boolean)'.
  Source=NServiceBus.Core
  StackTrace:
       at Autofac.Core.Activators.Reflection.ReflectionActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.Core.Resolving.InstanceLookup.Execute() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
       at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
       at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
       at Autofac.Features.Collections.CollectionRegistrationSource.<>c__DisplayClass4.<RegistrationsFor>b__0(IComponentContext c, IEnumerable`1 p) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.Core.Activators.Delegate.DelegateActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.Core.Resolving.InstanceLookup.Execute() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.Core.Resolving.ResolveOperation.Execute(IComponentRegistration registration, IEnumerable`1 parameters) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable`1 parameters, Object& instance) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at Autofac.ResolutionExtensions.ResolveService(IComponentContext context, Service service, IEnumerable`1 parameters) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\IncomingMessage.cs:line 0
       at NServiceBus.AutofacObjectBuilder.ResolveAll(IComponentContext container, Type componentType) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\ObjectBuilder\Autofac\AutofacObjectBuilder.cs:line 180
       at NServiceBus.CommonObjectBuilder.BuildAll[T]() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\ObjectBuilder\Common\CommonObjectBuilder.cs:line 48
       at NServiceBus.InitializableEndpoint.<RunInstallers>d__13.MoveNext() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\InitializableEndpoint.cs:line 164
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at NServiceBus.InitializableEndpoint.<Initialize>d__1.MoveNext() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\InitializableEndpoint.cs:line 58
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at NServiceBus.Endpoint.<Start>d__1.MoveNext() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Endpoint.cs:line 27
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
       at Program.<AsyncMain>d__1.MoveNext() in C:\Users\andreas.ohlund\Downloads\gateway_gateway_2\Headquarters\Program.cs:line 24
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
       at Program.Main() in C:\Users\andreas.ohlund\Downloads\gateway_gateway_2\Headquarters\Program.cs:line 11
  InnerException: 

Looking at the code I think we just need to add a call to .EnableFeature<Gateway>() in https://github.com/Particular/NServiceBus.Gateway/blob/develop/src/NServiceBus.Gateway/Settings/GatewayExtensions.cs#L14

In order to support to redirect various messages to different endpoints depending on some business logic (not via namespace/assembly/type).

See original request in http://stackoverflow.com/q/30352155/90882

Instead add it to the collection and use that info to set the header at a later stage in the pipeline.

See #104 (comment)

Because the installer expects the settings object to be present. The settings is created in the Defaults section of the feature so it only gets invoked if the feature is enabled.

Task force: @janovesk (lead) @SzymonPobiega

Background

Sending messages with databus properties through the gateway is currently implemented by using one HTTP request or the message and one additional HTTP request for each databus property. The receiver correlates the calls with an in-memory cache and eventually reconstructs the message. In a scaled out scenario this will not work because physical gateway instances do not share the cache.

Running the gateway on the master node/distributor is recommended in version 5. It is by design a single process, so this works.

Version 6 enables non-brokered scale-out by sending messages directly to individual endpoint instances. End users will need to either

a) only enable the gateway on one particular endpoint instance and route all HTTP request to it, or
b) enable it for all the instances and use a load balancer in front to direct the incoming HTTP traffic to one of them.

Scenario B is a highly likely deployment scenario, and the HTTP request for the message might end up on a different gateway instance than HTTP request for the individual databus properties. If that happens, none of them will have the complete message and it will be stuck in transit.

It's worth noticing that this is already broken for scaled out brokered transports in version 5, but MSMQ + gateway is presumed to be the most frequent scenario.

Alternative solutions

1. Use a single HTTP call instead of one for the message and one per individual databus property

Benefits

• Very easy to implement and understand
• Nothing to correlate
• Fewer moving parts, the solution is pretty much to delete code.

Drawbacks

• Change from 1+N to 1 HTTP requests for a message with N databus properties. The rationale behind sending the databus properties in separate HTTP requests in the first place needs to be considered. Is it more reliable/efficient/failure tolerant to use 1+N HTTP requests instead of 1?
• Would not be backwards compatible as it can not handle version 5 messages with databus properties. No way to correlate across gateway instances.

2. Return the new databus ids from the receiver

Include a new header in the response for each of the individual databus HTTP requests. The sender aggregates them and included them in the final message HTTP request.

Benefits

• Keep the 1+N HTTP requests for a message with N databus properties.
• Moves the correlation from receiver to the sender.

Drawbacks

• Slightly more complicated implementation.
• More moving parts
• Would also not be backwards compatible as it can not handle version 5 messages with databus properties. No way to correlate across gateway instances.

3. Store receiver databus properties in the deduplication store

Here we can extend the deduplication store/gateway persistence to store the id of the databus properties. The gateway instances share it, so correlation works as before, just using the persisted data instead of the in-memory cache.

Benefits

• Keep the 1+N HTTP requests for a message with N databus properties.
• Fully backwards compatible. Version 6 gateway can send to version 5 gateway and vice versa.

Drawbacks

• The complicated implementation
• More moving parts
• The gateway persistence would need to double duty for both deduplication and correlation.

4. Store metadata about the incoming gateway message in the databus store

Pretty much same as 3, but use databus store instead of deduplication store.

Here we can extend the databus store to save the id of the incoming message. The gateway instances share it, so correlation works as before, just using the persisted data instead of the in-memory cache.

Benefits

• Keep the 1+N HTTP requests for a message with N databus properties.
• Fully backwards compatible. Version 6 gateway can send to version 5 gateway and vice versa.

5. Guidance

Document that the gateway is designed to run on single instance. When scaling out with the new client side solution customers should pick one of the instances to run the gatway.

Benefits

• Fully backwards compatible. Version 6 gateway can send to version 5 gateway and vice versa.
• No implementation needed.
• Less moving parts.

Drawbacks

• All gateway requests go to a single instance. Need to make that fault resistant by using a LB in front that can detect a failing instance switch to one of the other instances instead.

Chosen solution

5 - Guidance

Relates to https://github.com/Particular/PlatformDevelopment/issues/133
Task force is the https://github.com/Particular/PlatformDevelopment/issues/133 task force.

Raised by @SimonCropp
Migrated from by Particular/NServiceBus#1945

The tostring of this https://github.com/Particular/NServiceBus/blob/develop/src/NServiceBus.Core/Gateway/Channels/Channel.cs#L12

results in this being logged

2014-02-04 17:13:19,645 [12] INFO  NServiceBus.Gateway.Receiving.GatewayReceiver [(null)] <(null)> - Receive channel started: 
http,http://localhost:25894/subtractminor/NumberOfWorkerThreads=1Default=True

We should format that a little better