Support for message queues about dev-spaces HOT 6 CLOSED

Dev Spaces currently routes http calls only, but we have discussed adding support for additional protocols at a later date. Thank you for your feedback -- we will consider that when prioritizing future features.

from dev-spaces.

I'm using Azure Service Bus to provide event-based integration between microservices where each microservice has its own subscription on a single system-wide topic. This follows guidance obtained from the eshopOnContainers guidance project.

As @MortenChristiansen points out, whilst using Dev Spaces is great for propagating space context to downstream HTTP services, it doesn't currently help for these event-based scenarios.

I appreciate this use case comes with some difficulties, such as how to ensure a child space's service receives an event over the parent space. Then there's the issue of propagating the space context (maybe as event metadata?) such that it can be passed on to a downstream HTTP call for example.

@lisaguthrie you commented that support for additional protocols may be added. I was wondering whether this would then mean Dev Spaces would support these use cases, and if it's closer to becoming a reality?

from dev-spaces.

@markphillips100, it is an unfortunate limitation of dev spaces that communication via queues cannot be naturally routed correctly today. There is an inherent conflict in the routing logic, which works when making a request from one service to another, and a piece of code that is listening on receiving a message from a specific queue instance.

The only realistic way to make this work today is to run both the sender and the listener in the child space and give them both an isolated queue instance to use for communication. Given the increased demand for a generalized service mesh I would expect solutions for this to start cropping up, but it will take some time.

from dev-spaces.

Thanks @stepro for your insights.

On a previous project, when we needed to develop locally so as to be isolated from other devs, we would manually create a personalised service bus subscription (rather than a topic) and attach the event-driven microservice to this subscription. This didn't necessarily isolate the original (parent) microservice from receiving an event and handling it but it at least meant the child (local dev) service did always receive an event.

One thing I was considering here was to include some filtering in the subscriber infrastructure code that would infer whether to handle an event, or ignore it, from 2 pieces of information; the name of the subscription and some metadata or header data in the event itself. This assumes that an upstream producer of the event has added the necessary metadata header if it existed in its execution. The subscriber code would also need to add some execution context if it decided to handle the event so that event handlers could easily pass the routing data on to either further http calls or events to be published.

The above changes (to app code, not dev-spaces) don't offer the dev an automatic fallback to a parent service but it would add some control over routing of messages based upon a dev using a child front-end URI.

I'd be interested in yours or anyone's thoughts on this approach.

from dev-spaces.

@markphillips100, your idea seems reasonable, and is something we had thought about but couldn't implement in a general fashion as it requires specific app logic changes. We'd love to hear your experience trying this out and if it proves to work for you. One thing we're looking to learn is how far people are willing to go with app logic changes in order to keep the value of dev spaces, and this scenario is one of the key ones that we know will apply to a number of apps.

from dev-spaces.

@stepro I'll for sure give feedback when I have it.

It's likely to be a while as our new project hasn't decided yet if we'll just use a message broker alone or a framework like NServiceBus or MassTransit. I'm not sure yet but if we went for the latter frameworks, we'd need to decide whether the extra app code, if any, would need to be coupled to the framework semantics or just the underlying transports.

from dev-spaces.