Custom delivery options about nameko HOT 6 OPEN

Completely agree that this should be possible. We are generally inconsistent about exposing delivery and other messaging options and to callers and there have been a number of issues related to it.

I like Option B, because it allows us to keep the simple invocation in the normal case. If you really needed per-call options you could do it with a locking context-manager around the invocation.

Are there other AMQP options/properties we should expose here as well?

On a related note we have AMQP publishing code scattered all over the codebase, and it's all pretty similar. The following all wrap a kombu publisher and pass slightly different options to it:

• nameko.standalone.event_dispatcher
• nameko.messaging.Publisher.get_dependency
• nameko.rpc.Responder.send_response
• nameko.rpc.MethodProxy._call

It feels like we should have a nameko.amqp module that consolidates all this repeated code.

from nameko.

n.rpc.greeting_service.set_delivery_options is problematic because we can't distinguish it from an rpc call to set_delivery_options

is the main use case the standalone proxy or injected rpc proxies? i wonder if a better option is to make it easy to subclass the extension and set these options there

from nameko.

Subclassing is nice but tricky to do in this case because MethodProxy is quite buried. You'd have to subclass ServiceProxy as well, and you'd have to do it for each permutation of options you wanted.

Since this is RPC we can probably get away with using a property. Something called delivery_options is unlikely to be a remote method.

from nameko.

hm. elsewhere we work pretty hard to not infringe on "user namespaces", so would prefer to avoid. there might be workarounds (e.g off the top of my mind set_delivery_options(rpc_proxy, options), though with some thought we might be able to come up with something nicer)

i didn't mean to just say "subclass and override yourself", but rather for us to see if we can tweak/reorg the classes and their interactions to make it fairly straightforward to create a subclass that changes rabbit options

from nameko.

I've got another proof of concept ready where all the delivery options are provided when instantiating the service proxy. I think I found all the places where those options could be applied and the only real trouble is ensuring that delivery options of responses follow those of requests.

In addition to that I extended @rpc decorator to allow providing delivery options for responses specifically. The need for us was to allow defining compression on message for some methods only - the ones that return large lists of entries. So it made sense that the service decides whether to apply compression or not.

I will push both later today (will still be WIP). Unit tests are pain though, cause requests cannot be sniffed as easily as response messages (nameko tests suite only supports sniffing responses).

In regard to other AMQP options, there are more of course, but those 3 I mentioned cover the majority of use cases and they are the ones fully supported (virtually or natively) by other kombu transports (should you want to support them later).

from nameko.

So my new proposal is as follows. When creating ServiceRpcProxy or calling on dependency provider to get such proxy, developer can provide delivery_options attribute in order to override kombu's defaults whenever interacting with a target service.

If, for some reason, different set of attributes is required for specific calls, developer can just create another proxy with different configuration.

Ideally, whatever parameters are being applied to the request, should also be applied to the response message.

In some cases, we'd like to be able to override those options for all responses from a particular method. For example one that returns large chunks of data (compression).

Sample usage:

large_msg_opt = {'compression': 'gzip', 'delivery_mode': 'transient'}
fast_msg_opt = {'delivery_mode': 'transient', 'timeout': 5}


class Service(object):
    name = "service"

    @rpc
    def echo(self, arg):
        return arg

    @rpc(delivery_options=large_msg_opt)
    def echo_compressed(self, arg):
        return arg


with ServiceRpcProxy('service', rabbit_config, delivery_options=fast_msg_opt) as rpc_proxy:
    rpc_proxy.echo(test_data)
    rpc_proxy.echo_compressed(test_data)
    rpc_proxy.echo.async(test_data)
    rpc_proxy.echo_compressed.async(test_data)

Initial development, far from finished, but you may get some idea what I'm trying to achieve:
master...astawiarski:custom-delivery

from nameko.