Channel Connection Pool about aioamqp HOT 8 CLOSED

Hello @kracekumar,

Since each channel is designed to one purpose, and is linked to a queue, an exchange … and is not as low as creating a huge resource (like a thread, or a new tcp connection) what is the benefit of a pool ?

A new channel is created on all call to channel():
https://github.com/Polyconseil/aioamqp/blob/master/examples/publisher.py#L20

from aioamqp.

Hi @dzen

Other implementations like kombu and Bunny have same concept.

I haven't benchmarked myself with and without pool, it was general suggestion since amqp request is sent when a channel is created every time.

from aioamqp.

Frankly I don't see the problem.

If some messages take too much time to get processed, wrap it in asyncio.ensure_future(). Or delegate them to a subprocess. Or, if large messages are the problem you're trying to solve, create a new aioamqp instance for exchanging them.

from aioamqp.

@kracekumar what is your use case ? are you using asyncio in a webframework ? and then you're creating a new amqp channel on each incoming request ? If so I'm not sure if it's the right way to do it.

from aioamqp.

@dzen I am yet to start the new websocket project. I am benchmarking/reading between asyncio (aiohttp) and gevent (flask/uwsgi). Every incoming message will be enqueued to rabbitmq, so I am contemplating between opening channel for every message or fetch the channel from connection pool like SQLAlchemy. For 100 concurrent connection opening new channel vs fetching from connection pool (queue).

If this doesn't make sense, please close the issue.

from aioamqp.

@kracekumar This may be a stupid question, but why would you need a separate channel per message in the first place? simply put the messages onto a queue (along with a future, to signal success to the http task) and start a worker task feed the data to rabbitmq. if that's not fast enough (it should be), start more workers to process the queue, each with its own rabbitmq connection, no pooling needed.

from aioamqp.

@smurfix That is an alternate architecture, by queue you meant Python asyncio queue. Thanks for that.

from aioamqp.

@kracekumar I'm closing this issue: imho I'll create one channel per queue / exchange and try to keep them open.

from aioamqp.