Comments (10)
RabbitMQ will block publishers that overrun consumers. In theory it should not cause any socket exceptions but presumably it does. Network error handling is very simplistic right now, it is true.
from bunny.
Can you count after about what # of messages published you get this behavior? And post what's in the RabbitMQ log? Thanks!
from bunny.
Also, what's approximate median message size? So far I cannot reproduce this with small messages (1K) and 100,000
messages published.
from bunny.
It happens at about 270K messages. Some other details about my environment:
- I'm writing to a topic exchange.
- There is a durable queue that gets the messages from the exchange.
The message size doesn't really seems to matter. It happens even with 4B messages.
I believe the queue getting messages from the exchange is relevant.
These are the last lines in my Rabbit logs:
=INFO REPORT==== 15-Feb-2013::10:30:55 ===
closing AMQP connection <0.24366.1> (127.0.0.1:64089 -> 127.0.0.1:5672)
=INFO REPORT==== 15-Feb-2013::10:30:56 ===
accepting AMQP connection <0.24407.1> (127.0.0.1:64092 -> 127.0.0.1:5672)
=ERROR REPORT==== 15-Feb-2013::10:30:56 ===
closing AMQP connection <0.24407.1> (127.0.0.1:64092 -> 127.0.0.1:5672):
{bad_header,<<1,0,2,0,0,0,25,65>>}
from bunny.
That means RabbitMQ somehow gets an empty frame header, considers it a fatal error and closes the connection.
Do you have an example script that reproduces the issue? What RabbitMQ version do you run?
from bunny.
Yes, the script I posted in the first message reproduces the issue when there is a durable queue subscribed to the exchange.
I'm running 3.0.1
from bunny.
Can you check that RabbitMQ log does not have anything related to alarms when this happens?
I cannot reproduce this. My theory is that when your machine goes low on memory or disk space (< 1 GB), RabbitMQ blocks the publisher, eventually OS socket buffer gets full and this results in exceptions
that clear @socket
in the transport. And because the publisher is not stopped, you see this nil pointer
exception.
from bunny.
Setting RabbitMQ memory watermark to a really low value suggests it is not the issue:
sudo rabbitmqctl set_vm_memory_high_watermark 0.05
will cause the publisher quickly get blocked but it does not cause the script to fail with any exceptions.
from bunny.
I can't reproduce it across 2 OSes, 2 RabbitMQ versions, 2 Ruby implementations. We have a test for a very
similar scenario, unless an example that reproduces it can be found, I consider this issue resolved.
from bunny.
0.9.0.pre8
is out, please give it a try.
from bunny.
Related Issues (20)
- losing published messages HOT 2
- Gem includes test certificates HOT 3
- Cannot subscribe to existing queue with no configure permission HOT 2
- Redeliver publisher confirms acks again and again from 0 till current tag HOT 4
- Binding a queue to same exchange twice but using different routing keys not working as expected HOT 1
- any plans to support the new rabbitMQ stream plugin with RabbitMQ 3.9? HOT 4
- Queue subscribe: Calling thread is no longer blocked after connection failure HOT 2
- TLS 1.3 support HOT 2
- QueueDeclare / Timeout issue
- Bunny does not recover from Rabbitmq Broker restart but reconnects on manual restart of service
- Confusion of parameters
- NameError: uninitialized constant OpenSSL::SSL::TLS1_3_VERSION (bunny-2.20.0, ruby 2.6) HOT 4
- Bunny::Channel#quorum_queue does not result in a QQ declaration
- Conditionally alias constants for TLSv1.3 to support older OpenSSL releases
- OpenSSL error is causing unrecoverable issue HOT 1
- Bunny::Channel#default_exchange returns a new object each time
- Missing client notification when recovery fails (after specified recovery attempts) HOT 1
- Reader Loop: undefined method handle_frameset for nil:NilClass
- Heartbeat sender uses Time.now which is unreliable HOT 6
- Channel callback from delivery acknowledgement timeout.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bunny.