conduitio-labs / conduit-connector-nats-jetstream Goto Github PK

Feature description

here is a link back to the discussion: ConduitIO/conduit#981 where all the current info is.

Bug description

After the NATS system recreates a connection, the connector ceases to function. This issue has been observed consistently following each instance of NATS connection recreation. Consequently, data transmission and communication processes stop working, leading to major operational setbacks.

Steps to reproduce

1. Start the connector and NATS server.
2. Create a pipeline file with NATS Jetstream as the source and log as the destination.
3. Use NATS benchmark to create some load: nats bench test --js --pub 1 --size 16 --msgs 10
4. Run conduit and observe logs of messages received working.
5. Stop NATS server
6. Stop NATS benchmark
7. Start NATS server
8. Start NATS benchmark
9. Observe logs of messages received NOT working.

Version

0.3.1

Bug description

After the NATS system recreates a connection, the destination connector ceases to function by not being able to write to the new nats connection.

Steps to reproduce

• Start the connector and NATS server.
• Create a pipeline file with NATS Jetstream as the source and destination
• Run nats bench test --js --storage=memory --pub 1 --size 32 --msgs=200000 to send msgs to NATS
• Run conduit and observe by running nats bench test --js --sub=1 --pull
• Stop NATS server
• Stop NATS benchmark
• Start NATS server
• Send more messages
• Start NATS benchmark
• Observe logs of messages that stop reaching the destination.

Version

v0.3.1