Comments (5)
You can follow issue #67
from kafkajs.
Hi @al66, you can call disconnect
, it will wait for your consumers to finish the current set of messages and then disconnect. You don't need pause
for that, example:
const consumers = [...]
await Promise.all(consumers)
console.log('disconnected')
from kafkajs.
Hi @tulios . That was my first try.
But with the same result.
I think the reason is, that the cluster is also disconnected in the first call.
Would it possible to add a method "stop", which doesn't disconnect the cluster directly?
/**
* @return {Promise}
*/
const disconnect = async () => {
try {
if (runner) {
await runner.stop()
logger.debug('consumer has stopped, disconnecting', { groupId })
}
await cluster.disconnect()
} catch (e) {}
logger.info('Stopped', { groupId })
}
Here is my original coding:
/**
* Service stopped lifecycle event handler
*/
async stopped() {
await Promise.all(this.subscriptions.map(async (subscription) => {
try {
await subscription.consumer.disconnect();
this.logger.info(`Consumer for subscription ${subscription.id} stopped`);
} catch(err) {
this.logger.warn(`Stopping consumer for subscription ${subscription.id} failed`);
}
Promise.resolve();
}));
this.logger.info(`All consumers disconnected`);
}
from kafkajs.
You are right, I think we usually subscribe to different topics instead of creating several consumers. I'll create an issue to add the stop
method. In the meantime you can create a new client for each consumer (new Kafka(...)
)
from kafkajs.
This wasn't necessary, please check issue #67 (just to document)
from kafkajs.
Related Issues (20)
- 消费者端无法连接到kafka,连接地址被固定 HOT 10
- Provide function on producer that returns partition for message key of topic
- KafkaJSRequestTimeoutError during consumption leading to partition backlog
- more than one runner with one consumer.
- Kafka Api Versions missing
- Consumer receives eachBatch over the maxBytes HOT 1
- Treat throttling errors as retriable
- Re Consuming KafkaJs
- Section anchor tags on right sidebar not working HOT 1
- While using eachBatch(), the consumer isn't processing messages from each batch and just calling multiple batches
- TooManyRequestsError doc questions (doubts) HOT 2
- Introduce Kafkajs startup project prompt Invalid or unexpected token
- Check pending requests runs in an infinite loop
- Filter Messages by Header Before Deserialization
- Enhance the instrumentations events to support metrics collection
- What is max size of Message[]? HOT 3
- Issue with Message Processing or consumption in few Partitions
- Contribution Proposal: SASL_PLAINTEXT GSSAPI Authentication Support
- KafkaJSConnectionClosedError: Closed connection
- Crash: KafkaJSNonRetriableError: Specified group generation id is not valid
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kafkajs.