Comments (11)
Hi, Could you give more description about your issue?
I have tried to set the above interceptor, it's working in my local.
from mop.
KoP needs to be added after version 2.80
brokerentrymetadatainterceptors = org. Apache. Pulsar. Common. Intercept. Appendindexmetadatainterceptor
KoP works normally, but when I add MoP, the mqtt client can publish messages normally, but cannot subscribe to messages
ERROR:
java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6
at org.apache.pulsar.common.api.proto.LightProtoCodec.skipUnknownField(LightProtoCodec.java:270) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0]
at org.apache.pulsar.common.api.proto.MessageMetadata.parseFrom(MessageMetadata.java:1370) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0]
at org.apache.pulsar.common.protocol.Commands.parseMessageMetadata(Commands.java:426) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0]
at org.apache.pulsar.common.protocol.Commands.parseMessageMetadata(Commands.java:416) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0]
at io.streamnative.pulsar.handlers.mqtt.utils.PulsarMessageConverter.toMqttMessages(PulsarMessageConverter.java:71) ~[?:?]
at io.streamnative.pulsar.handlers.mqtt.support.MQTTConsumer.sendMessages(MQTTConsumer.java:85) ~[?:?]
at io.streamnative.pulsar.handlers.mqtt.support.MQTTConsumer.sendMessages(MQTTConsumer.java:41) ~[?:?]
at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.sendMessagesToConsumers(PersistentDispatcherMultipleConsumers.java:538) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readEntriesComplete(PersistentDispatcherMultipleConsumers.java:469) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.bookkeeper.mledger.impl.OpReadEntry.lambda$checkReadCompletion$2(OpReadEntry.java:156) ~[org.apache.pulsar-managed-ledger-2.8.0.jar:2.8.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[org.apache.pulsar-managed-ledger-2.8.0.jar:2.8.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.1.jar:4.14.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.63.Final.jar:4.1.63.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
from mop.
I think you can provide more info for this, some like your topic, qos, etc..
And you are use mqtt consumer and mqtt producer, right?
mqtt producer --> mqtt connsumer
from mop.
When I use the above configuration, remove it
brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor
MQTT clients can publish and subscribe to messages
but, Kafka cannot publish and subscribe messages .
If I modify the configuration as above add it
brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor
Kafka client can publish and subscribe messages.
but the mqtt client can publish messages normally, cannot subscribe to messages
View the broker log with the following error message:
java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6
at org.apache.pulsar.common.api.proto.LightProtoCodec.skipUnknownField(LightProtoCodec.java:270) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0]
at org.apache.pulsar.common.api.proto.MessageMetadata.parseFrom(MessageMetadata.java:1370) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0]
at org.apache.pulsar.common.protocol.Commands.parseMessageMetadata(Commands.java:426) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0]
at org.apache.pulsar.common.protocol.Commands.parseMessageMetadata(Commands.java:416) ~[org.apache.pulsar-pulsar-common-2.8.0.jar:2.8.0]
at io.streamnative.pulsar.handlers.mqtt.utils.PulsarMessageConverter.toMqttMessages(PulsarMessageConverter.java:71) ~[?:?]
at io.streamnative.pulsar.handlers.mqtt.support.MQTTConsumer.sendMessages(MQTTConsumer.java:85) ~[?:?]
at io.streamnative.pulsar.handlers.mqtt.support.MQTTConsumer.sendMessages(MQTTConsumer.java:41) ~[?:?]
at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.sendMessagesToConsumers(PersistentDispatcherMultipleConsumers.java:538) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readEntriesComplete(PersistentDispatcherMultipleConsumers.java:469) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
at org.apache.bookkeeper.mledger.impl.OpReadEntry.lambda$checkReadCompletion$2(OpReadEntry.java:156) ~[org.apache.pulsar-managed-ledger-2.8.0.jar:2.8.0]
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[org.apache.pulsar-managed-ledger-2.8.0.jar:2.8.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.14.1.jar:4.14.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.63.Final.jar:4.1.63.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
from mop.
The above is the detailed configuration and information.
I'm using a local build nar.
commit id is f6602d8
@Technoboy-
from mop.
I tried to update the latest version of the mop code and found that the MQTT client could not subscribe to the messages with or without the addition of brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor
to the configuration file
from mop.
@fanyouyong0526 Which Pulsar version are you using, please make sure the Pulsar version is the same as the mop plugin version.
from mop.
@Technoboy- Looks we need to add an integration test in the streamnative-tests to make sure MoP can work with KoP together.
from mop.
pulsar version:2.8.0
mop:A locally built version,commitid is f6602d8
from mop.
Could you try 2.8.1? It's because Commands#parseMessageMetadata
didn't skip the BrokerEntryMetadata in 2.8.0, but apache/pulsar#10968 added the skipBrokerEntryMetadataIfExist
call in parseMessageMetadata
. After that, parseMessageMetadata
should work.
from mop.
Please upgrade pulsar to 2.8.1.
from mop.
Related Issues (20)
- Can not support shared subscription
- Connection is null HOT 1
- Flaky-test: SimpleIntegrationTest.testConsumerDisconnectNotMissingMessage
- Is this still maintained? HOT 7
- MoP proxy does not function properly when running without TLS HOT 1
- MoP Proxy, keep alive is not managed correctly causes connection to fail HOT 13
- MoP Proxy seems redundant, introduces unnecessary complexity HOT 2
- java.nio.file.NoSuchFileException: /tmp/pulsar-nar/pulsar-protocol-handler-mqtt-2.8.1.23.nar-unpacked/UdB3VJeg7mGXymmwakz1bw/META-INF/services/pulsar-protocol-handler.yml HOT 1
- Cannot push/receive Mosquitto Bridge messages on MOP
- The mqtt plug-in shows that the loading is successful, but ports 5682 and 1833 still fail to listen. HOT 2
- [WIP] MIP-? Introduce new implementation for MoP
- Please redo the lookup. Request is denied: namespace=public/default HOT 2
- OPENSSL_internal:WRONG_VERSION_NUMBER excpetion when using MOP TLS Proxy HOT 1
- MQTT retain publishes do not make it to Pulsar HOT 3
- Readme.md states "only MQTT 3.1.1 supported" but code base seems to indicate MQTT 5 is also supported HOT 1
- LWT does not reach Pulsar topic, short cuts QoSPublishHandlers so Qos nor Retain function properly HOT 2
- Resubscribing on previously unsubscribed topic closes connection HOT 1
- LWT fires on graceful disconnect but not on connection loss HOT 1
- MQTT QOS support level? HOT 1
- Unsubscribed topic, repeatedly kicked out using client connection only. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mop.