Comments (7)
sclasen
commented on August 17, 2024
Hmm, perhaps there is an off by one error. What happens if you use
commitConfig = new CommitConfig(Some(10 seconds), Some(10), Timeout(10 second))
also, is there any process sending messages to kafka?
from akka-kafka.
sclasen
commented on August 17, 2024
also, what version are you using?
from akka-kafka.
richiesgr
commented on August 17, 2024
Thanks for super quick help !!
using 0.9 from maven repo
try your commitConfig don't help
here is my log with debug
[test-akka.actor.default-dispatcher-6] INFO com.sclasen.akka.kafka.ConnectorFSM - at=created-streams
2015-01-13 19:21:51 DEBUG ClientCnxn:839 - Reading reply sessionid:0x14ae3aafd0c0028, packet:: clientPath:null serverPath:null finished:false header:: 16,4 replyHeader:: 16,453,0 request:: '/brokers/ids/0,F response:: #7b226a6d785f706f7274223a2d312c2274696d657374616d70223a2231343231313538383536393333222c22686f7374223a223139322e3136382e35302e3731222c2276657273696f6e223a312c22706f7274223a393039327d,s{218,218,1421158856939,1421158856939,0,0,0,93137065843884032,90,0,218}
[test-akka.actor.default-dispatcher-6] INFO akka.actor.ActorSystemImpl - at=consumer-started
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Verifying properties
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property client.id is overridden to 1
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property metadata.broker.list is overridden to 192.168.50.71:9092
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property request.timeout.ms is overridden to 30000
2015-01-13 19:21:51 INFO ClientUtils$:68 - Fetching metadata from broker id:0,host:192.168.50.71,port:9092 with correlation id 0 for 1 topic(s) Set(HTTP)
2015-01-13 19:21:51 DEBUG BlockingChannel:52 - Created socket with SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 530904 (requested -1), SO_SNDBUF = 102400 (requested 102400).
2015-01-13 19:21:51 INFO SyncProducer:68 - Connected to 192.168.50.71:9092 for producing
2015-01-13 19:21:51 INFO SyncProducer:68 - Disconnecting from 192.168.50.71:9092
2015-01-13 19:21:51 DEBUG ClientUtils$:52 - Successfully fetched metadata for 1 topic(s) Set(HTTP)
2015-01-13 19:21:51 DEBUG ConsumerFetcherManager$LeaderFinderThread:52 - [1_richard-Latitude-E6440-1421169710915-2dd7b19d-leader-finder-thread], {TopicMetadata for topic HTTP ->
Metadata for partition [HTTP,0] is partition 0 leader: 0 (192.168.50.71:9092) replicas: 0 (192.168.50.71:9092) isr: 0 (192.168.50.71:9092) isUnderReplicated: false}
2015-01-13 19:21:51 INFO ConsumerFetcherThread:68 - [ConsumerFetcherThread-1_richard-Latitude-E6440-1421169710915-2dd7b19d-0-0], Starting
2015-01-13 19:21:51 DEBUG BlockingChannel:52 - Created socket with SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 65536 (requested 65536), SO_SNDBUF = 1313280 (requested -1).
2015-01-13 19:21:51 DEBUG PartitionTopicInfo:52 - reset fetch offset of ( HTTP:0: fetched offset = 20996: consumed offset = -1 ) to 20996
2015-01-13 19:21:51 DEBUG PartitionTopicInfo:52 - reset consume offset of HTTP:0: fetched offset = 20996: consumed offset = 20996 to 20996
2015-01-13 19:21:51 INFO ConsumerFetcherManager:68 - [ConsumerFetcherManager-1421169710957] Added fetcher for partitions ArrayBuffer([[HTTP,0], initOffset -1 to broker id:0,host:192.168.50.71,port:9092] )
2015-01-13 19:21:53 DEBUG ClientCnxn:759 - Got ping response for sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:55 DEBUG ClientCnxn:759 - Got ping response for sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:57 DEBUG ClientCnxn:759 - Got ping response for sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:59 DEBUG ClientCnxn:759 - Got ping response for sessionid: 0x14ae3aafd0c0028 after 0ms
2015-01-13 19:22:01 DEBUG PartitionTopicInfo:52 - updated fetch offset of (HTTP:0: fetched offset = 20997: consumed offset = 20996) to 20997
2015-01-13 19:22:01 DEBUG ClientCnxn:759 - Got ping response for sessionid: 0x14ae3aafd0c0028 after 0ms
[test-akka.actor.default-dispatcher-2] INFO com.sclasen.akka.kafka.ConnectorFSM - at=transition from=Receiving to=Committing uncommitted=0
[test-akka.actor.default-dispatcher-4] WARN com.sclasen.akka.kafka.ConnectorFSM - state=Committing msg=StateTimeout drained=1 streams=1
from akka-kafka.
sclasen
commented on August 17, 2024
OOh, sorry I think 0.0.9 is broken. .Can you try 0.0.10-SNAPSHOT from
oss.sonatype. I am publishing 0.0.10 right now!!!
On Tue, Jan 13, 2015 at 9:23 AM, richiesgr [email protected] wrote:
Thanks for super quick help !!
using 0.9 from maven repo
try your commitConfig don't helphere is my log with debug
[test-akka.actor.default-dispatcher-6] INFO
com.sclasen.akka.kafka.ConnectorFSM - at=created-streams
2015-01-13 19:21:51 DEBUG ClientCnxn:839 - Reading reply
sessionid:0x14ae3aafd0c0028, packet:: clientPath:null serverPath:null
finished:false header:: 16,4 replyHeader:: 16,453,0 request::
'/brokers/ids/0,F response::
#7b226a6d785f706f7274223a2d312c2274696d657374616d70223a2231343231313538383536393333222c22686f7374223a223139322e3136382e35302e3731222c2276657273696f6e223a312c22706f7274223a393039327d,s{218,218,1421158856939,1421158856939,0,0,0,93137065843884032,90,0,218}[test-akka.actor.default-dispatcher-6] INFO akka.actor.ActorSystemImpl -
at=consumer-started
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Verifying properties
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property client.id is
overridden to 1
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property
metadata.broker.list is overridden to 192.168.50.71:9092
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property
request.timeout.ms is overridden to 30000
2015-01-13 19:21:51 INFO ClientUtils$:68 - Fetching metadata from broker
id:0,host:192.168.50.71,port:9092 with correlation id 0 for 1 topic(s)
Set(HTTP)
2015-01-13 19:21:51 DEBUG BlockingChannel:52 - Created socket with
SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 530904 (requested -1),
SO_SNDBUF = 102400 (requested 102400).
2015-01-13 19:21:51 INFO SyncProducer:68 - Connected to 192.168.50.71:9092
for producing
2015-01-13 19:21:51 INFO SyncProducer:68 - Disconnecting from
192.168.50.71:9092
2015-01-13 19:21:51 DEBUG ClientUtils$:52 - Successfully fetched metadata
for 1 topic(s) Set(HTTP)
2015-01-13 19:21:51 DEBUG ConsumerFetcherManager$LeaderFinderThread:52 -
[1_richard-Latitude-E6440-1421169710915-2dd7b19d-leader-finder-thread],
{TopicMetadata for topic HTTP ->
Metadata for partition [HTTP,0] is partition 0 leader: 0 (
192.168.50.71:9092) replicas: 0 (192.168.50.71:9092) isr: 0 (
192.168.50.71:9092) isUnderReplicated: false}
2015-01-13 19:21:51 INFO ConsumerFetcherThread:68 -
[ConsumerFetcherThread-1_richard-Latitude-E6440-1421169710915-2dd7b19d-0-0],
Starting
2015-01-13 19:21:51 DEBUG BlockingChannel:52 - Created socket with
SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 65536 (requested 65536),
SO_SNDBUF = 1313280 (requested -1).
2015-01-13 19:21:51 DEBUG PartitionTopicInfo:52 - reset fetch offset of (
HTTP:0: fetched offset = 20996: consumed offset = -1 ) to 20996
2015-01-13 19:21:51 DEBUG PartitionTopicInfo:52 - reset consume offset of
HTTP:0: fetched offset = 20996: consumed offset = 20996 to 20996
2015-01-13 19:21:51 INFO ConsumerFetcherManager:68 -
[ConsumerFetcherManager-1421169710957] Added fetcher for partitions
ArrayBuffer([[HTTP,0], initOffset -1 to broker
id:0,host:192.168.50.71,port:9092] )
2015-01-13 19:21:53 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:55 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:57 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:59 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 0ms
2015-01-13 19:22:01 DEBUG PartitionTopicInfo:52 - updated fetch offset of
(HTTP:0: fetched offset = 20997: consumed offset = 20996) to 20997
2015-01-13 19:22:01 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 0ms
[test-akka.actor.default-dispatcher-2] INFO
com.sclasen.akka.kafka.ConnectorFSM - at=transition from=Receiving
to=Committing uncommitted=0
[test-akka.actor.default-dispatcher-4] WARN
com.sclasen.akka.kafka.ConnectorFSM - state=Committing msg=StateTimeout
drained=1 streams=1—
Reply to this email directly or view it on GitHub
#26 (comment).
from akka-kafka.
sclasen
commented on August 17, 2024
0.0.10 should be live in just a few minutes
On Tue, Jan 13, 2015 at 9:24 AM, Scott Clasen [email protected] wrote:
OOh, sorry I think 0.0.9 is broken. .Can you try 0.0.10-SNAPSHOT from
oss.sonatype. I am publishing 0.0.10 right now!!!On Tue, Jan 13, 2015 at 9:23 AM, richiesgr [email protected]
wrote:Thanks for super quick help !!
using 0.9 from maven repo
try your commitConfig don't helphere is my log with debug
[test-akka.actor.default-dispatcher-6] INFO
com.sclasen.akka.kafka.ConnectorFSM - at=created-streams
2015-01-13 19:21:51 DEBUG ClientCnxn:839 - Reading reply
sessionid:0x14ae3aafd0c0028, packet:: clientPath:null serverPath:null
finished:false header:: 16,4 replyHeader:: 16,453,0 request::
'/brokers/ids/0,F response::
#7b226a6d785f706f7274223a2d312c2274696d657374616d70223a2231343231313538383536393333222c22686f7374223a223139322e3136382e35302e3731222c2276657273696f6e223a312c22706f7274223a393039327d,s{218,218,1421158856939,1421158856939,0,0,0,93137065843884032,90,0,218}[test-akka.actor.default-dispatcher-6] INFO akka.actor.ActorSystemImpl -
at=consumer-started
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Verifying properties
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property client.id is
overridden to 1
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property
metadata.broker.list is overridden to 192.168.50.71:9092
2015-01-13 19:21:51 INFO VerifiableProperties:68 - Property
request.timeout.ms is overridden to 30000
2015-01-13 19:21:51 INFO ClientUtils$:68 - Fetching metadata from broker
id:0,host:192.168.50.71,port:9092 with correlation id 0 for 1 topic(s)
Set(HTTP)
2015-01-13 19:21:51 DEBUG BlockingChannel:52 - Created socket with
SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 530904 (requested -1),
SO_SNDBUF = 102400 (requested 102400).
2015-01-13 19:21:51 INFO SyncProducer:68 - Connected to
192.168.50.71:9092 for producing
2015-01-13 19:21:51 INFO SyncProducer:68 - Disconnecting from
192.168.50.71:9092
2015-01-13 19:21:51 DEBUG ClientUtils$:52 - Successfully fetched metadata
for 1 topic(s) Set(HTTP)
2015-01-13 19:21:51 DEBUG ConsumerFetcherManager$LeaderFinderThread:52 -
[1_richard-Latitude-E6440-1421169710915-2dd7b19d-leader-finder-thread],
{TopicMetadata for topic HTTP ->
Metadata for partition [HTTP,0] is partition 0 leader: 0 (
192.168.50.71:9092) replicas: 0 (192.168.50.71:9092) isr: 0 (
192.168.50.71:9092) isUnderReplicated: false}
2015-01-13 19:21:51 INFO ConsumerFetcherThread:68 -
[ConsumerFetcherThread-1_richard-Latitude-E6440-1421169710915-2dd7b19d-0-0],
Starting
2015-01-13 19:21:51 DEBUG BlockingChannel:52 - Created socket with
SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 65536 (requested 65536),
SO_SNDBUF = 1313280 (requested -1).
2015-01-13 19:21:51 DEBUG PartitionTopicInfo:52 - reset fetch offset of (
HTTP:0: fetched offset = 20996: consumed offset = -1 ) to 20996
2015-01-13 19:21:51 DEBUG PartitionTopicInfo:52 - reset consume offset of
HTTP:0: fetched offset = 20996: consumed offset = 20996 to 20996
2015-01-13 19:21:51 INFO ConsumerFetcherManager:68 -
[ConsumerFetcherManager-1421169710957] Added fetcher for partitions
ArrayBuffer([[HTTP,0], initOffset -1 to broker
id:0,host:192.168.50.71,port:9092] )
2015-01-13 19:21:53 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:55 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:57 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 1ms
2015-01-13 19:21:59 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 0ms
2015-01-13 19:22:01 DEBUG PartitionTopicInfo:52 - updated fetch offset of
(HTTP:0: fetched offset = 20997: consumed offset = 20996) to 20997
2015-01-13 19:22:01 DEBUG ClientCnxn:759 - Got ping response for
sessionid: 0x14ae3aafd0c0028 after 0ms
[test-akka.actor.default-dispatcher-2] INFO
com.sclasen.akka.kafka.ConnectorFSM - at=transition from=Receiving
to=Committing uncommitted=0
[test-akka.actor.default-dispatcher-4] WARN
com.sclasen.akka.kafka.ConnectorFSM - state=Committing msg=StateTimeout
drained=1 streams=1—
Reply to this email directly or view it on GitHub
#26 (comment).
from akka-kafka.
richiesgr
commented on August 17, 2024
Thanks I give it a try
from akka-kafka.
richiesgr
commented on August 17, 2024
Thanks a lot for your help working now on 0.0.10
from akka-kafka.
Related Issues (20)
- Any reason for losing the key HOT 2
- StateTimeout after 1 message is processed HOT 7
- Consumer with a Router HOT 5
- Question : How can I consume my kafka topic as fast as possible ? HOT 22
- No license explicitly stated HOT 2
- Race / off-by-one issue in BatchFSM? (Stuck temporarily in WaitingToSendBatch) HOT 3
- Empty Queue HOT 1
- Unable to cleanly stop AkkaConsumer HOT 1
- Batch consumer priority HOT 2
- ConsumerTimeoutException HOT 5
- Guidance Requested: Error Handling HOT 2
- Can AkkaConsumber inherit from a common base trait? HOT 2
- New Kafka .9 consumer HOT 3
- Multiple workers actors HOT 1
- `unhandled event StartProcessing` Warning
- Specifying multiple topic HOT 1
- Version in sonatype.org HOT 3
- Question, how to set SupervisorStrategy for StreamFSM?
- NPE in BatchConnectorFSM
- What exactly is outstanding? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from akka-kafka.