Comments (8)
In the function below is hard-coded"snappy" instead of reading it from the config KAFKA_COMPRESSION_TYPE. This affects Windows environment as there is no snappy installed.
public function getProducerOptions(): array
{
$config = [
'compression.codec' => 'snappy',
'bootstrap.servers' => $this->broker,
'metadata.broker.list' => $this->broker,
];
return collect(array_merge($config, $this->customOptions, $this->getSaslOptions()))
->reject(fn ($option) => in_array($option, self::CONSUMER_ONLY_CONFIG_OPTIONS))
->toArray();
}
from laravel-kafka.
Hey @grunk please check this confluentinc/librdkafka#2496 issue and let me know if it helps
from laravel-kafka.
Don't know why but I'm not able to reproduce this nor #105
from laravel-kafka.
Thanks for your answer.
I'm not sure that the build of librdkafka is at fault here as i manage to consume my message with php-rdkafka.
To be honest, i'm in a kind of special dev environment.
Apache/PHP on windows and Kafka running on an other windows machine with docker. Not really representative of production , maybe this is why i have this kind of error.
For now i'm using php-rdkafka directly so you can close this issue if you think it's something too specific.
from laravel-kafka.
As soon as I have time I'll try to setup a environment as close to what you mentioned here and then I can get back to you.
from laravel-kafka.
In the function below is hard-coded"snappy" instead of reading it from the config KAFKA_COMPRESSION_TYPE. This affects Windows environment as there is no snappy installed.
public function getProducerOptions(): array { $config = [ 'compression.codec' => 'snappy', 'bootstrap.servers' => $this->broker, 'metadata.broker.list' => $this->broker, ]; return collect(array_merge($config, $this->customOptions, $this->getSaslOptions())) ->reject(fn ($option) => in_array($option, self::CONSUMER_ONLY_CONFIG_OPTIONS)) ->toArray(); }
Can you tell me file and path where i must insert this code?
from laravel-kafka.
In the function below is hard-coded"snappy" instead of reading it from the config KAFKA_COMPRESSION_TYPE. This affects Windows environment as there is no snappy installed.
public function getProducerOptions(): array { $config = [ 'compression.codec' => 'snappy', 'bootstrap.servers' => $this->broker, 'metadata.broker.list' => $this->broker, ]; return collect(array_merge($config, $this->customOptions, $this->getSaslOptions())) ->reject(fn ($option) => in_array($option, self::CONSUMER_ONLY_CONFIG_OPTIONS)) ->toArray(); }
Can you tell me file and path where i must insert this code?
The path is vendor/mateusjunges/laravel-kafka/src/Config/Config.php. But you dont need to insert this code, what I meant is that this code is wrong because of the line
'compression.codec' => 'snappy',
This line should be something like 'compression.codec' => env(KAFKA_COMPRESSION_TYPE, 'snappy'),
.
So on Windows we changed to 'compression.codec' => 'none',
temporarily so that our developers can use the library.
from laravel-kafka.
Hey folks, I've just merged a PR to fix this. Now, instead of the hardcoded snappy
compression type, this library uses the one defined in the compression
key of the kafka.php
configuration file. Please let me know if it really solves this issue. Released on v1.8.4
from laravel-kafka.
Related Issues (20)
- Failed to resolve `broker:9092` HOT 4
- Getting a timeout issue, while trying to publish a message. HOT 7
- Undefined constant \"Junges\\Kafka\\RD_KAFKA_PARTITION_UA\ HOT 3
- Upstash Support - Producer not sending messages. HOT 3
- Wrong return type in Junges\Kafka\Config\Config->getCommit HOT 2
- Cannot Consume published messages in laravel 11 and php 8.3.6 HOT 15
- Issue consuming records from Avro topic HOT 1
- Cannot consume produced messages in v2.0 HOT 2
- What do we need to set for the `--consumer` flag. HOT 1
- Message consuming is still an issue in v2.1 HOT 10
- Encoded payloads: Is it possible with Avro Serializer to Save encoded messages in Kafka when sending it using AvroSerialize and doing the opposite when reading with Deserializer HOT 1
- Is there any support to Retry policies ? HOT 2
- [QUESTION] Possibility to describe topic / calculate read backlog HOT 1
- [Question/bug] Consuming Tombstones and send it to dlq throws $payload must be of type string, null given HOT 1
- cant send message HOT 8
- Error "Specified group generation id is not valid" during consumption HOT 3
- How to commit each message in withHandler HOT 1
- Missing messages HOT 2
- Customizable flush retried and timeout
- stopAfterLastMessage not working as expected. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from laravel-kafka.