[QUESTION] Getting "Unsupported value "snappy" for configuration property "compression.codec" while creating a consumer about laravel-kafka HOT 8 CLOSED

In the function below is hard-coded"snappy" instead of reading it from the config KAFKA_COMPRESSION_TYPE. This affects Windows environment as there is no snappy installed.

public function getProducerOptions(): array
    {
        $config = [
            'compression.codec' => 'snappy',
            'bootstrap.servers' => $this->broker,
            'metadata.broker.list' => $this->broker,
        ];

        return collect(array_merge($config, $this->customOptions, $this->getSaslOptions()))
            ->reject(fn ($option) => in_array($option, self::CONSUMER_ONLY_CONFIG_OPTIONS))
            ->toArray();
    }

from laravel-kafka.

Hey @grunk please check this confluentinc/librdkafka#2496 issue and let me know if it helps

from laravel-kafka.

Don't know why but I'm not able to reproduce this nor #105

from laravel-kafka.

Thanks for your answer.
I'm not sure that the build of librdkafka is at fault here as i manage to consume my message with php-rdkafka.

To be honest, i'm in a kind of special dev environment.
Apache/PHP on windows and Kafka running on an other windows machine with docker. Not really representative of production , maybe this is why i have this kind of error.

For now i'm using php-rdkafka directly so you can close this issue if you think it's something too specific.

from laravel-kafka.

As soon as I have time I'll try to setup a environment as close to what you mentioned here and then I can get back to you.

from laravel-kafka.

In the function below is hard-coded"snappy" instead of reading it from the config KAFKA_COMPRESSION_TYPE. This affects Windows environment as there is no snappy installed.

public function getProducerOptions(): array
    {
        $config = [
            'compression.codec' => 'snappy',
            'bootstrap.servers' => $this->broker,
            'metadata.broker.list' => $this->broker,
        ];

        return collect(array_merge($config, $this->customOptions, $this->getSaslOptions()))
            ->reject(fn ($option) => in_array($option, self::CONSUMER_ONLY_CONFIG_OPTIONS))
            ->toArray();
    }

Can you tell me file and path where i must insert this code?

from laravel-kafka.

In the function below is hard-coded"snappy" instead of reading it from the config KAFKA_COMPRESSION_TYPE. This affects Windows environment as there is no snappy installed.

public function getProducerOptions(): array
    {
        $config = [
            'compression.codec' => 'snappy',
            'bootstrap.servers' => $this->broker,
            'metadata.broker.list' => $this->broker,
        ];

        return collect(array_merge($config, $this->customOptions, $this->getSaslOptions()))
            ->reject(fn ($option) => in_array($option, self::CONSUMER_ONLY_CONFIG_OPTIONS))
            ->toArray();
    }

Can you tell me file and path where i must insert this code?

The path is vendor/mateusjunges/laravel-kafka/src/Config/Config.php. But you dont need to insert this code, what I meant is that this code is wrong because of the line
'compression.codec' => 'snappy',
This line should be something like 'compression.codec' => env(KAFKA_COMPRESSION_TYPE, 'snappy'),.
So on Windows we changed to 'compression.codec' => 'none', temporarily so that our developers can use the library.

from laravel-kafka.

Hey folks, I've just merged a PR to fix this. Now, instead of the hardcoded snappy compression type, this library uses the one defined in the compression key of the kafka.php configuration file. Please let me know if it really solves this issue. Released on v1.8.4

from laravel-kafka.