UnsupportedCodecError for snappy when producing events without snappy compression about aiokafka HOT 4 CLOSED

Does by any chance the topic contain existing messages, produced by an other producer ? The consumer auto reset to "earliest" so it will consume the topic from the beginning, and in Kafka, the broker sends messages as they are stored, as they want to delegate most of the CPU work to the clients. So you could face :

• you have/had a producer producing snappy messages to the topic
• the consumer fetch these messages, the metadata says "this message is a snappy one" and you don't have the appropriate library to decode it

from aiokafka.

Does by any chance the topic contain existing messages, produced by an other producer ?

No this is not possible, I am running this example on a brand new topic where my event in the script is the first and only one published.

from aiokafka.

Do you have access to the segment files on the broker ? If yes, you can check the compression with a kafka tools provided in the kafka binaries. Maybe there are 3rd party tools also offer that while consuming a topic.
In aiokafka, it seems the error you have should happen when reading messages with header compression saying "snappy" :

• it doesn't seem to come from your input topic (but I would rather double check it, maybe something you are not aware added messages over there)
• it maybe could have come from the __consumer_offsets topic, but you are not using any consumer group here

from aiokafka.

I found out that the topic I was creating was being created with compression.type: "snappy" without me providing that config to the API call. I am using a shared cluster, so I suspect that is the cluster's default in some way.

I re-creating my topic using compression.type: "producer" and my example above now works.

Thank you for your help, sorry it ended up being a dumb mistake!

from aiokafka.