Proposal: Support for protobuf in http_server about connect HOT 4 CLOSED

Hey @waldauf, would the intention be to use Benthos to proxy the protobuf payloads to some destination or would you expect to be able to parse the payloads and process them?

from connect.

@Jeffail In my case I would lie to use Benthos like stream processor between Prometheus and Kafka:

[ Prometheus : remote_write to InfluxDB ] ---> [ http_server : Benthos : kafka ] ---> [ Kafka ] ---> [ Telegraf ] ---> [ InfluxDB ]

In this flow I'm not able to write time series metrics produced by Prometheus into InfluxDB. I checked Prometheus and all metrics are serialized by protobuf and then sent to InfluxDB.

I'm thinking during writing this message and I have additional question: In this configuration do Benthos some changes on received packages from Prometheus before it writes them into Kafka?

from connect.

It's my understanding that in order to do any kind of processing on a protobuf message the service would need to be compiled with the specific .proto generated definitions of the types being received. I think this might be too specific to add to the general project, but you could always fork and hard code your own HTTP input with the ability to parse the prometheus metrics into any format you like.

from connect.

Yep, that is what I thought. This is to specific request. Thx for your answer.

from connect.