Custom Inputs/Outputs about connect HOT 6 CLOSED

Hey @cludden, if the input/output targets are generally useful then I'd be interested in adding them to the project, otherwise you can fork and I'll still be happy to guide you. The process is pretty simple, check out the examples in https://github.com/Jeffail/benthos/tree/master/lib/input/reader.

I'm trying to avoid plugins for now as I like the simplicity of having "one true Benthos", but obviously if the list of components gets outrageously large I'll revisit, not sure how I'd manage builds in that case but the technical side shouldn't be an issue.

from connect.

Marking this as documentation as it would be nice for me to outline the current process for contributing components.

from connect.

Hey @Jeffail thanks for the quick response, what you've stated was the same as my initial thoughts, and I have no problem submitting PR's for my use cases. However, the particular input I would like to implement (mysql binlog consumer) requires an additional storage layer for persisting consumer state (files and offsets), which can also be pluggable. As opposed to creating a truly generic mysql input (as in, bring-your-own backend), it seemed it would be simpler if I could just define my own structure that satisfied the benthos input interface and be on my way, plus i wasn't sure how useful it would be as part of the benthos core. Would you prefer this as a core input (i'll gladly submit a PR), or would a fork be the better solution?

from connect.

Let's do a fork for now. There's similar patterns I'm currently looking at where we want to have a consumer and then a separate storage layer so maybe in the future there will be a general pattern we could fit something like this into.

from connect.

sounds good!

from connect.

i'll leave it open for the documentation tag you added, thanks again!

from connect.