Config template "processors" section shadows custom ECS processors about o365beat HOT 3 CLOSED

The concatenation of o365beat's _meta/beat.yml and the libbeat _meta/config.yml.tmpl happens in beats/dev-tools/mage/config.go. There doesn't appear to be any configuration that'll suppress the inclusion of that second processors section, as the merge is just a simple file concat and the .tmpl file conditionals don't have any option for leaving it out entirely.

We'll have to consider removing it later in the build, which will be tricky because we won't have any control of future changes to that file. Maybe there's a more robust way to pre-process the config file, merge duplicate sections, and re-write it. Research ongoing.

from o365beat.

Opened a discussion at https://discuss.elastic.co/t/config-template-processors-section-shadows-custom-processors-in-custom-beats/215578, the first step to an issue or pull request to correct the underlying problem.

from o365beat.

Fixed this by re-writing the concatenated config files in the build magefile (see 0074704). It's not the "right answer," but it works and shouldn't be too brittle so long as we keep an eye on any updates to libbeat (the upstream issue remains, so any custom beat that wants to add a processors section will have to do something similar - either ask their users to fix/merge the configs, or rewrite during the build process like we did here).

from o365beat.