kube-logging / roadmap Goto Github PK

Namespace level isolation:

• add routing logic to fluentbit so that it can map multiple namespaces to a single aggregator with a logging ref

Node level isolation already works with:

• multiple FluentbitAgents sending to separate logging refs
• multiple FluentbitAgents sending to a single logging ref

Resource size limitations

Multi-agent:

• heterogen clusters for example with different os (windows) or different arch (amd vs arm)

Multi tenancy:

• separate RBAC
• separate agents on different nodes sending to a centralized or separate logging ref

Supporting rolling upgrade of nodes with different config

Combination of performance and flexibility: utilize a low consumption agent with the routing flexibility of fluentd

This item is only about introducing syslog-ng as a node agent with the same capabilities and the ability to send to a syslog-ng or a fluentd aggregator.

Add flexible routing and filtering capabilities to the agent so that it can send logs not just to an aggregator, but also to supported destinations directly

Depends on #1

• Guides for the most common use cases

Stretch

• CRD documentation revamp

Vector (vector.dev) has some benefits (performance, VRL language) that could bring value to a logging infrastructure managed by the logging operator.

Things to consider:

• is there any existing operator that the logging operator could integrate with to avoid reinventing the wheel?
• is there someone (company or individual) who would pick up the task of adding support and maintaining it?
• do we want/need a major refactor to be able to add another provider to the aggregation layer? (most probably yes)

• Add unit and e2e tests for syslog-ng
• Test coverage for configcheck
• OCI Helm chart release and image signing (in progress)
• Automated continuous releases of the latest version

Stretch

• Helm library chart or reusable examples for common scenarios
• Create efficient reusable log consumer and producer apps for e2e tests

Aggregator (per namespace) for

• hard isolation of log configuration, resources and data
• dedicated resource to avoid noisy neighbours
• reduce configuration size, complexity and conflicts
• limit the blast radius of configuration errors
• more granular scalability

Note: this might not requires additional work on top of #3