KubeStateWatch functions as a surveillance system for Kubernetes. Tracking different resources for any changes and letting users know exactly what has been changed.
It can be used standalone or deployed in Kubernetes. But its main purpose is to be deployed in Kubernetes.
KubeStateWatch is an extended and simplified version of kubewatch to meet the needs of our team ####Whats the difference between kubewatch and KubeStateWatch? It has been extended to support more the one connector, better support on multiple namespaces, visiblity to what was changed,simplified configuration, extended resource specific configura, added metrics and few other small stuff.
##UseCase Imagine you're managing a large Kubernetes cluster that has many different areas (namespaces) used by various people or teams. You need a way to keep an eye on any changes that happen in these areas that were made without the use of CI/CD pipelines ( for example using kubectl, lens, k9s etc.). In such cases you want to get notified about such changes,you also want to see what exactly was changed. This is what KubeStateWatch is for.
There are basically two kind of notifications:
- Notifications for Updated Items: The core purpose here is to focus on tracking meaningful changes to the items under our watch, while disregarding minor alterations like metadata or status updates. Rather than just receiving a basic message that something has changed, we aim to gain precise insight into what specifically was altered and the timing of these changes.
- Notifications for Added/Deleted Items: The foundational concept of kubewatch was to monitor and report on items that are newly added or removed.
Although this aspect is important, our primary focus is on the first scenario: tracking modifications to the items we are monitoring, such as deployments, replica sets (rs), horizontal pod autoscalers (hpa), and configmaps. We aim to be promptly informed about any and all changes occurring within these elements.
- Added support for configuration per resource (breaking change in the configuration file)
This gives you the ability to configure each resource separately. For example, you can configure monitoring only the UPDATES on deployments, while monitoring ADD and DELETE on replica sets and DELETE on pods and etc.Also, now you can ignore specific paths in the diff per resource. Which gives you more control over what you want to monitor.
#OLD
resourcesToWatch:
configmap: true
#NEW
configmap:
enabled: true
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
- Added support for muting notifications during deployment.
This feature allows you to mute notifications during deployment. This is useful when you have a deployment ongoing and you don't want to be notified about every change that happens during the deployment. You can mute notifications for a specific time period. This keeps the notifications clean and relevant. Also this keeps your webhook from being overloaded.Some webhooks have a rate limit per minute, so this feature can help you avoid hitting the rate limit and missing important notifications after the deployment.
It accepts two parameters:
- namespace: the namespace you want to mute notifications for
- duration: the duration you want to mute notifications for in minutes The path looks like this: PUT url/deploy/namespace/duration PUT url/deploy/namespace - uses default duration of 2 minutes POST url/reset - clears all muted namespaces
#Examples:
curl -X PUT http://localhost:8080/deploy/default/5
#Mutes notifications for the default namespace for 5 minutes
curl -x Delete http://localhost:8080/deploy/default
#Clears the default namespace from the muted list
curl -X PUT http://localhost:8080/deploy/default
#Mutes notifications for the default namespace for 2 minutes
curl -X POST http://localhost:8080/reset
#Clears all muted namespaces
docmarr/kubestatewatch:2.0.0
To install the chart with the release name my-release
:
$ helm repo add kubestatewatch https://marvasgit.github.io/kubernetes-statemonitor/
$ helm install my-release kubestatewatch -n NS
The command deploys statemonitor on the Kubernetes cluster in the default configuration. With the default configuration, the chart monitors all namespaces.
$ helm install my-release -f values.yaml statemonitor
Tip: You can use the default values.yaml
To uninstall/delete the my-release
deployment:
$ helm delete my-release
The command removes all the Kubernetes components associated with the chart and deletes the release.
Open https://my.slack.com/services/new/bot to create a new Slack bot.
The API token can be found on the edit page (it starts with xoxb-
).
Invite the Bot to your channel by typing /join @name_of_your_bot
in the Slack message area.
Once you have a Teams account and have created a team to work with, take the following steps to create a webhook:
- Channel -> Connectors -> Incoming Webhook -> Configure -> Add -> Name -> Create
- Copy the webhook URL and paste it into the
msteams.webhook
value in thevalues.yaml
file. - Change the
msteams.enabled
value totrue
in thevalues.yaml
file.
IMPORTANT Note: There is a msg rate limit per webhook per minute. If you exceed the limit, you will receive a 429 error code. Here is a link for more information on rate limits.
Based on the desired communication channel, you need to configure the following values in the values.yaml
file:
- Channel .enabled - where the channel is your desired communication channel (slack, msteams, discord, etc.)
- Relevant values for the channel (slack.token, msteams.webhook, etc.)
namespaceconfig.include & namespaceconfig.exclude
- the namespaces you want to monitor, By default you monitor everything. If you want to monitor only specific namespaces, you can use the include and exclude options. If you use both, the exclude option will be ignored. You probably want to exclude the kube-system namespace.resources
- the resources you want to monitorignore
- the resources you want to ignorediff.ignorePath
- this configuration affects all components that you watch. the paths you want to ignore in the diff ( Usually /metadata, /status, and everything that is not relevant to you)
message:
title: "XXXX"
diff:
ignorePath:
#- "/metadata"
#- "/spec/template/metadata"
#- "/status"
#- "/spec/replicas"
#- "/lastTimestamp"
#- "/data/status"
#- "/count"
# - "/metadata"
# - "/status"
# - "/metadata/replicas"
namespacesconfig:
include:
exclude:
#- "kube-system"
#- "cattle-fleet-system"
# changed on V2.0.0
resourcesToWatch:
configmap:
enabled: true
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
daemonset:
enabled: true
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
deployment:
enabled: true
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
event:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
coreevent:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
hpa:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
job:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
persistentvolume:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
pod:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
replicaset:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
replicationcontroller:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
node:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
services:
enabled: false
includeEvenTypes:
#- "add"
#- "update"
#- "delete"
ignorePath:
# - "/status"
You can configure multiple connectors, for example slack and msteams, or slack and smtp, or slack and webhook, etc.
slack:
enabled: false
channel: "XXXX"
token: "XXXX"
slackwebhook:
enabled: false
channel: "XXXX"
username: ""
emoji: ""
slackwebhookurl: "XXXX"
hipchat:
enabled: false
room: ""
token: ""
url: ""
mattermost:
enabled: false
channel: ""
url: ""
username: ""
flock:
enabled: false
url: ""
msteams:
enabled: false
webhookurl: "XXXX"
webhook:
enabled: false
url: ""
cloudevent:
enabled: false
url: ""
lark:
enabled: false
webhookurl: ""
smtp:
enabled: false
to: ""
from: ""
hello: ""
smarthost: ""
subject: ""
auth:
username: ""
password: ""
secret: ""
identity: ""
requireTLS: ""
To Run statemonitor Container interactively, place the config file in /path/to/your/appsettings.json
location and use the following command.
docker run --rm -it --network host -v /path/to/your/appsettings.json:/config/appsettings.json --name <container-name> docmarr/kubestatewatch:1.0.2
If you want to build the docker image locally, you can use the following commands:
clone repo
git clone github.com/marvasgit/kubernetes-statemonitor.git
build docker image and run it
docker build -t <docker.tagname> .
docker run --rm -it --network host -v /path/to/your/appsettings.json:/config/appsettings.json --name <container-name> <docker.tagname>
Clone the repository anywhere:
$ git clone https://github.com/marvasgit/kubernetes-statemonitor.git
$ cd statemonitor
$ go build
or
You can also use the Makefile directly:
$ make build
- You need to have Go (v1.5 or later) installed. Make sure to set
$GOPATH
$ make docker-image
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
statemonitor latest 919896d3cd90 3 minutes ago 13.9MB
- you need to have docker installed.
None at the moment. But if you have any suggestions, please create an issue.
Refer to the contribution guidelines to get started.