Giter Club home page Giter Club logo

karma's Introduction

karma

Alert dashboard for Prometheus Alertmanager.


Alertmanager >=0.22.0 is required.


See GitHub Releases for release changelog.

Feature overview

Alertmanager UI is useful for browsing alerts and managing silences, but it's lacking as a dashboard tool - karma aims to fill this gap.

Alert aggregation and deduplication

Starting with the 0.7.0 release it can aggregate alerts from multiple Alertmanager instances, running either in HA mode or separate. Unique alerts are displayed by filtering duplicates. Each alert is tagged with the names of all Alertmanager instances it was found at and can be filtered based on those tags (@alertmanager). Note that @alertmanager tags will be visible only if karma is configured with multiple Alertmanager instances. If alertmanger is configured to use HA clusters then @cluster will be available as well, to set a custom name for each cluster see CONFIGURATION.md.

Screenshot

Alert visualization

Alert groups

Alerts are displayed grouped preserving group_by configuration option in Alertmanager. Note that a unique alert group will be created for each receiver it uses in alertmanager as they can have different group_by settings. If a group contains multiple alerts only the first few alerts will be presented. Alerts are expanded or hidden using - / + buttons. The default number of alerts can be configured in the UI settings module. Each group can be collapsed to only show the title bar using top right toggle icon. Each individual alert will show unique labels and annotations. Labels and annotations that are shared between all alerts are moved to the footer.

Active alerts will show recently expired silences, to allow re-silincing if needed. This is controlled via silences:expired setting. 10m value would show silences expired in the last 10 minutes but only for alerts that started firing more than 10 minutes ago.

Example

Alert history

Alertmanager doesn't currently provide any long term storage of alert events or a way to query for historical alerts, but each Prometheus server sending alerts stores metrics related to triggered alerts. When history:enabled is true karma will use source fields from each alert to try querying alert related metrics on remote Prometheus servers. The result is the number of times given alert group triggered an alert per hour in the last 24h, displayed as 24 blocks. The darker the color the more alerts were triggered in that hour, as compared by all other hours.

Example

For this feature to work karma must be able to connect to all Prometheus servers sending alerts. Be sure to set --web.external-url Prometheus flag to a publicly reachable URL of each server.

Inhibited alerts

Inhibited alerts (suppressed by other alerts, see Alertmanager docs) will have a "muted" button.

Inhibited alert

Clicking on that button will bring a modal with a list of inhibiting alerts.

Inhibiting alerts

Silence deduplication

If all alerts in a group were suppressed by the same silence then, to save screen space, the silence will also be moved to the footer.

Deduplicated silence

Label based multi-grid

To help separate alerts from different environments or with different level of severity multi-grid mode can be enabled, which adds another layer of visually grouping alert groups. To enable this mode go to the configuration modal and select a label name, all alerts will be grouped by that label, each label value will have a dedicated grid, including an extra grid for alerts without that label present.

Example

Silence management

Silence modal allows to create new silences and manage all silences already present in Alertmanager. Silence ACL rules can be used to control silence creation and editing, see ACLs docs for more details.

Silence browser

Alert overview

Clicking on the alert counter in the top left corner will open the overview modal, which allows to quickly get an overview of the top label values for all current alerts.

Overview

Alert acknowledgement

Starting with v0.50 karma can create short lived silences to acknowledge alerts with a single button click. To create silences that will resolve itself only after all alerts are resolved you can use kthxbye. See configuration docs for details.

Dead Man’s Switch support

Starting with v0.78 karma can be configured to check for Dead Man’s Switch style alerts (alert that is always firing). If no alert is found in given alertmanager karma will show an error in the UI. See healthcheck:filters option on configuration docs for details.

Dark mode

Starting with v0.52 release karma includes both light and dark themes. By default it will follow browser preference using prefers-color-scheme media queries.

Dark mode

Demo

Online demo is running latest main branch. It might include features that are experimental and not yet ready to be included.

Release notes

Release notes can be found on GitHub Release Page.

To get notifications about new karma releases go to GitHub karma page, click Watch and select Releases only. This requires GitHub user account. To subscribe to email notifications without GitHub account you can subscribe to the RSS feed that GitHub provides. To get email notifications from those feeds use one of the free services providing RSS to email notifications, like Blogtrottr.

History

I created karma while working for Cloudflare, originally it was called unsee. This project is based on that code but the UI part was rewritten from scratch using React. New UI required changes to the backend so the API is also incompatible. Given that the React rewrite resulted in roughly 50% of new code and to avoid confusion for user I've decided to rename it to karma, especially that the original project wasn't being maintained anymore.

Supported Alertmanager versions

Alertmanager's API isn't stable yet and can change between releases, see VERSIONS in internal/mock/Makefile for list of all Alertmanager releases that are tested and supported by karma. Due to API differences between those releases some features will work differently or be missing, it's recommended to use the latest supported Alertmanager version.

Security

karma doesn't in any way alter alerts in any Alertmanager instance it collects data from. This is true for both the backend and the web UI. The web UI allows to manage silences by sending requests to Alertmanager instances, this can be done directly (browser to Alertmanager API) or by proxying such requests via karma backend (browser to karma backend to Alertmanager API) if proxy mode is enabled in karma config.

If you wish to deploy karma as a read-only tool without giving users any ability to modify data in Alertmanager instance, then please ensure that:

  • the karma process is able to connect to the Alertmanager API
  • read-only users are able to connect to the karma web interface
  • read-only users are NOT able to connect to the Alertmanager API
  • readonly is set to true in alertmanager:servers config section for all alertmanager instances, this options will disable any UI elements that could trigger updates (like silence management)

To restrict some users from creating silences or enforce some matcher rules use silence ACL rules. This feature requires proxy to be enabled.

Metrics

karma process metrics are accessible under /metrics path by default. If you set the --listen.prefix option a path relative to it will be used.

Building and running

Building from source

To clone git repo and build the binary yourself run:

git clone https://github.com/prymitive/karma $GOPATH/src/github.com/prymitive/karma
cd $GOPATH/src/github.com/prymitive/karma

To finally compile karma the binary run:

make

Note that building locally from sources requires Go, nodejs and yarn. See Docker build options below for instructions on building from withing docker container.

Running

karma can be configured using config file, command line flags or environment variables. Config file is the recommended method, it's also the only way to configure karma to use multiple Alertmanager servers for collecting alerts. To run karma with a single Alertmanager server set ALERTMANAGER_URI environment variable or pass --alertmanger.uri flag on the command line, with Alertmanager URI as argument, example:

ALERTMANAGER_URI=https://alertmanager.example.com karma
karma --alertmanager.uri https://alertmanager.example.com

There is a make target which will compile and run a demo karma docker image:

make run-demo

By default it will listen on port 8080 and will have mock alerts.

Docker

Running pre-build docker image

Official docker images are built and hosted on Github.

Images are built automatically for:

  • release tags in git - ghcr.io/prymitive/karma:vX.Y.Z
  • main branch commits - ghcr.io/prymitive/karma:latest

NOTE karma uses uber-go/automaxprocs to automatically adjust GOMAXPROCS to match Linux container CPU quota.

Examples

To start a release image run:

docker run -e ALERTMANAGER_URI=https://alertmanager.example.com ghcr.io/prymitive/karma:vX.Y.Z

Latest release details can be found on GitHub.

To start docker image build from lastet main branch run:

docker run -e ALERTMANAGER_URI=https://alertmanager.example.com ghcr.io/prymitive/karma:latest

Note that latest main branch might have bugs or breaking changes. Using release images is strongly recommended for any production use.

Building a Docker image

make docker-image

This will build a Docker image locally from sources.

Health checks

/health endpoint can be used for health check probes, it always responds with 200 OK code and Pong response body.

Configuration

Please see CONFIGURATION for full list of available configuration options and example.yaml for a config file example.

Contributing

Please see CONTRIBUTING for details.

License

Apache License 2.0, please see LICENSE.

karma's People

Contributors

aalexk avatar benoitknecht avatar carlpett avatar cinder-fish avatar david-caro avatar davidkarlsen avatar dependabot-support avatar dependabot[bot] avatar dmke avatar donbowman avatar ephur avatar filippog avatar fvbommel avatar giganteous avatar greenkeeper[bot] avatar greenkeeperio-bot avatar honkinggoose avatar jamesog avatar luke-orden avatar mattbostock avatar mhrabovcin avatar msvticket avatar nledez avatar prymitive avatar renovate-bot avatar renovate[bot] avatar solarnz avatar supertassu avatar svenmueller avatar swoga avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

karma's Issues

Karma shouldn't enforce email for Author

Karma should relax the constraint, either via a configuration parameter or general, that an email is required for the author field. Alertmanager makes no restriction that the author field should be an email, Karma shouldn't either.

Feature Request : Collapse All + colorize severity

Hi, thanks for the job, we like Unsee, and we appreciate Karma for reading gain.

We have two requests if that's possible, the first is the possibility of a "Collapse All" function to minimize the current view.
The seconds requests is the possibility to colorize the card according to the severity label (or other label). We need to differentiate quickly a warning or a critial alert.

We made some test with custom.js but I thnink we can't easily achieved this feature like this.

What do you think about this?

Regards

Custom colors wildcard

We should be able to specify a wildcard in the custom colors dict. This allows us to set a custom color for the all the label values. Useful to change the default static color.

labels:
  color:
    custom:
      severity:
        info: "#87c4e0"
        warning: "#ffae42"
        critical: "#ff220c"
        "*": "#ffffff"

Silences blocked by CORS.

Karma v0.36
Alertmanager v0.17
Both behind nginx proxy with their domains.
After submitting silence i get error:
Access to fetch at 'https://alertmanager..../api/v2/silences' from origin 'https://karma....' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: The value of the 'Access-Control-Allow-Origin' header in the response must not be the wildcard '*' when the request's credentials mode is 'include'.

Feature Request: Card view disorienting

Hello! Really like your project.
Its hard to see and visually sort the alerts when there are more than a few.
A good'ol sortable table would work really well.
Any plans to implement a "table view"-mode?

/api/v2/silences: context deadline exceeded

After upgrading from Alertmanager 0.15.3 to .0.17.0, Karma 0.39 stopped working.

curling the endpoint works without any issues.

./karma-linux-amd64 --version
"v0.39"
alertmanager --version
alertmanager, version 0.17.0 (branch: non-git, revision: c7551cd)
  build date:       20190624-21:36:45
  go version:       go1.11.10
INFO[0000] [am] Configured Alertmanager source at https://alertmanager:9093 (proxied: true) 
INFO[0000] Initial Alertmanager query                   
INFO[0000] Pulling latest alerts and silences from Alertmanager 
INFO[0000] [am] Collecting alerts and silences    
INFO[0000] GET https://alertmanager:9093/metrics timeout=0s 
INFO[0000] Upstream version: 0.17.0                     
INFO[0000] GET https://alertmanager:9093/api/v1/status timeout=0s 
INFO[0000] [am] Remote Alertmanager version: 0.17.0 
ERRO[0000] [am] Get https://alertmanager:9093/api/v2/silences: context deadline exceeded```

Not working at all on Firefox based signage system

Moving from Unsee to Karma does not work on our signage system.

On one screen I get

Internal error
TypeError: Object.entries is not a function
This page will auto refresh in xxx s

The number of seconds (xxx) is changing quickly - for every second that passes it is reduced by 4 seconds. There is a graphical exclamation mark in front of Internal.

On another screen I tried Karma and it just returns a blank turquoise screen without any elements.

The browser in our signage system is a Firefox "Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0". Unfortunately I can't access any developer tools to give more information.

Basic auth: unknown error (status 401)

Let's test basic auth credentials:

curl -v -u admin:secretPassword https://alertmanager.example.com
[...]
< HTTP/2 200

Works fine.

karma.yaml:

    alertmanager:
      interval: 1m
      servers:
      - name: example
        uri: https://admin:[email protected]

karma generates error message:

time="2019-05-08T15:01:23Z" level=info msg="GET https://admin:[email protected]/api/v1/status timeout=10s"
time="2019-05-08T15:01:23Z" level=info msg="[example] Remote Alertmanager version: 0.17.0"
time="2019-05-08T15:01:23Z" level=error msg="[example] unknown error (status 401): {resp:0xc0006c4750} "

karma:v0.35

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper App’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.

Allow stripping of annotations

Hi,

We use annotations for storing display informations which are used in alertmanagers.

We'd like to strip them completely in karma (as we can do with labels), not only hide them.

Thanks!

authentication not passed to /silence

We authenticate using nginx auth_request.
When are using proxy mode in the configuration file.

When submitting a new silence, cookies are not sent with the POST, resulting in a 401.

Seems to be related to #24 ?

Feature request: Nightmode

First of all, thank you for developing this dashboard! It is very handy for us at trivago.

Would it be possible to have a "Nightmode" or "Dark mode" setting in the dashboard?

Some of us work in a low luminosity environment and the karma dashboard is bright. It strains the eye after a while.

If the little alarm windows could have a dark gray background for example, and the window could have an even darker shade of gray in this option, that would be great! Grafana does this very well.

Do you think this would be possible? It would certainly make some of your users very happy!

make it possible to skip tls validation of https alertmanager backend

Our alertmanager instance comes with self-signed certs by vendor (IBM ICP).
It would be nice if you could provide a flag to skip validation of certificate.
I could not find any env var to set this which go would act on - so it seems it has to be coded as an option/flag to pass.

Get https://monitoring-prometheus-alertmanager.kube-system:9093/alertmanager/api/v1/silences: x509: certificate signed by unknown authority

Single flag to specify configuration file path and no need to append .yaml (please...)

It would be great if we could specify the path to the configuration file in a single command flag (instead of using two), e.g.:

$ karma --config.file /etc/karma/karma.yaml

We've also noticed that karma is appending a .yaml suffix to the file name we provide, which is a bit counter-intuitive, especially with yaml as some prefer .yml (like in our case 😄) while others use .yaml

Again, thanks!!!

Not finding karma.yaml in root

Howdy Lukasz,

Great work on this, I just found out about your work on Karma after CloudFlare. I'm evaluating it now to replace unsee in swarmstack (https://www.youtube.com/watch?v=3FpTcVnvfRg - https://github.com/swarmstack/swarmstack) - which rebuilt Stefan Prodan's swarmprom starter kit for Docker Prometheus/Grafana/Alertmanager/Unsee and adds Docker swarm node installation and ongoing management with ease of installation as a focus.

Dockerfile below works, but requires addition of the CONFIG_FILE env variable. My reading of docs/CONFIGURATION.md was that Karma would look for a file named karma.yaml in the same directory. Didn't see an existing ticket on this issue, but might be pebkac as I just found out about Karma an hour ago :)

#cat docker-compose.yml
version: "3.4"
configs:
  karma-conf:
    file: ./karma/karma.yaml

services:
  karma:
    image: lmierzwa/karma:latest
    configs:
      - source: karma-conf
        target: /karma.yaml
    deploy:
      mode: replicated
      replicas: 1
      labels:
        prometheus.enable: "true"
        prometheus.port: "8080"
        prometheus.path: "/metrics"
    environment:
      - CONFIG_FILE=/karma.yaml

Add an icon next to silence and gear for linking to Monitoring main page

We all have a starting point for our monitoring systems: In our case it is a Grafana page that has a bunch of links to dashboard/documentation, some metrics, a live log of events, etc.
When we arrive in Karma, there is no way to get back to our starting page. Having an icon and a link for going back to our monitoring start page would be very helpful. Also document where the image is located and that it can be replaced by a user provided image (so we can put our system related icon)
It is a minor enhancement, but I think it would greatly improve usability.
Keep up the good work!

Sometimes, no rendering alerts

We use Karma on top of an AlertManager high availability cluster (with 2 or more AM). We use filter on labels. Sometimes no alerts rendering on Karma.
In this example, the number on top indicate that no alert match 'pf="sdr"' but in AM we can show the alerts.

2019-07-09 08_51_23-Greenshot

2019-07-09 08_51_31-Greenshot (2)

We don't see error in Karma logs

time="2019-07-09T07:06:46Z" level=info msg="Pulling latest alerts and silences from Alertmanager"
time="2019-07-09T07:06:46Z" level=info msg="[sdr] Collecting alerts and silences"
time="2019-07-09T07:06:46Z" level=info msg="[xymon] Collecting alerts and silences"
time="2019-07-09T07:06:46Z" level=info msg="GET http://alertmanager.xymon:9093/metrics timeout=10s"
time="2019-07-09T07:06:46Z" level=info msg="GET http://alertmanager.prometheus:9093/metrics timeout=10s"
time="2019-07-09T07:06:46Z" level=info msg="Upstream version: 0.18.0"
time="2019-07-09T07:06:46Z" level=info msg="GET http://alertmanager.prometheus:9093/api/v1/status timeout=10s"
time="2019-07-09T07:06:46Z" level=info msg="[sdr] Remote Alertmanager version: 0.18.0"
time="2019-07-09T07:06:46Z" level=info msg="[sdr] Got 3 silences(s) in 764.867µs"
time="2019-07-09T07:06:46Z" level=info msg="[sdr] Detecting JIRA links in silences (3)"
time="2019-07-09T07:06:46Z" level=info msg="Upstream version: 0.17.0"
time="2019-07-09T07:06:46Z" level=info msg="GET http://alertmanager.xymon:9093/api/v1/status timeout=10s"
time="2019-07-09T07:06:46Z" level=info msg="[xymon] Remote Alertmanager version: 0.17.0"
time="2019-07-09T07:06:46Z" level=info msg="[xymon] Got 0 silences(s) in 4.333911ms"
time="2019-07-09T07:06:46Z" level=info msg="[xymon] Detecting JIRA links in silences (0)"
time="2019-07-09T07:06:46Z" level=info msg="[sdr] Got 136 alert group(s) in 67.698158ms"
time="2019-07-09T07:06:46Z" level=info msg="[sdr] Deduplicating alert groups (136)"
time="2019-07-09T07:06:46Z" level=info msg="[sdr] Processing unique alert groups (56)"
time="2019-07-09T07:06:46Z" level=info msg="[sdr] Merging autocomplete data (480)"
time="2019-07-09T07:06:46Z" level=info msg="[xymon] Got 997 alert group(s) in 585.932343ms"
time="2019-07-09T07:06:46Z" level=info msg="[xymon] Deduplicating alert groups (997)"
time="2019-07-09T07:06:46Z" level=info msg="[xymon] Processing unique alert groups (838)"
time="2019-07-09T07:06:46Z" level=info msg="[xymon] Merging autocomplete data (2908)"
time="2019-07-09T07:06:46Z" level=info msg="Pull completed"
time="2019-07-09T06:59:34Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 15.087184ms"
time="2019-07-09T07:00:34Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 18.785712ms"
time="2019-07-09T07:01:35Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 15.025122ms"
time="2019-07-09T07:02:35Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 11.880314ms"
time="2019-07-09T07:03:36Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 16.137345ms"
time="2019-07-09T07:04:37Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 19.860792ms"
time="2019-07-09T07:05:37Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 18.43056ms"
time="2019-07-09T07:06:38Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 21.778087ms"
time="2019-07-09T07:07:38Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 22.852808ms"
time="2019-07-09T07:08:39Z" level=info msg="[10.244.6.0 MIS] <200> GET /alerts.json?q=pf%3Dsdr&q=%40state%3Dactive took 14.763488ms"

Did you encounter this problem ?

Use ENTRYPOINT instead of CMD in Dockerfile

Hey!

First of all this tool is totally dope! thanks 🙌

Small request, it would be great if you could change the Dockerfile to use ENTRYPOINT instead of CMD, this would make things much simpler in terms of passing cmd line arguments to the container, as these are set into CMD and are appended by default to ENTRYPOINT.

Thanks!

Tooltips not rendering correctly in Chrome

Tooltip text when hovering over an alertname is not rendering correctly on Chrome 73/Mac. It's rendering as faint black text, as seen in the image below.

Karma v0.32

Screen Shot 2019-04-09 at 2 11 21 PM

allow config test options

Hi!

I'm trying to check karma config on ci before applying it to production.
I want to check it with live version of karma to ensure syntax is ok.
I see two approaches for that

  1. launch timeout 5 karma --config.file karma/karma.yml and wait for something like
FATA[0000] yaml: unmarshal errors:
  line 18: cannot unmarshal !!str `flase` into bool 

but image build on top of gcr.io/distroless/base that does not contain timeout and even sh

  1. add --configtest key to karma and exit if config is wrong

of cause i can rebuild image by myself but that is probably not the best idea

Quick overview label

I imagine that it would be helpful to add new line with the same size like search bar and show label and count in it.

Currently, except suppresed, i have 102 alerts.
i want to get quick overview of alert count across team label. for example
"team: red: 23, blue: 15, white: 12"
and so on.

Don't work with the latest version api (v2) alertmanager

Hi!

In new version of alertmanager API version will be changed to v2. URI /api/v1/ will no longer work. And dashboard not work - request to http://alertmanager:9093/api/v1/alerts/groups failed with 404 Not Found

docker image for test prometheus/alertmanager:master

Improve support for clustered Alertmanagers

When multiple Alertmanagers are configured in Karma, they are treated as separate standalone instances, even if they are part of an HA cluster. This has the disadvantage that when creating silences, it is by default sent to all configured Alertmanagers.

By design, such HA Alertmanagers will propagate the silence via gossip, and it ends up being duplicated.

When Karma then scrapes e.g. two Alertmanagers that are part of a cluster, it sees two silences from each Alertmanager, and ends up displaying four silences. I expect that this will multiply, so 3 Alertmanagers would show 9 silences, 4 Alertmanagers would show 16, and so on.

Got status 404 when submit a silence request.

alertmanager version

alertmanager, version 0.15.2 (branch: HEAD, revision: d19fae3bae451940b8470abb680cfdd59bfa7cfa) build user: root@3101e5b68a55 build date: 20180814-10:53:39 go version: go1.10.3
alertmanager has command line options --web.external-url=https://prom.ctcfin.com/am --web.route-prefix=am

karma version "v0.21"

karma config

alertmanager:
  interval: 15s
  servers:
    - name: alertmanager-headq
      uri: http://1.2.3.4:9093/am/
      timeout: 10s
      proxy: true
debug: true
labels:
  color:
    static:
      - job
    unique:
      - instance_ip
      - app_name
      - env

  keep: []
  strip: []
listen:
  address: "0.0.0.0"
  port: 6677
  prefix: /alert
log:
  config: false
  level: debug
receivers:
  keep: []
  strip: []

When I try to add a silence request in karma, the following url returns a 404 response:

https://karma.example.com/alert/proxy/alertmanager/alertmanager-idc/api/v1/silences

And when I set the proxy off, a CORS issue comes up.

Feature request: view and edit silences in Karma

Creating silences in Karma is super nice! we use it all the time, especially that we can define the same silence on multiple alert managers (in our case in different datacenters).

It would be great if we could also view and edit silences, as we need to jump to the AM UI for that now.

WDYT? @prymitive

Disable silence functionality

Hello,
is there any way to disable silence functionality?
We want to provide an ability to see a list of alerts to our Dev teams, but we don't want to allow them to create or delete any silences.
That is why I'm looking for a way to turn it off

Multiple @alertmanager labels not stripped

If Karma is configured with multiple alertmanager URLs, it seems that it insists on displaying a @alertmanager label for each in the alerts on the dashboard.

I've tried to strip the label, without success.

labels:
  strip:
  - "@alertmanager"

If only a single alertmanager URL is specified, the label is not shown.

Support HTTP/SOCKS proxies for Alertmanager URLs

When using the proxy: true option, it would be useful to have the option to use an HTTP/SOCKS proxy. The use case is for an Alertmanager endpoint running in another network (or the Internet) only reachable through a corporate proxy.

Unable to copy value of annotation

When clicking on an annotation the value is hidden which makes it impossible to select and copy the value.

Could the hiding feature be binded to a double click or just activated when clicking on the key of the annotation ?

Work with unmatched silences

Not sure if I am not missing something or it is currently not possible.
I was following: #87
The creator of the issue had a point: where to view silences.

To clarify, I mean silences which do NOT match any current alert.

Use case:

  1. You create a silence in preparation of maintenance (match non-existing alerts, for example whole env=production)
  2. During the maintenance, the silence matches alerts you wanted to silence (no issues here)
  3. The maintenance lasted less then expected so you want to delete the silence, but currently no alerts are triggered -> can't do it via karma, have to go to alertmanager directly.

Please let me know if I am missing something obvious.
And still want to thank you for your great work, really like the project.

Inhibition rule for Karma?

Is there any current method to mimic the inhibition rule behaviour of Alertmanager in Karma?

For example, two alert rules that fire at different thresholds, one with severity: warning, the other with severity: critical. Alertmanager is able to inhibit the notification for the warning alert if the critical alert is firing. However, Karma displays both alerts, since this is obviously what is returned by the Alertmanager API.

Dependabot can't resolve your JavaScript dependency files

Dependabot can't resolve your JavaScript dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Error while updating /ui/yarn.lock:
Couldn't find any versions for "scheduler" that matches "^16.6.2"

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

Label key "alertname" should be less visually dominating

This is a great tool and it really helps to see things clearer. I read about it on the Prometheus-User-Mailinglist. Thank you very much for presenting it there and moving it over here.

Following the spirit of clarifying I think the actual alertname (the value) should be more prominent than the label’s key. So in alertname: HardToRead the string HardToRead should be more prominent than the alertname.
It is good that the keys are so prominent for other labels, but for alertname less would do because one might use this to find an alert by skimming over and every alert has this label, seeing the precise letters of its key in detail is not as important compared to other labels’ keys.

I am not sure how to do this the best way, but here are a few proposals/ideas:

  • left-align the label (to simplify vertical skimming)
  • change the font redendering for the alertname key
    • less bold
    • less contrast
    • not render the key alertname at all

What do you think?

authentication not passed to /alert.json

We authenticate to karma using apache

But now somehow with the rewrite to react, the /alerts.json is not authenticated like it was with unsee. In the console I see 401 errors on that endpoint.

Unsee CSS

For those nostalgic of the unsee colors like me I made a custom css :

    body {
      font-size: 1.03rem !important;
      background-color: #2c3e50 !important;
    }

    .bg-primary-transparent {
      background-color: #2c3e50 !important;
    }

    .flex-shrink-1 {
      text-align: center !important;
    }

    .components-label-name {
      color: #fff !important;
    }

    .components-label-bright .components-label-name {
      color: #000 !important;
    }

    .text-muted {
      color: #000 !important;
      font-weight: bold !important;
    }

    .card {
      border: 1px solid #e74c3c !important;
    }

    .card > .card-header.bg-light  {
      background-color: #e74c3c !important;
    }

    .card .card-header.bg-light .badge-danger {
      color: #e74c3c !important;
      background-color: #fff !important;
    }

    .card .card-header.bg-light .text-muted {
      color: #fff !important;
    }

    .card-footer {
      line-height: 1;
    }

    .components-label .components-label-name,
    .components-label .components-label-value,
    .components-label-with-hover time {
      font-size: 1.1em !important;
    }

Remove fonts.google.api dependancy

Problem : delay when loading the page

Request: When running in environments where there is no internet access, remove the dependancy to fonts.google.api. Otherwise there is always an annoying delay/timeout when loading the karma web page.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

  • chore(deps): update dependency @percy/cli to v1.12.0

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

dockerfile
Dockerfile
  • node 19.0.0-alpine
  • golang 1.19.2-alpine
demo/Dockerfile
  • node 19.0.0-alpine
  • golang 1.19.2-alpine
  • alpine 3.16
  • ghcr.io/prymitive/kthxbye v0.15
  • prom/alertmanager v0.24.0
internal/mapper/v017/Dockerfile
  • quay.io/goswagger/swagger 0.30.3
github-actions
.github/workflows/actionlint.yml
  • actions/checkout v3
  • reviewdog/action-actionlint v1.33.0
.github/workflows/deps.yml
  • actions/checkout v3
  • actions/dependency-review-action v2
.github/workflows/duplicates.yml
  • wow-actions/potential-duplicates v1.1.0
.github/workflows/go-maintenance.yml
  • actions/checkout v3
  • actions/setup-go v3
  • peter-evans/create-pull-request v4.2.0
  • peter-evans/enable-pull-request-automerge v2.2.1
.github/workflows/go-mod-tidy.yml
  • actions/checkout v3
  • actions/setup-go v3
  • peter-evans/create-pull-request v4.2.0
  • peter-evans/enable-pull-request-automerge v2.2.1
.github/workflows/go-version.yml
  • actions/checkout v3
  • peter-evans/create-pull-request v4.2.0
  • peter-evans/enable-pull-request-automerge v2.2.1
.github/workflows/lock.yml
  • dessant/lock-threads v3
.github/workflows/node-maintenance.yml
  • actions/checkout v3
  • actions/setup-node v3
  • peter-evans/create-pull-request v4.2.0
  • peter-evans/enable-pull-request-automerge v2.2.1
.github/workflows/node-version.yml
  • actions/checkout v3
  • actions/setup-node v3
  • peter-evans/create-pull-request v4.2.0
  • peter-evans/enable-pull-request-automerge v2.2.1
.github/workflows/openapi.yml
  • actions/checkout v3
  • peter-evans/create-pull-request v4.2.0
  • peter-evans/enable-pull-request-automerge v2.2.1
.github/workflows/scan.yml
  • actions/checkout v3
  • github/codeql-action v2
  • github/codeql-action v2
.github/workflows/stale.yml
  • actions/stale v6.0.1
.github/workflows/test.yml
  • actions/checkout v3
  • actions/setup-go v3
  • actions/cache v3
  • codecov/codecov-action v3.1.1
  • actions/checkout v3
  • actions/setup-go v3
  • actions/cache v3
  • actions/checkout v3
  • actions/setup-node v3
  • codecov/codecov-action v3.1.1
  • actions/checkout v3
  • actions/setup-node v3
  • actions/checkout v3
  • actions/setup-go v3
  • actions/checkout v3
  • actions/setup-node v3
  • actions/checkout v3
  • prymitive/depcheck-action v1.4.3
  • actions/checkout v3
  • actions/setup-node v3
  • actions/checkout v3
  • actions/checkout v3
  • actions/checkout v3
  • wagoid/commitlint-github-action v5.2.0
  • actions/checkout v3
  • avto-dev/markdown-lint v1.5.0
  • actions/checkout v3
  • dorny/paths-filter v2.11.1
  • actions/setup-go v3
  • actions/cache v3
  • actions/checkout v3
  • dorny/paths-filter v2.11.1
  • actions/setup-node v3
  • actions/checkout v3
  • dorny/paths-filter v2.11.1
  • actions/setup-node v3
  • actions/checkout v3
  • actions/setup-go v3
  • actions/cache v3
  • actions/setup-node v3
  • bruceadams/get-release v1.2.3
  • AButler/upload-release-assets v2.0
  • actions/checkout v3
  • docker/setup-qemu-action v2
  • docker/setup-buildx-action v2
  • docker/login-action v2
  • docker/login-action v2
  • docker/build-push-action v3
gomod
go.mod
  • go 1.19
  • github.com/Masterminds/semver/v3 v3.1.1
  • github.com/beme/abide v0.0.0-20190723115211-635a09831760@635a09831760
  • github.com/cnf/structhash v0.0.0-20201127153200-e1b16c1ebc08@e1b16c1ebc08
  • github.com/fvbommel/sortorder v1.0.2
  • github.com/go-chi/chi/v5 v5.0.7
  • github.com/go-chi/cors v1.2.1
  • github.com/go-openapi/errors v0.20.3
  • github.com/go-openapi/runtime v0.24.2
  • github.com/go-openapi/strfmt v0.21.3
  • github.com/go-openapi/swag v0.22.3
  • github.com/go-openapi/validate v0.22.0
  • github.com/google/go-cmp v0.5.9
  • github.com/hashicorp/golang-lru v0.5.4
  • github.com/jarcoal/httpmock v1.2.0
  • github.com/klauspost/compress v1.15.11
  • github.com/knadh/koanf v1.4.4
  • github.com/mitchellh/mapstructure v1.5.0
  • github.com/pmezard/go-difflib v1.0.0
  • github.com/prometheus/client_golang v1.13.0
  • github.com/prometheus/common v0.37.0
  • github.com/prymitive/randomcolor v0.0.0-20210705210145-26c3401033a6@26c3401033a6
  • github.com/rogpeppe/go-internal v1.9.0
  • github.com/rs/zerolog v1.28.0
  • github.com/spf13/pflag v1.0.5
  • go.uber.org/automaxprocs v1.5.1
  • gopkg.in/go-playground/colors.v1 v1.2.0
  • gopkg.in/yaml.v3 v3.0.1
tools/benchstat/go.mod
  • go 1.19
  • golang.org/x/perf v0.0.0-20220920022801-e8d778a60d07@e8d778a60d07
tools/gofumpt/go.mod
  • go 1.19
  • mvdan.cc/gofumpt v0.4.0
tools/goimports/go.mod
  • go 1.19
  • golang.org/x/tools v0.1.12
tools/golangci-lint/go.mod
  • go 1.19
  • github.com/golangci/golangci-lint v1.50.0
tools/looppointer/go.mod
  • go 1.19
  • github.com/kyoh86/looppointer v0.1.8-0.20220224024524-f953a93c424a
npm
ui/package.json
  • @floating-ui/react-dom 1.0.0
  • @fortawesome/fontawesome-svg-core 6.2.0
  • @fortawesome/free-regular-svg-icons 6.2.0
  • @fortawesome/free-solid-svg-icons 6.2.0
  • @fortawesome/react-fontawesome 0.2.0
  • @juggle/resize-observer 3.4.0
  • body-scroll-lock 3.1.5
  • bootstrap 5.2.2
  • bootswatch 5.2.2
  • bricks.js 1.8.0
  • copy-to-clipboard 3.3.2
  • csshake 1.7.0
  • date-fns 2.29.3
  • downshift 6.1.12
  • favico.js 0.3.10
  • fontfaceobserver 2.3.0
  • intersection-observer 0.12.2
  • lodash.debounce 4.0.8
  • lodash.merge 4.6.2
  • lodash.throttle 4.1.1
  • lodash.uniqueid 4.0.1
  • mobx 6.6.2
  • mobx-react-lite 3.4.0
  • mobx-stored 1.1.0
  • promise-retry 2.0.1
  • qs 6.11.0
  • react 17.0.2
  • react-app-polyfill 3.0.0
  • react-cool-dimensions 2.0.7
  • react-day-picker 8.3.5
  • react-dom 17.0.2
  • react-hotkeys-hook 3.4.7
  • react-idle-timer 4.6.4
  • react-intersection-observer 9.4.0
  • react-json-pretty 2.2.0
  • react-linkify 0.2.2
  • react-media-hook 0.5.0
  • react-range 1.8.14
  • react-select 5.5.4
  • react-transition-group 4.4.5
  • typeface-open-sans 1.1.13
  • @percy/cli 1.10.4
  • @percy/storybook 4.3.4
  • @storybook/builder-webpack5 6.5.12
  • @storybook/manager-webpack5 6.5.12
  • @storybook/preset-create-react-app 4.1.2
  • @storybook/react 6.5.12
  • @testing-library/jest-dom 5.16.5
  • @testing-library/react 12.1.5
  • @testing-library/react-hooks 8.0.1
  • @testing-library/user-event 14.4.3
  • @types/body-scroll-lock 3.1.0
  • @types/bricks.js 1.8.2
  • @types/enzyme 3.10.12
  • @types/fontfaceobserver 2.1.0
  • @types/jest 29.2.0
  • @types/lodash.debounce 4.0.7
  • @types/lodash.merge 4.6.7
  • @types/lodash.throttle 4.1.7
  • @types/lodash.uniqueid 4.0.7
  • @types/node 16.11.68
  • @types/promise-retry 1.1.3
  • @types/qs 6.9.7
  • @types/react 17.0.50
  • @types/react-dom 17.0.17
  • @wojtekmaj/enzyme-adapter-react-17 0.6.7
  • diffable-html 5.0.0
  • enzyme 3.11.0
  • eslint-config-prettier 8.5.0
  • eslint-config-react-app 7.0.1
  • eslint-plugin-jest 27.1.3
  • eslint-plugin-prettier 4.2.1
  • fetch-mock 9.11.0
  • prettier 2.7.1
  • purgecss 5.0.0
  • react-scripts 5.0.1
  • sass 1.55.0
  • timemachine 0.3.2
  • typescript 4.8.4
  • npm >=8.3 <9.0
  • node 19.0.0
  • autoprefixer 10.4.12

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.