metrico / promcasa Goto Github PK

View Code? Open in Web Editor NEW

18.0 3.0 3.0 65.25 MB

ClickHouse Custom Query / Metrics Exporter for Prometheus

License: GNU Affero General Public License v3.0

Makefile 0.37% Go 99.63%

clickhouse prometheus metrics prometheus-exporter cloki

promcasa's Introduction

PromCasa

Query, Aggregate and Publish anything from ClickHouse to Prometheus metrics in zero time

Unlike other exporters limited to internal metrics, PromCasa is designed to unleash data from any query

⭐ Functionality

Execute recurring Clickhouse queries
Exctract mapped labels and values
Aggregate results using metric buckets
Publish as prometheus metrics

Instructions

Download a binary release or build from source

📦 Download Binary

curl -fsSL github.com/metrico/promcasa/releases/latest/download/promcasa -O && chmod +x promcasa

📄 Configuration

PromCasa acts according to the query bucket parameters configured in /etc/promcasa.json

▶️ Query Buckets

To provision and publish a new metrics bucket, extend the configuration with a query set:

{
   "_info": "Custom Metrics from Clickhouse",
   "name": "my_status", // must be unique
   "help": "My Status",
   "query": "SELECT status, group, count(*) as counter FROM my_index FINAL PREWHERE (datetime >= toDateTime(now()-60)) AND (datetime < toDateTime(now()) ) group by status, group",
   "labels": ["status","group"], // must match columns
   "counter_name": "counter",
   "refresh": "60s", //  Refesh takes unit sign: (ns, ms, s, m, h)
   "type":"gauge" // gauge, histogram, counter
}

For a complete usage example, check out the wiki

License

©️ qxip/metrico Licensed under AGPLv3 as part of qryn

promcasa's People

Contributors

Stargazers

Watchers

Forkers

mtgnoah luis-sousa-pinto compunctus

promcasa's Issues

Concurrent queries limit

Hi,

we are trying Promcasa against Clickhouse with quite a large list of queries.
We are currently hitting this error:

{"level":"error","msg":"Error in future process:code: 202, message: Too many simultaneous queries. Maximum: 100","time":"2023-03-06T11:52:20Z"}

There seems to be nothing in place to limit the number of concurrent queries in promcasa?

Examples for histogram

Hello.

I can ask you to add to the wiki examples of datasource, sql queries and settings to form histogram type metrics, please.

Unfortunately, the existing example does not give an understanding of how to do this.

Thank you.

No configuration file loaded - checking env: While parsing config: invalid character '\x7f' looking for beginning of value ERRO[0000] No configuration file loaded - using defaults - checking env We don't have any active DB session configured. Please check your config

Dear Promcasa devs,
I encountered a problem while executing binary file of promcasa after configuring /etc/promcasa.json file.
Could you expand your tutorial with configuring promcasa.json file with another example of database and table.
For example I have a table orders that has a column date. I want to see the count of table for the last hour and its' configuration in /etc/promcasa,json

Also it will be good if you add another information about how to configure /endpoint of clickhouse so it can show metrics with curl. Because I am struggling with it, when I run curl 'localhost:8123/metrics' it is shows me 0 result.

Thank you!

Feature request: Scraper/Writer

The scheduler and config could be extended to support additional useful features:

optionally writing metrics directly to a remote_write endpoint

{
   "_info": "Custom Metrics from Clickhouse to remote_write",
   "name": "my_status", 
   "help": "My Status",
   "query": "SELECT status, group, count(*) as counter FROM my_index ...",
   "labels": ["status","group"], 
   "counter_name": "counter",
   "refresh": "60s", 
   "type":"gauge",
   "remote_write": "http://qryn:3100/api/prom/remote/write"
}

optionally using the scheduler to pull remote metrics and convert to writes

{
   "_info": "Remote Scraper to Local Writer",
   "name": "my_scraper", 
   "refresh": "60s", 
   "source":"http://target:9096/metrics",
   "remote_write": "http://qryn:3100/api/prom/remote/write"
}

Is it possible to run one promcasa exporter in one instance but with two clickhouse service?

Hello dev team!
I am currently working with cluster of clickhouse servers. I have in 1 - instance, 2 - services of Clickhouse. Is it possible to collect metrics from both of them via promcasa? I just thought about changing a config file like as below:

{
  "database_data": [
    {
      "help": "Settings for Clickhouse Database (data)",
      "user": "promcasa",
      "pass": "12345",
      "name": "db",
      "proto": "http",
      "host": "10.10.10.10",
      "port": 8123,
      "primary": true,
      "debug": true
    }
  ]

and add the same block but with port 8124

{
  "database_data": [
    {
      "help": "Settings for Clickhouse Database (data)",
      "user": "promcasa",
      "pass": "12345",
      "name": "db",
      "proto": "http",
      "host": "10.10.10.10",
      "port": 8124,
      "primary": true,
      "debug": true
    }
  ]

Will it work? If no, could you add this type of feature in futher time and would you mind to give me advice how to deal with my problem?

Please support http protocol in clickhouse connection