How to integrate PushProx's service discovery into Promgen? about promgen HOT 9 CLOSED

Hmm, that's an interesting use case that has not come up yet. I am familiar with PushProx but have not used it myself

Right now, Promgen does have an import method, but that was intended to import from other Promgen instances (in the case you wanted to move a group from a beta environment to a release environment or something like that)

It currently expects that there would be labels for service,project,job so that promgen would know where to group things.

1. If you wanted to just manage rules + alerts, that's a viable option even without importing the list
2. If you know a bit of Django, it would also not be difficult to write a small cli command to import the list

I can write a more detailed explanation about either of those if that sounds like something that would work in your environment. I will also take another glance at PushProx to refresh myself and see what it does in terms of labels

from promgen.

from promgen.

Hi again Paul,

I'm thinking that for now I'll keep it simple, and will just try to add the proxied targets manually.

I've run into an issue, though: how can I set properties such as "proxy_url" and "params" when adding a host manually in Promgen?

For instance, this is the relevant portion of my current prometheus.yml file for a job that scrapes the Node Exporters of hosts server1 and server2 through the PushProx proxy. (PushProx is running on the same server as Prometheus on port 8080) :

  - job_name: proxied_node
    proxy_url: 'http://127.0.0.1:8080'
    params:
      collect[]:
        - filesystem
        - systemd
        - cpu
        - meminfo
    static_configs:
    - targets: ['server1:9100']
    - targets: ['server2:9100']

from promgen.

Sorry for the delay.
proxy_url and params can be set in the Prometheus configuration. Currently there is not a way to set those in Promgen. Params could be a good option to add to Promgen (and actually metrics_path is set in a similar way) but there's not a good way to allow Promgen to manage proxy_url since that's entirely handled by Prometheus.

from promgen.

Hey Paul,

Thanks again - I have sorted my proxy_url and params needs via the Prometheus config, along with 'relabel_configs' and 'keep'.

I am sort of back to square one regarding my discovery needs, though... We'll actually be using Ansible to deploy monitoring to new hosts, and our goal is to end up with a script (cli or otherwise) that Ansible can kick off to add hosts to a Promgen farm.

I'll appreciate it if you can point me in the right direction.. I'm new to Django but not to Python, so I may not need much more than a little bit of info to get started ;)

from promgen.

Right now, there's not a simple way to do this via ansible.

There is a promgen import command, that expects a list of targets in the file_sd_config format, but that's probably not easy to generate from Ansible.

One possible option, that probably would not require much code, and would probably be easy to use from ansible, is to add a CLI command that allows you to add a monitoring target. You can look in promgen/management/commands for some examples, but perhaps a command like

promgen register-job <project> <job> <port> [path]
could be paired with a command to register a host
promgen register-host <project> <host>

though there's probably a few minor things regarding farms that I haven't thought through yet.

Once Promgen had commands for that, it should be (reasonably) easy to call those command from your ansible scripts

from promgen.

I wrote #192 as an example command to handle the first half, I will add the next command shortly. There's a possibility I may want to rename the commands slightly in the future but hopefully that would not be a major issue

from promgen.

Thanks Paul! This is appreciated. Looking forward to the next command.

from promgen.

Added a register-host command in #193 so hopefully that will take care of that part.

from promgen.