Targeting multiple DNS records about blackbox_exporter HOT 12 CLOSED

just in case someone will need to solve same issue:
blackbox.yml

modules:
  facebook.com:
    prober: dns
    timeout: 5s
    dns:
      query_name: "facebook.com"
      query_type: "A"
      preferred_ip_protocol: "ip4"
  imdb.com:
    prober: dns
    timeout: 5s
    dns:
      query_name: "imdb.com"
      query_type: "A"
      preferred_ip_protocol: "ip4"

prometheus scrape

  - job_name: 'blackbox-dns'
    scrape_interval: 30s
    metrics_path: /probe
    static_configs:
      - targets:
        - facebook.com:8.8.8.8
        - imdb.com:8.8.8.8
        - facebook.com:1.1.1.1
        - imdb.com:1.1.1.1
        - facebook.com:127.0.0.1
        - imdb.com:127.0.0.1
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - source_labels: [__param_target]
        regex: '(.*):.*$'
        replacement: '$1'
        target_label: 'domain'
      - source_labels: [__param_target]
        regex: '.*:(.*)$'
        replacement: '$1'
        target_label: 'instance'
      - source_labels: [__param_target]
        regex: '(.*):.*$'
        replacement: '$1'
        target_label: __param_module
      - source_labels: [__param_target]
        regex: '.*:(.*)$'
        replacement: '$1'
        target_label: __param_target
      - target_label: __address__
        replacement: 172.17.0.1:9115  # The blackbox exporter's real hostname:port.

from blackbox_exporter.

This was the top of my google search for "blackbox dns poll multiple domains prometheus" and I found @mancubus77's comment very helpful, I've improved his example Prometheus configuration slightly and would like to share it so everyone can benefit:

- job_name: blackbox_dns
  metrics_path: /probe
    relabel_configs:
  # You can set internal labels that are called __param_<name>.
  # Those set URL parameter with the key <name> for the scrape request.
  # https://prometheus.io/docs/guides/multi-target-exporter/

  # Populate domain label with domain portion of __address__
  - source_labels: [__address__]
    regex: (.*):.*$
    replacement: $1
    target_label: domain
  # Populate instance label with dns server IP portion of __address__
  - source_labels: [__address__]
    regex: .*:(.*)$
    replacement: $1
    target_label: instance
  # Populate module URL parameter with domain portion of __address__
  # This is a parameter passed to the blackbox exporter
  - source_labels: [domain]
    target_label: __param_module
  # Populate target URL parameter with dns server IP
  - source_labels: [instance]
    target_label: __param_target
  # Populate __address__ with the address of the blackbox exporter to hit
  - target_label: __address__
    # Real address of blackbox exporter (Assumed to be running on port 9115 on localhost)
    replacement: 127.0.0.1:9115
  static_configs:
  - targets:
    - facebook.com:8.8.8.8
    - google.com:8.8.8.8
    - imdb.com:8.8.8.8
    - facebook.com:1.1.1.1
    - google.com:1.1.1.1
    - imdb.com:1.1.1.1
    - facebook.com:127.0.0.1
    - google.com:127.0.0.1
    - imdb.com:127.0.0.1

See https://prometheus.io/docs/guides/multi-target-exporter/ for the official docs.

from blackbox_exporter.

I can see @knyar's point regarding the module configuration and customization.

However, I also agree with @mattbostock that it is probably more common to want to probe multiple DNS records against the same DNS server. For example, in my case, I want to probe 54 DNS records against 2 DNS servers. But with the current implementation, I find myself writing a Perl script to generate the 54 modules in my blackbox.yaml config. I then also have to generate 54 Prometheus scrape jobs to scrape each module.

Is there an easier way for me to accomplish this? Or is auto-generating really the way to go? Thanks in advance.

from blackbox_exporter.

@brian-brazil To query multiple records then, would you expect to create a module for each one?

from blackbox_exporter.

@mancubus77 Hello, i take your example but now i want to use Grafana to create a percentage graph based on the values 1 & 0. Do you have the equation ?

from blackbox_exporter.

I think this way around is correct, and is similar to how all the other modules work. Each configured module is one type of test to run against some to-be-specified server.

from blackbox_exporter.

I agree with the "one type of test" argument.

Also, keep in mind that any validation options specified in DNSProbe are often only going to be useful in the context of a specific query_name and query_type - i.e. you would check that your MX record contains your mail server, and that your DNS balancer returns a low TTL for your dynamic records.

Since you normally would want to keep prometheus server configuration in sync with blackbox_exporter anyway (you actually need to use the modules you defined), you are hopefully using a higher level abstraction to generate config files for both. This means that the number of modules that end up in the config file does not practically matter.

from blackbox_exporter.

Makes sense, thanks both!

from blackbox_exporter.

Auto-generation is the way to go.

The expected use case is that you'll run one or two tests against hundreds or thousands of servers. Attempting to go the other via HTTP params wouldn't be sane, there's too many possibilities.

from blackbox_exporter.

So I've got my different probe configurations, one for each domain. How can I relabel so that I can change the query param for probe which each domain against each target?

from blackbox_exporter.

You can use relabelling on the module url parameter.

from blackbox_exporter.

Hi,
How can I check multiple MX records (200+)? I try manual from https://www.robustperception.io/probing-dns-servers-with-the-blackbox-exporter, but can't check status in Grafana, because blackbox don't sent domain name and I can't filtering. Thanks.

from blackbox_exporter.