Prometheus Blackbox Exporter: Probe HTTP and TCP (2026)

The most surprising thing about Prometheus’ Blackbox Exporter is that it doesn’t actually do the probing itself; it acts as a proxy, allowing Prometheus to check external services by having the Blackbox Exporter make the requests on Prometheus’ behalf.

Let’s see this in action. Imagine you have a web service running at http://my-external-app.com and you want Prometheus to monitor its availability.

Here’s a snippet from a Prometheus configuration file (prometheus.yml):

scrape_configs:
  - job_name: 'external-http'
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets:
        - http://my-external-app.com
        - https://another-service.net
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: blackbox-exporter.my-namespace.svc.cluster.local:9115 # The address of your Blackbox Exporter

And here’s a minimal Blackbox Exporter configuration (blackbox.yml):

modules:
  http_2xx:
    prober: http
    http:
      method: GET
      fail_if_ssl: false
      fail_if_not_ssl: false
      valid_status_codes: [] # Defaults to 2xx
      method: GET
      body_file: /path/to/some/optional/body.txt # Optional request body
      headers:
        User-Agent: Prometheus-Blackbox-Exporter

When Prometheus scrapes the external-http job:

1. It sees http://my-external-app.com as a target.
2. The relabel_configs tell Prometheus to change the scrape target to the Blackbox Exporter’s address (blackbox-exporter.my-namespace.svc.cluster.local:9115).
3. Crucially, it adds the original target (http://my-external-app.com) as a query parameter named target to the Blackbox Exporter’s /probe endpoint. So, Prometheus is actually trying to scrape http://blackbox-exporter.my-namespace.svc.cluster.local:9115/probe?target=http://my-external-app.com&module=http_2xx.
4. The Blackbox Exporter, listening on port 9115, receives this request.
5. It looks at the module parameter (http_2xx) and finds the corresponding configuration in blackbox.yml.
6. It then performs the actual HTTP GET request to http://my-external-app.com as defined in the http_2xx module.
7. Finally, the Blackbox Exporter exposes Prometheus metrics (like probe_success, probe_duration_seconds, probe_http_status_code) based on the result of its probe, which Prometheus then scrapes.

This setup allows Prometheus to monitor endpoints it cannot directly reach, or to perform checks that require specific protocols or authentication that Prometheus itself doesn’t natively support. The Blackbox Exporter becomes a flexible probe agent.

You can configure various modules for different protocols. For TCP probes, you’d define a tcp module:

modules:
  # ... other modules ...
  tcp_connect:
    prober: tcp
    tcp:
      timeout: 5s
      ip_protocol: ip4 # or ip6
      query_response:
        - expect: "some expected response" # Optional: check for a specific response after connecting

And in prometheus.yml:

scrape_configs:
  - job_name: 'external-tcp'
    metrics_path: /probe
    params:
      module: [tcp_connect]
    static_configs:
      - targets:
        - my-tcp-service.local:8080
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: blackbox-exporter.my-namespace.svc.cluster.local:9115

The probe_success metric will be 1 if the TCP connection to my-tcp-service.local:8080 succeeds within the 5s timeout, and 0 otherwise.

The most powerful aspect, and often overlooked, is the relabel_configs within Prometheus. They are not just for routing to the Blackbox Exporter; they are essential for transforming the target address into the __param_target query parameter that the Blackbox Exporter expects, and then mapping the Blackbox Exporter’s own address (__address__) to the actual scrape target for Prometheus. Without these relabeling rules, the Blackbox Exporter wouldn’t know what to probe, and Prometheus wouldn’t know where to send the probe request.

Once you have basic HTTP and TCP probes working, the next logical step is exploring how to use TLS probes with certificate validation and custom CA certificates.