Prometheus can scrape Pi-hole’s metrics, but the built-in /admin/api.php?summary=true endpoint is missing crucial context for effective alerting.
Let’s get Pi-hole’s metrics into Prometheus and set up some alerts.
First, we need to expose Pi-hole’s data in a Prometheus-friendly format. Pi-hole has a built-in API, but it’s not ideal for Prometheus scraping out of the box. We’ll use a small exporter to bridge this gap.
Install pihole-exporter on your Pi-hole host or a separate machine that can access Pi-hole’s API. You can usually find pre-built binaries or build it from source. For example, on a Debian/Ubuntu system, you might download a .deb file.
wget https://github.com/nonet/pihole-exporter/releases/download/vX.Y.Z/pihole-exporter_X.Y.Z_linux_amd64.deb
sudo dpkg -i pihole-exporter_X.Y.Z_linux_amd64.deb
Now, configure pihole-exporter. The default configuration usually works if pihole-exporter is on the same host as Pi-hole, but you might need to specify the Pi-hole API endpoint if it’s remote:
/etc/pihole-exporter/config.yaml
pihole:
api_url: "http://127.0.0.1/admin/api.php" # Or your Pi-hole's IP if remote
api_token: "YOUR_PIHOLE_API_TOKEN" # Generate this in Pi-hole's settings
To get your API token, log into your Pi-hole web interface, go to Settings -> API/Web interface, and click "Show API Token."
Start and enable the pihole-exporter service:
sudo systemctl start pihole-exporter
sudo systemctl enable pihole-exporter
Verify that pihole-exporter is running and exposing metrics on port 9100 by default:
curl http://localhost:9100/metrics
You should see output like:
# HELP pihole_ads_blocked_today Total number of ads blocked today.
# TYPE pihole_ads_blocked_today gauge
pihole_ads_blocked_today 12345
# HELP pihole_dns_queries_today Total number of DNS queries today.
# TYPE pihole_dns_queries_today gauge
pihole_dns_queries_today 54321
# HELP pihole_ads_percentage_today Percentage of ads blocked today.
# TYPE pihole_ads_percentage_today gauge
pihole_ads_percentage_today 22.72
# HELP pihole_clients_unique_today Number of unique clients that made DNS queries today.
# TYPE pihole_clients_unique_today gauge
pihole_clients_unique_today 15
# HELP pihole_domains_total Total number of domains in the blocklists.
# TYPE pihole_domains_total gauge
pihole_domains_total 100000
# HELP pihole_gravity_last_updated_timestamp Timestamp of the last gravity update.
# TYPE pihole_gravity_last_updated_timestamp gauge
pihole_gravity_last_updated_timestamp 1678886400
# HELP pihole_queries_forwarded_today Total number of DNS queries forwarded today.
# TYPE pihole_queries_forwarded_today gauge
pihole_queries_forwarded_today 40000
# HELP pihole_queries_cached_today Total number of DNS queries served from cache today.
# TYPE pihole_queries_cached_today gauge
pihole_queries_cached_today 1321
Now, configure Prometheus to scrape these metrics. Edit your prometheus.yml file:
prometheus.yml
scrape_configs:
- job_name: 'pihole'
static_configs:
- targets: ['localhost:9100'] # If pihole-exporter is on the same host
# - targets: ['<pihole-exporter-host>:9100'] # If pihole-exporter is on a different host
Reload your Prometheus configuration. You should see the pihole job appear in the "Targets" page of your Prometheus UI.
With metrics flowing, let’s set up some alerts in Alertmanager via Prometheus rules. Create a pihole_alerts.yml file in your Prometheus rules directory.
pihole_alerts.yml
groups:
- name: pihole.rules
rules:
- alert: PiholeAdsBlockedLow
expr: pihole_ads_blocked_today < 100
for: 15m
labels:
severity: warning
annotations:
summary: "Pi-hole ads blocked is low"
description: "Pi-hole has blocked fewer than 100 ads in the last 15 minutes (current: {{ $value }})."
- alert: PiholeHighDnsQueries
expr: pihole_dns_queries_today > 50000
for: 10m
labels:
severity: info
annotations:
summary: "Pi-hole experiencing high DNS query volume"
description: "Pi-hole has processed over 50,000 DNS queries in the last 10 minutes (current: {{ $value }})."
- alert: PiholeGravityStale
expr: time() - pihole_gravity_last_updated_timestamp > 86400 # 24 hours
for: 5m
labels:
severity: critical
annotations:
summary: "Pi-hole gravity is stale"
description: "Pi-hole's ad lists (gravity) have not been updated in over 24 hours. Last updated at {{ pihole_gravity_last_updated_timestamp | human_readable_timestamp }}."
- alert: PiholeHighAdsPercentage
expr: pihole_ads_percentage_today > 50
for: 30m
labels:
severity: warning
annotations:
summary: "Pi-hole blocking an unusually high percentage of ads"
description: "Pi-hole is blocking over 50% of DNS queries, which might indicate an issue or a flood of ad requests (current: {{ $value }}%)."
Make sure to add this new rules file to your prometheus.yml:
prometheus.yml
rule_files:
- "path/to/your/pihole_alerts.yml"
Reload Prometheus again. Now, if any of these conditions are met for their specified for duration, an alert will fire to Alertmanager.
The pihole_gravity_last_updated_timestamp metric is particularly useful because it’s a direct timestamp of when Pi-hole last updated its blocklists. By comparing time() (the current Unix timestamp) to this value, we can detect if the gravity files are becoming stale, which means Pi-hole is no longer blocking new ads effectively.
The next step is to monitor Pi-hole’s DNS resolution time, which pihole-exporter doesn’t directly expose but can be inferred or monitored separately with tools like blackbox_exporter.