RabbitMQ Prometheus metrics are surprisingly opt-in, requiring explicit configuration for both enablement and scraping.
Let’s see what that looks like in practice. Imagine you have a RabbitMQ cluster running, and you want to monitor its health and performance using Prometheus. You’ve got Prometheus set up and running, but it’s not seeing your RabbitMQ instance.
First, we need to enable the metrics endpoint on RabbitMQ itself. This is done by installing the prometheus plugin. You’d typically do this on each node in your RabbitMQ cluster.
rabbitmq-plugins enable rabbitmq_prometheus
After enabling the plugin, you need to restart the RabbitMQ server on each node for the plugin to become active.
Once the plugin is enabled and the server restarted, RabbitMQ will expose metrics on a new port, usually 15692 (the same port as the management UI, but a different path). The default endpoint for Prometheus metrics is /metrics. So, if your RabbitMQ node is at rabbitmq.example.com:15692, you’d access the metrics at http://rabbitmq.example.com:15692/metrics.
Now, Prometheus needs to know where to find these metrics. This is configured in your Prometheus prometheus.yml file. You’ll add a new scrape job for your RabbitMQ instances.
scrape_configs:
- job_name: 'rabbitmq'
static_configs:
- targets: ['rabbitmq1.example.com:15692', 'rabbitmq2.example.com:15692']
labels:
env: 'production'
instance: 'rabbitmq-cluster'
In this configuration:
job_name: 'rabbitmq'is a label that Prometheus will apply to all metrics scraped from this configuration.static_configsmeans we’re providing a static list of targets.targetsis a list of RabbitMQ nodes, specifying their hostname/IP and the port where therabbitmq_prometheusplugin is listening.labelsallow you to add custom metadata to your metrics, which is incredibly useful for filtering and aggregation in Prometheus and Grafana.
After updating prometheus.yml, you need to reload the Prometheus configuration. This can be done by sending a SIGHUP signal to the Prometheus process or by making an HTTP request to its reload endpoint:
curl -X POST http://localhost:9090/-/reload
Once Prometheus has reloaded its configuration, it will start scraping the /metrics endpoint on your specified RabbitMQ targets. You can verify this by going to your Prometheus UI, navigating to "Status" -> "Targets," and looking for the rabbitmq job. The targets should show up as "UP."
The rabbitmq_prometheus plugin exposes a rich set of metrics covering various aspects of RabbitMQ, including:
- Node-level metrics: CPU usage, memory, disk space, network traffic.
- Channel and connection metrics: Number of open channels and connections, rates of connection/disconnection.
- Queue metrics: Number of messages, ready messages, unacknowledged messages, message rates (publish, deliver, acknowledge).
- Exchange metrics: Message rates.
- Consumer metrics: Number of active consumers, delivery rates.
The specific metrics available can be seen by visiting the /metrics endpoint directly in your browser or using curl. For instance, you might see metrics like rabbitmq_node_disk_free_bytes, rabbitmq_queue_messages_ready, or rabbitmq_channel_open_count.
The most surprising thing about the rabbitmq_prometheus plugin is that it doesn’t automatically expose all possible metrics. You can actually configure which metrics are exposed by modifying the rabbitmq.conf file. For example, to disable certain verbose metrics that might impact performance on very busy clusters, you can add entries like:
prometheus.disabled_metrics.queue = queue_messages_published_total
prometheus.disabled_metrics.exchange = exchange_message_out_total
This granular control allows you to tailor the metrics collection to your specific monitoring needs and resource constraints, ensuring you’re not overwhelming either RabbitMQ or Prometheus with unnecessary data.
The next logical step after enabling and scraping RabbitMQ metrics is to visualize them, typically by importing a pre-built Grafana dashboard or creating your own.