Prometheus Relabeling: Drop, Keep, and Transform Labels (2026)

Prometheus relabeling lets you surgically modify metric labels before they’re stored, and it’s way more powerful than just filtering out unwanted metrics.

Let’s say you’re scraping a bunch of targets, and they all send back a metric called http_requests_total. The problem is, some of them have a service_name label, others have app, and some have neither. You want to normalize this so you always have a service label, and you also want to drop any instance labels that are just IP addresses.

Here’s a sample Prometheus scrape configuration:

scrape_configs:
  - job_name: 'my_services'
    static_configs:
      - targets: ['192.168.1.10:8080', '192.168.1.11:9090']
        labels:
          service_name: 'frontend'
          instance: '192.168.1.10:8080'
      - targets: ['192.168.1.20:8080']
        labels:
          app: 'backend'
          instance: '192.168.1.20:8080'
      - targets: ['192.168.1.30:9100']
        labels:
          job: 'node_exporter'
          instance: '192.168.1.30:9100'

And here’s the metric as it comes in from 192.168.1.10:8080:

http_requests_total{service_name="frontend", instance="192.168.1.10:8080"} 12345

And from 192.168.1.20:8080:

http_requests_total{app="backend", instance="192.168.1.20:8080"} 67890

You can see the inconsistency. We want to use relabel_configs to fix this.

Relabeling happens in two phases: relabel_configs (which modifies labels before scraping or during discovery) and metric_relabel_configs (which modifies labels after scraping but before storage). For this example, we’ll focus on metric_relabel_configs.

Here’s a metric_relabel_configs block that addresses our needs:

scrape_configs:
  - job_name: 'my_services'
    static_configs:
      # ... (previous static_configs) ...
    metric_relabel_configs:
      # Rule 1: If a 'service_name' label exists, create a 'service' label with its value and then drop 'service_name'.
      - source_labels: [service_name]
        regex: (.*)
        target_label: service
        action: replace
      - source_labels: [service_name]
        regex: .
        action: labeldrop

      # Rule 2: If an 'app' label exists, create a 'service' label with its value and then drop 'app'.
      - source_labels: [app]
        regex: (.*)
        target_label: service
        action: replace
      - source_labels: [app]
        regex: .
        action: labeldrop

      # Rule 3: Drop any 'instance' label that looks like an IP address.
      - source_labels: [instance]
        regex: (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(:\d+)?
        action: labeldrop

      # Rule 4: If we still don't have a 'service' label (e.g., from node_exporter), set it to 'unknown'.
      - source_labels: [service]
        regex: ^$
        target_label: service
        replacement: unknown
        action: replace

Let’s break down what each rule does:

Rule 1 & 2 (Handling service_name and app)

• source_labels: [service_name] (or [app]): This tells Prometheus to look at the service_name label on the incoming metric.
• regex: (.*): This is a regular expression that captures any value in the service_name label. The parentheses create a capturing group.
• target_label: service: This is where the captured value will be placed. We’re creating a new label called service.
• action: replace: This action means "take the captured value from source_labels and put it into target_label."

Then, the next rule in the sequence for service_name comes into play:

• source_labels: [service_name]
• regex: .: This regex matches any single character. It’s a simple way to say "if the service_name label exists and has any value at all…"
• action: labeldrop: This action removes the source_labels from the metric.

So, for http_requests_total{service_name="frontend", instance="192.168.1.10:8080"}, the first rule copies "frontend" to a new service label, resulting in http_requests_total{service_name="frontend", instance="192.168.1.10:8080", service="frontend"}. The second rule then sees service_name exists and drops it, leaving http_requests_total{instance="192.168.1.10:8080", service="frontend"}. The same logic applies to the app label.

Rule 3 (Dropping IP Instances)

• source_labels: [instance]: We’re looking at the instance label.
• regex: (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(:\d+)?: This regex specifically matches an IPv4 address (e.g., 192.168.1.10) optionally followed by a colon and port number (e.g., :8080).
• action: labeldrop: If the instance label matches this IP address pattern, it’s removed.

So, http_requests_total{instance="192.168.1.10:8080", service="frontend"} becomes http_requests_total{service="frontend"}.

Rule 4 (Fallback Service)

• source_labels: [service]: We check if a service label already exists after the previous rules.
• regex: ^$: This regex matches an empty string. It’s a common way to check if a label is missing or empty after previous transformations.
• target_label: service: We’re still aiming to populate the service label.
• replacement: unknown: This is the value to use if the regex matches (i.e., if service is empty or missing).
• action: replace: This action replaces the (empty) service label with "unknown".

This rule ensures that even if a metric comes in without service_name or app (like our node_exporter example), it will still get a service label, preventing potential issues with queries that expect it.

After applying these rules, our metrics would look like this:

From 192.168.1.10:8080: http_requests_total{service="frontend"} 12345

From 192.168.1.20:8080: http_requests_total{service="backend"} 67890

From 192.168.1.30:9100 (assuming node_exporter doesn’t send service_name or app): http_requests_total{service="unknown"} <value>

The order of metric_relabel_configs matters. Prometheus processes them sequentially. If an action is keep or drop, subsequent rules are not processed for that metric. replace and labelmap continue processing.

There’s also action: keep which drops all metrics except those that match the source_labels and regex. Similarly, action: drop drops any metric that matches.

The regex field is powerful. You can use named capture groups like (?P<name>...) which can then be referenced in replacement strings using %{name}. This allows for complex string manipulation.

The most surprising thing is how action: replace can be used with regex: ^$ to effectively provide default values for labels that might otherwise be missing. It’s a common pattern to ensure label consistency across your entire metric landscape.

The next thing you’ll likely run into is needing to manipulate labels based on the scrape job itself, which is where relabel_configs (applied before scraping) comes into play.