RabbitMQ’s memory alarm is preventing new messages from being published because the broker is close to exceeding its configured memory limit.

The Problem: The broker’s memory usage has hit a critical threshold, triggering a flow-blocked state to protect itself from running out of memory and crashing. This means publishers will receive basic.nack or basic.return responses, and new messages won’t be accepted until memory usage drops.

Here are the common culprits and how to fix them:

  1. Unacknowledged Messages: Large numbers of unacknowledged messages consume memory for message bodies and delivery tracking.

    • Diagnosis: Connect to the RabbitMQ management UI (usually http://localhost:15672). Navigate to Queues. Look for queues with a high number of Unacked messages. You can also use the rabbitmqctl command:
      rabbitmqctl list_queues name messages_unacknowledged
      
    • Fix: Ensure your consumers are properly acknowledging messages. If there’s a legitimate backlog, scale up your consumers or investigate why they are slow. For a quick (but potentially data-losing) purge of an unacknowledged queue:
      rabbitmqctl purge_queue <queue_name>
      
      This command removes all messages from the specified queue, including those that are unacknowledged. It works by iterating through the queue’s message store and discarding each message.
    • Why it works: Removing unacknowledged messages frees up memory that RabbitMQ was holding onto to track delivery status and potentially redeliver messages.
  2. Large Message Bodies: Publishing very large individual messages can quickly exhaust memory.

    • Diagnosis: This is harder to diagnose directly with rabbitmqctl for individual messages. However, if you see a sudden spike in memory usage corresponding to a publishing burst, and your unacknowledged message count isn’t the primary driver, large messages are suspect. Check your application logs for indications of large payloads being sent.
    • Fix: Optimize message payloads. Compress data before sending, break large messages into smaller ones, or reconsider if RabbitMQ is the right tool for transmitting very large binary blobs. For example, if you’re sending images, store them in object storage and send a URL in RabbitMQ.
    • Why it works: Reducing the size of each message directly decreases the memory footprint per message stored or in transit.
  3. Memory Leaks in Application/Plugins: A bug in a custom plugin or in the application logic publishing/consuming messages can lead to memory continuously accumulating.

    • Diagnosis: Monitor RabbitMQ’s memory usage over time using the management UI or rabbitmqctl status. If memory steadily climbs and never returns to a baseline even after message processing, a leak is likely. Check application logs for any unusual errors or resource exhaustion warnings.
    • Fix: Identify and fix the leak in your application code or custom plugins. This often requires profiling your application’s memory usage. For built-in plugins, ensure they are up-to-date.
    • Why it works: Eliminating the leak prevents the continuous, unreleased consumption of memory by errant processes.
  4. Insufficient Erlang VM Memory Limit: The default Erlang VM (BEAM) memory limit for RabbitMQ might be too low for your workload.

    • Diagnosis: Check the vm_memory_high_watermark setting in your rabbitmq.conf or rabbitmq-env.conf file. The default is often 0.4 (40% of total system RAM). If your total system RAM is low, this limit might be too restrictive.
    • Fix: Increase the vm_memory_high_watermark in your RabbitMQ configuration. For example, to set it to 60% of system RAM:
      # In rabbitmq.conf
      vm_memory_high_watermark.relative = 0.6
      
      Then restart RabbitMQ:
      sudo systemctl restart rabbitmq-server
      
      This setting tells the Erlang VM to start garbage collection more aggressively or, in this case, to trigger the flow-control mechanism when 60% of available memory is used.
  5. High Number of Queues/Exchanges/Connections: While less direct, a massive number of these objects can consume memory for their internal metadata.

    • Diagnosis: Use rabbitmqctl list_queues, rabbitmqctl list_exchanges, and rabbitmqctl list_connections to get counts.
    • Fix: Consolidate queues where possible (e.g., using topic exchanges with routing keys that fan out to fewer queues). Review your application’s connection management – are you opening and closing connections unnecessarily? Consider using connection pooling.
    • Why it works: Reducing the number of managed objects decreases the overhead RabbitMQ incurs for tracking and managing them.
  6. Large Number of Non-Durable Messages: If you have many transient (non-durable) messages that are not being consumed quickly, they still occupy memory.

    • Diagnosis: Check queue definitions in the management UI or rabbitmqctl list_queues for durability flags. The durable column should ideally be true for critical queues.
    • Fix: Make your critical queues durable. This means they will survive broker restarts. For transient messages that need to be processed, ensure consumers are keeping up. If they are truly transient and can be lost on restart, ensure your consumers are robust enough.
    • Why it works: While durability primarily affects disk persistence, the internal management of message state still has a memory component. However, the main impact here is ensuring that non-durable messages are truly transient and processed quickly, or making queues durable if their contents must persist.
  7. Memory Fragmentation: In rare cases, the Erlang VM’s memory allocator can lead to fragmentation, where available memory is broken into small, unusable chunks.

    • Diagnosis: This is difficult to diagnose directly without deep Erlang VM introspection. High memory usage that doesn’t correlate with message counts or obvious leaks, especially after long uptime, could suggest this.
    • Fix: The most common fix is to restart RabbitMQ. This forces the Erlang VM to reallocate its memory from scratch.
    • Why it works: Restarting the broker effectively clears the Erlang VM’s memory space, including any fragmented portions, and allows it to allocate memory more contiguously upon startup.

After applying these fixes, you should see the memory alarm clear and publishers regain the ability to send messages. The next common issue you might encounter is a channel_error if you’ve aggressively purged queues or if consumers are still struggling to keep up.

Want structured learning?

Take the full Rabbitmq course →