RabbitMQ doesn’t actually have "priority queues" in the way you might expect; instead, it allows you to assign priorities to individual messages within a standard queue.
Let’s see this in action. We’ll set up a direct exchange, bind two queues to it with different priority levels, and then publish messages with varying priorities.
import pika
import sys
# Connection parameters
params = pika.ConnectionParameters('localhost')
connection = pika.BlockingConnection(params)
channel = connection.channel()
# Declare an exchange
channel.exchange_declare(exchange='priority_exchange', exchange_type='direct')
# Declare two queues, one for high priority, one for low
# The 'x-max-priority' argument is key here. It tells RabbitMQ
# that this queue supports message priorities. The value is the
# maximum priority level allowed (e.g., 10 means priorities 0-10).
channel.queue_declare(queue='high_priority_queue', durable=True, arguments={'x-max-priority': 5})
channel.queue_declare(queue='low_priority_queue', durable=True, arguments={'x-max-priority': 5})
# Bind queues to the exchange
channel.queue_bind(exchange='priority_exchange', queue='high_priority_queue', routing_key='high')
channel.queue_bind(exchange='priority_exchange', queue='low_priority_queue', routing_key='low')
print("Queues and exchange declared. Publishing messages...")
# Publish messages with different priorities
# The 'priority' attribute is set in the message properties.
channel.basic_publish(
exchange='priority_exchange',
routing_key='high', # This routing key will go to high_priority_queue
body='This is a very important message (priority 5)!',
properties=pika.BasicProperties(delivery_mode=2, priority=5) # delivery_mode=2 makes message persistent
)
channel.basic_publish(
exchange='priority_exchange',
routing_key='low',
body='This is a less important message (priority 1)!',
properties=pika.BasicProperties(delivery_mode=2, priority=1)
)
channel.basic_publish(
exchange='priority_exchange',
routing_key='high',
body='Another important message (priority 4)!',
properties=pika.BasicProperties(delivery_mode=2, priority=4)
)
channel.basic_publish(
exchange='priority_exchange',
routing_key='low',
body='Another less important message (priority 1)!',
properties=pika.BasicProperties(delivery_mode=2, priority=1)
)
channel.basic_publish(
exchange='priority_exchange',
routing_key='high',
body='Yet another important message (priority 5)!',
properties=pika.BasicProperties(delivery_mode=2, priority=5)
)
print("Messages published. Check your consumers.")
connection.close()
When a consumer connects to high_priority_queue and starts consuming, it will receive messages in descending order of priority. Messages with the same priority will be delivered in the order they were enqueued.
The core problem this solves is ensuring that critical messages are processed before less urgent ones, especially in systems where processing time varies significantly or where timely delivery of certain events is paramount. Think of it like an emergency room: critical patients are seen before those with minor ailments, even if the minor ailment arrived first.
Internally, RabbitMQ maintains a separate list for each priority level within a queue that has x-max-priority configured. When a message arrives, it’s placed into the list corresponding to its priority. When a consumer requests a message, RabbitMQ scans these lists from the highest priority down to the lowest, delivering the first available message it finds. If a consumer requests a message and the highest priority list is empty, it checks the next highest, and so on.
The primary lever you control is the x-max-priority argument when declaring a queue. This defines the upper bound of priority levels your queue will support. Any priority assigned to a message that exceeds this maximum will be capped at the maximum. Priorities are integers, and by convention, higher numbers indicate higher priority. The default priority for messages not explicitly assigned a priority is 0.
It’s crucial to understand that priorities are per-queue. If you have multiple queues, RabbitMQ will not prioritize messages across queues. A message in a low-priority queue might be delivered before a message in a high-priority queue if the consumer is connected to the low-priority queue. You need to ensure consumers are configured to connect to the appropriate priority-enabled queues.
When you declare a queue with x-max-priority, RabbitMQ internally creates a set of internal queues, one for each priority level up to the maximum specified. A message with priority N is published to the internal queue for priority N. The main queue then acts as a sort of "fan-out" or aggregator for these internal priority queues, presenting them to consumers in the correct order. This internal structure is why the x-max-priority argument is essential for the queue itself to understand and manage priorities.
The next concept you’ll likely encounter is how to handle consumers that need to acknowledge messages in a specific order, especially when dealing with priorities and potential network interruptions.