Pulsar Reader API: Replay Messages from Any Offset (2026)

Pulsar’s Reader API, designed for sequential consumption, can actually jump back and replay messages from any arbitrary offset within a topic partition, a capability often overlooked.

Let’s see this in action. Imagine we have a Pulsar topic persistent://public/default/my-topic with a partition 0. We’ve consumed some messages, and now we want to re-process a specific message.

import org.apache.pulsar.client.api.*;

public class ReplayExample {

    public static void main(String[] args) throws Exception {
        PulsarClient client = PulsarClient.builder()
                .serviceUrl("pulsar://localhost:6650")
                .build();

        String topic = "persistent://public/default/my-topic";
        int partition = 0;

        // Get the current latest message ID (or any message ID you're interested in)
        // In a real scenario, you'd likely store these IDs or derive them.
        // For this example, let's assume we want to replay from a known message ID.
        // We'll simulate getting a message ID.
        MessageId targetMessageId = MessageId.latest; // Placeholder: Replace with actual MessageId

        // Create a reader at a specific message ID
        Reader<String> reader = client.newReader(Schema.STRING)
                .topic(topic + "-partition-" + partition)
                .startMessageId(targetMessageId)
                .create();

        System.out.println("Starting replay from message ID: " + targetMessageId);

        int messagesToReplay = 5;
        int count = 0;
        while (reader.hasMessage() && count < messagesToReplay) {
            Message<String> msg = reader.readNext();
            System.out.println("Replayed message: " + msg.getValue() + " at ID: " + msg.getMessageId());
            count++;
        }

        reader.close();
        client.close();
    }
}

This code snippet demonstrates how to create a Reader instance and explicitly set its starting point using startMessageId(). The targetMessageId can be any valid MessageId you’ve obtained previously. This could be from a stored log, a timestamp-based lookup (though Pulsar doesn’t directly support timestamp-to-ID mapping, you’d need to find an ID near that timestamp), or simply the MessageId.latest to start from the most recent message.

The core problem Pulsar’s Reader API solves is providing a cursor-like interface for topics without the complexities of consumer groups and acknowledgments. You get a stream of messages from a specified point. The "replay" functionality is a natural extension of this: instead of starting at the beginning of a topic or the latest message, you can instruct the reader to begin at any MessageId. This is invaluable for scenarios like:

• Data Reprocessing: Fixing bugs in consumer logic or re-calculating aggregates.
• Testing and Debugging: Replaying specific message sequences to isolate issues.
• Auditing: Replaying messages for compliance or verification purposes.

Internally, when you create a Reader with startMessageId(someId), Pulsar locates the segment file containing someId and seeks to that specific entry within the segment. It then begins delivering messages from that point onwards. Unlike a standard consumer, the Reader doesn’t manage cursors or acknowledgments in a topic subscription. It’s a stateless cursor for a specific topic partition.

The MessageId itself is a composite value, typically ledgerId:entryId:partitionId. When you provide a MessageId to startMessageId(), Pulsar uses this information to efficiently find the exact message. If you want to replay from a point before a specific message, you would typically find the MessageId of the message before the one you want to start replaying from. Pulsar’s MessageId ordering is strictly chronological.

A common point of confusion is how to get the MessageId to start from. While you can’t directly query for a MessageId by timestamp, you can:

1. Store MessageIds as you consume them.
2. Use MessageId.earliest to start from the very beginning.
3. Use MessageId.latest to start from the most recent message.
4. If you have a MessageId from a previous read operation (e.g., in logs or a database), you can reuse it.

The startMessageId parameter is crucial. If you provide a MessageId that doesn’t exist or is out of range for the partition, the Reader might fail to initialize or start from the nearest available message, depending on the exact Pulsar version and configuration. Always ensure the MessageId you provide is valid for the target topic partition.

When replaying, remember that the Reader does not acknowledge messages. If you were using this to reprocess data that a consumer group would normally handle, you’d need a strategy to prevent that consumer group from re-consuming the same messages. You might temporarily disable the consumer group, or use a separate topic for replayed messages.

The next challenge you’ll likely encounter is efficiently finding the correct MessageId for a given point in time or an event, which often involves external systems or custom logic to correlate events with Pulsar MessageIds.