The Pulsar broker failed to register with ZooKeeper because its configured brokerServiceUrl was not discoverable by other components.

Common Causes and Fixes

  1. Incorrect brokerServiceUrl in broker.conf:

    • Diagnosis: Check the brokerServiceUrl setting in your Pulsar broker’s configuration file (conf/broker.conf).
      grep brokerServiceUrl conf/broker.conf
      
    • Fix: Ensure brokerServiceUrl is set to the IP address and port that other brokers and clients can reach. If your broker is running on 192.168.1.100 and listening on port 1665 for internal communication, it should be:
      brokerServiceUrl=pulsar://192.168.1.100:1665
      
      This allows other Pulsar components to know how to connect to this specific broker instance.
    • Why it works: This explicitly tells the Pulsar cluster how to address this broker for inter-broker communication and client connections.
  2. advertisedAddress or advertisedListeners Misconfiguration:

    • Diagnosis: If brokerServiceUrl is not explicitly set, Pulsar attempts to derive it from advertisedAddress or advertisedListeners. Check these settings in conf/broker.conf.
      grep advertisedAddress conf/broker.conf
      grep advertisedListeners conf/broker.conf
      
    • Fix: If using advertisedAddress, set it to the broker’s publicly accessible IP or hostname:
      advertisedAddress=192.168.1.100
      
      If using advertisedListeners (which is more flexible for different protocols and ports), ensure it includes the correct pulsar:// or pulsar+ssl:// entry for the broker’s service port. For example:
      advertisedListeners=pulsar://192.168.1.100:1665
      
      Then, ensure brokerServiceUrl is either unset or matches this configuration.
    • Why it works: These settings provide a mechanism for the broker to advertise its network identity to the rest of the cluster, especially in dynamic or multi-network environments.
  3. Firewall Blocking Broker Port:

    • Diagnosis: Verify that the brokerServiceUrl port (default 1665) is open on the broker’s host and accessible from other nodes in the cluster. Use telnet or nc from another broker or a client machine.
      telnet 192.168.1.100 1665
      # or
      nc -vz 192.168.1.100 1665
      
    • Fix: Configure your firewall (e.g., firewalld, iptables, or cloud security groups) to allow inbound TCP traffic on port 1665 (or your custom brokerServiceUrl port) from the IP addresses of your other Pulsar nodes and clients.
      • firewalld example:
        sudo firewall-cmd --zone=public --add-port=1665/tcp --permanent
        sudo firewall-cmd --reload
        
      • iptables example:
        sudo iptables -A INPUT -p tcp --dport 1665 -j ACCEPT
        sudo service iptables save
        
    • Why it works: Network connectivity is essential. If other brokers or ZooKeeper cannot reach the brokerServiceUrl, they cannot establish or maintain connections, leading to registration failures.
  4. ZooKeeper Not Reachable or Incorrectly Configured:

    • Diagnosis: While not directly brokerServiceUrl, ZooKeeper is where brokers register. If ZooKeeper is inaccessible, the broker can’t register. Check zookeeperServers in conf/broker.conf.
      grep zookeeperServers conf/broker.conf
      
      Then, test connectivity from the broker node to the ZooKeeper ensemble.
      telnet zk1.example.com 2181
      
    • Fix: Ensure zookeeperServers points to the correct ZooKeeper ensemble addresses and ports. If there’s a network issue, open the ZooKeeper port (default 2181) in the firewall.
      zookeeperServers=zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181
      
    • Why it works: The broker needs to communicate with ZooKeeper to register itself and discover other cluster members. If this communication fails, the broker cannot join the cluster.
  5. Hostname Resolution Issues:

    • Diagnosis: If brokerServiceUrl or advertisedAddress uses hostnames, ensure these hostnames resolve correctly to the broker’s IP address on all nodes in the cluster (brokers, clients, ZooKeeper). Use ping or nslookup.
      ping broker.example.com
      nslookup broker.example.com
      
    • Fix: Correct your DNS records or update the /etc/hosts file on all relevant nodes to map the hostname to the correct IP address. For example, in /etc/hosts:
      192.168.1.100  broker.example.com
      
    • Why it works: Incorrect DNS resolution means other components cannot translate the advertised hostname into a routable IP address, preventing them from connecting to the broker.
  6. Internal Network Routing Problems:

    • Diagnosis: Even if ports are open and DNS is correct, underlying network routes might prevent communication between the broker’s advertised IP and other cluster members. Use traceroute from another node to the broker’s brokerServiceUrl IP.
      traceroute 192.168.1.100
      
    • Fix: Work with your network administrator to correct any routing misconfigurations between the nodes. This might involve updating gateway configurations or network policies.
    • Why it works: Packets need a valid path to travel. If network devices between nodes don’t know how to reach the destination IP, communication will fail silently or with network-level errors.

After fixing the brokerServiceUrl and ensuring network connectivity, you might encounter the next error: "ZooKeeper session expired" if the broker was unable to establish a stable connection to ZooKeeper due to the previous configuration issues.

Want structured learning?

Take the full Pulsar course →