Setting up TLS and SASL authentication for Redpanda might seem like a complex dance of certificates and secrets, but it boils down to ensuring Redpanda can securely identify itself to clients and that clients can securely identify themselves to Redpanda.
Let’s watch Redpanda in action. Imagine a simple producer trying to send messages to a topic. Without TLS/SASL, this is like shouting your secrets across a crowded room.
rpk topic produce my-topic --broker-list localhost:9092
> Hello, world!
> Another message.
Now, let’s secure it.
First, TLS. This is about encrypting the communication channel. Redpanda acts as a server, and clients (producers, consumers, admin tools) act as clients. Both need to trust each other.
-
Generate Certificates: You need a Certificate Authority (CA) to sign your server and client certificates. For testing,
opensslis your friend.# Create a CA openssl genrsa -out ca.key 2048 openssl req -new -x509 -key ca.key -out ca.crt -days 365 -subj "/CN=RedpandaTestCA" # Create a server key and CSR openssl genrsa -out server.key 2048 openssl req -new -key server.key -out server.csr -subj "/CN=localhost" # Sign the server CSR with your CA openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 365 -
Configure Redpanda: You’ll tell Redpanda where to find its certificate and key, and how to handle TLS. This goes into your
redpanda.yamlor command-line arguments.# redpanda.yaml listeners: - name: plaintext port: 9092 ssl_cert_file: /path/to/your/server.crt ssl_key_file: /path/to/your/server.keyAnd you’ll need to tell the client (e.g.,
rpk) to trust your CA.# Place ca.crt in a known location, e.g., ~/.config/redpanda/ca.crt -
Connect with TLS: Now, your producer needs to know to use TLS and trust the CA.
# Ensure ca.crt is in ~/.config/redpanda/ca.crt rpk topic produce my-topic --broker-list localhost:9092 --tls-enabled > Hello, TLS world!Notice the
--tls-enabledflag. Redpanda will now presentserver.crtto the client. The client, usingca.crt, will verify thatserver.crtwas indeed signed by a trusted CA. This prevents man-in-the-middle attacks.
SASL (Simple Authentication and Security Layer) is about who is allowed to connect. It’s like a bouncer at the door.
-
SASL Mechanisms: Redpanda supports PLAIN and SCRAM-SHA-256/512. SCRAM is more secure as it doesn’t transmit credentials in plaintext. Let’s use SCRAM-SHA-256.
-
Create Users and Secrets: You need to define users and their passwords. Redpanda stores these in its security configuration.
# Create a user 'appuser' with password 'supersecret' # This command needs to be run when Redpanda is NOT configured with TLS/SASL enabled yet, # or you can use rpk with appropriate TLS/SASL flags if already configured. rpk user create appuser --password supersecretThis command internally creates a file (usually in Redpanda’s data directory) that maps
appuserto its hashed password. -
Configure Redpanda for SASL: You’ll enable SASL and specify the mechanisms.
# redpanda.yaml listeners: - name: plaintext port: 9092 # ... TLS config if used ... sasl_enabled: true sasl_mechanisms: "SCRAM-SHA-256" # Or "SCRAM-SHA-512", "PLAIN" -
Connect with SASL: The client now needs to provide credentials.
rpk topic produce my-topic \ --broker-list localhost:9092 \ --tls-enabled \ --sasl-enabled \ --sasl-mechanism SCRAM-SHA-256 \ --sasl-username appuser \ --sasl-password supersecret > Hello, SASL/TLS world!When you run this, the client and server perform a handshake specific to SCRAM-SHA-256. The password
supersecretis never sent over the wire directly. Instead, both sides use it to generate cryptographic challenges and responses. If they match, authentication succeeds.
Combining TLS and SASL means both encryption and strong authentication. The --tls-enabled flag ensures the connection is encrypted, and the SASL flags ensure only authorized users can speak over that encrypted channel.
The most surprising true thing about Redpanda’s security setup is that the rpk tool manages user creation and secret storage directly, and it handles the complexities of interacting with the Redpanda API for these operations. You don’t manually edit a separate user database file in most common scenarios.
If you’re using rpk and get an error like "Authentication failed" even though your username and password are correct, it’s often because the client’s system clock is significantly out of sync with the Redpanda broker’s clock, especially for SCRAM mechanisms that rely on time-based challenges.