Rancher on Kubernetes is surprisingly easy to get wrong, and the most common failure point is a subtle misconfiguration of its internal etcd cluster, leading to cascading failures that are hard to debug.
Let’s get Rancher running in a High Availability (HA) setup using Helm, which is the standard way to manage applications on Kubernetes. This means if one Rancher pod goes down, another one immediately takes over, ensuring your management plane stays available.
First, ensure you have a Kubernetes cluster already running. This could be a managed service like EKS, GKE, AKS, or a self-hosted cluster. You’ll also need kubectl configured to talk to your cluster and Helm installed.
We’ll be using Helm to deploy Rancher. Add the Rancher Helm repository:
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
helm repo update
Now, create a namespace for Rancher:
kubectl create namespace cattle-system
Before we install, we need to create a Kubernetes Secret that holds your Rancher SSL certificate and private key. For production, you should use a valid certificate signed by a trusted Certificate Authority (CA). You can generate a self-signed certificate for testing, but it’s not recommended for production.
Let’s assume you have your certificate file (tls.crt) and private key file (tls.key) ready. Create the secret like this:
kubectl -n cattle-system create secret tls rancher-tls-secret --cert=./tls.crt --key=./tls.key
Now, we can install Rancher using Helm. We need to specify some critical parameters for HA:
hostname: This should be the DNS name that users will use to access Rancher. It must resolve to the external IP or LoadBalancer service of your Rancher deployment.replicas: Set this to 3 for HA.ingress.enabled: Set totrueif you want Rancher to manage its own Ingress, orfalseif you are managing Ingress externally. For simplicity here, we’ll enable it.privateCA: If you’re using a private CA, you’ll need to provide its certificate here. For most production setups with a public CA, you can omit this.
Here’s the values.yaml file you’ll use with Helm:
hostname: rancher.yourdomain.com
replicas: 3
ingress:
enabled: true
tls:
Source: secret
And the Helm install command:
helm install rancher rancher-latest/rancher \
--namespace cattle-system \
-f values.yaml
This command deploys Rancher into your cluster. The replicas: 3 setting ensures that three Rancher pods are running. Kubernetes will automatically manage these pods, restarting them if they fail and ensuring that if one node goes down, the other Rancher pods remain available. The ingress.enabled: true along with hostname and tls.Source: secret tells Rancher to create an Ingress resource that exposes it externally via the rancher.yourdomain.com hostname using the TLS certificate stored in cattle-system/rancher-tls-secret.
After a few minutes, you should be able to access Rancher at https://rancher.yourdomain.com.
The most surprising thing about Rancher’s HA setup is how tightly coupled its internal etcd cluster is with the Rancher application pods. While Kubernetes handles the pod scheduling and restarts, the etcd cluster itself needs to maintain quorum. If more than half of the etcd members become unavailable, the cluster will halt all operations, including Rancher’s ability to manage resources. This is why setting replicas: 3 is crucial; it ensures that even if one etcd member (and thus one Rancher pod) fails, the remaining two can still maintain quorum.
Crucially, you need to ensure that the hostname you configure is resolvable from outside your cluster and that your Kubernetes cluster’s ingress controller (or the one Rancher creates) is correctly configured to route traffic to the Rancher pods. If your DNS isn’t set up or your ingress isn’t pointing to the right service, you won’t be able to reach Rancher, even if all the pods are running perfectly.
The next thing you’ll likely encounter is configuring the initial administrator password, which is stored in a Kubernetes secret and can be retrieved using kubectl get secret --namespace cattle-system bootstrap-secret -o jsonpath='{.data.bootstrapPassword}' | base64 --decode.