Rancher audit logging is actually a distributed system where events are generated by multiple Rancher components and then aggregated by a central logging service.
Let’s see this in action. Imagine a user, alice, who is an administrator. She decides to create a new Kubernetes cluster.
First, she logs into the Rancher UI. The UI itself is a web application, and when she clicks the "Create Cluster" button, it doesn’t directly create the cluster. Instead, it sends an API request to the Rancher API server.
The Rancher API server receives this request. Before it even considers creating the cluster, its audit logging middleware intercepts the request. It records the crucial details: alice’s username, the timestamp, the HTTP method (POST), the requested URL (/v3/cluster), and a representation of the request body containing the cluster configuration. This event is then immediately sent to the configured audit log collector.
Concurrently, the Rancher API server processes the request to create the cluster. This involves interacting with other Rancher components, like the cluster provisioning controller.
Meanwhile, the audit log collector, which is typically running as a separate process or within a dedicated logging pod in your Kubernetes cluster (often managed by Rancher itself), receives this event. It might then forward this event to a persistent storage backend like Elasticsearch, Splunk, or even a simple file.
If alice later decides to delete that cluster, the same process repeats. The UI sends a DELETE request to the API server. The audit middleware captures this event – alice, timestamp, DELETE, /v3/cluster/<cluster_id>, etc. – and sends it to the collector. The API server then proceeds to tear down the cluster.
The core problem audit logging solves is providing an irrefutable, auditable trail of who did what, when, and to which resource within Rancher. This is critical for security, compliance, and debugging. Without it, you’d have no way to definitively know who modified a specific cluster setting, who granted access to a project, or who deleted a critical workload.
Internally, Rancher’s audit logging works by leveraging Kubernetes’ own audit logging mechanisms where applicable, but also by implementing its own middleware within the Rancher API server. When a request hits the API server, a dedicated handler examines it and generates an audit event. This event is structured JSON containing fields like userPrincipal, verb, requestURI, requestBody, responseBody (for success/failure), sourceIPs, and userAgent.
These events are then sent to a configured endpoint. This endpoint is typically a local listener within the Rancher deployment that forwards events to a backend. The configuration for this endpoint is crucial. You’ll often find it specified in the Rancher Helm chart values or within the Rancher deployment configuration itself, pointing to the desired logging backend.
The levers you control are primarily the destination of these logs and their level of detail. You can configure where the audit events are sent (e.g., to a specific Elasticsearch index, a syslog server, or a file path). You can also, in some cases, tune the verbosity, deciding whether to log request bodies, response bodies, or just metadata.
The most surprising thing about Rancher audit logging is that the events themselves are not typically stored by the Rancher API server long-term. The API server’s primary job is to process API requests, not to act as a persistent log store. It generates the events and pushes them out to a separate logging system. If that downstream logging system fails or is misconfigured, the audit events are lost forever, even if the Rancher API server itself is healthy.
The next concept you’ll likely encounter is how to effectively search and visualize these audit logs to gain meaningful insights, often involving the configuration of dashboards in your chosen logging backend.