Rancher Logging Stack: Ship Logs to Elasticsearch or Loki (2026)

The Rancher Logging stack allows you to collect, aggregate, and route logs from your Kubernetes clusters to various backends like Elasticsearch or Loki.

Here’s a look at the logging stack in action, collecting logs from a simple Nginx deployment and sending them to Loki.

Scenario: A Kubernetes cluster with a few pods running an Nginx application. We want to capture stdout logs from these Nginx pods and send them to a Loki instance for centralized storage and analysis.

Setup:

1. Rancher UI: Navigate to Cluster Explorer -> Apps -> Chart Repositories. Add the rancher-charts repository if it’s not already present.
2. Install Logging App: Go to Apps -> Marketplace. Search for "Rancher Logging" and install it. During installation, you’ll configure the common.defaultCluster and common.clusterName parameters to match your cluster.
3. Configure Log Output Plugin: Once the logging app is installed, navigate to Cluster Explorer -> Logging -> Configurations. Click "Create".
   • Name: loki-output
   • Output Plugin: Select Loki.
   • Loki URL: http://loki.rancher-monitoring.svc.cluster.local:3100 (This assumes Loki is deployed in the rancher-monitoring namespace, which is common when installing the Rancher Monitoring stack).
   • Client Settings:
     • Batch Size: 100
     • Batch Wait: 1s
   • Click "Create".

Log Collection in Action:

Now, let’s deploy a simple Nginx application to generate some logs.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

Apply this deployment: kubectl apply -f nginx-deployment.yaml

Within a minute or two, you should start seeing logs from the nginx-app pods appearing in your Loki instance. You can access Loki via its UI (if exposed) or query it using tools like promtail or the Grafana Loki datasource.

Mental Model Breakdown:

The Rancher Logging stack is built around Fluentd, a powerful open-source data collector. When you install the logging app, it deploys a DaemonSet on your cluster. This DaemonSet runs a Fluentd agent (often as a sidecar or a node-level agent) on each Kubernetes node.

1. Log Collection: The Fluentd agent, configured via the Rancher UI, watches for log files from your pods. By default, it targets /var/log/containers/*.log. It uses a tail input plugin to continuously read new log entries.
2. Log Parsing & Filtering: Fluentd can parse these log lines (often JSON) and enrich them with Kubernetes metadata (pod name, namespace, labels, etc.). You can define filters to include or exclude specific logs based on namespace, labels, or even log content.
3. Log Routing (Output Plugins): This is where you specify where the logs go. The Rancher Logging stack provides pre-configured output plugins for Elasticsearch, Loki, Kafka, and others. You configure the connection details (URL, authentication, etc.) for your chosen backend.
4. Buffering: To ensure log durability, Fluentd can buffer logs to disk if the output destination is temporarily unavailable. This prevents log loss during network glitches or backend downtime.

Key Configuration Levers:

• logging.cattle.io/v1 API: Rancher exposes logging configurations through custom resources. You’ll interact with ClusterLoggings and Loggings resources.
  • ClusterLogging: Configures cluster-wide logging, including the output plugins that apply to all namespaces by default.
  • Logging: Configures logging for specific namespaces, allowing you to override or add to the cluster-wide settings.
• Filters: You can define filters within your ClusterLogging or Logging resources. These are crucial for selective log shipping.
  • namespace: Target specific namespaces.
  • labelSelector: Target pods with specific labels.
  • container: Target specific containers within a pod.
  • regex: Filter logs based on regular expressions matching the log content.
• Output Plugins: The core of routing. You define connection details for your chosen backend.
  • Elasticsearch: elasticsearch.urls, elasticsearch.index_name.
  • Loki: loki.url, loki.tenant_id.
• Spec.Flush: Controls how often Fluentd attempts to send logs.
  • @type: "flush"
  • interval: 5s (How often to check if there are logs to flush)
  • max_bytes: 1M (Maximum buffer size before flushing)
  • max_num: 100 (Maximum number of records before flushing)

The most surprising thing about the Rancher Logging stack is its reliance on Fluentd’s underlying plugin architecture. While Rancher abstracts much of the complexity, understanding the fluentd.conf configuration that Rancher generates behind the scenes can unlock advanced filtering and routing scenarios that aren’t immediately obvious through the UI. For instance, you can define custom parser plugins to handle non-standard log formats or filter plugins that perform complex data transformations before logs are sent to the backend.

The next step after successfully shipping logs is often setting up alerting based on log patterns or implementing log retention policies.