Longhorn’s primary innovation is its ability to provide distributed, resilient block storage for Kubernetes, essentially making your Kubernetes nodes act like a sophisticated SAN.
Here’s Longhorn in action, managing a persistent volume for an Nginx deployment:
First, we create a PersistentVolumeClaim (PVC) requesting a longhorn storage class.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nginx-pvc
spec:
accessModes:
- ReadWriteOnce
storageClassName: longhorn
resources:
requests:
storage: 1Gi
When Kubernetes provisions this PVC, Longhorn kicks in. It doesn’t just allocate space; it creates a replica of that volume on another node. For a 1Gi volume, Longhorn might create a base image file and then a snapshot, effectively creating a thin-provisioned block device.
Let’s say we have two nodes, node-1 and node-2. If the Nginx pod lands on node-1, Longhorn will create the primary volume replica on node-1 and a secondary replica on node-2. This is managed by the Longhorn manager pods running within the cluster. The nginx-pvc is now bound to a PersistentVolume (PV) that Longhorn created, pointing to this distributed block device.
The Nginx pod then mounts this PV. From Nginx’s perspective, it’s just a standard block device. However, under the hood, Longhorn is intercepting I/O operations. Reads and writes go to the primary replica on node-1. If node-1 fails, Longhorn’s control plane detects this, promotes the replica on node-2 to be the new primary, and the Nginx pod (which will likely be rescheduled to node-2) can resume I/O with minimal disruption.
The problem Longhorn solves is the ephemeral nature of node storage in Kubernetes. Without a distributed storage solution, if a pod requiring persistent storage dies with its data on a specific node, that data is lost. Longhorn makes that data redundant and accessible from any node.
The core components are:
- Longhorn Manager: A Deployment running as pods (usually one per control-plane node) that orchestrates volume creation, replica management, and health checks.
- CSI (Container Storage Interface) Driver: This
csi-driver-nodeDaemonSet runs on every Kubernetes node, translating Kubernetes storage requests into Longhorn API calls. - Instance Manager: A DaemonSet that runs on every node, responsible for managing the actual block device processes (like
qemu-imgfor volume operations) on that node.
You control Longhorn via its Custom Resource Definitions (CRDs): Volume, Replica, Setting, etc. The storageClassName: longhorn in your PVC is the user-facing entry point. Behind the scenes, the CSI driver translates that into creating a Volume CRD, which the Longhorn Manager then uses to spin up Replica CRDs on the chosen nodes.
The most surprising thing about Longhorn is how it achieves its performance and resilience: it uses qemu-img and a custom kernel module (longhorn-dm) to create thin-provisioned, COW (Copy-on-Write) block devices. Each volume is essentially a set of disk image files managed by qemu-img on the node’s local storage. The longhorn-dm module then exposes these as block devices to the pods. Replicas are synchronized over the network using a custom protocol, and data integrity is maintained through checksums and robust snapshotting.
When you set numberOfReplicas to 3, Longhorn doesn’t just create three identical copies of the entire volume immediately. It creates one primary volume and then begins the process of creating the secondary and tertiary replicas. During this initial creation, if a node goes down, you might end up with a degraded volume state until the replica can be rebuilt. The system is designed for eventual consistency and resilience, not immediate, synchronous replication across all nodes upon creation.
The next concept you’ll likely encounter is managing Longhorn backups, either to S3-compatible object storage or NFS, for disaster recovery scenarios beyond node failure.