Rancher projects don’t inherently limit resource consumption, allowing runaway applications to consume all available cluster capacity.
Let’s see this in action. Imagine we have a project named dev-project where developers are spinning up pods. Without quotas, a few memory-hungry pods could easily exhaust the nodes.
apiVersion: v1
kind: Pod
metadata:
name: memory-hog
namespace: dev-project
spec:
containers:
- name: hog
image: busybox
command: ["sh", "-c", "while true; do echo $(date) >> /dev/null; done"]
resources:
requests:
memory: "2Gi" # Requesting a large amount of memory
limits:
memory: "4Gi" # And even more for limits
If we deploy this memory-hog pod into dev-project and there are no quotas, it will happily start consuming memory. If the nodes can’t satisfy the requests, the pod will remain pending. If it exceeds its limits, it will be OOMKilled. But the core issue is that this single pod could starve other applications in the same project, or even other projects if the cluster is small, by consuming all available node resources.
Rancher Resource Quotas are the mechanism to prevent this. They allow you to set hard limits on the total amount of compute resources a project can consume. This includes CPU, memory, storage, and even the number of objects like pods and services.
Here’s how you’d set up a quota for our dev-project:
apiVersion: v1
kind: ResourceQuota
metadata:
name: dev-project-quota
namespace: dev-project
spec:
hard:
requests.cpu: "4" # Total CPU requests for all pods in this project cannot exceed 4 CPU cores.
requests.memory: "8Gi" # Total memory requests cannot exceed 8 GiB.
limits.cpu: "8" # Total CPU limits cannot exceed 8 CPU cores.
limits.memory: "16Gi" # Total memory limits cannot exceed 16 GiB.
pods: "20" # Maximum of 20 pods allowed in this project.
services: "10" # Maximum of 10 services allowed.
When you apply this ResourceQuota to the dev-project namespace, Kubernetes, managed by Rancher, starts enforcing these limits. Any new pod creation or update that would violate these hard constraints will be rejected.
How it works internally: The ResourceQuota object is a policy enforced by the Kubernetes API server. When a request to create or update a resource (like a Pod) arrives, the API server checks if the namespace associated with that resource has a ResourceQuota object. If it does, it calculates the current resource consumption in that namespace and compares it against the hard limits defined in the ResourceQuota. If the requested change would push the total consumption over any of the limits, the API server returns an error, typically a Forbidden status code with a message indicating the quota violation.
Key levers you control:
requests.cpu/requests.memory: These define the guaranteed resources your project’s pods will receive. When a pod is scheduled, its requests are summed up. If the total requests exceed this limit, the pod won’t be scheduled. This is crucial for ensuring your applications have the baseline resources they need.limits.cpu/limits.memory: These define the maximum resources a pod is allowed to consume. If a pod tries to exceed its limits, it will be throttled (for CPU) or terminated (for memory, OOMKilled). This prevents runaway processes from hogging the entire node.pods/services/replicationcontrollersetc.: These are counts of specific Kubernetes objects. They prevent a project from creating an excessive number of resources, which can strain the Kubernetes control plane and lead to instability.
The most impactful aspect of ResourceQuota is that it operates on aggregated requests and limits across all resources within a namespace. It’s not just about individual pod limits; it’s about the collective resource footprint. This means if you have 10 pods each requesting 0.5 CPU and 1 GiB memory, and your quota requests.cpu is set to 4 and requests.memory to 8Gi, you can only add 10 more pods before hitting the CPU quota and 8 more before hitting the memory quota.
The real power comes from setting both requests and limits. If you only set limits, pods might still be scheduled onto nodes that don’t have enough guaranteed resources, leading to noisy neighbor problems where one pod’s bursty behavior impacts others. By setting requests, you ensure that the sum of requested resources for all pods in the project can be accommodated by the cluster’s capacity.
Once your dev-project has a ResourceQuota in place, attempting to create that memory-hog pod with requests.memory: "2Gi" and limits.memory: "4Gi" would fail if the project has already consumed 7Gi in requests and 10Gi in limits. The error message would clearly state the quota violation, guiding you to adjust either the pod’s resource needs or the project’s quota.
The next thing you’ll likely encounter is the need to differentiate resource needs between projects, leading you to explore ResourceQuota scopes or multiple quotas per namespace.