The core idea behind per-user rate limiting is that resource consumption should be tied to individual users, not just to the overall system load.
Let’s see this in action. Imagine a simple API service where each user gets a certain number of requests per minute.
{
"user_id": "user-123",
"timestamp": "2023-10-27T10:00:01Z",
"request_type": "GET /data"
}
If user-123 makes 10 requests in one minute, and their limit is 5 requests per minute, the 6th request onwards will be rejected.
{
"status": 429,
"error": "Too Many Requests",
"message": "You have exceeded your rate limit. Please try again later."
}
This prevents a single "noisy" user from monopolizing shared resources like database connections, CPU cycles, or network bandwidth, ensuring a more equitable experience for everyone.
Internally, this typically involves a distributed key-value store like Redis. Each user ID acts as a key. The value associated with that key is a counter and a timestamp. When a request comes in for user-abc:
- Check the counter: Is the current request count for
user-abcwithin the defined limit for the current time window? - Increment and update timestamp: If within limits, increment the counter and update the timestamp associated with the counter.
- Reject if over limit: If the counter exceeds the limit, reject the request with a
429 Too Many Requestsstatus code.
The most common implementation pattern uses the "sliding window" algorithm. This algorithm tracks requests within a rolling time window (e.g., the last 60 seconds). It’s more sophisticated than a fixed "per-minute" counter because it avoids "bursts" at the window edge. For example, if a user makes 5 requests at 00:59 and another 5 at 01:00, a fixed window would count them as 10 requests in the first minute, potentially exceeding a limit of 5. A sliding window would only count the requests within the actual last 60 seconds, making it more accurate.
The configuration for such a system often looks like this in a gateway or proxy:
rate_limit:
rules:
- match:
user_id: "*" # Apply to all users
limit: 100 # Max requests
period: 60s # Per second
policy:
type: "user" # Key the limit by user ID
Here, user_id: "*" indicates that this rule applies to all users. limit: 100 means a user can make at most 100 requests. period: 60s defines the time window. policy.type: "user" is the critical part, telling the system to use the user_id from the request as the unique identifier for rate limiting.
The period can be granular. While 60s is common, you might see 1m, 1h, or even 1s for very high-throughput scenarios. The limit is the hard cap. It’s crucial to set these values based on observed traffic patterns and the capacity of your downstream services. A limit too low causes user frustration; a limit too high defeats the purpose of rate limiting.
When you’re debugging, you’ll often find yourself looking at logs that show 429 responses. These logs should ideally include the user ID and the timestamp of the request, which helps you pinpoint which user is hitting the limit and when. Tools like Prometheus with a Redis exporter can give you real-time insights into the current request counts per user. For instance, a Prometheus query might look like:
rate(redis_commands_total{command="incr"}[1m])
This shows the rate of INCR commands (which is what happens when a request is processed and the counter is incremented) over the last minute, which can be a proxy for request volume. You’d then filter this by user ID if your Redis metrics are tagged appropriately.
The underlying storage mechanism for these counters is critical for performance. Redis is popular because it offers atomic operations (like INCR and EXPIRE combined) and low latency. Using a single Redis instance for all rate limiting can become a bottleneck. For large-scale systems, consider sharding your Redis cluster by user ID to distribute the load. Each shard would manage the rate limits for a subset of your users.
A common pitfall is how to handle users who aren’t authenticated or don’t have a user_id readily available. In such cases, systems often fall back to a global rate limit or use an IP address as the key. However, IP addresses are less ideal for per-user fairness, as multiple users can share a single IP (e.g., in corporate networks or public Wi-Fi). A more robust solution involves using API keys or session tokens that are directly tied to a user account.
The distinction between "requests per second" and "requests per minute" is more than just a unit of time; it impacts how the sliding window behaves. A "requests per second" limit enforced by a sliding window means that at any given second, the number of requests made within that second cannot exceed the limit. This is much tighter than a "requests per minute" limit, which allows for bursts as long as the total over the minute is respected.
The Retry-After header is a standard part of the 429 response. It tells the client how long, in seconds, they should wait before retrying. For example:
Retry-After: 15
This is implemented by setting an expiration time on the counter in Redis. When the counter expires, it’s automatically removed, and the user can start making requests again. The duration of this expiration is often tied to the period configured in the rate limiting rule.
You’ll discover that managing rate limiting across different environments (development, staging, production) requires careful consideration of limits. Development environments often need higher or no limits to facilitate quick iteration, while production needs strict enforcement. This often means having different configuration files or environment variables for rate limiting rules per environment.
The next logical step after implementing per-user rate limiting is to consider tiered rate limiting, where different classes of users (e.g., free vs. premium) get different limits.