The N+1 query problem isn’t just about inefficient database calls; it’s a symptom of your ORM (Object-Relational Mapper) actively working against your application’s performance by fetching related data too eagerly, then too lazily, in a loop.

Let’s see this in action. Imagine a simple User model with a has_many relationship to Posts.

# models/user.rb
class User < ApplicationRecord
  has_many :posts
end

# models/post.rb
class Post < ApplicationRecord
  belongs_to :user
end

Now, if you want to display all users and the count of their posts:

# controller
@users = User.all

# view
<% @users.each do |user| %>
  <%= user.name %> - Posts: <%= user.posts.count %>
<% end %>

This innocent-looking code triggers the N+1 problem. The ORM first executes SELECT * FROM users (1 query). Then, for each user in the result set, it executes SELECT COUNT(*) FROM posts WHERE user_id = ? (N queries, where N is the number of users). If you have 100 users, that’s 101 queries!

The core issue is that the user.posts.count call within the loop triggers a separate database query for each User object. The ORM, by default, doesn’t "know" you’ll need the posts for every user when it initially fetches the users. It fetches the User objects, and only when user.posts (or user.posts.count) is accessed for the first time for a given user does it go to the database to fetch those specific posts.

Diagnosis:

The easiest way to spot this is by enabling SQL query logging in your application. In Rails, you can do this by setting config.log_level = :debug in your config/environments/development.rb. When you run the code that exhibits the N+1, you’ll see a repetitive pattern of identical or very similar SELECT statements for the related records.

Fixing the N+1 Problem:

The solution is to tell your ORM upfront that you’ll need the related data for all the objects. This is typically done using "eager loading."

  1. includes: This is the most common and often the best approach. includes tells the ORM to fetch the primary records and their associations in as few queries as possible. It intelligently decides whether to use a LEFT OUTER JOIN or separate queries based on what’s being requested.

    # controller
    @users = User.includes(:posts) # Eager load posts for all users
    
    # view (no change needed!)
    <% @users.each do |user| %>
      <%= user.name %> - Posts: <%= user.posts.count %>
    <% end %>
    

    Why it works: With includes(:posts), Rails will typically generate a single query using a LEFT OUTER JOIN:

    SELECT "users".* FROM "users"
    LEFT OUTER JOIN "posts" ON "posts"."user_id" = "users"."id"
    

    Then, it efficiently associates the posts with their respective users in memory. When user.posts.count is called, Rails can count the associated posts already loaded in memory without hitting the database again. If you were fetching user.posts.size instead of user.posts.count, Rails would use a COUNT(*) query for the association, still only running one additional query for all users.

  2. preload: This method always uses separate queries for the primary records and their associations. It’s useful when you have complex associations or when includes might default to an inefficient join.

    # controller
    @users = User.preload(:posts)
    
    # view (no change needed!)
    <% @users.each do |user| %>
      <%= user.name %> - Posts: <%= user.posts.count %>
    <% end %>
    

    Why it works: preload generates two queries: one for users and one for all associated posts for those users.

    SELECT "users".* FROM "users";
    SELECT "posts".* FROM "posts" WHERE "posts"."user_id" IN (user_id_1, user_id_2, ...);
    

    This is often more efficient than a join if you’re only selecting a few columns from the posts or if the join would be very complex.

  3. eager_load: This method always uses a LEFT OUTER JOIN to fetch the primary records and their associations in a single query. It’s similar to includes but forces the join behavior.

    # controller
    @users = User.eager_load(:posts)
    
    # view (no change needed!)
    <% @users.each do |user| %>
      <%= user.name %> - Posts: <%= user.posts.count %>
    <% end %>
    

    Why it works: Like the includes example with a join, this fetches everything in one go.

    SELECT "users"."id" AS t0_id, "users"."name" AS t0_name, ..., "posts"."id" AS t1_id, ...
    FROM "users"
    LEFT OUTER JOIN "posts" ON "posts"."user_id" = "users"."id"
    

    This can be less memory-efficient if you have many associated records, as Rails has to de-duplicate the main records.

When to use which:

  • includes: Your go-to. Let Rails decide the best strategy (join vs. separate queries).
  • preload: Use when includes might generate a bad join, or you explicitly want separate queries. Good for counting or when you don’t need all columns from the association.
  • eager_load: Use when you always want a single query with a join, perhaps for complex filtering or ordering across associations.

Advanced Scenario: Counting within Eager Loading:

If you need to count associations and filter by that count, includes can sometimes still result in N+1 if you’re not careful, because user.posts.count will be called on the in-memory collection, not the database. To count directly in the database when eager loading, you can use includes with a counter cache, or more directly, use joins and group.

# To count posts per user efficiently in one query:
@users = User.joins(:posts).select('users.*, COUNT(posts.id) as posts_count').group('users.id')

# view
<% @users.each do |user| %>
  <%= user.name %> - Posts: <%= user.posts_count %>
<% end %>

Why it works: joins(:posts) creates an INNER JOIN (or LEFT OUTER JOIN if specified) between users and posts. select('users.*, COUNT(posts.id) as posts_count') tells the database to count the posts for each user. group('users.id') ensures the count is per user. The posts_count attribute is then available directly on the User object.

Common Pitfalls:

  • Nested includes: User.includes(posts: :comments) is great. But User.includes(:posts).includes(:groups) is redundant; the second includes will be ignored. Chain them: User.includes(:posts, :groups).
  • where after includes: If you use where after includes on the association, it might not behave as expected. For example, User.includes(:posts).where(posts: { published: true }) might still issue separate queries or not filter correctly. Use joins for filtering on associations: User.joins(:posts).where(posts: { published: true }). However, this will exclude users with no posts. To include users with no posts but filter those that do have published posts, you’d need something more complex like User.left_outer_joins(:posts).where("posts.id IS NULL OR posts.published = true").
  • Loading specific fields: If you only need user.name and user.email, but user.posts.count, includes(:posts) will still fetch all columns for posts. To optimize, you might need to select specific columns or use pluck.

The next hurdle you’ll likely encounter after solving N+1 is understanding how to optimize complex queries involving multiple joins and eager loading scenarios, especially when dealing with polymorphic associations or custom scopes.

Want structured learning?

Take the full Performance course →