The pip resolver doesn’t just find a way to install your dependencies; it finds the best way, and "best" means minimizing the number of times you have to deal with it again.
Let’s see it in action. Imagine you have a requirements.txt file like this:
requests==2.25.0
urllib3<1.26
And you try to install it with an older pip (pre-20.3):
pip install -r requirements.txt
Pip might happily install requests==2.25.0 and then discover that requests==2.25.0 requires urllib3==1.26.0. But your requirements.txt explicitly says urllib3<1.26. Boom. Conflict. Pip would tell you about the conflict, and you’d have to manually figure out which version to downgrade or upgrade.
Now, with pip’s new resolver (v2, introduced in pip 20.3), here’s what happens:
pip install -r requirements.txt
Pip starts by looking at requests==2.25.0. It knows this version needs urllib3==1.26.0. It then checks your requirements.txt and sees urllib3<1.26. Pip immediately recognizes this as a conflict. Instead of just installing and then failing, it re-evaluates. It knows requests==2.25.0 cannot be installed with urllib3<1.26.
The new resolver operates on a concept of "dependency sets." When you request a package, pip doesn’t just grab the latest version; it looks at the entire set of packages required by that version. It then tries to find a version of all required packages that satisfies all constraints, including those from your requirements.txt and from other dependencies.
Think of it like a highly organized librarian. You ask for "Book A, edition 2.25" and "Book B, edition less than 1.26." The librarian knows "Book A, edition 2.25" requires "Book C, edition 1.26." They also know you’ve specified "Book C, edition less than 1.26." Instead of giving you "Book A" and then telling you the librarian can’t fulfill the "Book C" requirement, they’ll say, "I can’t give you Book A, edition 2.25, because it conflicts with your requirement for Book C."
The core problem pip v2 solves is pre-resolution of conflicts. Older pip versions would install packages greedily and then check for conflicts at the end. This often led to a tangled mess where you’d get an error deep into the installation process, and it wasn’t always clear which of your direct or indirect dependencies caused the problem.
The new resolver builds a complete graph of all possible dependency versions that satisfy your initial constraints. It then searches this graph for a "consistent" solution. If it finds one, it installs it. If it cannot find a consistent solution (meaning no combination of package versions satisfies all requirements), it reports the conflict clearly, pointing to the specific packages and versions that are incompatible.
Here’s a simplified view of the process:
- Collect Candidates: Pip gathers all available versions of the packages you’ve requested (directly or indirectly).
- Build Requirement Sets: For each candidate package version, it determines the full set of requirements that version introduces.
- Identify Conflicts: It actively looks for overlaps and contradictions between these requirement sets and your explicit constraints.
- Find a Consistent Solution: Pip tries to find a single version for each package such that all requirements are met. This is an iterative process. If it hits a dead end (a conflict it can’t resolve), it backtracks.
- Install: Once a consistent set of packages is found, pip installs them.
The mental model you should have is that pip is no longer just a package downloader; it’s a constraint solver. You provide the initial constraints (requirements.txt, setup.py dependencies), and pip finds a valid state of your environment.
The most surprising thing about the new resolver is how it handles seemingly simple version pins. If you have package-a==1.0 and package-b==2.0, and package-a==1.0 requires dependency-x>=1.5 while package-b==2.0 requires dependency-x<1.5, the old pip might have installed one, then failed on the other. The new pip will immediately recognize that dependency-x cannot satisfy both package-a==1.0 and package-b==2.0 simultaneously, and will report a conflict between package-a and package-b due to dependency-x. It doesn’t just tell you about the unmet requirement; it tells you which direct dependencies are causing the problem.
The next concept you’ll likely encounter is understanding how to interpret the detailed conflict reports pip generates, especially when dealing with complex dependency chains.