The most surprising thing about pip in production is that it’s fundamentally not designed for it, and the standard way people use it for reproducible installs is actually a ticking time bomb.
Here’s a common scenario: you’ve got a web application, it’s working fine on your development machine. You pip freeze > requirements.txt and deploy. Weeks later, you redeploy, and suddenly your app is broken. Why? Because pip freeze only records what is installed, not which specific versions were resolved by pip’s dependency resolver at that exact moment. The next time you install from requirements.txt, pip might pull down newer, incompatible versions of your dependencies.
Let’s see what this looks like in practice. Imagine you have a simple setup.py:
from setuptools import setup, find_packages
setup(
name='my_app',
version='0.1.0',
packages=find_packages(),
install_requires=[
'flask==2.0.0',
'requests<2.28.0',
],
)
And a requirements.txt generated by pip freeze from a working environment:
click==8.0.3
Flask==2.0.0
itsdangerous==2.0.1
Jinja2==3.0.3
MarkupSafe==2.0.1
requests==2.27.1
Werkzeug==2.0.2
Now, you deploy this requirements.txt. Everything is fine. A month later, you need to deploy an update. You run pip install -r requirements.txt. pip sees requests<2.28.0 in your setup.py (or implicitly from a previous pip install that resolved it), but it also sees requests==2.27.1 in requirements.txt. It’s happy with both. However, pip might also notice that there’s a newer version of Flask available (say, 2.1.0) that still satisfies Flask==2.0.0 (because pip’s version specifiers are more flexible than you might think). If Flask==2.1.0 has a subtly different dependency tree, or if a new dependency like redis is added to the system and pip needs to resolve its version, it might pull in requests==2.28.0 because that’s the latest version compatible with the new Flask==2.1.0 and the new redis version. Suddenly, your application breaks because Flask==2.0.0 (which you explicitly wanted) doesn’t work with requests==2.28.0.
The mental model here is that pip is a package installer, not a deterministic build system. When you run pip install, it consults PyPI, checks version constraints, and resolves a compatible set of packages. pip freeze just dumps what’s currently installed. It doesn’t capture the resolution process.
To achieve reproducible installs, you need to capture the resolved state, not just the installed state. This is where pip-tools comes in. You use pip-compile to generate a fully pinned requirements.txt.
First, you create a requirements.in file:
Flask==2.0.0
requests<2.28.0
Then, you run pip-compile requirements.in. This command resolves the entire dependency graph and produces a requirements.txt that looks like this:
#
# This file is autogenerated by pip-compile with Python 3.9
# To update, run:
#
# pip-compile requirements.in
#
click==8.0.3
Flask==2.0.0
itsdangerous==2.0.1
Jinja2==3.0.3
MarkupSafe==2.0.1
requests==2.27.1
Werkzeug==2.0.2
Notice how every single package is pinned to an exact version. Now, when you deploy and run pip install -r requirements.txt, pip will install exactly these versions, no more, no less. The dependency resolver has already done its work once, and you’re just replaying that exact outcome.
The lever you control is the requirements.in file. This is your source of truth for the direct dependencies and their high-level constraints. pip-compile is the tool that translates these constraints into a concrete, reproducible set.
One thing most people don’t know is that pip-compile can also handle transitive dependencies explicitly. If you know that a specific version of a transitive dependency is causing issues, you can add it to requirements.in with a specific version. For example, if you discover that requests==2.27.1 is problematic and you need requests==2.27.1 but with a specific patch for a bug, you could add requests==2.27.1 to your requirements.in. pip-compile will then respect that specific version, even if a newer compatible version exists, and it will ensure that all other dependencies are resolved against that pinned version.
The next concept you’ll run into is managing multiple environments (like dev, staging, prod) with different dependency sets.