Python packaging has a few major players, and picking the right one can feel like a minefield. You’ve got pip with requirements.txt, and then there’s Poetry. They both manage your Python project’s dependencies, but they do it in fundamentally different ways, leading to vastly different developer experiences.
Let’s see pip and requirements.txt in action. Imagine you’re starting a new project. You create a requirements.txt file:
flask==2.2.2
requests>=2.28.1
numpy
Then, you install these dependencies using pip:
pip install -r requirements.txt
This is straightforward. pip reads the file and installs the specified packages and their transitive dependencies. If you want to record the exact versions that are currently installed (which is crucial for reproducible builds), you’d run:
pip freeze > requirements.txt
This creates a file like:
click==8.1.3
Flask==2.2.2
itsdangerous==2.1.2
Jinja2==3.1.2
MarkupSafe==2.1.1
numpy==1.23.5
requests==2.28.1
Werkzeug==2.2.2
Now, Poetry. It uses a pyproject.toml file, which is a more structured and comprehensive way to define your project.
[tool.poetry]
name = "my-awesome-project"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.10"
flask = "^2.2.2"
requests = ">=2.28.1"
numpy = "*"
[tool.poetry.group.dev.dependencies]
pytest = "^7.2.0"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
To install these, you’d use:
poetry install
Poetry then creates a poetry.lock file, which pins down the exact versions of all dependencies, including transitive ones, ensuring identical installations across environments.
The core problem Poetry solves is dependency resolution. With pip and requirements.txt, if you have conflicting dependency requirements (e.g., Package A needs lib==1.0, but Package B needs lib==2.0), pip will often fail, or worse, install something that seems to work but will break later. It doesn’t have a sophisticated solver. Poetry, on the other hand, has a robust dependency resolver. When you add a new package, it attempts to find a set of compatible versions for all your dependencies. If it can’t find a solution, it tells you clearly which packages are in conflict.
Here’s how Poetry handles versioning. The ^ symbol (e.g., flask = "^2.2.2") is called "caret-compatible versioning." It means "allow updates that do not change the leftmost non-zero digit." So, ^2.2.2 allows 2.2.3, 2.3.0, but not 3.0.0. The * for numpy means "any version," which is generally discouraged in production but useful for initial exploration. The > symbol is standard semantic versioning.
When you run poetry add flask, Poetry updates your pyproject.toml and then resolves and updates your poetry.lock file. If you run poetry install in a new environment, it reads poetry.lock and installs the exact versions specified there. This makes your builds highly reproducible.
The most surprising thing about Poetry’s approach is how it separates development dependencies from production dependencies. In the pyproject.toml above, pytest is in [tool.poetry.group.dev.dependencies]. When you run poetry install, it only installs production dependencies. To install development dependencies, you’d run poetry install --with dev. This is a significant improvement over pip and requirements.txt, where you often end up with a single, monolithic requirements.txt that includes everything, leading to over-installation in production environments.
The mental model is this: pyproject.toml is your declaration of what your project needs, specifying version constraints. poetry.lock is the guarantee of exactly what is installed, ensuring consistency. pip and requirements.txt often blur these lines, making it harder to manage complex dependency graphs and ensure reproducibility.
If you’re encountering dependency conflicts that pip can’t seem to untangle, or if you’re struggling with inconsistent environments, Poetry is likely your next step.