pytest’s slowest tests are often not what you think, and optimizing them requires looking beyond simple time.sleep() calls.
Let’s see pytest’s profiling in action. Imagine a simple test file, test_slow.py:
import time
import pytest
def test_fast():
time.sleep(0.05) # 50ms
def test_medium():
time.sleep(0.2) # 200ms
@pytest.mark.parametrize("i", range(10))
def test_many_fast(i):
time.sleep(0.03) # 30ms
def test_slow_setup():
time.sleep(0.1) # 100ms setup
assert True
def test_slow_teardown():
assert True
time.sleep(0.1) # 100ms teardown
If we run this with standard pytest, it takes about 400ms (give or take network latency if these were actual network calls). But where is that time really going? We need to profile.
The pytest-xdist plugin, primarily known for parallel execution, also has a powerful profiling capability. Install it:
pip install pytest-xdist
Now, run pytest with the --durations=0 flag. This tells pytest to report the duration of all tests, sorted by duration.
pytest --durations=0 test_slow.py
The output will look something like this:
============================= slowest 5 durations ==============================
0.21s call test_slow.py::test_medium
0.11s call test_slow.py::test_slow_setup
0.11s teardown test_slow.py::test_slow_teardown
0.06s call test_slow.py::test_fast
0.03s call test_slow.py::test_many_fast[9]
0.03s call test_slow.py::test_many_fast[8]
0.03s call test_slow.py::test_many_fast[7]
...
This is good, but it only shows the total time per test. What if test_medium is slow because of its setup, not the test body itself? This is where pytest-cov (for coverage) and pytest-xdist’s profiling come together, or more directly, the --trace flag combined with pytest-profiling.
Install pytest-profiling:
pip install pytest-profiling
Now, run with --profile:
pytest --profile test_slow.py
This generates a profile.prof file. To make sense of it, we use snakeviz:
pip install snakeviz
snakeviz profile.prof
This opens a browser window with an interactive flame graph or icicle plot. You’ll see that test_medium consumes the most CPU time. But the real insight comes from understanding how pytest executes tests and where the overhead is.
The core problem pytest solves is test execution management. It discovers tests, loads them, runs them, and reports results. Slowdowns can occur at any of these stages, but most commonly they are within the test function itself, its fixtures, or external dependencies.
The pytest-profiling tool, when run with --profile, actually instruments your tests. It uses Python’s built-in cProfile module to record function call times. The output profile.prof is a raw profile data file. snakeviz then visualizes this data, showing you which functions consumed the most time during the test run.
Here’s the mental model:
- Discovery:
pytestfinds your tests (files namedtest_*.pyor*_test.py, functions namedtest_*). - Collection: It analyzes these tests, identifies fixtures, parameters, and dependencies.
- Execution: For each test item, it:
- Resolves and executes any required fixtures (setup phase).
- Runs the test function itself.
- Executes teardown code for fixtures (teardown phase).
- Reporting: Gathers results and presents them.
Slowdowns are typically in step 3. pytest --profile helps you pinpoint which functions within step 3 are the culprits.
The key levers you control are:
- Test Function Logic: The actual code you write in your
test_*.pyfiles. This is the most common place for performance issues. - Fixture Logic: Setup and teardown code in your fixtures. Complex fixtures can significantly impact suite runtime.
- External Dependencies: Network calls, database interactions, file I/O. These are often the biggest bottlenecks.
- Parameterization: While powerful, heavily parameterized tests can add up.
test_many_fastin our example takes 10 * 30ms = 300ms, which is a substantial chunk of the total.
The one thing most people don’t know is that pytest’s fixture teardown runs even if the test function itself fails. This is a critical feature for resource cleanup, but it means that a slow teardown in a fixture can mask an actual test failure’s duration, or simply add unnecessary time to every test that uses that fixture, regardless of test outcome. If test_slow_teardown was a fixture used by test_fast and test_medium, its 100ms teardown time would be added to both tests’ execution time, even if test_fast and test_medium passed quickly.
The next concept to explore is how to effectively mock external dependencies to speed up tests without compromising their integrity.