pytest-rerun failures is a plugin that can automatically re-run tests that fail due to transient issues, often referred to as "flaky" tests.
Let’s see it in action. Imagine you have a test that sometimes fails because a network resource isn’t immediately available.
# test_flaky.py
import pytest
import random
def test_sometimes_fails():
if random.random() < 0.3: # 30% chance of failure
assert False, "This test failed randomly!"
assert True
Normally, you’d see this:
$ pytest test_flaky.py
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-7.4.0, pluggy-1.3.0
rootdir: /path/to/your/project
plugins: rerunfailures-11.0
collected 1 item
test_flaky.py F [100%]
=================================== FAILURES ===================================
_________________________________ test_sometimes_fails _________________________________
def test_sometimes_fails():
if random.random() < 0.3: # 30% chance of failure
> assert False, "This test failed randomly!"
E AssertionError: This test failed randomly!
test_flaky.py:7: AssertionError
=========================== short test summary info ============================
FAILED test_flaky.py::test_sometimes_fails - AssertionError: This test failed randomly!
============================== 1 failed in 0.05s ===============================
Now, let’s tell pytest-rerun-failures to retry failed tests up to 3 times. We install it first:
pip install pytest-rerun-failures
Then, we run pytest with the --reruns flag:
pytest --reruns 3 test_flaky.py
If the test fails on its first attempt, pytest will automatically re-run it. It will keep running it up to the specified number of times until it passes or all retries are exhausted. You’ll see output like this if it eventually passes:
$ pytest --reruns 3 test_flaky.py
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-7.4.0, pluggy-1.3.0
rootdir: /path/to/your/project
plugins: rerunfailures-11.0
collected 1 item
test_flaky.py R [100%]
============================== 1 passed in 0.12s ===============================
The R indicates that the test was rerun. If it had failed all 3 times, you’d see F and the final failure report.
The core problem pytest-rerun-failures solves is that many tests fail not because of a bug in the code under test, but due to external, non-deterministic factors. Think of tests that interact with databases, networks, or external APIs. These dependencies can be temporarily unavailable, slow, or return inconsistent results. Instead of manually re-running the entire test suite, which is tedious and error-prone, this plugin automates the process. It isolates the flaky tests and gives them another chance to pass without human intervention.
Internally, pytest-rerun-failures hooks into pytest’s collection and execution phases. When a test fails, it doesn’t immediately mark it as a failure for the entire session. Instead, it captures the failure and schedules a re-run for that specific test. It maintains a count of how many times each test has been rerun. This process repeats until either the test passes or the configured maximum number of reruns is reached. The plugin is designed to be lightweight and integrates seamlessly with pytest’s existing reporting mechanisms, meaning you don’t need to learn a new way to interpret test results.
You can configure the number of reruns globally using the --reruns command-line option, or you can apply it to specific tests using a marker.
# test_flaky_marker.py
import pytest
import random
@pytest.mark.rerun(3)
def test_flaky_with_marker():
if random.random() < 0.3: # 30% chance of failure
assert False, "This test failed randomly!"
assert True
Running pytest test_flaky_marker.py will now automatically attempt to rerun test_flaky_with_marker up to 3 times upon failure, without needing the --reruns flag on the command line. This allows for fine-grained control over which tests benefit from the retry mechanism.
One mechanism that’s particularly useful is the ability to specify different rerun counts for different types of failures. By default, --reruns N applies to any failure. However, you can target specific exceptions. For instance, if you want to retry only tests that raise ConnectionError but not other exceptions, you can use the --rerun-exceptions option. However, the plugin’s documentation is a bit sparse on its exact syntax for this, and it’s often simpler to use the @pytest.mark.rerun(N) marker on tests that are known to be susceptible to specific transient issues, rather than trying to configure exception-based retries globally.
The next common challenge you’ll face is dealing with tests that fail due to race conditions in concurrent or parallel test execution.