Playwright’s automatic retries are not about making flaky tests pass, but about ensuring that transient environmental issues don’t derail your CI/CD pipeline.
Let’s watch a test fail and then succeed after a retry.
Here’s a simple test that sometimes fails due to a brief network hiccup when fetching an element’s text:
import pytest
from playwright.sync_api import sync_playwright
def test_flaky_example():
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("https://playwright.dev")
# Simulate a transient failure
try:
assert page.locator("text=Playwright").text_content(timeout=1000) == "Playwright"
except Exception as e:
# In a real scenario, this exception might be due to a brief network issue
# or a DOM element not being immediately ready.
print(f"\nAttempt failed: {e}")
raise # Re-raise to fail the test if it's not a retryable condition
finally:
browser.close()
Now, let’s configure Playwright to retry this test. We’ll use pytest for this example and set the playwright_retries fixture.
In your pytest.ini file, add the following:
[pytest]
playwright_retries = 2
With this configuration, if test_flaky_example fails, pytest-playwright will automatically re-run it up to 2 more times.
Here’s the same test, but now it’s wrapped by the retry mechanism:
import pytest
from playwright.sync_api import sync_playwright
# The 'playwright_retries' fixture is automatically available if pytest-playwright is installed
# and configured in pytest.ini. No explicit import is needed here for the fixture itself.
def test_flaky_example_with_retries(playwright_retries):
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.goto("https://playwright.dev")
# Simulate a transient failure
try:
# The timeout here is intentionally short to increase the chance of a transient failure
assert page.locator("text=Playwright").text_content(timeout=1000) == "Playwright"
except Exception as e:
print(f"\nAttempt failed: {e}")
# Playwright retries will handle re-running the test if it fails
# due to an exception within the test function.
raise
finally:
browser.close()
When you run pytest, you’ll see output indicating retries if the test fails initially. The playwright_retries = 2 in pytest.ini means the test will run a maximum of 3 times in total (1 initial run + 2 retries).
The core problem retries solve is non-determinism in your test environment. This can manifest as:
- Intermittent Network Glitches: A brief packet loss or latency spike between your test runner and the browser, or between the browser and the application under test. This can cause
page.goto(),locator.click(), orlocator.text_content()to fail with timeouts or connection errors. - Race Conditions in Application Load: The application under test might be slow to initialize certain components or APIs on a particular test run. This leads to elements not being present or interactive when Playwright tries to interact with them, resulting in
TimeoutErrororElementNotFoundError. - CI/CD Environment Instability: Shared CI runners can experience temporary resource contention, slow disk I/O, or brief network disruptions that affect test execution. Retries give the test a chance to run on a more stable snapshot of the environment.
- Browser/Driver Instability: Less common, but the underlying browser process or WebDriver communication can sometimes hit transient issues.
- External Service Dependencies: If your application relies on external APIs (e.g., a payment gateway mock, a third-party data service) that are experiencing temporary slowness or unavailability, your tests might fail.
To diagnose a retryable failure, look for common Playwright exceptions like TimeoutError, ProtocolError, or AssertionError related to element states. The key is that the failure is transient – it resolves itself if the test is simply given another chance to execute.
The fix is simply setting playwright_retries in your pytest.ini to a value greater than 0. For instance, playwright_retries = 3 would mean 1 initial attempt plus 3 retries, for a total of 4 runs.
The underlying mechanism is that pytest-playwright intercepts test failures. If the failure is due to a condition it deems retryable (typically exceptions like TimeoutError during Playwright operations), and the retry count hasn’t been exhausted, it will re-execute the test function. This happens seamlessly from your test code’s perspective; you just write your test as if it will pass on the first try.
The most surprising aspect of Playwright retries is how often they mask deeper architectural issues. While they are excellent for CI stability against ephemeral environmental flakiness, a test that consistently requires retries to pass is likely a sign of an unstable application component or an unreliable test setup, rather than just a flaky network. The retry count serves as a blunt instrument; if a test fails on its 3rd or 4th attempt consistently, it’s almost certainly not an intermittent environmental problem anymore.
The next step after configuring retries is to explore how to selectively retry tests or to implement custom retry logic for specific failure conditions.