Pytest’s HTML reporting plugin is surprisingly powerful, but most users only scratch the surface, treating it as a glorified diff viewer instead of a strategic tool for understanding test suite health.
Let’s see it in action. Imagine we have a simple test suite:
# test_math.py
def test_add():
assert 1 + 1 == 2
def test_subtract():
assert 5 - 3 == 2
def test_multiply_fail():
assert 2 * 3 == 7 # This will fail
def test_divide():
assert 10 / 2 == 5
To generate an HTML report, we first need to install the plugin:
pip install pytest-html
Then, we run pytest with the --html flag, specifying a report file name:
pytest --html=report.html
This command executes all tests and generates report.html in the current directory. Open this file in your browser. You’ll see a clean, interactive summary of your test run. It lists each test, its status (passed, failed, skipped), and crucially, provides detailed output for failures, including the assertion error and the values involved.
AssertionError: assert (2 * 3) == 7
+ where (2 * 3) = 6
This is where the plugin starts to shine. It’s not just about seeing if a test failed, but why, with context.
But the real power comes from understanding how this report can inform your development workflow. The core problem it solves is transforming raw, often noisy, test output into a digestible, actionable format. Internally, pytest-html hooks into pytest’s reporting mechanisms. When a test starts, passes, fails, or is skipped, the plugin receives these events. It then aggregates this information and uses an HTML template to render a structured document.
You can customize the report extensively. For instance, adding environment information or custom metadata can make reports more informative. Let’s say you want to include the Python version and the operating system. You can use the pytest_configure hook in your conftest.py:
# conftest.py
import sys
import os
def pytest_configure(config):
config.option.html_environment = True
config.option.html_title = "My Awesome Test Report"
config.option.html_show_environments = True
config.option.html_show_datetime = True
config.option.html_show_metadata = True
# Add custom metadata
config.option.html_extra_metadata = [
("Python Version", sys.version),
("OS", os.name),
("Environment", "Staging")
]
Running pytest --html=report.html again will now include these details directly in the report, often under an "Environment" section. This allows you to quickly identify if test failures are specific to a particular environment or Python version.
The plugin also offers more granular control. You can specify which tests to include or exclude using standard pytest markers and selection options. For example, to only include tests marked with @pytest.mark.smoke, you’d run:
pytest -m smoke --html=report.html
This selective reporting is incredibly useful for focusing on critical paths or specific feature sets during different stages of development.
Here’s a detail that trips many people up: the report.html file itself is a static snapshot. If you run pytest again, it overwrites the file. This means you can’t easily compare test runs within the same report file. To compare runs, you’d typically generate separate reports (e.g., report_v1.html, report_v2.html) and then manually compare them, or use external tools that can diff HTML files. The plugin’s strength is in its detailed view of a single run, not historical trend analysis out of the box.
The next logical step after generating these detailed reports is to integrate them into your CI/CD pipeline, where they become invaluable for automated quality gates and debugging production issues.