GitHub Actions is a bit like having a virtual machine that lives in GitHub, ready to run code whenever you push changes. When you want to run your Python tests with pytest in this environment, you’re essentially telling GitHub to spin up a machine, install your code, set up Python, and then execute pytest.
Here’s a pytest test file we can use for demonstration:
# tests/test_example.py
def test_addition():
assert 1 + 1 == 2
def test_subtraction():
assert 5 - 3 == 2
Now, let’s set up a GitHub Actions workflow to run these tests. In your GitHub repository, create a directory named .github/workflows/ and inside it, a file named test.yml.
# .github/workflows/test.yml
name: Pytest Automation
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
- name: Test with pytest
run: pytest
When you push this file and your test file to your repository, GitHub Actions will automatically trigger this workflow. The on: [push] line means it runs on every push. The runs-on: ubuntu-latest specifies that the workflow will execute on a fresh Ubuntu Linux virtual machine.
The steps section details the sequence of actions. actions/checkout@v3 downloads your repository’s code onto the virtual machine. actions/setup-python@v3 is a convenient action that installs a specified Python version (here, 3.10).
The Install dependencies step ensures that pytest is available. We upgrade pip first, which is good practice, and then install pytest. The Test with pytest step is where the magic happens: it simply executes the pytest command. By default, pytest will discover and run all tests in files named test_*.py or *_test.py in your project.
If your tests pass, the workflow will show a green checkmark. If any test fails, it will show a red 'X', and you can click on the workflow run to see the detailed output, including the specific test that failed and the error message.
The most surprising thing about pytest in GitHub Actions is how easily it integrates with different Python versions and operating systems without you needing to manage any infrastructure. You can define multiple jobs within a single workflow, each running on a different OS or Python version, to ensure your code works everywhere. For example, you could add another job to test against Python 3.9 on macos-latest.
# .github/workflows/test.yml (expanded)
name: Pytest Automation
on: [push]
jobs:
build-ubuntu:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
- name: Test with pytest
run: pytest
build-macos:
runs-on: macos-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
uses: actions/setup-python@v3
with:
python-version: "3.9"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
- name: Test with pytest
run: pytest
This expanded workflow will run your tests on both Ubuntu (with Python 3.10) and macOS (with Python 3.9) concurrently. The jobs key allows you to define independent tasks, and you can specify dependencies between them using needs if one job must complete before another.
A common pattern is to install your project’s dependencies using pip install . or pip install -r requirements.txt before installing pytest, especially if your tests rely on your own package. This ensures that your tests are run against the actual code as it would be installed.
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
pip install . # Or pip install -r requirements.txt
The run commands in GitHub Actions are executed in a shell. This means you can chain commands with && for sequential execution or use | to run multiple commands in a single shell instance, as shown in the Install dependencies step. You can also use environment variables within these commands, which are exposed by GitHub Actions.
When pytest runs, it looks for tests in a specific way. By default, it finds files named test_*.py or *_test.py and then functions or methods within those files that start with test_. You can customize this discovery behavior using pytest command-line options or configuration files like pytest.ini. For example, to run only tests in a specific file, you could change the run: pytest command to run: pytest tests/test_specific.py.
Beyond basic execution, you can configure GitHub Actions to cache dependencies, upload test artifacts (like coverage reports), and even trigger workflows based on pull requests. For instance, to cache your Python dependencies and speed up subsequent runs, you’d add a caching step.
- name: Cache pip dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
pip install -r requirements.txt
This caching action saves the pip cache directory (~/.cache/pip) and creates a cache key based on the operating system and a hash of your requirements.txt file. If a matching cache is found on a subsequent run, it will be restored, significantly speeding up the installation phase.
The next step in mastering pytest with GitHub Actions is exploring how to generate and upload test reports, such as HTML reports or JUnit XML files, which provide much richer insights into your test suite’s performance and failures.