Run tests for your Python package#

Running your tests across different Python versions and operating systems is critical to ensuring your package works for your users. Your users may be running different versions of Python and operating systems than you are.

This page teaches you how to run tests locally in isolated environments and across multiple Python versions. You’ll learn about two main automation tools: Hatch and Nox. In the next lesson, you will learn about running your tests online in continuous integration (CI).

Why run tests across multiple environments?#

When you develop a package on your computer, it works in one specific environment: your Python version, your operating system, and your installed dependencies. Your users, however, will run your code in many different environments. By running your tests across multiple Python versions and operating systems, you catch compatibility issues before users do.

Additionally, running tests in isolated environments ensures that your tests pass because of your code, not because of unexpected dependencies installed on your computer. This gives you confidence that your package will work when others install it.

On this page, you will learn about the tools that you can use to both run tests in isolated environments and across Python versions.

Tools to run your tests#

There are three categories of tools that will make it easier to setup and run your tests in various environments:

Testing framework (pytest): Provides the syntax and tools for writing and running your tests. Learn more from the pytest documentation. Below you will learn about pytest, the most commonly used testing framework in the scientific Python ecosystem. Testing frameworks are essential for running tests, but they don’t provide an easy way to run tests across Python versions or in isolated environments—that’s where automation tools come in.
Automation tools (Nox, Tox, Hatch): Allow you to run tests in isolated environments and across multiple Python versions with a single command. We focus on Hatch and Nox below. These tools create virtual environments automatically and ensure your tests run consistently. However, they typically only test on your local operating system.
Continuous Integration (CI): Runs your tests online across different operating systems (Windows, Mac, and Linux) and Python versions. CI integrates with platforms like GitHub Actions to automatically test every pull request and code change.

Learn about CI here.

Quick comparison: what each tool does#

Testing Framework (pytest):

Runs your tests locally in your current Python environment
Provides the core syntax for writing tests (assertions, fixtures, etc.)
Can be extended with plugins (like pytest-cov for coverage)

Automation Tools (Nox, Tox, Hatch):

Run tests locally across multiple Python versions
Create and manage isolated virtual environments automatically
Can automate other tasks like building documentation
Make it easy to reproduce test environments

Continuous Integration (GitHub Actions):

Runs tests online automatically for every pull request
Tests across different operating systems (Windows, Mac, Linux)
Tests across multiple Python versions in parallel
Can automate deployments, releases, and other workflows

What testing framework / package should I use to run tests?#

We recommend using Pytest to build and run your package tests. Pytest is the most common testing tool used in the Python ecosystem.

The Pytest package also has a number of extensions that can be used to add functionality such as:

pytest-cov allows you to analyze the code coverage of your package during your tests, and generates a report that you can upload to codecov.

Note

Your editor or IDE may add additional convenience for running tests, setting breakpoints, and toggling the –no-cov flag. Check your editor’s documentation for more information.

Run tests using pytest#

If you are using pytest, you can run your tests locally by calling:

pytest

Or if you want to run a specific test file - let’s call this file “test_module.py” - you can run:

pytest test_module.py

Learn more about pytest here.

Running pytest on your computer is going to run your tests in whatever Python environment you currently have activated. This means that tests will be run on a single version of Python and only on the operating system that you are running locally.

An automation tool can simplify the process of running tests in various Python environments.

Tests across operating systems

If you want to run your tests on different operating systems you can use continuous integration. Learn more here.

Tools to automate running your tests#

To run tests on various Python versions or in various specific environments with a single command, you can use an automation tool such as nox or tox. Both nox and tox can create an isolated virtual environments. This allows you to easily run your tests in multiple environments and across Python versions.

We will focus on Hatch on this page as Hatch is the default tool that we use in our tutorials and for our Python package template.

If you are not a hatch fan, then Nox is an alternative tool that we cover in the next lesson. nox is a Python-based automation tool that builds upon the features of both make and tox. nox is designed to simplify and streamline testing and development workflows. Everything that you do with nox can be implemented using a Python-based interface. You will learn more about using nox here.

Other automation tools you’ll see in the wild

Tox is an automation tool that supports common steps such as building documentation, running tests across various versions of Python, and more.
Make is a build automation tool that some developers use for running tests due to its versatility. However, Make’s unique syntax can be challenging to learn, and it won’t manage environments for you like Hatch and Nox do.

Run tests with Hatch#

Hatch is a modern Python packaging and environment manager that integrates test running capabilities directly into your pyproject.toml. Unlike Nox (which uses a separate noxfile.py), Hatch keeps all your project configuration in one place, making it ideal if you’re already using Hatch for packaging workflows.

Why Hatch for testing?#

Configuration lives in pyproject.toml alongside your project metadata
Integrates seamlessly with Hatch’s packaging and build workflows
No separate Python file needed (unlike Nox)
Easy to share standardized test environments across your team

Setting up Hatch environments#

Hatch environments are defined in your pyproject.toml. Rather than duplicating dependencies, use dependency-groups to reference your test dependencies:

[dependency-groups]
tests = [
    "pytest>=7.0",
    "pytest-cov",
]

[tool.hatch.envs.test]
dependency-groups = [
     "tests",
]

[tool.hatch.envs.test.scripts]
run = "pytest {args:--cov=test --cov-report=term-missing --cov-report=xml}"

This approach keeps your test dependencies in one place and avoids duplication. For a complete example, see our packaging template tutorial which shows a full pyproject.toml configuration.

Running tests with Hatch#

Once you’ve defined your test environment, you can run tests with simple commands:

List available environments:

hatch env show

Run pytest in the test environment:

# This command is how tests are run if you use the pyos-package-template
hatch run test:run

Testing across Python versions#

To test across multiple Python versions, define a matrix in your pyproject.toml:

[dependency-groups]
tests = [
    "pytest>=7.0",
    "pytest-cov",
]

[tool.hatch.envs.test]
dependency-groups = [
     "tests",
]

[[tool.hatch.envs.test.matrix]]
python = ["3.10", "3.11", "3.12"]

Then run all versions with a single command:

hatch run test:run

Hatch will automatically run your tests on Python 3.10, 3.11, and 3.12. If you only want to test a specific Python version:

hatch run test.py3.11:run

Using Hatch in GitHub Actions#

Hatch integrates well with CI/CD. Here’s a minimal GitHub Actions setup:

name: Run tests

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@8c6903cd8c0fde910a37f88322edcfb5dd907a8 # v4
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          python -m pip install hatch
      - run: hatch run test:run

Since all of your test dependencies are declared in the dependency-group table of your pyproject.toml, your CI environment is reproducible and consistent with the environments that you are using for local testing.

Nox vs Hatch: choosing the right tool#

Both Hatch and Nox are excellent automation tools / task runners for running tests across Python versions. Here’s how they compare to help you decide which fits your workflow:

Hatch#

Configuration: Hatch uses a declarative configuration approach. You tell it what goes into an environment and that empowers Hatch to create the environment for you. All configuration settings live in your pyproject.toml alongside your project metadata
Integration: Hatch is package management tool that also has an integrated automation / task runner. Using Hatch means you are using the same tool for all of your packaging and automation needs.
Learning curve: Easier if you prefer declarative configuration over a code based workflow
packaging scope Hatch is better for simpler workflows that focus on testing and packaging. If you have more complex builds or are creating a non pure python package you might prefer Nox. The scientific Python development guide has more details on this.
Best for: Teams using Hatch for packaging, or those who want standardized configuration in one place

Nox#

Configuration: Python-driven via noxfile.py for maximum flexibility
Customization: Great for complex workflows that need custom logic
Learning curve: Easier if you already know Python and want flexible session control
Best for: Complex automation needs, building docs alongside tests, or workflows that don’t fit the standard model

Next steps#

Now that you understand how to run tests locally across Python versions, you can learn about running tests automatically in GitHub Actions with continuous integration. You can also review test types and write tests for your package.