Run tests for your Python package

Running your tests across different Python versions and operating systems is critical to ensuring your package works for your users. Your users may be running different versions of Python and operating systems than you are.

This page teaches you how to run tests locally in isolated environments and across multiple Python versions. You'll learn about two main automation tools: Hatch and Nox. In the next lesson, you will learn about running your tests online in continuous integration (CI).

Why run tests across multiple environments?

When you develop a package on your computer, it works in one specific environment: your Python version, your operating system, and your installed dependencies. Your users, however, will run your code in many different environments. By running your tests across multiple Python versions and operating systems, you catch compatibility issues before users do.

Additionally, running tests in isolated environments ensures that your tests pass because of your code, not because of unexpected dependencies installed on your computer. This gives you confidence that your package will work when others install it.

このページでは、隔離された環境でテストを実行したり、Python のバージョンを越えてテストを実行したりするために使えるツールについて学びます。

参考

Related pages:

• Write tests for best practices on writing test suites

• Test types to understand unit, integration, and end-to-end tests

• Run tests online with CI for GitHub Actions setup

• Code coverage to measure how much code your tests cover

Tools to run your tests

There are three categories of tools that will make it easier to setup and run your tests in various environments:

1. Testing framework (pytest): Provides the syntax and tools for writing and running your tests. Learn more from the pytest documentation. Below you will learn about pytest, the most commonly used testing framework in the scientific Python ecosystem. Testing frameworks are essential for running tests, but they don't provide an easy way to run tests across Python versions or in isolated environments—that's where automation tools come in.

2. Automation tools (Nox, Tox, Hatch): Allow you to run tests in isolated environments and across multiple Python versions with a single command. We focus on Hatch and Nox below. These tools create virtual environments automatically and ensure your tests run consistently. However, they typically only test on your local operating system.

3. Continuous Integration (CI): Runs your tests online across different operating systems (Windows, Mac, and Linux) and Python versions. CI integrates with platforms like GitHub Actions to automatically test every pull request and code change.

Learn about CI here.

Quick comparison: what each tool does

Testing Framework (pytest):

• Runs your tests locally in your current Python environment

• Provides the core syntax for writing tests (assertions, fixtures, etc.)

• Can be extended with plugins (like pytest-cov for coverage)

Automation Tools (Nox, Tox, Hatch):

• Run tests locally across multiple Python versions

• Create and manage isolated virtual environments automatically

• Can automate other tasks like building documentation

• Make it easy to reproduce test environments

Continuous Integration (GitHub Actions):

• Runs tests online automatically for every pull request

• Tests across different operating systems (Windows, Mac, Linux)

• Tests across multiple Python versions in parallel

• Can automate deployments, releases, and other workflows

テストを実行するには、どのテストフレームワーク/パッケージを使用すればよいですか？

パッケージテストのビルドと実行には Pytest を使うことを推奨します。 Pytest は Python のエコシステムで最もよく使われているテストツールです。

Pytestパッケージ には、以下のような機能を追加するために使用できる多くの拡張機能もあります：

• pytest-cov allows you to analyze the code coverage of your package during your tests, and generates a report that you can upload to codecov.

注釈

お使いのエディタや IDE には、テストを実行したり、ブレークポイントを設定したり、 -no-cov フラグを切り替えたりするための便利な機能が追加されているかもしれません。 詳しくはエディタのドキュメントを参照してください。

pytest を使ってテストを実行する

pytest を使用している場合、テストをローカルで実行することができます:

pytest

また、特定のテストファイル - このファイルを "test_module.py" と呼ぶことにします - を実行したい場合は、次のようにします:

pytest test_module.py

Learn more about pytest here.

あなたのコンピュータで pytest を実行すると、現在有効になっている Python 環境でテストが実行されます。 つまり、テストは単一のバージョンのPythonで、ローカルで実行しているオペレーティングシステム上でのみ実行されます。

自動化ツールは、様々な Python 環境でテストを実行するプロセスを単純化することができます。

オペレーティングシステムをまたいだテスト

If you want to run your tests on different operating systems you can use continuous integration. Learn more here.

テストの実行を自動化するツール

一つのコマンドで様々な Python のバージョンや特定の環境でテストを実行するには、 nox や tox のような自動化ツールを使うことができます。 nox も tox も、隔離された仮想環境を作成することができます。 これにより、複数の環境や Python のバージョンをまたいだテストを簡単に実行することができます。

We will focus on Hatch on this page as Hatch is the default tool that we use in our tutorials and for our Python package template.

If you are not a hatch fan, then Nox is an alternative tool that we cover in the next lesson. nox is a Python-based automation tool that builds upon the features of both make and tox. nox is designed to simplify and streamline testing and development workflows. Everything that you do with nox can be implemented using a Python-based interface. You will learn more about using nox here.

その他の自動化ツール

• Tox は自動化ツールで、ドキュメントのビルドや、Pythonの様々なバージョン間でのテストの実行など、一般的なステップをサポートしています。

• Make is a build automation tool that some developers use for running tests due to its versatility. However, Make's unique syntax can be challenging to learn, and it won't manage environments for you like Hatch and Nox do.

Run tests with Hatch

Hatch is a modern Python packaging and environment manager that integrates test running capabilities directly into your pyproject.toml. Unlike Nox (which uses a separate noxfile.py), Hatch keeps all your project configuration in one place, making it ideal if you're already using Hatch for packaging workflows.

Why Hatch for testing?

• Configuration lives in pyproject.toml alongside your project metadata

• Integrates seamlessly with Hatch's packaging and build workflows

• No separate Python file needed (unlike Nox)

• Easy to share standardized test environments across your team

Setting up Hatch environments

Hatch environments are defined in your pyproject.toml. Rather than duplicating dependencies, use dependency-groups to reference your test dependencies:

[dependency-groups]
tests = [
    "pytest>=7.0",
    "pytest-cov",
]

[tool.hatch.envs.test]
dependency-groups = [
     "tests",
]

[tool.hatch.envs.test.scripts]
run = "pytest {args:--cov=test --cov-report=term-missing --cov-report=xml}"

This approach keeps your test dependencies in one place and avoids duplication. For a complete example, see our packaging template tutorial which shows a full pyproject.toml configuration.

Running tests with Hatch

Once you've defined your test environment, you can run tests with simple commands:

List available environments:

hatch env show

Run pytest in the test environment:

# This command is how tests are run if you use the pyos-package-template
hatch run test:run

Testing across Python versions

To test across multiple Python versions, define a matrix in your pyproject.toml:

[dependency-groups]
tests = [
    "pytest>=7.0",
    "pytest-cov",
]

[tool.hatch.envs.test]
dependency-groups = [
     "tests",
]

[[tool.hatch.envs.test.matrix]]
python = ["3.10", "3.11", "3.12"]

Then run all versions with a single command:

hatch run test:run

Hatch will automatically run your tests on Python 3.10, 3.11, and 3.12. If you only want to test a specific Python version:

hatch run test.py3.11:run

Using Hatch in GitHub Actions

Hatch integrates well with CI/CD. Here's a minimal GitHub Actions setup:

name: Run tests

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@8c6903cd8c0fde910a37f88322edcfb5dd907a8 # v4
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          python -m pip install hatch
      - run: hatch run test:run

Since all of your test dependencies are declared in the dependency-group table of your pyproject.toml, your CI environment is reproducible and consistent with the environments that you are using for local testing.

Nox vs Hatch: choosing the right tool

Both Hatch and Nox are excellent automation tools / task runners for running tests across Python versions. Here's how they compare to help you decide which fits your workflow:

Hatch

• Configuration: Hatch uses a declarative configuration approach. You tell it what goes into an environment and that empowers Hatch to create the environment for you. All configuration settings live in your pyproject.toml alongside your project metadata

• Integration: Hatch is package management tool that also has an integrated automation / task runner. Using Hatch means you are using the same tool for all of your packaging and automation needs.

• Learning curve: Easier if you prefer declarative configuration over a code based workflow

• packaging scope Hatch is better for simpler workflows that focus on testing and packaging. If you have more complex builds or are creating a non pure python package you might prefer Nox. The scientific Python development guide has more details on this.

• Best for: Teams using Hatch for packaging, or those who want standardized configuration in one place

Nox

• Configuration: Python-driven via noxfile.py for maximum flexibility

• Customization: Great for complex workflows that need custom logic

• Learning curve: Easier if you already know Python and want flexible session control

• Best for: Complex automation needs, building docs alongside tests, or workflows that don't fit the standard model

What we recommend

If you're using Hatch for packaging: Use Hatch for testing too. You get everything in one place and one consistent tool.

If you need maximum flexibility: Choose Nox. Its Python-driven approach lets you implement almost any workflow.

If you're just starting out: Start with Hatch. It's simpler to set up and understand, and you can always switch to Nox later if you need to.

Both tools are good choices. For a more comprehensive guide to using Nox, see Run tests with Nox and the Scientific Python testing guide.

Next steps

Now that you understand how to run tests locally across Python versions, you can learn about running tests automatically in GitHub Actions with continuous integration. You can also review test types and write tests for your package.