pytest-codspeed

Pytest plugin to create CodSpeed benchmarks

---

Documentation: https://codspeed.io/docs/reference/pytest-codspeed

---

Installation

pip install pytest-codspeed

Usage

Creating benchmarks

In a nutshell, pytest-codspeed offers two approaches to create performance benchmarks that integrate seamlessly with your existing test suite.

Use @pytest.mark.benchmark to measure entire test functions automatically:

importpytestfromstatisticsimportmedian@pytest.mark.benchmarkdeftest_median_performance():
input= [1, 2, 3, 4, 5]
output=sum(i**2foriininput)
assertoutput==55

Since this measure the entire function, you might want to use the benchmark fixture for precise control over what code gets measured:

deftest_mean_performance(benchmark):
data= [1, 2, 3, 4, 5]
# Only the function call is measuredresult=benchmark(lambda: sum(i**2foriindata))
assertresult==55

Check out the full documentation for more details.

Testing the benchmarks locally

If you want to run the benchmarks tests locally, you can use the --codspeed pytest flag:

$ pytest tests/ --codspeed
============================= test session starts ====================
platform darwin -- Python 3.14.0, pytest-7.4.4, pluggy-1.5.0
codspeed: 4.2.0 (enabled, mode: walltime, timer_resolution: 41.7ns)
rootdir: /home/user/codspeed-test, configfile: pytest.ini
plugins: codspeed-4.2.0
collected 1 items
tests/test_sum_squares.py . [ 100%]
Benchmark Results
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┓
┃ Benchmark ┃ Time (best) ┃ Rel. StdDev ┃ Run time ┃ Iters ┃
┣━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━╋━━━━━━━━━━━━━╋━━━━━━━━━━╋━━━━━━━━┫
┃test_sum_squares┃ 1,873ns ┃ 4.8% ┃ 3.00s ┃ 66,930 ┃
┗━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━┻━━━━━━━━━━━━━┻━━━━━━━━━━┻━━━━━━━━┛
=============================== 1 benchmarked ========================
=============================== 1 passed in 4.12s ====================

Running the benchmarks in your CI

You can use the CodSpeedHQ/action to run the benchmarks in Github Actions and upload the results to CodSpeed.

Here is an example of a GitHub Actions workflow that runs the benchmarks and reports the results to CodSpeed on every push to the main branch and every pull request:

name: CodSpeedon:
push:
branches:
- "main"# or "master"pull_request: # required to have reports on PRs# `workflow_dispatch` allows CodSpeed to trigger backtest# performance analysis in order to generate initial data.workflow_dispatch:
jobs:
benchmarks:
name: Run benchmarksruns-on: ubuntu-lateststeps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6with:
python-version: "3.14"
- name: Install dependenciesrun: pip install -r requirements.txt
- name: Run benchmarksuses: CodSpeedHQ/action@v4with:
mode: simulation # or `walltime`token: ${{ secrets.CODSPEED_TOKEN }}run: pytest tests/ --codspeed