refactor(pathfinder): replace info_summary_append fixture with Python logging by cpcloud · Pull Request #1593 · NVIDIA/cuda-python

cpcloud · 2026-02-10T17:30:05Z

What

Replaces the custom info_summary_append pytest fixture with standard Python logging, writing test diagnostic info to a file uploaded as a GHA artifact instead of printing a terminal summary.

Why

The terminal INFO summary was verbose and noisy in CI output
Grep-based verification of the summary (grep '^INFO test_' in run-tests) was brittle
A file artifact is easier to inspect on-demand without cluttering logs

Changes

conftest.py -- info_summary_append fixture and pytest_terminal_summary hook replaced by two fixtures: a session-scoped _info_summary_handler (manages a logging.FileHandler) and a function-scoped info_log (yields a LoggerAdapter with the test node name). Log file is keyed off CUDA_PATHFINDER_TEST_LOAD_NVIDIA_DYNAMIC_LIB_STRICTNESS env var so both CI runs produce separate files.
3 test files -- info_summary_append(...) calls become info_log.info(...).
ci/tools/run-tests -- Removed tee pipe and grep verification for pathfinder.
Both test-wheel workflows -- Added upload-artifact step (pinned v6.0.0) after the last pathfinder test run with if-no-files-found: error.

Made with Cursor

copy-pr-bot · 2026-02-10T17:30:09Z

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

cpcloud · 2026-02-10T17:32:32Z

/ok to test

cpcloud · 2026-02-10T17:33:44Z

/ok to test

cpcloud · 2026-02-10T17:34:34Z

/ok to test

github-actions · 2026-02-10T17:45:21Z

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-1593/
https://nvidia.github.io/cuda-python/pr-preview/pr-1593/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-1593/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-1593/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

rwgk

I'm OK with it, although I want to note that breaking currently certainly coherent output into pieces has pitfalls. Definitely, we need to clean up at the beginning. We should also add the new log file to .gitignore.

rwgk · 2026-02-10T21:17:05Z


+@pytest.fixture(scope="session")
+def _info_summary_handler(request):
+ strictness = os.environ.get("CUDA_PATHFINDER_TEST_LOAD_NVIDIA_DYNAMIC_LIB_STRICTNESS", "default")


I think it's better and simpler to hardwire the filename. The strictness level is only for controlling hard asserts. Also, maybe make it easy to know what the file is about even when it's pulled into another context, e.g. by adding in pathfinder:
log_path = request.config.rootpath / f"pathfinder-test-info-summary-log.txt"
Naming it .txt makes for the smoothest experience viewing, uploading, etc. Other extensions tend to confuse one tool or another.

Is there ever a case where the pathfinder tests are run twice CI? Once with see_what_works and once with the other setting?

CI runs pathfinder tests twice per job -- once with see_what_works (test-wheel-linux.yml:250, test-wheel-windows.yml:223) and once with all_must_work (test-wheel-linux.yml:295, test-wheel-windows.yml:272). The strictness level is embedded in the filename so the second run does not silently overwrite the first run's log. pytest_configure only deletes the file matching the current strictness level, so both logs coexist for artifact upload.

rwgk · 2026-02-10T21:34:09Z

+@pytest.fixture(scope="session")
+def _info_summary_handler(request):
+ strictness = os.environ.get("CUDA_PATHFINDER_TEST_LOAD_NVIDIA_DYNAMIC_LIB_STRICTNESS", "default")
+ log_path = request.config.rootpath / f"test-info-summary-{strictness}.log"


We want to be sure we're not accidentally using a file from a previous run (e.g. if there is a segfault but some follow-on logic still looks for that file). Cursor generated the code below. If we hard-wire the filename as suggested, we don't even need the glob.
diff --git a/cuda_pathfinder/tests/conftest.py b/cuda_pathfinder/tests/conftest.py index 2a82cca0c..a10278f2d 100644 --- a/cuda_pathfinder/tests/conftest.py+++ b/cuda_pathfinder/tests/conftest.py@@ -7,9 +7,15 @@ import os import pytest +_INFO_LOG_GLOB = "test-info-summary-*.log" _LOGGER_NAME = "cuda_pathfinder.test_info" +def pytest_configure(config):+ for log_path in config.rootpath.glob(_INFO_LOG_GLOB):+ log_path.unlink(missing_ok=True)++ @pytest.fixture(scope="session") def _info_summary_handler(request): strictness = os.environ.get("CUDA_PATHFINDER_TEST_LOAD_NVIDIA_DYNAMIC_LIB_STRICTNESS", "default")

See #1593 (comment), it's also related to your comment here.

rwgk · 2026-02-10T21:38:54Z

+ handler.setFormatter(logging.Formatter("%(test_node)s: %(message)s"))

-def pytest_terminal_summary(terminalreporter, exitstatus, config): # noqa: ARG001
- if not config.getoption("verbose"):


Hm, what about deleting only the verbose condition, but keeping the other two? Otherwise the file will be overwritten 99 times or so?
However, we should unconditionally remove the file when the _info_summary_handler fixture runs. (I've been really confused a few times in the past by stale output, correlating it wrongly to a more recent action.)

What's getting overwritten 99 times?

The _info_summary_handler fixture is scope="session", so it's created exactly once per pytest invocation regardless of how many times pytest-repeat re-runs the tests. The FileHandler opens the file once with mode="w" at session start and appends to it for the duration -- repeated tests just add more lines to the same open handle.

Hm, that's basically spamming the filesystem.

cpcloud · 2026-02-12T12:19:18Z

/ok to test

cpcloud · 2026-02-12T12:22:40Z

/ok to test ab07049

rwgk

See #1621 for a super simple solution to the original problem.

rwgk · 2026-02-13T17:14:13Z

 CUDA_PATHFINDER_TEST_FIND_NVIDIA_HEADERS_STRICTNESS: all_must_work
 run: run-tests pathfinder
+
+ - name: Upload cuda.pathfinder test info summary


I posted #1621.
Introducing these artifacts makes something simple weirdly complex. The INFO lines are a tiny fraction of the total log (see PR #1621 description for numbers).

It's still too noisy, and the mechanism that we are using to track this information lacks any sort of real structure. It's very ad-hoc and can't really ever be integrated into or compose well with another system.

Can you point to a concrete, measurable problem (similar to the numbers I showed under #1621)?
The fraction of INFO lines in a typical CI log file is around 2.5%
I'm using those INFO lines regularly when working on pathfinder, or if random issues pop up. Hiding that information in disconnected artifacts would force me to puzzle together information from pieces that was perviously available at a glance.

It's not a problem in CI anymore it seems. It's still a big code smell though. Is that measurable? Not exactly. Does it have to be to be a valid concern? Definitely not. Closing the PR.

rwgk · 2026-02-13T17:14:32Z

+ handler.setFormatter(logging.Formatter("%(test_node)s: %(message)s"))

-def pytest_terminal_summary(terminalreporter, exitstatus, config): # noqa: ARG001
- if not config.getoption("verbose"):


Hm, that's basically spamming the filesystem.

… logging Replace the custom `info_summary_append` pytest fixture and terminal summary with standard Python `logging`. Test diagnostic info now writes to a per- strictness log file (`test-info-summary-{strictness}.log`) via a `logging.FileHandler`, uploaded as a GHA run artifact. - Drop `custom_info` list, `pytest_terminal_summary`, and the `info_summary_append` fixture from conftest - Add session-scoped `_info_summary_handler` yield fixture (FileHandler lifecycle) and function-scoped `info_log` fixture (LoggerAdapter with test node name) - Update test call sites to use `info_log.info(...)` instead of `info_summary_append(...)` - Remove `tee` + `grep` verification from `ci/tools/run-tests` - Add `upload-artifact` step to both test-wheel workflows with `if-no-files-found: error` Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

- Hardwire log filename with `pathfinder` prefix and `.txt` extension - Retain strictness suffix so both CI runs produce separate logs - Add `pytest_configure` to clean up stale log for the current strictness level (not all levels, preserving the other run's log) - Add line count echo to `run-tests` for quick CI log grep - Add `.gitignore` entry for the new `.txt` log files - Update artifact upload paths in both workflow files - Align default strictness fallback with `test_load_nvidia_dynamic_lib` Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

test_driver_lib_loading.py, test_find_bitcode_lib.py, and test_find_static_lib.py landed on main while this branch was open and still reference the info_summary_append fixture that this PR removes. Switch them to the info_log LoggerAdapter fixture so the rename is complete across the pathfinder test suite.

Removed preview folders for the following PRs: - PR #1593 - PR #1908 - PR #1923 - PR #1925 - PR #1933 - PR #1934 - PR #1938 - PR #1939

rwgk reviewed Feb 10, 2026
View reviewed changes

rwgk mentioned this pull request Feb 13, 2026
Make pathfinder INFO line counting non-failing unconditionally. #1621
Merged

rwgk reviewed Feb 13, 2026
View reviewed changes

cpcloud mentioned this pull request Feb 13, 2026
feat(pathfinder): support "cuda"/"nvml" driver libs and reject unsupported libnames #1602
Merged

cpcloud modified the milestone: cuda.pathfinder backlog Feb 17, 2026

rparolin added this to the cuda.pathfinder 1.4.0 milestone Feb 18, 2026

rparolin assigned cpcloud Feb 18, 2026

rwgk removed this from the cuda.pathfinder 1.4.0 milestone Mar 4, 2026

cpcloudand others added 6 commits April 16, 2026 17:25

chore: lints
e83b6f5

refactor(pathfinder): inline log filename into fixture
ee20cd6
Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pathfinder): set test info logger level to INFO, not DEBUG
37044ad
Co-authored-by: Cursor <cursoragent@cursor.com>

chore: style
93373c4
Co-authored-by: Cursor <cursoragent@cursor.com>

cpcloud force-pushed the move-test-info-summary-to-file-and-logging branch from ab07049 to 93373c4Compare April 16, 2026 21:38

cpcloud added the cuda.pathfinder Everything related to the cuda.pathfinder module label Apr 17, 2026

cpcloud added this to the cuda.pathfinder next milestone Apr 17, 2026

cpcloud requested a review from rwgk April 17, 2026 13:08

cpcloud added the test Improvements or additions to tests label Apr 17, 2026

cpcloud closed this Apr 17, 2026

cpcloud deleted the move-test-info-summary-to-file-and-logging branch April 17, 2026 14:12

github-actionsBot pushed a commit that referenced this pull request Apr 18, 2026
Clean up PR preview folders for 8 closed/merged PRs
e9c5835
Removed preview folders for the following PRs: - PR #1593 - PR #1908 - PR #1923 - PR #1925 - PR #1933 - PR #1934 - PR #1938 - PR #1939

Uh oh!

Conversation

cpcloud commented Feb 10, 2026

What

Why

Changes

Uh oh!

copy-pr-botBot commented Feb 10, 2026

Uh oh!

cpcloud commented Feb 10, 2026

Uh oh!

cpcloud commented Feb 10, 2026

Uh oh!

cpcloud commented Feb 10, 2026

Uh oh!

github-actionsBot commented Feb 10, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

rwgk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cpcloud commented Feb 12, 2026

Uh oh!

cpcloud commented Feb 12, 2026

Uh oh!

rwgk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants