allow notebooks to exit early (e.g. with `sys.exit(0)` or `pytest.skip(reason, allow_module_level=True)`)

[tswast]

Is your feature request related to a problem? Please describe.

I have some notebooks that I test with multiple versions of Python, but they are not compatible with Python 3.13. I would love to be able to exit early from these notebooks without causing the test to be reported as a failure.

Describe the solution you'd like

I have tried the following cells:

import sys

# Python 3.13 is not yet a supported runtime.
if sys.version_info >= (3, 13, 0):
    sys.exit(0)

also

import sys

# Python 3.13 is not yet a supported runtime.
if sys.version_info >= (3, 13, 0):
    import pytest

    pytest.skip("Python 3.13 not supported", allow_module_level=True)

I would expect at least the latter to work, if not both. Note: one can disambiguate if SystemExit exception is "ok" / intended from a failure by looking at SystemExit.code == 0.

Describe alternatives you've considered

• I've also tried creating an autouse fixture in a conftext.py file that wraps the yield in a try / except block to capture these expected exceptions, but it seems conftest.py fixtures don't run. Perhaps an alternative to consider to allow such flexibility.

• I've tried the "metadata": { "tags": [ "raises-exception" ] } on the cell, but this actually continues execution after the failed cell. I suspect this is desirable to have code cells that demonstrate exceptions, but it wasn't the right solution to be able to exit early.

Additional context

N/A

[tswast]

I think the solution will end up being somewhere around here:

nbmake/src/nbmake/nb_run.py

Lines 117 to 138 in 5de6e18

	error = self._get_error(nb)
	except CellTimeoutError as err:
	trace = err.args[0]
	error = NotebookError(
	summary=trace.split("\n")[0],
	trace=trace,
	failing_cell_index=self._get_timeout_cell(nb),
	)
	except Exception as err:
	# if at top causes https://github.com/jupyter/nbclient/issues/128
	# from jupyter_client.kernelspec import KernelSpecManager, NoSuchKernel
	# trace=f"{summary}\n\nInstalled Kernels: {str(KernelSpecManager().find_kernel_specs())}", # noqa
	# https://github.com/treebeardtech/nbmake/runs/1536896858?check_suite_focus=true
	if str(type(err)) != "<class 'jupyter_client.kernelspec.NoSuchKernel'>":
	raise err
	summary = f"Error - No such kernel: '{err.name}'" # type: ignore
	error = NotebookError(
	summary=summary,
	trace=f"{summary}",
	failing_cell_index=0,
	)

We could explicitly catch SystemExit (if exc.code == 0: pass; else error = NotebookError(...);) / the pytest.skip.Skipped exception. Though, maybe those end up being wrapped by one of those CellExecutionErrors?

[alex-remedios]

hey @tswast - sorry I don't maintain this project v. actively but I think your suggested approach is on the right track.

The challenge here is that any code executed in a notebook cell is in a different process to pytest, hence no ability to imperatively skip.

I also don't want to allow pytest calls via scripting in notebook metadata as we did with post cell execute as it leaves open the opportunity to write hidden malicious code.

So it would make sense to give you a convention for raising an Exception in the nb that will be wrapped with a skip/xfail or whatever. Not sure when I'll get around to this but any PRs welcome.