test(containers): make the container fixture able to fail (#108)

[mmcky]

The container smoke tests could not fail. This is the substance of #108, though the fixture was inert for three independent reasons rather than the one the issue names.

Why it was inert

#	Cause
1	execute_notebooks: "off" in _config.yml
2	the one code block was a plain ```python fence, not a {code-cell}
3	no jupytext front matter, so the file was never treated as a notebook at all

grep -rn 'code-cell' over the fixture returned zero hits. An image whose numpy, matplotlib or kaleido was completely broken still produced a green container test.

And fixing only 1 and 2 would still not be enough: myst-nb reports a failed cell as a warning, so a raising cell yields a published book and exit 0 unless -W or raise_on_error is set. Both are now set — deliberately, because a typo in the raise_on_error key is accepted silently by jupyter-book's config validator, and -W is then the only thing still catching a failing cell.

What it tests now

Cells that assert rather than print. A stack that imports cleanly but computes wrong answers is exactly what a smoke test should catch, and printing alone would not.

• numpy/scipy eigenvalues against an exact closed form
• a pandas groupby
• a real matplotlib PNG render (buffer, not just figure construction — that's what exercises the backend and fonts)
• a plotly static export through kaleido — the Pin kaleido<1.0 in both containers to fix ChromeNotFoundError #85 path, and the reason both images pin kaleido<1.0

container: jobs, not docker run

This is the other half, and it is not cosmetic. A GitHub Actions container: job forces HOME=/github/home; docker run leaves HOME=/root. That difference is precisely why #85 passed these tests while failing test-containers-lectures.yml, which does use a container job — kaleido v1 dropped its bundled chromium, and a chromium provisioned under /root at image build time became unreachable at run time.

So both images now run as a real container: job matrix, and the job asserts HOME=/github/home explicitly so a silent revert to a weaker environment is caught. The lean image also gains pdflatex coverage it never had.

The self-test

The defect here was silence — a fixture that cannot fail reads as coverage. So smoke-test.sh --self-test stages a deliberate exception and fails if the build does not go red, and it attributes the red rather than accepting any non-zero (which a bad mount, an OOM, or an unrelated -W warning would also satisfy).

It greps captured stdout rather than reports/*.err.log, unlike build-fail-guard. Verified: with raise_on_error: true myst-nb raises before Sphinx writes the report, so that file does not exist here.

Verification

Run locally against jupyter-book 1.0.4.post1 / myst-nb 1.4.0, minus the plotly cell (kaleido is not in my local env):

check	result
cells genuinely execute	numpy 2.3.5: eigenvalues [1.38 3.62], pandas 2.3.3: group means {a: 2.0, b: 5.0}, matplotlib 3.10.6: rendered 12311 byte PNG — all present in the built HTML
clean under -W	exit 0, zero warnings
fixture can fail	staged raise → exit 1
sentinel in captured output	present
reports/ written	absent — confirms the stdout-attribution decision

The plotly/kaleido cell and the HOME assertion can only be verified in CI. I'll trigger workflow_dispatch on this branch before asking for merge — workflow_run only fires for workflow files already on the default branch, so that dispatch is the only possible pre-merge gate.

Deliberately not in scope

• Directory consolidation. test: unify container smoke tests and make the fixture exercise the Dockerfiles #108 also proposes unifying three copies of the container-runner logic under containers/tests/. Left alone: it would dangle at least six doc references, and this PR is already the behavioural change. Worth a follow-up.
• fontpkg: "". Removing it would engage the FreeSerif/fontspec chain that broke in FreeSerif font not accessible to Jupyter Book in Docker container #3 — real added coverage, but a separate and riskier change that I cannot verify locally.
• Pinning plotly. It is bare in containers/quantecon-build/environment.yml:49 while kaleido is pinned <1.0. Since this PR makes fig.to_image() load-bearing for CI, a defaults-channel plotly bump past the kaleido v0 window would redden the smoke test with no code change here. Pinning it is an image change and belongs in its own PR — flagging it rather than doing it silently.

Refs #108

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Updates the container CI smoke tests so they can genuinely fail when the container’s scientific / rendering stack is broken, aligning the tests with the repo’s goal of validating published lecture build images (not just “does jupyter-book run”).

Changes:

• Add a new in-container smoke test runner (smoke-test.sh) with a --self-test mode to ensure the fixture is capable of failing.
• Make the minimal Jupyter Book fixture execute real {code-cell} blocks (with jupytext front matter, execute_notebooks: force, and raise_on_error) and assert on results (numpy/scipy, pandas, matplotlib render, plotly+kaleido export).
• Switch test-container.yml to run both images as true container: jobs (plus a separate manifest-based compressed size report), and update related docs/changelog.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
TESTING.md	Clarifies how action-level fixtures differ from container image smoke fixtures.
containers/quantecon/tests/smoke-test.sh	New smoke test script for running inside container jobs; adds --self-test inert-fixture guard.
containers/quantecon/tests/run-local-tests.sh	Strengthens local replication script (now uses pipefail and validates the Python stack).
containers/quantecon/tests/minimal-jupyter-book/intro.md	Converts fixture content into executable {code-cell} tests with assert-based checks.
containers/quantecon/tests/minimal-jupyter-book/_config.yml	Forces notebook execution and raises on cell errors to ensure failures are surfaced.
containers/quantecon/README.md	Updates container testing documentation to reflect new smoke test and fixture behavior.
CHANGELOG.md	Documents the behavioral change: container smoke tests now actually exercise the stack and can fail.
.github/workflows/test-container.yml	Runs smoke tests in real container: jobs for both images and reports manifest-based compressed sizes.
.github/fixtures/mini-lectures/README.md	Updates fixture documentation to distinguish action-logic testing vs image-contents testing.