[Experimental] Build the lectures with QuantEcon/mystmd instead of jupyter-book

[kp992]

Experimental setup for building the lectures with the QuantEcon MyST stack. This PR is built on top of #345.

Current toolchain

Both workflows now build with the QuantEcon fork of the MyST CLI and the new theme, so the Netlify preview shows the latest content rendered by the latest tooling.

Piece	What the branch uses	Where it is set
MyST CLI	QuantEcon/mystmd, cloned and built from source with bun, but the two workflows differ: ci.yml is pinned to the exact SHA 96ee78f5 so failures stay bisectable, while build-ipynb.yml clones --branch main and so tracks the fork's latest. quantecon/VERSION.yml is printed to the log, so every build records the qe-vN state it ran with — the ipynb build is currently on qe-v10, the HTML build on the pinned June engine	.github/workflows/ci.yml (pinned), .github/workflows/build-ipynb.yml (main)
Theme	quantecon-theme.mystmd v3.0.0, pinned to the release zip (taken in #629)	lectures/myst.yml (site.template)
Build command	myst build --html --execute	.github/workflows/ci.yml
Content	merged up to date with main	—

The theme moves off the archived QuantEcon/quantecon-theme bundle (which the branch was still pulling as a floating main.zip) onto a pinned release of the new theme repo. Bump the vX.Y.Z in the site.template URL to take a newer theme; the fork stays on main deliberately, so the preview tracks the latest tooling.

The build was not actually using the fork

Worth calling out separately, because it had been green and invisible: until this update the HTML preview was built by jupyter book build, which cannot use the fork. jupyter-book 2.x is a thin Python shim — jupyter_book/__main__.py execs a compiled mystmd bundle vendored inside its own wheel (jupyter_book/dist/jupyter-book.cjs) and exposes no hook to point it at another CLI. The globally installed fork was simply ignored, and only the build-ipynb workflow (which calls myst directly) ever exercised it.

The last run before the fix shows it plainly: the jupyter-book HTML build emitted 27 × 'output' unknown export output extension: exports/<page>.ipynb, because the per-page ipynb export is a fork feature (myst-to-ipynb, qe-v1) that upstream does not know about. The same commit built by the fork's myst build --ipynb emitted none and exported 23 clean notebooks. The HTML build now runs myst build --html --execute — the same engine and the same flags, since jupyter-book 2.x is mystmd, just our build of it.

Everything else in this update

• Merged main. Brings in the new Polars lecture, the numba rewrite, the pandas_panel switch to remote CSVs, and the translation-sync workflows. The one conflict was ci.yml, where main and this branch have rewritten the same job for different builders — resolved in favour of this branch's job, keeping main's runner spec and action bumps.
• myst.yml TOC re-synced with _toc.yml. It had drifted: autodiff and the new polars lecture were missing, workspace appeared twice, and one part title differed. The two files now list exactly the same 27 pages.
• project.github corrected to lecture-python-programming — it pointed at lecture-python-programming.myst, which now only survives as a rename redirect. At the time this mattered because the theme derived the notebook-launch repo from the value plus a .notebooks suffix; theme v3.0.0 removed that derivation, so project.github now only drives the commit and "edit this page" links, and the launch repository is named explicitly (see chore: take theme v3.0.0, migrating the launch and footer config #629).
• JAX is now installed in the build. Previous preview builds were rendering ModuleNotFoundError cells for jax_intro, autodiff and numpy_vs_numba_vs_jax, because this job installed only myst_requirements.txt while the jupyter-book 1.x job on main installs jax[cuda13] separately. Same install, same runner family.
• CI actions moved to current majors and Node 20 → 24 (the theme's .nvmrc; it runs as a Remix server during myst build --html). This also clears the Node 20 runtime deprecation warnings. The unused configure-pages step is gone — this job deploys to Netlify, not Pages.
• Removed committed execution artifacts (foo.py, newfile.txt, numbers.txt, output.txt, output2.txt, test_table.csv, us_cities.txt). All are written by %%file / %%writefile cells in the lectures themselves, all are matched by .gitignore, and none exist on main.

Still open

• Nothing fails the build on a notebook execution error. Execution errors are reported and the build carries on, which is how the JAX failures above survived green builds. main gained a gate for the jupyter-book 1.x path in CI: gate the first jb build so notebook errors cannot pass green #588; the fork's CLI has myst build --strict for this, but it should be turned on only once the remaining build diagnostics are triaged, or it will fail on unrelated warnings.
• Launch-button URLs are left on the theme defaults. Resolved by chore: take theme v3.0.0, migrating the launch and footer config #629. Theme v3.0.0 makes Launch opt-in, so site.options now names launch_notebook_repo, launch_notebook_source_dir and launch_colab, ported from lectures/_config.yml rather than invented; the branch is left unset because main is the default. Confirmed on that PR's preview: the button resolves to …/lecture-python-programming.notebooks/blob/main/need_for_speed.ipynb.
• The ci.yml fork pin's comment is out of date. It says the SHA is "the exact SHA of the fork's main (unmoved since 2026-06-12)", but the fork's main has since moved to c1f274b8a (2026-09-10) and several qe-vN checkpoints have landed. The pin itself is a deliberate choice — it keeps failures bisectable — but the HTML build is now running a materially older engine than build-ipynb.yml, which is worth a conscious decision rather than drift.

[github-actions]

🚀 Deployed on https://6aa4e1e19e2cb93dc5a0a97d--epic-agnesi-957267.netlify.app

[mmcky]

@kp992 thanks for opening this. Excellent work! It is exciting to see how close we are now.

From my review it looks like there a just a few issues and features we need before launch.

Bugs:

1. LHS TOC menu flickers when using the Home page or a top level Part section. It is like it has an auto-stow feature that is triggering when pages are reloaded. It doesn't look great when pages are refreshed.
2. Code block formatting has too much indentation for Solution blocks

Features:

1. I see a bunch of new export notebooks being added to the yaml headers of each lecture. Do we need to do some development work on supporting the download notebook link in the theme?
2. Is index support on the jb2 / mystmd roadmap?

[DrDrij]

Netlify preview only, not a production concern: assets here are served with cache-control: public,max-age=0,must-revalidate, so every navigation makes a blocking revalidation round-trip (~390ms for a 304 with 0 bytes) before first paint. GitHub Pages serves max-age=600, so the deployed lectures are unaffected. If it is worth making the preview behave like production, a _headers file with /build/_assets/* Cache-Control: public,max-age=31536000,immutable is safe — those filenames are content-hashed, so a rebuild changes the URL.

[mmcky]

@DrDrij I have just updated this to the latest release of quantecon-theme.mystmd incorporate recent merges.

[mmcky]

Bumped the theme pin in lectures/myst.yml from v2.3.0 to v2.5.0 (7a392b8), released today. Between those tags the theme picked up the static-build asset URL fixes (2.3.1), lecture content typography and the inline-code backticks regression fix (2.4.0), and in 2.5.0 the icon-only search trigger, PT Serif headings on the Sphinx scale, QuantEcon-blue content links, the Colab-only launcher (the Private JupyterHub option is gone), and the back-to-top / FOUC first-paint fixes. Release notes: https://github.com/QuantEcon/quantecon-theme.mystmd/releases/tag/v2.5.0. No other change on the branch; the existing conflict with main is untouched.

[mmcky]

Bumped the pinned theme to v2.6.0 in b9524ff. It brings the QuantEcon code-token palette and seoul256 text colours (Phase 3), and the language switcher, hreflang alternates, RTL support and translator credit (Phases 4–5). This edition sets none of the new multilingual options, so the only visible change here is the code and text colouring; the switcher appears once languages lists the translated editions.

[mmcky]

Pinned theme bumped to v2.6.1, the patch release carrying the fix for QuantEcon/quantecon-theme.mystmd#186 (Back after an in-page anchor showed an "Application Error" screen on the Netlify preview; "↑ Top" did a full reload). No configuration changes. The preview built from this commit is the first real-content check of that fix.

[mmcky]

Pinned theme bumped to v2.7.0 in 41b1f83. It carries the last items of the book-theme parity gate: the pinned, scroll-tracking "On this page" panel with nested subsections (QuantEcon/quantecon-theme.mystmd#182), stderr output folded behind a "Code warnings" disclosure (QuantEcon/quantecon-theme.mystmd#92), and a layout correction for desktop windows 1280–1327px wide (QuantEcon/quantecon-theme.mystmd#198). No configuration changes are needed; this branch sets no site.options, so of the new OpenGraph/Twitter tags only og:type and og:site_name appear.

[mmcky]

Theme v3.0.0 is now on this branch: #629 merged into jb2 as 69ce7e5fd.

Three things in the description above are now out of date, listed here rather than edited in so the history stays readable.

The toolchain table says theme v2.2.0. It is v3.0.0 as of #629 — and the table had already drifted, since lectures/myst.yml was pinned at v2.7.0 before this. v3.0.0 is the release that closes the book-theme parity gate.

The project.github note no longer holds. It says the theme derives the notebook-launch repo from that value plus a .notebooks suffix, so a stale name means broken launch buttons. v3.0.0 removed that derivation entirely: Launch is opt-in and the repository is named explicitly. project.github still drives the commit and "edit this page" links, so keeping it correct matters for a different reason now.

The "Still open" bullet about launch defaults is resolved. The URLs are no longer on theme defaults — site.options now carries launch_notebook_repo, launch_notebook_source_dir and launch_colab, ported from lectures/_config.yml (nb_repository_url, path_to_docs, launch_buttons.colab_url) rather than invented, with the branch left unset because main is the default. Verified on the #629 preview: the button resolves to …/lecture-python-programming.notebooks/blob/main/need_for_speed.ipynb, so the page location does resolve as that bullet asked someone to confirm.

Also landed in #629, for completeness: lectures/footer.md and the then-orphaned _static/ccbysa.png are gone, because v3.0.0 renders the licence footer by default and a declared footer part replaces it — the default is a superset, restoring the "A theme by QuantEcon" credit the file had dropped. site_url is set from the Sphinx html.baseurl, which is what makes the theme emit a canonical link and og:url. And the {raw} notebook-header blocks are rewritten out of need_for_speed.md and polars.md, so their source no longer reaches readers as escaped text under the page title.

[mmcky]

Engine bumped in f1c7a8d55: the ci.yml fork pin moves from 96ee78f5 (2026-06-12, qe-v8) to c1f274b8a (2026-09-10, qe-v10). Still an exact SHA rather than a branch — the pin exists so failures stay bisectable, so only its value changed, and the comment above it now records when the SHA was taken instead of claiming it is the fork main.

The reason for bumping: the pin had fallen 40 commits behind, so the HTML build was rendering with qe-v8 while build-ipynb.yml — which clones --branch main — was already on qe-v10. The same lecture commit was being built by two materially different engines depending on which workflow you looked at.

Worth reading this run's log rather than just its check mark. Per the "Still open" note above, execution errors are reported without failing the build, so a green run does not by itself establish that the newer engine is clean.

[mmcky]

Followed my own advice and read the log rather than the check mark. The engine move is clean.

Both builds pass (HTML 4m50s, ipynb 2m04s), and the bump is confirmed active in the logs: the previous run recorded qe_version: qe-v8, this one records qe-v10.

Diagnostics are unchanged across the 40 commits of engine:

	before (qe-v8)	after (qe-v10)
⛔ lines	24	24
error	6	6
warning	10	10
unknown directive	1	1

For the record, those 24 ⛔ lines are two known classes and neither is an execution failure: 23 are one-per-page Resource "exports/<page>.ipynb" in downloads should be a URL or path to static file — the downloads entry names an ipynb that the HTML build does not produce, since build-ipynb.yml is what creates them — and 1 is status.md:16 unknown directive: nb-exec-table, the Sphinx directive with no mystmd counterpart.

Worth noting against the "Still open" item above: there are currently zero cell-execution errors in the build, so the "errors are reported but do not fail the build" hazard is latent rather than active right now, and this green run means what it appears to mean.