Skip to content

fix(artifacts): verify what we ship, not what we asked for — Vulkan, capability checks, artifact runnability, CI resilience - #25

Merged
ryan-morris merged 32 commits into
mainfrom
fix/artifact-resilience
Sep 16, 2026
Merged

ryan-morris merged 32 commits into
mainfrom
fix/artifact-resilience

Conversation

@ryan-morris

@ryan-morris ryan-morris commented Sep 14, 2026

Copy link
Copy Markdown
Member

Closes the remaining artifact-audit items, the CI failures that were not our code, and — after a defect-class hunt — a class of bug where the repo verified intent and never result.

Everything here was found by inspecting published artifacts or reproducing locally, and verified the same way rather than by pushing and watching CI.

The core defect: we checked that we asked, never that we got it

check_config greps the configure string embedded in the artifact. gen-matrix.sh reads the BUILD_* flags. Both answer "did we request this". FFmpeg and the dependency build systems decline requests silently as a matter of course.

Measured in the published 9.0.1.7 and 8.1.2.7 artifacts — every one of these shipped green:

RID claims --enable-vulkan Vulkan filters present
linux-x64, linux-arm64 yes 18
win-x64, android-arm64, ios-arm64, maccatalyst, osx-arm64 yes 0

Cause: FFmpeg's Vulkan filters need spirv_compiler, which configure obtains by probing a glslc binary with --target-env=vulkan1.4. glslc was only ever installed on the manylinux/Alpine path, so spirv_compiler was off everywhere else. Our own guard made it worse: the pinned newer glslc was built only if ! command -v glslc, so hosts with a too-old distro glslc skipped it — and it lived inside the linux-x64|linux-arm64 branch, so it could never have run for armhf, Windows, Android or Apple at all.

Fixed: probe capability rather than presence (compile a real shader with configure's exact flags), hoisted out of the platform branch and gated on BUILD_VULKAN; build glslc as a host tool with the cross toolchain cleared; add shaderc + ninja to the macOS package list; widen the SPIR-V-headers gate from "whisper uses Vulkan" to "Vulkan is enabled at all".

Verification that asks the binary

  • check_claimed_capabilities — for every --enable-* in the artifact's own configure string, something matching it must actually be registered. Expectations come from the artifact, so there is no per-RID list to maintain.
  • check_claimed_capabilities_static — the same question for cross-built slices that ship no CLI (Android .so, iOS/Catalyst frameworks), read out of the libraries with strings. Those were previously verified by intent alone, which is why the mobile Vulkan gap went unseen.

Both are driven by one table, scripts/test/capabilities.tsv (43 rows), so they cannot drift apart.

Proven against six published artifacts — ~200 assertions, no false positives; the only failures are the real Vulkan defect.

Writing it corrected three things of mine:

  • Two false-negative classes. strings skips sequences under 4 chars (so srt never appeared), and the linker tail-merges a suffix (rist inside librist). A name a longer string could have absorbed is now reported inconclusive, not missing — without that, CI would have failed linux-x64 on a correct build.
  • Two rows that passed regardless of the flag. libsoxr asserted the aresample filter, which FFmpeg builds unconditionally; it now queries the resampler engine via -h full. A row that cannot fail is worse than no row.
  • The check ran before ffmpeg existed on all three desktop scripts, so an empty listing would have reported ~35 missing capabilities on all nine desktop jobs. My earlier evidence came from a harness where I set FFMPEG myself — I had verified the function and never its call site.

Two more silent losses found by the same hunt

  • libvmaf shipped with no built-in models on the manylinux RIDs. Its meson marks xxd required: false and emits the model sources only inside if xxd.found(), so the filter registers and every lookup returns -EINVAL. xxd added to all package lists, plus a hard assert.
  • The Vulkan runtime probe scored a MISSING filter as a passNo such filter: scale_vulkan matched the "no GPU on this runner" tolerance branch.

Four more capabilities are now asserted at build time, because only build-time evidence can prove them: SRT encryption (libsrt's HaiCrypt layer), RIST encryption (what meson actually resolved), libass x86 assembly, and libvmaf's NASM floor for AVX-512. All four were measured present first, so they are regression guards, not new failures.

Artifacts that could not start

  • Windows v3: avfilter-12.dll carried a hard vulkan-1.dll import, so ffmpeg.exe would not start without a GPU driver. Replaced the dlltool import library with BtbN's MIT Vulkan-Shim-Loader: identical capability, 0 hard imports.
  • musl: libstdc++.so.6 / libgcc_s.so.1 in DT_NEEDED, and Alpine ships neither. Linked statically rather than bundled — bundling would pull a GPLv3 §6 obligation into every musl artifact, including the lgplv2 cell whose whole value is a clean licence position. Staging now refuses to publish a musl artifact that still depends on a host C++ runtime.
  • fontconfig compiled the build machine's paths into every artifact. Fixed with DESTDIR staging.
  • Dependencies were not built in Release (-O3 -DNDEBUG) — now explicit for the 14 CMake-built deps.

CI failing for reasons that were not our code

curl's --retry-delay replaces its exponential backoff with a fixed wait, so passing it was actively downgrading behaviour. --retry also does not cover connection-level failures, and --retry-all-errors needs curl ≥ 7.71 while manylinux ships 7.61.1 — so the wrapper retries an allowlist of transient exit codes itself. Git retry now covers fetch / submodule / ls-remote, not just clone, with exponential jittered backoff (124–144s measured, was ~15s). All 13 checkout steps across 9 workflows wait for github.com first. Four unprotected network operations closed, including the release tag push — a blip there meant the release silently did not happen.

Tooling that could not run

  • gen-matrix.sh was dead on any host without an Android NDK — exit 2 with no message, because it sources the platform config with stderr redirected. It now uses an NDK-shaped stub, and output is byte-identical between Git Bash and WSL.
  • command -v python3 answers "is something named python3 on PATH", not "does it work" — on Windows it is a Store stub that exits 49. resolve_python() in lib.sh replaces four copies of the probe.
  • Matrix consistency is now asserted (15 built, 9 desktop-tested, 6 mobile-tested), and mbedtls no longer renders as "not built" in all 8 matrices when it is built and linked on every v3 cell.

Documentation

56 factual defects in shipped docs, mostly one mistake repeated: a rule stated globally that is true of most RIDs but not all. TLS is not "OpenSSL on v3, GnuTLS on gplv2, none on lgplv2" — that ladder is Linux/Android/Catalyst only; Windows and macOS/iOS have a native backend in all four cells. That claim was also being written into the legal notice of every Windows and macOS v2 artifact; it is now RID-aware.

A further 39 defects were closed by removing docs/superpowers/ — working notes that read as authoritative in-tree. Now gitignored.

Deliberately not addressed

alpine:latest and the manylinux tag remain unpinned — tracked in #26, a dependency-management decision with its own bump cadence.

🤖 Generated with Claude Code

…k blips

Closes the remaining artifact-audit items. Everything here was found by
inspecting published artifacts or reproducing locally, and verified the
same way rather than by waiting for CI.

Windows v3 could not start without a Vulkan runtime
---------------------------------------------------
Confirmed in the shipped 9.0.1.6 artifact: avfilter-12.dll carries
"DLL Name: vulkan-1.dll" as a hard import. libavfilter carries
af_whisper, so a missing vulkan-1.dll is not a degraded filter -- the
library will not load and ffmpeg.exe does not start. vulkan-1.dll
arrives with GPU drivers, which is why this is invisible on a developer
desktop and fatal on a headless server, a container or a fresh VM.

Cause: mingw ships no Vulkan import library, so whisper.sh synthesised
one with dlltool for ggml-vulkan, which creates a hard import by
construction. Fixed by linking BtbN's MIT-licensed Vulkan-Shim-Loader
instead: a static stub that LoadLibraryExA's the real loader on first
use. Comparing published artifacts of the same FFmpeg version, BtbN has
2 vulkan symbols and 0 hard imports where we had 2 and 1 -- identical
capability, only ours undeployable. So the answer was never to drop
Vulkan from Windows v3.

Proven with mingw locally before wiring it in: the same consumer object
links to 1 vulkan import via the dlltool library and 0 via the shim.
Scoped to win-x64, the only RID with both the Vulkan whisper backend and
this import. The shim is in deps.json, so Renovate tracks it (59 deps,
each matched by exactly one manager).

musl artifacts could not start on a bare Alpine image
-----------------------------------------------------
The C++ codec libraries put libstdc++.so.6 and libgcc_s.so.1 in
DT_NEEDED and Alpine ships neither, so a stock container failed at
startup. This was documented as "apk add libstdc++ libgcc", but
documenting a missing dependency does not make an artifact runnable.

The runtime is now linked statically (-l:libstdc++.a, -static-libgcc)
rather than bundled. Bundling would also have worked, but it means
redistributing GPLv3 libraries: the GCC Runtime Library Exception covers
our linked output, not the runtime shipped as a library, so it would
pull a GPLv3 section 6 corresponding-source obligation into every musl
artifact -- including the lgplv2 cell, whose whole value is a clean
licence position. Static linking makes the result "Target Code" under
the Exception, with no such obligation, and matches what BtbN ships.

Verified on a real shared library exercising C++ exceptions: DT_NEEDED
drops to libc and the loader alone, and it still runs. whisper.sh fails
the build up front, naming the package to install, if libstdc++.a is
missing -- so the static link is guaranteed rather than hoped for -- and
staging refuses to publish a musl artifact that still depends on a host
C++ runtime.

Build-tree paths baked into every artifact
------------------------------------------
fontconfig compiles its sysconfdir/cachedir into the library, so the
build machine's path shipped in every asset. An earlier attempt pinned
those paths and was reverted in 72f3c53, because the same options decide
where meson INSTALLS, so it tried to create /var/cache and hit
permission denied. DESTDIR staging separates the two: configure with the
runtime prefix, install under DESTDIR, relocate into DEPS_DIR, repoint
the .pc. Same separation BtbN uses.

Dependencies were not built in Release
--------------------------------------
build_cmake_dep now passes CMAKE_BUILD_TYPE=Release. For GCC/Clang that
is -O3 -DNDEBUG, so it does disable assert() in the 14 CMake-built
dependencies -- stated plainly rather than described as an optimisation
change. Not forced on the autotools/Meson ones, which handle it
themselves; that matches BtbN, which sets Release per project.

armhf Vulkan loader
-------------------
Documented rather than changed: armhf genuinely ships no bundled loader,
but it is not a startup dependency (no DT_NEEDED; it uses FFmpeg's
dlopen path). Its Whisper is CPU-only, so the system loader affects
Vulkan filters only.

CI: builds failing for reasons that are not our code
-----------------------------------------------------
Three separate failures this cycle were network blips that "retrying"
did not survive.

A release job died on "Could not resolve host" after 4 attempts -- but
the delay was a fixed 5s, so all four landed inside ~15 seconds. Clone
retry is now exponential with jitter (124-144s measured) and covers
fetch, submodule and ls-remote as well as clone; libplacebo.sh's
submodule fetch previously had no protection at all.

curl's --retry-delay turns out to REPLACE curl's exponential backoff
with a fixed wait, so passing it was actively downgrading behaviour.
Removed, and retry policy now lives in one place instead of being
re-specified by nine dependency scripts.

A linux-x64 build then died on curl exit 35 (TLS connect), which none of
that would have retried: curl's --retry covers timeouts and transient
HTTP responses, not connection-level failures, and --retry-all-errors
needs curl 7.71+ while manylinux ships 7.61.1. The wrapper now retries
an allowlist of transient exit codes itself. Configuration errors (2, 3,
37) fail immediately, which also keeps 07_build_ffmpeg.sh's
--retry-all-errors capability probe instant on old curl.

actions/checkout has no retry input and makes 3 attempts with random
10-20s waits, and cannot be wrapped: a local composite action needs the
repo already checked out, and retry actions only wrap run steps. So all
13 checkout steps across 9 workflows wait for github.com first, with the
same exponential backoff and no pointless sleep after the final attempt.

Remaining unprotected network operations
----------------------------------------
The retry work above left four gaps, all now closed:

  - release.yml pushed the release tag with a bare `git push`. A blip
    there is not cosmetic: the tag exists locally and is never published,
    so the release silently does not happen.
  - check-updates.yml pushed its PR branch and probed upstream tags with
    bare `git push` / `git ls-remote`.
  Both run in workflow steps, where the git() wrapper in scripts/lib.sh
  does not apply, so they get an equivalent inline helper.

  - scripts/gen-matrix.sh and gen-coverage.sh fetch FFmpeg's configure
    with the system curl, outside the wrapper. They now source lib.sh and
    drop their own --retry so the policy lives in one place. Verified:
    a resolve failure now retries instead of failing outright, and
    gen-matrix still produces byte-identical output.

  - 03_install_packages.sh runs shaderc's git-sync-deps, which clones
    glslang/SPIRV-Tools over the network from a PYTHON child process --
    shell functions are not exported to children, so it had no retry at
    all. The invocation is retried as a whole.

Not addressed deliberately: the alpine:latest and manylinux image tags
remain unpinned. Pinning them is a dependency-management decision with
its own bump cadence, not part of this fix.

Vulkan filters were missing from four RIDs
------------------------------------------
Found by a consumer asking whether scale_vulkan was actually in the
artifact. It was not -- on linux-armhf, win-x64, win-arm64 and
android-arm64, measured in the published 9.0.1.6 and 8.1.2.6. The other
four Linux RIDs had them (linux-x64 9, linux-arm64 11, musl 9/11).

FFmpeg's Vulkan filters need spirv_compiler, which configure obtains by
probing a glslc binary with --target-env=vulkan1.4. Ubuntu/Debian ship
shaderc 2023.8, which predates Vulkan 1.4 and rejects the flag, so
configure silently disables spirv_compiler and drops every Vulkan
filter. Nothing errors, and libavcodec/vulkan_*.o still compile, so it
looks like Vulkan is working.

Our own guard made it worse in an inverted way: the pinned newer glslc
was built only `if ! command -v glslc`, so hosts WITH a too-old distro
glslc skipped it, while hosts without one (manylinux, Alpine) built or
packaged a good one and worked. It also lived inside the
linux-x64/arm64 manylinux branch, so it could never have run for armhf,
Windows or Android at all.

Now: probe capability rather than presence (compile a real shader with
configure's exact flags), hoisted out of the platform branch and gated
on BUILD_VULKAN, and assert after building that the new glslc actually
wins on PATH.

musl static C++ runtime needed the .pc rewrite too
---------------------------------------------------
Setting -l:libstdc++.a in WHISPER_SYS_LIBS covered only whisper's
contribution. openh264, libass and other C++ dependencies declare
-lstdc++ in their .pc files, so the FFmpeg link still resolved the
shared libstdc++ and staging refused the artifact:

  ERROR: libavcodec.so.62.28.102 depends on a host C++ runtime:
    libstdc++.so.6

The .pc files are now rewritten the same way win-arm64 already rewrites
libc++. The staging guard catching this is the guard working as
designed -- it refused to publish an artifact that would not start.

Planning documents removed
--------------------------
docs/superpowers/ held 39 factual defects at the point of removal --
stale cell counts, pre-Catalyst assumptions, sample commands that no
longer run. They are working artifacts, not documentation, and in-tree
they read as authoritative. Removed and gitignored.

Verify capabilities against the binary, not the configure line
---------------------------------------------------------------
The root reason all of the above shipped: every check in this repo
verified INTENT. check_config greps the configure string embedded in the
artifact; gen-matrix.sh reads the BUILD_* flags. Both answer "did we ask
for it", never "did we get it" -- and FFmpeg and the dependency build
systems decline requests silently as a matter of course.

check_claimed_capabilities now asserts the other direction: for every
flag in the ARTIFACT's own configure string, something matching it must
actually be registered in the binary. Expectations come from the
artifact, so there is no per-RID list to maintain and the table cannot
drift out of sync with what a cell enabled.

Proven against published artifacts: it FAILS 8.1.2 linux-x64 (scale_vulkan
claimed, not registered) and PASSES 9.0.1 linux-x64, with 27 other
capabilities confirmed present in both. Writing it also corrected two of
my own measurements -- 8.1.2 has 2 Vulkan filters rather than 0 (the two
that need no SPIR-V compiler), so the check asserts scale_vulkan
specifically rather than a loose /_vulkan/ match that a degraded build
would pass.

Two more silent-loss defects found by the same hunt:

  - libvmaf shipped with NO built-in models on the manylinux RIDs. Its
    meson marks xxd `required: false` and emits the model sources only
    inside `if xxd.found()`, so the filter registers and every lookup
    returns -EINVAL: FFmpeg's default version=vmaf_v0.6.1 cannot load.
    A comment in linux.sh asserted a "non-xxd fallback" that does not
    exist, which is why it went unnoticed. vim-common added to the dnf
    list, xxd added to all five apt lists, and libvmaf.sh now refuses to
    build without it.

  - The Vulkan runtime probe scored a MISSING filter as a pass:
    "No such filter: scale_vulkan" matched the "no GPU on this runner"
    tolerance pattern. That is how a build with no Vulkan filters
    reported "path exercised". Missing/unknown filters are now hard
    failures, ordered before the tolerance branch.

  - check_symbol's fallback dropped --defined-only, so a symbol the
    library merely REFERENCES satisfied it. Verified against a library
    that only calls an external function: the old path matched, the new
    one does not.

Matrix consistency is now checked, not remembered
-------------------------------------------------
Adding a RID means touching three matrices plus a runner mapping, and
every way of getting it wrong is silent -- a RID in no test matrix is
simply built and never tested, and a mismatched include ADDS
combinations rather than erroring, which is how the Catalyst work
produced 26 jobs with an empty licence. Now asserted: 15 built, 9
desktop-tested, 6 mobile-tested, no gaps.

Licence rationale recorded
--------------------------
04_select_license.sh claimed OpenSSL "can't be used without version3".
That is not what FFmpeg's configure does -- it rejects OpenSSL 3+ only
when --enable-gpl is set without --enable-version3, so an LGPL build may
use it. Stating it wrongly invited the question repeatedly. The comment
now concedes the premise and gives the real reasons for declining, as
conservative policy pending counsel rather than settled law, and records
that one smaller project does publish such builds.

Verification
------------
Each fix checked against the property that matters, not a green run:
import tables of real mingw-linked DLLs; DT_NEEDED of a real shared
library; stubbed git/curl on Ubuntu 24.04 under WSL for retry counts,
preserved exit codes and zero-retry success paths. The matrix gate and
the Vulkan guard were both proven to FAIL on broken input, not merely
pass on good input.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ryan-morris
ryan-morris force-pushed the fix/artifact-resilience branch from 79d7839 to 6d4867f Compare September 14, 2026 18:02
ryan-morris and others added 10 commits September 14, 2026 13:25
…es them

The docs audit found 95 factual defects. 39 were in docs/superpowers/ and
closed by removing it; these are the remaining 56, in shipped documentation
and in the scripts that generate documentation and legal notices.

Nearly all of them are the same mistake: a rule stated globally that is
true of most RIDs but not all. The corrections are per-RID.

  - TLS is not "OpenSSL on v3, GnuTLS on gplv2, none on lgplv2". That
    ladder applies to Linux, Android and Mac Catalyst only. Windows
    (SChannel) and macOS/iOS (SecureTransport) have an OS-native backend
    in all four cells, so v3 is not required for TLS and the lgplv2 cell
    is not TLS-less there. Corrected in README, docs/install/README,
    DEVELOPMENT.md, gen-matrix.sh's footnote and both generated
    preambles, and in 10_write_legal.sh -- which was writing "TLS is
    GnuTLS or omitted" into the LEGAL NOTICE of every Windows and
    macOS/iOS v2 artifact. That sentence is now RID-aware.

  - Whisper's GPU backend is per RID, not per platform. Vulkan on
    linux-x64/arm64, both musl RIDs, win-x64 and both Android RIDs;
    Metal on every Apple target in every cell; CPU on linux-armhf and
    win-arm64 in every cell. So choosing v3 does not buy GPU Whisper on
    armhf or win-arm64, and v2 does not cost it on Apple. whisper.md now
    carries the table instead of a sentence.

  - The Vulkan loader is bundled on linux-x64 and linux-arm64, not on
    every glibc build; linux-armhf ships none.

  - gpl means x264+x265 and lgpl means kvazaar everywhere EXCEPT the
    lean ios-sim-arm64 slice, which carries none of the three (nor
    libvpx, libaom, Opus or libass). ios.md additionally claimed the
    simulator has "no software encoders" -- it has OpenH264, LAME,
    Vorbis, OpenCORE AMR, OpenJPEG and libjxl.

  - One RID does not always mean one artifact: ios-arm64,
    ios-sim-arm64, maccatalyst-arm64 and maccatalyst-x64 are four
    builds merged into a single ios-token asset (the two Catalyst RIDs
    lipo-fuse into one universal slice, giving three xcframework
    slices). README, docs/install/README and the RID vocabulary note
    all described per-RID tarballs.

  - musl has needed no `apk add` since the C++ runtime became statically
    linked. docs/install/README still told consumers to install packages,
    and a comment in platform/linux.sh still called it "a standard
    apk add libstdc++ libgcc on any musl host".

Also corrected: SECURITY.md said upstream sources are never patched (the
build patches FFmpeg for Catalyst, in the open, in scripts/); LICENSE and
SOURCE_OFFER.txt described legal/ contents and a version list that do not
exist -- versions live in deps.json, license texts in legal/licenses/;
android.md told consumers to link MediaCodec themselves when libavcodec.so
already carries DT_NEEDED libmediandk.so; ios.md's MoltenVK framework list
omitted Catalyst's -framework IOKit; future-platforms.md described work
that has already landed as pending; docs/install/README's releases link
resolved under /blob/ instead of the releases page.

gen-matrix.sh could not run at all without an Android NDK
---------------------------------------------------------
Regenerating the matrices is how these docs stay true, and the generator
was dead on any host without an NDK -- exiting 2 with NO message, because
it sources the platform config with stderr redirected to /dev/null.

platform/android.sh resolves the toolchain host directory with
`ls "$NDK/toolchains/llvm/prebuilt"`. Under `set -euo pipefail` a failing
`ls` there takes the whole script down silently. gen-matrix.sh knew and
worked around it by using the runner's REAL NDK, with a /tmp fallback its
own comment admitted "isn't enough" -- so it worked on CI and nowhere
else, and the generated docs could vary with the host's toolchain
directory name.

Both ends fixed: android.sh now names the problem instead of dying in
silence, and gen-matrix.sh builds an NDK-SHAPED stub (it only reads
BUILD_* flags; no compiler is ever invoked), the same trick already used
for xcrun. Verified: output is now byte-identical between Git Bash on
Windows and Ubuntu 24.04 under WSL.

The same intent-vs-reality gap in the python probes
----------------------------------------------------
`command -v python3` answers "is something named python3 on PATH", not
"does it work". On Windows it is a Store alias stub that resolves and then
exits 49. Three scripts had already hit this and grown three copies of a
correct execution probe; matrix-consistency-test.sh (added in this same
PR) used the naive check and gen-coverage.sh used bare python3, so neither
could run outside CI.

resolve_python() in lib.sh now holds the one copy, and all five callers
use it. matrix-consistency-test.sh runs and passes locally for the first
time.

Verification
------------
All gates green on Ubuntu 24.04 under WSL: matrix-consistency (15 built,
9 desktop-tested, 6 mobile-tested), pinned-deps, renovate-coverage 59/59,
renovate-config, select-versions 11/11, gen-manifest 6/6, ledger-validate
and its test 8/8, repo-wide shellcheck -x -S warning, and every workflow
YAML parses. The nine matrix docs are regenerated from the corrected
generator, and the table cells are unchanged except shaderc, which now
correctly shows as built on the 8.x v3 cells following the libshaderc fix.

Every per-RID claim above was read out of the scripts that implement it
(platform/*.sh, 04_select_license.sh, release.yml, moltenvk.sh,
test/android.sh), not inferred from the previous documentation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…obile

Two fixes to the glslc guard, and the check that would have caught both.

glslc was being cross-compiled for the target
---------------------------------------------
Hoisting the guard out of the manylinux branch made it run for every
Vulkan RID -- including the cross-compiled ones, where 02_configure.sh
has already exported the TARGET toolchain. CMake reads CC/CXX from the
environment, so win-x64 was configuring a WINDOWS build of glslc on a
Linux runner and died at generate time:

  The install of the spirv-as target requires changing an RPATH from the
  build tree, but this is not supported with the Ninja generator unless
  on an ELF-based or XCOFF-based platform.

CMake decides ELF-ness from the compiler's output format, and mingw
emits PE. Had it generated, it would have produced a glslc.exe that
cannot run on the runner -- a worse failure than the one we got.

The host-tool build now runs in a subshell with CC, CXX, AR, RANLIB, LD,
NM, STRIP, the *FLAGS, the PKG_CONFIG_* triple, CMAKE_TOOLCHAIN_FILE and
SDKROOT cleared, so CMake picks the platform's default host compiler --
the condition this code had when it lived in the manylinux branch.

Reproduced and verified with a minimal CMake project under WSL Ubuntu
24.04: with mingw CC/CXX exported, cmake -G Ninja fails with that exact
RPATH error; with them unset it configures, builds, and emits a native
x86-64 ELF executable.

Two more ways the same hoist broke Apple
-----------------------------------------
  - `-j"$(nproc)"`: macOS has no nproc. The repo already carries NPROC
    for this (02_configure.sh sets nproc, apple.sh sets
    "sysctl -n hw.ncpu"); this call was the one place not using it.
  - the macOS brew list had neither shaderc nor ninja, so Apple had no
    glslc at all and no generator for the fallback source build. Both
    added: Homebrew's shaderc is the fast path, the capability probe
    still verifies it, and ninja backs the fallback.

Mobile slices were verified by intent alone
--------------------------------------------
check_claimed_capabilities has to RUN ffmpeg, so it only ever covered
the RIDs whose binaries execute on a runner. The mobile slices were
checked with check_config -- which greps the configure string, i.e. asks
"did we request it".

Measured in the published 9.0.1.7 and 8.1.2.7 artifacts: android-arm64,
ios-arm64 and the maccatalyst slice each carry --enable-vulkan (iOS also
--enable-vulkan-static) and ZERO Vulkan filters. Every mobile job was
green. The earlier count of "four affected RIDs" came from the CLI
listing, which these slices do not have -- so they were never measured
at all, not measured and found clean.

check_claimed_capabilities_static closes it. A cross-built library can't
be executed but it can be READ: every registered component's name is a
string in the library that registers it (the .name field of AVFilter /
FFCodec / AVOutputFormat). The ff_* registration symbols are useless for
this -- hidden in a shared build and stripped besides, measured at 0 hits
via nm, nm -D and readelf on a library whose filters are definitely
present. Matching is exact-line, never substring: "scale" occurs 67 times
in libavfilter and would pass anything.

capabilities.tsv grows two columns (static-library, static-string) so one
table drives both checks and they cannot drift apart. 28 of its 33 rows
are statically checkable; the five that are not are desktop-only
hwaccels.

Proven against the published artifacts, which is the point -- it FAILS
all three mobile slices on exactly the real defect and confirms the other
23 claimed capabilities are genuinely present, with no false positives
across 72 assertions:

  [FAIL] capability: --enable-vulkan is in the configure line but
         'scale_vulkan' is NOT registered in libavfilter
  [INFO] capability check (static): 24 claimed, 1 silently missing

Also regression-checked that widening the table did not break the CLI
variant: `read -r flag opt re` would have folded the two new columns into
${re} and silently stopped matching every row. It reads five fields now,
and still matches 25 rows against a real configure string.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…by guard

Two remaining findings from the defect-class hunt, both the same shape as
the rest: a thing is requested, and nothing checks that it arrived.

SPIR-V headers were gated on the wrong consumer
------------------------------------------------
spirv-headers.sh installed only when WHISPER_BACKEND == vulkan, and then
only for an explicit RID allowlist. But FFmpeg's own Vulkan code needs
these headers independently of whisper's backend -- n9's swscale SPIR-V
backend compiles only behind HAVE_SPIRV_HEADERS_SPIRV_H /
HAVE_SPIRV_UNIFIED1_SPIRV_H.

The allowlist excluded linux-armhf, win-arm64 and every Apple target --
all of which set BUILD_VULKAN=1 on the v3 cells. Those RIDs enabled
Vulkan in configure and then built it against whatever headers the host
happened to have, or none at all.

Now gated on either consumer. The install is header-only (plus its CMake
config) and adding headers can only enable code paths, never remove
them. The RID allowlist is gone: every RID that enables Vulkan gets the
pinned headers, so the set cannot drift from BUILD_VULKAN again.

Honest scope note: whether this specific header gap is what removed the
swscale SPIR-V backend on those RIDs is NOT confirmed. Confirming it
needs a build's config.h, which no published artifact carries -- probing
libswscale's strings was inconclusive. The gate was wrong either way, and
check_claimed_capabilities_static now reports what actually lands.

mbedTLS was shown as "not built" in every cell of every matrix
--------------------------------------------------------------
gen-matrix.sh derives what-we-build by scanning scripts/deps/*.sh for
--enable- tokens. mbedTLS has none by design: it is SRT's and librist's
transport crypto, linked into those libraries rather than into FFmpeg
("No FFmpeg --enable-* flag", mbedtls.sh's own header says). So its row
matched no token and rendered as an em dash on all 15 RIDs in all 8
matrix files -- including the v3 cells, where BUILD_MBEDTLS=1 and it is
genuinely built and linked into both transports.

Rather than special-case mbedTLS, any active dep that has a BUILD_ guard
but no --enable- token now drives its row from that guard, keyed by the
script's own name, for the rows that exist in DESC. Support libraries
with no row of their own (brotli, highway, kissfft, libexpat, libpng)
are unaffected.

That generalisation immediately found a second instance: libdrm's
Version column was blank because the same scan never associated it with
its ledger key. It now shows libdrm-2.4.134.

Regenerated output: mbedTLS is now the correct per-cell answer -- built
on v3 (v3.6.7), licence-excluded on v2, which is right, since mbedTLS is
Apache-2.0 and the v2 series takes no Apache-2.0 dependency.

Verification
------------
The regenerate touches exactly two rows across the eight matrix files
(16 lines), and nothing else. All gates green on Ubuntu 24.04 under WSL:
matrix-consistency, pinned-deps, renovate-coverage, renovate-config,
select-versions 11/11, gen-manifest 6/6, ledger-validate and its test
8/8, repo-wide shellcheck -x -S warning.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lity gaps

linux-armhf failed on the new libvmaf xxd guard
------------------------------------------------
The guard was right, the package list was not. xxd went into five of the
six apt/apk lists and not the armhf one, so armhf hit the hard failure the
guard is supposed to prevent:

  ERROR: xxd not found; libvmaf would build with NO built-in models.

Audited all eight package arrays this time instead of patching the one
that failed: only armhf was short. The guard's own message also named the
wrong Debian package -- xxd was split out of vim-common in Debian 11 /
Ubuntu 22.04, so "vim" was never right.

Sub-capabilities: measured, then asserted
------------------------------------------
A top-level stack can be enabled while codecs inside it are silently
dropped: FFmpeg probes NV_ENC_PIC_PARAMS_AV1, CUVIDAV1PICPARAMS and the
VAAPI HEVC/AV1 structs SEPARATELY from the stack, so an older header set
yields --enable-nvenc with no AV1 NVENC and nothing notices.

Rather than guess, every candidate was measured in the published 9.0.1.7
artifacts of all six RIDs that claim the relevant flags. All were present,
so the ten new rows are regression guards that pass today, not new
failures:

  --enable-nvenc        hevc_nvenc, av1_nvenc
  --enable-cuvid        av1_cuvid
  --enable-vaapi        hevc_vaapi, transpose_vaapi
  --enable-libvpl       hevc_qsv, av1_qsv
  --enable-mediacodec   hevc_mediacodec
  --enable-videotoolbox hevc_videotoolbox
  --enable-d3d11va      hevc_d3d11va
  --enable-libsrt       srt (static side added)

Deliberately NOT asserted: scale_vt. Measured present on osx-arm64 and
absent on ios-arm64, both claiming --enable-videotoolbox. It needs
VTPixelTransferSessionCreate, which the iOS SDK does not expose -- a real
platform difference, not a silent loss, and this table has no per-RID
scoping by design.

Two false-negative classes in the static check, found by measurement
---------------------------------------------------------------------
Running the widened table against real artifacts immediately failed
linux-x64 on librist. `ffmpeg -protocols` lists rist there, so the CHECK
was wrong, not the build. Two causes, both real:

  1. strings(1) defaults to a 4-character minimum, so a 3-character name
     like "srt" never appears at all. Now -n 2.
  2. The linker tail-merges a string that is a SUFFIX of another: linux-x64
     libavformat stores only "librist", so "rist" has no standalone entry,
     while linux-arm64 has one. Same shape as "flip_vulkan" inside
     "hflip_vulkan", which is why strings finds 16 of linux-x64's 18
     Vulkan filters.

Class 2 cannot be told apart from genuine absence by reading strings, so
when the exact name is missing but some string ENDS with it, the row is
reported INCONCLUSIVE and does not fail. The check now only fails when the
name is absent and nothing could have absorbed it.

Had this shipped as written, it would have failed linux-x64 in CI on a
build that is correct.

Verification
------------
~200 assertions across six published 9.0.1.7 artifacts (linux-x64,
linux-arm64, win-x64, android-arm64, ios-arm64, osx-arm64):

  linux-x64    33 claimed, 0 missing, 1 inconclusive (rist, CLI-confirmed present)
  linux-arm64  31 claimed, 0 missing, 0 inconclusive
  win-x64      32 claimed, 1 missing  -- scale_vulkan
  android      26 claimed, 1 missing  -- scale_vulkan
  ios-arm64    26 claimed, 1 missing  -- scale_vulkan
  osx-arm64    28 claimed, 1 missing  -- scale_vulkan

No false positives. The only failures are the real Vulkan defect, which
is now measured on FOUR more RIDs than previously believed: glslc was
only ever installed on the manylinux/Alpine path, so spirv_compiler was
off everywhere else. linux-x64 and linux-arm64 have all 18 filters;
win-x64, android, ios, catalyst and osx have none.

That also corrects an earlier claim of mine that macOS was verified good
because its test executes scale_vulkan: assert_vulkan_device is new in
this branch and is not on main, so it has never run against a release.

All gates green on Ubuntu 24.04 under WSL; repo-wide shellcheck clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e can prove

The capability table can only check things FFmpeg registers by name. Three
suspected gaps from the defect-class hunt are invisible to it, because they
are properties of a dependency's own build, not a registered component.
The evidence exists at build time -- it just was not being read.

All three are the same shape as the xxd/libvmaf bug already fixed here: a
build system treats something as optional, finds it missing, warns (or says
nothing), and produces a working-but-degraded library that nothing notices.

libsrt: encryption requested, never verified
---------------------------------------------
-DENABLE_ENCRYPTION=ON with USE_ENCLIB is a REQUEST. If find_package
resolves to nothing usable, the crypto layer can drop out while the build
succeeds and srt:// still works -- in the clear. FFmpeg's -passphrase and
-pbkeylen options prove nothing either way: they live in FFmpeg's own
libsrt wrapper and exist regardless of what libsrt was built with.

Now asserted through libsrt's HaiCrypt layer, which is compiled only with
ENABLE_ENCRYPTION=ON. Measured present in the published linux-x64 9.0.1.7
artifact first, so this is a regression guard on known-good behaviour.

librist: same, via meson introspection
---------------------------------------
-Duse_mbedtls=true is likewise a request. Rather than infer the outcome
from source layout, ask meson what it actually resolved, through
build/meson-info/intro-dependencies.json.

The query is schema-tolerant (meson has shipped that file as a bare array
and wrapped in an object) and treats a recorded found:false as not found,
since meson records missed dependencies rather than omitting them. Tested
against all three shapes: found -> pass, found:false -> fail, absent ->
fail. shellcheck caught a real bug while writing it -- jq's $n was inside
double quotes, so bash expanded it to empty and the query would have
matched everything.

libass: x86 assembly can vanish into a warning
-----------------------------------------------
meson defaults -Dasm to auto and downgrades a missing or pre-2.10 NASM to
a warning, leaving scalar subtitle rasterisation that works and is simply
slower -- so nothing downstream notices. We control NASM through the
package lists, so on the x86-64 RIDs it is now an error. android-x64 stays
the deliberate exception (meson's Nasm backend cannot emit PIE).

If CONFIG_ASM is absent from config.h entirely, that is also an error: the
probe cannot answer the question, so it fails rather than passing blind.
Tested: ASM=1 passes, ASM=0 fails, macro-absent fails, missing config.h
fails, non-x86 RIDs skip.

libvmaf: AVX-512 can vanish the same way
-----------------------------------------
VMAF hard-fails below NASM 2.13.02 but only WARNS below 2.14, silently
dropping its AVX-512 kernels. The package lists pin no NASM version, so
the floor is now asserted on the x86-64 RIDs. Tested with stubbed nasm:
2.13.02 rejected, 2.14 and 2.16.03 accepted, absent rejected.

Verification
------------
Every gate was exercised against fabricated inputs on BOTH sides, not just
the passing one -- the same standard applied to the matrix gate and the
Vulkan guard earlier in this branch. All CI gates green on Ubuntu 24.04
under WSL; repo-wide shellcheck -x -S warning clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…sktop RID

check_claimed_capabilities was wired into linux.sh, macos.sh and win.sh up
with the structural checks -- before FFMPEG and RUNNER are established,
which happens ~25-75 lines later in each script.

_enum() is `"${RUNNER[@]}" "$FFMPEG" -hide_banner "$1" 2>/dev/null || true`.
With FFMPEG unset under `set -u` the command substitution dies and returns
an EMPTY listing, and an empty listing means every claimed capability looks
absent. All nine desktop test jobs would have reported ~35 missing
capabilities each: a wall of failures that reads exactly like a
catastrophic build regression, caused entirely by the checker.

My own earlier "proven against published artifacts" evidence for this
function came from a standalone harness where I set FFMPEG myself, so it
never exercised the wiring. That is the gap: I verified the function and
not its call site.

Two changes:

  - The call moves after the functional suite in all three scripts, where
    FFMPEG and RUNNER are known good (and, on armhf/Wine, where the runner
    prefix is set).

  - check_claimed_capabilities now refuses to run without a usable binary:
    if FFMPEG is unset or missing, or ffmpeg yields no -filters listing at
    all, it emits ONE clear failure saying verification did not run,
    instead of 35 saying the build lost everything. A check that cannot do
    its job must say so, not produce confident wrong answers -- the same
    principle as the rest of this branch.

Verified end-to-end rather than by inspection: scripts/test.sh run against
the published 9.0.1.7 linux-x64 artifact under WSL Ubuntu 24.04 now reports

  [INFO] capability check: 35 claimed, 0 silently missing
  Passed: 129  Failed: 2

(the 2 are include/ and lib/pkgconfig, absent because the test used the
runtime tarball rather than the -dev one). Guard tested both ways: FFMPEG
unset and FFMPEG pointing at a nonexistent path each produce exactly one
failure, not thirty-five.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A capability row that cannot fail is worse than no row: it reports
confidence it has not earned, which is the exact failure this branch
exists to remove. Two rows were doing that.

libsoxr asserted the wrong thing
---------------------------------
The row checked for the `aresample` filter. FFmpeg builds aresample
unconditionally -- its only dependency is swresample -- so the row passed
on a build with no libsoxr whatsoever. It had been "verifying" libsoxr
since the table was written.

libsoxr registers no filter, codec or format at all: it adds a RESAMPLER
ENGINE to libswresample. Measured against the artifact, the only CLI
evidence that discriminates is `-h full`, which lists

  soxr    1    ....A...... select SoX Resampler

only when libsoxr is linked. The row now queries that. _enum word-splits
its argument so a row can pass a multi-word option; every value comes from
the in-repo table.

No static equivalent: the engine name lives in libswresample, which the
static check does not map. Marked "-" rather than approximated.

d3d11va sub-capability had no CLI form
---------------------------------------
The row I added earlier used `-decoders hevc`, which matches the built-in
HEVC decoder on every build. Measured: ffmpeg lists zero per-codec d3d11va
entries under -decoders, and -hwaccels names only the stack. There is no
CLI-visible form, so the row is now static-only (hevc_d3d11va in
libavcodec), with empty CLI columns the reader skips.

Verified end-to-end against the published linux-x64 9.0.1.7 artifact:
libsoxr now reports "2 entries in -h full" instead of matching an
unconditional filter, d3d11va correctly does not apply, and the totals are
unchanged at 35 claimed / 0 silently missing / 129 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DEVELOPMENT.md still pointed at
docs/superpowers/specs/2026-08-22-dependency-ledger-design.md as the
ledger's design record. That directory was removed earlier in this branch
(39 factual defects, working notes rather than documentation), so the link
was dead on arrival.

Found by sweeping every relative markdown link in all 21 tracked .md files
against the tree rather than by noticing this one. It was the only dead
link; the sweep is clean now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…TH alone

check_claimed_capabilities_static reads CROSS-BUILT libraries, so the
host's own binutils may not understand the target object format. lib.sh
already resolves NM and READELF as `llvm-<tool> || <tool>` for exactly
that reason, and android.sh puts the NDK's llvm-* ahead of them on PATH
because macOS has no GNU readelf and its nm reads Mach-O, not ELF.

The new check used a bare `strings`, which today happens to be fine --
android-* tests land on ubuntu-latest and ios/maccatalyst on macOS, so
each host's strings already understands the format it is handed. But that
is a property of the runner matrix in test-mobile.yml, not of the check:
moving an android RID onto a macOS runner would have made it read an ELF
.so with Mach-O tooling and report capabilities as missing.

STRINGS now resolves llvm-strings first, matching the surrounding code,
and the guard fails loudly if neither exists.

Re-verified against all six published 9.0.1.7 artifacts: unchanged --
0 silently missing on both Linux RIDs, 1 inconclusive (librist tail-merge),
and the real scale_vulkan failure on win-x64, android, ios and osx.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The comment said the iOS SDK "does not expose" VTPixelTransferSessionCreate.
It does expose it -- Apple marks the API available from macOS 10.8 and iOS
16.0, and apple.sh builds iOS against -miphoneos-version-min=13.0, so
FFmpeg's configure probe cannot use it at that deployment target.

Same measured outcome (present on osx-arm64, absent on ios-arm64, both
claiming --enable-videotoolbox) but the stated cause was wrong, and this
comment is the justification for deliberately leaving a capability
unasserted -- so it needs to be right.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ryan-morris ryan-morris changed the title fix(artifacts): make published builds runnable (Windows Vulkan, musl runtime, build paths) + CI resilience fix(artifacts): verify what we ship, not what we asked for — Vulkan, capability checks, artifact runnability, CI resilience Sep 14, 2026
ryan-morris and others added 8 commits September 14, 2026 14:58
…nfig.h

The probe I added checked build/config.h for CONFIG_ASM right after
`meson setup`. It failed every Apple and x86-64 Linux/Windows build with

  ERROR: libass config.h has no CONFIG_ASM line at all on osx-x64

on builds whose own meson summary two lines earlier said

  ASM optimizations: YES

Two mistakes, both mine. libass generates config.h through a
configure_file followed by TWO vcs_tag targets, so the file does not exist
until `meson compile` -- the probe ran a build step too early. And parsing
a dependency's generated header for a macro name is fragile even when the
timing is right.

libass already answers this question itself: meson.build does

  if enable_asm
      conf.set('CONFIG_ASM', 1)
  elif asm_option.enabled()
      error(...)

so -Dasm=enabled makes libass fail with its own accurate diagnostic when
assembly cannot be enabled, instead of downgrading to a warning and
building scalar subtitle rasterisation that nothing notices. That is the
whole point of the check, obtained without a timing dependency or a macro
name to track.

Applied to every RID except android-x64, which stays explicitly disabled
(meson's Nasm backend cannot emit PIE). It is a no-op on the aarch64 RIDs,
where libass enables assembly unconditionally already.

Also switched the libsrt encryption check from strings(1) to `grep -a`.
That check runs in the BUILD environment, where the package lists do not
guarantee binutils but grep is always present. Verified both directions on
a binary fixture.

Incidentally confirmed from the same CI logs that the libvmaf NASM floor
is safe where I could not check it locally: the manylinux container has
nasm 2.15.03, above the 2.14 AVX-512 threshold.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dency entry

The check I added queried meson's intro-dependencies.json for the crypto
backend. Reading librist v0.2.20's actual sources shows that cannot work,
and that the failure it claimed to catch is not the real one.

contrib/mbedtls/meson.build resolves the external library as

    dependency('MbedTLS', method: 'cmake', modules: ['MbedTLS::mbedcrypto'])

and falls back to cc.find_library('mbedcrypto'). meson does not record
find_library results in intro-dependencies.json at all, so a perfectly good
build resolved through the fallback would have been reported as having no
crypto backend -- a false failure on every v3 cell.

The stated consequence was also wrong. If both lookups miss, librist does
not disable encryption: it sets builtin_mbedtls = true and compiles its own
vendored copy from contrib/mbedtls/library/*.c. rist:// still encrypts. The
real damage is that the artifact would carry an UNPINNED, un-Renovate-
tracked mbedTLS vendored inside librist instead of the version deps.json
pins -- silently diverging from the ledger the dependency policy rests on,
which is the more valuable thing to catch.

That fallback is unambiguous in the build tree: it declares
static_library('mbedcrypto'), so a libmbedcrypto.a anywhere under build/
means the external library was not found. Checked after `meson compile`,
which is when it would exist.

My earlier commit message for this check said rist:// "would transport in
the clear". That was wrong; correcting it here rather than leaving it in
the history unqualified.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The encryption check hardcoded ${DEPS_DIR}/lib/libsrt.a. build_cmake_dep
controls the install prefix and libdir, but the archive's filename is
SRT's to choose, and a wrong guess fails a perfectly good build.

That is the same mistake this branch has now had to correct three times:
assuming where a build system puts something (libass's config.h, which does
not exist until compile), assuming what it records (librist's mbedTLS,
resolved through a path meson does not introspect), and assuming what it
names things here. The check now finds libsrt*.a under the install libdir
and reports the directory listing if there is none.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI on this branch died with

  docker: Error response from daemon: received unexpected HTTP status:
  502 Bad Gateway

pulling quay.io/pypa/manylinux_2_28_x86_64. Nothing to do with our code --
exactly the failure class the curl and git wrappers exist to remove, in the
one network operation this branch had not covered.

`docker run` pulls its image implicitly, and that pull has no retry of any
kind. All three sites were affected: the manylinux build container
(linux-x64 / linux-arm64), the alpine build container (linux-musl-arm64),
and the alpine test container.

Each now pulls explicitly first, with the same exponential+jitter backoff
scripts/lib.sh uses. Retrying the PULL rather than the `docker run` is the
point: re-running the run would repeat the entire build. The image is then
referenced through one variable in both the pull and the run, so the two
cannot drift apart.

Verified with a stubbed docker on PATH, both directions:
  - always failing -> 6 attempts, 5 sleeps, 134s of backoff, one clear
    error, exit 1 (the git wrapper's window is 124-144s)
  - failing twice then succeeding -> 2 retries, then proceeds to docker run

All workflow YAML still parses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…2-bit ARM

The -Dasm=enabled I added applied to every RID except android-x64, which
broke linux-armhf:

  meson.build:337:8: ERROR: Problem encountered: Assembly was requested,
  but cannot be built; see prior messages.

on a RID that had been building correctly all along.

Reading libass 0.17.5's meson.build rather than assuming: it takes the
nasm path only for generic_cpu_family 'x86' (which covers x86 and x86_64)
and sets enable_asm unconditionally for 'aarch64'. Every other
architecture falls through to a warning with assembly off. armv7 is in
that last group -- libass ships no 32-bit ARM assembly at all, so
requiring it can only fail.

linux-armhf now keeps meson's `auto`. Every other RID is x86_64 or
aarch64, where the option is either the whole point (x86_64: a silent
scalar fallback becomes an error) or a no-op (aarch64: already
unconditional). android-x64 stays explicitly disabled.

That is twice this option has had to be corrected. Both times the cause
was the same: applying a rule to every RID when the upstream condition it
depends on is architecture-specific.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dored copy

The guard added earlier fired across the matrix:

  ERROR: librist fell back to its VENDORED mbedTLS
         (build/contrib/mbedtls/libmbedcrypto.a)

on win-arm64, android-arm64, android-x64 and osx-arm64 v3 cells -- and
osx-arm64 is a NATIVE build, so this was never a cross-compilation quirk.
Every v3 artifact this repo has published carries librist linked against
an unpinned, un-Renovate-tracked mbedTLS vendored inside librist, instead
of the version deps.json pins. -Dbuiltin_mbedtls=false has been set the
whole time and silently ignored.

Cause: librist resolves the library with

  dependency('MbedTLS', method: 'cmake', modules: ['MbedTLS::mbedcrypto'])

which is a CMake package lookup. Nothing pointed meson's CMake search at
DEPS_DIR -- the cross files set pkg_config_libdir only, and a native build
has no reason to look there either. The cc.find_library('mbedcrypto')
fallback missed for the same reason (no -L for DEPS_DIR), so librist took
its third path and compiled its own copy, successfully and silently.

A comment in mbedtls.sh asserted that librist consumed it "via
pkg-config". That was wrong, and being wrong in the place someone would
check is why this survived: the .pc file it describes is real, and no
consumer of it exists.

Fixed by passing -Dcmake_prefix_path="${DEPS_DIR}" for the mbedtls cell,
scoped to librist rather than changing every cross file, so the blast
radius is one dependency. Verified that cmake_prefix_path is a real meson
core option (meson 1.3.2: "List of additional prefixes for cmake to
search") rather than assuming the name.

This is the check working exactly as intended: it turned a silent
substitution of a security-relevant dependency into a build failure, on a
defect that predates this branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…gaps

Independent read-only audit of the branch. Every item below was verified
in the source or reproduced before changing anything.

A failed FFmpeg configure exited SUCCESSFULLY
----------------------------------------------
  if ! "${CONFIGURE_CMD[@]}"; then
    rc=$?          # <- 0, not configure's status
    ...
    exit "${rc}"   # <- exit 0 on failure

Inside a negated test, $? is the status of the NEGATION, which is 0
precisely when the command failed. So a failed configure ended the build
with status 0, before compile, staging or verification: the job reported
success and simply produced nothing. Reproduced in isolation (a function
returning 42 yields $?=0 in the `!` branch, and the script exits 0).

From #16, not this branch, but this branch added the config.log dump
right beside it and did not notice.

The capability table had a field-parsing bug of its own making
---------------------------------------------------------------
The d3d11va row used adjacent tabs for its two unused CLI fields. TAB is
an IFS *whitespace* character, so bash's `read` collapses runs of it: both
readers parsed `avcodec` as the ffmpeg list-option and `hevc_d3d11va` as
its regex, and the static fields came back empty. Measured, not reasoned.
Unused fields are now the sentinel "-", which both readers skip, and the
format comment says why empty is not allowed.

A static marker that could never fail
--------------------------------------
libsrt's static marker was the string "srt" -- which also names FFmpeg's
BUILT-IN SubRip subtitle muxer/demuxer, in the same libavformat. It would
have passed on a build that lost the protocol entirely. Removed; the CLI
row (^[[:space:]]*srt$ against -protocols) stays, because a protocol
listing cannot be satisfied by a subtitle format.

check_config matched prefixes
------------------------------
`grep " ${flag}\b"` treats "-" as a word boundary, so --enable-vulkan-static
satisfied a check for --enable-vulkan. Both check_config and
check_config_absent now match whole space-delimited tokens. Verified over
five cases, including check_config_absent no longer failing a build that
legitimately has only the variant.

Release policy had holes
-------------------------
The M5 fix covered build_cmake_dep's callers; four deps invoke cmake
directly and were still taking upstream defaults: libexpat, mbedtls,
whisper and -- Codex missed this one -- libx265, a video encoder. All four
now pass -DCMAKE_BUILD_TYPE=Release explicitly.

x265's ARM64 hold silently missed two ARM64 RIDs
-------------------------------------------------
deps.json holds x265 at 3.6 on ARM64 because 4.0+ ships broken aarch64
NEON intrinsics, and lists the affected RIDs. win-arm64 and
maccatalyst-arm64 were added to the repo in later PRs and never backfilled,
so both took the broken default. Exactly the shape this branch already
guards for test matrices: adding a RID means touching several places and
every way of forgetting one is silent.

Backfilled, and matrix-consistency now ASSERTS it: a ledger hold whose
platform list is entirely ARM64 must contain every ARM64 RID we build.
Proven to fail on broken input (remove win-arm64 -> exit 1) and pass on
the real ledger.

Ledger metadata that rots by design
------------------------------------
libgsm's origin said gsm-1.0.22.tar.gz while its tag was 1.0.24, because
the tarball managers update `tag` only. Harmless to the build (the URL is
built from dep_version) but wrong where a reader would check -- which is
how the librist/mbedTLS misdescription survived. origin is now the release
DIRECTORY, and ledger-validate rejects any tarball origin whose embedded
version disagrees with its tag. Proven against the real stale value.

Provisioning ran before the licence was applied
------------------------------------------------
03_install_packages ran before 04_select_license, so every v2 cell still
saw BUILD_VULKAN=1 and would build shaderc/glslc from source for a build
that then disables Vulkan entirely. 04 is pure flag logic -- no command
substitution, no external tool -- so the two are swapped: provisioning now
sees the final state.

Two more in the same block: the shaderc fallback cloned into a fixed
/tmp/shaderc, so any rerun failed on "destination already exists" rather
than on anything real (now mktemp with cleanup); and the glslc capability
PROBE sat inside the SKIP_DEPS guard, so the documented local-build path
skipped it and could produce Vulkan-less artifacts silently. The probe now
always runs; only the shaderc build is suppressed, with a message naming
the version to install.

All gates green on Ubuntu 24.04 under WSL; repo-wide shellcheck clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… to check

Answering two questions that came out of the audit: is anything we link
missing from the coverage matrix, and what is actually worth verifying.

The matrix was almost complete
-------------------------------
Of the 57 libraries/features the build enables, exactly ONE had no matrix
row: zlib. It is enabled on every RID but is not in any of FFmpeg's
EXTERNAL_LIBRARY_* lists -- it is a system library, not an external one --
so the universe the matrix is built from never contained it. Added through
EXTRA, the same mechanism MediaFoundation/SChannel/SecureTransport use.

The regenerate also surfaces the x265 ledger backfill from the previous
commit: win-arm64's column moves from 4.2 to 3.6, which is what the ARM64
hold was always supposed to do.

Verification had 15 gaps; 7 were real
--------------------------------------
Diffing every --enable- flag against everything any test actually checks
(capabilities.tsv, check_registry's _enc_iff, exercise_tls, the Vulkan and
whisper probes) leaves 15 unverified. Eight of those register no component
of their own -- zlib, lcms2, libdrm, fontconfig, fribidi, harfbuzz, cuda,
ffnvcodec are support libraries whose effect is only observable through
another feature. A row naming one could only ever match something else,
which is the false confidence this table exists to remove, so they stay
out and the table says why.

The other seven register a named component and are now rows: libdav1d,
both opencore-amr variants, libvo-amrwbenc, audiotoolbox (aac_at),
mediafoundation (h264_mf) and v4l2-m2m (h264_v4l2m2m).

Hardware: the half that needs no hardware
------------------------------------------
The hw stacks are the likeliest thing to break and CI has no GPU -- but
that risk splits in two, and only one half needs one. FFmpeg probes
NV_ENC_PIC_PARAMS_AV1, CUVIDAV1PICPARAMS and the VAAPI HEVC/AV1 structs
SEPARATELY from the stack, so an SDK/header change yields --enable-nvenc
(or vaapi, or amf) with the AV1 entry point quietly gone. That is a
build-time drop, plainly visible in the registry.

Swept all of them against published artifacts: 28 hw sub-capabilities
across nvenc, cuvid, vaapi, qsv, amf, mediafoundation, mediacodec and
videotoolbox are present today -- and AMF had no row of ANY kind. Added
h264_amf (stack anchor), av1_amf, av1_vaapi and av1_mediacodec: the
SDK-gated AV1 probes plus the unanchored stack, rather than all 28, to
keep the maintenance honest.

Whether a GPU actually encodes is a different question that genuinely
needs hardware. Not faked here.

Adding rows is now free
------------------------
_enum is called once per ROW, and rows share a handful of options, so each
addition used to cost another ffmpeg process -- which makes broadening
coverage quietly expensive and is a real argument against doing it. It is
memoised per option now: one invocation per distinct listing no matter how
many rows use it.

Verified against six published 9.0.1.7 artifacts: coverage rises (linux-x64
32 -> 37 claimed) with ZERO new failures; the only failure anywhere is
still the real scale_vulkan defect. All gates green; shellcheck clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ryan-morris and others added 3 commits September 14, 2026 16:19
…stale docs

Working through the remainder of the audit. Behavioural items (real GPU,
encrypted loopback, on-device execution) are tracked in #27 rather than
faked; everything concrete is here.

The release path had no network retry at all
---------------------------------------------
The resilience work covered curl, git, checkout and docker, and stopped at
the release. Two places, both of which run when the release is ALREADY
PUBLIC -- so a blip does not fail retryably, it leaves a published release
missing an artifact or its SHA256SUMS:

  - release.yml's `gh release upload` for every asset and -dev tarball.
    Retried with the same exponential+jitter backoff; --clobber keeps it
    idempotent, so a partially-uploaded asset is replaced rather than
    duplicated.
  - gen-manifest.sh's two I/O points (asset enumeration and the
    digest-fallback download). Both read-only and idempotent, so repeating
    one is always safe.

check_tls could not count
--------------------------
It claimed "exactly one TLS backend" but used a first-match `case`, which
cannot tell one from two. Now counts: 0 is a pass only on the lgplv2
signature, 1 names the backend, and more than one fails. Tested over five
configure strings including the two-backend case.

exercise_tls accepted the absence of evidence
----------------------------------------------
`[ -z "$out" ]` counted as a successful handshake, and the exit status was
never captured. Empty output is also exactly what a silently-dead ffmpeg
produces. It now requires positive evidence: the demux complaint that
proves bytes arrived, or empty output WITH exit 0. Empty output with a
nonzero status is a failure naming the status, not a pass.

The genuinely ambiguous case (unclassified output) stays a skip on purpose
-- it is a runner-network symptom, and turning it into a gate would make
CI flaky without finding defects.

Docs that contradicted verified behaviour
------------------------------------------
  - DEVELOPMENT.md said glibc "had no xxd at all and fell back cleanly".
    There is no clean fallback: libvmaf emits its built-in models only
    inside `if xxd.found()`, so those artifacts shipped a libvmaf filter
    that registered and then returned -EINVAL for its own default model.
  - Two places still told contributors to hand-build a fake NDK before
    running gen-matrix. It makes its own stub now, which is what keeps its
    output byte-identical across hosts.
  - The contributor guide defined "done" as "compiles, and config.h shows
    CONFIG_MYLIB=1" -- the intermediate-result standard this whole branch
    exists to replace. config.h says configure accepted the library, not
    that anything it provides reached the artifact. Done is now: the
    component is REGISTERED in the staged artifact, with a row in
    capabilities.tsv naming it.
  - 06_build_libraries.sh still described SPIRV-Headers as a whisper-only
    dependency on a subset of platforms, after the gate was widened to
    every Vulkan RID for FFmpeg's own use.
  - libsrt.sh's header called the library "dynamic-linked" while the build
    sets ENABLE_STATIC=ON / ENABLE_SHARED=OFF. Corrected, with the actual
    MPL-2.0 reasoning (file-level copyleft; static linking is fine as long
    as the library's own sources stay available, which SOURCE_OFFER.txt
    and legal/licenses/ cover).

And three whitespace-only lines in 08_stage_artifacts.sh that made
`git diff --check main...HEAD` fail. It is clean now.

All gates green; shellcheck clean; gen-manifest-test still 6/6 and still
instant, so the retry wrapper does not slow the stubbed paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…asked for

Whisper could silently fall back to CPU
----------------------------------------
whisper.sh assembles the ggml link line from whichever archives happen to
be installed. That is correct for the OPTIONAL backends -- ggml
auto-enables BLAS on Apple -- but it silently tolerates the REQUESTED one
going missing.

If ggml's cmake cannot find Vulkan or Metal it falls back to CPU without
failing. libggml-vulkan.a simply would not exist, the loop would skip it,
and whisper would register and transcribe correctly on the CPU. Every test
passes: the filter is present, inference works, and nothing anywhere
states which backend ran. It is the GPU equivalent of the Vulkan-filter
defect this branch exists for, in the path least visible from outside.

The archive matching WHISPER_BACKEND is now required, and libggml-cpu.a is
required unconditionally (it is the fallback path). A miss names the
backend, the RID and the archives that WERE installed.

This is the part of the hardware question that needs no hardware: not
"does the GPU encode", but "was the GPU backend even built in".

A failed upload left a PUBLIC, incomplete release
--------------------------------------------------
The release was created public, with --latest, and the assets uploaded
afterwards by two matrix jobs and the SHA256SUMS manifest. Any failure in
those left a published release -- possibly badged "Latest" -- missing
artifacts or its checksums, which consumers would fetch.

It is created as a DRAFT now, and a new `finalize` job publishes it after
`manifest`, i.e. once every asset and the checksum file are attached. If
anything earlier fails the release stays a draft: invisible, and there for
a human to inspect or delete. --latest moves to finalize, where it is
meaningful; MARK_LATEST is recomputed from deps.json rather than passed
through, because `release` is a matrix job whose outputs would be
last-wins across version lines.

Honest caveat: this path is NOT exercised by PR CI -- release.yml only
runs on a release. The YAML parses and the job graph is
prepare -> ... -> publish/publish-ios -> manifest -> finalize, but the
first real proof comes from an actual release.

All gates green; shellcheck and `git diff --check` clean; all workflow
YAML parses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… the artifact

Cross builds still shipped a vendored mbedTLS
----------------------------------------------
The -Dcmake_prefix_path added last commit fixed the NATIVE cells; armhf and
maccatalyst-arm64 still failed the guard. Rather than guess again, I
measured what meson actually honours, with a minimal project and a stub
MbedTLSConfig.cmake:

  native, no hint ................................. not found
  native, -Dcmake_prefix_path ..................... FOUND
  native, CMAKE_PREFIX_PATH env ................... FOUND
  cross, [properties] cmake_prefix_path ........... not found
  cross, control (no property) .................... not found

meson's cmake dependency lookup cannot see our prefix on a cross build at
all -- including through the documented machine-file property. So the
cmake route is a dead end there, and librist's second path,
cc.find_library('mbedcrypto', has_headers: ['mbedtls/aes.h']), was missing
only for want of -L/-I. Confirmed the same way:

  cross, no search paths .......................... not found
  cross, c_args -I + c_link_args -L ............... FOUND

Every cross file now carries -I${DEPS_DIR}/include and -L${DEPS_DIR}/lib,
appended to the existing arrays on the Apple targets so their -target /
-isysroot survive. pkg_config_libdir already covered deps that ship a .pc;
this covers the ones a consumer looks up any other way, which is why the
vendored-mbedTLS fallback went unnoticed for so long.

Both halves are needed -- -Dcmake_prefix_path for native, search paths for
cross -- and the post-build guard proves whichever one applied.

The GPL boundary was checked only against the configure string
---------------------------------------------------------------
check_license_boundary read the embedded flags and stopped. A stale
DEPS_DIR carrying a previous GPL cell's libx264.a would yield an
LGPL-CONFIGURED build with a GPL encoder inside it, and every check passed.
This is the one place where being wrong is a licensing problem rather than
a capability one.

It now also reads libavcodec directly and asserts the encoder is absent.
Works on every RID including the cross-built slices, since it needs no CLI;
the configure string lives in libavUTIL, so it cannot cause a false
positive. Proven both ways on real artifacts: PASSES on the published
lgplv3 linux-x64, and FAILS when libx264 is injected into its libavcodec.

Static-only rows were unreachable on the platforms they describe
-----------------------------------------------------------------
Desktop RIDs ran only the CLI checker, which can see a row only if it has a
CLI form -- so hevc_d3d11va, which ffmpeg lists nowhere, was never checked
on Windows, the one platform it applies to. All three desktop scripts now
run the static variant too, via a resolver that handles every layout this
repo stages. linux-x64 goes from 41 assertions to 41 + 37.

All gates green; shellcheck clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…coverage

Three more from the audit, all the same shape: a check or a comment that
claims more than it delivers.

Windows PE audits went quiet when their tool was missing
---------------------------------------------------------
win.sh's runtime-DLL, Vulkan-hard-import and import-library-architecture
audits each began with

  command -v llvm-readobj || { info "...skipping..."; return; }

so if the runner image ever stopped shipping llvm-readobj, all three would
report nothing and the job would pass. Those are precisely the checks that
caught the undeployable Windows artifact (a hard vulkan-1.dll import), so
silence is the one outcome they must not produce.

They use _tool_missing now, which the repo already has for exactly this:
fail under GITHUB_ACTIONS, degrade to a skip on a developer machine.

The mobile smoke test claimed more than it did
-----------------------------------------------
Two of its messages overstated: "whisper filter present" and "https + tls
protocols present (TLS backend wired)". The first is registration only --
no inference runs -- and the second enumerates protocol names without a
handshake. Both now say so. (The encode+decode roundtrip message stays as
it is: that one really does execute.)

Relabelling rather than implementing: real Whisper inference and a real
handshake on mobile need runtime infrastructure, tracked in #27.

A code comment asserted a blocker that no longer exists
--------------------------------------------------------
windows.sh explained win-arm64's CPU-only Whisper as "the ggml-vulkan path
needs a dlltool-synthesised vulkan-1 import library ... follow-up work".
This branch REMOVED the dlltool approach -- it created the hard
vulkan-1.dll import that made Windows v3 artifacts fail to start -- and
win-x64 now uses the Vulkan-Shim-Loader. The blocker named in the comment
had ceased to exist while the comment stayed.

What actually keeps Vulkan off win-arm64 is that BUILD_VULKAN_SHIM is set
for win-x64 only. Whether to extend it is a genuine open question (does the
shim build under llvm-mingw/aarch64; is Vulkan on Windows-on-ARM worth the
surface), so it is issue #28 now, and the comment states the real position
instead of a stale one.

That failure mode is worth naming: a comment that is wrong in the place
someone would check is how the librist/mbedTLS substitution survived for
months.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ryan-morris and others added 9 commits September 14, 2026 17:20
… was never set

The staging guard caught linux-musl-arm64 depending on libgcc_s.so.1. The
diagnostics artifact from the failing job gave the answer directly:

  LD=gcc
  LDFLAGS= -static-libgcc -L/work/.build/linux-musl-arm64/deps/lib ...
  EXTRALIBS=-lm -lstdc++ -lstdc++ -lstdc++ -lstdc++

-static-libgcc WAS being passed (and does work -- verified separately that
it removes libgcc_s from a C++ shared library). The problem was the four
dynamic -lstdc++ entries dragging the shared C++ runtime in behind it.

Cause: chromaprint, libjxl, libplacebo, libsrt and libvmaf each append
${CXX_RT_LIB--lstdc++}, and CXX_RT_LIB was set for win-arm64 ONLY. On musl
all five took the dynamic default.

whisper.sh was already static on musl -- but through its own variable,
CXX_STATIC_LIB. That is precisely why this survived the earlier musl work:
one of the six consumers was fixed, and because the name differed, nothing
showed that the other five still had the dynamic default. The same shape as
xxd going into five package lists and missing armhf.

Three layers, so it cannot come back:
  - platform/linux.sh sets CXX_RT_LIB=-l:libstdc++.a for both musl RIDs, so
    all five deps emit the archive form.
  - whisper.sh derives CXX_STATIC_LIB from CXX_RT_LIB; the two can no longer
    drift.
  - 07_build_ffmpeg.sh refuses to configure a musl build whose EXTRA_LIBS
    still contains a bare -lstdc++. Staging already caught the symptom after
    the fact; this names the cause before the build. Tested: rejects
    "-lm -lstdc++", accepts "-lm -l:libstdc++.a" and "-lm".

Scope, measured against the published 9.0.1.7 artifacts: BOTH musl RIDs
ship libstdc++.so.6 AND libgcc_s.so.1 on nearly every library today --
libavcodec, libavfilter, libavformat, libavutil, libswscale. main sets no
-static-libgcc at all, so every musl artifact this repo has released is
unable to start on a bare Alpine image, exactly as the docs promised it
could. This branch fixed the libstdc++ half earlier; this is the rest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard tripped again on linux-musl-arm64, this time with everything it
was supposed to check already correct:

  LD=gcc
  LDFLAGS= -static-libgcc -L.../deps/lib -Wl,--as-needed ...
  EXTRALIBS=-lm -l:libstdc++.a -l:libstdc++.a -l:libstdc++.a -l:libstdc++.a
  musl: EXTRA_LIBS carries no dynamic C++ runtime.     <- the new pre-configure guard passed

and libavfilter STILL needs libgcc_s.so.1. Since --as-needed is in effect,
that DT_NEEDED means symbols really are being resolved from the shared
libgcc, not merely offered.

Verified separately on glibc that -static-libgcc DOES remove libgcc_s from
a C++ shared object, linked either by g++ or by gcc with -l:libstdc++.a. So
the remaining cause is specific to the Alpine/musl toolchain or to aarch64,
and there is no Alpine available locally (no docker in WSL or on the host)
to reproduce it.

Rather than infer from outside the container a third time, the guard now
prints what distinguishes the possibilities:

  - the LDFLAGS and EXTRALIBS the build actually configured
  - whether this toolchain even HAS libgcc_eh.a / libgcc.a
    (if -static-libgcc has no static unwinder to point at, it cannot work)
  - the exact symbols the offending library takes from libgcc_s.so.1

That last line is the one that decides it: _Unwind_* means C++ exception
unwinding and wants a different link fix; __aarch64_* means outline atomics
and is an -mno-outline-atomics / libgcc.a question; anything else points
somewhere new. Each has a different answer, and guessing between them has
already cost two rounds.

The two previous root causes on this branch (the librist vendored mbedTLS
and the five deps still linking -lstdc++) both came from a CI diagnostics
artifact rather than from reasoning. This makes the next failure carry its
own diagnosis instead of requiring the artifact to be fetched and read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ling on it

The self-diagnosing guard answered its own question on the first run:

  LDFLAGS= -static-libgcc ...                              present
  EXTRALIBS=-lm -l:libstdc++.a -l:libstdc++.a ...          fully static
  libgcc_eh.a  /usr/lib/gcc/aarch64-alpine-linux-musl/15.2.0/libgcc_eh.a
  symbols taken from the shared libgcc:                    <empty>

Everything the guard was checking for was already correct, and libavfilter
needed ZERO symbols from libgcc_s.so.1. Alpine's gcc adds -lgcc_s for
-shared in a form that -Wl,--as-needed does not strip, so the entry is pure
noise -- but noise with teeth: the loader would demand a library that a bare
Alpine image does not ship and that nothing in the artifact calls into.

Three rounds were spent inferring this from outside the container. The
diagnostic found it on its first run, which is the argument for
instrumenting a guard rather than reasoning at it.

Staging now distinguishes the two cases instead of treating every
DT_NEEDED as fatal:

  - no symbol taken from it -> remove the entry with patchelf (exactly what
    --as-needed is for) and say so; the removal is then VERIFIED, because
    "strip the error away" must not be indistinguishable from hiding it.
  - symbols genuinely taken -> unchanged, still a hard failure naming the
    library.

It also fails closed: if the dependency cannot be located to compare
symbols against, it is treated as used.

Both paths tested with stubbed patchelf/nm/gcc: a spurious entry is
removed, a real one is left for the existing failure to catch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…'s limits

Two accuracy fixes from the audit tail. Neither changes behaviour; both stop
a check claiming more than it delivers, which is the failure mode that let
the librist substitution and the Vulkan gap survive.

09_verify_build.sh described config_components.h entries as "registered".
They are not: that header records what FFmpeg's configure DECIDED, an
intermediate result. The two genuinely differ -- the artifacts shipped with
CONFIG_*_VULKAN set and zero Vulkan filters in the binary on 9 of 15 RIDs.

The gate keeps its place: it fails early, cheaply, before staging, naming a
precise macro. But its header now says it is a first filter rather than the
finish line, and points at the artifact-level check in scripts/test. It also
says outright: do not offer "it is in config_components.h" as evidence that
something works; add a capabilities.tsv row instead.

capabilities.tsv now documents the static side's three known limits instead
of leaving them to be rediscovered:
  - it proves a NAME is present, not that the component is reachable
    through a registry;
  - it cannot tell an encoder from a decoder, since both share one name
    (the CLI side can, and does, because it queries -encoders/-decoders);
  - a name that is a suffix of a longer string is inconclusive, not missing.

All three are why the CLI variant stays the stronger check and the static
one exists only for slices with no runnable binary. Closing them properly
means verifying against a runtime registry, which is tracked in #27 rather
than approximated here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… nothing

The first test jobs of the run reached the musl cells and failed twice each.
The capability system itself passed everywhere it ran -- 34 claimed / 0
missing on the CLI side, 30 / 0 on the static side -- so both failures are
in the harness, not the artifacts.

patchelf was missing from the Alpine test containers
-----------------------------------------------------
  [FAIL] patchelf unavailable - cannot verify the musl C++ runtime invariant

The check behaved correctly: it refuses to pass when it cannot do its job.
But the container's apk list never included the tool it needs, so the musl
C++ runtime invariant -- the one this branch spent three rounds fixing --
could not be verified at all. Added to both Alpine test containers.

The ffprobe check threw away its own evidence
----------------------------------------------
  [PASS] ffmpeg runs (ffmpeg version 8.1.2 ...)
  [FAIL] ffprobe does not run

That was the entire message. It ran ffprobe with >/dev/null 2>&1, so the
interesting part -- why ffprobe fails where ffmpeg, in the same directory
against the same libraries, succeeds -- was discarded. Confirmed from the
artifact that both have identical DT_NEEDED and neither needs a host C++
runtime, so the cause is something the check refused to report.

It now reports exit status and output like the ffmpeg check immediately
above it, and additionally says whether the file exists and is executable.
A check that cannot say why it failed is precisely the pattern this branch
exists to remove; it should not survive inside the suite enforcing that.

Also corrected the comment above the apk list, which still described
libstdc++/libgcc as "a standard runtime dep ... not something we can
avoid". That was true of the artifacts as published and is the thing this
branch fixed: they are installed for the TEST HOST, to build the smoke
program, and emphatically not because the artifact needs them -- linux.sh
asserts the opposite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…cannot have

First full CI run: 228 pass, 22 fail — 16 musl, 2 ios-sim, 4 roll-ups.

My own fix was shipping a broken musl artifact
-----------------------------------------------
The ffprobe check, now that it reports instead of discarding its output,
named the cause on the first run:

  ffprobe does not run (exit 127): Error relocating libavfilter.so.12:
    __popcountdi2: symbol not found
    _Unwind_GetRegionStart: symbol not found
    _Unwind_SetGR: symbol not found

Those are libgcc symbols. libavfilter genuinely needs libgcc_s, and the
previous commit's `patchelf --remove-needed` stripped it, producing a
library that cannot load at all.

The flaw was in my reasoning, not the tooling: I treated an EMPTY `nm`
result as "no symbols required" when it actually meant "could not
determine" -- nm returned nothing for Alpine's libgcc_s.so.1, and I read
that silence as proof of absence. That is precisely the mistake this branch
exists to eliminate, committed inside the guard meant to enforce it.

Reverted. A host C++ runtime dependency is treated as REAL again and fails
the build. Whether the answer is a link change or accepting the dependency
is a decision to make deliberately, not something to patch away in staging.

Two things worth keeping from the episode: the diagnostic I added one
commit earlier is what caught it, and it caught it immediately. And the
musl artifacts are no worse than before -- this restores the previous
behaviour, where the guard fails loudly rather than shipping something
broken.

ios-sim-arm64 claimed Vulkan it structurally cannot provide
------------------------------------------------------------
  capability: --enable-vulkan is in the configure line but 'scale_vulkan'
  is NOT registered in libavfilter

Real finding, caught the first time the capability check reached that RID.
The chain: the lean simulator slice sets BUILD_LIBPLACEBO=0; shaderc.sh
returns early unless BUILD_LIBPLACEBO=1; without shaderc there is no SPIR-V
toolchain, so FFmpeg dropped every Vulkan filter while --enable-vulkan
stayed in the configure string.

Re-adding the whole shaderc/SPIR-V chain to a slice that deliberately drops
x264, x265, kvazaar, vpx, aom, opus and the entire text stack would
contradict what the slice is for. So the fix goes the other way: stop
claiming it. If a flag is on the feature must be there, and here it cannot
be. Matrix regenerated accordingly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…eeded

Goal restated from the requirement: a Dockerfile should be able to grab a
release artifact, extract it, and run — no package juggling, and no
dependency list that can quietly grow later.

Measured split in the branch build (musl-arm64):

  libavutil, libavcodec, libswscale   0 libgcc references   clean
  libavformat                        13 versioned refs
  libavfilter                        24 versioned refs

and the references are exclusively C++ exception unwinding plus two
128-bit float helpers:

  _Unwind_Resume@GCC_3.0  _Unwind_GetIPInfo@GCC_4.2.0
  _Unwind_RaiseException@GCC_3.0  __multf3@GCC_3.0  __divtf3@GCC_3.0

The @GCC_3.0 tags are the whole problem: a versioned reference can only be
satisfied by a SHARED library carrying that version node. libgcc_eh.a
exports the same symbols unversioned, so once a reference is bound to
@GCC_3.0 no archive can satisfy it and libgcc_s.so.1 is a hard runtime
dependency.

Why only those two libraries: FFmpeg links its DSOs with LD=gcc, not g++.
The C driver honours -static-libgcc for libgcc itself but does not add
libgcc_eh.a, where the unwinder lives, so _Unwind_* falls through to the
shared libgcc and picks up its version tags. Libraries whose C++
dependencies never throw don't reach that path -- exactly the observed
clean/dirty split.

So the archives are appended to EXTRA_LIBS, which becomes FFmpeg's
EXTRALIBS at the very end of the link line, where an archive can satisfy
references from objects scanned earlier.

Honest about verification: this is proven HARMLESS on glibc (identical
clean result with and without, since glibc's driver already pulls the
unwinder) but could NOT be proven correct locally -- there is no Alpine or
docker on this machine. CI is the test. If it fails, the staging guard
fails the build rather than shipping an artifact that cannot start, which
is the behaviour restored in the previous commit.

Context for why this is worth a round rather than accepting the dependency:
Alpine's own ffmpeg package DOES depend on so:libgcc_s.so.1, and its C++
library packages (chromaprint, libsrt, zimg) depend on libstdc++ and
libgcc_s as well -- Alpine links those deps SHARED, so the C++ runtime
lives in them rather than in libav*. We static-link everything, which is
what pulls the references inward. Matching Alpine would mean accepting
`apk add libgcc`; the requirement here is explicitly not to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… for Windows

Root cause found, and it was a scoping bug rather than anything subtle:
07_build_ffmpeg.sh already strips -lgcc_s from dependency .pc files, with a
comment naming SRT's srt.pc as the culprit that over-captures the
compiler's implicit link line into Libs.private. That strip runs for
win-* only. The musl block right below it rewrites -lstdc++ to
-l:libstdc++.a and leaves -lgcc_s alone.

An explicit -lgcc_s DEFEATS -static-libgcc: the linker resolves the
unwinder against the SHARED libgcc and stamps the references with its
version tags, and a versioned reference can never afterwards be satisfied
by an archive. Proven in an alpine:latest container, same toolchain as the
build (gcc 15.2.0, musl):

  without -lgcc_s : DT_NEEDED libgcc_s = 0, versioned undefined = 0
  with    -lgcc_s : DT_NEEDED libgcc_s = 1, versioned undefined = 3
                    _Unwind_Resume@GCC_3.0, __register_frame_info@GCC_3.0, ...

exactly the symbol pattern the musl artifacts carry. It also explains the
split that made this confusing: only libavformat (13 refs) and libavfilter
(24) were affected -- the DSOs whose C++ dependencies actually throw --
while libavutil, libavcodec and libswscale never need the unwinder and were
already clean under -static-libgcc.

Three hypotheses were eliminated by measurement before this one, all
refuted in a container rather than argued about: link order (reproduced
FFmpeg's exact ordering on glibc, clean); LDFLAGS not reaching the DSO link
(library.mak has LINK_SO_ARGS = $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS)); and an
Alpine spec that adds libgcc_s unconditionally (refuted -- on real Alpine
-static-libgcc alone gives a clean result, and in the published artifacts
libavdevice and libswresample carry no libgcc_s at all).

The speculative -l:libgcc_eh.a/-l:libgcc.a addition from the previous
commit is removed: the container test showed -static-libgcc alone is
sufficient once the explicit -lgcc_s is gone, so it was unnecessary.

This keeps the requirement that motivated it: grab a release artifact in a
Dockerfile, extract, run -- no apk add, and no dependency list that can
grow quietly, because the staging guard fails the build if any host C++
runtime dependency reappears.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…uld not fail

Two ios-sim-arm64 v3 failures in run 34977613367, both the same defect class this
branch exists to close: the test inferred a capability instead of measuring it.

f3863cd set BUILD_VULKAN=0 for ios-sim-arm64 because the lean slice has
BUILD_LIBPLACEBO=0 -> shaderc.sh returns early -> no SPIR-V compiler, so every
Vulkan filter would drop out of configure while --enable-vulkan stayed on the
command line. ios.sh still hardcoded "v3 => Vulkan" and "v3 => MoltenVK
attribution required", so it failed a correct artifact. Verified against the
built artifacts: ios-sim carries no --enable-vulkan* in its configure string and
no legal/licenses/MoltenVK; ios-arm64 carries both.

Fixing that surfaced a worse one. The check advertised as "the real gate, read
out of the BINARY" was av_vkfmt_from_pixfmt -- but libavutil/Makefile has

    OBJS-$(!CONFIG_VULKAN) += hwcontext_stub.o

and hwcontext_stub.c defines av_vkfmt_from_pixfmt and av_vk_frame_alloc as
ABI-preserving stubs. Both are therefore exported by every slice on both n8.1.2
and n9.0.1, so the assertion could not fail -- the same "row that cannot fail"
shape as the libsoxr/aresample row. Measured with llvm-nm: ios-sim (Vulkan off)
exports 2 av_vk* symbols, ios-arm64 exports 4. The two extra ones live only in
hwcontext_vulkan.c, so av_vk_get_optional_device_extensions is the discriminator
and is now what both branches assert.

- ios.sh: VULKAN_CELL computed once from version3 AND the RID; the negative
  branch now also proves the symbol is absent, so re-enabling Vulkan without the
  SPIR-V chain fails instead of passing quietly. MoltenVK attribution keyed off
  the same variable, so a cell that does not build it may not ship its licence.
- lib.sh: add check_symbol_absent. Empty nm output reports inconclusive rather
  than absent -- treating "could not determine" as "not there" is exactly how the
  patchelf change shipped a broken artifact earlier on this branch.
- gen-matrix.sh: the per-cell prose claimed "v3 adds Vulkan" globally while
  naming the lean slice only for x264/x265/kvazaar. The per-RID column was
  already correct; only the summary sentence was wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ryan-morris
ryan-morris merged commit 91c5ac4 into main Sep 16, 2026
256 checks passed
ryan-morris added a commit that referenced this pull request Sep 17, 2026
* fix(renovate): one un-writable file was silently discarding every dependency bump

Run 35140763146 reported success and opened nothing. It had found four ledger
updates (libvmaf v3.2.0->v3.2.1, whisper.cpp v1.9.3->v1.9.4, shaderc
v2026.3->v2026.4, libmp3lame) plus a renovate-version bump, pushed no branch
(`git ls-remote` shows no renovate/*), and exited 0.

Cause, from the run log:

    INFO: Workflows update rejection - aborting branch.
          (branch=renovate/ffmpeg-+-library-dependencies)

UPDATE_PR_TOKEN has no `workflow` scope, so Renovate may not write under
.github/workflows/. It does not skip that one file -- it discards the ENTIRE
branch. Because `groupName: "FFmpeg + library dependencies"` batches every
custom.regex manager into one PR, and one of those managers bumps
`renovate-version:` inside .github/workflows/renovate.yml, that single
un-writable file took the whole ledger batch down with it.

A second, independent failure in the same run:

    Request failed with status code 403 (Forbidden): PATCH .../issues/23

so the Dependency Dashboard cannot be written either. Issue #23 last changed
2026-09-14 while the run was 2026-09-16: everything it reports, including its
"Other Branches" entry pointing at a branch that no longer exists, is stale, and
its force-a-PR checkboxes do nothing.

Changes:

- renovate.json: give the renovate-version manager its own group. The blast
  radius of a missing `workflow` scope is then that one PR instead of the entire
  ledger, whatever the token can do.
- renovate.json: drop `"onboarding": false`. It is a global-only option; every
  run logged it as a Configuration Error and ignored it. This is the repo problem
  behind the dashboard's "Found renovate config warnings".
- renovate.json: set gitAuthor. Renovate warns against its default
  renovate@whitesourcesoftware.com on github.com (a Mend-owned address).
- renovate-config-test.sh: add --no-global --strict. Without them the test
  validated renovate.json AS A GLOBAL CONFIG -- it literally printed "Validating
  renovate.json as global config" -- where `onboarding` is legal, and exited 0 on
  warnings regardless. So the guard meant to catch config defects validated a
  different mode than production and passed on a file Renovate was rejecting.
  Verified both ways: reintroducing `"onboarding": false` now exits 1 with the
  same message the real run logged.
- renovate.yml: pre-flight the PAT's X-OAuth-Scopes for repo + workflow and fail
  with a specific message. Absence of the header (fine-grained PAT) means "cannot
  verify", not "no scopes", so that warns rather than fails. Not enforced under
  dryRun, which exists to preview.
- renovate.yml: after the run, read the Dependency Dashboard back and fail if it
  carries a "Repository Problems" section. Renovate publishes its own failures
  there, so this converts a silent exit 0 into a red job. Verified against the
  current dashboard body (fires) and a cleaned copy (does not).
- renovate.yml: document the required PAT scopes, and correct the `permissions:`
  block, which implied it governed Renovate. It does not -- Renovate uses the PAT,
  so nothing in that block can fix a Renovate 403.

Note the shape: a green check over discarded work, and a test that verified the
wrong thing. Same defect class as #25, in the dependency pipeline.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(renovate): keep onboarding:false, in the config layer that accepts it

renovate.json is passed as configurationFile (Renovate's GLOBAL config) and is
also read from the default branch as the REPO config. So `"onboarding": false`
did take effect globally, while the repo-config pass rejected it on every run --
which is why simply deleting it would have changed behaviour rather than just
silencing a warning. Set it as RENOVATE_ONBOARDING instead: same effect, no
Configuration Error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: don't rebuild every artifact because a CI validator changed

release.yml triggers on scripts/**, excluding scripts/test/**. scripts/ci/ is
not excluded, so this PR's one-line change to renovate-config-test.sh -- a
validator that reads a config file and can affect no artifact -- would rebuild
15 RIDs x 4 cells x 2 FFmpeg lines and cut a release.

Same reasoning as the existing scripts/test/** exclusion, scoped narrowly:
scripts/ci also holds gen-manifest.sh, select-versions.sh and
impacted-versions.sh, which release.yml genuinely runs, so only the *-test.sh
validators are excluded. Verified against the directory: the three release-used
scripts still trigger, the five validators no longer do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant