Skip to content

deps: lock the runtime dependencies with hashes - #49

Merged
z33b0t merged 1 commit into
mainfrom
deps/lock-runtime-requirements
Jul 29, 2026
Merged

deps: lock the runtime dependencies with hashes#49
z33b0t merged 1 commit into
mainfrom
deps/lock-runtime-requirements

Conversation

@z33b0t

Copy link
Copy Markdown
Contributor

The second half of #10, taking your call on the fork:

yes, exclude torch (and its CUDA deps) from the compile

187 packages pinned exactly with hashes; Dockerfile installs with --require-hashes.

What's excluded, and what deliberately isn't

Excluded (all installed by the cu130 step that runs first): torch, triton, and the sixteen nvidia-* runtime packages.

Not excluded: torchaudio and torchmetrics. Worth flagging since they look like they belong with torch — they don't. I checked the running container: the cu130 install doesn't provide them, they come from PyPI as ordinary NeMo dependencies, so leaving them out would mean an unpinned hole in the middle of the tree.

The torch pin earns its keep

requirements.in pins torch==2.12.1 even though torch never reaches the output. That isn't decoration — without it the resolver drifts to PyPI's newest torch (2.13.0) and pins everything else against a torch the image never installs.

With the pin, the resolution reproduces the running container exactly:

locklive container
torchaudio2.11.02.11.0
torchmetrics1.9.01.9.0
numpy2.4.62.4.6
nemo-toolkit2.7.32.7.3

The Dockerfile's torch==2.12.1 and the requirements.in pin must stay in lockstep; both carry a comment saying so, and Dependabot is configured to ignore torch so it can't move one without the other.

Verified by building and running it, not by reading it

On the GPU box, separate tag/port/volume so the live daemon was untouched:

  • Build succeeds under --require-hashes
  • torch survives as 2.12.1+cu130, cuda available: True, bf16 supported: True — this was the real risk, since the install upgrades cuda-bindings/cuda-pathfinder that the cu130 step had brought in
  • Transcription works: both audio/clean.wav and audio/noisy.wav come back correct
  • The hash check actually bites — forged hashes are refused (Expected sha256 1111… Got 0f17b89f…), genuine ones install

Also here

  • scripts/lock-requirements.sh holds the exclusion list, so it isn't a comment someone has to retype. It fails loudly if a cu130-provided package ever leaks into the lock.
  • Dependabot gains the pip ecosystem. Read its first PR rather than merging on green: both lockfiles are uv-generated, and Dependabot's pip-compile support keys off the pip-tools header, so it may edit requirements.txt directly instead of regenerating it. If it does, close the PR and re-run the script.

Not included

requirements.in isn't added to server.yml's path filters — that file is under .github/workflows/, which I can't merge without the workflow scope. It belongs in #40, which already edits those filters.

Closes#10

requirements.txt carried only floors, so `docker build` today and next month
resolved different transitive trees — a bug reported against an image tag
couldn't be reliably reproduced, because the tag didn't identify what was in it.
A compromised upstream release also landed on the next rebuild with no PR, no
diff and no alert. 187 packages are now pinned exactly with hashes, and the
Dockerfile installs with --require-hashes.
Per the decision on #10, torch is excluded from the lock along with triton and
the nvidia-* runtime packages: the Dockerfile installs torch from the cu130
index first, because those wheels bundle the CUDA + cuDNN runtime, and letting
PyPI install it again would clobber that with a generic build. torchaudio and
torchmetrics are deliberately *not* excluded — they come from PyPI, not from
the cu130 step, so they belong in the lock.
torch is still pinned in requirements.in even though it never reaches the
output, and that turned out to matter: without it the resolver drifts to PyPI's
newest torch (2.13.0) and pins the whole tree against a torch the image never
installs. With the pin, the resolution reproduces the running container exactly
— torchaudio 2.11.0, torchmetrics 1.9.0, numpy 2.4.6, nemo-toolkit 2.7.3.
The exclusion list lives in scripts/lock-requirements.sh rather than a comment
someone has to retype, and the script fails if a cu130-provided package ever
leaks into the output.
Dependabot gains the pip ecosystem now that there is something to bump, with
torch ignored so it can't move the pin out of lockstep with the Dockerfile.
Verified by building and running the image on the GPU box: build succeeds under
--require-hashes, torch survives as 2.12.1+cu130 with CUDA available and bf16
supported, and both sample clips transcribe correctly. Also confirmed the hash
check actually bites — a forged hash is rejected, the genuine one installs.
@z33b0t
z33b0t merged commit ab9db8b into mainJul 29, 2026
9 checks passed
@z33b0t
z33b0t deleted the deps/lock-runtime-requirements branch July 29, 2026 18:33
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pin/lock Python dependencies — requirements.txt uses >= floors

1 participant

@z33b0t