Uh oh!
There was an error while loading. Please reload this page.
feat(ENG-56): derive Debian source package members from the .dsc on push deb - #351
Open
BartoszBlizniak wants to merge 4 commits into
Open
feat(ENG-56): derive Debian source package members from the .dsc on push deb#351BartoszBlizniak wants to merge 4 commits into
push deb#351BartoszBlizniak wants to merge 4 commits into
Conversation
…nges files Adds a deb-only --dsc-file option that parses a Debian .dsc control file's Files:/Checksums-Sha256: stanza and auto-fills --sources-file/--changes-file, resolving referenced filenames relative to the .dsc's directory (closes GitHub issue cloudsmith-io#56). This is layered on top of create_push_handlers()'s generic, API-model-driven option loop rather than adding per-format branching to it: the deb subcommand alone gets the extra option, and a small resolver module (cloudsmith_cli/cli/dsc_parser.py) does the parsing with the stdlib email parser (no new dependency). Precedence: explicit --sources-file/--changes-file always win over values derived from --dsc-file, so passing both is a deliberate override rather than an error. A .dsc referencing a multi-component source package (*.orig-*.tar.*) or a detached signature (*.asc) raises a clear click.UsageError before any network call, since Cloudsmith's deb upload format has no field for either. Tests cover the parser directly (happy path with/without a .changes file, the Checksums-Sha256 fallback, missing referenced files, multi-component/detached-signature rejection, ambiguous multi-tarball .dsc files) and the CLI integration (flag precedence for both --sources-file and --changes-file, --help wiring restricted to `deb`, and that rejected .dsc files abort before validate_create_package is ever called). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verified against the live API, a Debian source push must name the `.dsc` as PACKAGE_FILE: a binary `.deb` carrying source archives is rejected ("You can only upload a sources and changes files with a source package") and a `.dsc` on its own is rejected too ("A sources archive is required when uploading a source package"). Requiring `--dsc-file` as well therefore meant writing the same path twice, where issue cloudsmith-io#56 asks for the members to be derived "when passed only a .dsc file". The `.dsc` is now parsed whenever PACKAGE_FILE ends in `.dsc` and no `--sources-file` was given; `--dsc-file` remains for naming a different `.dsc`. Since a `.dsc`-only push already failed server-side, this cannot regress a working invocation.
Fixes found while testing against real packages:
- Detached upstream signatures hard-failed, which made the feature unusable for Debian's own `hello` source package. The issue notes signatures can be ignored, so they are now skipped and reported via `utils.should_use_stderr(opts)`; `resolve_dsc_files()` returns a frozen `ResolvedDscFiles` carrying `ignored_files` so the parser reports nothing itself, mirroring `ResolvedMetadata`.
- A signature also matched as a second source archive, since `pkg.orig.tar.gz.asc` satisfies the `pkg.orig.tar.` prefix test. Masked by the hard failure above; classification now runs over the non-signature members only.
- `3.0 (git)` and `3.0 (bzr)` were declared supported but could never resolve: per dpkg-source(1) and Dpkg::Source::Package::V3::Bzr they produce a git bundle and a `.bzr.tar.*`, neither matching `<source>_<version>.tar.*`, and neither being a source archive the deb upload model can index. Both now fall through to the unsupported-format error alongside `3.0 (custom)`.
- Symlinked members were rejected because the resolved target had to sit in the `.dsc` directory. `mk-origtargz --symlink` is the uscan default and links the `.orig` tarball in from a download cache. Traversal is already blocked at the name level, so the check is now "regular file next to the .dsc", still returning the canonical path so a symlink cannot be swapped between check and read. A member symlinked to a directory is still rejected.
- Errors named `--dsc-file` even when the `.dsc` came from PACKAGE_FILE, and now lead with the file.
- `--dsc-file` used `click.Path` where every other file option in `push.py` uses `ExpandPath`.
Verified end-to-end against a live repository with `hello_2.10-3` (`3.0 (quilt)`, clearsigned, ships a `.asc`), a `3.0 (native)` package and a `1.0` non-native package with a `.diff.gz`; all three synchronised with the expected members stored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>Contributor
There was a problem hiding this comment.
Pull request overview
Adds Debian source-package ergonomics to cloudsmith push deb by parsing .dsc control files to automatically derive and upload the associated upstream and Debian packaging archives, keeping the existing API-model-driven push machinery intact.
Changes:
- Implement
.dscparsing (plain or OpenPGP-clearsigned) to derivesources_file/changes_file, with validation and safe member resolution. - Extend
push debto auto-parse whenPACKAGE_FILEis a.dsc, plus add a deb-only--dsc-fileoverride option and warning behavior for ignored detached signatures. - Add CLI-level and parser-level regression tests, and document the new behavior in the README and changelog.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents pushing a Debian source package by passing a .dsc. |
| cloudsmith_cli/cli/tests/test_push_dsc_cli.py | End-to-end-ish CLI command tests for push deb.dsc behavior and option precedence. |
| cloudsmith_cli/cli/tests/test_dsc_parser.py | Unit tests for .dsc parsing, validation, and member resolution edge cases. |
| cloudsmith_cli/cli/dsc_parser.py | New parser/resolver for Debian .dsc member classification and canonical path resolution. |
| cloudsmith_cli/cli/commands/push.py | Integrates .dsc derivation into the deb push handler and registers --dsc-file for deb only. |
| CHANGELOG.md | Notes the new push deb.dsc-driven source upload behavior under Unreleased. |
| AGENTS.md | Documents the intended pattern for format-specific behavior layered atop the model-driven push handler generation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
BartoszBlizniak
marked this pull request as ready for review
August 14, 2026 12:09
cloudsmith-iduffy
approved these changes
Aug 14, 2026
cloudsmith-iduffy
left a comment
Contributor
There was a problem hiding this comment.
If we know of someone internally that knows the debian package format well it would be good to get them to give this a glance. Otherwise looks fine to me.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes#56.
cloudsmith push debnow works out a Debian source package's members from its.dsc, so this is enough:where callers previously had to determine
--sources-fileand--changes-filethemselves, across suffixes that vary per package (.orig.tar.gz,.orig.tar.bz2,.debian.tar.xz,.diff.gz, ...).The
.dscis parsed plain or OpenPGP-clearsigned, from itsChecksums-Sha256:field in preference toFiles:. The upstream/native source archive becomessources_fileand the Debian packaging archive becomeschanges_file, for the1.0,2.0,3.0 (native)and3.0 (quilt)source formats. Because uploading a Debian source package already means passing the.dscasPACKAGE_FILE— the API rejects a binary.debcarrying source archives, and rejects a.dscwithout them — the.dscis parsed by default and no extra flag is needed.--dsc-fileremains for naming a.dscother thanPACKAGE_FILE, and an explicit--sources-fileor--changes-filestill wins for its own field. Passing--sources-fileskips derivation entirely, which keeps an escape hatch for anything the parser will not accept.Members must be bare filenames resolving to regular files next to the
.dsc; absolute paths and directory traversal in the.dscare rejected, and each member is uploaded by its canonical path so a symlink cannot be swapped between the check and the read. Symlinked members are allowed, sincemk-origtargz --symlink(theuscandefault) routinely links the.origtarball in from a download cache.Following the discussion on the issue, a detached upstream signature (
*.orig.tar.*.asc) is skipped with a warning — the deb package format has no field to carry it, and nothing in the uploaded source is lost — while a multi-component source package (*.orig-<component>.tar.*) is rejected, because dropping a component would upload incomplete source.3.0 (git),3.0 (bzr)and3.0 (custom)are reported as unsupported source formats rather than being half-handled.The implementation stays a small
deb-specific layer over the API-model-driven push command generation. Other formats and the generic upload flow are unchanged.Type of Change
Additional Notes
Verified end-to-end against a live repository with three real source packages: Debian's own
hello_2.10-3(3.0 (quilt), including the.orig.tar.gz.ascit ships), a3.0 (native)package, and a1.0non-native package with a.diff.gz. All three uploaded and synchronised, with the.dscand its archives present on the resulting package.Regression coverage invokes the registered
push debcommand tree and asserts both the local paths seen at validation and the uploaded identifiers seen at package creation, for native, quilt and legacy non-native layouts; plus.dsc-as-PACKAGE_FILEderivation, a binaryPACKAGE_FILEnever being parsed, explicit-option precedence in both directions, OpenPGP-clearsigned control data, absolute paths, traversal, symlinked members, multi-component rejection, skipped detached signatures, unsupported source formats, names and versions containing.orig-, malformed file-list rows, conflicting checksum lists, anddeb-only option registration.