Add Deb822 sources support (issue #149) - #150
Open
aybanda wants to merge 43 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Any reason why were the rgrepositorywin.* files renamed to rgrepositorywindow?
This probably won't work properly with the rest of the build process and also the translatable strings present in these files won't get translated.
Author
There was a problem hiding this comment.
I'll revert this, renaming was unintentional
mvo5
commented
May 5, 2025
Owner
I tried this real quick (did not really look into the code) with my "apt modernize-source" updates sources but I only get an empty window when I open the repository dialog. My debian sources: $ cat /etc/apt/sources.list.d/debian.sources
# Modernized from /etc/apt/sources.listTypes: deb deb-srcURIs: http://ftp.de.debian.org/debian/Suites: trixieComponents: main non-free-firmware Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
# Modernized from /etc/apt/sources.listTypes: deb deb-srcURIs: http://security.debian.org/debian-security/Suites: trixie-securityComponents: main non-free-firmware Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
# Modernized from /etc/apt/sources.listTypes: deb deb-srcURIs: http://ftp.de.debian.org/debian/Suites: trixie-updatesComponents: main non-free-firmware Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg |
aybanda
commented
May 5, 2025
Author
I've made several improvements, Could you please test again with your file? You should see all three sources listed in the window. |
mvo5
reviewed
May 5, 2025
| #endif | ||
| } | ||
| bool RPackageLister::handleFailedInstallation(const string &pkgName) |
Owner
There was a problem hiding this comment.
This change looks unrelated to deb822? It looks interesting and maybe worth its own PR but I would prefer if we could keep this PR strictly focused on deb822 support :)
Owner
There was a problem hiding this comment.
I still get compiler errors for this function and when I remove it the build fails now with:
$ make.../usr/bin/ld: ../common/libsynaptic.a(rsources.o): in function `SourcesList::ReadDeb822SourcePart(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)':/home/mvogt/devel/synaptic/trunk.git/common/rsources.cc:592:(.text+0x386e): undefined reference to `RDeb822Source::ParseDeb822File(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<RDeb822Source::Deb822Entry, std::allocator<RDeb822Source::Deb822Entry> >&)'/usr/bin/ld: /home/mvogt/devel/synaptic/trunk.git/common/rsources.cc:600:(.text+0x39c3): undefined reference to `RDeb822Source::ConvertToSourceRecord(RDeb822Source::Deb822Entry const&, SourcesList::SourceRecord&)'/usr/bin/ld: ../common/libsynaptic.a(rsources.o): in function `SourcesList::WriteDeb822Source(SourcesList::SourceRecord*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)':/home/mvogt/devel/synaptic/trunk.git/common/rsources.cc:651:(.text+0x5c24): undefined reference to `RDeb822Source::ConvertFromSourceRecord(SourcesList::SourceRecord const&, RDeb822Source::Deb822Entry&)'/usr/bin/ld: /home/mvogt/devel/synaptic/trunk.git/common/rsources.cc:656:(.text+0x5d5e): undefined reference to `RDeb822Source::WriteDeb822File(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<RDeb822Source::Deb822Entry, std::allocator<RDeb822Source::Deb822Entry> > const&)'/usr/bin/ld: ../common/libsynaptic.a(rsources.o): in function `SourcesList::UpdateSources()':/home/mvogt/devel/synaptic/trunk.git/common/rsources.cc:340:(.text+0x6a80): undefined reference to `RDeb822Source::ConvertFromSourceRecord(SourcesList::SourceRecord const&, RDeb822Source::Deb822Entry&)'/usr/bin/ld: /home/mvogt/devel/synaptic/trunk.git/common/rsources.cc:345:(.text+0x6ad5): undefined reference to `RDeb822Source::WriteDeb822File(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<RDeb822Source::Deb822Entry, std::allocator<RDeb822Source::Deb822Entry> > const&)'this is a standard debian/unstable system I'm testing with. Any hints how to resolve this?
Author
There was a problem hiding this comment.
The linker errors persist because we need to regenerate the build system after adding rsource_deb822.cc and rsource_deb822.h to Makefile.am. try running:
./autogen.sh
./configure
make clean
make
This will ensure the new files are properly integrated into the build system.
Owner
There was a problem hiding this comment.
Thanks, I tried this but no luck - when I pulled now I get a different error and it looks like common/rsource_deb822.cc is now urf-16 instead of the expected ascii or utf-8. this makes g++ unhappy:
$ file common/rsource_deb822.cccommon/rsource_deb822.cc: C source, Unicode text, UTF-16, little-endian text, with CRLF line terminators...
$ file common/rswig.h common/rswig.h: C++ source, ASCII textcan you please update again? (in GH the file is also shown as binary data now).
mvo5
commented
May 5, 2025
Owner
It currently fails to build in the new "handleFailedInstallation" - if I remove this new function it builds and runs but still no luck, still an empty repositories window for me. |
mvo5
reviewed
May 5, 2025
mvo5
left a comment
Owner
There was a problem hiding this comment.
Thanks a lot for working on this! Some quick feedback inline, still have not read the code line-by-line but went real quick over it)
| }; | ||
| /** |
Owner
There was a problem hiding this comment.
Lets put this into either "rsources.{cc,h}" or a new "rsource_deb822.{cc,h}" - the rpackagemanager file is not quite the right place for this (and/or rsources_deb822 and rsources_manager).
| @@ -0,0 +1,252 @@ | |||
| /* rsource_deb822.cc - Deb822 format sources support | |||
Contributor
There was a problem hiding this comment.
I may be wrong, but shouldn't this new file be also added to common/Makefile.am?
Contributor
There was a problem hiding this comment.
Could you please add it? Thanks! :-)
mvo5
commented
May 13, 2025
Owner
Thank you, I still get compiler errors when I run this locally, if you could rebase the branch to current master that would be great, this way we will get working CI again (the github action was on ubuntu-20.04 and 15b6a17 which is EOL now moved it to ubuntu-latest) |
aybandaforce-pushed
the
feature/deb822-sources
branch
from
May 15, 2025 15:49
0e07c9b to
15b6a17CompareContributor
Why was this closed and all commits removed? |
…. Fix UTF-8 encoding issues.
… UTF-8 support using wide strings and codecvt_utf8 for file I/O operations.
aybanda
commented
May 15, 2025
Author
While rebasing, The PR closed due to a force-push that temporarily removed my changes. |
aybanda
commented
May 15, 2025
Author
I've rebased the branch to the current master, which includes the CI workflow update from ubuntu-20.04 to ubuntu-latest. This should resolve the CI issues. Regarding the compiler errors you're seeing locally, could you please share the specific error messages? |
AsciiWolf
commented
May 18, 2025
Contributor
@aybanda Not sure about Michael's errors, but this is how it looks like on my (Debian Testing) system: |
aybanda
commented
May 19, 2025
Author
hey @mvo5 I tried few things, fixed few issues |
AsciiWolf
commented
May 19, 2025
Contributor
Sadly even more errors now on my system. |
AsciiWolf
commented
Jul 8, 2025
Contributor
I have also tested the modernized (deb822) source file and it sadly got broken. Synaptic returned a file format error after attempting to disable one of the entries. Here is a diff of the changes Synaptic has made: As you can see, the (By the way, also notice the empty line added at the end of the file - probably the same problem as with classic sources.list.) |
aybanda
commented
Jul 8, 2025
Author
Thanks @AsciiWolf for the quick and detailed review! I’ll go through the points and fix them once I get a bit of time. Appreciate the thorough testing! |
AsciiWolf
commented
Jul 8, 2025
Contributor
Thanks again Ajay for working on this! I have sent you another 40 USD from the bounty via PayPal. :-) |
…ng, and Signed-By preservation
AsciiWolf
commented
Jul 11, 2025
Contributor
Thanks for the update! Quick test: Legacy
Deb822
|
Contributor
One more problem with legacy sources: The disabled (cdrom) source seems to have funny colors that change when Synaptic is restarted:   |
…e, preserve comments and fields, and show deb-src entries
aybanda
commented
Jul 13, 2025
Author
Hi @AsciiWolf |
Contributor
The legacy format is sadly still not correct with latest code from your branch. When I opened the Sources window in Synaptic, I got an "Ignoring invalid record(s) in sources.list file!" message (even though there were no invalid records). When closing the window, the sources list got changed: Please note the hash characters incorrectly added. The Deb822 formatted sources are now not working at all. Only an empty Sources window appears:  |
AsciiWolf
commented
Jul 14, 2025
Contributor
Here are the source files I used for testing.
|
…o prevent corruption of .sources files
… (portable, non-GTK)
…rces, add debug output for config path
…c to trace stanza and field parsing
…ser and source management
…n; preserve Deb822 flag on edit
Contributor
I have just repeated the test with latest code: Legacy: Deb822: Not sure why this happens, but it certainly does not look right and indeed apt does not like it: |
AsciiWolf
commented
Sep 24, 2025
Contributor
@aybanda Any update? :) |
Contributor
I have found out that some of the small formatting issues that I pointed out before, for example the trailing space added at the end of the line, also happen in older Synaptic builds with legacy sources.list. We can probably ignore these small issues if they cause no real problems. But we must make sure that there is no invalid deb822 or legacy output generated by Synaptic. Adding, modifying, removing, enabling/disabling etc. the entries must work flawlessly. edit:This issue also happens with older Synaptic and looks harmless. |
SanchezTheCoder
commented
May 20, 2026
I opened a small follow-up against this branch: aybanda#1 It wires the existing Deb822 integration test into Verified in a Debian testing container with:
|
AtesComp
commented
Jul 16, 2026
Just adding this as I haven't see a reference to using the "X-Repolib-Name" field yet as opposed to using the file's comment to display the repo name. Many apps are automating the repo additions and update the repo file with its own comment overwriting any previous comment (i.e., Google Chrome, MS VSCode, etc.) Example: See https://repolib.readthedocs.io/en/latest/deb822-format.html |
tzh476
commented
Aug 31, 2026
@AsciiWolf@aybanda — offering to help finish the rebase here, if help is wanted. I have no claim on this and I am not trying to jump anyone's work. @aybanda did the implementation and was paid for it; @AsciiWolf's split offer above ("both parties will receive 60 USD") is his to make, not mine to assume. So this is a question, not a PR. What I checked before asking. The branch no longer merges cleanly onto That is a contained rebase — the conflicts are in the files this PR was always touching, not a broad divergence. What I would do, if you want it:
Where I would put it. Against @aybanda's fork, the way @SanchezTheCoder did, so the commits land on his branch and this PR updates in place. His authorship and history stay intact; nothing needs to be re-opened elsewhere. On the bounty: genuinely not why I am asking first — I would rather the rebase happen than argue about the split. Whatever @AsciiWolf and @aybanda decide, including "no thanks, Ajay is picking it back up," is fine and I will drop it. @aybanda — if you are still on this and just short of time, say so and I will stay out of the way, or do a narrow piece you name. Disclosure: I work with LLM assistance and take responsibility for what I submit; I can explain any part of a diff I open. |
AsciiWolf
commented
Aug 31, 2026
Contributor
Thanks for offering to finish the Deb822 support! It would be great, but I am afraid I won't be able to provide much help/testing in the next month or two due to my current time schedule. |
mvo5 added a commit
that referenced
this pull request
Sep 5, 2026
To read apt's deb822 source records we now use libapt's pkgTagFile and parse into our own SourceRecords. However we start with read only support for now to keep the diff smaller. Thanks to tzh476 and aybanda for the work on this in PR #274 and PR #150. The test fixtures are taken from tzh476's tests in PR #274. Co-authored-by: Stephen Smith <tzh476@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements support for the Deb822 source format in Synaptic, as requested in issue #149.
What’s new
RDeb822SourceandRSourceManagerclasses for parsing, validating, and serializing Deb822 sources.Testing
tests/test_deb822_integration.cc) to verify parsing, file I/O, and validation.Sample Python test results
Created test directory: /tmp/tmpabcd1234
Initialized source manager
Source validation: PASS
Source string representation:
deb http://example.com stable main
Successfully wrote source to file: /tmp/tmpabcd1234/test.sources
Read source from file:
Type: deb
URI: http://example.com
Suite: stable
Components: main
Disabled source test: PASS
Multiple URIs test: PASS
Multiple suites test: PASS