feat(desktop): dedicated attachment index for the Files tab with bounded, recency-aware retention (T3b) by wiggdevin · Pull Request #25 · ZeroSum-Solutions/buzz

wiggdevin · 2026-09-05T23:10:02Z

feat(desktop): dedicated attachment index for the Files tab with bounded, recency-aware retention (T3b)

Summary

T3b builds a dedicated attachment index over the landed T3 Files tab. Today's
useChannelFiles only reads the loaded message window
(useChannelMessagesQuery), which is top-level only and 200 messages per page.
This ticket adds channelFilesIndex.ts: open the channel's live subscription
first, then paginate its full history including reply-kind messages, keep only
events carrying imeta, overlay edits and deletions on top of the retained
set, and expose deterministic pagination over the result. Bulk drag-drop moves
behind a new setting that defaults off, with a batch cap of 20.

This session's own change, on top of that base: the index used to refuse
every new attachment, edit, or deletion once its retention cap (5000 entries
each) was full, so a channel past the cap silently dropped later live arrivals
while the banner claimed "Showing the most recent attachments only." The caps
now evict instead of refuse — recency- and relevance-ranked, not plain
oldest-first, so evicting to make room never un-hides an attachment whose
deletion was the thing evicted, and never permanently loses a genuine edit to
a flood of edits for messages the index never retained.

Fork deviations

From build.md (fix round) and the repair round, plus pr-inputs.md:

Truncation banner copy changed and is no longer verbatim from the ported
PR feat(desktop): add Files tab with folders, bulk select, and drag-and-drop block/buzz#4316 text, and no longer opens with "Showing the most recent
attachments only" because that is false when only the edit or deletion cap
fired. channel-files-tab.spec.ts does not assert the string.
FilesIndex.deletions changed type from ReadonlySet<string> to
ReadonlyMap<string, number> (deletion event timestamp) so the cap can
order by recency. Exported type change; no consumer outside
channelFilesIndex.ts reads the field, .has/.size callers stayed
source-compatible, and desktop-typecheck is clean.
Added boundEventTimestamp (an unusable relay timestamp normalises to 0)
so a NaN created_at cannot jam a cap by comparing false against
everything.
admitNewest renamed admitRetained with an IsRelevant parameter;
oldestRetained became lowestRanked; the source cap passes
ALWAYS_RELEVANT.
A pair-eviction branch (evict the oldest deletion and its target
source when only real pairs remain) is implemented but currently
unreachable while MAX_INDEXED_DELETIONS >= MAX_INDEXED_ATTACHMENT_EVENTS;
the blind critic forced it with a reduced cap and confirmed it is correct.
It ships without a dedicated test — see follow-up 2.

Defect checklist

(Verbatim from build.md's last fix round.)

DTO caps — unchanged and still enforced (boundIndexSource,
boundIndexEdit cap pubkey, content and imeta parts). This round adds
boundEventTimestamp for the one new relay-sourced field the retention
order reads; test 6 binds it.
Write order — eviction and insert happen inside one mutable draft that
is published only at the end of ingestIndexEvents, and an evicted source
takes its edit and deletion with it in the same draft, so no published
snapshot carries an overlay for a row that is gone.
Child-process environment — not applicable; this module spawns nothing.
Gate bounds what costs — the caps still bound entries retained
(sources, deletion targets, edits), which is the memory the index costs;
eviction changes which entries are kept, not how many.
Falsifiable guards — mutation table in build.md; six mutations in the
fix round, each caught, plus four more in the repair round, each caught.
No swallowed failure — nothing is caught here. A drop is reported to
the user through truncated -> the banner, which now names what was
dropped; ChannelFilesTab.test.mjs asserts that wording.

Gates

Tested base OID: d478fcf4a3c81dfeb7b19276e68333124e610aa1 (origin/zs/main
before this branch's merge). This PR's tip merges that base in at
3d91302ad65fe948fd77ff2f121a857de60d9431 (clean, no conflicts).

On cbfea5a9b (pre-merge tip), all exit 0:

just fmt-check clippy desktop-check desktop-tauri-fmt-check desktop-tauri-clippy file-size-check
just desktop-typecheck
channel-files node suite (132 tests)
just desktop-test (6521 tests) under the gate lock
channel-files-tab.spec.ts and channel-files-index.spec.ts (3 passed) under the gate lock
desktop-e2e-smoke on 481ed97f0: 1360 passed, four unrelated failures shown
flaky or pre-existing on the clean base
(workflow-local-controls.spec.ts:148 reproduces with the diff stashed).

After merging origin/zs/main (93 files changed via the merge, so the fast
gates were rerun per the merge policy), on merge commit 3d91302ad6, all exit 0:

just fmt-check clippy desktop-check desktop-tauri-fmt-check desktop-tauri-clippy file-size-check
just desktop-typecheck
channel-files node suite (132 tests, 0 fail)

Pre-push hook (full gate suite, this session, first attempt): push-head-scope,
branch-skew, file-size-check, desktop-check, desktop-typecheck,
desktop-test, desktop-tauri-checks, rust-tests — all green, including all
12 Rust test suites (buzz-core, buzz-auth, buzz-voice, buzz-cli,
buzz-db, buzz-conformance, buzz-push-gateway, buzz-backend-kubernetes,
buzz-agent, buzz-acp, buzz-secret-store, buzz-mcp-launch).

Gemini 3.8 Flash tester

Ran in an earlier session against this branch; not rerun this session (report
was session-scoped, see the earlier workflow journal, wf_d128b3c8-44d).

Blind critic

Fresh-context Claude blind critic, run twice against this branch:

Round 1 (head 481ed97f0): FAIL. One BLOCK-class defect: an evicted
deletion entry could un-hide an attachment that was still retained
(secret.png reappeared after one newer, unrelated deletion evicted its
deletion marker), plus follow-ups on the edits cap ranking the wrong
population, an edit-keyed deletion surviving source eviction, and banner
copy false for two of the three caps. Also confirmed: caps still bound
retained entries under a 7000×3 stress ingest; the retained set is exactly
the newest N under any arrival order (shuffled 5500-event probe); pagination
stays deterministic after eviction; a bad relay timestamp cannot jam a cap.
Recheck (head cbfea5a9b, after the repair round): PASS. Notes: both
reproductions from round 1 are dead (secret.png stays hidden; orig.png
renames correctly); under sustained pressure 500/500 deleted files stay
hidden and 2000/2000 new deletions land; the claimed-unreachable
pair-eviction branch verified genuinely unreachable at current cap values;
both new relevance guards are falsifiable. Full report: critic.md in this
session's scratch directory (not committed to the repo).

GPT-5.6 Sol audit

Three rounds in the earlier session, ending block-open: a full cap
refused every new attachment/edit/deletion arrival while the banner claimed
recency, which this session's fix round (481ed97f0) closed for the case Sol
named. This session ran two further rounds (the fix round and the repair
round after the blind critic's round-1 FAIL) plus the blind critic's own
recheck, which is what closes the loop here — no fourth Sol run (the
three-run cap was already reached in the earlier session).

Follow-ups

channelFilesIndex.ts:431-447 (new, from the repair round): an edit
arriving before its message is "irrelevant" at that instant, so at an
edits cap full of relevant edits it is refused and lost; the row later
shows the un-edited version. Needs 5000 edited attachments in one channel;
costs an overlay, exposes nothing.
channelFilesIndex.ts:400-407: the pair-eviction safety net is unreachable
only because MAX_INDEXED_DELETIONS >= MAX_INDEXED_ATTACHMENT_EVENTS and
nothing binds that coupling; add a static assertion or a reduced-cap test.
channelFilesIndex.ts:270-300: lowestRanked is an O(size) scan per
arrival once a cap is full (0.53 ms per live deletion at 5000; a
30000-event backfill costs 0.8-0.9 s); bounded by MAX_BACKFILL_PAGES ×
page size; an ordered index would make it O(log n).
ChannelFilesTab/e2e: no e2e spec asserts the banner copy.
Carried from the earlier session's Sol audit (per build.md): the
O(size)-per-arrival scan was originally described as free ("the backfill
fills once and hits the floor check without a scan"); that claim was wrong
(a refusal scans too) and has been corrected in build.md — see follow-up
3 above for the accepted, measured cost.

Test plan

cd desktop && node --import ./test-loader.mjs --experimental-strip-types --test src/features/channel-files/*.test.mjs — 132 tests, 0 fail.
just desktop-typecheck — clean (the FilesIndex.deletions type change).
just fmt-check clippy desktop-check desktop-tauri-fmt-check desktop-tauri-clippy file-size-check — all exit 0.
just desktop-test under the gate lock — 6521 tests, 0 fail.
channel-files-tab.spec.ts and channel-files-index.spec.ts under the gate lock — 3 passed (seeds 250 attachments incl. a thread reply, asserts sub-500ms over three runs per the plan's bar).
just desktop-e2e-smoke — 1360 passed; four failures are flaky/pre-existing on the clean base, not a regression from this diff.
Pre-push full gate suite (this session, on push) — green, including all 12 Rust test crates.

🤖 Generated with Claude Code

https://claude.ai/code/session_012Z6iidtozXxgx58BUZUKnu

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

The Files tab projected the loaded message window, which is top-level only and 200 events per page, so a file posted in a thread or older than the window was simply absent (issue block#4428). Add the tab's own model instead. `createChannelFilesIndexController` opens the live subscription first and only then walks the channel's history in keyset pages over the reconnect-repair command — replies, deletions and edits included, no `top_level` filter — so backfill and live overlap instead of leaving a gap (AGENTS.md review-proven rule 2). Every page is ingested before its cursor advances, so an interrupted walk resumes on a consistent prefix, and everything is keyed by event id, so an event that arrives through both paths is indexed once. Relay input is bounded where it enters the index, not where it renders: imeta tags per event, parts per tag, part and content length, pubkey length, indexed events, deletions and edits, pages per run, and the relay detail carried into an error message. An edit may only rewrite the attachments of a message signed by the same key, and a forged edit is dropped outright once its target is known. `useChannelFiles` keeps its pure projection; its hook is superseded by `useChannelFilesIndex` and goes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

Dragging a selection into a folder is a batch of relay writes started by a gesture that is easy to make by accident. It is now off by default, turned on from the tab's own "Drag selection" toggle while selecting, and capped at 20 files: a larger selection is refused whole and says so, rather than silently moving the first twenty. The tab also grows the two affordances the index needs. A partial failure — a history page that failed, or live updates that stopped — banners over the rows it did load instead of blanking the list, and a history walk that stopped at its page bound offers "Load older files" so the user is never left with no way to continue. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

ChannelScreen feeds the tab from `useChannelFilesIndex`, so the list is the channel's attachments rather than the ones inside the loaded message window, and passes through the partial-failure message and the load-older control. `channel-files-index.spec.ts` (registered in the smoke allow-list) seeds 250 attachments — the last of them a thread reply, which the window projection could never see — and asserts the tab lists all 250 and reaches that state in under 500 ms on each of three runs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

…file backfill `fetchPage` returns whatever `get_channel_reconnect_repair` sent. A page of the wrong shape, or an event whose fields cannot be read, threw out of the backfill loop and became an unhandled rejection under the hook's `void controller.start()`: the history walk stopped, the cursor kept its old value, and the Files tab showed a partial list with no error and no way back. Reading a page is now allowed to fail. The failure is surfaced through the same `historyError` the fetch path uses, and it returns before the cursor advances, so a retry refetches the failed page instead of skipping it. Both halves get a test that fails when the guard is removed: a page that is not a list is refused by name, and a page whose events cannot be read leaves the cursor where it was and re-requests it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

…guards A refused live subscription was caught, banner-ed and then never retried. It was opened only from `start()`, which latched after one call, and the banner's Retry ran `loadMore()`, which returns as soon as the history walk is complete — so the Files tab stopped following the channel for the life of the tab and its only recovery affordance did nothing (AGENTS.md review-proven rules 1 and 6). Live state is now tracked apart from history state. `retryLive()` re-opens the subscription and is idempotent on every state that makes a second relay call wrong; `retry()` — what the banner's Retry runs — drives the re-subscription and the history walk, because either failure alone leaves the other standing. The banner is cleared only by a subscription that actually opened, and the attempt budget is the terminal state rule 4 asks for (every attempt after the first is a user gesture that schedules nothing, so there is no self-amplifying loop to delay). The activation gate is keyed by channel id instead of a latched boolean. The index effect commits before the tab-reset effect, so on a channel switch the old boolean was still true and opened a subscription for a channel whose Files tab was never opened. `isFilesIndexEnabled` is the seam ChannelScreen calls and the test binds. Tests added for the guards that survived mutation: the pubkey cap and the timestamp check in `boundIndexSource`, the index retention cap reaching the tab's partial-list notice, newest-edit-wins, a deleted edit, the projection sort tie-break, the edit id check, the cursor's timestamp check, dispose awaiting an in-flight subscription, `hasMore` after dispose, the out-of-order edit cleanup and the projection's own authorship check, the hook's error and loading shape, and that nothing reaches the relay before Files is opened. `loadMore`'s duplicate complete gate is removed rather than tested: `backfill` owns it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

… subscription The controller took its live subscription through a call that resolves for every outcome. A relay that answered the REQ with CLOSED -- `error: too many subscriptions` is the likely answer once this branch adds a per-channel subscription on top of the ones a channel view already opens -- still produced an unsubscribe closure, so the tab recorded a refusal as an open subscription: no banner, no Retry, and no further event for the life of the channel view. A terminal CLOSED arriving after the readiness window was quieter still, because the shared client deletes the subscription and told nobody. Both halves are now reported. `subscribeLive` carries an `onTerminalClose` alongside `onReady`; a terminal CLOSED fires it once, guarded so a throwing handler cannot abandon the rest of the inbound frame dispatch. The Files controller subscribes through that seam, treats readiness "closed" as a failed attempt, and takes the subscription down on a later terminal CLOSED -- both leaving a capped banner and a working Retry. A malformed live event no longer throws out of the controller's callback. The relay's EVENT payload is JSON.parse'd and cast, so `pubkey` or `content` can be a number and reading one as a string threw a TypeError into the relay's shared dispatcher, which flushes one buffer for every subscription in a bare loop -- discarding the rest of that batch for the timeline, unread counts, typing and huddles too. Every relay-supplied string is now type-checked and capped at the DTO, and the callback body is wrapped so a failure becomes the tab's live-error banner instead. Tests bind the production wiring to a real RelayClient and feed it relay frames: a refused REQ, a capped CLOSED message, a terminal CLOSED after EOSE, a retryable CLOSED that the client still recovers from on its own, and a malformed event followed by a good one. Seven mutations across the new guards were run and all seven are killed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

…g new ones The Files index caps stopped at "refuse": once a channel reached MAX_INDEXED_ATTACHMENT_EVENTS the index took nothing further, so a channel with more than 5000 attachment events dropped every later live arrival while the tab said it was showing the most recent attachments. The deletion and edit caps had the same shape, so past them a new deletion no longer hid its file and a new edit no longer rewrote its row. The caps now keep the newest instead. Past a cap, an entry newer than the retention floor — the oldest retained entry — evicts that floor and takes its place; only an entry older than the floor is refused, which is the entry a "keep the newest N" cap exists to leave out. Evicting a message takes its edit and deletion entries with it, so no overlay outlives its row. The retention order is oldest-first with the entry id breaking ties, the same total order the projection sorts into, so an eviction cannot shuffle the page a caller is reading. The caps still bound the entries retained, which is the memory the index costs. Deletions now carry the timestamp of the deletion that removed them, so they can be ordered; an unusable relay timestamp normalises to 0 rather than being retained raw, because a NaN compares false against everything and a retained one would be an entry nothing could displace and a cap nothing could evict from again. Both the refusal and the eviction path set `truncated`, and the banner now names what is missing rather than only that something is. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Z6iidtozXxgx58BUZUKnu Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

…chment The eviction round bounded the deletion cap by recency alone, so a full cap gave up its oldest deletion entry with no regard for what that entry was doing. A deletion entry is the only thing hiding a deleted row: if its message was still indexed, dropping it put the file back on screen — an attachment its author deleted, returning silently, while the banner talked only about older attachments. Deletions of reactions, edits and ordinary messages all consume the cap, so a channel with a few hundred attachments and thousands of deletion markers reaches this on the next live delete. Eviction now ranks rather than only orders. An entry that still describes a message the index holds outranks one that describes nothing, and within a class the newer outranks the older; the cap gives up the lowest-ranked entry and refuses an arrival only when the arrival is lowest of all. For deletions that means orphan markers go first and a deletion doing real work is never the victim; if only real pairs are left, the message goes with its deletion, so the file is dropped rather than revealed. The edit cap was bounded on the same wrong population: a burst of edits for messages the index never saw could refuse a visible row's genuine edit for good. It now counts edits for retained sources and drops the rest first. Two smaller repairs: forgetting an evicted message now also drops a deletion keyed by the id of the edit that rewrote it, which the projection honours as well; and the truncation banner no longer claims the channel has more attachments than the tab keeps, since the edit and deletion caps set the same flag — it now says the oldest entries are dropped and names what may be missing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Z6iidtozXxgx58BUZUKnu Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

wiggdevin and others added 9 commits September 5, 2026 09:18

Merge remote-tracking branch 'origin/zs/main' into feat/files-index

3d91302

Signed-off-by: wiggdevin <202901685+wiggdevin@users.noreply.github.com>

wiggdevin marked this pull request as ready for review September 5, 2026 23:10

wiggdevin added this pull request to the merge queue Sep 6, 2026

Merged via the queue into zs/main with commit 6d68993 Sep 6, 2026
76 of 77 checks passed

wiggdevin deleted the feat/files-index branch September 6, 2026 01:29

Navigation Menu

feat(desktop): dedicated attachment index for the Files tab with bounded, recency-aware retention (T3b) - #25

feat(desktop): dedicated attachment index for the Files tab with bounded, recency-aware retention (T3b)#25
wiggdevin merged 9 commits into
zs/mainfrom
feat/files-index

wiggdevin commented Sep 5, 2026 •

edited by blacksmith-sh Bot

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

wiggdevin commented Sep 5, 2026 • edited by blacksmith-sh Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Fork deviations

Defect checklist

Gates

Gemini 3.8 Flash tester

Blind critic

GPT-5.6 Sol audit

Follow-ups

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

wiggdevin commented Sep 5, 2026 •

edited by blacksmith-sh Bot

Loading