Design the trusted-list refresh: what a pass is, when it runs, and what a failure leaves behind

[LKSNDRTMLKV]

Decision 4 of #324. The storage landed in #343 and nothing fills it, which is what this decides.

The direction was agreed — per-list, driven by each list's own NextUpdate, never a global sweep, and a failed refresh leaves the previous good copy in place. put in #343 is already shaped for that. What is not settled is the part below.

What a pass is

A refresh is not one operation. It is:

1. fetch the LOTL and verify it against the pinned anchor;
2. filter to national pointers — national_pointers already drops the self-pointer and the PDF variants several Member States publish beside their XML;
3. for each pointer, fetch and verify, and write verified or unavailable.

Step 1 failing is categorically different from step 3 failing for one territory: without a verified LOTL there is no authorised certificate list for any national list, so nothing in step 3 can be checked at all. A failed LOTL fetch must not mark 27 territories unavailable — that would turn one outage into a cache that says the Union cannot be checked, and put cannot prevent it because the caller decides.

Open questions

Cadence. Lists carry NextUpdate and the LOTL does too. Per-list scheduling is more faithful and needs per-list timers or a wakeup that scans due rows. A single periodic tick that refreshes whatever is due is far simpler and is what spawn_scan_prune / spawn_idempotency_purge / spawn_job_cleanup already do. The tick is probably right; the due-check needs NextUpdate persisted, which #343 does not store — it was left out deliberately rather than guessed, and this is where it gets decided and added.

Concurrency and politeness. 27+ documents, several MB each, from 27 Member State servers. Sequential is slow and kind; parallel is fast and looks like a small crawl. The guarded outbound path (dpp_common::outbound) is mandatory either way — the LOTL names the URLs, and until it is verified those URLs are attacker-influenceable, which is what fetch_lotl_pointers parsing before verifying already means.

First fill on a fresh node. A node booting with an empty cache has 27 fetches to make and cannot block boot on them. This is the same question as #344 from the other side.

Removal. A territory the LOTL stops naming should eventually leave the cache. #343 grants no DELETE on odal.trusted_list_cache deliberately — adding it means adding the row to ops/pg/README.md's DELETE set, which just grants-check gates. Decide whether stale rows are deleted or simply ignored; ignoring them is safer and leaks rows slowly.

Failure taxonomy. unavailable_reason is free text, and #343 argues that is right because what stops a list being read is not enumerable. But the retry behaviour differs sharply: a fetch timeout should be retried soon, a signature that does not verify should not be hammered, and a document over a parser ceiling (#320, Germany) will never succeed until the parser changes. If retry policy needs to distinguish them, the reason needs structure beside the prose.

The anchor interaction

A refresher is the only thing positioned to act on #321's anchor-freshness signal, and it is also the thing that breaks first when the pin goes stale: the LOTL stops verifying, step 1 fails, and every national list goes unrefreshed. The current signing certificate expires 2027-11-17. See #346.

Not blocking

Nothing here can produce a qualified seal, so an unfilled cache costs nothing today. It becomes the blocker the moment a QTSP credential exists.

Related: #324 · #343 (storage) · #344 (what is reported meanwhile) · #345 (anchor) · #320 (a territory that will not succeed)

[LKSNDRTMLKV]

#353 says it answers this, and it answers most of it — but not all of it, and it reverses one decision recorded here. Worth settling before this is closed on that merge, because two of the open questions above would go with it silently.

What #353 settles

• What a pass is — LOTL first, its failure abandons the whole pass and leaves the cache untouched, exactly as the "step 1 is categorically different" paragraph asks. A failed LOTL fetch does not mark 27 territories unavailable.
• First fill on a fresh node — 60 s after boot, then daily; boot does not block. (Though the lists do not reach a running node's verdicts at all until it restarts — The seal inspector holds the trusted lists it booted with, so a refresh never reaches a running node #361.)
• Concurrency and politeness — sequential, through the guarded outbound path.
• The filling trap in Decide what the seal route reports while the trusted-list cache is cold or partial #344 — claim_unheld gives every territory the LOTL names a row before any of them is read, so a half-filled cache reports the unreached ones as unchecked instead of answering unchecked: 0. That is Decide what the seal route reports while the trusted-list cache is cold or partial #344's middle row closed.

What it reverses

The direction was agreed — per-list, driven by each list's own NextUpdate, never a global sweep

#353 does the opposite: a global daily sweep, and per-list NextUpdate scheduling deliberately not used. Its argument is that notListed claims completeness over a set of territories, so the set is what has to be coherent, and per-list scheduling makes the cache a patchwork of ages with no moment at which it describes the Union.

That argument is good and I am not disputing it. But it is the opposite of what is written above as agreed, and the reversal should be recorded here rather than left as a surprise to whoever reads this issue next. NextUpdate is still unstored and still unread by the parser, so the refinement is not foreclosed — it just is not the schedule.

What it leaves open

• Removal. Untouched. A territory the LOTL stops naming keeps its row, and since the row may be Verified, a verdict goes on consulting a list the Union no longer names. The note above that odal.trusted_list_cache has no DELETE grant — so deleting means adding it to ops/pg/README.md's set and through just grants-check — is the part that makes this a decision rather than a patch. Ignoring stale rows is still the safer option and is what feat(seal): fetch and hold the EU Trusted Lists #353 does by default, without having chosen it.
• Failure taxonomy. unavailable_reason stays free text, and retry stays "wait a day" for every cause — a timeout, a signature that will not verify, and Germany's parser ceiling (Germany's trusted list fails on a second ceiling the xml-sec fork does not raise #320) are indistinguishable to the scheduler. A fourth cause has since appeared and is the most retryable of them: the write itself failing, which A failed trusted-list write is counted as a refresh, and leaves the stale copy the fail-closed rule exists to drop #362 covers.

So: close this only for the parts above, or keep it open scoped to removal + failure taxonomy. I would keep it open — the two remaining questions are real and neither is answered by code that is about to merge.