Uh oh!
There was an error while loading. Please reload this page.
✨ The fields a tailored wizard needs: voters, auth flow, client presets - #2988
Draft
edulix wants to merge 197 commits into
Draft
✨ The fields a tailored wizard needs: voters, auth flow, client presets#2988edulix wants to merge 197 commits into
edulix wants to merge 197 commits into
Conversation
Three additions to the plan and the profile, each one a thing the wizard cannot currently remember. **`Blueprint.voters`.** A plan describes an election and a census is a separate, much larger, much more sensitive thing, so most plans have none and the wizard says so. But a client whose membership does not change between elections has every reason to keep the two together, and telling them to hold the list somewhere else is telling them to hold it somewhere worse. `PlannedVoter` stays deliberately close to a row of `export_voters.csv` and keeps every column it does not recognise — the same passthrough the workbook path has, so a census exported by one route imports through the other. **`Blueprint.auth_preset`.** `CompileOptions` has had this since the builder was ported, so compiling with a preset already worked; what did not work was *remembering* the choice. A setting that vanishes when you reopen a plan is one somebody makes twice and gets differently the second time. `None` leaves the environment's own configuration alone, which is what every plan written before this field did. **`ClientProfile.presets`.** The wizard offers permissive, standard and strict — a sensible spread for elections in general, and frequently not the spread for *one organisation*. A client whose rules describe two ways they run a ballot is better served by those two, named after their own rules. Additive to ours unless `only_our_presets` says otherwise, and a profile that hides ours while offering none of its own is refused rather than leaving the client nothing to choose. `Behaviour::accepts` is the gate, and it is why presets live in Rust rather than as strings a browser renders: a client-facing button cannot select a behaviour no importer accepts. Two earlier versions of this tool shipped three such values between them. It is spelled by round-tripping through the patch's own deserializer rather than by a `match` over field names — a `match` is a second list of the policies, and the way a second list fails is a policy added to one and forgotten in the other. Five tests, and the two that matter were confirmed to fail with the gate bypassed: an under-vote `not-allowed` (a real value, for blank votes, and one of the three the old mappings invented) and a mistyped rule name. 452 election_config tests, up from 447. Refs #12769
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueComment |
`Blueprint.voters` had nowhere to go: the plan could hold a census and `to_workbook` did not emit it, so carrying one changed nothing. `voters_sheet` emits it in the columns `build_tables` already reads, minus the two the builder derives — `id` comes from `ids::uid` and `authorized-election-ids` from the areas, so handing those over would be supplying values the builder is about to overwrite. Everything else a client carries passes through, which is how a reporting breakout column survives the round trip. `None` rather than an empty sheet when the plan has no census. `build_tables` reads a present-but-empty Voters sheet as "this election has no voters", which is a different claim from "this plan does not carry the census" — and produces a bundle importing an election nobody can vote in. The columns are spelled locally rather than imported from `VOTER_LEADING_COLUMNS`, because that lives behind `election_config_templates` and this module has no feature gate — reaching for it would drag the whole builder into a front end that only wants to describe a plan. Two lists can drift, so `the_voters_sheet_matches_what_the_builder_reads` asserts every column this emits is one the builder reads. `check_census` catches the two failures nobody sees until a voter cannot vote: - **A duplicate username.** The importer derives the account from it, so two rows sharing one become one account and the second silently replaced the first. - **An area name no area has.** Voters are matched to areas *by name*, so a misspelling gives a voter no ballot at all — and a census is exactly where somebody retypes a name instead of copying it. Plus one warning, said **once** rather than per voter: an election with areas whose census names none of them. Ten thousand copies of one sentence is a report nobody reads. 458 election_config tests, up from 452. Both validation tests confirmed failing with the check bypassed. Refs #12769
`policyCatalog()` gave a front end the seven policies and left it to invent the counting method. That is the half of the value space where getting it wrong is worst: a preferential contest counted by a plurality algorithm reads a voter's rankings as an unordered set, and it imports cleanly. So the catalog now carries the voting types, every counting algorithm, and **which of them are preferential** — the real `COUNTING_ALGORITHMS` and the `PREFERENTIAL_ALGORITHMS` subset rather than a screen's idea of them. A front end can refuse the combination while somebody is choosing it instead of reporting it afterwards. Same reasoning as the policies: nothing in the UI knows what the values are, so a dropdown cannot offer one the platform lacks. `alphabetic` where the platform says `alphabetical` is exactly the class of bug two earlier versions of this tool shipped. Refs #12769
`Lint & Prettify` on #2988. Two chained iterator calls and a short const array that rustfmt wants laid out differently. Refs #12769
The wizard's review step should show the ballot. Drawing one from a second reading of the plan would agree with itself and not necessarily with the platform, and the whole value of a preview is being able to say "that is what voters get". So `election_config::preview` decides nothing. It takes the entities the plan compiled into — `ImportElectionEventSchema`, deserialized from the bundle about to be imported — and hands them to `create_ballot_style`, the same function windmill runs on *Generate ballot publication*. The output is byte-compatible with the publication preview windmill uploads, which is what `PreviewPublicationEvent.tsx` already opens, so the Voting Portal itself can render it at whatever version a client is running. Three things this needed: * `create_ballot_style` read `DEMO_PUBLIC_KEY` from the process environment on its happy path, with a `?`. So an election *with* a real key still could not generate a ballot style where that variable was unset — and `std::env::var` always fails on wasm32, which put the function out of reach of the browser entirely. The read moved into the branch that needs it, and the key can be passed in. * windmill's area→contest walk moved into `sequent_core::ballot_style` beside `create_ballot_style`, with windmill calling it. Composing ballots two ways would show a voter contests they will not get. * Ballot style ids come from the plan's deterministic factory, and the document is serialized through `to_document`. windmill mints a v4 uuid per style and translations live in a `HashMap`, either of which would make two previews of one plan different files. Two bugs fell out of pointing it at a two-area plan, both about the same misunderstanding — that a contest assigned to a parent area is not on its children's ballots. It is: the platform walks the path from the root down. `PlannedArea`'s doc comment claimed otherwise, and `check_ballot_coverage` warned that a child area's voters "would see an empty ballot" while the preview generated them a ballot with a contest on it. Both fixed, both now tested. Also `step-cli step compile-plan`, which builds a plan the way `build-election-event` builds a workbook and with `--preview` writes `ballot-preview.json` beside the archive. Refs #12769
`/preview/:tenantId/:documentId/:areaId/:publicationId` renders a publication preview the Admin Portal has already published to the public bucket. Before an event is imported there is no bucket and no publication, so a plan's ballots had nowhere to be seen. `/preview/file` takes the same document directly. `step-cli step compile-plan --preview` writes it, and so does the Election Architect's review step — so a delivery engineer can look at the ballot in the client's own portal, at whatever version the client is running, and what they see is drawn by the portal rather than by anything of ours. Hydration is the existing `updateBallotStyleAndSelection`, unchanged; the only new code is choosing the file and choosing which area's ballot to look at. The areas are labelled by the contests on them, because the document names an area by id and a list of uuids is not a choice anybody can make. Deliberately a file picker and not a URL parameter or a `postMessage` listener: this page is unauthenticated and frameable, so a channel letting another origin push a document into it would be new attack surface bought for a convenience. A file the operator picked is a document they have already seen. One change outside the new route. `App` sends a demo session back to `/preview/:tenantId/…` after signing in, which for a file-borne preview would be `/preview/undefined/undefined/…` and a blank screen; it now returns to `/preview/file`, which keeps the document in `sessionStorage` across that reload. Refs #12769
`PreviewFromFile.tsx` was formatted by hand. The repository pins prettier 3.8.1 in its lockfile; a newer one disagrees about an unrelated file that has been on `main` for months, so that is the version to run. Also `PublicationPreview::areas`. A ballot style names its area by id, which is what the platform writes, so the names travel *beside* the document rather than inside it — adding a sixth key would mean the preview was no longer the same file the Voting Portal opens. It is there because a picker offering four uuids is not a choice anybody can make. Refs #12769
`event_sheet` wrote `presentation.language_conf.default_language_code` as `languages.first()`, and nothing could say otherwise. So the languages were configurable and the default was not: a client wanting Spanish first had to reorder the list, and nothing told them that was what the order meant. `Blueprint.default_language`, `None` meaning the first — so every plan written before this compiles to exactly the bytes it did. A default naming a language the ballot is not offered in is an **error**, not a warning. The builder would fall back to the first, so it imports cleanly and opens in a language nobody chose, which is the class of failure nobody notices until a voter mentions it. `event_sheet` also filters the chosen language against the list rather than trusting it, so even with validation skipped it cannot emit one the event does not have. Refs #12769
Two more from the audit of what the Admin Portal's own contest form exposes and the wizard cannot reach — `EA-68d` and `EA-68e` on meta#12769. **`tally_configuration.tie_breaking_policy`.** The platform has two values and the plan could name neither, so every wizard-built contest took whatever `contest.hbs` said. It lands on `Tally` because settling a tie is a counting decision, and it writes a *dotted* column — the Admin Portal keeps it under the contest's tally configuration, and a flat `tie_breaking_policy` would have gone somewhere nothing reads. The default is `external-procedure`, not `random`: a random tie-break is defensible and also a result nobody can derive from the ballots, which is exactly the result that gets challenged. **A third group, `Layout`.** `presentation.columns`, `collapsible_lists`, `enable_checkable_lists` and `max_selections_per_type`. Its own group rather than more fields on `Policies`, for two reasons: the `policy_set!` macro requires every field to be an enum with a `COLUMN` and an `as_str()` and two of these are numbers; and they are a different kind of decision — `Policies` is what happens when a voter does something unusual, this is what the page looks like before they do anything. `Overrides` and `Behaviour` gain the member, so it inherits down the same three levels. `columns` is the pointed one: the Voting Portal reads it, and the wizard's own ballot preview has honoured it since the preview existed — so it was visible in the preview and unsettable in the plan. Validation, each check confirmed to fail against a bundle that breaks it: `TIE_BREAKING_POLICIES`, `COLLAPSIBLE_LISTS` and `CHECKABLE_LISTS` as lists rather than as a third copy of types that live in `ui-core`; zero columns refused; more than four warned about rather than refused, because it renders and it renders unusably on a phone, which is a judgement about the electorate; and a per-type cap above the contest's own maximum pointed out, since somebody who set it believes a limit is in force that never applies. One new invariant, from reading the changeset back rather than from a failure: `apply` is a struct literal so a forgotten field will not compile, but `columns()` builds a `Vec` and a field left out of it is a setting chosen on screen that never reaches the bundle. `no_setting_is_carried_on_screen_and_dropped_on_the_way_out` counts fields through serde for all three groups, so adding one fails until its column exists. 489 election_config tests pass. Not built for `wasm32` here — this machine's clang has no wasm32 target, so `ring` cannot compile; the new catalog block is type-checked through `--features wasmtest` instead, and CI covers the real target. meta#12769 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… for `EA-68f` on meta#12769. `raw_ballot::encode` packs a typed name into the ballot integer a byte at a time and `contest_context::bases` reserves the encoding slots for it. Both have been there from the beginning; nothing could produce a contest that used them, so the feature was reachable only by hand-editing a bundle. `PlannedContest` gains `allow_writeins` and `write_in_slots`. The slots are **minted at sheet time rather than kept in the plan**: a write-in slot is not a candidate anybody named, it is a blank line, and the number of them is the whole decision — keeping them in `contest.candidates` would put empty rows in the ballot's own list and make "how many candidates are standing" wrong. They are named `Write-in 1`, `Write-in 2` in every language the plan lists, because the Voting Portal draws the name as the slot's label and an unnamed one is a blank box. Validation refuses each half alone, and both rules are read off the codec rather than guessed: - **allowed with no slot** reserves no bases, so a voter is offered a feature with nowhere to type; - **a slot with the switch off** gets no bases either, and appears on the ballot as an ordinary option with a name nobody chose. Neither is refused anywhere downstream. Both import cleanly and are found by a voter. `check_contests` now counts write-in slots separately from candidates, because the arithmetic rules ask whether enough people are standing to fill the seats and a blank line is not one — a contest with one real candidate and two slots must not be able to claim it elects three. There is a test for exactly that. `base32_writeins` is deliberately **not** exposed. It selects the character map the text is packed with, which changes the size of the ballot integer; it is an encoding decision, not a client's, and the template's value is the right one. 495 election_config tests pass. meta#12769 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two corrections.
**Every description is per language, and I had one of them as a `String`.**
Checked properly this time, in the Admin Portal's own four forms rather than
inferred: `EditElectionEventDataForm`, `ElectionDataForm`, `EditContestDataForm`
and `CandidateDataForm` all edit `presentation.i18n.<lang>.description`, and each
keeps the flat `description` column as a mirror of the English one —
`newContest.description = newContest.presentation.i18n.en.description`. So a plan
carrying a single text put the same words in front of every voter whatever
language they read, and the reference page I wrote said that was a limitation of
the field. It was a limitation of my implementation.
`PlannedContest.description` becomes `Translated`, and the three entities that had
none get one: the event, the election and the candidate. `i18n_columns` takes the
field name, so `description` gets exactly what `name` already had, and every sheet
writes both the i18n block and the flat mirror.
A plan saved during the one release where it was a string still opens:
`translated_or_plain` reads a bare string as English, and an empty one as absent
rather than as `{"en": ""}` — a description that exists and is blank would put an
empty string where a blank cell belongs.
**And wasm32 builds on an Apple-silicon Mac without Nix.**
I said this machine could not do it and left the wasm-gated half of
`election_config` type-checked only through `--features wasmtest` on the host.
That was wrong. The failure —
unable to create target: 'No available targets are compatible with triple
"wasm32-unknown-unknown"'
— is `ring`'s build script, not Rust: `rustup target add` does not help because
the missing piece is a **C** compiler, and Apple's clang has the WebAssembly
backend disabled. `packages/sequent-core/flake.nix` already solves it with LLVM
19's clang; `.devcontainer/scripts/wasm32-env-without-nix.sh` does the same from
Homebrew's LLVM, reading the resource-directory version rather than pinning it so
the next upgrade does not break it.
`cargo check --target wasm32-unknown-unknown` with
`wasmtest,default_features,election_config_xlsx,election_config_templates,election_config_archive`
is clean. `nix develop` remains the supported path and the one CI uses; this
exists so that "it does not build for wasm32 here" stops being a reason to check
less.
498 election_config tests pass.
meta#12769
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>`EA-68g` on meta#12769. Six election-level settings, all of them reaching a voter, none of them settable from a plan. `num_allowed_revotes`, `spoil_ballot_option`, `presentation.grace_period_policy` and `grace_period_secs`, `presentation.start_screen_title_policy`, and `permission_label` — the last only because a delivery occasionally has to match an existing label; empty means the builder derives one. **`num_allowed_revotes` counts casts, not re-votes, and zero means unlimited.** I wrote a check that refused zero as "an election nobody can vote in", read straight off the field's name. The Voting Portal is the authority and it is unambiguous: `castVotes.length < num_allowed_revotes`, with "If num_allowed_revotes is 0, allow voting" immediately above it. So zero is unlimited, one is cast-once-and-final, two is one change. An existing build fixture caught it, which is the only reason it did not ship as a validator that refuses a legitimate bundle. `unlimited_revoting_is_not_an_error` now pins it, and the comment says where the authority is — the name will mislead the next reader too. The spoil warning is corrected with it: spoiling a cast ballot needs another cast to make, so it fires on exactly one, not on "fewer than two" — under the wrong reading, unlimited would have been flagged. The grace period is checked as a pair, both ways round: a policy with no seconds is no grace period, and seconds with no policy means voting closes on the deadline. Either way somebody believes voting stays open a little longer than it does, which is the sort of thing a voter finds at one minute past the close. `PlannedElection` gets a hand-written `Default` rather than a derived one, because `derive(Default)` gives zero casts and an empty `grace_period_policy` — neither of which is a value the platform has. `#[serde(default = "…")]` applies only when deserialising; a struct literal gets the impl. Fixtures use `..Default::default()` now, so the next field does not break all of them. 506 election_config tests pass, and wasm32 builds. meta#12769 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`EA-68i` on meta#12769. The wizard already wrote each contest's `presentation.sort_order` from its place in the list, and nothing said whether the portal should *honour* it — so a carefully arranged election could be shuffled anyway, and one left alone could be expected to shuffle. `elections_order` on the event and `contests_order` on the election say which. One value space, three places: `custom`, `alphabetical`, `random` are also a contest's `candidates_order`, and `ui-core`'s `sortContestList` and `sortCandidatesInContest` are the same function twice over. So it is one `ORDERINGS` const and one check, and a fourth value is refused rather than silently ignored. **And a guard on `sheet_of`, from making the mistake it catches.** Every sheet builds its column names in one place and its cells in another, with nothing tying the two together. Adding `elections_order`'s value without its column shifted every later cell one place left, so `presentation.sort_order` was read as a language code — and the failure surfaced as "no entry found for key" in a test about *languages*, three sheets away from the edit. `sheet_of` is the single point every sheet passes through, so it now refuses a row whose length does not match the header. The two lists stay separate on purpose: pairing them would mean a `Vec<(String, Cell)>` per row with the column names repeated on every one. `a_ragged_sheet_is_refused_rather_than_written` asserts it directly, since producing one from a `Blueprint` now takes editing the builder. 511 election_config tests pass, and wasm32 builds. meta#12769 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`EA-68k` on meta#12769. The plan has collected trustees and a threshold from the beginning and emitted no ceremony, so an event imported with a key nobody had been asked to generate. **`trustee_ids` carries names, not identifiers**, and that is the platform's own convention rather than a shortcut: `windmill/src/services/import/import_election_event.rs` builds a `HashMap<name, id>` from `get_all_trustees(tenant_id)` and maps the field through it, the same way a voter's area name is resolved. And the same trap — line 763's `.unwrap_or_default()` turns an unmatched name into an **empty string**, so the ceremony imports with a member who does not exist, the count looks right, and it surfaces when the key is generated or, worse, when the threshold cannot be met at the tally. A nameless trustee is now an error, since a blank name is that failure with certainty rather than by accident. Three things came out of doing it rather than out of a failing test: - **A warning on every plan is not information.** I first pushed one unconditionally saying the ceremony resolves its members by name against the tenant — true and important, and it fires on every sound plan, which teaches people to skim the report. `a_sound_plan_has_nothing_to_report` caught it. It is a handover fact, so it belongs in the reference page and the trustees section's hint. - **The ceremony belongs in `build`, not patched on after it.** The first version edited `bundle.export` inside `compile_plan`, so `build` alone produced none — and every test asserting through this file's own `compiled()` helper saw an empty array and passed for the wrong reason. It is now `BuildOptions::keys_ceremony`, which both paths go through; the workbook path passes `None` and keeps the template's empty array, since it has no trustee sheet to draw from. - **The wasm32 check earned its keep on its second day.** Adding a field to `BuildOptions` broke a literal in `wasm.rs`, which no native build compiles — `cargo check --target wasm32-unknown-unknown` caught it, and last week I would have pushed it. 515 election_config tests pass. meta#12769 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Part of `EA-68j` on meta#12769. `presentation.show_cast_vote_logs` — the Voting Portal's ballot-locator page reads it and shows a tab where somebody holding a ballot identifier can confirm theirs was received. A real decision about how much a voter can verify for themselves, in the Admin Portal's event form, and unsettable from a plan. Shown by default. Verifiability is the kind of thing an election should have to argue itself out of rather than into. Two smaller things: - **One event-presentation check instead of two.** `check_elections_order` was a single-purpose function, and adding a second would have made two of the same shape. It is one loop now, and the third value will cost a line. - **"is not a elections_order".** The field-name template produced that. Naming the field is right and the article was not, so every one of these messages says "is not a **valid** X" — including the three that already read awkwardly (`under_vote_policy`, `collapsible_lists`, `grace_period_policy`). Four assertions updated with it; these strings reach a client. `presentation.cast_vote_gold_level` is deliberately left out. It changes which buttons the review screen offers and I could not establish what it means well enough to write a sentence a client should act on — guessing at documentation for a voter-facing setting is worse than not shipping it. Recorded as `EA-84`. 518 election_config tests pass, and wasm32 builds. meta#12769 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`voting_channels` was in `election_event.hbs` and `election.hbs` and settable from nothing. Four flags, and every one is a *precondition* rather than a switch: ticking it makes the platform offer the channel, and something else has to exist before a voter can use it. That asymmetry is why this is validated rather than just written. online nothing else. Off means nobody votes, and that is now an error. early_voting the areas. `PlannedArea.allow_early_voting` is the other half, and the Voting Portal needs both — `isEarlyVotingOpen` is `area_presentation.allow_early_voting == allow_early_voting && early_voting_status == OPEN`. Either alone is refused, both ways round, and the message names the areas. kiosk an auth client the bundle cannot create. `AuthContextProvider` answers a `?kiosk` URL with `<client>-kiosk`. A warning, because the missing half is in the environment, which this pass cannot see. telephone the event's IVR tab, which `ElectionEventTabs` reveals on this flag and which no part of a bundle fills. Also a warning. Written to the event *and* every election, from one field: the Publish screen reads the block off `useRecordContext<ElectionEvent | Election>`, so writing it in one place leaves the start control present at one level and missing at the other. A difference between the two is now reported — by whether each channel is *on*, not by which keys are present, because `election_event.hbs` names three channels and `election.hbs` names four and comparing presence would flag every ordinary build about `telephone`. Two things deliberately not done, both recorded rather than guessed: `paper` is in `hasura_core::VotingChannels` and in nothing else — no `VotingStatusChannel` variant, no status block, no Publish control, no label, not in the Admin Portal's own `defaultChannels`. No field for it. A bundle naming it gets a warning rather than a refusal: it does no harm, it just does nothing, and somebody who set it believes otherwise. `is_kiosk` on the election is left as the template's `false`. Its only reader is `GetTallyData`, which selects the whole row — no behaviour hangs off it, and the voting portal decides kiosk-ness from the URL. Setting it alongside `voting_channels.kiosk` would be a guess about an election that allows both. `an_early_voting_policy_the_platform_does_not_know_is_refused` closes a real gap: an existing builder test overrides `area.hbs` with `early_voting_allowed`, a plausible rearrangement of the real `allow_early_voting`, and nothing anywhere would have refused it. The portal compares the string exactly, so a near miss reads as "no early voting" in silence. 531 election_config tests, wasm32 clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`EA-68l` was going to add candidate images as `images/<candidate>.<ext>` plus a
manifest in `layout.auxiliary`, accepting an `image_document_id` when one was
known. Reading the importer says that cannot work, so this reports it instead of
shipping a field that quietly produces blank ballots.
An image is two fields that have to agree, and both are UUIDs:
presentation.urls[] {url, is_image: true}, where url is
`tenant-<t>/document-<d>/<name>` appended to
PUBLIC_BUCKET_URL. What a *voter* sees —
`Answer.tsx` through `ui-core`'s `getImageUrl`.
image_document_id What the *Admin Portal* needs afterwards, to show,
replace or delete it.
`replace_ids` hands the whole serialized bundle to `replace_realm_ids`, which
calls `replace_uuids` — a regex over the **raw text** swapping every UUID-shaped
substring for a fresh `Uuid::new_v4()`. The keep list is the old tenant id, the
old event id, Keycloak authenticator config values and
`ELECTION_EVENT_FIXED_UUIDS`. A document id is on none of them, and the one
inside the url string is rewritten too — it is just text to the regex.
So `image_document_id` names no row and the path names a document directory that
has never existed. The candidate shows a blank space, on the ballot, silently.
The bytes cannot ride along either: `ImportElectionEventSchema` has no
`documents` array and the upload is a live presigned-URL call.
This is the same property that makes `keys_ceremonies[].trustee_ids` carry
trustee *names*, and it is worth stating as a rule because it decides every
future field of this shape: **an identifier a bundle carries that points at a row
the bundle does not contain is destroyed on import.**
`check_carried_images` warns — not errors, since everything but the pictures
works — when one arrives from a hand-written workbook or a `base_export` taken
from an event that had them. One problem naming every candidate rather than one
each: forty photographs would otherwise bury every other finding, and the answer
is the same for all of them. Non-image `urls` are left alone; `getLinkUrl` looks
those up by title, they carry no document id, and they survive import intact.
534 election_config tests, wasm32 clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>Reverts the warning added in 804be82, which was wrong, and replaces it with a check of what the mechanism actually requires. I concluded an image could not survive an import from two true facts — `ImportElectionEventSchema` has no `documents` array, and `replace_uuids` rewrites every identifier in the JSON — and never looked at what the importer does with **zip entries**. I even noted that `replace_ids` returns a replacement map without asking who consumes it. The answer is `process_s3_file`: the `images/` branch of `import_election_event` hands it that same map, which * pulls the old identifier from the entry name with `document_([0-9a-f-]{36})`, * looks it up in the map, * and creates the document **with the new identifier**. So `image_document_id`, the identifier inside `presentation.urls`, and the uploaded file all move to the same new value together. `is_public: true` on that branch is what lets `PUBLIC_BUCKET_URL` serve it. The contract is therefore three parts that must agree: candidates[].image_document_id <document> candidates[].presentation.urls[] {"url": "tenant-<t>/document-<d>/<file>", "is_image": true} zip entry images/document_<d>_<file> `check_images` checks the half a JSON-only pass can see. A url naming a different identifier than the field beside it is an **error**: the two are kept together only because they are the same string before the rewrite, so differing means they are rewritten to two different values. Each half present without the other is a warning — a url alone still puts the picture on the ballot but leaves the Admin Portal unable to change it; a document alone is uploaded and shown nowhere. Not checked here, because this pass sees the JSON and not the zip, and recorded in the engineering page instead: an entry whose identifier appears nowhere in the JSON **fails the whole import**, since `process_s3_file` does `replacement_map.get(uuid).ok_or_else(…)` and the map only holds identifiers that were actually rewritten. What I should have taken from `keys_ceremonies[].trustee_ids` is narrower than what I did take. It is true that an identifier pointing at a row the bundle does not contain is destroyed. It does not follow that such a thing cannot be carried — a bundle *can* contain the row, as a zip member, and then the importer keeps both in step. The question is not "is it an identifier?" but "does the archive carry what it points at, and does the importer know to look?" No control yet: the tenant is resolved inside the builder from four sources, so only the builder can compose the url, and the plan/UI half is its own change. The organiser-facing pages therefore say nothing about photographs rather than promising an upload that is not there. 536 election_config tests, wasm32 clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The feature the previous two commits established the contract for. Three parts, and
they are only correct together:
candidates[].image_document_id derived from the candidate's external_id
candidates[].presentation.urls[] {url: "tenant-<t>/document-<d>/<file>",
is_image: true}
images/document_<d>_<file> the bytes, inside the zip
**The identifier is derived, not stored.** `image_document_id(ids, external_id)` off
the same uuid5 factory as every other identifier, so two builds of one plan produce
the same archive and a rebuild is a diff somebody can read. `candidates_sheet` and
`plan_images` derive it independently rather than passing one around: a stored
identifier is a thing that can be edited into disagreement, and this one has no
reason to be editable.
**The url is composed by the builder**, not the sheet. It embeds the tenant, and
`resolve_tenant_id` draws on the explicit option, the Parameters sheet, the base
export and the id-factory fallback — so the builder is the only thing that knows
which tenant the bundle will claim. Composing it upstream would have written a
tenant the bundle does not carry, and import's textual replacement only rewrites the
one it does. An existing `is_image` entry is replaced rather than appended, matching
the Admin Portal's own uploader: `getImageUrl` takes the *first* one, so a second
would surface as a stale picture.
**The bytes travel through `BuildOptions`**, like the key ceremony, because a
workbook cell cannot hold an image. The workbook path therefore has none, which is
correct — a spreadsheet has no bytes to offer.
**Base64 in the saved plan.** A plan is what somebody reopens next month when a
candidate withdraws; one that lost its photographs on reopening would mean uploading
forty again, which is the tedium this removes. It makes such a plan large, and that
is the right trade.
A url is only written when the archive actually carries the file, so a row naming a
document with nothing behind it gets no url — `check_images` reports it rather than
putting a broken picture on a ballot. Write-in slots get a blank cell: a slot is a
blank line, not a person, and the cell still has to be there because `sheet_of`
refuses a ragged row.
Two things caught by tests rather than by reading:
* `compiled()` did not pass `images`, so three tests failed against a generator
that was already correct. Second time this helper drifting from `compile_plan`
has produced a wrong answer — the first was the key ceremony.
* `wasm.rs`'s `BuildOptions` literal broke again. No native build compiles it;
the wasm32 check caught it, for the second time in three days.
543 election_config tests, wasm32 clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>…is refused `EA-70`. Three things a plan could carry that make the encryption decorative, all previously allowed: no trustees was a warning; built a bundle with an empty `keys_ceremonies`, which imports and has nobody to generate a key one trustee was allowed silently threshold of one was a warning: "one trustee alone can open the tally" All three are now errors. The guarantee threshold encryption buys is that **no single party can read the votes**, and each of these hands it away — one person decrypts every ballot, alone, and nobody else can tell. It is not recoverable either: by the time anybody looks, the votes have been cast under that key. **This reverses `a_threshold_of_one_is_allowed_but_said_out_loud`**, which is now `a_threshold_of_one_is_refused`, and the code carries the reason rather than leaving a reader to find the old test in the history (`INV-26`). A warning on the last screen somebody reads, about a property they cannot check afterwards, is the exact shape of defect this validation pass exists to refuse. Also new: **every trustee needs an email address**, because that is how they are invited to the ceremony. A trustee who never receives the invitation does not attend, and the shortfall is discovered when the count cannot be opened — the same failure as a name resolving to nobody, arriving by a different route. `looks_like_email` is deliberately a shape check and not RFC 5322. It cannot know whether an address receives mail, and being stricter than the specification would refuse a working address — a worse defect than accepting one that bounces. What it catches is what people actually type: a name in the email box, a missing domain, a stray space, a doubled `@`. Both directions are tested, including `"quoted"@example.org` and `ada+trustee@example.co.uk`, which a hand-rolled regex usually rejects. 547 election_config tests, wasm32 clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ealm A census column the wizard has no field for becomes a **Keycloak user attribute** — that passthrough is why a client can carry a reporting breakout without a code change. But Keycloak only stores an attribute its user profile declares. An undeclared one is dropped or refused depending on the realm's unmanaged-attribute policy, and either way the column is in the file, in the import, and **not on the voter**. Nothing said so. `declare_census_attributes` adds them. Deliberately separate from `patch_user_profile`, which *warns* when a preset wants an attribute the realm lacks — that is right, because a preset and a realm disagreeing is a mismatch somebody has to resolve. A census column is the opposite situation: the author is declaring a new attribute rather than expecting an existing one, so the answer is to add it. Added minimally and permissively: a name, and permissions letting an administrator read and write it. Not required, no validator — this knows the column exists and nothing about what belongs in it, and a guessed validator would refuse data the client's own file contains. A test asserts exactly that, because the tempting version of this adds `required` and breaks every census with a blank cell. Read from the sheet's **headers** rather than its rows: a column present in the header and empty in every row is still a column the author declared, and skipping it would mean an attribute that materialises the moment somebody fills one cell in. Nothing is reported. `Severity` is `Error` or `Warning` and neither fits — this is the wizard doing exactly what was asked, and a warning saying so teaches people to skim the ones that matter. It is not silent: the wizard shows a dialog at the moment the column is added, which is when somebody can still change their mind. `authPresets()` now also carries **`profile_attributes`** — the user-profile attributes each preset actually reads off a voter, taken from what each `build` patches: `otp_email_or_sms` reads `email` and `mobile`; `voter_link_plus_dob` reads `dateOfBirth` and `mobile`, matching its own `search-attributes`; the certificate and SAML presets read none, because the identity comes from the certificate or the assertion. Static rather than derived by calling `build` with a dummy input, because the census's column chooser needs it *before* anything is decided. 549 election_config tests, wasm32 clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…izard down
Found by clicking "Show the ballot" in the running wizard on a plan that builds:
white screen, and `TypeError: Cannot read properties of undefined (reading
'filter')` from `BallotPreview.tsx:163`, `document.ballot_styles.filter(…)`.
`serde_wasm_bindgen` renders anything that goes through `serialize_map` as a JS
**`Map`**, and a `Map`'s entries are not properties — so `ballot_styles` was
`undefined`. `preview_ballot` returns a `serde_json::Value` deliberately (a
`Value`'s object is a `BTreeMap`, so a saved preview's keys are sorted and two
previews of one plan diff cleanly), and `PublicationPreview`'s own fields are
`Value` too, so every key of the document was affected.
What made it survive: `JSON.stringify` prints a `Map` as `{}`. Inspecting the
output showed `"preview": {}`, which reads as an empty document — a plan that
produced no ballot — rather than as a document whose keys are in the wrong kind of
container. The `report` and `areas` beside it are structs, so they arrived as
plain objects and looked fine.
`to_js` now sets `serialize_maps_as_objects`. Not `Serializer::json_compatible()`,
though that is what `wasm/areas.rs` and `wasm/wasm.rs` use: it also turns `None`
into `null` rather than `undefined` and changes how bytes serialise, and
`compile_plan` hands its artifacts across as bytes. One flag, for the one thing
that was wrong.
No test in either half could have caught this. Every React test injects a fake
core, and no Rust test can run `serde_wasm_bindgen` — it needs a JS context. This
is the second bug of that shape on this ticket (the first was the vendored build
simply being old), which is what makes the e2e-against-the-real-WASM item the
next one rather than a nice-to-have.
561 election_config tests pass; wasm32 compiles.`COUNTING_ALGORITHMS` is the value space and stays exactly as it is: ten algorithms, any of which a plan may name, all of which validate and build. `OFFERED_COUNTING_ALGORITHMS` is a second, shorter list, for a dropdown. The two answer different questions. A validator asks whether a value is real; a picker asks what somebody should be choosing between. Four of the ten — `borda-mas-madrid`, `desborda`, `desborda2`, `desborda3` — are one family written for a specific Spanish municipal process and its quota rules, and the difference between `desborda2` and `desborda3` is not something a label can carry. Offering four near-identical names to somebody choosing how their union election is counted is offering four ways to get it wrong. Nothing is taken away. A client whose rules genuinely name one has a plan written by hand or through `step-cli`: it validates, it builds, it counts, and a front end that keeps the current value in its list still shows it. Turning a narrower menu into a validation error would break a real election to tidy a dropdown. Two tests. One asserts every offered algorithm is one the platform accepts — the failure it prevents is a rename in `COUNTING_ALGORITHMS`, after which the dropdown would offer a value nothing takes. The other asserts the four hidden ones are still real, still preferential, and therefore still bound by the pairing rule `check_tally` enforces; it also asserts they are *not* offered, so it cannot pass by asserting nothing. The catalog carries both lists, so which is which is the front end's to read rather than its to decide. 563 election_config tests. wasm32 compiles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`validate_plan` writes 38 English sentences and they cross the WASM boundary as text, so a Spanish build of the architect shows Spanish everywhere and English on the one screen that stops somebody building. The obvious key does not work, and it is worth recording why because it looks as though it should. `Code` is a *category*: those 38 share eleven of them, and adding the path does not separate them either — `ContestArithmetic` produces four different sentences about the same contest, because the path names the contest and not the complaint. `ProblemList.tsx` already knew this; it says so where it builds a React key out of code, path and index. So `Problem` gains `id` — a stable name for the sentence — and `details`, the specifics it interpolates, since sixteen of them name an address, a username or a pair of numbers that no identifier can reconstruct. Both optional, and both absent means "show the English", which is what all 143 call sites do today. The 53 bundle-level messages in `validate.rs` are therefore unchanged and still correct: adding an id is opt-in per message rather than a rewrite. Two left deliberately unnamed. The `ConflictingColumns` one is a sheet builder disagreeing with itself during `to_workbook`, and the bundle-schema one says in its own text that it is a bug in this tool — neither is an author's mistake, and translating a bug report helps nobody who has to send it to us. One site became two. A threshold of zero opens the tally to nobody and a threshold of one opens it to anybody on the list; they were one push with a conditional message, and one id cannot name two sentences. Both new checks were proved failing first: a copied id fails `every_named_problem_has_its_own_name`, and a message left unnamed fails `every_complaint_about_a_plan_carries_its_name_and_its_specifics`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`BuildOptions` gained `images` and `keys_ceremony` for the wizard's sake and the two `step-cli` call sites were never updated, so `Tests` and `Build Step CLI Binary` have been red on this branch for days with `E0063`. Nobody looked; the compiler had been saying it the whole time. Both set explicitly rather than with `..Default::default()`. The shorthand would have prevented this error and that is precisely the objection to it — the compiler refusing to build is the signal that caught this, and turning it off because it fired is the wrong lesson. What was missing was somebody reading CI, which is now `INV-33`. Neither value is lost by being empty. `compile_plan` builds both from the plan itself — the trustees become the ceremony, the candidates' photographs travel as files — and passes them through `..options.clone()`, so what the CLI supplies is replaced. The workbook path has no photographs to offer, because a cell cannot hold bytes, and builds no ceremony because only a plan carries a trustee list. Checked the whole workspace rather than the one crate this time, per the new `INV-0`: `windmill`, `harvest` and `step-cli` each compile on their own. Checking `-p sequent-core` alone is what let this through, and it is the second time on this issue that a per-crate check would have caught something the workspace build hid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"No trustees" and "one trustee" are different complaints, and they were one
push whose lead was chosen by a `format!`. One id cannot name two sentences —
that is what `id` exists for — and the count could not rescue it either: a
number crossing the wasm boundary becomes a *string*, and i18next pluralises on
numbers, so `{{count}}` would have produced "0 trustees" and "1 trustees" in
every language that inflects.
The same shape as the threshold split beside it, for the same reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>The wizard now draws as many trustee rows as the threshold asks for, at a floor of two, so a new plan arrives showing two empty boxes instead of an empty list and an instruction to add some. Counting those rows as trustees would report the commonest state of a new plan as complete — and would replace the complaint that matters, that nobody is holding the key, with two lesser ones about empty fields. So `check_trustees` counts a trustee with neither a name nor an address as absent, and the threshold check compares against that count rather than the list length. Each blank row is still checked individually, so nothing is accepted quietly; they simply do not count towards having anybody. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each is a real field on `ElectionEventPresentation`, each is already written by `election_event.hbs`, and none of them could be chosen — so every plan shipped whatever the template said and nobody was asked. **Language detection policy** is the half of the language question that had no control. A client could say which languages the ballot offers and which is the default, and then the default only applied to browsers that asked for nothing else. Two values, `browser-detect` and `force-default`, published through `policyCatalog` so the wizard's dropdown is the platform's own list rather than one kept in step by hand (INV-8), and validated, because it is the one language setting whose value space is not a list of the plan's own languages and therefore the only one that can carry a word from nowhere. **Skip election screen** and **show user profile**, which the voting portal both read. **Support materials** gets its `activated` switch and per-language title and subtitle — and no more, deliberately: `ElectionEventMaterials` carries only that boolean, and `ImportElectionEventSchema` has no documents array, so the Admin Portal's per-material rows cannot ride in a bundle at all. Offering them would produce a bundle that drops them silently. Every field is emitted **only when the plan says something**, so a plan written before any of them existed compiles to the bytes it used to — the template already carries a value for each, and an absent column leaves it alone. There is a test for exactly that rather than a claim. Both fixtures gained `..Default::default()`, which is what makes the next field added here a one-line change instead of a compile error in two files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`build` takes a `&Sources`, and `census_attributes` reads `CensusSource::columns()` where a build is handed one. This is the place the census's *shape* decides something outside the census: every column becomes a Keycloak user-profile attribute, and Keycloak drops an attribute its profile does not declare — so a column that fails to reach this list is a value the sign-in flow reads as absent, with no error anywhere. It is the one consequence of moving the census that has no symptom, which is why it gets a test rather than a comment. `the_realm_declares_the_same_attributes_either_way` builds the same census twice, once as the sheet and once as a source, and compares what the realm declares. Sabotage-checked by truncating the source's columns. **`build_voters` deliberately still reads the Voters sheet**, and the reason is worth having written down. A source yields `PlannedVoter`, which cannot carry `enabled`, `email_verified` or `authorized-election-ids`: all three are in `RowShape::OWNED`, so they are excluded from `extra`, and there is no field for them. Routing the builder through a source today would switch on a voter the census switched off and authorise a restricted voter for every election in the event — silently, in the delivery. `a_source_cannot_yet_say_what_the_voters_sheet_says` pins the gap in both directions: nothing through the shape, all three through the sheet. Nor would it save anything yet. The Voters tab stays in the workbook, so the census is already in memory at build time whatever `build_voters` reads; iterating a source instead would add a walk and move no bytes. Also fixes a feature combination `EA-F4-054` broke: `sources` was left ungated on the theory that a plan describes these things whether or not the build can render a template. It cannot — `validate_plan` lives in `architect`, which is behind `election_config_templates` — so `--features election_config_xlsx` alone would not compile. All eight combinations check clean now. The wasm boundary does not move: `buildFromWorkbook` passes an empty `Sources`, so the JS side is unchanged and beyond's CI keeps building green against this tip. Refs: meta#12769
Nine commits landed on this branch while EA-F4-056 was being written. Two of them do not merge as text. `Requirement::kind` became a typed `RequirementKind` — because the `&str` it replaced fell through a `_ =>` arm meaning "authenticator", so a misspelled kind was checked against the wrong collection of the realm and reported nothing. But `NeedsDoc::kind` is a *profile's* text, read at run time, and is exactly where a misspelling comes from. So `as_requirement` returns `Option<Requirement>` and refuses a kind it does not recognise, rather than reintroducing the guess one layer up. `a_requirement_kind_nobody_recognises_is_not_guessed_at` covers the three that parse, whitespace around them, and the ones that used to be treated silently as authenticators. `areas_inside_each_other_are_refused_by_the_plan_validator` arrived written against both the older `PlannedArea` and the older `validate_plan`; it goes through the module's `checked` helper like every other test here. 924 tests, every feature combination clean, `cargo fmt --check` clean. Refs: meta#12769
The wasm boundary grows two optional fields and loses nothing. `compilePlan` and `previewBallot` accept `options.census` — a `CensusPull`, which is three methods rather than an array — and `options.files`, the bytes the plan's file names refer to. A host that passes neither behaves exactly as it did: the census and the files are read off the plan. That is the whole design constraint here. beyond's CI checks out step's live tip and builds the WASM in the same job, so a boundary that only grows is the only kind that can be pushed before the browser half exists. This is the gate the beyond commits sit behind, and it can sit on the branch indefinitely without breaking anything. `CensusPull` is an extern type, not a deserialised one, and the reason is the point of the exercise: ten million members reaching Rust as one JS array is precisely what `serde_wasm_bindgen::from_value` would do with them. Its shape is `CensusCsvReader`'s own, which is not a coincidence — that class already exists and the wizard's census store already speaks to it, so a CSV the browser has parsed once is not parsed again to be handed over. **A `CensusCsvReader` is a `CensusPull`.** `JsCensus` reads the columns once, at construction, and keeps them. `CensusSource::columns` hands back a borrowed slice so it cannot call into JavaScript — and should not, since `census_attributes` asks per build. Reading them up front is also what makes the promise true: the column list is available before the first row. `compile_plan` takes a `Compile` struct rather than a fifth positional argument. Rust would have caught a misordered call, but five arguments is where the next `compile_plan_js` mistake hides — a profile passed as a third positional that the browser never sent, so every profiled build compiled with `None` and no locked value reached a bundle, silently, on both sides. Two things that had to move with it, or the new option would be decorative: - `plan_images` and `plan_materials` prefer a named file from `sources.files` over the plan's own bytes. Which way round matters — a plan opened from a save file carries names and no bytes; one the wizard is holding carries both. - `check_logo` refused a logo that names a file and carries nothing. That is the exact shape of a reopened plan, so it now asks the sources too. Without this the first thing the new option does is fail validation. `the_caller_may_bring_the_census_and_the_files` is the Rust half of the browser feature and the only half testable without one: a plan carrying no voters and no logo bytes, beside sources carrying both, looked for in the finished bundle. 925 tests, every feature gate clean, `cargo fmt --check` clean, and the wasm feature builds. Refs: meta#12769
`archive::save_file` writes `<external_id>-plan.zip` — `blueprint.json`, `census.csv`, `files/<name>`. The wizard stops handing over a bare `blueprint.json`, because a plan on its own is no longer the whole document: it is a document with the members' names and the candidates' photographs missing, which is the kind of loss nobody notices until the file is reopened somewhere else. A bare `blueprint.json` still *opens*, and always will — somebody has one saved from last month, and refusing it would be refusing their work. `open_delivery` reads a save file and a delivery with one function, because they are one layout twice: both carry `blueprint.json` at the root, and what differs is where the bulk sits. The root is looked at first, the importable zip second. **The delivery branch used to read `blueprint.json` and stop.** It returned `Report::default()` and never looked inside `official_election_setup.zip`, so reopening a delivery gave back whatever the JSON still happened to carry. That is fine today and is nothing at all the moment a plan stops carrying its own bulk — a defect made invisible by the very duplication this programme removes. `a_delivery_brings_its_census_and_its_files_back` is the test; it fails against the old behaviour. `Opened` gains `sources`, derived from the plan for every door with nothing else to offer and read from the archive for the two that have. A caller hands it straight to `compile_plan` without asking what kind of file it opened. Two smaller things the round trip forced: - `sources::cell_of` is the inverse of `RowShape::voter`, written beside it so the writer is not a fourth copy of *what a census row means*. - `plan_file_name` turns `images/document_<uuid>_<name>` back into the `<name>` a plan points at, splitting exactly twice — a photograph called `photo_of_ada.jpg` keeps every underscore it came with, and a greedy split would invent a file name nothing references. The comma is in the round-trip test on purpose. `O'Brien, Jr.` is an ordinary member's name and an unquoted CSV field turns them into two members. 928 tests, every feature gate clean, wasm builds, `cargo fmt --check` clean. Refs: meta#12769
Three new exports beside the old ones, and nothing removed. beyond's CI builds this crate's live tip, so additive is the only thing that can be pushed first; `openConfiguration` and `planInDelivery` stay until the browser half has moved. `CensusHandle` is `CensusPull`'s three methods pointing the other way. A host that opens a save file gets one and can hand it straight back to `compilePlan` as `options.census` — so ten million members are read out of the zip, checked, counted and written into a bundle without ever being a JavaScript value. `CensusCsvReader` is the same shape and stays, because a dropped CSV has no `Opened` to come from. That the two are interchangeable is the reason the interface was written before either of them. `takeCensus()` takes rather than borrows. A handle owns its cursor and two of them over one census would each believe they were at the start; the second call returns `undefined` instead of a second reader of the same rows. `saveFile` returns the zip. `files()` serialises in one go, unlike the census, because a logo and some photographs are values a host puts in state rather than a stream it pulls. Refs: meta#12769
…rt declares `check_sources` asks three questions of a plan and its files, and the first has no other home. **Two owners, one file name.** Bytes are keyed by name from the moment they leave the plan — `sources.files`, a save file's `files/` directory, `BuildOptions::images` — so two candidates whose photographs are both `photo.jpg` are not two photographs. One silently becomes the other's, and the wrong face is on a ballot. Nothing catches this today, because a plan carrying its bytes inline holds two different values under two identical names. The moment it stops, they are one, and the check has to exist before the field goes rather than after. **A file the plan names and nobody holds** is an error, said against the field somebody filled in. The build already says this — `material.file-missing`, `logo.file-missing` — at build time, in the workbook's vocabulary, about a spreadsheet column the author may never have seen. **A file nobody names** is a warning: dead weight in a delivery rather than a broken one, and usually a photograph whose candidate was renamed afterwards. And the cross-check, which is a different kind of thing and deliberately narrow. When an election-event archive carries both a realm and a voters CSV, the census's columns are compared with the user-profile attributes that realm declares. A column Keycloak has never heard of reaches nobody — the platform drops an attribute its profile does not declare, with no error and nothing in a log — so an export whose own realm does not declare its own census's columns is worth one sentence, said where somebody can still ask for a better export. Two artifacts that arrived together, compared where both are in hand. **Nothing is stored about what a census ought to contain**, so there is nothing to keep in step and nothing to be wrong. It is also asserted the other way: the check stays quiet on the wizard's own output, because `declare_census_attributes` puts every census column into the profile, and a warning that fired on every export this tool writes would be worth nothing. `sources::OWNED` is public now — it is the list of columns the platform owns, and the cross-check needs the same one the reader uses rather than a fourth copy. 931 tests. Refs: meta#12769
`read_plan` documented itself as "the only way a plan should be deserialized" and
had **no callers outside the tests**. `open`, `compilePlan`, `previewBallot`,
`applyProfile` and `step-cli` each reached for `serde_json` or
`serde_wasm_bindgen` directly, so `migrate_v1` and `migrate_v2` ran nowhere but in
a unit test.
That is a shipped defect, not a tidiness point. A version 2 plan keys its voters
by area *name* and the builder reads `area.external_id`; opened unmigrated, the
name lands in the identifier field and every voter's area dangles — in a plan that
opened without a word of complaint. `an_older_plan_is_migrated_on_the_way_in` is
the test.
`read_plan_value` is the funnel for callers who already have the document parsed:
the wasm boundary hands over a `JsValue` and `step-cli` a file it has read, and
neither should have to serialise back to text to be read properly. Both return
`ReadPlan { plan, sources }` — the sources derived from the plan's own fields
today, and lifted out of the JSON by the migration when those fields go. That is
the reason this is a commit of its own and comes before the removal: a caller
still holding its own `serde_json::from_str` would hand back a plan whose census
is silently empty, against a green suite.
`nothing_else_deserializes_a_plan` reads the sources rather than behaviour,
because the failure mode is a *new* call site and nothing at run time can notice a
migration that was skipped.
932 tests.
Refs: meta#12769Telling a save file from a delivery by "does it carry a `census.csv`" looked equivalent to telling them apart by the nested zip, and is not. A plan with no members writes an archive whose single member is `blueprint.json`, and that was read as a delivery — so the wizard's own save file was the one file it could not recognise, and only for the plans most likely to be saved early. Found by beyond's core contract check on the first plan it tried, which is the argument for that file existing. Refs: meta#12769
The wizard validates on every keystroke, and this is where a census that is no longer inside the plan has to arrive — otherwise duplicate usernames and dangling areas stop being reported while somebody is still in a position to fix them. Optional, like the other two, and a caller that passes nothing still means "read them off the plan". Refs: meta#12769
They reached one file and it was the wrong shape of place. `admin_portal/communication_templates.json` is a *loose* member of the delivery — `archive::admin_portal_member` does not match that name — so it sits outside `admin_portal_settings.zip` while the CSV of the same concept sits inside it. Two files, two places, one idea, and whoever loads them has to know that. `compile_plan` appends each message to `bundle.templates`, which is the list `export_templates-<tenant>.csv` and `templates/*.hbs` are both written from. The seam is deliberate: the *builder* still mints nothing from a message — `messages_leave_as_two_files_outside_the_bundle` still holds — and the wizard's compile adds them afterwards. **Not a Templates sheet, and the first attempt was.** A sheet has to be merged with the one a janitor's workbook carries, and the merged row then has to survive `a_plan_round_trips_to_the_same_workbook`: the same cells, in the same order, coerced the same way, with every `Translated` filled per language exactly as the Messages sheet fills it. It failed on all three, and for no gain — the delivery's spreadsheet already has a Messages tab, so the sheet would have said the same thing twice. Deleted rather than debugged. This does not put a template in the election-event import. `templates/` and `export_templates-` are both `admin_portal_member`, so they travel in the Portal's settings zip, which is what a tenant's templates are loaded through and what that invariant is really about. Asserted both ways. A template the workbook already carries under the same alias wins: two with one alias is what `build_templates` refuses outright, and a client's own wording is not a screen's to replace. **The loose JSON stays**, as asked — "also add". It is now a second copy of content that has a home, which is the duplication this programme removes everywhere else; retiring it is in the open list. 936 tests. Refs: meta#12769
…inst it (#2981) Parent issue: sequentech/meta#12769 **1 of 4**, base `main`. Stacked above: [#2982](#2982) → [#2983](#2983) → [#2988](#2988). **The only one of the four that changes existing behaviour.** The rest are additions. ## Why Three tools built or checked an election-event import and shared no code: windmill's importer, janitor's Python, the Election Architect's TypeScript. Each had its own bundle schema, CSV byte shapes and validation, and each got them slightly differently. Every defect found while building janitor was one implementation disagreeing with the platform. ## What changes | | | |---|---| | **Moved** | `ImportElectionEventSchema` and the report types → `sequent-core::election_config`. windmill re-exports both, so its ~10 call sites are untouched. `TryFrom<Row>` stays in windmill: it needs `tokio_postgres`, which cannot compile to WASM. | | **Reshaped** | `keycloak_event_realm` → opaque `serde_json::Value`; `tenant_id` → `String` with a shape check in validation. The `keycloak` crate would pull `reqwest`, `uuid` would pull `getrandom`. No field loses anything, and the export path still runs `parse_uuid_v4`. | | **New** | `election_config::validate` — pure checks returning structured `Problem`s, each with a machine-readable code, a dotted path and the entity's `external_id`. No database, no IO, no clock, because it has to run in a browser. | | **Wired** | windmill validates the bundle **as written**, before `replace_ids` rewrites identifiers, and refuses the import on anything fatal. The operator gets every problem at once, in the wording the browser-side tools use. | | **Example** | `validate_bundle` — the same call `step-cli` and the browser make, for checking an export from a shell. | ## The part that deserves scrutiny Wiring validation into the import path means **a bundle that used to import can now be refused** — including the platform's own export, re-imported for disaster recovery. Four rules were fatal and should not have been: a contest with no candidates, a contest not yet on any area's ballot, an area with no contests assigned, and — reported since — an event with no areas at all. An event still being configured legitimately looks like that; those bundles are self-consistent and import into a working event. They just mean nobody votes there yet. | | | |---|---| | **Error** | Internally inconsistent, or the platform would reject or corrupt it | | **Warning** | Consistent, but the configuration looks unintended | Authoring tools treat warnings as blocking through a strict mode rather than by lying about severity here. Pinned from both sides by `an_event_still_being_configured_can_be_re_imported` and `an_inconsistent_bundle_is_still_refused`. ## Verified - `sequent-core` — 219 passed at `keycloak,default_features`, in the devcontainer - `windmill` — 308 passed - `cargo fmt --check`, `reuse lint` clean - Against the generated SEIU1000 bundle: 0 errors, 1 warning — the permission label that made every election invisible on that event's first real import ## Reading it Four commits, one thing each: report types moved in → bundle schema moved out of windmill → the shared validation → windmill validating before importing. Then two from the review: the algorithm list and the preferential split now come from `CountingAlgType` instead of being written out beside it, negative vote counts are refused rather than passing every relational check, and a report's permission label reports `reports[].permission_label` instead of pointing at `elections`. Plus the two boundaries that were untested — an event with no areas, and a fatal bundle failing the import. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added comprehensive election bundle validation for required fields, references, identifiers, contest configuration, ballot coverage, permissions, and scheduling. * Added structured reports with categorized errors and warnings. * Added shared election import schemas, report definitions, scheduling, and encryption settings. * Added a command-line tool to validate bundles and return failure status when errors are found. * **Improvements** * Import processing now validates bundles and reports warnings before completion. * Tenant and realm data are preserved consistently during import and export. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…r editor (#3046) (#3080) Parent issue: sequentech/meta#12824 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added grouped user-profile fields with configurable headings, descriptions, ordering, responsive layouts, and tenant-specific styling. * Added support for checkbox-based multi-select profile attributes. * Added profile configuration retrieval for tenant and election-specific user management. * **Bug Fixes** * Improved attribute visibility and prevented duplicate area-name columns in user exports. * **Documentation** * Expanded administrator guidance for profile groups, selectors, styling, ordering, and customization. * **Tests** * Added coverage for grouping, field types, metadata, fixed fields, and export correctness. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Version 4. `plan.voters` is gone, and a plan describing ten million people is no longer ten million rows of JSON — saved, re-serialised on every compile, and copied across the wasm boundary each time, while the same rows also went into the workbook and into `export_voters-<id>.csv`. `migrate_v3` is the one migration that hands something back. `migrate_v1` and `migrate_v2` rewrite a document in place and that is the whole of their job; this one *removes* a field, and the rows in it are somebody's members. Dropping them would open a saved plan as an election with nobody in it — silently, against a file that still looks complete. So it lifts them out, `read_plan_value` puts them in `ReadPlan::sources`, and the caller hands that to `compile_plan` exactly as it would a census read from a file. It also **removes** the key, so nothing downstream can find a second copy, and it keys on version 3 so a stray `voters` in a current document is not silently adopted as a census the plan does not believe it has. Both are tested, as is the client's own column surviving the lift. The two other doors that carried a census now hand one back beside the plan: `plan_from_workbook::ReadPlan` gains `sources`, and `fill_from_archive` returns them. `voters_into` became `voters_from_csv`, which returns rows rather than writing them into a plan that has nowhere to put them. `voters` joins the control paths in `profile.rs`. A delivery profile hides the Census screen, and there is no `voters` in the shape of a plan any more — the honest field path would name a row of a file the plan does not contain. Refused in `defaults` like the rest, because a starting value for a census is not a thing. `Sources::from_plan` keeps only its files half. The three `bytes` fields stay for now and the reason is the sequencing rule: the browser reads a logo and a candidate's photograph off the plan, so removing them here would lose every photograph on reopening until beyond holds those beside the plan too. That is the next pair, and it is a much smaller one — a logo is kilobytes. Also fixes a feature gate `EA-F4-067` broke: `add_message_templates` carried `election_config_archive` while its caller is behind `election_config_templates`, so that combination alone would not compile. Every combination checks clean now — and checking only the three I had changed is how it got through. 938 tests, wasm builds, `cargo fmt --check` clean. Refs: meta#12769
Parent issue: sequentech/meta#12713 `main` counterpart of #2930 (`release/10.0`), cherry-picked from `abbfa30bc7`. `register.ftl` and `login.ftl` (both `sequent.admin-portal` and `sequent.voting-portal` themes) had drifted apart: `MultiAttributePasswordAuthenticator`'s `login.ftl` rendered its `matchAttributes` fields from a bespoke `{name,type}` list built in Java, so it was missing field-specific input types, User Profile helper text, select filtering, and the shared tel-input widget that `register.ftl` already had - plus every configured attribute was unconditionally mandatory to match, with no way to make some optional. ## Changes - Added `LoginBean` (mirrors Keycloak's own `RegisterBean`), exposing `profile.attributesByName` to `login.ftl` so its `matchAttributes` loop renders through the same `user-profile-commons.ftl` macros `register.ftl` uses - same field types, helper text, select filtering, tel-input handling. - Extracted the duplication this surfaced into shared macros under `sequent.admin-portal/login/`: `field-helper-text.ftl`, `tel-input-widget.ftl`, `select-filter-widget.ftl`, `social-providers.ftl` - now shared by `register.ftl` and both portals' `login.ftl`. - Removed a dead `mobile`-attribute special case in `register.ftl` (no realm ever configures a User Profile attribute literally named `mobile`). - Added an opt-in `honorUserProfileRequired` config property on `MultiAttributePasswordAuthenticator`: when enabled, a `matchAttributes` field's required-ness (asterisk, HTML5 `required`, and whether it's optional for matching) follows the realm's User Profile `required` setting for that attribute, instead of every configured attribute being unconditionally mandatory. Disabled by default, so existing realms are unaffected. The optional-attribute filtering lives entirely in the authenticator (`effectiveMatchAttributes`), so `MultiAttributeCredentialResolver` - shared with the IVR direct grant flow - is unchanged. ## Documentation Not in #2930 - added here: - New reference page **Configuring Login and Registration Fields** documenting how User Profile attributes and their annotations drive both the registration form and the attribute-based login form (input types, labels/translations, helper text, field limits, option lists and dependent dropdowns, phone-number fields, required-ness, hiding, login-hint prefilling). - Cross-linked from *Adding User Attributes to Keycloak* and *Logging In Without a Username*, and documented the new **Honor User Profile required attributes** setting in the latter's config steps. ## Conflict resolution vs #2930 `main` has moved on since `abbfa30bc7`. Resolved while cherry-picking: - `buildAttributeFields` (and its `inputTypeMax` → `max` forwarding, added on `main` by #3000) is deleted, as in #2930 - the `max` attribute and its `9999-12-31` default now come from `user-profile-commons.ftl`'s shared `inputTag` macro, which both forms go through. - Kept `TemplateSyntaxTest#multiAttributeDateInputsHonorConfiguredMaxInBothPortals` from #3000, rewritten against the new `profile.attributesByName` model so the four-digit-year regression stays covered on both portals. Its `renderLogin(portal, model)` helper is kept alongside #2930's new `profileWithAttributes` / `mockAttribute` helpers. - Dropped the four `buildAttributeFields_*` unit tests, whose subject no longer exists; the behaviour they covered is now asserted at the template level. ## Testing `mvn -pl sequent-theme test` - 28 tests pass. `message-otp-authenticator` tests were not run locally (its `action-token-login-bridge` dependency fails to build under JDK 21; this reproduces on a clean `main` and is unrelated to this change) - CI covers them on JDK 17. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Login and registration forms support configurable credential placement, validation modes, field types, labels, helper text, dropdowns, date inputs, and localized phone widgets. - Multi-attribute login can optionally follow profile-defined required fields. - Social sign-in providers display consistently, with policy-based digital certificate visibility. - Phone fields provide improved formatting and timezone-based country detection. - **Bug Fixes** - Improved focus behavior, autocomplete, accessibility, required-field handling, annotation support, fallback fields, and form layout. - **Documentation** - Expanded guidance for profile fields, annotations, requiredness, multi-attribute login, and certificate-login provider visibility. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Félix Robles <felix@sequentech.io>
Parent issue: sequentech/meta#12939 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Updated select option labels to display configured or translated descriptions directly. * Stored option values now appear only when no description is available. * **Tests** * Updated coverage to verify the revised label display behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…ifier and results-portal (#3075) (#3084) Parent issue: sequentech/meta#12862 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added portal-specific localization overrides for Voting, Ballot Verifier, Results, Admin, and global scopes. * Added scope selection, duplicate-key validation, legacy override compatibility, and clearer localization management. * Ballot Verifier now displays the correct published ballot style for the selected election event. * Results Portal applies election-event presentation translations from published data. * **Bug Fixes** * Improved translation cleanup when switching events, portals, languages, or leaving pages. * Prevented unrelated localization scopes from affecting Voting Portal date and time formats. * **Documentation** * Updated localization setup, override behavior, and language-extension guidance. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Parent issue: sequentech/meta#12891 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved election event prompt validation to catch invalid structures and missing language or key names. - Updated change detection to more accurately identify edits after prompt formatting or normalization. - Allowed valid empty prompt values where appropriate while continuing to require text-based entries. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
The bottom of this stack — "One definition of the import bundle" — is on main now, as a squash. That is why six files conflicted with no real disagreement in them: main has `problem`, `report`, `schema` and `validate` from that squash, and this branch has the same content plus everything the four commits above it added, so ours is a strict superset. Checked rather than assumed: `git diff ded5be6..origin/main` over `election_config` and `types/ceremonies.rs` is **empty**, so nothing on main has touched those files since the squash and there is nothing of theirs to lose. Taking ours for all six. 948 tests (main brings ten), step-cli green, the wasm feature builds, `cargo fmt --check` clean. Refs: meta#12769
edulix added a commit
that referenced
this pull request
Aug 24, 2026
A fast-forward would have left this branch pointing at a commit that already belongs to `architect-fields`, and GitHub attributes one check suite per commit — so the milestone pull request inherited #2988's suite and ran nothing of its own. `build_wasm` in particular never ran for this branch name, which is what beyond's end-to-end job resolves through `.github/step-branch` to fetch the WebAssembly it tests against. A merge commit gives the branch a commit of its own with the same tree, so the milestone is built and tested as itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…on ids EA-F4-077. Two defects an import found, both in what the importable zip says about voters. **An empty census is now no census.** `layout` wrote `export_voters-<id>.csv` unconditionally, and with no Voters sheet `build_voters` returns a table with no columns *and* no rows — so the member came out as a single newline: no header, no data. The platform's importer still reads it as a census and refuses the whole import, which made an election whose membership list has not arrived yet impossible to import at all. Now written only when there are voters, the same rule `reports` already followed. **A voter who names no election gets no election ids.** `voter_elections` expanded a blank or absent `authorized-election-ids` to *every* election in the event, on the reasoning that an empty attribute would deny access to all of them. Importing it says otherwise: the area already carries the voter's ballot, and an `AreaContests` row is what puts it in front of them. So the expansion wrote a restriction the census never expressed — which is what a client saw as an election id appearing against every voter they had not restricted — and it goes stale the moment an election is added. Blank now stays blank; a voter who names elections still has them resolved to ids, and naming one that does not exist is still refused. Two comments that claimed the old behaviour are corrected rather than left to mislead: `voters_sheet`'s "`authorized-election-ids` from the areas", and `build_voters`'s note on what a source cannot yet say. 950 tests pass and all seven election-config feature combinations compile. Verified in the wizard against a freshly built wasm: the sample builds a zip whose census member is present with an empty elections column, and the same plan with its two voters removed builds one with no census member at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EA-F4-079. `locked` requires a matching `defaults` key, for a good reason: a
lock with nothing to lock to fixes the field at whatever the plan happens to say.
There is one class of path where no value could ever be right, and it made
*fixed* unreachable for the two settings on the Areas screen a delivery profile
most wants to take away.
`defaults` holds one value per path and `apply_profile` writes it to every
element the path resolves to. One identifier shared by every area is a duplicate
by construction — `check_unique_identifiers` refuses the build it makes — and an
area's identifier is derived from its name anyway, one per area. So *fixed* here
means what a delivery engineer means by it: the client does not get to type one.
`parent_external_id` is the same read the other way round: locking **Inside**
says a client's districting is flat, and no value expresses that better than the
absence of one.
So `derives_its_own_value` names those two paths, exempts them from needing a
default, and — the half that matters more — **refuses them one**. `is_fixed`
writes a default unconditionally, so `{"areas[].external_id": ""}` would blank
every area's identifier on every compile and report "an area needs an
identifier" about a box the client cannot see. That is EA-F4-052 one rung
deeper, and this closes the door before anybody walks through it.
Additive: the rule accepts a profile it used to refuse, and refuses only a
profile nobody could have wanted. 953 tests pass, all feature combinations
compile, fmt and clippy clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>edulix added a commit
that referenced
this pull request
Aug 25, 2026
Two commits from `architect-fields`. Kept as a merge commit rather than a fast-forward for the reason the last one was: GitHub attributes one check suite per commit, so a tip shared with `architect-fields` leaves this branch's pull request inheriting #2988's suite and running nothing of its own — including `build_wasm`, which beyond's end-to-end job resolves through `.github/step-branch` to fetch the WebAssembly it tests against. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EA-F4-081. Reported as: download a delivery whose voters have areas, drop the zip back on Getting Started, and the voters arrive with no area. `RowShape::voter` and `cell_of` are meant to be each other's inverse — the comment above them says so, "the two together are the whole answer to what a census row means". They were not. `voter` has always read `area_name`, because that is the column the platform's own export writes; `cell_of` had no arm for it and fell through to `extra`, which is empty for a column a `PlannedVoter` owns. So a census whose header says `area_name` came out of `next_batch` with that cell blank. The CSV inside a delivery says exactly that, so reopening a delivery handed the wizard a census whose every area was empty — while the plan's own areas came back intact, which is what made it look like a wizard bug. The identifier is what goes back, not the display name: the identifier is what the voter holds and what `voter()` put there. `build_tables::voter_area_name` is still the one place that translates back, at the boundary that writes the platform's CSV — the only reader that wants a name. Pinned by a round-trip property test over three headers rather than by a case for `area_name`: read a row through a shape, write it back out, get the row. It fails with the old `cell_of` on the delivery's own header, and that is the check that would have caught this when the shape was introduced. 955 tests pass, every feature combination compiles, fmt and clippy clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
edulix added a commit
that referenced
this pull request
Aug 25, 2026
One commit: a census written back keeps its area. Kept as a merge commit rather than a fast-forward, because GitHub attributes one check suite per commit — a tip shared with `architect-fields` leaves this branch's pull request inheriting #2988's suite and running nothing of its own, `build_wasm` included, which is what beyond's end-to-end job resolves through `.github/step-branch`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EA-F4-082, the core half. A census has always been able to carry a `password` column — the platform's importer hashes it — but somebody had to produce ninety values, which in practice meant a spreadsheet formula nobody could reproduce a week later. **One random thing, and it is not the passwords.** `PasswordRecipe` carries a seed, and every password is HMAC-SHA256 of that seed over the voter's username. So a plan built twice gives the same passwords, a delivery reopened from its own zip regenerates exactly what was sent out, and a client who lost the CSV is handed it again rather than having every credential reissued. Regenerating the seed reissues all of them, which is what that button should mean. The seed comes from the wizard's `crypto.getRandomValues`, so no randomness enters `election_config` — three comments in this crate already explain why a `getrandom` in the WebAssembly build is a cost with no benefit, and a derivation wants a hash rather than an RNG. `sha2` and `hmac` are RustCrypto siblings of the `sha1` already here and already in the lockfile. Configurable: length, the four character classes, and whether to leave out the characters a person reads back wrong — `O`/`0`, `l`/`1`, `S`/`5`, `Z`/`2`, `B`/`8`. Each class is spelled as two constants, safe and confusable, and a test asserts the halves partition the class exactly. Characters are drawn by rejection sampling rather than `byte % len`, which would favour the first `256 % len` of the alphabet — invisible in one password and a real weakening across ninety thousand. A test measures the spread. The column is written by `voters_sheet` and carried to `export_voters-<id>.csv` by the passthrough that already existed, and only when the recipe can actually fill it: `get_copy_from_query` reads the *presence* of the header as "hash a password for each of these voters", so a column of blanks would issue every voter an empty credential. A census that already carries the column is refused rather than resolved — the client's value and a derived one are both credible and choosing silently would hand somebody the wrong credential. `shape_of_a_plan` fills the new `Option`, or every `passwords.*` path a profile names would "name nothing a plan has" — the trap every optional field on the plan has fallen into. 972 tests pass, every feature combination compiles, fmt and clippy clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EA-F4-082, the half I left out of the previous commit. Both crates were already in the lock as transitive dependencies, so this only adds sequent-core to their dependents — no new downloads, and `--locked` builds keep working. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…orted EA-F4-082, and the wizard's own validation found it: generate, export, drop the zip back on Getting Started, rebuild — the one path the whole feature exists for — was refused with "this census already carries a `password` column". The build writes the generated column into the census it exports, and that export is what comes back through the door. Read as an ordinary passthrough it landed in the reopened census, so the plan then held two answers to one question and `check_passwords` rightly refused it. So a column this plan's own recipe would write is dropped on the way in and derived again on the way out — the same rule `census_csv::DERIVED` already follows for `id` and the two flags the platform sets itself. A password a client *typed* has no recipe behind it and is untouched. `RowShape::ignoring` is the seam: it forgets a column from the lists while leaving every other cell's recorded position alone, because those positions are absolute into the raw row. `CsvCensus::ignoring` passes it through, and `open` asks for it only when the plan carries a recipe that is ready. The round trip is pinned end to end — build, open, rebuild, and the same two passwords come out. Asserted on the passwords rather than on the archive's bytes, which differ in ways that say nothing. 973 tests pass, fmt and clippy clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parent issue: https://github.com/sequentech/meta/issues/12769
4 of 4, on #2983 — this diff shows only what is added on top. Its beyond half is beyond#805.
What a tailored wizard needs from the platform: fields that survive being saved and reopened, the voter's own screens as reusable components so a preview cannot drift from the portal, and a client profile that can constrain any of it. Nothing here decides anything a front end could decide for itself — that is the rule the whole ticket runs on.
Plan and profile fields
Blueprint.votersPlannedVoterstays close to a row ofexport_voters.csvand keeps every column it does not recognise, so a census exported by one route imports through the other.Blueprint.auth_presetNoneleaves the environment alone.ClientProfile.presetsonly_our_presets; a profile that hides ours and offers none of its own is refused rather than leaving the client nothing to choose.ClientProfilepathsTallyPatchBehaviour::acceptsis the gate, and the reason presets live in Rust rather than as strings a browser renders: a client-facing button must not be able to select a behaviour no importer accepts. Two earlier versions of this tool shipped three such values between them — each an election that imports cleanly and then behaves in a way nobody chose. It is spelled by round-tripping through the patch's own deserializer rather than amatchover field names, because amatchwould be a second list of the policies.The workbook and the export both read back into a plan
plan_from_workbook): the authoring workbook is the whole configuration now, and the round trip is the test.plan_from_event): a.jsonor an export.zip, so an event configured by hand can be reopened in the wizard. Detection ordering is load-bearing and pinned by tests; the archive brings everything it carries, and the refusals are explicit rather than a silent partial read.step-cli step compile-planand/preview/filein the Voting Portal: the same core, reachable from a shell and from a browser.A round trip found two real defects that reading the code did not.
The voter's screens, lifted into
ui-essentialsThe wizard's Ballot Preview has to show what a voter will actually see. Copying the portal's markup into the wizard is how a preview comes to be a picture of something that no longer exists — so the portal's screens are now components the portal and the preview both render:
StartLayout·ElectionListLayout·BallotScreenLayout·ReviewLayout·ConfirmationLayout·SupportMaterialsLayout·BallotSteps·BallotActions·ReviewActions·ConfirmationActions·IvrCallTwo rules held throughout, and each was broken once and fixed:
packages/<package>/src/translations/<lng>.tson the same i18n paths. Clients depend on those keys; a lifted component reads the portal's catalogue rather than carrying English of its own.reviewScreen.titlechanges the portal and the picture of it through one string rather than two.The IVR emulator moves out of the Admin Portal for the same reason, and
EVotingPortalCountdownPolicyis exported fromui-core's pure entry, which no ballot build could previously find.The plan references its bulk data, it does not embed it
Blueprintembeds every kind of bulk data the delivery already carries in its own format, so aplan describing ten million voters would be ten million rows of JSON — serialised three times
into one delivery:
blueprint.json,election_workbook.xlsxandexport_voters-<id>.csv. Thisis the first half of removing that, and it is deliberately additive: nothing is deleted yet.
election_config::sourcesholds aCensusSourcetrait —columns(),rewind(),next_batch(size)— and aSources { census, files }beside the plan.validate_planandto_workbooktake it;check_censusandvoters_sheetwalk it a batch at a time.Sources::from_planderives it from the fields the plan still carries, so behaviour is identicaland the suite is unchanged.
Three details are load-bearing rather than incidental:
columns()answers before a row is read.build_realm::census_attributesdeclares oneKeycloak user-profile attribute per census column; a source that could only answer by reading
the whole census would not be answering.
name the realm's attributes, and to write the CSV — in three modules at three depths.
RefCellpanic inside wasm unwindsthrough the boundary and leaves a blank page.
RowShapecollapses three separate implementations of what a census row means — the droppedCSV, the CSV inside an export, and the workbook's Voters sheet — which had already drifted: only
one resolved
area_nameagainst the areas, and only one preferredarea.external_idwhere a rowcarried both.
Two defects came out of the rewrite.
check_censusmade a second pass over the whole census onlyto ask whether any voter named an area; one pass answers both. And the per-row half of the
no-area report had no test — only the aggregate did — so merging the passes could have dropped
every row-level error silently.
buildtakes the same&Sources, andcensus_attributesreadsCensusSource::columns()where abuild is handed one — finding 1 above, and the only consequence of moving the census that has no
symptom.
the_realm_declares_the_same_attributes_either_waybuilds one census twice, as the sheetand as a source, and compares what the realm declares.
build_votersdeliberately still reads the Voters sheet. A source yieldsPlannedVoter, whichcannot carry
enabled,email_verifiedorauthorized-election-ids; routing the builder throughone today would switch on a voter the census switched off and authorise a restricted voter for
every election in the event, silently. Nor would it save anything: the Voters tab stays in the
workbook, so the census is in memory at build time whatever that function reads. It is pinned by a
test in both directions rather than left as an intention.
The boundary then grows and loses nothing:
compilePlanandpreviewBallotacceptoptions.census— aCensusPull, three methods rather than an array — andoptions.files. A hostthat passes neither behaves exactly as before.
openFilereturns aCensusHandle, which isCensusPullpointing the other way, so a census read out of a save file can be handed straightback to
compilePlanwithout ever being a JavaScript value.openConfigurationandplanInDeliverystay until the browser half has moved.archive::save_filewrites<external_id>-plan.zip— plan, census,files/<name>. The wizardstops handing over a bare
blueprint.json, which is no longer the whole document. It stillopens, and always will.
Four defects this turned up, none of which had a symptom
blueprint.json,returned
Report::default()and stopped — fine while the JSON carries a second copy ofeverything, and nothing at all the moment it does not.
read_planhad no callers. It documented itself as the only way to deserialize a plan whilefive production call sites reached for
serde_jsondirectly, somigrate_v1andmigrate_v2ran nowhere but in a unit test. A version 2 plan opened in the wizard kept its area names in
the field the builder reads as identifiers — every voter's area dangling, silently.
photographs are both
photo.jpgare one photograph and the wrong face is on a ballot. Invisiblewhile the plan holds the bytes inline; certain the moment it does not.
sourceswas left ungated, so one feature combination did not compile.Plus the cross-check the owner asked for and no more than it: where an election-event archive
carries both a realm and a voters CSV, the census's columns are compared with the user-profile
attributes that realm declares. Two artifacts that arrived together, compared where both are in
hand — nothing is stored about what a census ought to contain.
check_sourcesasks what the plan names, what it was handed, and whether two owners share a filename — bytes travel keyed by name, so two candidates whose photographs are both
photo.jpgare onephotograph and the wrong face is on a ballot. And where an election-event archive carries both a
realm and a voters CSV, the census's columns are compared with the user-profile attributes that
realm declares: two artifacts that arrived together, compared where both are in hand, with nothing
stored about what a census ought to contain.
Voter Messaging's templates now reach
export_templates-<tenant>.csv, appended tobundle.templatesbycompile_planrather than written as a Templates sheet — a sheet would haveto merge with a janitor's own and then survive the workbook round trip, for no gain, since the
delivery's spreadsheet already has a Messages tab.
And then the field goes
BLUEPRINT_VERSIONis 4, andBlueprint::votersis gone.migrate_v3is the one migrationthat has to hand something back —
migrate_v1andmigrate_v2rewrite a document in place, andthis one removes a field whose rows are somebody's members, so dropping them would open a saved
plan as an election with nobody in it. It lifts them into
ReadPlan::sources, removes the keyso nothing downstream finds a second copy, and keys on version 3 so a stray
votersin a currentdocument is not silently adopted.
plan_from_workbook::ReadPlangainssourcesandfill_from_archivereturns them;votersjoins the control paths, because a delivery profilestill hides the Census screen and there is no such field to name any more.
Verified end to end in a browser against this core (beyond#805): the wizard's save file is
["blueprint.json", "census.csv"], the plan reads"version": 4and has novoterskey, andreopening restores every member.
The three
bytesfields stay for now, and the reason is the sequencing rule again: the browserreads a logo and a candidate's photograph off the plan, so removing them here would lose every
photograph on reopening until beyond holds those beside the plan too. That is the next pair, and a
much smaller one. The trait shape is chosen so streaming is a new impl rather than a second breaking change.
Also here
EA-F4-047,EA-F4-048).declare_census_attributesadds what the census carries and the profile lacks, andAuthPresetgainedprofile_attributesso the column chooser offers what this election's sign-in actually reads.EA-F2-025). The wizard has drawn a Withdrawn checkbox for a while;PlannedCandidatehad no such field and no flatten catch-all, so serde discarded every tick in silence — accepted on screen, absent from the delivery, gone on reopening. It isdisablednow, named for the platform's own field, whichcandidate.hbshas defaulted tofalseall along.getrandom0.4'swasm_jsturned on — the WASM is built from the lockfile, which is where the wasm pins live.Verified
sequent-corekeycloak,default_features,election_config_{xlsx,templates,archive}cargo check --libcleanstep-cliui-essentialsui-core44 ·voting-portal77 ·results-portal16cargo fmt --check,yarn lint(0 errors),yarn prettify,reuse lintall cleanRun Rust tests (velvet)fails on this branch and fails onmainthe same way — ninecli::test_allreport tests losing their headless-Chrome connection during PDF rendering, reproduced locally onorigin/mainwith no part of this stack applied. Nothing here touches velvet or the report path.messagesneeds nothing here, and that is worth writing downbeyond's
EA-F4-076lets a Client Profile hold the two voter emails — a starting value, fixed, orhidden — and it landed as a beyond-only commit. Checked against this branch before any of it was
written:
messagesis a real field ofBlueprint, soapply_profileseedsdefaults["messages"]andcovers()honourslocked: ["messages"]with no addition tonames_a_control.messages.invitation-to-voteandmessages.get-out-the-votestay where they are — innames_a_control, accepted inhidden, refused indefaultsandlocked. They name a control,not a plan field, so there is nothing for a default to be written into. That refusal is the reason
fixing is all-or-nothing on beyond's side and hiding remains per message; it is correct, and no
change is proposed for it.
Recorded so the asymmetry reads as a design decision rather than a gap, if someone later wonders why
one path in that list takes a default and its siblings do not.
EA-F4-077— an empty census is no census, and a voter who names no election gets no idsTwo defects an actual import found, both in what the importable zip says about voters.
An empty census is now no census.
archive::layoutwroteexport_voters-<id>.csvunconditionally, and with no Voters sheet
build_votersreturns aPlainTablewith no columns andno rows — so
plain_csvproduced a single newline: not a header row, one blank line. The platform'simporter still reads that as a census and refuses the whole import, which made an election whose
membership list has not arrived yet impossible to import at all. Now written only when
!bundle.voters.is_empty()— the rulereportsalready followed, andthe_reports_member_appears_only_when_there_are_reportsalready said why: an empty reports CSV isnot a valid one. The blank line has its own test so the reason stays a fact rather than a
recollection.
A voter who names no election gets no election ids.
voter_electionsexpanded a blank orabsent
authorized-election-idsto every election in the event. The old comment gave the reasoning —an empty attribute would deny access to all of them — and importing it says otherwise: the area
already carries the voter's ballot, and an
AreaContestsrow is what puts it in front of them. Theexpansion therefore wrote a restriction the census never expressed, which is what the owner saw as an
election id appearing against every voter they had never restricted, and it goes stale the moment an
election is added. Blank now stays blank; a voter who does name elections still has them resolved
to UUIDs, and naming one nothing configured is still refused with
DanglingReference.Worth knowing where this lands:
voters_sheetfiltersauthorized-election-idsout of the sheet itwrites, so a wizard-authored census always took the absent branch — every voter in every plan the
wizard built carried the full election list. One function, and it reaches the whole wizard path.
Two comments that claimed the old behaviour are corrected rather than left to mislead:
voters_sheet's "authorized-election-idsfrom the areas", andbuild_voters's note on what aCensusSourcecannot yet say — the hazard there is the same size but a different shape now, asource-derived voter losing a restriction rather than gaining every election.
AdminUsers is deliberately untouched.
sheet.rstreats the column as multi-valued on that sheettoo, but it is read by a different path and
admin_users.csvnever enters the importable zip. Theinstruction was about importing the census.
cargo test(keycloak + all three election-config features)cargo check --libcombinations cleancargo fmt --check, clippyexport_voters-<id>.csvwith an empty elections column, and the schedule; same plan with both voters removed → no census member; zero console errors in a clean tabmessagesand the ballot wildcards: still nothing needed herebeyond's
EA-F4-078makes a profile'selections[].name,elections[].description,elections[].contests[].nameandelections[].contests[].descriptionreach an election or contestadded after the plan was seeded. Also beyond-only, and the reason is worth recording against
this branch because it is a property of this core:
apply_profileresolves a[]path against whatever elements are present, so applying the profileto a plan holding one new row writes that row's defaults and nothing else. No new export, no change
to
Path::resolve.is_unsetalready reads a fresh{"en": ""}as nothing said — the recursive rule added whentranslated starting values were disappearing. A shallow
object.is_empty()here would have madethe whole fix impossible, so that comment is now load-bearing for a second caller.
Both are now pinned from the JavaScript side by
check-core-contract.mjs, which runs this branch'sWebAssembly: a wildcard default written into a one-row plan, and a name somebody already gave left
alone. Every Jest test fakes
applyProfile, so without that check the wizard's suite would staygreen against a core that ignored wildcards entirely.
EA-F4-079— a derived path may be locked without a valuelockedrequires a matchingdefaultskey, for a good reason written at the check: a lock withnothing to lock to fixes the field at whatever the plan happens to say. There is one class of path
where no value could ever be right, and requiring one made fixed unreachable for the two settings
on the Areas screen a delivery profile most wants to take away.
defaultsholds one value per path andapply_profilewrites it to every element the pathresolves to. One identifier shared by every area is a duplicate by construction —
check_unique_identifiersrefuses the build it makes — and an area's identifier is derived from itsname anyway, one per area.
parent_external_idis the same read the other way round: lockingInside says a client's districting is flat, and no value expresses that better than the absence
of one.
So
derives_its_own_valuenames those two paths and does two things:locked: ["areas[].external_id"]with an emptydefaultsnow reads; andis_fixedwrites a defaultunconditionally, so
{"areas[].external_id": ""}would blank every area's identifier on everycompile and report "an area needs an identifier" about a box the client cannot see. That is
EA-F4-052one rung deeper, and this closes the door before anybody walks through it.Additive, which the ordering rule requires: the change accepts a profile the core used to refuse,
and refuses only a profile nobody could have wanted.
cargo test(keycloak + all three election-config features)cargo check --libcleancargo fmt --check, clippycheck-core-contract.mjsasserts both halves against this branch's build — locking either path without a value, and a starting value for either being refusedEA-F4-081— a census written back keeps its areaReported as: download a delivery whose voters have areas correctly assigned, drop the zip back on the
wizard's Getting Started screen, and the voters arrive with no area.
RowShape::voterandcell_ofare meant to be each other's inverse — the comment above them saysso, "the two together are the whole answer to what a census row means". They were not.
voterhasalways read
area_name, because that is the column the platform's own export writes;cell_ofhad no arm for it and fell through to
extra, which is empty for a column aPlannedVoterowns.So a census whose header says
area_namecame out ofnext_batchwith that cell blank. The CSVinside a delivery says exactly that, so reopening a delivery handed the wizard a census whose every
area was empty — while the plan's own areas came back intact, which is what made it look like a
wizard defect rather than a boundary one.
The identifier goes back, not the display name: the identifier is what the voter holds and what
voter()put there.build_tables::voter_area_nameremains the one place that translates back, atthe boundary that writes the platform's CSV — the only reader that wants a name.
Pinned as a property, not a case.
a_row_read_through_a_shape_comes_back_out_of_it_unchangedreads a row through a shape and writes it back, over three headers — the delivery's, a workbook's,
and one with a passthrough column. Sabotage-confirmed: with the old
cell_ofit fails on thedelivery's header with exactly the reported symptom. That is the check that would have caught this
when
RowShapewas introduced to stop the three doors disagreeing.The wizard had the mirror image of the same defect in
fieldOfColumn(beyond); either fix alone still shows an empty area.
cargo test(keycloak + all three election-config features)cargo check --libcleancargo fmt --check, clippycompilePlanthenopenFilein Node: two voters in different areas come back asnorthandsouth, where before both were""EA-F4-082— passwords a delivery generates, from a seed it keepsA census has always been able to carry a
passwordcolumn — the platform's importer hashes it — butsomebody had to produce ninety values, which in practice meant a spreadsheet formula nobody could
reproduce a week later.
One random thing, and it is not the passwords.
PasswordRecipecarries a seed; each password isHMAC-SHA256(seed, "sequent-voter-password-v1\0" + username)mapped into the alphabet. A plan builttwice gives the same passwords, a delivery rebuilt from its own zip reproduces what was sent out, and
a new seed reissues everything — which is what that button should mean.
The seed comes from the wizard's
crypto.getRandomValues, so no randomness enterselection_config: three comments in this crate already explain that agetrandomin theWebAssembly build is a cost with no benefit, and a derivation wants a hash rather than an RNG.
sha2and
hmacare RustCrypto siblings of thesha1already here and already in the lockfile — no newdownloads, and
--lockedbuilds keep working.Details worth a reviewer's eye:
byte % len. The modulo favours the first256 % lencharacters —invisible in one password, a real weakening across ninety thousand. A test measures the spread.
test asserts the halves cover the class exactly once.
get_copy_from_queryreads the header'spresence as "hash a password for each of these voters", so a column of blanks would issue every
voter an empty credential.
choosing silently would hand somebody the wrong credential.
shape_of_a_planfills the newOption, or everypasswords.*path a profile names would "namenothing a plan has".
The round trip was broken and the validation above is what caught it. Generate, export, reopen,
rebuild — the one path this exists for — was refused with "this census already carries a
passwordcolumn", because the build writes the column into the census it exports and that export is what
comes back. A column this plan's own recipe would write is now dropped on the way in and derived
again on the way out, the same rule
census_csv::DERIVEDfollows forid.RowShape::ignoringisthe seam: it forgets a column from the lists while leaving the other cells' positions alone, because
those are absolute into the raw row. A password a client typed is untouched.
cargo testcargo check --libcleancargo fmt --check, clippycheck-core-contract.mjsasserts the column, two voters differing, the excluded characters absent, two builds identical and a new seed not