Secure Boot enrolment for fleets: only Setup Mode + db scales (FOS cannot bootstrap its own trust)

[mastacontrola]

Rewritten 2026-08-03 after hardware testing. The original framing of this
issue was wrong in a way that mattered, and the correction is the most useful
thing to come out of it. See "What testing changed" below.

The gap

FOG generates a Secure Boot signing key by default and signs the FOS kernels
with it on every install and upgrade (_ensureSecureBootKeys / _resignKernels
/ _publishSecureBootKit). What it has no answer for is getting that
certificate trusted on more than a handful of machines. Both existing routes
need a technician with prepared media at each machine:

• the USB kit — make a stick, boot a stock Ubuntu/Debian live image, run
  fog-enroll-mok.sh, compare a fingerprint, reboot, answer MokManager
• PXE menu item 14 (_enrollSecureBootChoice) — chains MokManager directly,
  still needs MOK.der on local FAT media because MokManager has no network
  stack

The governing constraint

Whatever performs the enrolment must already be trusted by the firmware.
Every workable design is a different answer to "trusted by what?" Two
independent walls enforce this.

Wall 1 — MOK enrolment always requires a human. mokutil --import writes
MokNew, which is runtime-accessible. MokList — the store shim actually
consults — is boot-services-only, so the running OS physically cannot write
it. Only MokManager, in boot services, can promote one to the other, and it
demands a one-time password as proof of physical presence. That is shim's entire
security model; there is no bypass and there should not be one.

Wall 2 — FOS cannot bootstrap its own trust. This is what testing
established, and it invalidated the original plan. On real firmware with Secure
Boot enforcing and MokList empty:

NBP filename is secureboot/snponly-shimx64.efi <- MS-signed, accepted
Fetching Netboot Image secureboot/snponly.efi <- iPXE-signed, accepted
iPXE 2.0.0 ... boot.php... ok
bzImage... ok
Verification failed: Security Policy Violation
init.xz... ok
Verification failed: Security Policy Violation
Could not boot: Error 0x7f04819a

iPXE verifies the kernel and the initrd through shim. The signed chain
loads fine until FOG's own artefacts are checked against a MokList that does not
yet contain FOG's certificate, and both are refused.

So a FOG task running inside FOS can never be what establishes trust in FOG's
key. The task that would enrol the key cannot run on the machine that needs
it.

What testing changed

The original version of this issue proposed a task type as the fleet answer:
"Path B — staged MOK", pitched as replacing the USB-stick visit with a scheduled
task. That was built on the assumption FOS could boot on an enforcing machine.

It cannot. And notably 26 unit-test cases passed against that assumption and
still would — no harness can observe firmware policy. It took a real boot.

The task type still has a real, narrower use (below), but it is not the fleet
story, and the issue is rewritten so nobody builds on the old framing.

The three paths that survive, ranked

1. Setup Mode + db — the fleet answer. db/KEK/PK write policy follows
the presence of a PK, not the enforcement bit. In Setup Mode (PK absent)
nothing is enforcing, so FOS loads normally and can write FOG's certificate
straight into db — a plain efivarfs write, no signing tooling on the client.
No MokManager, no password, no media. Cost: one firmware visit to clear the
platform keys, which can be the same trip that enables Secure Boot. One BIOS
screen, once per machine, ever.

Secure Boot merely being switched off does NOT make db writable. A
machine in User Mode with enforcement disabled still has a PK. Only Setup Mode
helps. sbState() keeps setup and disabled as distinct answers for exactly
this reason.

2. Out-of-band (Redfish / vendor BMC) — the only zero-touch tier.
/redfish/v1/Systems/{id}/SecureBoot and its SecureBootDatabases collection
write db with the client powered off. No boot, no OS, no physical presence.
Dell cctk is the desktop equivalent. This is the only genuinely end-to-end
automated path and should be documented as a first-class tier, not a follow-on.

3. Borrowed trust (MS-signed live Linux) — the existing USB kit. Ubuntu live
boots under enforcing Secure Boot because Microsoft signed its shim. Still ends
at MokManager. Remains the answer for a machine that cannot be put into Setup
Mode — so it becomes more important, not less.

Where the staged-MOK task actually fits

Machines that currently have Secure Boot off and are going to have it turned
on. Common in practice — many sites disable Secure Boot precisely to use FOG
and want it back on afterwards. There, the task stages the key with no USB stick,
no live image and no fingerprint transcription, and the tech confirms once. A
genuine improvement on the USB kit for that case, and nothing more.

No in-code guard is needed for the enforcing case: the failure happens in iPXE,
before FOS exists. The task description states both limits so nobody schedules it
against 200 enforcing machines and watches them all fail to boot.

db baseline: ship Microsoft's certificates

Decided, implemented in Phase 2. KEK CA 2011, Windows Production PCA 2011, UEFI
CA 2011, plus the 2023 generation.

The decisive reason is FOG's own chain, not Windows compatibility. The chain
measured above is shimx64.efi → signed iPXE → FOG-signed kernel, and that shim
is signed by Microsoft Corporation UEFI CA 2011. A db without it breaks
FOG's own Secure Boot PXE boot.

Capturing each machine's factory keyset was rejected as the baseline — no answer
for the first machine of a model, an ordering trap the admin cannot recover from,
and restoring a stale dbx re-trusts revoked bootloaders. Kept as optional
enrichment for OEM-specific db entries (Dell SupportAssist, HP Sure Recover,
Lenovo diagnostics). dbx will not be baked into the shipped bundle.

Status

Phase 1 — done, validated end to end 2026-08-03.

• fstab mounted efivarfs as type efivars, which is not a mountable
  filesystem type (it was the old CONFIG_EFI_VARS sysfs interface). The
  entry silently failed under mount -a, so FOS has never had EFI variable
  access at all. Fixed.
• BR2_PACKAGE_MOKUTIL=y (+ KEYUTILS) on all three arches — mokutil is a
  stock Buildroot package.
• secureboot-funcs.sh, bin/fog.enrollsb, mode=enrollsb dispatch.
• taskTypes row 25, schema step 322.
• tests/checks/secureboot.sh — 26 cases, mutation-checked.
• Hardware validation: boot.php emits mode=enrollsb; FOS dispatches it;
  MokNew/MokAuth land in NVRAM with FOG's exact 810-byte DER at offset 44
  of 854; task reports Complete; shim shows "Shim UEFI key management" on the
  next boot.

Phase 2 — the actual fleet path, not started.

• fog-build-sb-authvars server helper (root-only, fog-sign-kernel sudo
  pattern).
• MS baseline keyset; db/KEK/PK write from FOS in Setup Mode.
• Refusal logic; Secure Boot page UI showing enrolment state.

Phase 3 — zero-touch and enrichment.

• Redfish / cctk for the firmware half.
• Capture-as-enrichment for OEM db entries.

Open

• Hardware validation of the db write. Not reversible from the OS —
  a mistake needs a firmware trip to recover, per model.
• 32-bit UEFI. No signed 32-bit shim exists, so the MOK path is moot on
  x86; the db path would still work. Decide refuse vs degrade.
• _keaSecureBootClassCommented() in fogproject says no signed snponly
  equivalent exists and that is why the DHCP Secure Boot class ships
  commented out. The fog-ipxe MANIFEST shows secureboot/snponly.efi is
  signed as of iPXE v2.0.0 (iPXE Secure Boot Automatic Code Signing G1A).
  That reasoning looks out of date and is worth revisiting separately.

Related

• fos ADR-0009 — docs/adr/0009-secure-boot-enrolment-paths.md
• fogproject ADR-0008 — task type scope and the db baseline decision

[mastacontrola]

Phase 2 is built: the Setup Mode path

The path this issue identified as the only one that scales is now implemented on
both sides.

Client (fos d57edbc) — fog.enrollsb picks its route from sbState(). In
Setup Mode it fetches the server's signed PK/KEK/db updates and writes them
straight into efivarfs, then finishes: no MokManager, no password, nobody at the
keyboard. Anything else stages a MOK request exactly as before. No new Buildroot
packages — coreutils, e2fsprogs and curl are already in every config, and
no signing tooling goes on the client.

Server (fogproject 75de023db) — the installer generates a PK and a KEK
alongside the existing MOK key, vendors Microsoft's published CAs under
packages/secureboot/mscerts/ with a MANIFEST, and fog-build-sb-authvars
produces db.auth (KEK-signed), KEK.auth (PK-signed) and PK.auth
(self-signed) beside MOK.der. efitools becomes an install dependency; where
it is absent the installer warns and skips, and the MOK paths are unaffected.

Four things that fail silently, so each is asserted rather than commented

• db lives under EFI_IMAGE_SECURITY_DATABASE_GUID, not the global GUID.
  Writing it to the global one creates a variable the firmware ignores — and
  efivarfs accepts the write.
• The attribute prefix is 0x27. Without TIME_BASED_AUTHENTICATED_WRITE_ACCESS
  the firmware stores the payload as data instead of applying it as an update.
• efivarfs requires the prefix and payload in a single write().
• PK is written last. Writing it is what leaves Setup Mode, after which
  every write must be signature-checked. PK first bounces the db and KEK
  writes and leaves a machine enforcing Secure Boot while trusting nothing —
  recoverable only at the firmware screen.

Per ADR-0003, dd's exit status is not taken as proof: sbEnrollDb() requires
SetupMode to flip 1 → 0, which is the firmware confirming it took the PK.
A failure stops before the PK write, so the machine is left as it was found,
and the message says so — "enrolment failed" otherwise reads like the machine may
now be unbootable.

One finding worth recording

cert-to-efi-sig-list reads PEM only, and on the DER that Microsoft publishes it
exits 0 after writing a 44-byte signature list containing no certificate. The
first run of the builder produced three well-formed .auth blobs with every
Microsoft CA silently missing from db. A client that enrolled that would have
lost Windows and FOG's own shim, needing a firmware trip per machine, with
nothing in any log to say why.

Guarded twice now: normalise every certificate to PEM whatever it arrived as, and
size-check the result (an ESL is exactly DER + 44 bytes) so a plausible-looking
empty list aborts the build. The test parses the finished db.auth back apart and
compares each embedded certificate by sha256, so shape alone cannot satisfy it.

Tests

tests/checks/secureboot.sh is now 40 cases; the 14 new ones were
mutation-checked against five deliberate breakages (wrong namespace, reversed
order, 0x07 attributes, no SetupMode re-read, no iflag=fullblock) and each
was caught. fogproject gains tests/secureboot-authvars.test.sh (7 cases,
mutation-checked, skips where efitools is absent).

What those cases prove is the bytes and the sequence. They cannot prove that a
given firmware accepts the update — that is a property of the machine, and this
issue already records one design that passed every test and was still wrong.

Status

• Phase 1 — staged MOK, validated end to end 2026-08-03
• Phase 2 — fog-build-sb-authvars, MS baseline keyset, db/KEK/PK
  write from FOS in Setup Mode, Secure Boot page UI
• Hardware validation of the db write. Still the open item, and not
  reversible from the OS — a mistake needs a firmware trip to recover, per
  model. Nothing here should be relied on until a real machine has been
  through it.
• Phase 3 — Redfish / cctk for the firmware half; capture-as-enrichment for
  OEM db entries
• 32-bit UEFI: no signed 32-bit shim exists, so the MOK path is moot on x86
  while the db path would still work. Refuse vs degrade still undecided.

[mastacontrola]

Merged, and validated end to end

• fos#135 — phases 1 + 2
• fos#136 — a bug the hardware test found (below)
• Secure Boot: build the signed variable updates, and make task type 25 do the enrolment (FOS GH-134) fogproject#990 — server side

The full path, on real firmware (VirtualBox 7.2 EFI, 2026-08-03)

Cleared the platform keys to enter Setup Mode, scheduled the task, and it enrolled with nobody at the keyboard. The firmware afterwards held exactly what it should:

Variable	Contents
db	Microsoft's 5 db CAs + FOG's signing certificate
KEK	Microsoft's 2 KEK CAs + this server's KEK
PK	this server's PK alone

Then Secure Boot was switched on, and the same machine PXE-booted FOG's signed chain:

bzImage... ok
init.xz... ok

Before enrolment, those two lines were Verification failed: Security Policy Violation. That is the whole feature in one diff.

What testing found that the tests could not

sbCertTrusted() grepped mokutil --db for the certificate's SHA-256. mokutil prints a SHA1 fingerprint there, so the match could never fire. On a machine already enrolled, the task therefore skipped its short-circuit, tried to stage a MOK, was refused by mokutil (which will not stage a request for a certificate already in db), and aborted. Re-running against an already-enrolled fleet would have failed on every host.

The harness stub for mokutil --db had been written to emit the SHA-256 the broken code was looking for. The stub and the bug were the same guess, so the case passed and always would have.

Fixed as a rule rather than a patch: answer from the data where a spec defines it, and from the tool only where the tool owns the format. db membership now searches the variable for the certificate's own bytes — the UEFI signature-list layout is published and cannot drift. MokList membership still uses mokutil, but against strings read out of the binary.

This is the second design in ADR-0009 that passed every test and was still wrong. Both were caught by booting a machine.

Still open

• Per-model validation on physical firmware. A VM is not a fleet. Writing PK/KEK/db is not reversible from the OS, so a model that rejects the update needs a firmware trip to recover.
• Phase 3 — Redfish / cctk for the firmware half; capture-as-enrichment for OEM db entries.
• 32-bit UEFI: refuse vs degrade.

[mastacontrola]

Closing this out

Everything this issue was opened to answer is answered, and everything it
scoped as buildable is built, merged and validated on firmware.

Phase 1 — staged MOK	fos#135, validated 2026-08-03
Phase 2 — Setup Mode db/KEK/PK	fos#135 + FOGProject/fogproject#990, validated 2026-08-03
db-membership detection bug	fos#136
Kernel lockdown groundwork	fos#131

The end state on real firmware: cleared to Setup Mode, task ran unattended,
db came back holding Microsoft's five CAs plus FOG's certificate, KEK
Microsoft's two plus this server's, PK this server's alone. Secure Boot
switched on afterwards and the machine PXE-booted FOG's signed chain —
bzImage... ok / init.xz... ok where it had been
Verification failed: Security Policy Violation.

The last unchecked item in the body — _keaSecureBootClassCommented() still
citing ipxe/ipxe#1776 as the reason no signed snponly equivalent existed — is
also resolved. FOGProject/fogproject#993 points both DHCP Secure Boot classes at
the snponly chain now that the premise is gone.

What is not being carried forward as a checkbox

Per-model validation on physical firmware. This can never be ticked — there
is no list of models that ends. It belongs where an admin will actually meet it,
and it is already there: the Secure Boot how-to in fog-docs states that per-model
validation on physical firmware is outstanding, that a mistake is not reversible
from the OS, and to treat the first machine of any model as a test. Leaving an
issue open for it would only make it look like work someone is doing.

Successors

The two genuinely open items are different work on different timescales, so they
get their own issues rather than keeping this one alive:

• fos#141 — Phase 3: zero-touch db enrolment via Redfish / vendor BMC,
  and capture-as-enrichment for OEM db entries.
• fos#140 — 32-bit UEFI. Decided as refuse, not degrade. The body above
  says "the db path would still work", which is true of the write and wrong
  about the outcome; the new issue explains why, and why the current absence of
  any architecture check is a false-success bug rather than a missing feature.

The thing worth keeping from this issue

Two designs here passed every test in the harness and were still wrong — the
original "run a task to enrol the key" framing, and sbCertTrusted() matching a
SHA-256 against mokutil's SHA-1 output, where the stub and the bug were the same
guess. Both were caught by booting a machine. That is recorded in ADR-0009 and is
the reason the db write is confirmed by re-reading SetupMode rather than by
dd's exit status.