Add ECMA-335 EnC metadata delta writer

[NatElkins]

Summary

Adds an internal, standalone ECMA-335 Edit-and-Continue metadata delta writer to AbstractIL, with focused System.Reflection.Metadata-readable metadata tests. It has no behavior change and no in-tree semantic consumer yet; the consumer is the F# hot reload work in #19941. This follows the same upstreaming pattern as #20017 and #20018: land isolated infrastructure first, with deterministic byte and layout evidence, then wire the feature in a later PR.

What

Ten new internal AbstractIL files (file names kept from the development branch for diffability):

• ILDeltaHandles - offset/handle primitives, coded-index discriminated unions, delta tokens
• ILMetadataHeaps - heap sizes and offset bookkeeping
• IlxDeltaStreams - user-string and standalone-signature token calculators (continue numbering from a baseline)
• DeltaMetadataEncoding - ECMA-335 II.24.2.6 coded-index encoding
• DeltaMetadataTypes - the writer's input model: plain row-description records (method/parameter/property/event rows, method-semantics updates, custom attribute rows) and the definition key types
• DeltaMetadataTables / DeltaTableLayout / DeltaIndexSizing / DeltaMetadataSerializer - delta #- table stream, heap construction, layout and index sizing, EncLog/EncMap emission, generation GUID chaining
• FSharpDeltaMetadataWriter - the coordinating writer

The writer's inputs are row records (names, tokens, signature blobs, RVAs) plus baseline heap offsets and row counts. It imports nothing from any diffing, baseline, or session machinery; a grep gate for such imports is part of how the extraction was validated. One line of ilwrite.fsi exposes the pre-existing markerForUnicodeBytes so the delta writer reuses the full writer's exact string-marker logic.

Tests

130 tests in tests/FSharp.Compiler.Service.Tests/DeltaMetadata/:

• coded-index encodings asserted against the production Encoding.CodedIndices definitions, with ECMA-335 II.24.2.6 table order cross-checked against System.Reflection.Metadata's TableIndex as an independent reference;
• System.Reflection.Metadata reader parity over emitted deltas (SrmReaderParityTests);
• EncLog/EncMap correctness, stream layout, heap and index sizing;
• multi-generation chaining with generation-2 heap offsets asserted against computed expected values;
• standalone-signature delta rows asserted at baseline + 1 from a real seeded baseline;
• serializer failure paths.

All additions are internal; no public surface area change.

Sequencing

This PR is part of splitting the F# hot reload work (#19941) into small, independently reviewable PRs. The planned order:

1. Wave 1 (independent of each other, no cross dependencies):Add ResetCompilerGeneratedNameState to compiler-generated name generators #20017 (generated-name counter reset), Add Roslyn-format EnC CustomDebugInformation codec and portable PDB method CDI emission #20018 (EnC CustomDebugInformation codec and method CDI emission), Add ECMA-335 EnC metadata delta writer #20019 (EnC metadata delta writer), plus two upcoming slices: stable synthesized-name replay and the typed-tree differ with rude-edit classification.
2. Wave 2: baseline reading and recorded CDI state (depends on Add Roslyn-format EnC CustomDebugInformation codec and portable PDB method CDI emission #20018 and the naming slice).
3. Wave 3: the delta emitter and symbol matcher.
4. Wave 4: the hot reload session, FCS surface, and the --test:HotReloadDeltas capture hook (F# hot reload: Edit-and-Continue delta emission behind --test:HotReloadDeltas #19941 in its final, much smaller form).
5. Last, explicitly experimental:Add an experimental flag-gated in-process compile path for hot reload sessions #20031 (the flag-gated in-process compile perf path).

Each slice is rebased on current main, carries its own tests, and is flag-off zero-cost. Everything stays draft until reviewed in order.
Role in the train: the hot reload session emits its metadata deltas through this writer.

Refresh status (2026-07-17)

• Refreshed against dotnet/fsharp main at 5928e91; the current reviewed head is e7d79c0.
• A dedicated review pass completed for this PR, its findings were fixed in the lowest owning slice, and the PR has no unresolved review threads.
• The downstream session, in-process compiler, umbrella, and SDK branches were restacked after the fixes, so this slice remains part of the decomposed review train.
• The complete compiler stack passed the 11-step hot reload verifier, 456 service tests, 243 component tests with 2 expected skips, and 1411 EmittedIL tests with 3 expected skips. Replacement CI passed on this exact head.

Refresh status (2026-07-22)

• Current head: 33fceef, refreshed against dotnet/fsharp main 69fca7f.
• Stack position: independent wave-1 metadata-writer infrastructure; consumed by Add the hot reload delta emitter and symbol matcher #20027 and every later composite slice.
• Both July 21 review findings are fixed and resolved: one shared ECMA-335 #US encoder and cached trace configuration.
• Local verification: full Debug build passed with 0 warnings/errors and all 134 DeltaMetadata tests passed.
• Exact-head Azure compiler CI completed with 45/46 jobs passing. The sole failure occurred after clean repository builds when the policy-injected Component Detection task hit a Node/libuv agent assertion; a privileged rerun has been requested. The base-branch check_release_notes workflow still fails before analysis because pull_request_target refuses fork checkout.

Refresh status (2026-07-24)

• Current head: 06e0ad8a7c, with dotnet/fsharp main 1dc395ad34 merged.
• Stack position: independent metadata-writer slice consumed by Add the hot reload delta emitter and symbol matcher #20027 and every later composite.
• All review feedback remains addressed and every thread is resolved. The focused delta-metadata suite passed 134/134 before the content-neutral final main merge.
• The current FCS release note links this PR exactly once. A maintainer request is posted to remove the stale NO_RELEASE_NOTES label.
• Exact-head Azure build 1525314 completed with 45 of 46 jobs passing. The only failure was a hosted UseDotNet Node/libuv clock assertion before restore, compiler, build, or test execution; a privileged rerun is requested. The base-branch fork-checkout failure remains the documented Secure release-note checks for fork pull requests #20081 exception.

[github-actions]

❗ Release notes required

You can open this PR in browser to add release notes: open in github.dev

---

✅ Found changes and release notes in following paths:

Change path	Release notes path	Description
src/Compiler	docs/release-notes/.FSharp.Compiler.Service/11.0.100.md

[azure-pipelines]

Commenter does not have sufficient privileges for PR 20019 in repo dotnet/fsharp

[T-Gro]

🤖 This review was generated by AI (@expert-reviewer agent). Findings may contain inaccuracies — please verify independently.

Summary

Reviewed the ten new AbstractIL delta-writer sources and the two heap/exposure edits (skipped the ~5k lines of test files beyond spot-checking their intent). The ECMA-335 encoding is careful and internally consistent, and the design leans hard on System.Reflection.Metadata reader-parity tests, which is the right way to pin byte layout. I verified the pieces most prone to silent corruption and found them correct:

• Coded-index tag/table definitions (DeltaMetadataEncoding, ILDeltaHandles.*.CodedTag) match ECMA-335 II.24.2.6, and the sizing math (codedBigness limit = 2^(16-tagBits), simple-index >= 0x10000) is right.
• Fat method-body header (flags byte + 0x30 size nibble), and both small (12-byte clause / 1-byte DataSize / <=0xFF cutoff) and fat (24-byte clause / 3-byte DataSize) exception sections are encoded per II.25.4.
• Compressed-integer length prefixes (1/2/4-byte forms) and the metadata-root / #~ stream header layout match the spec, and Valid/Sorted 64-bit masks correctly cover the PDB tables at 0x30–0x37 (TableCount = 64).
• EncLog parent/member Add* pairing stays adjacent and EncMap is token-sorted.

No correctness, security, or error-handling blockers found. The two inline notes below are non-blocking maintainability/hygiene items.

[NatElkins]

The only Azure failure in this run was WindowsStrictIndentation. Both repository builds completed with zero warnings and zero errors, and the validation steps passed. The failure came afterward in the policy-injected Component Detection task when Node hit the libuv assertion new_time >= loop->time and exited with code 57005. All other 45 jobs passed. Could someone with pipeline permissions rerun the failed job?

[T-Gro]

Approving. Deep review clean: both prior findings fixed; shared #US encoder verified across empty/ascii/non-ascii/2- and 4-byte cases; determinism byte-identical; 134/134 delta-metadata tests pass. Adding NO_RELEASE_NOTES.

[T-Gro]

CI is red: fsharp-ci / Build WindowsStrictIndentation is failing — needs a green run before this can merge.

[T-Gro]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[T-Gro]

Re-approving — the dismissal was a content-neutral merge of main (#20018/#20024/#20033 landing); no source changes since my prior approval on 33fceef.

[NatElkins]

@T-Gro this PR now has a current FCS release note for the metadata delta writer in docs/release-notes/.FSharp.Compiler.Service/11.0.100.md, with exactly one link to #20019. I tried to remove NO_RELEASE_NOTES, but GitHub requires maintainer permission to remove repository labels. Could you remove that label when you get a chance? It is not the cause of the current checkout failure, but it is no longer appropriate for this PR.

[NatElkins]

The only Azure failure in build 1525314 is IcedTasks_Test_Debug Regression Test, and it failed before the compiler or IcedTasks build ran. The UseDotNet task installing SDK 8.0.x crashed inside the hosted agent's Node/libuv process with Assertion failed: new_time >= loop->time and exit code 57005. Every restore, compiler configuration, build, and test step was skipped.

@T-Gro could you rerun that failed job when you get a chance? The other exact-head jobs are still clean.

[T-Gro]

Re-approving after rebase (approval dismissed on force-push). Content unchanged; the only real-CI red is the known IcedTasks file-lock flake.

[T-Gro]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[github-actions]

🔍 Tooling Safety Check — Affects-Compiler-Output
Affects-Compiler-Output: adds ECMA-335 EnC metadata delta writer

Generated by PR Tooling Safety Check · opus46 12.1M · ◷

[T-Gro]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).