IL: add ILPreNamespace, make ILPreTypeDef creation lazy by auduchinok · Pull Request #20092 · dotnet/fsharp

auduchinok · 2026-07-28T20:09:05Z

This PR groups lazy type definitions into lazy namespaces structure, allowing to make nested namespaces lazy and defer lazy type defs creation. It improves the custom module readers that we have in Rider.

In addition to that, the PR adds tests asserting the type defs are not eagerly read, which discovered several issues. This PR, #20088, and #20090 all fix these issues. The constraint fix has the most effect for the most users, since without it the type defs were effectively not lazy at all.

The following was measured with the other two PRs included.

Retained memory after ParseAndCheckProject (retain-project, avg of 3):

Project	main `8c0e444de`	branch + other PRs	vs main
Console — 168 refs / 3 src	77.7 MB	59.9 MB	−17.8 (−22.9 %)
FSharp.Common — 486 refs / 58 src	1319.2 MB	766.0 MB	−553.2 (−41.9 %)
FCS — 124 refs / 397 src	2308.7 MB	2289.2 MB	noise

Type defs actually read out of the references:

Project	top-level TypeDef rows	read before constraint fix	after	namespace levels realised
Console — 168 refs	3 905	3 905 / 3 905 (100 %)	2 280 / 3 905 (58 %)	428 → 300
FSharp.Common — 486 refs	106 980	107 356 (≈100 %) †	34 877 / 106 980 (33 %)	11 262 → 3 964
FCS — 124 refs	not instrumented	—	—	—

† Denominator is top-level rows only while the read counter also fires for nested type defs.

ParseAndCheckProject — warm (1 warm-up, then samples with the IL reader cache cleared each time):

Project	main: time (min / median)	branch: time (min / median)	main: allocated	branch: allocated
Console — 168 refs / 3 src	109 / 122 ms	95 / 110 ms (−13 % / −10 %)	110.2 MB	96.4 MB (−12.5 %)
FSharp.Common — 486 refs / 58 src	1832 / 2027 ms	1699 / 1968 ms (−7 % / −3 %)	3167 MB	2670 MB (−15.7 %)
FCS — 124 refs / 397 src	3060 ms (min)	2906 ms (min)	14 347 MB	14 340 MB (−0 %)

github-actions · 2026-07-30T09:21:24Z

🔍 Tooling Safety Check — Affects-Compiler-Output
Affects-Compiler-Output: modifies IL reader (il.fs, ilread.fs), import, and static linking

Generated by PR Tooling Safety Check · opus46 3M · ◷

auduchinok · 2026-07-30T13:11:59Z

This is ready.

T-Gro

🤖 This review was generated by AI (@expert-reviewer agent). Findings may contain inaccuracies — please verify independently.

Reviewed the lazy-namespace refactoring end to end (il.fs/il.fsi, ilread.fs, import.fs, illib.fs, and the order-sensitive consumers). No correctness defects found — the change is well engineered and the tradeoffs are documented in-line. Notes on what I checked:

Concurrency: DelayInitValue and ILTypeDefs.RealiseNamespaces are correct double-checked locking (volatile stores, non-cached exceptions). They lock this while DelayInitArrayMap locks a private syncObj, so the two paths don't interfere.
Enumeration order: AllPreTypeDefs regroups by namespace only for tables built via mkILTypeDefsGroupedComputed (the metadata reader). Flat tables (mkILTypeDefsFromArray, as used by FSI/ilreflect for generated modules) have no child pre-namespaces, so GetArray() order is preserved. The one read-order-sensitive consumer, StaticLinking, now sorts by MetadataIndex. ✓
ImportILAssemblyExportedType: moving FindByName inside the try that maps KeyNotFoundException to the friendly impReferenceToDllRequiredByAssembly error is a slight improvement over the old path (which let KeyNotFoundException escape), not a regression.
TypedTree.DemangledModuleOrNamespaceName and NameResolution reorderings are semantically equivalent to the originals while avoiding a forced read.
noTypars singleton: LazyWithContext.NotLazy [] is an already-forced, never-mutated record, so sharing one instance across all no-typar entities is safe.

One non-blocking readability note inline.

github-actions · 2026-08-12T10:11:48Z

❗ Release notes required

You can open this PR in browser to add release notes: open in github.dev

✅ Found changes and release notes in following paths:

Change path Release notes path Description
`src/Compiler` docs/release-notes/.FSharp.Compiler.Service/11.0.100.md

T-Gro · 2026-08-12T12:01:57Z

This looks great.
CI is red, pls ping for reapproval once it is fully green 👍

Hold the child namespaces + their by-name lookup in a single InterruptibleLazy<struct (ILPreNamespace[] * Dictionary<..>)> instead of two separate lazies, and share one pre-computed empty instance for namespace-less levels (every nested-type container - the vast majority of ILTypeDefs). On a single-file FCS check against a project with ~486 references this cut the namespace InterruptibleLazy wrappers from ~171,600 to ~8,800 and the retained IL-namespace machinery from ~16.5 MB to ~11.3 MB. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Follow-up cleanup on the ILPreNamespace work: - ImportILNamespaceLevel now buckets a level's flat entries and child pre-namespaces in one pass instead of via multisetDiscriminateAndMap - drop the now-unused readBlobHeapAsSplitTypeName/seekReadPreTypeDef and the mkILPreTypeDefEntry signature entry - TryFindPreTypeDef is a self-recursive member; simplify namespacesOf - flattening a grouped table (AsArray/AsList/the enumerator) restores the TypeDef row order, which static linking relies on to emit the types of a --standalone assembly in the reader's order - tests for laziness, lookup, hybrid levels, duplicate namespace nodes, imported entity order and metadata order of a read module Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Flattening a grouped table walks namespace by namespace, which is not the TypeDef row order when a namespace is split across the table. Doing the restoration inside ILTypeDefs cost a flag in the namespaces payload plus a sort in AllPreTypeDefs, in a memory-critical type; static linking is the only order-sensitive consumer, and one stable sort by MetadataIndex there covers it. Verified on a --standalone build: all 296 of FSharp.Core's top-level types are emitted in FSharp.Core's metadata order. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Temporary commit of the in-progress work on top of 9e331e8: - il.fs/il.fsi: one namespace-bucketing implementation (ILPreNamespaceOfEntries) reached both by the grouped reader and, via ilTypeDefsAsNamespaceLevel, by any flat table; DelayInitValue extracted into illib for the three hand-rolled lazies; fixed a race where AsArrayOfPreNamespaces could return an empty array. - import.fs: ImportILNamespaceLevel and its copy of the bucketing removed; ImportILTypeDefs walks the two arrays directly. Plus three unrelated memory tweaks (Nullness.GetFlags, shared noTypars, closure capture in ImportILTypeDef). - CompilerImports.fs: skip addConstraintSources for non-F# CCUs (also ported to its own branch off main, to be proposed and measured separately). - tests: namespace/cancellation tests updated, SurfaceArea baseline. - benchmarks: NamespaceImportBenchmarks with the retained-memory and retain-project probes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AddModuleOrNamespaceRefsToNameEnv runs over the root namespaces of every referenced assembly, and two checks in it forced each one's contents - which for an imported namespace means importing every type in it: - Entity.DemangledModuleOrNamespaceName forced the ModuleOrNamespaceType only to read its ModuleOrNamespaceKind. Only FSharpModuleWithSuffix demangles, and only a suffixed name can be of that kind, so testing the name first avoids the read. - `modref.IsModule && EntityHasWellKnownAttribute ... AutoOpenAttribute`: IsModule forces the contents while the attribute check only reads entity_attribs, so the operands are swapped. Retained memory when checking ReSharper.FSharp's FSharp.Common (486 references): 874.7 -> 780.4 MB (-10.8%). Framework-only projects are unchanged, since their root namespaces get imported anyway. Also un-skips `Type defs 02 - assembly import`: no type of the reference is imported now, closing dotnet#16166. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The reader tests above them pin the API in isolation; these pin the guarantee it exists for. A synthetic reference assembly records every GetTypeDef, member, nested-type and attribute read, and each test checks a file against it and asserts what came out - so a walk introduced far from the reader (addConstraintSources did exactly that) fails here. What they pin: naming Ns1.A reads the root level and Ns1's own types and nothing else; members, nested types and the base type stay unread until something needs them; a member lookup is what forces the base type's namespace; attributes are read for the types that enter scope, so an open pays for all of a namespace's types and a qualified use does not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

A level is now one shape everywhere: the types declared in it and its child namespaces, each half realised once and independently. Importing a level's types must not read its children, nor the other way round, and a child level needs no ILTypeDefs table of its own. ILPreNamespace becomes an abstract class carrying the name and caching both halves, rather than an interface handing back an ILTypeDefs. A store that knows its namespaces inherits it; ILTypeDefs is left for the two places a level is the declared type - a module's own level and a type's nested types - and its map is keyed by simple name again. That leaves mkILTypeDefsComputed flat, drops mkILTypeDefsAndNamespacesComputed for mkILTypeDefsOfNamespace, and turns mkILTypeDefsGroupedComputed into the one entry point for a store with no namespace structure to hand: namespaced entries plus any namespaces supplied directly, merged so a name coming from both becomes one child. The grouped reader no longer needs a maker to defer building a pre-type-def, because the pre-type-def is now the thing that defers: it holds the name index and reads the string heap through the store's getName. Grouping a table therefore reads namespaces only, and an un-imported namespace pays for neither its types' names nor their defs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

auduchinok · 2026-08-12T14:48:56Z

@T-Gro It's green 🙂

auduchinok requested a review from a team as a code owner July 28, 2026 20:09

github-project-automationBot added this to F# Compiler and Tooling Jul 28, 2026

github-project-automationBot moved this to New in F# Compiler and Tooling Jul 28, 2026

github-actionsBot added the AI-Tooling-Check-Scanned-Clean Tooling check: diff analyzed, no interesting infrastructure files label Jul 28, 2026

auduchinok force-pushed the il-pre-namespace branch from 5ddd1ca to 28c10e2Compare July 30, 2026 08:54

github-actionsBot added the ⚠️ Affects-Compiler-Output Tooling check: PR touches IL emission or codegen label Jul 30, 2026

auduchinok force-pushed the il-pre-namespace branch from 28c10e2 to 2e7cbc0Compare July 30, 2026 10:36

auduchinok force-pushed the il-pre-namespace branch from 2e7cbc0 to 8029661Compare July 30, 2026 13:16

T-Gro reviewed Aug 3, 2026
View reviewed changes

Comment threadsrc/Compiler/Driver/CompilerImports.fs

T-Gro added the AI-reviewed PR reviewed by AI review council label Aug 3, 2026

T-Gro self-requested a review August 3, 2026 19:20

auduchinok force-pushed the il-pre-namespace branch from 8029661 to 7a92a6dCompare August 12, 2026 10:11

T-Gro approved these changes Aug 12, 2026
View reviewed changes

github-project-automationBot moved this from New to In Progress in F# Compiler and Tooling Aug 12, 2026

auduchinok force-pushed the il-pre-namespace branch from f6e4a9d to 0beabc7Compare August 12, 2026 12:37

auduchinokand others added 11 commits August 12, 2026 15:23

IL: add ILPreNamespace, make ILPreTypeDef creation lazy
54d8880

Release notes
b421408

Cleanup comments
ea64a16

Fantomas
c23f792

Fix
6d95690

auduchinok force-pushed the il-pre-namespace branch from 0beabc7 to 6d95690Compare August 12, 2026 13:23

T-Gro approved these changes Aug 12, 2026
View reviewed changes

T-Gro merged commit d3fab44 into dotnet:mainAug 12, 2026
49 checks passed

dotnet-maestroBot mentioned this pull request Aug 13, 2026
[main] Source code updates from dotnet/fsharp dotnet/dotnet#8269
Merged

auduchinok deleted the il-pre-namespace branch August 13, 2026 10:30

dotnet-bot mentioned this pull request Aug 18, 2026
[Automated] PRs inserted in VS build main-12117.157 #20279
Closed

Navigation Menu

Uh oh!

IL: add ILPreNamespace, make ILPreTypeDef creation lazy - #20092

IL: add ILPreNamespace, make ILPreTypeDef creation lazy#20092
T-Gro merged 12 commits into
dotnet:mainfrom
auduchinok:il-pre-namespace

auduchinok commented Jul 28, 2026•
edited
Loading

Uh oh!

github-actionsBot commented Jul 30, 2026

Uh oh!

auduchinok commented Jul 30, 2026

Uh oh!

T-Gro left a comment

Uh oh!

Uh oh!

github-actionsBot commented Aug 12, 2026

Uh oh!

T-Gro commented Aug 12, 2026

Uh oh!

auduchinok commented Aug 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

auduchinok commented Jul 28, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Jul 30, 2026

Uh oh!

auduchinok commented Jul 30, 2026

Uh oh!

T-Gro left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actionsBot commented Aug 12, 2026

❗ Release notes required

Uh oh!

T-Gro commented Aug 12, 2026

Uh oh!

auduchinok commented Aug 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

auduchinok commented Jul 28, 2026•
edited
Loading