IL: intern the attributes and type references read from metadata - #20486
IL: intern the attributes and type references read from metadata#20486auduchinok wants to merge 2 commits into
Conversation
A metadata reader dedups a row within its own assembly, but the same value is read again in every assembly that mentions it, and readers are shared process-wide. Reading the 489 references of a ReSharper F# plugin project produces 22983 custom attributes that are 2896 distinct values, served by 338 distinct constructors, and 9350 type references that are 1186 distinct. Three capped tables in ilread keep one instance of each across readers. They are cleared from ClearAllILModuleReaderCache, so FSharpChecker.ClearCaches reaches them: an entry outlives the reader that produced it. Interning past the cap is skipped rather than evicting, since an unshared value is correct, just larger. The values pin nothing: a blob is copied out of the metadata view, so an interned attribute keeps neither the mapping nor its reader alive. Attribute constructors are interned inside the cached seekReadCustomAttrType rather than at the use site, so a constructor is already shared by the time the attribute carrying it is interned, and attributes then compare their constructor by reference instead of walking its method ref, argument types and assembly ref. fsc gets one call per constructor row from that cache; FCS disables it through reduceMemoryUsage, so there the interning runs per attribute row. Comparing attributes structurally instead cost small projects about 4.5% of their check time. Retained memory, under editor options. Diagnostics identical with the change off and on: consoleapp 1 proj 29.2 -> 28.0 MB -1.2 (-4.2%) Oxpecker 16 proj 130.9 -> 126.7 MB -4.3 (-3.3%) Prime 5 proj 105.5 -> 103.4 MB -2.1 (-1.9%) FsToolkit 11 proj 161.3 -> 158.4 MB -2.9 (-1.8%) IcedTasks 7 proj 96.6 -> 94.9 MB -1.7 (-1.7%) ReSharper.FSharp 10 proj 385.1 -> 378.4 MB -6.7 (-1.7%) Fantomas 8 proj 349.6 -> 346.8 MB -2.8 (-0.8%) FCS solution 14 proj 878.1 -> 876.4 MB -1.8 (-0.2%) Attributes carry two thirds to four fifths of that. Interning type references alone is worth -0.3 to -1.3 MB, and the two halves sum to the whole within 0.3 MB on every subject. Warm time is unchanged. On ReSharper.FSharp the timed region profiles at 27188 ms per iteration against 27186 and 27193 ms for two runs of the baseline, and the interning adds one frame costing 7.9 ms. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
|
||
| let elements = [] | ||
| ILAttribute.Encoded(method, data, elements) | ||
| internedAttributes.Intern(ILAttribute.Encoded(method, data, elements)) |
There was a problem hiding this comment.
🤖🕵️ [P2] Editing one assembly's imported attribute also changes another assembly's exported attribute. Keep mutable blobs reader-local.
open FSharp.Compiler.AbstractIL.IL
open FSharp.Compiler.AbstractIL.ILBinaryReader
let opts =
{ pdbDirPath = None; reduceMemoryUsage = ReduceMemoryFlag.Yes
metadataOnly = MetadataOnlyFlag.Yes; tryGetMetadataSnapshot = fun _ -> None }
let clsBlob name =
let sdk = @".dotnet\sdk\11.0.100-preview.6.26359.118\"
let reader = AssemblyReader.GetILModuleReader(sdk + name + ".dll", opts)
reader.ILModuleDef.Manifest.Value.CustomAttrs.AsArray()
|> Array.pick (function
| ILAttribute.Encoded(m, data, _)
when m.MethodRef.DeclaringTypeRef.FullName = "System.CLSCompliantAttribute" -> Some data
| _ -> None)
let a = clsBlob "Microsoft.Build"
let b = clsBlob "Microsoft.Build.Framework"
a[2] <- 0uy
printfn "B CLSCompliant = %b" (b[2] <> 0uy) // base: true; head: falseReproduced with exact base/head builds in all four reduceMemoryUsage × metadataOnly combinations. Exporting B with AssemblyBuilder.SetCustomAttribute changes its runtime CLSCompliantAttribute.IsCompliant from true to false too.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
🔍 Tooling Safety Check — Affects-Bootstrap, Affects-Compiler-Output
|
A metadata reader dedups a row within its own assembly, but the same value is read again in every
assembly that mentions it, and readers are shared process-wide. Reading the 489 references of a
ReSharper F# plugin project:
ilreadILAttribute(constructor + blob)ILMethodSpecILTypeRef(TypeRef table)Each occurrence rebuilt the chain behind one attribute: constructor, argument types, type reference,
scope reference, assembly reference, public key. Three capped tables in
ilreadnow keep one instanceof each across readers.
ClearAllILModuleReaderCache, soFSharpChecker.ClearCaches()reaches them: an entryoutlives the reader that produced it.
Cache<_, _>from illib does not fit: noClear, and adding one means touching the evictionmachinery shared with the type-subsumption and overload-resolution caches.
seekReadCustomAttrType, not at the use site, so aconstructor chain is hashed once per row per assembly instead of once per attribute; attributes then
compare their constructor by reference. Without both, small projects paid ~4.5% of their check time.
Retained memory
Whole solution checked and held at once, checker options as an editor passes them
(
keepAllBackgroundResolutionsandkeepAllBackgroundSymbolUsesoff).The saving is a roughly fixed 1-7 MB per configuration rather than a share of the heap, so it reads
largest where the heap is smallest.