Add CoreCLR WASM R2R performance lane by lewing · Pull Request #5297 · dotnet/performance

lewing · 2026-09-02T02:39:05Z

Summary

add a dedicated CoreCLR browser-WASM R2R microbenchmark lane on V8
propagate the R2R selection through Helix and the microbenchmark harness
publish per-assembly WASM R2R with a distinct R2RType=r2r result dimension
resolve the CoreCLR browser runtime pack in shared WASM pack overrides
fail fast if CoreCLR/R2R/webcil/runtime-pack settings are not applied

Dependency

Depends on dotnet/runtime#133040 and finally dotnet/runtime#133203, which stages the matching host Crossgen2 pack in BrowserWasmCoreCLR.

Validation

focused Python tests for argument validation, environment propagation, Helix forwarding, and result dimensions
direct MSBuild evaluation of R2R and runtime-pack properties
fail-fast MSBuild validation for invalid Mono/R2R configuration
XML and YAML parsing

Remaining validation

Run runtime-wasm-perf with this branch and the runtime artifact together, confirm CreateReadyToRunImages executes, and verify uploaded results are labeled R2RType=r2r.

Copilot

🔵 Needs a closer look

It changes CI/Helix orchestration and MSBuild/runtime-pack resolution behavior where correctness is environment-dependent and should be validated by a human with end-to-end runs.

Pull request overview

Adds a dedicated CoreCLR browser-WASM ReadyToRun (R2R) microbenchmark lane (V8) and wires the R2R selection through the performance repo’s Helix/job plumbing and MSBuild configuration, including validation to fail fast when required settings aren’t applied.

Changes:

Adds CoreCLR-vs-Mono runtime-pack override logic for WASM builds and validates the expected CoreCLR runtime pack for R2R runs.
Introduces --wasm-ready-to-run end-to-end (CLI arg → env var → MSBuild properties → Helix forwarding) and emits a distinct R2RType=r2r run dimension.
Adds a non-release-branch pipeline lane for CoreCLR WASM R2R microbenchmarks on V8 plus focused pytest coverage.

File summaries

File	Description
src/scenarios/build-common/WasmOverridePacks.targets	Chooses CoreCLR vs Mono WASM runtime-pack IDs based on `UseMonoRuntime`.
src/benchmarks/micro/MicroBenchmarks.Wasm.targets	Enables/validates CoreCLR WASM R2R publish properties and validates runtime-pack selection.
scripts/tests/test_wasm_coreclr_r2r.py	Adds pytest coverage for argument validation, env propagation, Helix forwarding, and result dimensions.
scripts/run_performance_job.py	Forwards `--wasm-ready-to-run` to Helix work items and adds `R2RType=r2r` dimension.
scripts/micro_benchmarks.py	Adds `--wasm-ready-to-run` and propagates it via `PERFLAB_WASM_READY_TO_RUN`.
eng/pipelines/runtime-wasm-perf-jobs.yml	Adds a CoreCLR WASM R2R microbenchmark lane (non-release branches), V8 on linux_x64.

Review details

Files reviewed: 6/6 changed files
Comments generated: 0
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

lewing · 2026-09-02T03:29:20Z

dotnet/runtime#133040 is the runtime side

Copilot

🔵 Needs a closer look

End-to-end validation remains pending and the required runtime dependency is not yet merged.

Review details

Files reviewed: 6/6 changed files
Comments generated: 0 new
Review effort level: Balanced

lewing · 2026-09-02T19:01:55Z

test run at https://dev.azure.com/dnceng-public/public/_build/results?buildId=1579394

lewing · 2026-09-02T20:21:47Z

Pushed c10ff6c to fix the first end-to-end validation failure from runtime-wasm-perf build 1579394. The runtime artifact correctly contained the local WebAssembly SDK and Crossgen2 packages at 11.0.0-ci, but generated benchmark restore requested the runtime product version (11.0.0-rc.1.26431.109). The payload now detects the shared local toolchain package version, exports it to Helix, and applies it to both KnownWebAssemblySdkPack and KnownCrossgen2Pack. Focused tests cover detection, mismatch rejection, propagation, and MSBuild properties. A new pipeline run is required.

Copilot

🔵 Needs a closer look

The environment-dependent MSBuild and Helix integration still requires the stated end-to-end validation after the latest package-version fix.

Review details

Files reviewed: 7/7 changed files
Comments generated: 0 new
Review effort level: Balanced

lewing · 2026-09-02T21:29:02Z

https://dev.azure.com/dnceng-public/public/_build/results?buildId=1579662

lewing · 2026-09-02T21:30:41Z

is there a better (non-workload) way to use the crossgen from the build?

lewing · 2026-09-02T23:28:11Z

Build 1579662 was green but did not actually compile the benchmark app as R2R: the inner-TFM restore evaluation reset KnownWebAssemblySdkPack to the product version, restored feed package .118, and therefore never imported the new CoreCLR R2R targets. The publish binlog showed CreateReadyToRunImages completing in 1 ms with no _ReadyToRunFilesToPublish. Pushed a fix that reapplies both local toolchain versions immediately before ProcessFrameworkReferences, following the runtime WASM test pattern, and now fails the build if _CreateR2RImages produces no outputs. Another validation run is required.

Copilot

🔵 Needs a closer look

The configuration validation runs after publishing rather than failing before build work begins.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/benchmarks/micro/MicroBenchmarks.Wasm.targets:21

PrepareForWasmBuild is a BenchmarkDotNet-generated target that runs AfterTargets="Publish", so this validation occurs only after the full publish/R2R work has already completed. Invalid Mono/R2R or package settings therefore do not fail fast as described; hook this validation before framework-pack resolution (or another pre-build target) instead.

Files reviewed: 7/7 changed files
Comments generated: 0 new
Review effort level: Balanced

lewing · 2026-09-02T23:36:55Z

https://dev.azure.com/dnceng-public/public/_build/results?buildId=1579905&view=results

lewing · 2026-09-03T01:47:08Z

the crossgen we're building for the pipeline had a newer glibc because it wasn't crossbuilt dotnet/runtime#133143 it the fix for that and I will rereun the test once it is in.

Copilot

🟡 Changes recommended

Payload validation rejects the actual runtime artifact and regresses non-R2R CoreCLR WASM runs.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

scripts/build_runtime_payload.py:412

The linked BrowserWasmCoreCLR producer stages only the WebAssembly SDK/ref packages and the newly added Crossgen2 package in staging/built-nugets; it does not stage a Microsoft.NET.ILLink.Tasks nupkg. This check therefore raises “found 0” for every CoreCLR WASM job before Helix submission, including the existing non-R2R lane. Either stage the matching ILLink package in the runtime artifact or validate/use the ILLink pack already installed in dotnet-none instead.

    illink_packages = [
        package for package in Path(built_nugets_dir).glob("Microsoft.NET.ILLink.Tasks.*.nupkg")
        if not package.name.endswith(".symbols.nupkg")
    ]
    if len(illink_packages) != 1:
        raise ValueError(
            f"Expected one ILLink package in {built_nugets_dir}, found {len(illink_packages)}")

Files reviewed: 7/7 changed files
Comments generated: 2
Review effort level: Balanced

Copilot

🔵 Needs a closer look

Approval depends on coordinated runtime artifacts and final end-to-end validation of the cross-repository toolchain path.

Review details

Files reviewed: 7/7 changed files
Comments generated: 0 new
Review effort level: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Copilot

🟡 Changes recommended

The new package-version requirement breaks an existing wasm_coreclr unit-test path.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 7/7 changed files
Comments generated: 1
Review effort level: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Copilot

🔵 Needs a closer look

The cross-repository runtime artifact dependency and end-to-end R2R publishing behavior require final human validation.

Review details

Files reviewed: 8/8 changed files
Comments generated: 0 new
Review effort level: Balanced

maraf

Beside the amount of places that hardcode 11.* it looks good to me

radekdoulik · 2026-09-06T18:04:10Z

The PR description should also mention dependency on runtime's #133203

lewing · 2026-09-06T18:43:19Z

The PR description should also mention dependency on runtime's #133203

@radekdoulik fixed

lewing · 2026-09-06T18:47:51Z

Beside the amount of places that hardcode 11.* it looks good to me

This is all just test related to verify the different version formats all work so you can run locally, against unmerged prs, or against versioned builds.

…ort (#133203) ## Summary Stage the complete same-build CoreCLR browser-WASM package cohort consumed by the performance pipeline. This fixes two failures: - The Mono artifact job now downloads both the CoreCLR browser runtime pack and Crossgen2 pack before installing the wasm-tools workload. In runtime-wasm-perf build [1584559](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1584559), workload installation failed because `Microsoft.NETCore.App.Crossgen2.linux-x64` version `11.0.0-ci` was absent from the local feed. - The CoreCLR R2R artifact now includes the locally built `Microsoft.NET.ILLink.Tasks` package. In build [1581666](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1581666), restore requested ILLink product version `.109`, the artifact did not contain it, NuGet floated to `.118`, and `_RunILLink` then failed because the payload only contained the `.109` runtime. - Use the canonical SDK R2R output directory spelling when staging trimmed CoreCLR browser-WASM images for WebCIL conversion. Add a focused target evaluation test that compares the path ordinally so case-insensitive hosts catch regressions too. The existing CoreCLR-only Crossgen2 staging switch becomes one toolchain/cohort staging switch that: - packs `ILLink.Tasks.csproj` without rebuilding or restoring, because the normal `clr+libs+host+packs` build compiles the project but does not pack it; - requires exactly one non-symbol package for each local package ID; - rejects packages whose versions do not match the local WebAssembly SDK cohort; - publishes the CoreCLR runtime and Crossgen2 packs for the dependent Mono artifact job; - stages those packages into the Mono job's local feed before workload installation. This completes the runtime artifact consumed by dotnet/performance#5297 and repairs the prerequisite artifact build used by the existing Mono WASM performance lane. ## Complete staged cohort The CoreCLR performance artifact now carries the complete same-build publish inputs: - SDK: `staging/dotnet-none` - CoreCLR browser runtime pack: `staging/microsoft.netcore.app.runtime.browser-wasm` - `Microsoft.NETCore.App.Ref` - `Microsoft.NET.Sdk.WebAssembly.Pack` - `Microsoft.NETCore.App.Crossgen2.linux-x64` - `Microsoft.NET.ILLink.Tasks` The dependent Mono artifact job also receives the same-build CoreCLR browser runtime and Crossgen2 packages before installing its workload. WebAssembly SDK and ILLink are direct SDK tool restore dependencies, while Crossgen2 and the runtime/ref packs are selected through the SDK framework/tool-pack graph. ILLink's private dependencies are included in its tooling package, so no additional locally built SDK tool package is required. The performance-side override aligns `KnownWebAssemblySdkPack`, `KnownCrossgen2Pack`, and `KnownILLinkPack` to the staged local package version. ## Validation - Parsed both changed Azure Pipelines YAML files. - Simulated the extracted staging script with matching non-symbol, symbol, and unrelated nupkgs; only the required non-symbol cohort packages were selected. - Simulated an absent package and a mismatched package version; both failed with explicit diagnostics. - Audited build 1584559 and confirmed the Mono artifact job failed specifically because `Microsoft.NETCore.App.Crossgen2.linux-x64` version `11.0.0-ci` was missing from the local package feed; this change publishes and downloads that exact package from the CoreCLR build job. - Audited build 1581666's restore/publish binlogs and CoreCLR build artifact. This confirmed the NU1603 fallback, the `.118` ILLink/runtime mismatch, the package graph above, and that the regular runtime build compiles but does not pack `ILLink.Tasks.csproj`. - Ran `git diff --check` and an independent focused review of the final pipeline diff. ## Remaining end-to-end validation Queue `runtime-wasm-perf` pipeline 156 on Ubuntu 22.04 with repository resource overrides: ```json { "resources": { "repositories": { "self": { "refName": "refs/heads/lewing-stage-wasm-r2r-tools" }, "performance": { "refName": "refs/heads/lewing-wasm-coreclr-r2r-perf" } } } } ``` The Mono artifact job must install the workload from the local feed without a missing Crossgen2 package. The CoreCLR R2R lane must restore without NU1603/fallback, complete ILLink, emit non-empty per-app R2R images through Crossgen2, stage WebCIL, and start the benchmark under V8. Existing failed builds cannot provide that proof because their published artifacts predate the complete cohort staged by this change. > [!NOTE] > This pull request description was generated with the assistance of GitHub Copilot. --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 26a97530-a6d9-44e6-8754-2783961f1313

radekdoulik

Besides the imports nit it LGTM

Copilot AI lite review requested due to automatic review settings September 2, 2026 02:39

Copilot started reviewing on behalf of lewing September 2, 2026 02:39 View session

Copilot AI reviewed Sep 2, 2026

View reviewed changes

lewing mentioned this pull request Sep 2, 2026

[perf][wasm] Stage Crossgen2 pack for CoreCLR R2R benchmarks dotnet/runtime#133040

Merged

lewing requested a balanced review from Copilot September 2, 2026 03:29

Copilot started reviewing on behalf of lewing September 2, 2026 03:30 View session

Copilot AI reviewed Sep 2, 2026

View reviewed changes

lewing marked this pull request as ready for review September 2, 2026 15:07

Copilot AI review requested due to automatic review settings September 2, 2026 20:21

Copilot started reviewing on behalf of lewing September 2, 2026 20:22 View session

Copilot AI reviewed Sep 2, 2026

View reviewed changes

lewing marked this pull request as draft September 2, 2026 21:29

lewing requested a review from akoeplinger September 2, 2026 21:29

lewing marked this pull request as ready for review September 2, 2026 22:58

lewing requested review from DrewScoggins, LoopedBard3, ilonatommy and pavelsavara September 2, 2026 22:58

Copilot AI review requested due to automatic review settings September 2, 2026 23:28

Copilot started reviewing on behalf of lewing September 2, 2026 23:28 View session

Copilot AI reviewed Sep 2, 2026

View reviewed changes

lewing mentioned this pull request Sep 3, 2026

Cross-build browser WASM performance host tools dotnet/runtime#133143

Merged

Copilot started reviewing on behalf of lewing September 5, 2026 01:30 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

Comment thread scripts/build_runtime_payload.py

Comment thread src/benchmarks/micro/MicroBenchmarks.Wasm.targets Outdated

Copilot AI review requested due to automatic review settings September 5, 2026 01:43

Copilot started reviewing on behalf of lewing September 5, 2026 01:44 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

lewing added 8 commits September 4, 2026 21:13

Add CoreCLR WASM R2R performance lane

33f114e

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Align CoreCLR WASM toolchain package versions

7864135

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Ensure CoreCLR WASM R2R images are produced

04b0e43

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Use trimmed staging for WASM R2R benchmarks

e51c718

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Align ILLink with the WASM toolchain cohort

2f741ee

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Support regular SDK WASM R2R builds

22f1217

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Exclude invalid Jil WASM R2R output

d3a58c4

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Validate resolved WASM runtime packs

0790cb8

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Copilot AI review requested due to automatic review settings September 5, 2026 02:13

lewing force-pushed the lewing-wasm-coreclr-r2r-perf branch from c51c6c4 to 0790cb8 Compare September 5, 2026 02:13

Copilot started reviewing on behalf of lewing September 5, 2026 02:14 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

Comment thread scripts/run_performance_job.py

Update CoreCLR WASM prerequisite test

e496b5d

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f286edf-0a3c-43dc-92fe-26a0a57d5360

Copilot AI review requested due to automatic review settings September 5, 2026 02:30

Copilot started reviewing on behalf of lewing September 5, 2026 02:32 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

maraf reviewed Sep 6, 2026

View reviewed changes

Comment thread scripts/tests/test_run_performance_job.py

Comment thread scripts/tests/test_wasm_coreclr_r2r.py

Comment thread scripts/tests/test_wasm_coreclr_r2r.py

lewing requested a review from maraf September 6, 2026 18:43

radekdoulik approved these changes Sep 6, 2026

View reviewed changes

Comment thread scripts/tests/test_wasm_coreclr_r2r.py

lewing merged commit 2d66af3 into main Sep 6, 2026
78 checks passed

lewing deleted the lewing-wasm-coreclr-r2r-perf branch September 6, 2026 19:42

Uh oh!

Conversation

lewing commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Dependency

Validation

Remaining validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

lewing commented Sep 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

lewing commented Sep 2, 2026

Uh oh!

lewing commented Sep 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

lewing commented Sep 2, 2026

Uh oh!

lewing commented Sep 2, 2026

Uh oh!

lewing commented Sep 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Suppressed comments (1)

Uh oh!

lewing commented Sep 2, 2026

Uh oh!

lewing commented Sep 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

maraf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

radekdoulik commented Sep 6, 2026

Uh oh!

lewing commented Sep 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lewing commented Sep 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

radekdoulik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lewing commented Sep 2, 2026 •

edited

Loading

lewing commented Sep 6, 2026 •

edited

Loading

lewing commented Sep 6, 2026 •

edited

Loading