Infrastructure: Add bundle-size and dependency regression gates

[JasonVMo]

Summary

Create deterministic package-cost gates for the agentic component and design
packages. Measure representative production consumer bundles, verify optional
theming capabilities are absent unless imported, enforce the component source
and dependency boundary, and make CI reject unapproved regressions against
reviewed baselines.

This task combines the approved Components bundle-size and dependency-budget
addition with the approved Theming bundle-size regression addition. It replaces
an orphaned legacy bundle-size configuration with one maintained,
cross-workstream system.

Goal

Make the package-size and dependency claims in the Components and Theming
workstreams repeatable and enforceable. A contributor must be able to run one
declared command, reproduce the checked-in metrics, understand which modules
caused a delta, and distinguish an intentional baseline update from an
accidental regression.

Stage

Stages 1 through 3.

• Stage 1: define scenarios, metrics, and update policy; land the harness and
  record current baselines.
• Stage 2: report comparisons in pull requests and enforce dependency and
  optional-module reachability rules.
• Stage 3: make approved size budgets required CI gates.

Why it matters

Observed. The component package source guidance allows production source to
depend only on React and React Native native components,
@fluentui-react-native/design, and
@fluentui-react-native/framework-base
(packages/agentic/components/src/AGENTS.md).
Its manifest nevertheless declares
@fluentui-react-native/callout, and its Knip configuration suppresses that
unused production dependency
(packages/agentic/components/package.json).
No component source imports Callout.

Observed. The Theming workstream makes three measurable production claims:
generated defaults de-duplicate shared values, legacy Fluent theme construction
runs only when a legacy hook requests it, and runtime color derivation is absent
from bundles that do not import its submodule. The
Default Values Codegen,
Dynamic Theme Building, and
Runtime Color Utilities tasks
cannot close those criteria without a stable consumer-bundle measurement.

Observed. The repository has no bundle-size task in
lage.config.mjs
or the root
package.json.
The only remaining bundlesize configuration is in
scripts/package.json;
it targets apps/test-bundles/dist/office-ui-fabric-react-Button.min.js, but
apps/test-bundles no longer exists, and the bundlesize package is not
declared by that workspace or present in the lockfile.

Observed. Existing application bundles are not a stable proxy for these
packages. Storybook imports the complete story catalog and FluentTester imports
the legacy test catalog, so unrelated story, test-page, or application changes
can hide the cost of one component or one theming capability.

Inferred. A useful gate needs purpose-built consumer scenarios, not only a
maximum size on an existing application bundle. It also needs module
reachability evidence: a small byte delta cannot prove that optional code or an
unapproved dependency was excluded.

Scope

Phase 1: deterministic harness and current baselines

• Add a minimal bundle-measurement application or fixture whose entries import
  only the capability named by each scenario.
• Produce release-mode, minified Metro bundles for a fixed React Native version,
  platform, Node version, package-manager state, and bundler configuration.
• Record raw JavaScript bytes, source-map module counts, and the packed archive
  sizes for @fluentui-react-native/components and
  @fluentui-react-native/design.
• Emit a machine-readable result with scenario, platform, tool versions,
  current commit, raw size, module count, package archive size where
  applicable, and the comparison to its checked-in baseline.
• Normalize or exclude nondeterministic paths and timestamps so two clean runs
  at the same commit produce identical measurements.
• Remove the orphaned scripts/package.json bundlesize command and dead
  apps/test-bundles target rather than maintaining two systems.

Phase 2: dependency and reachability enforcement

• Check each measured package's production dependency allowlist against its
  manifest and reachable production source graph.
• Reject unused production dependencies rather than hiding them with Knip
  ignores. Remove @fluentui-react-native/callout from the component package
  unless an approved source-boundary change introduces a real production use.
• Use source-map module identities or an equivalent stable module manifest to
  assert both absence and presence:
  • a component-only entry does not reach undeclared or disallowed package
    families;
  • a Flex-only theme entry does not reach legacy Fluent construction;
  • a theme entry that does not import runtime color utilities excludes their
    implementation;
  • explicit opt-in entries include the corresponding legacy or runtime-color
    capability, proving that an absence assertion did not pass because the
    fixture was broken.
• Add an advisory CI comparison that publishes the scenario report and
  artifacts without silently rewriting baselines.

Phase 3: required regression budgets

• Define a checked-in absolute and percentage growth budget for each scenario
  delta after enough Stage 2 runs establish normal variance.
• Fail the required gate when a scenario exceeds either its approved budget or
  its dependency and reachability rules.
• Require an explicit baseline-update command and a reviewed rationale when a
  deliberate public API or capability change must increase a budget.
• Keep historical trend storage outside the source tree; commit only the
  current baseline, scenario definitions, and policy.

Measurement scenarios

The exact fixture path is an implementation decision, but these semantic
scenarios are required:

Scenario	Entry contract	Primary proof
React Native shell	Provider and host shell with no Fluent package import	Stable framework and platform floor used for deltas
Single component	Shell plus one representative component such as Button through the supported public import	Cost and reachable dependencies of adopting one agentic component
Component catalog	Shell plus every public component and primitive	Aggregate catalog growth as missing components are generated
Flex theme	Shell plus the normal Flex token and ThemeState path	Default design cost without optional or legacy capabilities
Legacy Fluent opt-in	Flex-theme entry plus the supported legacy compatibility path	Incremental cost and positive reachability of legacy construction
Runtime color opt-in	Flex-theme entry plus the runtime color submodule	Incremental cost and positive reachability of color derivation
Package archives	Packed components and design workspaces	Published package payload and accidental-file growth

Windows and macOS platform resolution must both be represented before the gate
becomes a production requirement. Stage 1 may use one fixed CI platform while
the harness is stabilized, provided the missing platform is explicit in the
report.

Out of scope

• Native binary, application package, startup-time, memory, render-time, and
  over-the-air update measurements.
• A general performance benchmark framework.
• Using the complete Storybook or FluentTester bundle as the regression
  baseline.
• Setting final byte thresholds before current-main baselines and normal CI
  variance are recorded.
• Automatically accepting a new baseline because a pull request exceeded its
  budget.
• Deciding whether a new component or theming capability is worth its cost; the
  owning workstream makes that product decision.

Deliverables

1. A minimal bundle-measurement fixture with one entry per required scenario.
2. A declared root command and Lage task that build, measure, compare, and
   report every active scenario.
3. Checked-in scenario definitions, dependency allowlists, baselines, and
   growth budgets.
4. A machine-readable result and concise human-readable comparison report.
5. Module reachability assertions for the component boundary, Flex-only path,
   legacy Fluent opt-in, and runtime-color opt-in.
6. Packed-package size measurements using the repository's existing pack
   worker or an equivalent deterministic path.
7. Pull-request CI integration, first advisory and then required.
8. Documentation for local reproduction, baseline updates, expected variance,
   and failure investigation.
9. Removal of the dead scripts/package.json bundle-size configuration.
10. Changesets for any published package surface or manifest changes.

Acceptance criteria

• One documented root command produces all active bundle, reachability,
  dependency, and package-archive measurements from a clean checkout.
• Two clean runs at the same commit and on the same declared environment
  produce identical budget inputs; nondeterministic metadata is excluded
  from comparison.
• Results record raw bytes and module counts, plus packed archive bytes for
  the two measured packages. A compressed size may be reported as
  supplemental data but is not the only budget.
• Scenario deltas are calculated against the React Native shell so
  framework or platform changes are distinguishable from package growth.
• The component package's production dependencies match its approved source
  boundary, every declared production dependency is reachable from
  production source, and no unused production dependency is hidden by a
  Knip ignore.
• The single-component and component-catalog reports list their reachable
  workspace package dependencies.
• The Flex-theme scenario excludes legacy Fluent construction and runtime
  color derivation.
• The legacy and runtime-color opt-in scenarios each include their expected
  implementation and report their incremental cost.
• Windows and macOS platform-resolution scenarios have checked-in baselines
  before the gate is marked production-ready.
• CI publishes a comparison report and diagnostic artifact when a scenario
  changes or fails.
• Required CI fails on an unapproved dependency, failed reachability rule,
  or size increase beyond the checked-in absolute or percentage budget.
• Baselines can change only through an explicit update command, and the
  resulting diff identifies the affected scenarios for reviewer approval.
• The orphaned legacy bundlesize command and target are removed.
• Repository build, test, lint, format, publishing, and lockfile checks pass,
  and changesets exist for published package changes.

Dependencies and ordering

• Phase 1 has no prerequisite and should land before component generation or
  theming consolidation changes the current package-cost baseline.
• The component allowlist is owned by the
  Components workstream; an implementation may
  not widen it solely to make the dependency check pass.
• The Flex-only and legacy scenarios follow the public boundaries established
  by Package Consolidation and
  Dynamic Theme Building.
• The de-duplication comparison closes the measurement requirement in
  Default Values Codegen.
• The runtime-color scenario becomes active when
  Runtime Color Utilities
  publishes its optional submodule.
• Required Stage 3 budgets should be set only after advisory CI has established
  normal variance for both platform-resolution scenarios.

Risks and open decisions

• Open decision. Which fixed CI operating system runs Metro measurement.
  The bundle platform and host operating system are separate inputs and both
  must be recorded.
• Open decision. Whether required budgets use raw Metro bytes only or raw
  bytes plus Hermes bytecode. Raw bytes are required for the initial gate
  because they are portable and inspectable; Hermes may be added as a separate
  metric rather than replacing them.
• Open decision. Whether package archive size is gated independently from
  consumer-bundle size. Inferred: both should be retained because accidental
  source maps or generated files can enlarge a package without changing one
  bundle scenario.
• Open decision. The initial absolute and percentage budgets. They must be
  derived from current baselines and observed CI variance, not selected to make
  the first run pass.
• Open decision. Whether the component package needs supported
  per-component export subpaths if Metro proves that importing one component
  from the root reaches the complete catalog.
• Open decision. Whether components and design can safely declare
  "sideEffects": false. Neither package declares it today; add it only after
  auditing module initialization and verifying the measured consumer output.
• Risk. Metro or React Native upgrades can move the shell baseline
  substantially. Scenario deltas and explicit tool-version metadata reduce
  this noise but do not eliminate the need for reviewer judgment.
• Risk. Minifier output can change without a source-level dependency
  regression. Module reachability is a separate required signal so byte changes
  are diagnosable.
• Risk. A positive opt-in scenario can become stale when an export path
  changes. The harness must fail if its expected module marker disappears,
  rather than interpreting that as an improvement.
• Risk. Measuring only package archives would reward code compression but
  miss tree-shaking failures; measuring only bundles would miss accidental
  files and manifest bloat. The two measurements are complementary.

Evidence and references

• packages/agentic/components/src/AGENTS.md:
  allowed component production dependencies.
• packages/agentic/components/package.json:
  current dependencies and the unused Callout Knip suppression.
• packages/agentic/design/package.json:
  current design entry points and dependency baseline.
• lage.config.mjs:
  task graph with no bundle-size measurement.
• scripts/package.json:
  orphaned legacy bundlesize command and deleted target.
• apps/storybook/package.json:
  current macOS and Windows production Metro bundle commands.
• Package Consolidation:
  optional design-submodule and root-export boundary.
• Default Values Codegen:
  de-duplication and before-and-after bundle measurement requirement.
• Dynamic Theme Building:
  lazy legacy Fluent construction requirement.
• Runtime Color Utilities:
  optional-submodule reachability requirement.

[JasonVMo]

Status update - 2026-09-03

This issue is being reopened so its state matches its original Stage 1 through
Stage 3 scope.

Completed: PR #4284 delivered the package-cost measurement foundation,
checked-in baselines, dependency/reachability checks, and an advisory pull
request pipeline. PR #4295 added cross-platform Flex-theme scenarios and proved
that modern Flex paths exclude reverse legacy compatibility payloads.

Remaining: establish normal CI variance, approve absolute and percentage
growth thresholds, make the budgets required, require reviewed rationale for
baseline changes, and finish any acceptance criteria not yet covered by the
advisory implementation. Advisory warnings do not satisfy this issue's
required-gate acceptance criteria.

This remains a direct child of #4275.