Port the WAS wealth imputation and regional property uprating as declarative source stages

[juaristi22]

Workstream E5 of the UK migration (#145, master epic #665). Closes #681. Closes #180. Originally stacked on #709 (E4); rebased onto main after #709 and #706 merged, now based on main. Supersedes stranded draft #181 (its WAS column mappings and citations are salvaged here; its pre-Frame code is not).

What

Ports the incumbent WAS round-8 wealth layer as two declarative source stages appended after frs_brma:

• was_wealth (household) — WAS round-8 household-tab donor (sha+size-pinned, UKDS SN 7215, DOI 10.5255/UKDA-SN-7215-20, caller-supplied local input), strict lower-case exact column matching (no fuzzy r↔w fallback), the incumbent's 11 imputed outputs plus the UK: surface cash ISA and stocks & shares ISA holdings in the wealth imputation #180 ISA split (cash_isa ← DVCISAVR8_aggr, stocks_and_shares_isa ← DVIISAVR8_aggr), a seeded weighted RegimeGatedQRF checkpointed chain mirroring the incumbent's sequential conditioning (output i conditions on outputs 1..i−1; corporate_wealth drawn as corporate_wealth_excl_isa + stocks_and_shares_isa and folded before later outputs condition on it; cash_isa appended last so the 11-chain conditioning is untouched), donor-support clipping, integer num_vehicles, recipient-side NORTHERN_IRELAND→WALES region remap (prediction only), and the id-keyed allocate_within_group_waterfall for student_loan_balance → person.
• regional_property_uprating (household, deterministic, rewrites only) — per-region factor avg_house_price / unweighted_mean(main_residence_value | owner) applied to main_residence_value and property_wealth from the committed regional_land_values.json resource (MHCLG dwellings + ONS UK HPI Dec 2025; regenerator tools/build_uk_regional_land_values.py from the public CSV fixture). NI has no reference row and is never scaled (port-as-is).

Supporting: three new generic operation kinds (fit_weighted_qrf_chain, allocate_within_group_waterfall, uprate_to_regional_reference) plus first semantics for the reserved fold_into; the uk_support gate (release-blocking) against a committed disclosure-safe bounds resource; uk_export_surface allowances for the two ISA columns (no policyengine-uk model variables yet — policyengine-uk#1791 tracks); gate-battery fingerprints re-cut; coverage-manifest families for both stages (the 11 wealth columns were already required); default-off metadata-keyed QRF model cache in microcosm-fit; acceptance instruments (e5_identity_receipt, household-grain weighted-integrity extension, support-bounds builder).

UK-specificity note (required by #681)

Generic contributions: the three operation kinds, fold_into semantics, the QRF chain usage pattern, the model cache, and the support gate binding pattern live in shared modules/vocabularies. UK-specific: WAS column semantics and cleaning, the two build/uk/ resources, the two uk_runtime transforms, and the gate declarations. The incumbent package name appears nowhere in the tree.

Signed differences

1. RNG/method class (standing): seeded (0) weighted (R8xshhwgt) RegimeGatedQRF chain replaces the incumbent's microimpute QRF (implicit seed 42, unweighted fit, per-output re-seeded RNG giving perfectly correlated quantile ranks across the 11 outputs). Parity is weighted-distributional only. The weighted fit is adjudicated in WAS wealth imputation with the ISA split, plus regional property uprating #681; incumbent seeds recorded as documentation.
2. student_loan_balance allocation off-by-one fixed (defect class): the incumbent looks up 1-based sernums in a 0-based RangeIndex (wealth.py:238–241), assigning each household its neighbor's balance and dropping household 1's. Ours allocates by entity id; a regression test pins non-positional alignment.
3. corporate_wealth as component-sum: drawn as excl-ISA + S&S-ISA and folded, per the UK: surface cash ISA and stocks & shares ISA holdings in the wealth imputation #180/WAS wealth imputation with the ISA split, plus regional property uprating #681 split adjudication; later chain outputs condition on the folded total, preserving incumbent conditioning semantics.
4. num_vehicles integer draw (rint + clip ≥0) replaces microimpute's incidental numeric-categorical RandomForestClassifier conditional-mode path.
5. Exact pinned column names replace the fuzzy r↔w rename fallback; missing columns fail loud. DVCISAVR8_aggr resolved against the real tab header (the bare DVCISAVR8 named in UK: surface cash ISA and stocks & shares ISA holdings in the wealth imputation #180/UK: household-wealth imputation stage with cash / S&S ISA split #181 does not exist).
6. Port-as-is quirks documented, not diverged: blanket fillna(0) with no negative-sentinel recode; GOR code 3 absent from the region map; unweighted uprating means; NI unscaled by the uprating; the WAS↔FRS predictor definitional mismatch (FRS-side predictors materialize through the rules-engine adapter with person/benunit values summed to household, the incumbent's map_to="household" semantics).

Licensed acceptance — PENDING (PR stays draft until run)

The licensed run (twin a/b builds → data/ukds/acceptance/e5/, weighted totals vs the 131-column register, incumbent head-to-head distributional receipt, nonzero shares vs the parity reference, e5_identity_receipt, ISA totals vs HMRC statistics as documented-uncalibrated context, owned_land stability receipt per policyengine-uk-data#448 with the expected reviewed-exclusion path, tail-concentration baselines) has not run yet. was_wealth_support_bounds.json currently carries pre-acceptance placeholder bounds — chosen without reference to licensed data, explicitly labeled as such in the resource — and must be regenerated by tools/build_uk_was_wealth_support_bounds.py from donor-realized ranges (outward-rounded, SDC-safe) during acceptance, with the SDC treatment flagged for reviewer adjudication.

Merge coordination with #706 — RESOLVED

#706 merged first, so this PR applied the agreed second-lander recipe on rebase: the gate entries are unioned (#706's re-baselined weight-ratio/input-mass/tail-concentration parameters kept intact; this PR adds uk_support and the two ISA export allowances) and the three _UK_GATE_BATTERY_* digests were re-cut from the producer payload (test_gate_battery_contract_pins driving), with both contract mirrors updated in lockstep.

Implementation notes

Codex implemented under /codex-implement-plan; one recorded deviation (the placeholder support bounds, covered above). Claude review added two fixes pre-commit, both in the class PR CI cannot see: recipient predictors are now aggregated to household grain by native entity (num_adults/num_children are benunit, the four income predictors person — the direct engine arrays would have crashed the licensed build; the fake-engine tests now serve native-grain arrays with entity metadata), and region is one-hot encoded jointly across donor and recipient (matching the incumbent's dummy encoding and the hmrc_spi_income precedent) instead of an ordinal code.

🤖 Generated with Claude Code