Skip to content

Declare the uk_national target profile over the ported UK facts (#133) - #164

Open
juaristi22 wants to merge 1 commit into
PolicyEngine:mainfrom
juaristi22:uk-national-target-profile
Open

Declare the uk_national target profile over the ported UK facts (#133)#164
juaristi22 wants to merge 1 commit into
PolicyEngine:mainfrom
juaristi22:uk-national-target-profile

Conversation

@juaristi22

Copy link
Copy Markdown
Collaborator

Summary

Declares the uk_national target profile over the ported UK facts — the last item in #133's scope. Closes #133.

186 value-free targets across 20 families mirror the 651-row national registry policyengine-uk-data compiles at the pinned migration ref ebf733c (NATIONAL + COUNTRY + REGION, the exact set create_target_matrix iterates): 609 rows map onto profile targets and 42 carry signed exclusions — every row is accounted for, no silent gaps. The mapping was machine-generated from a dump of uk-data's get_all_targets() and the generator asserts the accounting closes before emitting the JSON.

What each target carries:

  • chronicle_selector referencing raw Chronicle facts by source_concept / source_measure_id / groupby_dimension (concept-level selectors for OBR/DWP/ONS families whose measure_ids are generic and whose record_set_ids are period-scoped).
  • measurement with entity, map_to, model-concept filters, and source_dimensions pins naming the exact publisher rows (two-child-limit status, UC family types, SLC lines, household types) — dimension values are not selector vocabulary, so the pins live here as declarations for the consumer.
  • bindings.policyengine reifying the semantics uk-data keeps implicit in ~35 name-parsing dispatch branches and 31 compute functions: value_variable/value_expression, from_entity (native entities verified against policyengine-uk), map_to, filters, and household_conditions + reduce: any for the household-indicator counts uk-data overloads onto is_count.
  • bindings.axiom stubs (status: pending) following the existing profiles' convention.

New declarative counterfactual vocabulary (no values, no runtime hooks — the payload is the spec populace's metric-provider registry implements in populace#622):

  • input_substitution_counterfactual — salary-sacrifice relief: zero pension_contributions_via_salary_sacrifice, fold into employment_income, target the income-tax/NICs delta, IT relief banded on counterfactual adjusted_net_income against live parameter thresholds.
  • parameter_gated_threshold — CGT: taxpayer/gains totals gated on gov.hmrc.cgt.annual_exempt_amount read from the parameter tree (gating only one of count/amount would target a mean over two different populations).
  • baseline_flag_crosstab — two-child limit: baseline uc_is_child_limit_affected flag with per-shape crosstab conditions; no second simulation.

Assertion axis (#154): the 28 OBR EFO lines, 4 SLC borrower-forecast targets, and the Scottish budget line carry allow_source_projection; everything else rides the observed_only default, so ported projections (NPP 2024-based) stay dormant until populace activates them — porting never activates.

Row accounting

profile family targets uk-data rows covered
council_tax_stock 18 99
dwp_benefit_cap 1 1
dwp_legacy_benefits 4 4
dwp_pip 2 2
dwp_two_child_limit 15 15
dwp_universal_credit 14 114
hmrc_cgt 2 2
hmrc_salary_sacrifice 5 5
hmrc_spi 12 156
isc 1 1
obr 28 28
ons_employment 1 1
ons_household_composition 10 10
ons_land 3 3
ons_national_accounts 1 1
ons_population 49 148
scotgov_social_security 1 1
slc_borrowers 4 4
slc_repayments 6 8
slc_student_support 6 6
total 186 609

Plus 3 profile targets with no registry row: the salary-sacrifice IT-relief bands, restored from uk-data source code — their values never load at ebf733c because the pinned HMRC CSV now 410s, so uk-data silently drops them (link-rot already logged on the checklist).

Signed exclusions (42 rows, no ported facts):

Vintage and value-side differences carried populace-side (enumerated per the acceptance rule; the compile-parity gate in populace#622 adjudicates):

  • SPI property income ×1.9 undercount adjustment; UC family-type undercount inflation (+~0.021%); OBR salary-sacrifice growth uprating; Scottish child payment 3%-per-year extrapolation beyond the budget horizon; savings-interest and land-value flat-fills — all value-side adjustments uk-data applies to raw publisher numbers; the facts are the raw numbers.
  • The UC Stat-Xplore families resolve from the May-2025 extract packages, a later vintage than uk-data's hand-transcribed counts.
  • uk-data's SPI dedupe quirk (the 2025 matrix serves un-uprated 2024 ODS values; the projection CSV is bypassed on the matrix path) is inherited data, not profile semantics.
  • Three suspected uk-data binding defects are carried as-is with notes flagging them for adjudication rather than silently fixed: obr/ni (total-NI line bound to ni_employee only, alongside separate employee/employer/self-employed lines), obr/pip (EFO line covers DLA+PIP; uk-data binds pip only), dwp/jsa_claimants (bound to contributory JSA only).
  • The dead compute_uc_jobseeker dispatch branch (no source emits those names) is not ported.
  • docs/pe-uk-source-checklist.md's uk_national row will be pointed at this PR in a follow-up touch — docs/** is outside this lane's allowed paths.

Chronicle Governance

  • Approved Chronicle agent role: ledger-target-profile-author
  • Deterministic checks run:
    • no embedded target valuesFORBIDDEN_VALUE_KEYS/FORBIDDEN_RUNTIME_KEYS enforced recursively at load; the profile loads through target_profile_from_mapping in the tests.
    • sum-only operationdefaults.operation: "sum", no per-target operation or aggregation keys anywhere.
    • selector references raw Chronicle facts — the existing packaged-profile vocabulary test covers the new file via its glob, and the new resolution test proves every target × geography level matches ≥1 built fact (see Tests).
  • LLM judge verdicts:
    • ledger-source-fidelity: n/a (no source packages touched)
    • ledger-target-profile: pending review
    • ledger-contract: n/a (no contract/schema change)
    • ledger-boundary: pending review

Tests

Three additions to tests/test_policyengine_chronicle_target_profiles.py (the lane-owned file):

  1. Structural — pins the 186-target/20-family shape; asserts projection-backed families (and only they) carry allow_source_projection, no target pairs a policy with an assertion selector key, every target carries policyengine + axiom bindings, and counterfactual kinds come from the declared vocabulary.
  2. Counterfactual payloads — pins the three kinds' load-bearing fields (zeroed input/fold target/output variable; gate parameter/gated variable; baseline flag).
  3. Full resolution — builds a bundle of exactly the 32 packages the profile selects (with a guard that the selector source set equals the package source set), builds the uk_national consumer artifact, and asserts every target × geography level resolves ≥1 built fact with zero selector issues (signed-exception set present but empty), plus that the borrower-forecast facts really are projections. Runs in ~80s.

Full local suite green: 633 passed, 1 skipped. No bundle-expectation re-measure needed — a profile-only change moves no fact counts.

Resolution spot-checks against uk-data's registry values: benefit cap 110,637 (month 2025-11, observation), CGT 378,000 taxpayers / £65.937bn gains (tax year 2023), OBR income tax £305.905bn (FY2024, observation), UK population 69,281,437 (CY2024, MYE).

🤖 Generated with Claude Code

…cyEngine#133)

186 value-free targets across 20 families mirror the 651-row national
registry policyengine-uk-data compiles at the pinned migration ref
ebf733c (NATIONAL+COUNTRY+REGION): 609 rows map onto profile targets and
42 carry signed exclusions (no ported facts: computed products, lost
sources, or wave-3 scope). Selectors reference raw Chronicle facts by
source_concept/measure/groupby; measurement carries entity, map_to,
filters, and publisher-dimension pins; bindings.policyengine reifies the
semantics uk-data keeps in ~35 name-parsing dispatch branches and 31
compute functions.

New declarative binding vocabulary (no values, no runtime hooks):
input_substitution_counterfactual (salary-sacrifice relief),
parameter_gated_threshold (CGT above the live annual exempt amount), and
baseline_flag_crosstab (two-child limit), plus reduce/household_conditions
for household-indicator counts uk-data overloads onto is_count.

OBR EFO lines, SLC borrower forecasts, and the Scottish budget line carry
allow_source_projection (PolicyEngine#154); everything else rides observed_only so
ported projections (NPP) stay dormant until populace activates them.

Tests: structural pins (families, policies, counterfactual vocabulary)
and a full-resolution test that builds the 32 selected packages and
asserts every target x geography level resolves at least one built fact
with zero selector issues - the mechanical form of this issue's
"resolves every selector against built facts" acceptance line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UK source packages, wave 2: national fiscal/benefit families + uk_national target profile

1 participant