Evidence-tier publishing: contract sibling, latest-evidence.json, builder --evidence-release (#506) - #674
Draft
MaxGhenis wants to merge 5 commits into
Draft
Evidence-tier publishing: contract sibling, latest-evidence.json, builder --evidence-release (#506)#674MaxGhenis wants to merge 5 commits into
MaxGhenis wants to merge 5 commits into
Conversation
validate_evidence_release_dir as a sibling of the certified contract: same required files and shape/provenance/hash checks, plus a mandatory non-empty known_failures block (verbatim gate-failure strings, each with an owner issue ref). Evidence manifests declare a distinct schema marker (1-evidence) and tier field, so the certified contract structurally rejects them in one direction and the evidence contract rejects certified-shape manifests in the other; validate_release_dir itself is untouched. Scoped to national-default releases: local-area roles (#398) and UK exact-k releases (#611) are refused pending their own adjudicated evidence semantics. publish_release grows evidence=True: validates via the evidence contract, publishes the immutable tag as usual, and moves only latest-evidence.json (payload mirrors latest_pointer_payload + tier) — the certified latest.json path never appears in an evidence commit. latest_evidence_release reads the new pointer; latest_release now refuses a pointer payload naming a foreign tier. The publish CLI grows --evidence. Slack alerts label the evidence tier so they can never read as a certified release announcement. Ported from the pre-rename populace tree (branch evidence-tier-506, commit 9012391d) onto the microcosm namespace. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… owned known_failures (#506) On terminal-gate failure the certified path is unchanged: batched pre-export raise, #568 weight-evidence sidecar, no H5, no manifests. Under --evidence-release the same recorded failures instead ride into the release manifest's known_failures block and the run continues through the H5 write, reform smoke, take-up contract, and manifests. Every conversion point resolves owners immediately — a failure matching neither the standing US_EVIDENCE_FAILURE_OWNERS register (the two #506-adjudicated dense families: SOI Table 1.4 dollar fit -> #487, QRF tail concentration -> #481/#487) nor the --evidence-failure-owners per-run file refuses the export. An all-green run under the flag is refused: the flag is structurally incapable of minting a certified-shape manifest (the release manifest gets the 1-evidence schema marker + tier + known_failures via _evidence_release_manifest_fields, which raises on an empty record). Release ids: --evidence-release auto-ids carry the -evidence- segment; certified builds refuse ids that squat it. Incompatible with --exact-k (the ladder lane keeps its own tag-only contract). Preflight and mid-build source-stage gates, --audit-export-targets, and the dirty-worktree refusal abort in both modes: the tier relaxes terminal gate verdicts, never artifact auditability. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…test-evidence.json (#506) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r paths, harden scope (#506) Findings fixed from the cross-family adversarial review of the committed diff: - (HIGH) Pointer isolation was bypassable through manifest-declared root artifacts: an artifact entry whose path IS latest.json (or latest-evidence.json) uploaded at that root path. publish_release now refuses reserved pointer paths as root artifacts on BOTH tiers, pinned in both directions. - (HIGH) known_failures was unbound from the artifact's own record. The evidence contract now requires every build_manifest gates.calibration failure verbatim in known_failures, and recomputes the certified critical-target verdicts requiring each breach acknowledged by name — a softened or dropped entry refuses. (Over-disclosure stays legal: the tier can under-promise, never under-disclose.) - (MEDIUM) The evidence contract now refuses non-populace-us release ids: a generic id with the segment must not buy a weaker contract by deactivating the US-specific requirements. - (MEDIUM) latest_release now refuses ANY tier field in latest.json — explicit 'certified' and null included; no certified producer writes one. - (PLAUSIBLE, confirmed) An evidence attempt refused for an unowned failure now falls through to the same failed-run path as a certified gate failure, #568 weight-evidence sidecar included, before re-raising the refusal. - (COVERAGE) New AST guard pins the main() conversion shape: every terminal 'Release gates failed' raise after the #548 accumulator is conditioned on --evidence-release, every one before it is unconditional, the all-green refusal precedes the manifest write, and _build_manifests receives the owned record. Adjudicated without change: the certified-path refusal of '-evidence-' ids is the intentional reserved-namespace guard (disclosed in the PR body); LatestPointer.tier is a deliberate additive API field, now in the changelog fragment. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the microcosm#506 design (owner directive: the best available artifact should be publishable at all times WITHOUT weakening certified gating), generalizing the #490/#497 register pattern — adjudicate, record, carry the caveat in the artifact. Leaves #506 open for the owner to close after the first evidence publish (the current best dense artifact).
The tier, structurally
validate_evidence_release_dir(microcosm-data): a sibling ofvalidate_release_dir, not a relaxation — same required files, same shape/provenance/hash/cross-manifest checks, plus a mandatory non-emptyknown_failuresblock: every recorded gate failure verbatim, each with an owner issue ref. Gate-verdict requirements (US critical-target fit, source-coverage gate.passed) are exactly what the tier records instead of enforcing. Dirty-git, hash mismatches, and build-id mismatches still refuse: the tier relaxes verdicts, never auditability.schema_version: "1-evidence"+tier: "evidence"; the certified contract rejects that schema marker no matter which gates failed, and a certified-shape manifest (no tier, no known_failures, schema 1) fails the evidence contract.validate_release_diritself is untouched.Publish path
publish_release(evidence=True)validates via the evidence contract, creates the immutable tag exactly as today, and moves only a newlatest-evidence.jsonpointer (payload mirrorslatest_pointer_payload+tier). The certifiedlatest.jsonpath never appears in an evidence commit — pointer isolation is pinned in both directions by tests.latest_evidence_release()is the consumer;latest_release()now refuses a pointer payload naming a foreign tier (defense if an evidence payload ever landed inlatest.json).microcosm-publish-release --evidence; Slack alerts label the tier so they can never read as a certified release announcement.Builder
--evidence-release_default_release_id)._evidence_release_manifest_fieldsstamps the evidence schema marker + known_failures (raises on an empty record — the flag is structurally incapable of minting a certified-shape manifest). Preflight and mid-build source-stage gates still abort in both modes.US_EVIDENCE_FAILURE_OWNERSregister (the two Evidence-tier publishing: ship the best available artifact at all times, with its failures recorded in the manifest (dense first) #506-adjudicated dense families: SOI Table 1.4 dollar fit → PUF donor uprating overstates mean-reverting distribution series: non_sch_d at 2.39x the SOI concept #487, QRF tail concentration → QRF amount forests lose donor design weights after the n-of-n bootstrap: extreme-tail draws are a seed lottery #481/PUF donor uprating overstates mean-reverting distribution series: non_sch_d at 2.39x the SOI concept #487) or a--evidence-failure-ownersper-run JSON — an unowned failure refuses the export, at the earliest conversion point.-evidence-segment; certified builds refuse ids that squat it; incompatible with--exact-k(the ladder lane keeps its tag-only contract).Tests
49 new tests across the three suites (19 contract, 13 release/publish, 2 CLI guard, 11 builder + the #568 AST guard now also covering the evidence branch shape). Full workspace suite green (
uv run pytest: exit 0 on this branch), ruff clean.Notes for the merge gate
evidence-tier-506, commit 9012391d) onto microcosm paths/namespace; the One country-agnostic gate battery: phased evaluation, unconditional reporting, per-country gate selection (#578) #611 schema-4 verifier (Schema-4 gate-battery verification in the release contract (#611 A3) #663) is already in this base.gates.calibrationin the build manifest stays the battery-only verdict in both modes; the release manifest'sknown_failuresis the authoritative full terminal record (superset).🤖 Generated with Claude Code