Skip to content

feat(accuracy): propagate end-to-end guarantees for nested summaries (#172) - #303

Open
zzylol wants to merge 4 commits into
mainfrom
review-pr-299-work
Open

feat(accuracy): propagate end-to-end guarantees for nested summaries (#172)#303
zzylol wants to merge 4 commits into
mainfrom
review-pr-299-work

Conversation

@zzylol

@zzylol zzylol commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Reopens the work originally proposed in #299 for review. Refs #172.

This PR is intentionally not configured for auto-merge. It restores the reviewed branch state after #302 reverted the premature merge, so the accuracy model and planner integration can be evaluated before any merge decision.

Scope

  • Add machine-readable result guarantees with typed error metrics, symbolic bounds, failure probabilities, and provenance.
  • Add a conservative AccuracyModel that fails closed for unsupported or cross-metric compositions.
  • Propagate supported exact and approximate guarantees through nested summaries.
  • Add accuracy-budget allocation and reject candidates that cannot meet the applicable target before cost ranking.
  • Export guarantees and rejected candidates through the DAG representation.
  • Integrate with current recurrence costing and cross-consumer accuracy reconciliation.

Soundness boundaries

  • TopK sketch readouts do not receive a result guarantee: per-key frequency error does not certify membership in the true top-k set.
  • CountSketch does not reuse the CMS L1 bound: its L2 guarantee and parameter sizing are not modeled yet.
  • Hydra grouped summaries do not inherit the inner sketch guarantee: shared-grid collision error must be included before an end-to-end guarantee can be claimed.
  • KLL, HLL, KMV, and Theta retain their modeled error magnitudes, but their failure probabilities are unknown because current parameters do not encode a confidence target. They therefore cannot satisfy an EpsilonDelta requirement.
  • Unsupported statistics and compositions fail closed and are rejected before cost ranking.

Deliberately deferred

  • TopK membership-margin certificates.
  • CountSketch L2 sizing and guarantee propagation.
  • Hydra shared-grid error composition.
  • Parameter-derived confidence bounds for KLL/HLL/KMV/Theta.
  • Runtime posterior-observation integration.
  • Advanced nonlinear and correlation-aware propagation.

Issue #172 should remain open after this PR because those items are not implemented here.

Runtime prerequisites for completing the deferred guarantees

The four deferred sketch guarantees cannot be completed soundly by changing planner metadata alone. They require coordinated changes in the actual sketch runtime (ProjectASAP/asap_sketchlib / sketchlib-go, or whichever implementation serves the finalized plan) and then corresponding integration in ASAPPlanner.

Runtime work required:

  • TopK membership: return certified error intervals for the candidate keys around the boundary, including the estimated kth and (k+1)th frequencies. Membership is certified only when the kth lower bound is strictly above every excluded candidate's upper bound. A point-frequency bound by itself is not a TopK membership certificate.
  • CountSketch: expose or conservatively bound the stream frequency vector's L2 norm and use CountSketch-specific width/depth sizing. The runtime contract must support a statement of the form |f_hat(x) - f(x)| <= epsilon * ||f||_2 with failure probability at most delta; the CMS L1 formula must not be reused.
  • Hydra: expose the outer shared-grid collision term and the statistics needed to normalize it. The end-to-end guarantee must compose that term with the inner CMS/CountSketch guarantee instead of copying the inner per-subpopulation bound.
  • KLL confidence: provide a runtime-supported confidence/amplification contract, not just k, so the planner can derive both normalized rank error and failure probability from committed parameters.
  • HLL/KMV/Theta confidence: document and expose a parameter-derived tail bound (or an explicit confidence parameter) for the exact runtime estimator. Standard error alone is not an (epsilon, delta) guarantee.

Planner work required after those runtime contracts exist:

  • Extend the concrete sketch parameter/observation IR without breaking runtime compatibility.
  • Size every family from the allocated (epsilon, delta) budget using the same theorem implemented by the runtime.
  • Import runtime evidence into ResultGuarantee provenance and evaluate the symbolic L2, margin, and shared-grid terms.
  • Add propagation and satisfaction rules for CountSketch, certified TopK membership, and Hydra.
  • Continue to fail closed whenever required runtime evidence or a supported confidence contract is absent.

Merely recording the requested delta in SketchParams is not sufficient: it would claim confidence without changing the executed algorithm or providing evidence that the probability bound holds. These capabilities should therefore be delivered as linked runtime and planner changes rather than asserted solely in this PR.

Validation

  • cargo fmt --all
  • cargo test --workspace (212 mapping unit tests plus all workspace suites)
  • cargo clippy --workspace --all-targets -- -D warnings
  • git diff --check

All pass locally.

Review focus

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant