Skip to content

Feed observed posterior sketch error back into re-planning (execution-time → planning-time loop) #250

Description

@zzylol

Summary

#239 / PR #248 implement Chen et al.'s posterior (query-time) error
estimator for CMS/CU-Sketch/Count-Sketch as a pure, execution-time
computation: given a sketch's actual counter values at readout, it
returns a tighter error bound than the traditional a priori worst-case
guarantee. That estimator is correctly scoped to execution time — it
needs real counter values a query only has once a (currently nonexistent)
sketch runtime has actually ingested data and been read out.

This issue is the natural follow-up: a feedback loop from that
execution-time observation back into planning time. If a workload is
replanned periodically (re-run through ASAPPlanner as data/traffic
patterns evolve), the posterior error estimates observed for past
executions of a query are exactly the kind of empirical, non-adversarial
signal posterior_aware_size_params's ExpectedCaseSizing (PR #248,
crates/asap-aware-mapping/src/boundary.rs) currently asks a caller to
supply as a bare assumption (width_relaxation, chosen with no
grounding). Recorded history could instead derive that assumption from
what the workload's sketches actually experienced last time:

Why not now

Same blocker #239/#248 already hit: this repo has no sketch runtime, no
counter arrays, and (new for this issue) no persistence/history layer for
recording past execution outcomes at all. This is planning-adjacent
future work, not an immediate implementation task — filing to record the
idea and the concrete hook point (ExpectedCaseSizing/posterior_aware_size_params)
so it isn't lost.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    cost-modelCost formulas, statistics, ranking, and selectionenhancementNew feature or requesterror-estimationreplanningRuntime feedback and incremental re-optimization

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions