Skip to content

Evaluate range-queryable structured summaries (QTree-style) #290

Description

@zzylol

Context

Split from #173 after its multi-subpopulation/Hydra portion landed. ASAPPlanner now represents shared multi-subpopulation frequency-sketch state through Hydra grouping candidates, but it still has no first-class range-queryable structured summary comparable to Algebird's QTree:

  • state is organized over non-overlapping value ranges;
  • each range carries decomposable/mergeable inner state;
  • readout asks about a sub-range and may return lower/upper bounds rather than one scalar point estimate.

This issue evaluates and, if justified, models that remaining structured-summary capability. It does not reopen the already implemented Hydra grouping work.

Required semantics

  • Define range-shaped readout operations such as bounded range count/sum and range quantile.
  • Define whether inner state is a closed core vocabulary or a recursively referenced mergeable summary family.
  • State merge, compaction, and range-query legality must be explicit.
  • Exact and approximate inner states must retain their existing accuracy contract; nesting a sketch does not make it exact.
  • Runtime capability must gate candidate generation until a downstream implementation exists.

Cost model

Compare alternatives in cost units per second, with state size reported independently in bytes.

For maintained structured range state:

structured_cost_rate =
    update_rate * structured_update_cost
  + evaluation_rate * structured_range_read_cost

For recomputation from the pre-ASAP/raw path:

raw_recompute_cost_rate =
    evaluation_rate * raw_range_scan_cost

For several repeating consumers sharing the same structured state:

evaluation_rate = sum(1 / query_interval_i)

If one-shot and recurring work are compared over a finite horizon H:

total_cost(H) = recurring_cost_rate * H + one_shot_cost

State accounting:

structured_state_bytes = retained_range_nodes * bytes_per_range_node

Every cost must include units, input provenance, and model version. Unknown runtime/cardinality inputs remain unknown rather than being encoded as zero.

Scope

  1. Write an ADR defining the readout vocabulary, inner-state composition, accuracy semantics, and runtime contract.
  2. Decide whether the capability belongs in core post-ASAP IR or behind the deployment extension interface.
  3. If first-class core support is justified, add the representation and an experimental capability-gated replacement strategy.
  4. Export the selected physical representation and explicit cost/provenance metadata for DAG visualization.

Acceptance criteria

  • ADR compares a first-class structured-summary type with deployment extension and ordinary raw recomputation.
  • At least one range count or range sum fixture has a lossless readout representation.
  • Unsupported/non-mergeable inner state does not produce a candidate.
  • A deterministic cost model selects structured state for a high-frequency range workload and raw recomputation for a sufficiently cold workload.
  • Tests verify evaluation_rate = sum(1 / query_interval_i) for shared consumers and prevent shared-state double counting.
  • Accuracy targets and returned bound semantics survive representation, merge, serialization, and DAG export.
  • Candidate generation is absent without runtime capability.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    accuracyAccuracy requirements, guarantees, and propagationcost-modelCost formulas, statistics, ranking, and selectionenhancementNew feature or requestirPre-ASAP or post-ASAP intermediate representationreplacement-strategyASAP replacement candidates and physical strategiesresearchResearch evaluation, ADR, or experimental design

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions