Skip to content

(dev-tools) Export and visualize planner cost/benefit annotations #286

Description

@zzylol

Context

The DAG viewer should explain why a selected post-ASAP workload is beneficial even when sharing and replacement make the graph look structurally more complex.

The current viewer has one Pre/Post-ASAP workflow: selecting one query displays that query's DAG; selecting multiple queries displays their unioned workload DAG. There are no separate Single, Compare, or Union modes.

This issue adds explicit planner/exported cost and benefit annotations. The viewer must only render metadata present in generated JSON; it must not infer costs, match nodes by guessed signatures, or rely on hand-edited demo JSON.

Cost and benefit semantics

All modeled recurring costs use cost units per second:

maintained_cost_rate =
    update_rate * maintenance_cost_per_update
  + evaluation_rate * summary_read_cost

recompute_cost_rate =
    evaluation_rate * raw_recompute_cost

evaluation_rate = sum(1 / query_interval_i)

For any selected alternative:

estimated_benefit_rate = baseline_cost_rate - selected_cost_rate
estimated_benefit_ratio = estimated_benefit_rate / baseline_cost_rate

The baseline must be named explicitly, for example pre-ASAP recomputation or the highest-ranked legal non-selected candidate. If baseline_cost_rate <= 0, the ratio is unavailable.

Finite-run or one-shot totals require an explicit horizon H:

total_cost(H) = recurring_cost_rate * H + one_shot_cost

Rate costs and one-shot costs must never be silently added. Shared nodes must be counted once in workload totals.

Annotation schema

Define a structured optional annotation rather than an unqualified number. The exact Rust shape may vary, but it must carry equivalent semantics:

struct CostAnnotation {
    value: Option<f64>,
    unit: CostUnit,
    source: CostSource, // Modeled, Measured, Unavailable
    baseline: Option<BaselineRef>,
    delta: Option<f64>,
    model_version: Option<String>,
    benchmark_id: Option<String>,
    inputs: Vec<CostInput>,
}
  • Modeled: generated by a named/versioned cost model.
  • Measured: loaded from a reproducible benchmark artifact.
  • Unavailable: no value; never encode an unknown value as 0 or another synthetic number.

Every numeric value must have units and provenance.

Granularity

Initial supported annotations:

  • replacement-region baseline cost, selected cost, and benefit;
  • whole selected-workload cost and benefit;
  • avoided recomputation for a shared sub-DAG;
  • edge cost only when genuinely attributable to the edge, such as transfer or materialization.

Arbitrary path cost is out of scope until path ownership and shared-node double-counting semantics are defined.

Scope

  • Extend generated DAG-export JSON with optional, backward-compatible node/edge/replacement-region/workload annotations as justified above.
  • Populate modeled annotations from planner/cost-model output, not frontend inference.
  • Allow measured annotations only through a versioned benchmark artifact consumed by the exporter/backend.
  • Render concise on-graph benefit/cost badges and full inputs/provenance in the resizable sidebar.
  • Clearly distinguish Modeled, Measured, and Unavailable states.
  • Support the existing one-query and multi-query Pre/Post-ASAP workflow.

Acceptance criteria

  • Real queries run through lowering, ASAP-aware mapping, post-ASAP generation, and dag_export; demo JSON/HTML is regenerated from that pipeline and is not manually patched.
  • Viewer reads only explicit JSON mappings and annotations.
  • Missing values display Not estimated, never a fabricated number.
  • Modeled and measured values are visually distinct and show units, provenance, and model/benchmark version.
  • Workload totals count shared nodes once and tests detect double counting.
  • Baseline, selected cost, absolute benefit, and benefit ratio follow the formulas above.
  • One-shot totals require an explicit horizon; unit-incompatible aggregation is rejected.
  • Selecting one query and multiple queries both display correct Pre/Post-ASAP annotations.
  • Existing exports without annotations remain fully supported.

Follow-ups

Metadata

Metadata

Assignees

No one assigned

    Labels

    cost-modelCost formulas, statistics, ranking, and selectiondag-viewerDAG export and visualization toolingdev-toolsDeveloper tools, diagnostics, and demosdocumentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions