fix: nine-issue batch — sequences, kv scans, windows, graph DSL, and one instant conversion by EnRaiha · Pull Request #324 · NodeDB-Lab/nodedb

EnRaiha · 2026-09-14T03:57:30Z

Summary

Fixes nine open issues (#295, #296, #297, #305, #306, #311, #314, #317,
#318) across SQL planning and execution, the kv engine, the graph DSL,
sequences, and the timeseries read path. 26 commits over 124cc53a6;
verified at fdba3e378. A full local CI-parity run over this exact tree is
green (see Validation).

Expression errors fold silently over constant derived tables — 22012 raised on collection scans but not on FROM (SELECT ...) #295 — expression errors folded silently over constant derived tables: computed projections now evaluate on the materialized-row scan, aggregates over a non-Scan body lower to a ProviderScan sub-plan, unaliased window projections get one alias derivation, and windows evaluate in the window's own ORDER BY (SELECT x/0 FROM (SELECT 1 AS x) s raises 22012). Supersedes the closed PR fix(sql): raise expression errors over constant derived tables #321.
Graph DSL ignores unconsumed tokens — a typo'd clause keyword silently runs the statement with default semantics #296 — the graph DSL cursor claimed its tokens twice, so a mistyped clause key parsed as a value; the cursor now claims the tokens it reads and reports the mistyped key.
Multi-row INSERT emits one CommandComplete per row — drivers report rowcount 1; kv and timeseries DML answer a bare OK tag #297 — one INSERT 0 n per multi-row INSERT: plain document rows batch into one BatchInsert page; kv insert and put report an affected count and answer INSERT 0 n instead of a bare OK.
SEARCH synthetic columns are unresolvable from a derived table over a closed-schema collection #305 — s.distance over a SEARCH derived table on a closed schema: derived-relation inference declares the synthetic distance column when the query orders by vector_distance(...).
ARRAY vector literal loses every element on document_strict INSERT, reported as 0 elements under XX000 #306 — ARRAY[0.1, 0.2, 0.3] into a strict VECTOR(3): element coercion accepts Float/Integer/Decimal/numeric String and rejects a non-numeric element by index and type; the binary_tuple wrappers preserve BadRequest, so a client value error is never XX000.
INSERT..SELECT with a kv-engine source silently NULLs expression cells #311 — INSERT ... SELECT from a kv source copied 0 rows: the source scan routes on the collection's own engine (document and kv; other engines are refused by name).
Evaluate sequence accessors per row in SELECT-list contexts (0A000 today) #314 — per-row nextval(...) projections returned NULL: the value is stamped on the control plane at the response boundary, once per output row, across pgwire, native, and NDJSON responses. A from-less nextval projection evaluates; the stamp path records the session's currval; a missing sequence raises 42704 (the shaper keeps its own sqlstate instead of falling to XX000).
Timestamp unit conversion is per-path, so a join or a cross-node scan compares mismatched units #317 — every read path scaled stored milliseconds itself, so a join could compare milliseconds against microseconds: the Data Plane now carries stored milliseconds end to end and one conversion runs at the response boundary, including star reads and declared time-key columns.
A cross-field typeguard guard passes the DEFAULT gate at declaration and fails at every insert #318 — typeguard and convert identity: handlers resolve in the session database, CONVERT carries the source primary key, and a guard DEFAULT or VALUE that references another column is refused at declaration.

Also fixes two silent engine defects on the same paths:

kv scan projections — SELECT upper(v) FROM kv answered NULL: KvOp::Scan carries the projection list and computed columns and evaluates them per row.
materialized-sum pages — a multi-row INSERT into a source whose balance target is cross-shard timed out waiting for Calvin completion: such a page is un-batched to the per-row PointInsert shape (which settles correctly); purely co-resident pages keep their single page.

Behaviour changes

Path	Before	After
`SELECT`, `RETURNING`, native stream, NDJSON body with `nextval(...)`	NULL	one allocated value per output row
Window over a derived table	NULL cell	evaluated over the materialized rows, in the window's own order
Timeseries read after a join	unit depended on the path	milliseconds throughout, converted once at the boundary
`INSERT` with more than one row	one tag per row	one `INSERT 0 n`
`currval(...)` after a stamped `nextval(...)`	`not yet defined`	the last allocated value
`CONVERT` without the source key column	minted new row identities	refused with `42601`

Compatibility

Pre-1.0, no migration:

The response shaper gained a sequence-stamper parameter and every response path threads the session's sequence values. Internal to the crate.
A CONVERT that omits the source primary key fails instead of minting identities.
A multi-row INSERT into a materialized-sum source with a cross-shard target plans as per-row writes (the page shape is reserved for co-resident targets). Internal to the planner.

Validation

Check	Result
`cargo fmt --all -- --check`	clean
`cargo clippy --workspace --all-targets --all-features --profile ci -- -D warnings`	clean
static gates (`scripts/ci/*`)	11/11 pass
`cargo deny check`	advisories, bans, licenses, sources all ok (lockfile bumps `rustls` to 0.23.45 for RUSTSEC-2026-0285)
`cargo nextest run --workspace --all-features` (fast suite)	16,169 passed, 0 failed (5 flaky passed on retry, 3 leaky)
`cargo nextest run -p nodedb-cluster-tests --all-features`	363 passed, 4 skipped (the CI-profile exclusions)
`cargo check --target wasm32-unknown-unknown` trio	clean
fuzz targets (`scripts/ci/*`)	6/6 clean

Local parity reaches everything except CodeQL, release packaging, and secret
scanning, which need GitHub-hosted runners; those report as not run until the
PR CI does them.

Note on the test environment

nextest needs RUST_MIN_STACK=33554432, the value
.github/workflows/test.yml sets. On the 2 MiB default, two kv surrogate
tests (kv_incr_on_fresh_key_persists_a_real_surrogate,
kv_transfer_persists_two_distinct_surrogates) abort with a
tokio-rt-worker stack overflow — on clean main as well, so that gap
predates this branch. Four cluster tests excluded by .config/nextest.toml
from the CI profile (CI-only Calvin-completion hangs) remain excluded here.

Closes #295, closes #296, closes #297, closes #305, closes #306, closes #311, closes #314, closes #317, closes #318

The data-group replay gate aborted after a hardcoded 60s. A restart whose last log entry is an election no-op can sit one index short of commit_index for that whole budget, then fail startup. - add [server] data_group_recovery_timeout_ms, default 60000 - pass it through await_cluster_ready into await_data_group_recovery

Each value tuple became its own PointInsert task. One statement emitted INSERT 0 1 per row, and drivers read rowcount 1. - accumulate plain rows in insert/convert.rs and emit one BatchInsert - keep per-row tasks for ON CONFLICT DO NOTHING and CRDT rows

kv writes reported no affected count, so pgwire emitted a bare OK. - report affected = 1 in execute_kv_insert and execute_kv_put - map KvOp::Insert, KvOp::Put, KvOp::BatchPut to the INSERT tag

ARRAY[0.1, 0.2, 0.3] folds its literals to Decimal. extract_vector_floats kept Float and Integer, dropped the rest, and reported "got 0 elements" as XX000. - accept Float, Integer, Decimal, and numeric String elements - reject a non-numeric element by index and type - refuse non-finite values

The binary_tuple wrap sites turned every non-UnknownStrictField error into a Serialization error. A BadRequest surfaced as XX000 with no SQLSTATE. - preserve BadRequest at the four wrap sites

The document materializer scanned every source. A kv source read an empty document store, so INSERT ... SELECT copied no rows and reported INSERT 0 0. - route on the catalog engine in clone_materializer/auto_source.rs - kv normalizes the materialize-scan page to the copy entry shape - refuse columnar, timeseries, spatial, array by name

Derived-relation inference listed declared columns only. The response layer appends a synthetic distance cell, so s.distance failed 42703 on a closed schema. - append distance when ORDER BY carries vector_distance(...)

KvOp::Scan carried no projection list and no computed columns. Every expression projection over a kv source shaped to NULL. - add projection and computed_columns to KvOp::Scan - evaluate both in the kv scan handler; keep plain projections full-row - update all KvOp::Scan constructors - add the kv_select_expressions wire case

SELECT x/0 FROM (SELECT 1 AS x) s answered one NULL row. The same expression over a collection raised 22012. - evaluate computed columns per row on the materialized-row scan - lower aggregates over a non-Scan body to a ProviderScan sub-plan - carry window specs through the post-processor - add the derived_expression_errors wire case

The projection list named an unaliased expression by its lowercased SQL text; the window spec used the verbatim text. The alias match never fired and the projection shaped to NULL. - add unaliased_projection_alias and use it at every naming site

Partitions were consumed in arrival order. row_number numbered by arrival and the rank family compared wrong neighbours whenever the window ORDER BY differed from the query's. - sort each partition by the spec's order keys - evaluate keys on singleton partitions so ordering errors still fail

CREATE TYPEGUARD answered 42P01 on an existing collection. The handlers read DatabaseId::DEFAULT while collections live under the session database. - thread database_id through the seven handlers and validate_typeguard

CONVERT rebuilt the strict schema from the column list and dropped the source key. Every later insert failed "no resolved primary key". - mark the source key column in the converted schema - refuse a column list that omits it with 42601

A guard DEFAULT or VALUE naming another column became a strict column DEFAULT. That DEFAULT evaluates with no row in scope, so every insert raised UnevaluableDefault. - add default_expr_references_columns in nodedb-sql - refuse the guard at CONVERT, naming field and clause

EnRaiha · 2026-09-14T03:58:10Z

Superseded: settling locally before opening for review.

EnRaiha · 2026-09-15T03:28:53Z

Superseded by #325, which carries the settled branch at fdba3e3.

EnRaiha added 14 commits September 14, 2026 08:17

fix(pgwire): answer kv INSERT with the INSERT tag

d4c3458

kv writes reported no affected count, so pgwire emitted a bare OK. - report affected = 1 in execute_kv_insert and execute_kv_put - map KvOp::Insert, KvOp::Put, KvOp::BatchPut to the INSERT tag

fix(executor): keep value errors out of XX000

19f5234

The binary_tuple wrap sites turned every non-UnknownStrictField error into a Serialization error. A BadRequest surfaced as XX000 with no SQLSTATE. - preserve BadRequest at the four wrap sites

fix(sql): declare the SEARCH distance column

e764eed

Derived-relation inference listed declared columns only. The response layer appends a synthetic distance cell, so s.distance failed 42703 on a closed schema. - append distance when ORDER BY carries vector_distance(...)

fix(typeguard): resolve in the session database

7b84a84

CREATE TYPEGUARD answered 42P01 on an existing collection. The handlers read DatabaseId::DEFAULT while collections live under the session database. - thread database_id through the seven handlers and validate_typeguard

fix(convert): keep the source primary key

0324ecb

CONVERT rebuilt the strict schema from the column list and dropped the source key. Every later insert failed "no resolved primary key". - mark the source key column in the converted schema - refuse a column list that omits it with 42601

EnRaiha closed this Sep 14, 2026

EnRaiha deleted the pr/open-issues-20260914 branch September 14, 2026 03:58

EnRaiha changed the title ~~fix: resolve issues #295, #297, #305, #306, #311, #318 — SQL, kv, and strict-format correctness~~ fix: nine-issue batch — sequences, kv scans, windows, graph DSL, and one instant conversion Sep 15, 2026

EnRaiha mentioned this pull request Sep 15, 2026

fix: nine-issue batch — sequences, kv scans, windows, graph DSL, and one instant conversion #325

Closed

Navigation Menu

Uh oh!

fix: nine-issue batch — sequences, kv scans, windows, graph DSL, and one instant conversion - #324

fix: nine-issue batch — sequences, kv scans, windows, graph DSL, and one instant conversion#324
EnRaiha wants to merge 14 commits into
mainfrom
pr/open-issues-20260914

EnRaiha commented Sep 14, 2026 •

edited

Loading

Uh oh!

EnRaiha commented Sep 14, 2026

Uh oh!

EnRaiha commented Sep 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

EnRaiha commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Behaviour changes

Compatibility

Validation

Note on the test environment

Uh oh!

EnRaiha commented Sep 14, 2026

Uh oh!

EnRaiha commented Sep 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

EnRaiha commented Sep 14, 2026 •

edited

Loading