feat(server): cache physical inputs across workspaces

[ViTeXFTW]

Problem

The persistent index cache introduced in #33 and optimized in #95 is still keyed by the complete workspace/base-root set. That makes the expensive analysis of an unchanged baseIniRoots input unusable as soon as the user opens another workspace, adds or reorders a root, or changes a root's role.

A cache per root would reduce some misses, but roots are discovery/configuration boundaries rather than ownership boundaries: roots can overlap, the same physical file can move between base and workspace roles, and a large root changing should not invalidate every unchanged file inside it.

• Closes [Bug] Data cache is workspace dependend. #105.

Design

Replace the root-set JSON cache with one shared, transactional cache of physical inputs:

• Cache identity is the canonical input path plus its size/mtime fingerprint and a producer identity derived from the actual configured schema and scanner payload ABI.
• Workspace/base-root role, root ordering, and overlap deduplication are applied from the current scan plan rather than persisted in cached payloads.
• An INI, W3D, or raw asset is one input. A BIG archive is one physical input whose payload contains its virtual entries.
• One SQLite database in WAL mode provides batched I/O, atomic updates, and safe sharing between language-server instances.
• Cached entries use bounded Postcard binary serialization instead of rebuilding and rewriting one large JSON manifest.
• Exact warm scans do not rewrite payloads; retention touches are throttled to once per day.

This retains the warm-start work from #95 while removing the root-set coupling. It also supersedes the old multi-file retention model from #93 and preserves the clear/rebuild behavior restored in #65.

Correctness and recovery

• Failed scans are never cached as empty successes, so inaccessible or malformed inputs are retried next time.
• Inputs are fingerprinted again after scanning and retried once if they changed during the read.
• A corrupt payload invalidates and rebuilds only that input.
• A corrupt SQLite database is replaced and rebuilt; cache failures otherwise degrade to uncached scanning.
• Clear/rebuild increments a database epoch, preventing an older in-flight scan from repopulating records after the clear completes.
• Producer changes isolate incompatible schema/scanner results.
• Legacy index-v*-<hash>.json files are removed only after the new store commits successfully.

Retention

• Records unused for 30 days expire independently.
• A 1 GiB logical payload budget removes least-recently-used records first.
• Files not owned by ZeroSyntax remain untouched.

Performance

Local release-mode regression test with 256 tiny INI inputs:

Scan	Time	Payload rewrites
Cold	~90 ms	256
Warm	~43 ms	0

The warm path still performs directory discovery and inexpensive file metadata checks, but it avoids rereading, parsing, extracting, serializing, and rewriting unchanged inputs. Manual testing against real game data also confirmed a large startup improvement.

Tests

New regression coverage includes:

• reuse of base inputs across workspaces and changed root sets;
• current base/workspace role and root ordering on cache hits;
• overlapping-root deduplication;
• schema/producer invalidation;
• isolated changed-file and corrupt-payload recovery;
• BIG archives as one cached physical input;
• failed-input retry behavior;
• concurrent writers and clear-vs-in-flight-write safety;
• expiration and LRU budget pruning;
• 256 inputs sharing one store with zero warm payload rewrites.

Validation:

• cargo test — 205 passed, 2 corpus-dependent tests ignored
• cargo clippy --workspace --all-targets -- -D warnings
• cargo fmt --all -- --check
• server stdio E2E suite — all checks passed

[greptile-apps]

Greptile Summary

The PR replaces root-set JSON caches with a shared SQLite cache keyed by physical input identity and producer ABI.

• Reuses unchanged inputs across workspace and root-role changes.
• Adds transactional writes, epoch-based clearing, corruption recovery, and per-record retention.
• Uses bounded Postcard serialization for cached scan payloads.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the maintained serialization replacement is complete, and the bounded flavor prevents cache encoding from allocating beyond the configured payload envelope.

Important Files Changed

Filename	Overview
crates/server/src/scan.rs	Integrates physical-input cache lookup and storage, stable-file rescanning, and bounded Postcard serialization; the prior payload-limit concern is addressed.
crates/server/src/cache.rs	Implements the transactional SQLite store, producer isolation, epoch-protected clearing, corruption recovery, and retention pruning.
crates/server/src/backend.rs	Updates cache-path reporting and clear/rebuild commands to operate on the shared cache.
Cargo.toml	Adds the SQLite, Postcard, and hashing dependencies required by the new cache implementation.

_{Reviews (3): Last reviewed commit: "fix(server): bound cache payload seriali..." | Re-trigger Greptile}

[ViTeXFTW]

@greptileai

[ViTeXFTW]

@greptileai