v0.1.10-3: Define a Versioned LMCache-Aligned KDN Management Protocol

[rickisba]

Background

The current KDN service mixes knowledge metadata, file handling, and Legacy Redis injection. The long-term architecture must not turn this into a second KVCache storage system. LMCache already owns physical cache storage, backend selection, block lookup, load/save, movement, and cache-maintenance mechanisms.

CacheRoute needs a versioned management contract that expresses knowledge semantics and desired operations while adapting to LMCache's available controller, lookup, MP HTTP API, and event interfaces.

Objective

Define versioned, storage-neutral protocol models and API semantics for:

• LMCache endpoint/capability registration and health;
• lookup and observed-residency queries;
• logical operation submission for publish/save, prefetch/load, move/copy, pin/unpin, clear/evict, verify, and cancel where supported;
• leases or operation tickets when asynchronous coordination requires them;
• status query and structured result reporting;
• idempotency and protocol-version negotiation.

The protocol is a CacheRoute management facade over LMCache capabilities, not a new physical data-plane protocol or storage engine.

Implementation Scope

• Define versioned request and response models for endpoint capabilities, lookup observations, operation submission, status, cancellation, results, and idempotency.
• Represent LMCache capability differences explicitly; an unsupported operation must return a structured capability error rather than silently falling back to a CacheRoute-owned implementation.
• Model logical references to LMCache-managed objects or token ranges without exposing raw KV payloads, backend credentials, internal Redis keys, or LMCache-private serialization.
• Separate desired state/operation intent from observed LMCache state.
• Define freshness, generation, timestamp, and observation-source fields for lookup/status results.
• Support adapters for the interfaces available in the deployment, such as LMCache Controller APIs, LMCache MP HTTP APIs, lookup APIs, and cache events.
• Preserve the existing Legacy Redis path behind an explicitly named compatibility adapter.
• Provide a mock/in-process adapter suitable for CPU-only CI.
• Keep v0.1.10 focused on contracts; implementation of production adapters belongs to later roadmap releases.

Acceptance Criteria

• Every new management message carries an explicit protocol version.
• Unsupported protocol versions and unsupported LMCache capabilities fail with structured errors.
• Lookup/status responses identify source, timestamp/freshness, endpoint generation, and compatibility identity where available.
• Repeated logical operation submission with the same idempotency key does not create duplicate CacheRoute work.
• Lease or operation-ticket expiration has deterministic behavior where used.
• Protocol payload tests prove that no KV binary data, backend password, connection credential, raw Redis key, or LMCache-private serialized object is exposed.
• The contract can represent at least lookup, prefetch/load, move/copy, pin/unpin, clear/evict, verify/status, and result reporting without requiring every LMCache deployment to support every operation.
• Legacy Redis injection remains functional through the existing compatibility path.
• Automated tests cover versioning, idempotency, capability negotiation, lookup observations, asynchronous results, errors, and secret rejection.

Non-Goals

• Implementing a CacheRoute-owned KVCache store, block index, storage tier, serialization format, eviction engine, or data-transfer protocol.
• Reimplementing LMCache Controller or MP services.
• Running KDN and an independent CacheRoute Data Worker as production storage services.
• Implementing replication, migration, admission, or prefetch policy; this issue only defines mechanism contracts.

Parent Epic

• v0.1.10 Epic: Contract and Observability Baseline #137

Dependencies

• Completed v0.1.10-1: Add Instance Capability Fingerprints and Compatibility-Aware Registration #138 for Instance/LMCache capability identity.
• v0.1.10-2: Define Storage-Neutral Artifact, Replica, DataPlaneTask, and QueueWork State Models #139 for storage-neutral lifecycle and task states.
• v0.1.10-4: Add Unified Request Tracing and Queue-Stage Observability #141 for trace and observation provenance.