perf: bound KV page reclamation candidate scans

[T4t4KAU]

Summary

Stop collecting KV eviction candidates once enough unreferenced pages have been found to satisfy the requested free capacity. Candidate collection still finishes before the used-page set is mutated.

Motivation

A request needing one or a few pages previously scanned every used page and built a list of every unreferenced page. Bounded collection avoids the unused tail while retaining the existing eviction order and index cleanup.

Type of Change

• perf — performance improvement with unchanged behavior

Test Results of Involved Models on Supported Platforms

Local CPU validation covered capacity boundaries, existing free pages, shared hashes with live peers, failed reclamation, bounded metadata reads, and 4,000 seeded randomized reclaim/reallocate steps against the previous implementation. Return values, free-list order, used-page membership, hashes, and reference counts matched. These checks were performed during development; no unit-test or test-support files are included in the final PR diff. Repository formatting and git diff --check passed.

Single-request/offline/sanity/service model tests were not run: this is Python cache-metadata validation without local model weights. Accelerator and model throughput remain unmeasured.

Benchmark / Performance Impact

Intel Core i5-14600KF, Python 3.12.13, integer metadata (dtype N/A), CPU not pinned. Baseline uses the original try_free_blocks from 270feb3. Each pool starts fully allocated, with the first indicated pages unreferenced and indexed using three repeating hashes. Only try_free_blocks is timed with perf_counter_ns; reclaimed pages are reallocated, republished, and released outside timing. Ten warmups, median of seven rounds of 100 calls. Exact state/output comparison precedes timing. Temporary local harness is not committed.

Pool pages	Unreferenced	Requested free pages	Before (µs)	After (µs)
512	256	1	12.524	0.655
8192	4096	1	119.514	0.379
8192	4096	16	122.344	3.744
32768	16384	16	483.420	3.526
8192	0	1	110.024	109.478
8192	4096	8192 (fails)	881.324	956.735

The favorable cases find candidates early. Worst-case work remains linear; the insufficient-capacity case measured about 8.6% slower due to the additional limit check per candidate. These are microbenchmarks, not model throughput.

Notes for Reviewers

Directly based on 270feb3 / InfiniLM-v0.2.9c, independent of other optimization PRs. When free capacity already suffices, the legacy behavior still evicts one unreferenced page if one exists. Failed reclamation still evicts all available candidates before returning false. No live page is evicted.

CI / ChatOps

Manual CI has not been triggered; model/platform validation remains outstanding.

Checklist

• Conventional Commits title/commit, matching branch, squashable commit history, no merge/fixup/WIP commits.
• Direct base is the requested InfiniLM-v0.2.9c release branch rather than main.
• N/A — legacy issue-format exception.
• Minimal scope; no debug/dead code, unrelated formatting, or public API changes.
• English comments/docstrings, final newlines, and whitespace checked.
• Python files checked using scripts/format.py; no auto_config.py changes.
• N/A — C++ checks/native build: Python-only changes.
• CPU correctness and performance checks completed; skipped model tests explained above.
• Additional platform validation and reviewer assignment.
• Manual CI run.
• N/A — documentation/breaking-change notices: external behavior and workflows unchanged.
• No secrets, new third-party code, or unsafe memory operations.