Skip to content

fix(graph): preserve recurrent graph inputs and allocation lifetimes - #1560

Draft
big-hip wants to merge 3 commits into
InfiniTensor:InfiniLM-v0.2.9cfrom
big-hip:fix/graph-replay-lifetimes
Draft

big-hip wants to merge 3 commits into
InfiniTensor:InfiniLM-v0.2.9cfrom
big-hip:fix/graph-replay-lifetimes

Conversation

@big-hip

@big-hip big-hip commented Sep 17, 2026

Copy link
Copy Markdown

Graph capture can reuse an allocator block without freezing it, trim reactivated graph storage, or omit reduction/scalar-power/MetaX concatenation operations from replay. These defects break indexed recurrent inference. The three commits repair allocation ownership, reduction recording, and MetaX last-axis concatenation separately using existing graph interfaces.

Branch: fix/graph-replay-lifetimes. Base: InfiniLM-v0.2.9c.

The diff contains eight files (+185/-39), including focused allocator, changed-input reduction, axis-cache and strided-concatenation regressions. It adds no graph compiler or Prefill capture feature.

Validation: current C500 slice tests passed for reduction/scalar-power replay (5) and concatenation (6); the same run separately checked the precision PR (2). Previous matching-source validation covered NVIDIA concatenation, reduction and allocator behavior. The current production source is byte-identical to that validated implementation. Format and whitespace checks passed. The new split branch has not undergone a fresh standalone accelerator build; combined-runtime tests do not replace that check.

  • Minimal scoped commits and regression coverage.
  • Formatting and whitespace checks.
  • Fresh build of the isolated branch and upstream CI.

Published as a draft while independent branch builds and upstream CI are pending.

Saved output, source provenance, device conditions and archived measurements: immutable evidence. Attachments live on a separate fork branch and are not part of this source diff.

Rendering of saved regression output; not a fresh CI run

CI at publication: fork Ruff passed; the independent fork CPU build/test matrix is in progress. The upstream run reports action_required and needs repository-side approval. No successful upstream build is claimed.

Freeze reused blocks during capture and remove reactivated storage from free lists before trimming. Add allocator lifetime regressions.
Keep reduction descriptors and workspace alive, key the cache by actual axes, and initialize scalar sums without capturing stack addresses. Cover changed-input graph replay.
Reuse the existing slice-copy path so replay updates concatenated inputs. Test graph producers and noncontiguous views across supported activation dtypes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant