feat!: add canonical tool execution results

[bbednarski9]

Overview

This PR makes one canonical tool-result contract apply across the runtime, bindings, plugins, and observability surfaces.

Warning

BREAKING CHANGE: [Tool execution APIs and plugin wire contracts] Tool producers, execution-intercept continuations, managed execution returns, and manual completion no longer exchange raw JSON; they now use the canonical ToolExecutionResult { result, annotation } contract. Update forwarding intercepts to preserve or deliberately replace/remove annotation, and update Rust, Python, Node.js, C/FFI, and Go call sites to the new result shape. Relay 0.8 resets the tool-result semantics under native_api = "1" and worker_protocol = "grpc-v1". Rebuild every native and worker plugin and declare compat.relay so it excludes versions before 0.8 (recommended: >=0.8.0,<1.0). Native ABI v4 layouts remain unchanged. Workers retain the grpc-v1 identifier and nemo.relay.worker.v1 package, service names, and RPC method names, but the protobuf contract changes: RelayHostRuntime.ToolNext now returns ToolExecutionResultResponse, and ToolExecutionInterceptResult.outcome is a typed ToolExecutionInterceptOutcome instead of a JsonEnvelope. Regenerate custom protobuf bindings before rebuilding workers.

Tool producers, execution continuations, managed execution, and manual completion now exchange:

ToolExecutionResult {
    result: Json,
    annotation: Option<Json>,
}

Tool execution intercepts return the flat companion contract:

ToolExecutionInterceptOutcome {
    result: Json,
    annotation: Option<Json>,
    pending_marks: Vec<PendingMarkSpec>,
}

• I confirm this contribution is my own work, or I have the right to submit it under this project's license.
• I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

• Replaces raw JSON tool results with ToolExecutionResult in Rust, Python, Node.js, C/FFI, Go, native plugins, and the Rust/Python worker SDKs.
• Keeps pending_marks Relay-owned while allowing each execution intercept to explicitly preserve, replace, remove, or short-circuit the opaque annotation.
• Normalizes missing and JSON-null annotations to absence without interpreting either the application-owned result or annotation value.
• Preserves complete MCP-style results—including content, structuredContent, _meta, and isError—inside result; Relay annotations remain adjacent to, and independent from, MCP _meta.
• Tool response sanitizers receive and sanitize only result; they cannot inspect or rewrite annotation. Relay then attaches each non-null annotation at category_profile.tool_result_annotation before the general scope-end event-sanitizer chain, which can replace or remove data, category_profile (including the annotation), and event metadata before subscribers and exporters receive the event.
• The first-party PII redaction plugin's built-in action policies apply tool_output to result and the projected annotation as independent JSON documents. target_paths are relative to each document; root removal omits the annotation key, while other typed tool profile fields remain unchanged.
• Sanitizers affect observability only; Relay returns the original, unsanitized ToolExecutionResult to the application.
• Projects annotations through ATOF, ATIF, full OpenTelemetry, and OpenInference as one opaque value. The GenAI projection intentionally omits the Relay-specific attribute.
• Keeps native plugin ABI v4 layouts and callback signatures unchanged while making canonical tool-result JSON the Relay 0.8 native API 1 baseline. Native tool-result envelopes continue to use the versioned schema tags defined beside the shared DTOs.
• Keeps the grpc-v1 protocol identifier and nemo.relay.worker.v1 package, service names, and RPC method names, while structurally defining the ToolExecutionResult and ToolExecutionInterceptOutcome wrappers in protobuf. Their application-owned result, annotation, and pending-mark array values use lossless JsonValue bytes rather than google.protobuf.Value so arbitrary JSON and integer precision are preserved.
• Changes RelayHostRuntime.ToolNext from JsonResult to ToolExecutionResultResponse and ToolExecutionInterceptResult.outcome from JsonEnvelope to typed ToolExecutionInterceptOutcome. All workers must regenerate bindings and rebuild for the Relay 0.8 baseline.
• Requires dynamic-plugin manifests and persisted records to declare a compat.relay range that excludes pre-0.8 Relay releases. Plugin loading separately retains the existing check that the range matches the running host.
• Rejects legacy raw producer results on public and dynamic-plugin boundaries rather than maintaining parallel frame APIs or ambiguous adapters.
• Leaves managed tool_call_id behavior unchanged in this PR; [Enhancement]: Add MCP and tool ID for tool-call provenance and correlation #446 remains a focused follow-up.

Breaking surfaces include callback return values, execution-intercept next, managed execute return values, manual tool completion input, and the two worker protobuf type seams above. Native plugin ABI v4 itself does not change.

Focused validation completed on the current implementation:

• cargo check -p nemo-relay-worker-proto -p nemo-relay-worker -p nemo-relay --features worker-grpc --locked
• cargo clippy -p nemo-relay-worker-proto -p nemo-relay-worker -p nemo-relay --all-targets --features worker-grpc -- -D warnings
• cargo fmt --all -- --check
• cargo test -p nemo-relay-worker-proto; cargo test -p nemo-relay-worker --test worker_sdk_tests; and focused host-worker continuation/error tests passed, including structural wrappers, lossless JSON, null normalization, malformed payloads, and pending marks
• Python worker SDK focused protocol and descriptor tests: 21 passed; Ruff passed
• Python worker example: 4 passed; added a managed-environment regression for ToolNext, result mutation, annotation preservation, and pending marks. The test compiled locally and runs end to end when the lifecycle-managed Python environment is provisioned
• Pinned grpcio-tools generation verified the ToolNext, tool-result, and outcome descriptors
• Fern documentation checks and strict broken-link validation passed
• git diff --check

The local fixture rebuild and full matrix were intentionally deferred because the Mac data partition had only 2.7 GiB free. CI runs the complete generated-binding, fixture, and cross-language matrix for this head.

Where should the reviewer start?

Start with crates/types/src/api/tool.rs for the canonical application contracts and crates/worker-proto/proto/nemo/relay/worker/v1/plugin_worker.proto for their structural worker representation. Then review crates/core/src/plugin/dynamic/worker.rs, crates/worker/src/lib.rs, and python/plugin/src/nemo_relay_plugin/_api.py for the local host/SDK conversions and enforcement. crates/worker-proto/src/lib.rs intentionally remains a generic generated-protocol and JSON-value helper crate.

For runtime semantics, review crates/core/src/api/runtime/state.rs and crates/core/src/api/tool.rs. crates/core/tests/integration/middleware_tests.rs captures annotation propagation, repeated/concurrent continuation behavior, pending-mark ownership, sanitization order, error behavior, and MCP-shaped result regressions.

For dynamic compatibility, review crates/core/src/plugin/dynamic.rs, crates/core/src/plugin/dynamic/native.rs, crates/core/src/plugin/dynamic/worker.rs, and docs/reference/migration-guides.mdx together. The key design choice is a Relay 0.8 cutover under the existing v1 identifiers: every plugin must rebuild and exclude pre-0.8 Relay versions. Native ABI v4 remains unchanged; workers retain the v1 identifier/package/API names but must regenerate bindings for the new structural tool-result protobuf types.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• Closes [Enhancement]: Standardize tool execution results across Relay #445
• Relates to [Enhancement]: Add MCP and tool ID for tool-call provenance and correlation #446

Summary by CodeRabbit

• New Features

  • Introduced a standardized tool execution result with a required result and optional opaque annotation.
  • Preserved annotations across callbacks, middleware, manual completion, SDKs, ATIF, and OpenTelemetry outputs.

• Breaking Changes

  • Tool callbacks and execution APIs now require the structured result format; legacy raw results are rejected.

• Updates

  • Dynamic plugins retain native API 1 and grpc-v1, with a Relay 0.8 rebuild requirement and updated migration guidance.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 0dd19324-ae7e-4470-936f-46ca2c824177

📥 Commits

Reviewing files that changed from the base of the PR and between b49ce6c and e0e438d.

📒 Files selected for processing (4)

• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs
• crates/cli/tests/coverage/shared/plugins_tests.rs

📜 Recent review details

⏰ Context from checks skipped due to timeout. (41)

• GitHub Check: Rust / Package (macos-arm64)
• GitHub Check: Rust / Package (windows-amd64)
• GitHub Check: Rust / Package (linux-arm64)
• GitHub Check: Python / Package (linux-amd64)
• GitHub Check: Rust / Package (linux-amd64)
• GitHub Check: Rust / Test (linux-arm64)
• GitHub Check: Rust / Package (linux-musl-amd64)
• GitHub Check: Rust / Test (macos-arm64)
• GitHub Check: Rust / Package (linux-musl-arm64)
• GitHub Check: Python / Package (linux-arm64)
• GitHub Check: Node.js / Package (linux-musl-amd64)
• GitHub Check: Rust / Test (windows-arm64)
• GitHub Check: Rust / Package (windows-arm64)
• GitHub Check: Node.js / Package (macos-arm64)
• GitHub Check: Node.js / Package (linux-arm64)
• GitHub Check: Node.js / Package (linux-musl-arm64)
• GitHub Check: Rust / Test (windows-amd64)
• GitHub Check: Python / Package (windows-amd64)
• GitHub Check: Rust / Test (linux-amd64)
• GitHub Check: Node.js / Package (windows-arm64)
• GitHub Check: Go / Test (linux-amd64)
• GitHub Check: Go / Test (windows-arm64)
• GitHub Check: Node.js / Package (windows-amd64)
• GitHub Check: Node.js / Test (linux-amd64)
• GitHub Check: Python / Package (macos-arm64)
• GitHub Check: Node.js / Test (windows-arm64)
• GitHub Check: Go / Test (macos-arm64)
• GitHub Check: Python / Package (linux-musl-arm64)
• GitHub Check: Go / Test (windows-amd64)
• GitHub Check: Node.js / Package (linux-amd64)
• GitHub Check: Python / Package (linux-musl-amd64)
• GitHub Check: Node.js / Test (windows-amd64)
• GitHub Check: Python / Package (windows-arm64)
• GitHub Check: Node.js / Test (linux-arm64)
• GitHub Check: Python / Test (windows-arm64)
• GitHub Check: Python / Test (linux-amd64)
• GitHub Check: Python / Test (windows-amd64)
• GitHub Check: Python / Test (linux-arm64)
• GitHub Check: Python / Test (macos-arm64)
• GitHub Check: Check / Run
• GitHub Check: Preview docs

🧰 Additional context used

📓 Path-based instructions (11)

**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-ffi-surface/SKILL.md)

**/*.rs: Run cargo fmt --all for all FFI work since it is Rust work
Run just test-rust to validate FFI changes
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting on FFI work

When Rust files changed as part of Go work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all when Rust files are changed as part of Node work
Run cargo clippy --workspace --all-targets -- -D warnings when Rust files are changed as part of Node work
Run just test-rust when Rust files are changed as part of Node work

**/*.rs: If any Rust code changed, always run just test-rust.
If any Rust code changed, also run cargo fmt --all.
If any Rust code changed, also run cargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, run cargo fmt --all and cargo clippy --workspace --all-targets -- -D warnings even if relying on pre-commit.

**/*.rs: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.

**/*.rs: Formatting: cargo fmt (rustfmt defaults)
Linting: cargo clippy -- -D warnings -- all warnings are treated as errors
Dependency auditing: cargo deny check -- configured in deny.toml

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, use maintain-dynamic-plugins and include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, prefer uv run pre-commit run --files <changed files...>.
Before review or handoff, run uv run pre-commit run --all-files.

**/*: Every commit in a pull request must include a Developer Certificate of Origin sign-off.
CI must pass before merging.
Use SONAR_IGNORE_START / SONAR_IGNORE_END only for documented false
positives that cannot be resolved in code or by improving the analyzer
configuration.
Keep the ignored block as small as possible, add a brief comment
explaining why the suppression is needed, and call it out in the PR description
so reviewers can explicitly sign off on it.
Keep the first line under 72 characters. Use the body for additional context when the change is not self-explanatory.

**/*: - [ ] Branch scope is coherent and reviewable

• Relevant tests passed under validate-change
• Docs and examples updated for any public behavior changes
• Pull request title follows Conventional Commit style and uses the correct
  type
  Use Conventional Commit style for PR titles:
  Only check the contribution confirmation boxes when they are true. If either
  confirmation cannot be made, stop before opening the PR and surface the blocker.

SPDX license header on any new files

**/*: Expose the new middleware surface in every affected binding.
Registration and duplicate-name behavior
Deregistration and no-op missing-name behavior
Ordering by priority
Callback failure policy, including fail-open behavior when required
Scope-local registrati...

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*.{rs,py,go,js,ts}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

If a language surface changed, always run that language's test target even when Rust core did not change.

**/*.{rs,py,go,js,ts}: Run tests for every language affected by your changes. If your change touches the core Rust crate, run tests across all bindings since they all depend on it.
When adding new functionality, include tests in the appropriate test files for each affected language binding.

**/*.{rs,py,go,js,ts}: - [ ] Does every OpenTelemetry endpoint require a type and nonblank destination?

• Does each endpoint resolve header_env values at activation and reject
  missing, blank, or duplicate headers?
• Are OpenTelemetry and OpenInference dependencies unconditional rather
  than Cargo feature-gated?
• Does enable_full_payloads preserve complete sanitized LLM request input
  and annotations while leaving credential removal and sanitizers active?
• Does Relay derive compliant trace and span IDs consistently across typed
  OpenTelemetry endpoints while preserving lifecycle parentage?
• Are mark events, start/end events, and orphan cases still handled correctly?

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*.{rs,py,js,mjs,ts,go,c,h}

📄 CodeRabbit inference engine (AGENTS.md)

Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*.{rs,py}

📄 CodeRabbit inference engine (AGENTS.md)

Follow binding naming conventions: Rust and Python snake_case, C FFI exports prefixed nemo_relay_, Go PascalCase for public APIs, Node.js camelCase.

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*.{rs,py,js,mjs,ts}

📄 CodeRabbit inference engine (AGENTS.md)

Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*.{rs,py,go,js,ts,html,md,mdx,toml}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

All source files must include an SPDX license header.

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*.{rs,c,h}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Use the naming conventions appropriate to each language: Rust snake_case, C FFI exports prefixed nemo_relay_, Go PascalCase, Node.js camelCase, Python snake_case.

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*.{rs,toml}

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

**/*.{rs,toml}: - [ ] Any Rust change ran just test-rust

• Any Rust change ran cargo fmt --all
• Any Rust change ran cargo clippy --workspace --all-targets -- -D warnings

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

**/*.{rs,py,pyi,go,js,ts}

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

Tests added in every affected language surface

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}

⚙️ CodeRabbit configuration file

{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.

Files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

🧠 Learnings (16)

📓 Common learnings

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:05.642Z
Learning: Which bindings actually expose the new surface?

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Use `karpathy-guidelines` alongside this skill for implementation or review
work. Keep changes scoped, surface assumptions, and define focused validation
before editing.

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Use this skill when contributing an integration with a framework or plugin such
as LangChain, LangGraph, Deep Agents, or OpenClaw through its public APIs.

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Keep NeMo Relay optional

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Use stable, documented framework or plugin APIs

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Wrap tool and LLM paths at the correct framework boundary

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Preserve the framework's original behavior when NeMo Relay is absent

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Integration uses public framework or plugin APIs

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Managed tool adapters return `ToolExecutionResult` to Relay and unwrap
      `.result` only at the framework boundary; opaque annotations are
      preserved through forwarding execution intercepts

Learnt from: CR
Repo: NVIDIA/NeMo-Relay

Timestamp: 2026-08-13T03:23:27.285Z
Learning: Relevant integration tests or smoke path pass

📚 Learning: 2026-08-07T22:42:49.565Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/test-rust-core/SKILL.md:0-0
Timestamp: 2026-08-07T22:42:49.565Z
Learning: Applies to crates/{plugin,worker,worker-proto,types}/**/* : If native dynamic plugins, gRPC workers, or the plugin, worker, worker-proto, or types crates change, also use `maintain-dynamic-plugins`.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-04T01:50:51.021Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/contribute-docs/SKILL.md:0-0
Timestamp: 2026-08-04T01:50:51.021Z
Learning: Applies to **/*.{md,mdx} : Dynamic plugin manifests in documentation and examples should use `compat.relay = ">=0.5,<1.0"` unless deliberately narrower.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-04T01:51:02.652Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/prepare-code-freeze/SKILL.md:0-0
Timestamp: 2026-08-04T01:51:02.652Z
Learning: Applies to crates/core/tests/fixtures/worker_plugin/Cargo.lock : Regenerate the worker-plugin fixture lockfile so its path dependencies use the new workspace version.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-04T01:51:02.652Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/prepare-code-freeze/SKILL.md:0-0
Timestamp: 2026-08-04T01:51:02.652Z
Learning: Applies to crates/core/tests/fixtures/worker_plugin/** : Build the worker-plugin fixture with `cargo build --locked --manifest-path crates/core/tests/fixtures/worker_plugin/Cargo.toml` after regenerating its lockfile.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-07T22:42:49.565Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/test-rust-core/SKILL.md:0-0
Timestamp: 2026-08-07T22:42:49.565Z
Learning: Applies to crates/{core,adaptive,plugin,worker,worker-proto,types}/**/* : Use narrower crate-specific tests only as a local debug loop, not as the final validation for a Rust change.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-07T22:42:49.565Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/test-rust-core/SKILL.md:0-0
Timestamp: 2026-08-07T22:42:49.565Z
Learning: Applies to crates/{core,adaptive,plugin,worker,worker-proto,types}/**/* : If a public API, event shape, middleware behavior, plugin semantics, or `crates/core`/`crates/adaptive` behavior changes, also run `validate-change`.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-07T22:42:49.565Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/test-rust-core/SKILL.md:0-0
Timestamp: 2026-08-07T22:42:49.565Z
Learning: Applies to crates/{core,adaptive,plugin,worker,worker-proto,types}/**/*.{rs,toml} : For changes in the Rust core, adaptive, dynamic plugin, worker, worker-proto, or types crates, run `cargo fmt --all`, `just test-rust`, and `cargo clippy --workspace --all-targets -- -D warnings` as the default validation sequence.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-12T18:42:21.566Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/maintain-dynamic-plugins/SKILL.md:0-0
Timestamp: 2026-08-12T18:42:21.566Z
Learning: Applies to {justfile,.github/**/*,codecov.yml} : - [ ] `justfile`, Codecov, and CI package/test workflows include new plugin
      crates and packages.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-07T22:42:49.565Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/test-rust-core/SKILL.md:0-0
Timestamp: 2026-08-07T22:42:49.565Z
Learning: Applies to crates/{core,adaptive,plugin,worker,worker-proto,types}/**/* : For changes affecting `crates/core`, `crates/adaptive`, or shared Rust runtime semantics, expand validation to the full binding matrix with `validate-change`.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-07T22:42:49.565Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/test-rust-core/SKILL.md:0-0
Timestamp: 2026-08-07T22:42:49.565Z
Learning: Applies to crates/{core,adaptive,plugin,worker,worker-proto,types}/**/* : If the change is isolated to one binding wrapper while Rust semantics remain unchanged, prefer that binding's build/test skill instead.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-12T18:42:21.566Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/maintain-dynamic-plugins/SKILL.md:0-0
Timestamp: 2026-08-12T18:42:21.566Z
Learning: Applies to crates/worker/**/*.rs : - [ ] Worker activation covers process launch, token auth, handshake, validation,
      declarative registration, proxy rollback, cancellation, and shutdown.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs

📚 Learning: 2026-08-03T19:55:03.931Z

Learnt from: afourniernv
Repo: NVIDIA/NeMo-Relay PR: 558
File: crates/pii-redaction/src/rampart/mod.rs:265-274
Timestamp: 2026-08-03T19:55:03.931Z
Learning: In NeMo Relay first-party plugin registration helpers, treat the documented duplicate-registration `PluginError::RegistrationFailed` result from `register_plugin` as success when registration is intended to be idempotent. Do not locally reclassify this as `PluginError::Conflict`; changing the classification requires a core-wide review of the public API and FFI behavior.

Applied to files:

• crates/cli/tests/coverage/shared/plugins_tests.rs
• crates/cli/tests/cli_tests.rs
• crates/cli/tests/coverage/shared/config_tests.rs
• crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs

📚 Learning: 2026-08-12T18:42:21.566Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/maintain-dynamic-plugins/SKILL.md:0-0
Timestamp: 2026-08-12T18:42:21.566Z
Learning: Applies to docs/**/*.md : - [ ] When detailed dynamic plugin guides exist, they keep Rust native, Python
      worker, and `grpc-v1` protocol details on separate pages.

Applied to files:

• crates/cli/tests/cli_tests.rs

📚 Learning: 2026-07-02T18:33:14.361Z

Learnt from: yczhang-nv
Repo: NVIDIA/NeMo-Relay PR: 355
File: crates/core/tests/unit/codec/parity_tests.rs:500-548
Timestamp: 2026-07-02T18:33:14.361Z
Learning: In `crates/core/tests/unit/codec/parity_tests.rs` (NeMo Relay, Rust), when comparing a computed cost/float value against a hand-derived literal in a comment, use a tolerance-based comparison (e.g., `assert!((total - expected).abs() < 1e-9)`) instead of `assert_eq!`. However, cross-codec parity assertions (e.g., `assert_eq!(chat_cost, anthropic_cost)`) should remain exact `assert_eq!` comparisons deliberately, since all codecs share one estimation code path and bit-identical output across codecs is the intended parity contract.

Applied to files:

• crates/cli/tests/coverage/shared/config_tests.rs

📚 Learning: 2026-05-26T21:03:12.012Z

Learnt from: CR
Repo: NVIDIA/NeMo-Relay PR: 0
File: .agents/skills/review-doc-style/SKILL.md:0-0
Timestamp: 2026-05-26T21:03:12.012Z
Learning: Applies to **/README.md|docs/index.md|python/nemo_relay/README.md|crates/*/README.md : Public behavior changes must be reflected in corresponding entry-point documentation (Must-Fix)

Applied to files:

• crates/cli/tests/coverage/shared/config_tests.rs

🔇 Additional comments (8)

crates/cli/tests/cli_tests.rs (2)

134-135: LGTM!

---

184-185: LGTM!

crates/cli/tests/coverage/shared/config_tests.rs (2)

106-107: LGTM!

---

509-510: LGTM!

crates/cli/tests/coverage/shared/plugins_lifecycle_tests.rs (3)

272-273: LGTM!

---

365-366: LGTM!

---

754-755: LGTM!

crates/cli/tests/coverage/shared/plugins_tests.rs (1)

70-71: LGTM!

---

Walkthrough

The pull request introduces canonical ToolExecutionResult values with optional opaque annotations across core execution, lifecycle events, observability, bindings, plugins, workers, tests, and documentation. Native API 1 remains available for legacy raw results. Worker integrations move to grpc-v2.

Changes

Canonical execution result contract

Layer / File(s)	Summary
Typed result and lifecycle propagation crates/types/..., crates/core/src/api/..., crates/core/src/observability/...	Tool execution APIs now use { result, annotation? }. Pending marks remain separate. Tool-end events and observability projections preserve non-null annotations.
Plugin and worker transport contracts crates/core/src/plugin/..., crates/worker/..., crates/worker-proto/..., crates/plugin/...	Native API 1 and 2 contracts are selected per plugin. Worker handshakes and continuations use grpc-v2 and versioned schemas.
Language binding APIs crates/ffi/..., crates/node/..., crates/python/..., go/..., python/...	Public APIs, callback bridges, typed codecs, manual completion, and continuations use ToolExecutionResult.
Validation and documentation crates/*/tests/..., docs/..., examples/..., integrations/...	Tests and documentation cover nested results, annotation propagation, legacy return rejection, sanitizer boundaries, migration, and grpc-v2.

Estimated code review effort: 5 (Critical) | ~120 minutes

Mergeability Score: 🟡 Moderate · up to e0e43

The PR introduces canonical tool-result propagation and changes observability sanitization, but sanitizer callback failures can still drop telemetry contrary to the required fail-open behavior, and a modified plugin README lacks the required license header. The observability issue should be fixed or explicitly accepted before merge.

Sequence Diagram(s)

sequenceDiagram
  participant ToolCallback
  participant InterceptChain
  participant ToolLifecycle
  participant Sanitizer
  participant Exporters
  ToolCallback->>InterceptChain: return ToolExecutionResult
  InterceptChain->>InterceptChain: preserve or modify result and annotation
  InterceptChain->>ToolLifecycle: return result and pending marks
  ToolLifecycle->>Sanitizer: sanitize result and annotation at separate stages
  ToolLifecycle->>Exporters: project sanitized tool result annotation

Loading

Possibly related issues

• [Enhancement]: Add MCP and tool ID for tool-call provenance and correlation #446 — Both changes use the canonical annotated ToolExecutionResult contract across tool execution integrations.

Possibly related PRs

• NVIDIA/NeMo-Relay#350 — Extends the tool-execution intercept outcome pipeline introduced by this PR.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The PR introduces a breaking canonical schema and protocol changes, but #445 requires an additive frame API that preserves raw-result compatibility and avoids a canonical schema.	Align the implementation with #445, or update the linked issue and scope to authorize the breaking canonical contract, protocol cutover, and broad binding changes.
Out of Scope Changes check	⚠️ Warning	Canonical schemas, public binding changes, protocol upgrades, observability changes, and raw-result incompatibility exceed the scope defined by #445.	Remove unrelated breaking and protocol changes, or link issues that explicitly cover the canonical result contract, binding migrations, worker upgrades, and observability projections.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check	✅ Passed	The title follows Conventional Commits format, uses the required breaking-change marker, stays under 72 characters, and accurately describes the main change.
Description check	✅ Passed	The description includes all required template sections, completed confirmations, reviewer guidance, detailed changes, validation notes, and related issue links.

✨ Finishing Touches 💡 1

⚔️ Resolve merge conflicts 💡

• Resolve merge conflict in branch feat/tool-result-annotations-445

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Fern docs preview: https://nvidia-preview-pull-request-575.docs.buildwithfern.com/nemo/relay/dev

[github-actions]

License Diff

Compared against origin/main.

Lockfile license changes

Lockfile License Changes

Rust

Added

• None

Removed

• None

Updated/Changed

• None

Node

Added

• None

Removed

• None

Updated/Changed

• None

Python

Added

• None

Removed

• None

Updated/Changed

• None

Status output

[license-diff] selected languages: rust, node, python
[license-diff] generating current inventory
[license-diff] current: generating Rust inventory
[license-diff] current: Rust inventory complete (448 packages)
[license-diff] current: generating Node inventory
[license-diff] current: Node inventory complete (367 packages)
[license-diff] current: generating Python inventory
[license-diff] current: Python inventory complete (105 packages)
[license-diff] current inventory complete
[license-diff] checking out base ref origin/main into a temporary worktree
[license-diff] base: generating Rust inventory
[license-diff] base: Rust inventory complete (448 packages)
[license-diff] base: generating Node inventory
[license-diff] base: Node inventory complete (367 packages)
[license-diff] base: generating Python inventory
[license-diff] base: Python inventory complete (105 packages)
[license-diff] base inventory complete
[license-diff] removing temporary base worktree
[license-diff] comparing inventories
[license-diff] rendering Markdown output
[license-diff] done

[mnajafian-nv]

Great work, I don’t have anything new beyond Will’s open review comments here. I’m good to approve once those are addressed.

[bbednarski9]

/merge

[willkill07]

/merge