Skip to content

feat(chat): validate custom agent client data - #4646

Open
gtremper wants to merge 2 commits into
triggerdotdev:mainfrom
gtremper:graham/custom-agent-validation
Open

feat(chat): validate custom agent client data#4646
gtremper wants to merge 2 commits into
triggerdotdev:mainfrom
gtremper:graham/custom-agent-validation

Conversation

@gtremper

@gtremper gtremper commented Aug 17, 2026

Copy link
Copy Markdown

Summary

chat.withClientData({ schema }).customAgent() now validates and parses payload.metadata before passing it to run, chat.messages, or chat.createSession.

Previously, the schema only provided types for custom agents, so consumers had to repeat the validation themselves. Schema defaults and transforms are now preserved in the value passed to user code.

Custom agents without a schema keep the existing pass-through behavior. This does not change chat.agent(). customAgent() does not currently expose an action schema.

Validation failures

Invalid client data is logged and never passed to user code.

  • Async reads write an error chunk followed by turn-complete, then wait for the next frame.
  • An invalid initial payload waits for a valid frame before calling run.
  • chat.messages.on() skips the frame and calls onClientDataValidationError without ending an active response.
  • chat.messages.peek() throws synchronously.
  • Invalid head-start payloads drain the handover signal before recovery. A skipped handover ends the run; a partial handover is discarded with a warning.

Validation is automatic when a schema is declared, matching the managed agent behavior. This could be changed to an opt-in flag or typed failure result if maintainers prefer a different contract.

Testing

  • pnpm --filter @trigger.dev/sdk run test -- --run
  • pnpm --filter @trigger.dev/sdk run typecheck
  • pnpm run build --filter @trigger.dev/sdk
  • pnpm run lint
  • Formatting checks pass

✅ Checklist

  • I followed the contributing guide
  • The PR title follows the convention
  • I ran and tested the change

Changelog

Custom chat agents now validate and parse client data declared with chat.withClientData({ schema }) before passing it to agent code.

Screenshots

Not applicable.

@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The SDK adds schema-based validation and parsing for custom-agent client data. Invalid initial and subsequent frames are excluded from agent code, with configurable validation-error reporting. Message APIs and subscriptions now validate metadata and preserve stream behavior. ChatTurn and createChatSession carry generic client-data types. Documentation describes managed-agent and custom-agent behavior, and tests cover parsing, retries, ordering, buffering, callbacks, handover recovery, and schema-free pass-through.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the primary change: custom agent client-data validation.
Description check ✅ Passed The description covers the change, validation behavior, testing, checklist, changelog, and screenshots; only the issue-closing line is missing.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Hi @gtremper, thanks for your interest in contributing!

This project requires that pull request authors are vouched, and you are not in the list of vouched users.

This PR will be closed automatically. See https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md for more details.

@github-actions github-actions Bot closed this Aug 17, 2026
@gtremper gtremper changed the title feat(chat): runtime clientData validation for custom agents feat(chat): validate custom agent client data Aug 17, 2026
@matt-aitken matt-aitken reopened this Aug 17, 2026
@changeset-bot

changeset-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 80790a7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 29 packages
Name Type
@trigger.dev/sdk Patch
@trigger.dev/python Patch
@internal/dashboard-agent Patch
@internal/sdk-compat-tests Patch
@trigger.dev/build Patch
@trigger.dev/core Patch
@trigger.dev/react-hooks Patch
@trigger.dev/redis-worker Patch
@trigger.dev/rsc Patch
@trigger.dev/schema-to-json Patch
@trigger.dev/database Patch
@trigger.dev/otlp-importer Patch
@trigger.dev/rbac Patch
@trigger.dev/sso Patch
trigger.dev Patch
@internal/cache Patch
@internal/clickhouse Patch
@internal/llm-model-catalog Patch
@internal/metrics-pipeline Patch
@internal/redis Patch
@internal/replication Patch
@internal/run-engine Patch
@internal/run-store Patch
@internal/schedule-engine Patch
@internal/tracing Patch
@internal/tsql Patch
@internal/webhook-engine Patch
@internal/webhook-sources Patch
@internal/testcontainers Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@matt-aitken
matt-aitken marked this pull request as ready for review August 17, 2026 08:38

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/trigger-sdk/src/v3/ai.ts (1)

1456-1502: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider moving the synchronous schema adapter next to getSchemaParseFn.

This function mirrors getSchemaParseFn in packages/core/src/v3/types/schemas.ts (arktype, valibot, zod, yup, superstruct, scale branches) and only removes the parseAsync branch. When core adds support for a new schema library, this copy will silently fall through to the "cannot validate" error. A shared getSyncSchemaParseFn in core keeps the two adapters aligned.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 385a5718-97fe-480c-9d22-ff1beb403bd2

📥 Commits

Reviewing files that changed from the base of the PR and between 6e77102 and 80790a7.

📒 Files selected for processing (7)
  • .changeset/quiet-chats-validate.md
  • docs/ai-chat/client-protocol.mdx
  • docs/ai-chat/custom-agents.mdx
  • docs/ai-chat/reference.mdx
  • docs/ai-chat/types.mdx
  • packages/trigger-sdk/src/v3/ai.ts
  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts

Included review availability: Your plan includes up to 10 reviews per rolling hour; 9 remain after this review.

📜 Review details
⏰ Context from checks skipped due to timeout. (29)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (3, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (5, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (7, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (6, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (8, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (4, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (2, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (12, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (9, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (10, 12)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (11, 12)
  • GitHub Check: sdk-compat / Node.js 22.23 (warp-ubuntu-latest-x64-4x)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (1, 12)
  • GitHub Check: sdk-compat / Deno Runtime
  • GitHub Check: sdk-compat / Node.js 26.4 (warp-ubuntu-latest-x64-4x)
  • GitHub Check: sdk-compat / Node.js 20.20 (warp-ubuntu-latest-x64-4x)
  • GitHub Check: sdk-compat / Node.js 24.18 (warp-ubuntu-latest-x64-4x)
  • GitHub Check: sdk-compat / Bun Runtime
  • GitHub Check: sdk-compat / Cloudflare Workers
  • GitHub Check: packages / 🧪 Unit Tests: Packages (1, 3)
  • GitHub Check: e2e-webapp / 🧪 E2E Tests: Webapp
  • GitHub Check: runops-guard / runops-guard
  • GitHub Check: fk-cascade-guard / fk-cascade-guard
  • GitHub Check: internal / 🧪 Unit Tests: Internal
  • GitHub Check: packages / 🧪 Unit Tests: Packages (3, 3)
  • GitHub Check: packages / 🧪 Unit Tests: Packages (2, 3)
  • GitHub Check: typecheck / typecheck
  • GitHub Check: code-quality / code-quality
  • GitHub Check: check-broken-links
🧰 Additional context used
📓 Path-based instructions (9)
docs/**/*.mdx

📄 CodeRabbit inference engine (docs/CLAUDE.md)

docs/**/*.mdx: MDX documentation pages must include frontmatter with title (required), description (required), and sidebarTitle (optional) in YAML format
Use Mintlify components for structured content: , , , , , , /, /
Always import from @trigger.dev/sdk in code examples (never from @trigger.dev/sdk/v3)
Code examples must be complete and runnable where possible
Use language tags in code fences: typescript, bash, json

Files:

  • docs/ai-chat/client-protocol.mdx
  • docs/ai-chat/types.mdx
  • docs/ai-chat/reference.mdx
  • docs/ai-chat/custom-agents.mdx
packages/trigger-sdk/**/*.{ts,tsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code

Files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
**/*.{ts,tsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead

**/*.{ts,tsx}: Prefer static imports over dynamic imports. Only use dynamic import() when:

  • Circular dependencies cannot be resolved otherwise
  • Code splitting is genuinely needed for performance
  • The module must be loaded conditionally at runtime
    Zod is pinned to a single version across the entire monorepo (currently 3.25.76). When adding zod to a new or existing package, use the exact same version as the rest of the repo - never a different version or a range. Mismatched zod versions cause runtime type incompatibilities (e.g., schemas from one package can't be used as body validators in another).
    Do not reintroduce V1.
    Add crumbs as you write code — not just when debugging.
    Do not invent new namespaces — pick from this table or ask first.

Files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use function declarations instead of default exports

Files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
**/*.{test,spec}.{ts,tsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use vitest for all tests in the Trigger.dev repository

We use vitest exclusively. Never mock anything - use testcontainers instead.

Files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
**/*.ts

📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)

**/*.ts: When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs
Do not use high-cardinality attributes in OTEL metrics such as UUIDs/IDs (envId, userId, runId, projectId, organizationId), unbounded integers (itemCount, batchSize, retryCount), timestamps (createdAt, startTime), or free-form strings (errorMessage, taskName, queueName)
When exporting OTEL metrics via OTLP to Prometheus, be aware that the exporter automatically adds unit suffixes to metric names (e.g., 'my_duration_ms' becomes 'my_duration_ms_milliseconds', 'my_counter' becomes 'my_counter_total'). Account for these transformations when writing Grafana dashboards or Prometheus queries

Files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
packages/trigger-sdk/**/*.{js,ts,jsx,tsx}

📄 CodeRabbit inference engine (packages/trigger-sdk/CLAUDE.md)

Always import from @trigger.dev/sdk. Never use @trigger.dev/sdk/v3 (deprecated path alias)

Files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
packages/**/*.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

packages/**/*.{ts,tsx}: - Public packages (packages/*): Use build.
Always import from @trigger.dev/sdk. Never use @trigger.dev/sdk/v3 or deprecated client.defineJob.

Files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
**/*.test.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Test files go next to source files (e.g., MyService.ts -> MyService.test.ts).

Files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
🧠 Learnings (21)
📚 Learning: 2026-03-10T12:44:14.176Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 3200
File: docs/config/config-file.mdx:353-368
Timestamp: 2026-03-10T12:44:14.176Z
Learning: In the trigger.dev repo, docs PRs are often companions to implementation PRs. When reviewing docs PRs (MDX files under docs/), check the PR description for any companion/related PR references and verify that the documented features exist in those companion PRs before flagging missing implementations. This ensures docs stay in sync with code changes across related PRs.

Applied to files:

  • docs/ai-chat/client-protocol.mdx
  • docs/ai-chat/types.mdx
  • docs/ai-chat/reference.mdx
  • docs/ai-chat/custom-agents.mdx
📚 Learning: 2026-04-30T20:30:29.458Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3226
File: docs/ai-chat/quick-start.mdx:13-13
Timestamp: 2026-04-30T20:30:29.458Z
Learning: In this repo’s documentation MDX files (`docs/**/*.mdx`), use `ts` and `tsx` (not `typescript`) as the code-fence language tags for TypeScript/TSX snippets. Do not flag `ts`/`tsx` code-fence language tags as incorrect in any docs MDX file, since this is the site-wide Mintlify-compatible convention.

Applied to files:

  • docs/ai-chat/client-protocol.mdx
  • docs/ai-chat/types.mdx
  • docs/ai-chat/reference.mdx
  • docs/ai-chat/custom-agents.mdx
📚 Learning: 2026-06-16T13:14:09.440Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3964
File: docs/ai-chat/reference.mdx:482-482
Timestamp: 2026-06-16T13:14:09.440Z
Learning: When documenting or reviewing usage of `ChatTurn.complete(source?)` (in `packages/trigger-sdk/src/v3/ai.ts`), note that `source` is optional (`source?: UIMessageStreamable`). Calling `complete()` with no `source` is valid specifically for a final head-start handover (`handover.isFinal`), because the warm partial already contains the response. If examples or guidance omit `source`, ensure they are in this final-hand-over context so they remain correct.

Applied to files:

  • docs/ai-chat/client-protocol.mdx
  • docs/ai-chat/types.mdx
  • docs/ai-chat/reference.mdx
  • docs/ai-chat/custom-agents.mdx
📚 Learning: 2026-06-16T13:14:14.382Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3964
File: docs/ai-chat/reference.mdx:478-478
Timestamp: 2026-06-16T13:14:14.382Z
Learning: When reviewing RC-gated `ai-chat` docs under `docs/ai-chat/`, don’t immediately flag missing SDK type fields or implementation details just because the field isn’t present on the docs branch yet. Instead, find and cross-check the companion implementation PR that’s intended to land alongside the docs PR, and only report missing/incorrect fields if they are also absent in the companion SDK/type changes.

Applied to files:

  • docs/ai-chat/client-protocol.mdx
  • docs/ai-chat/types.mdx
  • docs/ai-chat/reference.mdx
  • docs/ai-chat/custom-agents.mdx
📚 Learning: 2026-03-22T13:26:12.060Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3244
File: apps/webapp/app/components/code/TextEditor.tsx:81-86
Timestamp: 2026-03-22T13:26:12.060Z
Learning: In the triggerdotdev/trigger.dev codebase, do not flag `navigator.clipboard.writeText(...)` calls for `missing-await`/`unhandled-promise` issues. These clipboard writes are intentionally invoked without `await` and without `catch` handlers across the project; keep that behavior consistent when reviewing TypeScript/TSX files (e.g., usages like in `apps/webapp/app/components/code/TextEditor.tsx`).

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-03-22T19:24:14.403Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 3187
File: apps/webapp/app/v3/services/alerts/deliverErrorGroupAlert.server.ts:200-204
Timestamp: 2026-03-22T19:24:14.403Z
Learning: In the triggerdotdev/trigger.dev codebase, webhook URLs are not expected to contain embedded credentials/secrets (e.g., fields like `ProjectAlertWebhookProperties` should only hold credential-free webhook endpoints). During code review, if you see logging or inclusion of raw webhook URLs in error messages, do not automatically treat it as a credential-leak/secrets-in-logs issue by default—first verify the URL does not contain embedded credentials (for example, no username/password in the URL, no obvious secret/token query params or fragments). If the URL is credential-free per this project’s conventions, allow the logging.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-05-18T08:21:27.694Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3632
File: apps/webapp/sentry.server.ts:4-21
Timestamp: 2026-05-18T08:21:27.694Z
Learning: When handling Prisma error P1001 ("Can't reach database server") in TypeScript, don’t assume a single error shape. Prisma can surface P1001 via two different error classes/fields: `PrismaClientKnownRequestError` exposes it as `err.code === "P1001"` (common during mid-query connection drops), while `PrismaClientInitializationError` exposes it as `err.errorCode === "P1001"` (common on client startup failure). Therefore, predicates should use `err.code === "P1001" || err.errorCode === "P1001"`. Do not flag `err.code === "P1001"` as “unreachable/never matches,” as it is expected in production.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-05-18T08:21:27.694Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3632
File: apps/webapp/sentry.server.ts:4-21
Timestamp: 2026-05-18T08:21:27.694Z
Learning: When handling Prisma errors for P1001 ("Can't reach database server"), do not assume it only appears under a single property name. Prisma may surface P1001 via either `PrismaClientKnownRequestError` (`err.code === "P1001"`, e.g., mid-query connection drops) or `PrismaClientInitializationError` (`err.errorCode === "P1001"`, e.g., client startup connection failure). To reliably detect the condition, check `err.code === "P1001" || err.errorCode === "P1001"`, and avoid review rules that would incorrectly flag `err.code === "P1001"` as unreachable/never-matching.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-06-13T19:53:13.759Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3937
File: packages/trigger-sdk/skills/realtime-and-frontend/SKILL.md:258-260
Timestamp: 2026-06-13T19:53:13.759Z
Learning: When reviewing code that uses `trigger.dev/react-hooks`’s `useRealtimeRun`, preserve the call signature where the first argument is the full realtime handle object (not `handle.id`). This is intentional to maintain type-safety and is consistent with the official docs; do not suggest changing the first argument from the handle object to `handle.id`.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-06-17T17:13:49.929Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 3948
File: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.bulk-actions.$bulkActionParam/route.tsx:48-62
Timestamp: 2026-06-17T17:13:49.929Z
Learning: In triggerdotdev/trigger.dev, within `dashboardLoader`/`dashboardAction` (or similar context resolver code) whenever you resolve an organization ID from an organization slug for RBAC/enterprise authorization scope, always read from the primary Prisma client (`prisma`), not `$replica`. Using `$replica` can hit replica-lag and cause the RBAC lookup/authorization to run without the correct org scope (bypassing intended role enforcement). Implement the slug→org lookup with `prisma.organization.findFirst(...)` (or equivalent primary-client query) and add an inline comment documenting why the primary client is required (replica lag could lead to unscoped RBAC checks).

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-06-23T13:04:21.413Z
Learnt from: carderne
Repo: triggerdotdev/trigger.dev PR: 4023
File: apps/webapp/app/services/upsertBranch.server.ts:14-18
Timestamp: 2026-06-23T13:04:21.413Z
Learning: In TypeScript, it’s valid to `import { type X }` and then use `typeof X` in a type-only position, e.g. `type Alias = z.infer<typeof X>`. The `type` modifier suppresses the runtime import, but the type checker still has the full exported type so `z.infer<typeof X>` can resolve correctly. In code reviews, don’t flag this as a TypeScript compile error as long as `typeof X` is used in a type context (e.g., with `z.infer`, `type` aliases, generics), not as a runtime value.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-05-18T14:40:02.173Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3658
File: packages/core/src/v3/realtimeStreams/manager.test.ts:1-147
Timestamp: 2026-05-18T14:40:02.173Z
Learning: In this repo’s trigger.dev codebase, the “never mock — use testcontainers” guideline should only be applied to integration tests that talk to real external services (e.g., Redis, Postgres, S2). For unit tests that validate in-memory logic (e.g., deduplication/cache behavior in StandardRealtimeStreamsManager and similar module-boundary call counting), it is allowed to use Vitest mocks like `vi.fn()` and to stub/mock `ApiClient` objects to count calls or simulate in-process collaborators. Do not flag `vi.fn()`-based mocks as policy violations in these unit-test scenarios; reserve the rule for true external-service integration tests.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
📚 Learning: 2026-05-18T14:40:02.173Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3658
File: packages/core/src/v3/realtimeStreams/manager.test.ts:1-147
Timestamp: 2026-05-18T14:40:02.173Z
Learning: In the triggerdotdev/trigger.dev repo, the policy “Never mock anything — use testcontainers instead” should only be enforced for integration tests that interact with real external services (e.g., Redis, Postgres) via actual infrastructure. For unit tests that exercise pure in-memory logic (e.g., cache semantics) it is OK to stub collaborators such as `ApiClient` using Vitest (`vi.fn()`) to assert call counts or control behavior. Do not flag `vi.fn()`-based `ApiClient` stubs in unit tests as violations of the testcontainers policy.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
📚 Learning: 2026-05-19T22:37:47.286Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3671
File: packages/trigger-sdk/test/recovery-boot.test.ts:456-457
Timestamp: 2026-05-19T22:37:47.286Z
Learning: In `packages/trigger-sdk` (Trigger.dev SDK), `logger.warn` (and other SDK logger methods) should route to the Trigger.dev structured logger sink, not to `console.warn`. In SDK tests, `vi.spyOn(console, "warn")` (or similar console spies) should only be used to suppress stray console output; reviewers should not suggest asserting on `console.warn` spies to verify SDK-internal warning/fallback log behavior. Use the SDK’s structured-logger outputs/capture approach instead of console spies.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-06-04T18:16:35.386Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 3836
File: apps/supervisor/src/backpressure/backpressureMonitor.ts:3-5
Timestamp: 2026-06-04T18:16:35.386Z
Learning: When reviewing TypeScript in this repo, apply the rule “prefer type aliases over interfaces” only to data/object shapes and union/intersection type modeling. If an interface is being used as a behavioral contract for collaborators to implement (e.g., method-shape interfaces that define required behavior, such as `BackpressureLogger` / `BackpressureSignalSource` in `apps/supervisor/src/backpressure/backpressureMonitor.ts`), keep it as an `interface` and do not flag it as a type-alias-vs-interface violation.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-06-09T17:58:04.699Z
Learnt from: 0ski
Repo: triggerdotdev/trigger.dev PR: 3879
File: apps/webapp/app/models/vercelIntegration.server.ts:619-630
Timestamp: 2026-06-09T17:58:04.699Z
Learning: In this codebase, outbound raw `fetch` calls should typically rely on Node/undici’s default request timeout (about ~300s) rather than adding a per-call `AbortController` + `setTimeout` wrapper inside individual functions (e.g. in files like `apps/webapp/app/models/vercelIntegration.server.ts`). During code review, do not flag the absence of a per-call timeout on a single `fetch` as an issue; if per-call timeouts are needed, they should be implemented via a codebase-wide convention (e.g., a shared fetch wrapper or documented pattern) rather than ad-hoc per-function changes.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-06-16T09:19:47.637Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3960
File: apps/webapp/test/prismaInfrastructureErrorCapture.test.ts:0-0
Timestamp: 2026-06-16T09:19:47.637Z
Learning: In this repo’s Vitest setup, `vitest.config.ts` uses `globals: true`, so identifiers like `vi`, `describe`, `it`, and `expect` are available as globals in Vitest test files. During code review, do not flag missing `vi`/`describe`/`it`/`expect` imports as a runtime error or correctness issue when they’re used in `*.test.ts/tsx` or `*.spec.ts/tsx` files. Explicit imports are still preferred for consistency, but they’re not required for runtime behavior.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
📚 Learning: 2026-08-16T18:36:58.179Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 4537
File: packages/trigger-sdk/test/normalizeKeyString.test.ts:1-2
Timestamp: 2026-08-16T18:36:58.179Z
Learning: For related SDK `chat.agent` tests in the Trigger.dev repository—including chat channels, handover, snapshot, and transport-event coverage—keep new test files under `packages/trigger-sdk/test/` rather than colocating them with the `packages/trigger-sdk/src/v3/` source files.

Applied to files:

  • packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts
📚 Learning: 2026-03-31T21:37:27.212Z
Learnt from: isshaddad
Repo: triggerdotdev/trigger.dev PR: 3283
File: docs/migration-n8n.mdx:19-21
Timestamp: 2026-03-31T21:37:27.212Z
Learning: When reviewing code in `packages/trigger-sdk/src/v3`, treat `tasks.triggerAndWait()` and `tasks.batchTriggerAndWait()` as real exported APIs. They are defined in `shared.ts` and re-exported via the `tasks` object in `tasks.ts`, and they take the task ID string as their first argument (not a task instance). This is distinct from the instance methods `yourTask.triggerAndWait()` and `yourTask.batchTriggerAndWait()`. Do not flag calls to `tasks.triggerAndWait()` or `tasks.batchTriggerAndWait()` as non-existent or incorrectly invoked.

Applied to files:

  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-05-17T08:08:12.370Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3644
File: packages/trigger-sdk/src/v3/ai.ts:8695-8746
Timestamp: 2026-05-17T08:08:12.370Z
Learning: In the Trigger v3 session resume/streams logic, ensure session resumption uses sequence cursors rather than timestamps. Specifically: for each turn-complete control record written to `session.out`, include a `session-in-event-id` header whose value is the committed-consume cursor (`session.in.lastDispatchedSeqNum`). On boot/resume, scan `session.out` for the latest turn-complete record, read the `session-in-event-id` header, and seed the `sessionStreams` manager for `.in` using both `lastSeqNum` and `lastDispatchedSeqNum` so previously processed user messages are not replayed. Do not use `setMinTimestamp`/`lastOutTimestamp` for resume ordering in this flow.

Applied to files:

  • packages/trigger-sdk/src/v3/ai.ts
📚 Learning: 2026-05-18T14:19:56.437Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3655
File: packages/trigger-sdk/src/v3/ai.ts:8667-8731
Timestamp: 2026-05-18T14:19:56.437Z
Learning: In the Trigger SDK (v3) when making raw `fetch` calls to the Trigger API (including override paths such as `createChatStartSessionAction`), set the request headers to match `ApiClient`: `Content-Type`, `Authorization`, and `x-trigger-source: "sdk"`. Also forward the current preview branch by setting `x-trigger-branch` to `apiClientManager.branchName`. Prefer using the shared `overrideRequestHeaders(accessToken)` helper instead of manually constructing headers, so requests route correctly to preview environments.

Applied to files:

  • packages/trigger-sdk/src/v3/ai.ts
🔇 Additional comments (17)
packages/trigger-sdk/src/v3/ai.ts (11)

1504-1567: LGTM!


1569-1587: LGTM!


1701-1736: LGTM!


1754-1762: LGTM!

Also applies to: 1790-1792, 1805-1813, 1828-1834


5524-5554: LGTM!


5564-5568: LGTM!

Also applies to: 5595-5606


5619-5682: LGTM!


8589-8593: LGTM!


9752-9760: LGTM!

Also applies to: 9859-9862, 9896-9896, 9958-9958, 10004-10004, 10102-10106


9970-9987: LGTM!

Also applies to: 10034-10066


1456-1502: 📐 Maintainability & Code Quality

No change needed. The synchronous adapter includes every synchronous branch supported by getSchemaParseFn; excluding parseAsync is intentional for chat.messages.peek().

docs/ai-chat/reference.mdx (1)

549-549: LGTM!

Also applies to: 559-560

docs/ai-chat/types.mdx (1)

143-143: LGTM!

Also applies to: 170-171

docs/ai-chat/client-protocol.mdx (1)

774-774: LGTM!

docs/ai-chat/custom-agents.mdx (1)

22-47: LGTM!

Also applies to: 54-106, 135-135

packages/trigger-sdk/test/custom-agent-client-data-validation.test.ts (1)

1-31: LGTM!

Also applies to: 33-199, 201-320, 322-422, 424-514, 516-552

.changeset/quiet-chats-validate.md (1)

2-5: 📐 Maintainability & Code Quality

Keep the patch bump.

CONTRIBUTING.md states that most user-facing changes use patch. Existing SDK patch releases also include additive APIs and behavior changes. The changeset summary is accurate.

			> Likely an incorrect or invalid review comment.

Comment on lines +1477 to +1479
if (typeof parser.parse === "function") {
return parser.parse.bind(parser);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Guard against a promise-returning parse in the synchronous path.

The function-form branch at Lines 1463-1475 detects a thenable result and throws a clear error. The object parse branch does not. If a schema exposes an asynchronous parse method, validateChatCustomAgentPayloadSync returns a pending Promise as metadata, and chat.messages.peek() hands that promise to the caller as parsed client data. The caller then reads properties on a promise instead of failing fast.

Extract the thenable check into a small helper and apply it to both branches.

🐛 Proposed fix to reuse the thenable guard
+function assertSyncParseResult(result: unknown): unknown {
+  if (result && typeof (result as { then?: unknown }).then === "function") {
+    void Promise.resolve(result).catch(() => {});
+    throw new Error(
+      "chat.messages.peek() cannot validate clientData with an asynchronous schema. " +
+        "Use chat.messages.once(), chat.messages.wait(), or chat.messages.waitWithIdleTimeout()."
+    );
+  }
+  return result;
+}
+
 function getChatCustomAgentSyncSchemaParseFn(schema: TaskSchema): (value: unknown) => unknown {
   const parser = schema as any;
 
   if (typeof parser === "function" && typeof parser.assert === "function") {
     return parser.assert.bind(parser);
   }
 
   if (typeof parser === "function") {
-    return (value) => {
-      const result = parser(value);
-      if (result && typeof result.then === "function") {
-        void Promise.resolve(result).catch(() => {});
-        throw new Error(
-          "chat.messages.peek() cannot validate clientData with an asynchronous schema. " +
-            "Use chat.messages.once(), chat.messages.wait(), or chat.messages.waitWithIdleTimeout()."
-        );
-      }
-      return result;
-    };
+    return (value) => assertSyncParseResult(parser(value));
   }
 
   if (typeof parser.parse === "function") {
-    return parser.parse.bind(parser);
+    return (value) => assertSyncParseResult(parser.parse(value));
   }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (typeof parser.parse === "function") {
return parser.parse.bind(parser);
}
function assertSyncParseResult(result: unknown): unknown {
if (result && typeof (result as { then?: unknown }).then === "function") {
void Promise.resolve(result).catch(() => {});
throw new Error(
"chat.messages.peek() cannot validate clientData with an asynchronous schema. " +
"Use chat.messages.once(), chat.messages.wait(), or chat.messages.waitWithIdleTimeout()."
);
}
return result;
}
function getChatCustomAgentSyncSchemaParseFn(schema: TaskSchema): (value: unknown) => unknown {
const parser = schema as any;
if (typeof parser === "function" && typeof parser.assert === "function") {
return parser.assert.bind(parser);
}
if (typeof parser === "function") {
return (value) => assertSyncParseResult(parser(value));
}
if (typeof parser.parse === "function") {
return (value) => assertSyncParseResult(parser.parse(value));
}

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 3 potential issues.

Open in Devin Review

Comment on lines +1717 to 1736
if (!locals.get(chatCustomAgentClientDataParserKey)) {
return subscribeToRawChatMessages(handler);
}

let delivery = Promise.resolve();
return subscribeToRawChatMessages((payload) => {
delivery = delivery
.then(async () => {
const result = await validateChatCustomAgentPayload(payload, {
// A subscription may receive a frame while the current turn is
// still streaming. Completing that turn here would close the
// active response, so on() reports through the callback and log.
writeErrorToStream: false,
});
if (result.ok) {
await handler(result.payload);
}
})
.catch(() => {});
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Handlers can still fire after subscription.off() once a schema is configured

With a clientDataSchema present, chat.messages.on() no longer invokes the handler synchronously at chunk arrival — it queues validation on a serialized delivery chain and calls the handler only after the async parse resolves. Since off() only detaches the underlying .in listener, a frame that arrived just before off() will still reach the handler afterwards (the new test delivers frames that arrived before chat.messages.on is removed codifies this).

This matters for chat.createSession's steering subscription (packages/trigger-sdk/src/v3/ai.ts:10035-10060), which is deliberately detached in the finally after piping so that "later arrivals must buffer for the next turn". A late delivery pushes into the previous turn's turnSteeringQueue (captured in the closure), which is discarded — the message is neither injected nor buffered for the next turn. The existing comment claims the frontend re-sends non-injected messages on turn complete, so it is likely recoverable, but the off() boundary is no longer exact for schema-configured agents and is worth confirming against the transport's re-send behavior.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +5655 to +5666

// The Session base payload is sticky across continuation runs. If it is
// invalid, returning here would boot the same bad metadata again on the
// next message. Stay attached and wait for a valid wire frame instead.
const next = await messagesInput.waitWithIdleTimeout({
idleTimeoutInSeconds: payload.idleTimeoutInSeconds ?? 30,
timeout: "1h",
spanName: "waiting for valid clientData",
});
if (!next.ok || next.output.trigger === "close") {
return;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 A sticky-invalid session base payload emits a spurious error + turn-complete on every run boot

The boot validation writes an error chunk plus turn-complete (via reportChatCustomAgentClientDataError at packages/trigger-sdk/src/v3/ai.ts:1533-1537) whenever the base payload's metadata fails to parse, and then stays attached waiting for a valid wire frame. Because the Session base payload is sticky across continuation runs, a session created with invalid clientData (but whose per-message frames carry valid metadata) will emit a spurious error + turn-complete at the start of every new run for that session before recovering. Clients that treat turn-complete as "generation finished" will end the turn, then receive the real response chunks afterwards for the recovered frame. Worth confirming the transport tolerates a turn-complete that is immediately followed by a full response.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +5670 to +5677
const recoveredPayload = {
...next.output,
continuation: next.output.continuation ?? payload.continuation,
previousRunId: next.output.previousRunId ?? payload.previousRunId,
sessionId: next.output.sessionId ?? payload.sessionId,
idleTimeoutInSeconds: next.output.idleTimeoutInSeconds ?? payload.idleTimeoutInSeconds,
headStartMessages: next.output.headStartMessages ?? payload.headStartMessages,
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Forwarded headStartMessages is ignored by the managed session loop after handover recovery

The recovery payload carries headStartMessages forward, but the recovered payload's trigger is whatever the new client frame carried (typically submit-message), and chat.createSession only reads currentPayload.headStartMessages inside the pendingHandoverSignal branch (packages/trigger-sdk/src/v3/ai.ts:10085-10089), which requires trigger === "handover-prepare" on turn 0. So for createSession-based custom agents the warm-server first-turn history is dropped along with the partial, and the recovered turn starts with an empty accumulator. Hand-rolled loops can still consume it via MessageAccumulator's handover boot helper (packages/trigger-sdk/src/v3/ai.ts:9554), so the forwarding isn't dead in general — but the managed path silently loses prior history here.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants