From 314fcaad68dec9e6f1228a7fafa6d084015c4af8 Mon Sep 17 00:00:00 2001 From: Nicholas Carter Date: Wed, 19 Aug 2026 13:05:11 +0000 Subject: [PATCH 1/2] docs: state Node authority boundary honestly --- README.md | 149 ++++++++---------- .../ADR-001-execution-envelope-enforcement.md | 27 ++-- package.json | 4 +- 3 files changed, 85 insertions(+), 95 deletions(-) diff --git a/README.md b/README.md index a86dc67..12689ec 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,16 @@ -# @bodanglin/verdict-node — TypeScript Middleware for Verdict Routing +# @bodanglin/verdict-node — TypeScript Gateway Adapter [![npm](https://img.shields.io/npm/v/@bodanglin/verdict-node.svg)](https://www.npmjs.com/package/@bodanglin/verdict-node) [![TypeScript](https://img.shields.io/badge/typescript-strict-blue.svg)](https://www.typescriptlang.org/) -[![Tests](https://img.shields.io/badge/tests-139%20passing-brightgreen.svg)](<>) [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE) -> **Enterprise LLM Criticality Router middleware for Express, Next.js** — policy-gated, capability-aware routing with OpenAI-compatible upstream proxy. +> **OpenAI-compatible gateway adapter for Express and Next.js** — includes pre-forward execution-envelope validation. --- ## What is @bodanglin/verdict-node? -`@bodanglin/verdict-node` is the TypeScript gateway integration layer for the **Verdict** ecosystem. It accepts an Express/Next.js request, classifies criticality, discovers models from your configured OpenAI-compatible upstream (default: OmniRoute at `http://localhost:20128/v1`), rewrites the selected model, and forwards non-streaming SSE responses. +`@bodanglin/verdict-node` is the TypeScript gateway adapter for the **Verdict** ecosystem. Verdict Core owns policy and execution authorization; Node supplies transport middleware that can validate an `ExecutionEnvelope` before forwarding a request to an OpenAI-compatible upstream. The canonical cross-language envelope contract and Core issuance path are still being reconciled, so this alpha must not be represented as complete end-to-end policy enforcement. Node also retains local classification, discovery, ranking, and fallback behavior for compatibility routing; those heuristics are not Core authorization. **Works with any OpenAI-compatible client**: Claude Code, Codex, Cursor, Cline, Hermes, Agents SDK, raw HTTP. @@ -21,12 +20,13 @@ **Alpha** — not production-ready. Current implementation provides: -- Heuristic criticality classification +- Fail-closed `ExecutionEnvelope` validation in the standalone forwarder by default +- A shared pre-forward envelope check for streaming and non-streaming requests when an envelope is supplied to the gateway - Zod request/response schemas -- Model catalog discovery from configured upstream +- Heuristic criticality classification and model catalog discovery for compatibility routing - Bounded fallback ladder for selected HTTP/network failures -- Explicit fail-open behavior: inconclusive model discovery, missing quota/headroom data, unavailable usage APIs, and middleware classification errors keep routing on the safe primary/fallback path instead of blocking client traffic -- Non-streaming SSE forwarding +- Explicit compatibility opt-outs for deployments that do not yet require Core decisions or envelopes +- Streaming SSE and non-streaming JSON forwarding - In-memory score cache (process-local) **Missing** (tracked on release board): @@ -65,126 +65,111 @@ yarn add @bodanglin/verdict-node ## Quick Start +### Express standalone forwarder + ```typescript import express from 'express'; -import { verdictMiddleware } from '@bodanglin/verdict-node/middleware'; +import { createForwarder } from '@bodanglin/verdict-node/middleware'; const app = express(); app.use(express.json()); -// Mount Verdict middleware app.use( '/v1', - verdictMiddleware({ - upstream: 'http://localhost:20128/v1', // OmniRoute or your proxy - criticality: 'auto', // auto | low | medium | high | critical + createForwarder({ + baseUrl: process.env.VERDICT_UPSTREAM ?? 'http://127.0.0.1:20132/v1', + apiKey: process.env.OMNIROUTE_API_KEY, + executionEnvelope: coreEnvelope, + expectedPolicyDigest: trustedPolicyDigest, }) ); app.listen(3000, () => console.log('verdict-node listening on :3000')); ``` -Next.js `/api` route: +`executionEnvelope` is currently configured on the middleware instance. Create or scope middleware instances so an envelope cannot be reused for unrelated requests, and derive `trustedPolicyDigest` from an independent trusted policy source rather than from the envelope itself. `requireExecutionEnvelope` defaults to `true`; setting it to `false` is an explicit compatibility opt-out, not Core-authorized execution. + +### Next.js `/api` route ```typescript // pages/api/chat/completions.ts -import { createNextApiHandler } from '@verdict/node'; +import { createNextApiHandler } from '@bodanglin/verdict-node'; export default createNextApiHandler({ baseUrl: process.env.OMNIROUTE_BASE_URL ?? 'http://127.0.0.1:20132/v1', apiKey: process.env.OMNIROUTE_API_KEY, + decisionEndpoint: process.env.VERDICT_CORE_DECISION_ENDPOINT, }); ``` -```bash -# Start OmniRoute (if not running) -docker run -d -p 20128:20128 omnibus/omniroute - -# Start your app -node dist/index.js -``` +The gateway requires a Core routing decision by default. Set `decisionEndpoint` or `VERDICT_CORE_DECISION_ENDPOINT`; without either, requests receive HTTP 503. `requireCoreDecision: false` enables local heuristic compatibility routing and is not Core-authorized execution. --- ## Configuration -```typescript -import { verdictMiddleware, VerdictConfig } from '@bodanglin/verdict-node/middleware'; +### `ForwarderConfig` -const config: VerdictConfig = { - // Upstream OpenAI-compatible endpoint - upstream: process.env.VERDICT_UPSTREAM ?? 'http://localhost:20128/v1', +```typescript +import { createForwarder, type ForwarderConfig } from '@bodanglin/verdict-node/middleware'; - // Criticality classification: 'auto' | 'low' | 'medium' | 'high' | 'critical' - criticality: 'auto', +const config: ForwarderConfig = { + baseUrl: process.env.VERDICT_UPSTREAM ?? 'http://127.0.0.1:20132/v1', + apiKey: process.env.OMNIROUTE_API_KEY, + executionEnvelope: coreEnvelope, + expectedPolicyDigest: trustedPolicyDigest, + timeoutMs: 30_000, + maxRetries: 3, +}; - // Optional: Custom model catalog (bypasses discovery) - modelCatalog: [ - { id: 'anthropic/claude-3-opus-20240229', capabilities: ['tools', 'vision'] }, - { id: 'openai/gpt-4o', capabilities: ['tools', 'vision'] }, - { id: 'auto/best-coding', capabilities: ['tools', 'reasoning'] }, - ], +app.use('/v1', createForwarder(config)); +``` - // Fallback ladder (ordered) - fallbacks: [ - { model: 'auto/best-fast', maxRetries: 2 }, - { model: 'auto/best-reasoning', maxRetries: 1 }, - ], +### `GatewayConfig` - // Request timeout - timeoutMs: 30000, +```typescript +import { createNextApiHandler, type GatewayConfig } from '@bodanglin/verdict-node'; - // Enable request/response logging - debug: process.env.NODE_ENV === 'development', +const config: GatewayConfig = { + baseUrl: process.env.OMNIROUTE_BASE_URL, + apiKey: process.env.OMNIROUTE_API_KEY, + decisionEndpoint: process.env.VERDICT_CORE_DECISION_ENDPOINT, + decisionTimeoutMs: 2_000, }; -app.use('/v1', verdictMiddleware(config)); +export default createNextApiHandler(config); ``` --- ## API -### `verdictMiddleware(config: VerdictConfig): express.RequestHandler` +### `createForwarder(config: ForwarderConfig): express.RequestHandler` + +The standalone Express forwarder validates the configured envelope before its first upstream fetch. By default it rejects missing, invalid, expired, or out-of-bounds envelopes with machine-readable denial codes. It then forwards non-streaming JSON or streaming SSE responses without substituting the request model. -Express middleware that: +### `createNextApiHandler(config: GatewayConfig): NextApiHandlerLike` -1. Intercepts `POST /v1/chat/completions` -2. Classifies request criticality (heuristic or explicit header `x-verdict-criticality`) -3. Discovers/uses model catalog from upstream -4. Selects best model via Verdict Core logic (or local heuristic) -5. Rewrites `model` field in request body -6. Forwards to upstream, streams response back +The higher-level gateway requests a Core routing decision by default and returns HTTP 503 when no decision is available. If a returned decision contains an envelope, the gateway validates it before forwarding. Current limitations tracked by NOD-002 include a missing-envelope enforcement gap, locally substituted ladder models that are not revalidated against the envelope, and missing policy-digest integrity evidence on this path. ### Types ```typescript -// From @bodanglin/verdict-node +import type { GatewayConfig, OpenAIChatCompletionRequest } from '@bodanglin/verdict-node'; import type { - VerdictConfig, - ModelInfo, - CriticalityLevel, - ChatCompletionRequest, - ChatCompletionResponse, -} from '@bodanglin/verdict-node'; + ForwarderConfig, + OpenAIChatCompletionResponse, + OpenAIChatCompletionChunk, +} from '@bodanglin/verdict-node/middleware'; ``` --- ## Integration with Verdict Core -For full policy-gated routing (not just heuristic), run Verdict Core alongside: - -```bash -# Terminal 1: Verdict Core API -verdict serve --host 0.0.0.0 --port 8000 - -# Terminal 2: Verdict Node middleware pointing to Core -export VERDICT_UPSTREAM=http://localhost:8000/v1 -node dist/index.js -``` +Verdict Core is the intended authority for policy-gated execution; Node is an edge and transport adapter. Core and Node do not yet share a fully reconciled, published `ExecutionEnvelope` contract or verified issuance-to-enforcement fixture. Until that work is complete, treat the envelope support here as partial enforcement rather than proof of end-to-end Core authorization. -Then `@verdict/node` will use Core's `/v1/route` endpoint for model selection. +For the higher-level gateway, point `decisionEndpoint` (or `VERDICT_CORE_DECISION_ENDPOINT`) at the Core routing-decision endpoint. The standalone forwarder instead accepts an envelope through `ForwarderConfig.executionEnvelope` and requires one by default. Both APIs expose explicit compatibility opt-outs; those modes are not policy-gated execution. NOD-002 remains open until shared Core fixtures, complete envelope parity, and removal of every production-path policy bypass are verified. --- @@ -217,18 +202,16 @@ npm run verify:package ``` verdict-node/ ├── src/ -│ ├── index.ts # Main exports -│ ├── middleware/ # Express middleware -│ │ ├── index.ts -│ │ ├── criticality.ts # Criticality classification -│ │ ├── catalog.ts # Model catalog discovery -│ │ ├── routing.ts # Model selection logic -│ │ └── proxy.ts # SSE proxy forwarding -│ ├── types/ # Zod schemas + TS types -│ └── utils/ -├── tests/ # 139 tests -├── scripts/ # verify-package.mjs -├── dist/ # Build output +│ ├── index.ts # Gateway and Next.js exports +│ ├── adapters/ +│ │ └── contract-to-middleware.ts # Canonical-decision adapter +│ └── middleware/ +│ ├── index.ts # Middleware exports +│ ├── forwarder.ts # Express JSON/SSE forwarder +│ └── validator.ts # Validation helpers +├── tests/ +├── scripts/ # Package verification +├── docs/adr/ └── package.json ``` diff --git a/docs/adr/ADR-001-execution-envelope-enforcement.md b/docs/adr/ADR-001-execution-envelope-enforcement.md index 52e0a14..908824f 100644 --- a/docs/adr/ADR-001-execution-envelope-enforcement.md +++ b/docs/adr/ADR-001-execution-envelope-enforcement.md @@ -1,29 +1,36 @@ -# ADR-021: ExecutionEnvelope Edge Enforcement for Gateway Adapters +# ADR-001: ExecutionEnvelope Edge Enforcement for Gateway Adapters -- **Status:** Accepted — Implemented in NOD-002 +- **Status:** Accepted — Partially implemented; NOD-002 remains open - **Date:** 2026-08-03 - **Scope:** Cross-language contract enforcement between Verdict Core and Verdict Node (`@bodanglin/verdict-node`) ## Context -Verdict Core is the authoritative policy-gated execution control plane. To ensure that transport middleware (such as `verdict-node`) cannot execute requests outside Core-authorized boundaries, edge adapters must enforce canonical `ExecutionEnvelope` constraints before forwarding requests upstream. +Verdict Core is the intended policy and execution-authorization authority. To keep transport middleware such as `verdict-node` from weakening Core's constraints, edge adapters must enforce a canonical `ExecutionEnvelope` before forwarding requests upstream. Core and Node do not yet share a reconciled envelope schema and issuance-to-enforcement fixture, so the implementation remains partial. ## Decision -We establish edge-level `ExecutionEnvelope` validation rules in Verdict Node (`src/middleware/forwarder.ts`): +We establish the required edge-level `ExecutionEnvelope` validation rules for Verdict Node: -1. **Pre-Forward Validation:** Edge middleware MUST validate the presence, schema version (`1`), expiration time, and policy digest of an incoming `ExecutionEnvelope` before initiating any HTTP/SSE forwarding. +1. **Pre-Forward Validation:** Every policy-gated forwarding path MUST validate the presence, schema version (`1`), expiration time, and policy digest of a canonical `ExecutionEnvelope` before initiating HTTP or SSE forwarding. 2. **Fail-Closed Policy Enforcement:** - Missing or unparseable envelopes fail closed with `envelope_missing` / `envelope_invalid` (HTTP 403). - Expired envelopes fail closed with `envelope_expired` (HTTP 403). - Policy digest mismatches fail closed with `envelope_tampered` (HTTP 403). - - Requests specifying models outside `execution_constraints.allowed_models` fail closed with `model_disallowed` (HTTP 403). + - Every actual upstream model, including substituted or fallback models, is checked against `execution_constraints.allowed_models`; violations fail with `model_disallowed` (HTTP 403). - Requests invoking tools outside `execution_constraints.allowed_tools` fail closed with `tool_disallowed` (HTTP 403). - Requests exceeding `execution_constraints.budget_usd` or `max_request_usd` fail closed with `budget_exceeded` (HTTP 403). -3. **Parity Assurance:** The TypeScript middleware consumes canonical `@bodanglin/verdict-contracts` definitions and does not re-implement eligibility or policy evaluation. +3. **Parity Assurance:** Core and Node MUST consume one versioned envelope contract and shared positive and negative fixtures. Node does not re-implement eligibility or policy evaluation. +4. **Compatibility Isolation:** Explicit opt-outs MAY bypass envelope enforcement only for compatibility deployments, which MUST NOT be represented as Core-authorized execution. + +## Implementation Status + +- `src/middleware/forwarder.ts` implements fail-closed envelope validation by default before non-streaming JSON and streaming SSE forwarding. It checks the request model and does not substitute it. +- `src/index.ts` validates an attached envelope in the higher-level gateway, but it currently skips enforcement when the envelope is absent, does not supply independent policy-digest evidence, and does not revalidate locally substituted ladder models. +- The published Core contract, Core issuance path, shared fixtures, and CI conformance coverage are not yet aligned. ## Consequences -- Edge gateway adapters guarantee that no un-authorized, expired, or out-of-bounds requests reach upstream model providers. -- Enforcement is applied uniformly to both non-streaming (JSON) and streaming (SSE) request flows. -- All denial responses return standardized, machine-readable `EnvelopeDenialCode` payloads. +- This ADR defines the required authority boundary; it does not certify that all production forwarding paths currently satisfy it. +- The standalone forwarder returns machine-readable envelope denial codes for the checks it implements. +- NOD-002 remains open until one canonical Core envelope is accepted and enforced across every production forwarding path and verified through shared fixtures and CI. diff --git a/package.json b/package.json index 51b94b2..e076658 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "@bodanglin/verdict-node", "version": "0.1.0", - "description": "Enterprise LLM Criticality Router middleware for Express, Next.js — policy-gated, availability-aware routing control plane", + "description": "OpenAI-compatible gateway adapter with execution-envelope validation for Express and Next.js", "license": "MIT", "main": "dist/index.js", "types": "dist/index.d.ts", @@ -49,7 +49,7 @@ "llm", "routing", "ai", - "control-plane", + "gateway-adapter", "safety", "middleware", "express", From 6d0928796339cd5e39285deed9a43cd8117723bd Mon Sep 17 00:00:00 2001 From: Nicholas Carter Date: Wed, 19 Aug 2026 13:16:23 +0000 Subject: [PATCH 2/2] docs: disclose nextApiHandler continuation-after-503 defect - Rewrite Next.js quickstart to explicitly state createNextApiHandler is not fail-closed - Update API reference for createNextApiHandler to document the continuation defect - Update Integration section to call out the critical defect - Add the continuation-after-503 defect to ADR-001 implementation status - Clarify coreEnvelope/trustedPolicyDigest as application-provided values --- README.md | 12 ++++++++---- docs/adr/ADR-001-execution-envelope-enforcement.md | 1 + 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 12689ec..18b2097 100644 --- a/README.md +++ b/README.md @@ -74,6 +74,10 @@ import { createForwarder } from '@bodanglin/verdict-node/middleware'; const app = express(); app.use(express.json()); +// Obtain these values from independent trusted Core outputs. +const coreEnvelope: unknown = await loadCoreEnvelope(); +const trustedPolicyDigest = await loadTrustedPolicyDigest(); + app.use( '/v1', createForwarder({ @@ -89,7 +93,7 @@ app.listen(3000, () => console.log('verdict-node listening on :3000')); `executionEnvelope` is currently configured on the middleware instance. Create or scope middleware instances so an envelope cannot be reused for unrelated requests, and derive `trustedPolicyDigest` from an independent trusted policy source rather than from the envelope itself. `requireExecutionEnvelope` defaults to `true`; setting it to `false` is an explicit compatibility opt-out, not Core-authorized execution. -### Next.js `/api` route +### Next.js `/api` route — Currently Not Fail-Closed ```typescript // pages/api/chat/completions.ts @@ -102,7 +106,7 @@ export default createNextApiHandler({ }); ``` -The gateway requires a Core routing decision by default. Set `decisionEndpoint` or `VERDICT_CORE_DECISION_ENDPOINT`; without either, requests receive HTTP 503. `requireCoreDecision: false` enables local heuristic compatibility routing and is not Core-authorized execution. +**Critical limitation:** The current `createNextApiHandler` implementation does **not** stop after a Core-decision denial. When `middleware()` writes HTTP 503 (no decision available or denied), it returns without calling `next()`, but `nextApiHandler()` unconditionally invokes `proxy()` afterward. The proxy path skips envelope validation when no envelope is present and may fetch upstream. **Do not treat this handler as fail-closed.** A source-level fix with regression test is tracked in NOD-002. --- @@ -150,7 +154,7 @@ The standalone Express forwarder validates the configured envelope before its fi ### `createNextApiHandler(config: GatewayConfig): NextApiHandlerLike` -The higher-level gateway requests a Core routing decision by default and returns HTTP 503 when no decision is available. If a returned decision contains an envelope, the gateway validates it before forwarding. Current limitations tracked by NOD-002 include a missing-envelope enforcement gap, locally substituted ladder models that are not revalidated against the envelope, and missing policy-digest integrity evidence on this path. +The higher-level gateway **intends** to request a Core routing decision by default and return HTTP 503 when no decision is available. However, the current implementation has a critical defect: after `middleware()` writes a 503 response (no decision or denied), it does not call `next()`, but `nextApiHandler()` unconditionally proceeds to call `proxy()`. The proxy path skips envelope validation when no envelope is attached and may execute an upstream fetch. **This path is not fail-closed.** Current limitations tracked by NOD-002 include this continuation-after-503 defect, a missing-envelope enforcement gap, locally substituted ladder models that are not revalidated against the envelope, and missing policy-digest integrity evidence on this path. ### Types @@ -169,7 +173,7 @@ import type { Verdict Core is the intended authority for policy-gated execution; Node is an edge and transport adapter. Core and Node do not yet share a fully reconciled, published `ExecutionEnvelope` contract or verified issuance-to-enforcement fixture. Until that work is complete, treat the envelope support here as partial enforcement rather than proof of end-to-end Core authorization. -For the higher-level gateway, point `decisionEndpoint` (or `VERDICT_CORE_DECISION_ENDPOINT`) at the Core routing-decision endpoint. The standalone forwarder instead accepts an envelope through `ForwarderConfig.executionEnvelope` and requires one by default. Both APIs expose explicit compatibility opt-outs; those modes are not policy-gated execution. NOD-002 remains open until shared Core fixtures, complete envelope parity, and removal of every production-path policy bypass are verified. +For the higher-level gateway, point `decisionEndpoint` (or `VERDICT_CORE_DECISION_ENDPOINT`) at the Core routing-decision endpoint. The standalone forwarder instead accepts an envelope through `ForwarderConfig.executionEnvelope` and requires one by default. Both APIs expose explicit compatibility opt-outs; those modes are not policy-gated execution. **Critical defect:** `createNextApiHandler` currently continues into `proxy()` after `middleware()` writes HTTP 503, bypassing envelope validation and potentially forwarding to upstream. NOD-002 remains open until shared Core fixtures, complete envelope parity, the continuation-after-503 fix, and removal of every production-path policy bypass are verified. --- diff --git a/docs/adr/ADR-001-execution-envelope-enforcement.md b/docs/adr/ADR-001-execution-envelope-enforcement.md index 908824f..cf4059a 100644 --- a/docs/adr/ADR-001-execution-envelope-enforcement.md +++ b/docs/adr/ADR-001-execution-envelope-enforcement.md @@ -27,6 +27,7 @@ We establish the required edge-level `ExecutionEnvelope` validation rules for Ve - `src/middleware/forwarder.ts` implements fail-closed envelope validation by default before non-streaming JSON and streaming SSE forwarding. It checks the request model and does not substitute it. - `src/index.ts` validates an attached envelope in the higher-level gateway, but it currently skips enforcement when the envelope is absent, does not supply independent policy-digest evidence, and does not revalidate locally substituted ladder models. +- **Critical defect:** `src/index.ts`'s `nextApiHandler()` unconditionally invokes `proxy()` after `middleware()` returns, even when `middleware()` wrote HTTP 503 without calling `next()`. The proxy path then skips envelope validation (no envelope attached) and may fetch upstream. This path is not fail-closed. - The published Core contract, Core issuance path, shared fixtures, and CI conformance coverage are not yet aligned. ## Consequences