Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 70 additions & 83 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
# @bodanglin/verdict-node — TypeScript Middleware for Verdict Routing
# @bodanglin/verdict-node — TypeScript Gateway Adapter

[![npm](https://img.shields.io/npm/v/@bodanglin/verdict-node.svg)](https://www.npmjs.com/package/@bodanglin/verdict-node)
[![TypeScript](https://img.shields.io/badge/typescript-strict-blue.svg)](https://www.typescriptlang.org/)
[![Tests](https://img.shields.io/badge/tests-139%20passing-brightgreen.svg)](<>)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)

> **Enterprise LLM Criticality Router middleware for Express, Next.js** — policy-gated, capability-aware routing with OpenAI-compatible upstream proxy.
> **OpenAI-compatible gateway adapter for Express and Next.js** — includes pre-forward execution-envelope validation.

---

## What is @bodanglin/verdict-node?

`@bodanglin/verdict-node` is the TypeScript gateway integration layer for the **Verdict** ecosystem. It accepts an Express/Next.js request, classifies criticality, discovers models from your configured OpenAI-compatible upstream (default: OmniRoute at `http://localhost:20128/v1`), rewrites the selected model, and forwards non-streaming SSE responses.
`@bodanglin/verdict-node` is the TypeScript gateway adapter for the **Verdict** ecosystem. Verdict Core owns policy and execution authorization; Node supplies transport middleware that can validate an `ExecutionEnvelope` before forwarding a request to an OpenAI-compatible upstream. The canonical cross-language envelope contract and Core issuance path are still being reconciled, so this alpha must not be represented as complete end-to-end policy enforcement. Node also retains local classification, discovery, ranking, and fallback behavior for compatibility routing; those heuristics are not Core authorization.

**Works with any OpenAI-compatible client**: Claude Code, Codex, Cursor, Cline, Hermes, Agents SDK, raw HTTP.

Expand All @@ -21,12 +20,13 @@

**Alpha** — not production-ready. Current implementation provides:

- Heuristic criticality classification
- Fail-closed `ExecutionEnvelope` validation in the standalone forwarder by default
- A shared pre-forward envelope check for streaming and non-streaming requests when an envelope is supplied to the gateway
- Zod request/response schemas
- Model catalog discovery from configured upstream
- Heuristic criticality classification and model catalog discovery for compatibility routing
- Bounded fallback ladder for selected HTTP/network failures
- Explicit fail-open behavior: inconclusive model discovery, missing quota/headroom data, unavailable usage APIs, and middleware classification errors keep routing on the safe primary/fallback path instead of blocking client traffic
- Non-streaming SSE forwarding
- Explicit compatibility opt-outs for deployments that do not yet require Core decisions or envelopes
- Streaming SSE and non-streaming JSON forwarding
- In-memory score cache (process-local)

**Missing** (tracked on release board):
Expand Down Expand Up @@ -65,126 +65,115 @@ yarn add @bodanglin/verdict-node

## Quick Start

### Express standalone forwarder

```typescript
import express from 'express';
import { verdictMiddleware } from '@bodanglin/verdict-node/middleware';
import { createForwarder } from '@bodanglin/verdict-node/middleware';

const app = express();
app.use(express.json());

// Mount Verdict middleware
// Obtain these values from independent trusted Core outputs.
const coreEnvelope: unknown = await loadCoreEnvelope();
const trustedPolicyDigest = await loadTrustedPolicyDigest();

app.use(
'/v1',
verdictMiddleware({
upstream: 'http://localhost:20128/v1', // OmniRoute or your proxy
criticality: 'auto', // auto | low | medium | high | critical
createForwarder({
baseUrl: process.env.VERDICT_UPSTREAM ?? 'http://127.0.0.1:20132/v1',
apiKey: process.env.OMNIROUTE_API_KEY,
executionEnvelope: coreEnvelope,
expectedPolicyDigest: trustedPolicyDigest,
})
);

app.listen(3000, () => console.log('verdict-node listening on :3000'));
```

Next.js `/api` route:
`executionEnvelope` is currently configured on the middleware instance. Create or scope middleware instances so an envelope cannot be reused for unrelated requests, and derive `trustedPolicyDigest` from an independent trusted policy source rather than from the envelope itself. `requireExecutionEnvelope` defaults to `true`; setting it to `false` is an explicit compatibility opt-out, not Core-authorized execution.

### Next.js `/api` route — Currently Not Fail-Closed

```typescript
// pages/api/chat/completions.ts
import { createNextApiHandler } from '@verdict/node';
import { createNextApiHandler } from '@bodanglin/verdict-node';

export default createNextApiHandler({
baseUrl: process.env.OMNIROUTE_BASE_URL ?? 'http://127.0.0.1:20132/v1',
apiKey: process.env.OMNIROUTE_API_KEY,
decisionEndpoint: process.env.VERDICT_CORE_DECISION_ENDPOINT,
});
```

```bash
# Start OmniRoute (if not running)
docker run -d -p 20128:20128 omnibus/omniroute

# Start your app
node dist/index.js
```
**Critical limitation:** The current `createNextApiHandler` implementation does **not** stop after a Core-decision denial. When `middleware()` writes HTTP 503 (no decision available or denied), it returns without calling `next()`, but `nextApiHandler()` unconditionally invokes `proxy()` afterward. The proxy path skips envelope validation when no envelope is present and may fetch upstream. **Do not treat this handler as fail-closed.** A source-level fix with regression test is tracked in NOD-002.

---

## Configuration

```typescript
import { verdictMiddleware, VerdictConfig } from '@bodanglin/verdict-node/middleware';
### `ForwarderConfig`

const config: VerdictConfig = {
// Upstream OpenAI-compatible endpoint
upstream: process.env.VERDICT_UPSTREAM ?? 'http://localhost:20128/v1',
```typescript
import { createForwarder, type ForwarderConfig } from '@bodanglin/verdict-node/middleware';

// Criticality classification: 'auto' | 'low' | 'medium' | 'high' | 'critical'
criticality: 'auto',
const config: ForwarderConfig = {
baseUrl: process.env.VERDICT_UPSTREAM ?? 'http://127.0.0.1:20132/v1',
apiKey: process.env.OMNIROUTE_API_KEY,
executionEnvelope: coreEnvelope,
expectedPolicyDigest: trustedPolicyDigest,
timeoutMs: 30_000,
maxRetries: 3,
};

// Optional: Custom model catalog (bypasses discovery)
modelCatalog: [
{ id: 'anthropic/claude-3-opus-20240229', capabilities: ['tools', 'vision'] },
{ id: 'openai/gpt-4o', capabilities: ['tools', 'vision'] },
{ id: 'auto/best-coding', capabilities: ['tools', 'reasoning'] },
],
app.use('/v1', createForwarder(config));
```

// Fallback ladder (ordered)
fallbacks: [
{ model: 'auto/best-fast', maxRetries: 2 },
{ model: 'auto/best-reasoning', maxRetries: 1 },
],
### `GatewayConfig`

// Request timeout
timeoutMs: 30000,
```typescript
import { createNextApiHandler, type GatewayConfig } from '@bodanglin/verdict-node';

// Enable request/response logging
debug: process.env.NODE_ENV === 'development',
const config: GatewayConfig = {
baseUrl: process.env.OMNIROUTE_BASE_URL,
apiKey: process.env.OMNIROUTE_API_KEY,
decisionEndpoint: process.env.VERDICT_CORE_DECISION_ENDPOINT,
decisionTimeoutMs: 2_000,
};

app.use('/v1', verdictMiddleware(config));
export default createNextApiHandler(config);
```

---

## API

### `verdictMiddleware(config: VerdictConfig): express.RequestHandler`
### `createForwarder(config: ForwarderConfig): express.RequestHandler`

The standalone Express forwarder validates the configured envelope before its first upstream fetch. By default it rejects missing, invalid, expired, or out-of-bounds envelopes with machine-readable denial codes. It then forwards non-streaming JSON or streaming SSE responses without substituting the request model.

Express middleware that:
### `createNextApiHandler(config: GatewayConfig): NextApiHandlerLike`

1. Intercepts `POST /v1/chat/completions`
2. Classifies request criticality (heuristic or explicit header `x-verdict-criticality`)
3. Discovers/uses model catalog from upstream
4. Selects best model via Verdict Core logic (or local heuristic)
5. Rewrites `model` field in request body
6. Forwards to upstream, streams response back
The higher-level gateway **intends** to request a Core routing decision by default and return HTTP 503 when no decision is available. However, the current implementation has a critical defect: after `middleware()` writes a 503 response (no decision or denied), it does not call `next()`, but `nextApiHandler()` unconditionally proceeds to call `proxy()`. The proxy path skips envelope validation when no envelope is attached and may execute an upstream fetch. **This path is not fail-closed.** Current limitations tracked by NOD-002 include this continuation-after-503 defect, a missing-envelope enforcement gap, locally substituted ladder models that are not revalidated against the envelope, and missing policy-digest integrity evidence on this path.

### Types

```typescript
// From @bodanglin/verdict-node
import type { GatewayConfig, OpenAIChatCompletionRequest } from '@bodanglin/verdict-node';
import type {
VerdictConfig,
ModelInfo,
CriticalityLevel,
ChatCompletionRequest,
ChatCompletionResponse,
} from '@bodanglin/verdict-node';
ForwarderConfig,
OpenAIChatCompletionResponse,
OpenAIChatCompletionChunk,
} from '@bodanglin/verdict-node/middleware';
```

---

## Integration with Verdict Core

For full policy-gated routing (not just heuristic), run Verdict Core alongside:

```bash
# Terminal 1: Verdict Core API
verdict serve --host 0.0.0.0 --port 8000

# Terminal 2: Verdict Node middleware pointing to Core
export VERDICT_UPSTREAM=http://localhost:8000/v1
node dist/index.js
```
Verdict Core is the intended authority for policy-gated execution; Node is an edge and transport adapter. Core and Node do not yet share a fully reconciled, published `ExecutionEnvelope` contract or verified issuance-to-enforcement fixture. Until that work is complete, treat the envelope support here as partial enforcement rather than proof of end-to-end Core authorization.

Then `@verdict/node` will use Core's `/v1/route` endpoint for model selection.
For the higher-level gateway, point `decisionEndpoint` (or `VERDICT_CORE_DECISION_ENDPOINT`) at the Core routing-decision endpoint. The standalone forwarder instead accepts an envelope through `ForwarderConfig.executionEnvelope` and requires one by default. Both APIs expose explicit compatibility opt-outs; those modes are not policy-gated execution. **Critical defect:** `createNextApiHandler` currently continues into `proxy()` after `middleware()` writes HTTP 503, bypassing envelope validation and potentially forwarding to upstream. NOD-002 remains open until shared Core fixtures, complete envelope parity, the continuation-after-503 fix, and removal of every production-path policy bypass are verified.

---

Expand Down Expand Up @@ -217,18 +206,16 @@ npm run verify:package
```
verdict-node/
├── src/
│ ├── index.ts # Main exports
│ ├── middleware/ # Express middleware
│ │ ├── index.ts
│ │ ├── criticality.ts # Criticality classification
│ │ ├── catalog.ts # Model catalog discovery
│ │ ├── routing.ts # Model selection logic
│ │ └── proxy.ts # SSE proxy forwarding
│ ├── types/ # Zod schemas + TS types
│ └── utils/
├── tests/ # 139 tests
├── scripts/ # verify-package.mjs
├── dist/ # Build output
│ ├── index.ts # Gateway and Next.js exports
│ ├── adapters/
│ │ └── contract-to-middleware.ts # Canonical-decision adapter
│ └── middleware/
│ ├── index.ts # Middleware exports
│ ├── forwarder.ts # Express JSON/SSE forwarder
│ └── validator.ts # Validation helpers
├── tests/
├── scripts/ # Package verification
├── docs/adr/
└── package.json
```

Expand Down
28 changes: 18 additions & 10 deletions docs/adr/ADR-001-execution-envelope-enforcement.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,37 @@
# ADR-021: ExecutionEnvelope Edge Enforcement for Gateway Adapters
# ADR-001: ExecutionEnvelope Edge Enforcement for Gateway Adapters

- **Status:** Accepted — Implemented in NOD-002
- **Status:** Accepted — Partially implemented; NOD-002 remains open
- **Date:** 2026-08-03
- **Scope:** Cross-language contract enforcement between Verdict Core and Verdict Node (`@bodanglin/verdict-node`)

## Context

Verdict Core is the authoritative policy-gated execution control plane. To ensure that transport middleware (such as `verdict-node`) cannot execute requests outside Core-authorized boundaries, edge adapters must enforce canonical `ExecutionEnvelope` constraints before forwarding requests upstream.
Verdict Core is the intended policy and execution-authorization authority. To keep transport middleware such as `verdict-node` from weakening Core's constraints, edge adapters must enforce a canonical `ExecutionEnvelope` before forwarding requests upstream. Core and Node do not yet share a reconciled envelope schema and issuance-to-enforcement fixture, so the implementation remains partial.

## Decision

We establish edge-level `ExecutionEnvelope` validation rules in Verdict Node (`src/middleware/forwarder.ts`):
We establish the required edge-level `ExecutionEnvelope` validation rules for Verdict Node:

1. **Pre-Forward Validation:** Edge middleware MUST validate the presence, schema version (`1`), expiration time, and policy digest of an incoming `ExecutionEnvelope` before initiating any HTTP/SSE forwarding.
1. **Pre-Forward Validation:** Every policy-gated forwarding path MUST validate the presence, schema version (`1`), expiration time, and policy digest of a canonical `ExecutionEnvelope` before initiating HTTP or SSE forwarding.
2. **Fail-Closed Policy Enforcement:**
- Missing or unparseable envelopes fail closed with `envelope_missing` / `envelope_invalid` (HTTP 403).
- Expired envelopes fail closed with `envelope_expired` (HTTP 403).
- Policy digest mismatches fail closed with `envelope_tampered` (HTTP 403).
- Requests specifying models outside `execution_constraints.allowed_models` fail closed with `model_disallowed` (HTTP 403).
- Every actual upstream model, including substituted or fallback models, is checked against `execution_constraints.allowed_models`; violations fail with `model_disallowed` (HTTP 403).
- Requests invoking tools outside `execution_constraints.allowed_tools` fail closed with `tool_disallowed` (HTTP 403).
- Requests exceeding `execution_constraints.budget_usd` or `max_request_usd` fail closed with `budget_exceeded` (HTTP 403).
3. **Parity Assurance:** The TypeScript middleware consumes canonical `@bodanglin/verdict-contracts` definitions and does not re-implement eligibility or policy evaluation.
3. **Parity Assurance:** Core and Node MUST consume one versioned envelope contract and shared positive and negative fixtures. Node does not re-implement eligibility or policy evaluation.
4. **Compatibility Isolation:** Explicit opt-outs MAY bypass envelope enforcement only for compatibility deployments, which MUST NOT be represented as Core-authorized execution.

## Implementation Status

- `src/middleware/forwarder.ts` implements fail-closed envelope validation by default before non-streaming JSON and streaming SSE forwarding. It checks the request model and does not substitute it.
- `src/index.ts` validates an attached envelope in the higher-level gateway, but it currently skips enforcement when the envelope is absent, does not supply independent policy-digest evidence, and does not revalidate locally substituted ladder models.
- **Critical defect:** `src/index.ts`'s `nextApiHandler()` unconditionally invokes `proxy()` after `middleware()` returns, even when `middleware()` wrote HTTP 503 without calling `next()`. The proxy path then skips envelope validation (no envelope attached) and may fetch upstream. This path is not fail-closed.
- The published Core contract, Core issuance path, shared fixtures, and CI conformance coverage are not yet aligned.

## Consequences

- Edge gateway adapters guarantee that no un-authorized, expired, or out-of-bounds requests reach upstream model providers.
- Enforcement is applied uniformly to both non-streaming (JSON) and streaming (SSE) request flows.
- All denial responses return standardized, machine-readable `EnvelopeDenialCode` payloads.
- This ADR defines the required authority boundary; it does not certify that all production forwarding paths currently satisfy it.
- The standalone forwarder returns machine-readable envelope denial codes for the checks it implements.
- NOD-002 remains open until one canonical Core envelope is accepted and enforced across every production forwarding path and verified through shared fixtures and CI.
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@bodanglin/verdict-node",
"version": "0.1.0",
"description": "Enterprise LLM Criticality Router middleware for Express, Next.js — policy-gated, availability-aware routing control plane",
"description": "OpenAI-compatible gateway adapter with execution-envelope validation for Express and Next.js",
"license": "MIT",
"main": "dist/index.js",
"types": "dist/index.d.ts",
Expand Down Expand Up @@ -49,7 +49,7 @@
"llm",
"routing",
"ai",
"control-plane",
"gateway-adapter",
"safety",
"middleware",
"express",
Expand Down