Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

## Active tickets

- [ ] [`ticket-034`](project/ticket-034/README.md) — scale each OpenRouter chat
deadline deterministically from input size, output budget and structural
complexity. Current state: `IN_PROGRESS / PUBLICATION`; governance, 349-test
verification, gold, SDK examples and Docker smoke pass on the validated
ticket-027 publication base.

- [ ] [`ticket-018`](project/ticket-018/README.md) — enforce the
`wellmanifest/new-project` manifest as policy-as-code through a deterministic
validator, trusted approval boundary, reusable governance CI, stack-specific
Expand Down
1 change: 1 addition & 0 deletions project/TICKETS.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,5 @@ of `project/README.md`, which remains a generated technical-analysis artifact.
| **ticket-026** | [`README.md`](./ticket-026/README.md) | [`preprompt.md`](./ticket-026/preprompt.md) | - | [`ai-codex.md`](./ticket-026/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-026/ai-codex-logs.txt) | [`changelog.md`](./ticket-026/changelog.md) |
| **ticket-027** | [`README.md`](./ticket-027/README.md) | [`preprompt.md`](./ticket-027/preprompt.md) | - | [`ai-codex.md`](./ticket-027/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-027/ai-codex-logs.txt) | [`changelog.md`](./ticket-027/changelog.md) |
| **ticket-031** | [`README.md`](./ticket-031/README.md) | [`preprompt.md`](./ticket-031/preprompt.md) | - | [`ai-codex.md`](./ticket-031/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-031/ai-codex-logs.txt) | [`changelog.md`](./ticket-031/changelog.md) |
| **ticket-034** | [`README.md`](./ticket-034/README.md) | [`preprompt.md`](./ticket-034/preprompt.md) | - | [`ai-codex.md`](./ticket-034/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-034/ai-codex-logs.txt) | [`changelog.md`](./ticket-034/changelog.md) |
<!-- AUTO:TICKET_INDEX:END -->
99 changes: 99 additions & 0 deletions project/ticket-034/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Ticket 034: Scale LLM timeout by input complexity

- **ID**: ticket-034
- **Owner**: unresolved:human
- **Status**: IN_PROGRESS
- **Workflow state**: PUBLICATION
- **Created**: 2026-08-04

## Goal and scope

Derive each OpenRouter request timeout from the configured base timeout, input
size, requested output size and structural complexity. Small requests retain the
current timeout. Crossing a baseline doubles it; each further doubling of load
doubles it again, up to a bounded maximum.

This responds to a live Subactor audit where a short NL request completed, while
the bounded multi-document pipeline legitimately ran for several minutes. The
change must distinguish one-request timeout from total pipeline duration and
must not hide exhausted-credit, schema or external-cancellation failures.

## Proposed deterministic policy

For a chat-completion body calculate:

- `inputRatio = serialized request characters / 8_000`;
- `outputRatio = max_tokens / 6_000`;
- `complexityRatio = complexity points / 4`, where message count contributes
one point, strict JSON Schema contributes two, and response healing contributes
one;
- `pressure = max(1, inputRatio, outputRatio, complexityRatio)`;
- `steps = ceil(log2(pressure))`;
- `multiplier = min(8, 2^steps)`;
- `effectiveTimeout = min(600_000 ms, baseTimeout * multiplier)`.

Therefore an input just above the baseline gets `2×`, above twice the baseline
gets `4×`, and above four times gets `8×`. The existing
`OPENROUTER_TIMEOUT_MS` and documentation-specific base timeout remain minimums,
not replaced defaults.

## Bounded implementation paths

- `src/llm/openrouter-timeout.ts`: pure pressure/timeout calculation.
- `src/llm/openrouter.ts`: apply the effective timeout to chat completion
requests and report base/effective values on timeout.
- `src/llm/audit.ts`: persist the non-secret scaling policy with LLM audit
configuration.
- `test/openrouter-timeout.test.ts`: boundary, cap and cancellation regressions.
- Governance evidence under `project/ticket-034/**` and indexes.

Model selection, token budgets, retry counts, chunking, concurrency, provider
fallback and the `/models` endpoint are out of scope.

## Acceptance criteria

- [x] AC-01: A human approves the formula and bounded paths after ticket-027 is
integrated or closed.
- [x] AC-02: Requests at or below all baselines retain the exact configured base
timeout.
- [x] AC-03: Crossing one, two and four baseline units produces `2×`, `4×` and
`8×` timeouts respectively.
- [x] AC-04: The result never exceeds 600 seconds and rejects non-finite or
malformed request values without silently granting an unbounded timeout.
- [x] AC-05: Structured schemas and response-healing complexity contribute to
scaling independently of raw character count.
- [x] AC-06: External `AbortSignal` cancellation remains immediate and is never
extended by adaptive timeout logic.
- [x] AC-07: Retry backoff remains inside one effective request deadline; the
change does not multiply each retry into a separate unbounded deadline.
- [x] AC-08: Timeout errors state both base and effective milliseconds; audit
configuration records the factor, baselines and cap without secrets.
- [x] AC-09: Focused tests, full `npm run verify`, Docker smoke and governance
pass on the integrated base.

## Validation

- `make governance`: PASS, 0 errors and 0 warnings.
- `npm run verify`: PASS, 349 tests, 348 passed, 1 optional JDK skip.
- `npm run evaluate:gold`: PASS, all measured precision and recall 100%.
- `npm run examples:check`: PASS, five SDK fingerprints agree.
- `make docker-smoke`: PASS.

## Resolved blockers

- Ticket-027 was closed on the validated repair line at `c51bf19`. The current
refactored base already contains its array narrowing and total edit-path
handling in the split helper modules, so importing its full historical stack
would only introduce unrelated conflicts.
- Implementation uses this clean ticket worktree. The unrelated edits in the
main development worktree remain untouched.

## Approval boundary

The user's `kontynuuj` on 2026-08-04 approves this formula and bounded scope.
The ticket may enter `EDIT`; protected review remains required for merge.

## Participants

- Human participant: unresolved; no `user-*` file was created.
- Agent participant: [ai-codex.md](ai-codex.md)
9 changes: 9 additions & 0 deletions project/ticket-034/ai-codex-logs.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
2026-08-04 user approval: "kontynuuj"; state WAIT_FOR_APPROVAL -> EDIT
2026-08-04 local OpenRouter model: z-ai/glm-5.2; ignored .env only, no secret changed
2026-08-04 ticket-027 closure verified at c51bf19; equivalent split behavior present on current base
2026-08-04 focused OpenRouter suites PASS: 26/26 including 7 adaptive timeout tests
2026-08-04 make governance PASS: 0 errors, 0 warnings
2026-08-04 npm run verify PASS: 349 total, 348 pass, 0 fail, 1 optional JDK skip
2026-08-04 gold v2 PASS: all measured precision/recall 100%; stability PASS
2026-08-04 examples PASS: five SDK fingerprints agree
2026-08-04 Docker smoke PASS; state EDIT -> PUBLICATION
43 changes: 43 additions & 0 deletions project/ticket-034/ai-codex.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
participant-id: agent:codex
participant: codex
role: agent
ticket: ticket-034
---
# Participant: codex (AI agent)

## Understanding

The configured timeout is currently a fixed deadline for the complete request,
including transport retries. It does not account for prompt/schema size or
requested output. Documentation has a separate 45-second base, but large strict
JSON requests can therefore receive less time than much smaller generic calls.

## Execution plan

1. Confirm the active LLM repair is closed and its behavior is present in the
current split implementation.
2. Add a pure bounded timeout calculator with explicit baselines and factor.
3. Apply it once per chat request before creating the abort timer.
4. Keep external cancellation and retry behavior unchanged.
5. Expose the policy in safe audit configuration and timeout errors.
6. Run boundary tests, full verification, Docker smoke and governance.

## Actual changes

- Created ticket-034 and recorded the proposed formula.
- Recorded the user's explicit continuation as approval and entered `EDIT`.
- Configured the ignored local OpenRouter environment to use `z-ai/glm-5.2`;
no API key or other secret was changed.
- Added a pure timeout policy and applied one effective deadline across HTTP
retries and their abortable backoff.
- Added timeout policy fields to secret-free audit configuration and base plus
effective durations to timeout errors.
- Added seven boundary, cap, malformed-input, audit and cancellation tests.
- Full verify, gold, SDK examples, governance and Docker smoke pass on the
validated ticket-027 publication base.

## Blockers

- Implementation and validation are complete. Protected review remains an
external publication requirement.
20 changes: 20 additions & 0 deletions project/ticket-034/changelog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Ticket Changelog (ticket-034)

## [0.1.0] - 2026-08-04

- Created the adaptive LLM timeout governance plan.
- Defined deterministic `1×`/`2×`/`4×`/`8×` scaling and a 600-second cap.
- Recorded ticket-027 and the dirty OpenRouter refactor as blockers.
- Stopped at `BACKLOG / WAIT_FOR_APPROVAL`; no executable files changed.
- Recorded the user's approval and moved to `IN_PROGRESS / EDIT`.
- Confirmed ticket-027 is closed on its validated repair line and that the
current split base already carries the relevant behavior.
- Selected `z-ai/glm-5.2` in the ignored local OpenRouter configuration.
- Added deterministic timeout pressure from serialized input, output token
budget, message count, strict JSON Schema and response healing.
- Kept retry backoff inside one adaptive deadline and external cancellation
immediate.
- Persisted the non-secret scaling policy in audits and expanded timeout errors
with base/effective values.
- Passed governance, 349-test verification, gold v2, five SDK examples and
Docker smoke on the validated publication base.
27 changes: 27 additions & 0 deletions project/ticket-034/intent.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{
"schema": "new-project.intent/v2",
"ticket": "ticket-034",
"summary": "Scale each OpenRouter request timeout from bounded input and complexity pressure",
"workstream": "llm",
"allowedPaths": [
"src/llm/openrouter-timeout.ts",
"src/llm/openrouter.ts",
"src/llm/audit.ts",
"test/openrouter-timeout.test.ts",
"project/ticket-034/**",
"TODO.md",
"project/TICKETS.md"
],
"forbiddenPaths": [
"project/ticket-*/user-*.md",
"src/config/**",
"src/pipeline/**",
"src/extractors/**",
"src/communication/**",
"src/summary/**"
],
"stacks": ["node", "docker"],
"dependsOn": [],
"conflictsWith": [],
"integrationTicket": null
}
8 changes: 8 additions & 0 deletions project/ticket-034/preprompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Ticket preprompt

- **Task ID**: ticket-034
- **Task title**: Scale LLM timeout by input complexity
- **Created**: 2026-08-04T12:49:32Z

Keep executable implementation outside this governance/evidence directory.
Read a human-owned user-*.md file only when one exists.
10 changes: 10 additions & 0 deletions src/llm/audit.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { T2CConfig } from '../config/env.js';
import type { JsonValue } from '../core/types.js';
import { OPENROUTER_TIMEOUT_POLICY } from './openrouter-timeout.js';

/** Safe, secret-free OpenRouter parameters persisted with standalone and pipeline audits. */
export function openRouterAuditConfiguration(
Expand All @@ -11,6 +12,15 @@ export function openRouterAuditConfiguration(
model,
baseUrl: config.openRouter.baseUrl,
timeoutMs,
adaptiveTimeout: {
baseTimeoutMs: timeoutMs,
inputCharactersBaseline: OPENROUTER_TIMEOUT_POLICY.inputCharactersBaseline,
outputTokensBaseline: OPENROUTER_TIMEOUT_POLICY.outputTokensBaseline,
complexityPointsBaseline: OPENROUTER_TIMEOUT_POLICY.complexityPointsBaseline,
scaleFactor: OPENROUTER_TIMEOUT_POLICY.scaleFactor,
maximumMultiplier: OPENROUTER_TIMEOUT_POLICY.maximumMultiplier,
maximumTimeoutMs: OPENROUTER_TIMEOUT_POLICY.maximumTimeoutMs,
},
maxTokens: config.openRouter.maxTokens,
temperature: config.openRouter.temperature,
requireStructuredOutput: config.openRouter.requireStructuredOutput,
Expand Down
136 changes: 136 additions & 0 deletions src/llm/openrouter-timeout.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
export const OPENROUTER_TIMEOUT_POLICY = Object.freeze({
inputCharactersBaseline: 8_000,
outputTokensBaseline: 6_000,
complexityPointsBaseline: 4,
scaleFactor: 2,
maximumMultiplier: 8,
maximumTimeoutMs: 600_000,
});

export interface OpenRouterTimeoutLoad {
serializedInputCharacters: number;
outputTokens: number;
messageCount: number;
strictJsonSchema: boolean;
responseHealing: boolean;
}

export interface OpenRouterTimeoutDecision extends OpenRouterTimeoutLoad {
baseTimeoutMs: number;
complexityPoints: number;
pressure: number;
multiplier: number;
effectiveTimeoutMs: number;
capped: boolean;
}

/** Calculate one bounded request deadline without reading environment state. */
export function calculateOpenRouterTimeout(
baseTimeoutMs: number,
load: OpenRouterTimeoutLoad,
): OpenRouterTimeoutDecision {
assertPositiveFinite(baseTimeoutMs, 'base timeout');
if (baseTimeoutMs > OPENROUTER_TIMEOUT_POLICY.maximumTimeoutMs) {
throw new Error(`OpenRouter base timeout must not exceed ${OPENROUTER_TIMEOUT_POLICY.maximumTimeoutMs} ms`);
}
assertNonNegativeInteger(load.serializedInputCharacters, 'serialized input characters');
assertNonNegativeInteger(load.outputTokens, 'output tokens');
assertNonNegativeInteger(load.messageCount, 'message count');
if (typeof load.strictJsonSchema !== 'boolean' || typeof load.responseHealing !== 'boolean') {
throw new Error('OpenRouter timeout complexity flags must be boolean');
}

const complexityPoints = load.messageCount
+ (load.strictJsonSchema ? 2 : 0)
+ (load.responseHealing ? 1 : 0);
const pressure = Math.max(
1,
load.serializedInputCharacters / OPENROUTER_TIMEOUT_POLICY.inputCharactersBaseline,
load.outputTokens / OPENROUTER_TIMEOUT_POLICY.outputTokensBaseline,
complexityPoints / OPENROUTER_TIMEOUT_POLICY.complexityPointsBaseline,
);
const steps = pressure <= 1 ? 0 : Math.ceil(Math.log2(pressure));
const multiplier = Math.min(
OPENROUTER_TIMEOUT_POLICY.maximumMultiplier,
OPENROUTER_TIMEOUT_POLICY.scaleFactor ** steps,
);
const scaledTimeoutMs = baseTimeoutMs * multiplier;
const effectiveTimeoutMs = Math.min(
OPENROUTER_TIMEOUT_POLICY.maximumTimeoutMs,
scaledTimeoutMs,
);

return {
...load,
baseTimeoutMs,
complexityPoints,
pressure,
multiplier,
effectiveTimeoutMs,
capped: effectiveTimeoutMs < scaledTimeoutMs,
};
}

/** Derive timeout pressure from the exact JSON-compatible OpenRouter body. */
export function openRouterRequestTimeout(
body: Record<string, unknown>,
baseTimeoutMs: number,
): OpenRouterTimeoutDecision {
let serialized: string;
try {
serialized = JSON.stringify(body);
} catch (error) {
throw new Error(`OpenRouter request body must be JSON-serializable: ${error instanceof Error ? error.message : String(error)}`);
}
if (serialized === undefined) {
throw new Error('OpenRouter request body must serialize to a JSON object');
}

const messages = optionalArray(body.messages, 'messages');
const plugins = optionalArray(body.plugins, 'plugins');
const responseFormat = optionalObject(body.response_format, 'response_format');
const jsonSchema = responseFormat?.type === 'json_schema'
? optionalObject(responseFormat.json_schema, 'response_format.json_schema')
: undefined;
const maxTokens = body.max_tokens === undefined ? 0 : body.max_tokens;
assertNonNegativeInteger(maxTokens, 'max_tokens');

return calculateOpenRouterTimeout(baseTimeoutMs, {
serializedInputCharacters: serialized.length,
outputTokens: maxTokens,
messageCount: messages?.length ?? 0,
strictJsonSchema: responseFormat?.type === 'json_schema' && jsonSchema?.strict === true,
responseHealing: plugins?.some((plugin) => {
if (plugin === null || typeof plugin !== 'object' || Array.isArray(plugin)) {
throw new Error('OpenRouter request plugins must contain objects');
}
return (plugin as Record<string, unknown>).id === 'response-healing';
}) ?? false,
});
}

function optionalArray(value: unknown, name: string): unknown[] | undefined {
if (value === undefined) return undefined;
if (!Array.isArray(value)) throw new Error(`OpenRouter request ${name} must be an array`);
return value;
}

function optionalObject(value: unknown, name: string): Record<string, unknown> | undefined {
if (value === undefined) return undefined;
if (value === null || typeof value !== 'object' || Array.isArray(value)) {
throw new Error(`OpenRouter request ${name} must be an object`);
}
return value as Record<string, unknown>;
}

function assertPositiveFinite(value: number, name: string): void {
if (typeof value !== 'number' || !Number.isFinite(value) || value <= 0) {
throw new Error(`OpenRouter ${name} must be a positive finite number`);
}
}

function assertNonNegativeInteger(value: unknown, name: string): asserts value is number {
if (typeof value !== 'number' || !Number.isSafeInteger(value) || value < 0) {
throw new Error(`OpenRouter ${name} must be a non-negative safe integer`);
}
}
Loading
Loading