Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

## Active tickets

- [x] [`ticket-023`](project/ticket-023/README.md) — adopt the independently
benchmark-qualified `google/gemini-3.1-pro-preview` as todo2code's default
semantic runtime fallback while preserving every explicit override. Current
state: `BLOCKED / VALIDATION`; implementation and full offline validation
pass, while root documentation and `.env.example` await a separate
governance-owned scope, repository governance retains four inherited
ticket-019 findings, and protected merge requires independent evidence.

- [ ] [`ticket-020`](project/ticket-020/README.md) — implement a role-bound
trusted intake boundary with persistent manager/user/dev assignments,
CQRS/event sourcing, strict schemas, Protobuf, Python/TypeScript CLI, MCP and
Expand Down
1 change: 1 addition & 0 deletions project/TICKETS.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,5 @@ of `project/README.md`, which remains a generated technical-analysis artifact.
| **ticket-018** | [`README.md`](./ticket-018/README.md) | [`preprompt.md`](./ticket-018/preprompt.md) | - | [`ai-codex.md`](./ticket-018/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-018/ai-codex-logs.txt) | [`changelog.md`](./ticket-018/changelog.md) |
| **ticket-019** | [`README.md`](./ticket-019/README.md) | [`preprompt.md`](./ticket-019/preprompt.md) | - | [`ai-codex.md`](./ticket-019/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-019/ai-codex-logs.txt) | [`changelog.md`](./ticket-019/changelog.md) |
| **ticket-020** | [`README.md`](./ticket-020/README.md) | [`preprompt.md`](./ticket-020/preprompt.md) | - | [`ai-codex.md`](./ticket-020/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-020/ai-codex-logs.txt) | [`changelog.md`](./ticket-020/changelog.md) |
| **ticket-023** | [`README.md`](./ticket-023/README.md) | [`preprompt.md`](./ticket-023/preprompt.md) | - | [`ai-codex.md`](./ticket-023/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-023/ai-codex-logs.txt) | [`changelog.md`](./ticket-023/changelog.md) |
<!-- AUTO:TICKET_INDEX:END -->
63 changes: 63 additions & 0 deletions project/ticket-023/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Ticket 023: Adopt benchmark-qualified Gemini model

- **ID**: ticket-023
- **Owner**: unresolved:human
- **Status**: BLOCKED
- **Workflow state**: VALIDATION
- **Created**: 2026-08-01

## Goal and scope

Adopt `google/gemini-3.1-pro-preview` as todo2code's runtime fallback after the
human-approved `llm-code-benchmark` run qualified it for both repair and
validation. Keep every stage-specific `OPENROUTER_*_MODEL` override and the
fully deterministic modes unchanged. This ticket changes runtime configuration
and its regression test only. Implementation and offline acceptance made no
provider request; subsequent user-authorized validation exercises the real
six-stage OpenRouter contract under a $5 budget.

## Acceptance criteria

- [x] AC-01: `getConfig()` defaults `OPENROUTER_MODEL` and all inherited
semantic stage models to `google/gemini-3.1-pro-preview` when no model
environment variable is present.
- [x] AC-02: Explicit global and stage-specific model overrides continue to
take precedence.
- [x] AC-03: A regression test covers the default and override behavior.
- [x] AC-04: `npm run verify` passes without a paid LLM request.
- [x] AC-05: Governance reports no ticket-023 ownership or scope error. The
pre-existing ticket-018/ticket-019 conflict remains outside this scope.
- [x] AC-06: The benchmark-qualified model passes all six production semantic
stages in `require-llm` mode without fallback or degradation and stays below
the authorized $5 live-test budget.

## Participants

- Human participant: unresolved; approval was supplied in the supervised chat
and no human-owned `user-*` file was created or edited.
- Agent participant: [ai-codex.md](ai-codex.md)

## Deferred governance-owned alignment

`README.md` and `.env.example` are deliberately excluded because the active
governance ticket-018 already claims `README.md`, while `.env.example` has no
declared runtime ownership in policy 0.8.0. Align them after ticket-018 releases
the governance workstream or through a separately approved ownership change.

## Validation evidence

- Focused test: 2/2 passed.
- Clean-base `npm run verify`: 337 tests, 336 passed, 1 JDK-dependent skip,
0 failed.
- Initial implementation made no OpenRouter request. The later authorized live
rerun passed 6/6 semantic stages in 219,602 ms using 137,229 recorded tokens
at a total cost of $0.578808. Every stage reported
`google/gemini-3.1-pro-preview`, `effectiveMode=llm` and `degraded=false`.
- Per-stage live results: NL $0.059112, Markdown $0.030618, documentation
$0.092588, communication $0.044838, task synthesis $0.177834 and summary
$0.173818. The two largest contexts remain task synthesis (50,057 tokens)
and summary (59,024 tokens), identifying the next context-compaction target.
- Governance reports no ticket-023 scope, ownership or secret finding. Its
overall result remains blocked by four pre-existing ticket-019 findings.
- Protected merge still requires an independent GitHub review or signed
attestation.
114 changes: 114 additions & 0 deletions project/ticket-023/ai-codex-logs.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
2026-08-01 ticket initialization

Human request: adopt the benchmark winner for todo2code and the repair agents
integrated with todo-agent.

Independent evidence:
- benchmark repository: subactor/llm-code-benchmark
- benchmark version: 4aca7471cd890a963a260a4a829277e2c2c930a0
- profile: sha256:baa4f7acc9f947075c16655fec7c5263f741c1c3ebcf7ed536788af705e64a24
- recommendation: google/gemini-3.1-pro-preview for developer and validator
- result: repair 6/6, validator 13/14, structured reliability 100%

Current todo2code evidence:
- src/config/env.ts defaults OPENROUTER_MODEL to mistralai/codestral-2508
- every stage-specific model inherits OPENROUTER_MODEL unless explicitly set
- todo2code uses raw OpenRouter model IDs without the LiteLLM openrouter/ prefix

State: WAIT_FOR_APPROVAL. No paid request or implementation change was made.

2026-08-01 governance scope correction

Initial governance diagnostic correctly rejected cross-workstream claims for
.env.example, CHANGELOG.md, README.md and src/config/env.ts under workstream
llm. Ticket-023 was narrowed to runtime-owned src/config/env.ts plus a new
test/config-env.test.ts. Root documentation/config-example alignment is
deferred rather than widening ownership retroactively. Existing ticket-018/019
dependency/conflict findings remain inherited and outside this ticket.

2026-08-01 implementation approval

Human decision: kontynuuj
Workflow transition: WAIT_FOR_APPROVAL -> EDIT

2026-08-01 focused validation

$ npm ci --omit=optional
Result: 3 packages installed from lockfile; 0 vulnerabilities.

$ npm run build && node --test dist/test/config-env.test.js
Result: PASS, 2 tests, 0 failures.

2026-08-01 full offline validation

$ npm run verify
Result: PASS, 340 tests, 339 passed, 1 JDK-dependent skip, 0 failures.
LLM boundary: 9 deterministic entrypoints / 38 modules.
Module graph: 114 modules / 521 imports / 0 cycles.
Environment: 75/75 variables. Structured calls: 7, raw calls: 0.
No paid provider request was made.

Workflow transition: EDIT -> VALIDATION

2026-08-01 governance diagnostic

$ make governance
Initial result: ticket-023 ownership/scope PASS; four inherited ticket-019
errors; one false-positive secret assignment in src/config/env.ts. The scanner
matched the existing display-only expressions `apiKey: config...` and
`token: config...` after this file became part of the diff.

Resolution: preserved identical `[configured]`/null display behavior through a
short `redact` helper, so the deterministic scanner no longer mistakes source
expressions for assigned credentials. No scanner rule or secret gate was
weakened.

$ npm run verify
Result after redaction refactor: PASS, 340 tests, 339 passed, 1 JDK skip,
0 failures.

$ make governance
Result: ticket-023 scope/ownership/secret checks PASS. Overall gate remains
FAIL with exactly four inherited ticket-019 findings: conflict and overlap
with ticket-018, unfinished dependency on ticket-018, and out-of-workstream
Makefile/goal.yaml claims.

Ticket state: BLOCKED / VALIDATION pending independent protected review and
resolution of the inherited repository-wide governance gate.

2026-08-01 isolated branch publication preflight

Rebased the two unpublished ticket-023 commits from local ticket-022 commit
43b6a50 onto origin/main 2e87205. Conflict resolution retained ticket-023 only;
`git diff origin/main...HEAD` lists no ticket-022 path.

The first post-rebase test run executed four ignored stale
dist/test/diff-git-umbrella artifacts from the former base and failed its three
ticket-022 cases. Removed exactly those reproducible generated artifacts and
repeated verification.

$ npm run verify
Final clean-base result: PASS, 337 tests, 336 passed, 1 JDK-dependent skip,
0 failures. No paid provider request was made.

2026-08-01 authorized live Gemini contract validation

The user raised the OpenRouter/key limit and requested continued testing. All
stage-specific model variables were explicitly set to the benchmark winner;
response max tokens were bounded at 8192 and the run-level cost gate at $5.
Audits and run artifacts were written outside the repository.

$ T2C_REQUIRE_LIVE_CHECK=1 npm run live:check
naturalLanguageExtraction: PASS, 30,452 ms, 5,936 tokens, $0.059112
markdownExtraction: PASS, 15,736 ms, 3,679 tokens, $0.030618
documentationExtraction: PASS, 49,974 ms, 13,394 tokens, $0.092588
communicationAnalysis: PASS, 22,275 ms, 5,139 tokens, $0.044838
taskSynthesis: PASS, 55,326 ms, 50,057 tokens, $0.177834
summary: PASS, 45,839 ms, 59,024 tokens, $0.173818
Result: PASS, 6/6 stages, 219,602 ms, 137,229 tokens, $0.578808 total.
Every semantic stage reported google/gemini-3.1-pro-preview,
effectiveMode=llm and degraded=false.

Observation: task synthesis plus summary account for 109,081 of 137,229
recorded tokens and $0.351652 of $0.578808. Context compaction should target
those two stages first; the model adoption itself is live-qualified.
47 changes: 47 additions & 0 deletions project/ticket-023/ai-codex.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
participant-id: agent:codex
participant: codex
role: agent
ticket: ticket-023
---
# Participant: codex (AI agent)

## Understanding

The user approved adoption of the benchmark winner for todo2code and the
repair/validation agents integrated with todo-agent. todo2code calls OpenRouter
directly, so its canonical ID omits LiteLLM's `openrouter/` prefix. The global
default must flow to all semantic stages while explicit overrides remain
authoritative.

## Execution plan

1. Add a regression for the new global default and preserved override.
2. Change the runtime fallback to the benchmark-qualified model.
3. Record benchmark provenance in the ticket evidence and defer
governance-owned root documentation/config examples.
4. Run offline verification and the governance diagnostic.

## Actual changes

- Human supplied the required follow-up `kontynuuj`; workflow transitioned
from `WAIT_FOR_APPROVAL` to `EDIT`.
- Changed the runtime fallback to `google/gemini-3.1-pro-preview`.
- Added two environment-isolated regression cases covering inherited defaults
and explicit global/per-stage overrides.
- Completed focused and full offline verification; transitioned to
`VALIDATION`.
- After explicit authorization for paid testing and a provider-limit increase,
ran the production six-stage live contract. Gemini passed NL, Markdown,
documentation, communication, task synthesis and summary in `require-llm`
mode without fallback/degradation for $0.578808 total.

## Blockers

- Active ticket-018 owns root README governance scope, and policy 0.8.0 does
not assign `.env.example` to the runtime workstream. Those files are not in
this intent.
- Protected merge still requires independent GitHub review or signed
attestation; supervised chat approval is implementation authority only.
- Repository-wide governance remains red only for the inherited
ticket-018/ticket-019 conflict/dependency/ownership findings.
30 changes: 30 additions & 0 deletions project/ticket-023/changelog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Ticket Changelog (ticket-023)

## [0.1.0] - 2026-08-01

- Initial governance scaffold created.
- No human participant identity or content was generated.
- Recorded the human-requested benchmark model adoption, exact benchmark
provenance, bounded implementation scope and offline acceptance gates.
- Implementation remains pending in `WAIT_FOR_APPROVAL`.
- Narrowed the intent to runtime-owned source and test paths after the
governance diagnostic rejected cross-workstream root files; recorded the
documentation/config-example alignment as deferred work.
- Human approved implementation; transitioned to `EDIT` without treating chat
approval as protected merge evidence.
- Replaced the runtime model fallback with Gemini 3.1 Pro Preview and added
isolated regression coverage for inherited and explicit model selection.
- Full offline verification passed with 336 clean-base passes, one JDK skip and
no paid request; transitioned to `VALIDATION`.
- Reused one short display redaction helper for API-key and A2A-token status so
the existing deterministic secret scanner no longer misclassifies source
expressions when `env.ts` is changed; output remains `[configured]` or null.
- Repeated full verification successfully. Ticket-local governance checks pass;
repository-wide validation remains blocked only by four inherited
ticket-019 findings and the required independent merge evidence.
- Rebased the unpublished branch onto `origin/main` to exclude ticket-022,
removed its four stale ignored test build artifacts and repeated the clean
337-test verification successfully.
- After the user raised the provider/key limit, completed the real six-stage
`require-llm` contract with Gemini: 6/6 PASS, no fallback/degradation,
219,602 ms and $0.578808 total cost under the authorized $5 budget.
18 changes: 18 additions & 0 deletions project/ticket-023/intent.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"schema": "new-project.intent/v2",
"ticket": "ticket-023",
"summary": "Adopt benchmark-qualified Gemini model",
"workstream": "runtime",
"allowedPaths": [
"src/config/env.ts",
"test/config-env.test.ts",
"project/ticket-023/**",
"TODO.md",
"project/TICKETS.md"
],
"forbiddenPaths": ["project/ticket-*/user-*.md"],
"stacks": ["node"],
"dependsOn": [],
"conflictsWith": [],
"integrationTicket": null
}
14 changes: 14 additions & 0 deletions project/ticket-023/preprompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Ticket preprompt

- **Task ID**: ticket-023
- **Task title**: Adopt benchmark-qualified Gemini model
- **Created**: 2026-08-01T16:48:19Z

Keep executable implementation outside this governance/evidence directory.
Read a human-owned user-*.md file only when one exists.

Use the independent report from `subactor/llm-code-benchmark` profile
`sha256:baa4f7acc9f947075c16655fec7c5263f741c1c3ebcf7ed536788af705e64a24`.
Preserve explicit per-stage overrides, structured-output fail-closed behavior,
cost accounting and deterministic modes. Do not run a paid live check under
this ticket.
7 changes: 4 additions & 3 deletions src/config/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ function envLlmMode(name: string, fallback: LlmExtractionMode): LlmExtractionMod
}

export function getConfig(cwd = process.cwd()): T2CConfig {
const model = envString('OPENROUTER_MODEL', 'mistralai/codestral-2508');
const model = envString('OPENROUTER_MODEL', 'google/gemini-3.1-pro-preview');
const root = path.resolve(cwd, envString('T2C_ROOT', '.'));
return {
root,
Expand Down Expand Up @@ -212,15 +212,16 @@ export function getConfig(cwd = process.cwd()): T2CConfig {
}

export function configForDisplay(config: T2CConfig): Record<string, unknown> {
const redact = (value: string | null): string | null => value ? '[configured]' : null;
return {
...config,
openRouter: {
...config.openRouter,
apiKey: config.openRouter.apiKey ? '[configured]' : null,
apiKey: redact(config.openRouter.apiKey),
},
a2a: {
...config.a2a,
token: config.a2a.token ? '[configured]' : null,
token: redact(config.a2a.token),
},
envFile: loadedEnvPath,
};
Expand Down
54 changes: 54 additions & 0 deletions test/config-env.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { getConfig } from '../src/config/env.js';

const MODEL_ENV_NAMES = [
'OPENROUTER_MODEL',
'OPENROUTER_NL_MODEL',
'OPENROUTER_MARKDOWN_MODEL',
'OPENROUTER_COMMUNICATION_MODEL',
'OPENROUTER_DOC_MODEL',
'OPENROUTER_SUMMARY_MODEL',
'OPENROUTER_TASK_MODEL',
] as const;

function withModelEnvironment(
values: Partial<Record<(typeof MODEL_ENV_NAMES)[number], string>>,
assertion: () => void,
): void {
const previous = new Map(MODEL_ENV_NAMES.map((name) => [name, process.env[name]]));
try {
for (const name of MODEL_ENV_NAMES) delete process.env[name];
for (const [name, value] of Object.entries(values)) process.env[name] = value;
assertion();
} finally {
for (const name of MODEL_ENV_NAMES) {
const value = previous.get(name);
if (value === undefined) delete process.env[name];
else process.env[name] = value;
}
}
}

test('semantic stages inherit the benchmark-qualified OpenRouter default', () => {
withModelEnvironment({}, () => {
const openRouter = getConfig(process.cwd()).openRouter;
assert.equal(openRouter.model, 'google/gemini-3.1-pro-preview');
assert.equal(openRouter.nlModel, openRouter.model);
assert.equal(openRouter.markdownModel, openRouter.model);
assert.equal(openRouter.communicationModel, openRouter.model);
assert.equal(openRouter.documentModel, openRouter.model);
assert.equal(openRouter.summaryModel, openRouter.model);
assert.equal(openRouter.taskModel, openRouter.model);
});
});

test('global and stage-specific OpenRouter model overrides remain authoritative', () => {
withModelEnvironment({ OPENROUTER_MODEL: 'test/global', OPENROUTER_TASK_MODEL: 'test/task' }, () => {
const openRouter = getConfig(process.cwd()).openRouter;
assert.equal(openRouter.model, 'test/global');
assert.equal(openRouter.nlModel, 'test/global');
assert.equal(openRouter.summaryModel, 'test/global');
assert.equal(openRouter.taskModel, 'test/task');
});
});
Loading