diff --git a/TODO.md b/TODO.md index 7ce22cb..35019d1 100644 --- a/TODO.md +++ b/TODO.md @@ -2,6 +2,11 @@ ## Active tickets +- [ ] [`ticket-030`](project/ticket-030/README.md) — repair the stale + confidence-hierarchy source lookup after the NL and Markdown clamp logic + moved into focused helper modules. Current state: `IN_PROGRESS / VALIDATION`; + the production ceilings remain unchanged and the implementation is test-only. + - [ ] [`ticket-019`](project/ticket-019/README.md) — publish the dependency-free Python SDK as the root PyPI distribution `todo2code` through `goal -a`, with one root `pyproject.toml` and SDK-only artifacts. Current state: diff --git a/project/TICKETS.md b/project/TICKETS.md index 071ccc8..b1968fc 100644 --- a/project/TICKETS.md +++ b/project/TICKETS.md @@ -27,4 +27,5 @@ of `project/README.md`, which remains a generated technical-analysis artifact. | **ticket-019** | [`README.md`](./ticket-019/README.md) | [`preprompt.md`](./ticket-019/preprompt.md) | - | [`ai-codex.md`](./ticket-019/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-019/ai-codex-logs.txt) | [`changelog.md`](./ticket-019/changelog.md) | | **ticket-020** | [`README.md`](./ticket-020/README.md) | [`preprompt.md`](./ticket-020/preprompt.md) | - | [`ai-codex.md`](./ticket-020/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-020/ai-codex-logs.txt) | [`changelog.md`](./ticket-020/changelog.md) | | **ticket-022** | [`README.md`](./ticket-022/README.md) | [`preprompt.md`](./ticket-022/preprompt.md) | - | [`ai-codex.md`](./ticket-022/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-022/ai-codex-logs.txt) | [`changelog.md`](./ticket-022/changelog.md) | +| **ticket-030** | [`README.md`](./ticket-030/README.md) | [`preprompt.md`](./ticket-030/preprompt.md) | - | [`ai-codex.md`](./ticket-030/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-030/ai-codex-logs.txt) | [`changelog.md`](./ticket-030/changelog.md) | diff --git a/project/ticket-030/README.md b/project/ticket-030/README.md new file mode 100644 index 0000000..da4d376 --- /dev/null +++ b/project/ticket-030/README.md @@ -0,0 +1,71 @@ +# Ticket 030: Repair confidence hierarchy test after extractor refactor + +- **ID**: ticket-030 +- **Owner**: unresolved:human +- **Status**: IN_PROGRESS +- **Workflow state**: VALIDATION +- **Created**: 2026-08-04 + +## Goal and scope + +Repair the confidence-hierarchy regression test that failed after the Markdown +and NL confidence clamping logic moved from the orchestration modules into +focused helper modules. The production ceilings remain correct (`0.94`, +`0.90`, `0.85`); the failure is a test-discovery defect, not evidence that an +Intent record contains `confidence: null`. + +The implementation is deliberately test-only. It will update the hierarchy +test so it verifies the canonical current locations without changing any +extractor behavior, confidence value, documentation contract or LLM boundary. +No unrelated refactor or generated artifact belongs to this ticket. + +## Planned changed paths + +- `test/nl-llm.test.ts`: replace the stale source-file assumptions with a + deterministic check of the current confidence-clamping implementations. +- `project/ticket-030/**`, `TODO.md`, `project/TICKETS.md`: ticket intent and + verification evidence only. + +## Acceptance criteria + +- [x] AC-01: Scope is approved by a human owner. +- [x] AC-02: The test distinguishes a missing source-code match from a runtime + confidence value and no longer reports the former as extractor failure. +- [x] AC-03: The hierarchy remains Markdown `0.94` > NL `0.90` > documentation + `0.85`, with all three ceilings below deterministic observation levels. +- [x] AC-04: Focused NL/Markdown/document tests pass without a live provider, + changing production code or accepting a looser confidence ceiling. +- [x] AC-05: The ticket diff contains only its allowed test and governance + paths; unrelated dirty/generated files remain untouched. + +## Participants + +- Human participant: unresolved; no user-* file was created by this script. +- Agent participant: [ai-codex.md](ai-codex.md) + +## Approval boundary + +- Current state: `IN_PROGRESS / VALIDATION`. +- Required response from: `unresolved:human`. +- Chat approval authorizes implementation in this interactive session but is + not trusted merge evidence. +- The ticket was renumbered from the locally drafted `ticket-023` to + `ticket-030` before publication because remote PR #3 already owns 023. The + approved implementation scope and code delta did not change. + +## Validation evidence + +- The freshly emitted `dist/test/nl-llm.test.js` passes `11/11`; the repaired + hierarchy assertion passes and no live provider is called. +- The reported 338-test run used an older `dist` tree: it correctly exposed the + stale hierarchy lookup, but by itself was not proof that current TypeScript + sources build cleanly. After rebasing onto base `caf6551`, a clean + `npm run build` passes. +- The full freshly built suite reports 338 tests: 327 passed, 10 failed and one + skipped. The repaired confidence-hierarchy test passes. The ten inherited + failures concern AST/linker relations, communication identity/enrichment and + gold-dataset relations; none is in ticket-030 scope or caused by its two-line + source-location update. +- Scoped `git diff --check` passes. Whole-repository governance continues to + report only the pre-existing ticket-018/ticket-019 conflict, dependency, + workstream ownership and shared `Makefile` overlap. diff --git a/project/ticket-030/ai-codex-logs.txt b/project/ticket-030/ai-codex-logs.txt new file mode 100644 index 0000000..6c2070d --- /dev/null +++ b/project/ticket-030/ai-codex-logs.txt @@ -0,0 +1,8 @@ +2026-08-04 ticket-030 validation +- `node --test dist/test/nl-llm.test.js`: PASS (11/11). +- `npm run build`: FAIL (17 inherited TypeScript diagnostics outside allowed implementation path). +- `npm test` after the failed fresh emit: FAIL (292 total, 253 pass, 38 fail, 1 skip); base import/runtime regressions exposed. +- The user-reported 338-test run was based on stale `dist`; useful for reproducing the hierarchy assertion only, not for clean-source verification. +- `make governance`: FAIL with the four inherited ticket-018/ticket-019 diagnostics (`GOV-CONFLICT-001`, `GOV-DEPENDENCY-002`, `GOV-WORKSTREAM-003`, `GOV-WORKSTREAM-004`); no ticket-030 diagnostic was reported. +- Rebased onto updated PR-5 base `caf6551`: `npm run build` PASS. +- Fresh `npm test`: FAIL (338 total, 327 pass, 10 fail, 1 skip). Failures are AST/linker relation (2 direct plus gold cohorts), communication identity/enrichment (3), and configuration/document relation coverage; the ticket-030 test passes. diff --git a/project/ticket-030/ai-codex.md b/project/ticket-030/ai-codex.md new file mode 100644 index 0000000..5d5ca36 --- /dev/null +++ b/project/ticket-030/ai-codex.md @@ -0,0 +1,56 @@ +--- +participant-id: agent:codex +participant: codex +role: agent +ticket: ticket-030 +--- +# Participant: codex (AI agent) + +## Understanding + +The failed assertion's `actual: null` is the result of +`source.match(...)`, not the confidence stored in an Intent record. The test +still reads `markdown-llm.ts` and `nl-llm.ts`, while their clamping expressions +now live in `markdown-llm-helpers.ts` and `nl-llm-helpers.ts`. The documented +and runtime ceilings remain `0.94`, `0.90` and `0.85`. + +The narrow repair belongs to the `llm` workstream and does not overlap the +active governance or SDK implementation paths. It must not absorb the user's +concurrent synthesis refactor or generated `project/` artifacts. + +## Execution plan + +1. Stop at `WAIT_FOR_APPROVAL` before editing the test. +2. Update only the confidence-hierarchy test to inspect the canonical current + implementations, retaining strict numeric assertions and ordering. +3. Run the focused compiled/TypeScript tests without a live OpenRouter call. +4. Review the exact diff and run scoped governance; record any inherited + repository failures separately from this test-only change. + +## Actual changes + +- The user explicitly approved the bounded ticket-030 plan in chat. The + implementation session is authorized; merge approval remains external. +- Updated only `test/nl-llm.test.ts`: the hierarchy test now reads the Markdown + and NL helper modules that own their confidence clamps after refactoring. +- Ran the freshly emitted compiled test file: 11 passed, 0 failed, with the + hierarchy assertion passing. No provider request was made. +- Rechecked the updated base from a clean dependency installation. Base + `caf6551` builds successfully, and the freshly built complete suite reports + 327 passed, 10 failed and one skipped out of 338. The confidence-hierarchy + test passes; the remaining failures are inherited relation and communication + regressions outside this ticket. +- Renumbered the unpublished local ticket from 023 to 030 after discovering + that GitHub PR #3 already owns ticket-023; no implementation path or + acceptance criterion changed. + +## Blockers + +- Whole-repository governance already fails because ticket-018 and ticket-019 + declare a conflict and unfinished dependency. This ticket does not rewrite + either intent or claim that unrelated blocker. +- The main worktree contains user-owned generated and synthesis changes. Work + remains isolated in a separate branch/worktree. +- Full verification is blocked by ten source-behaviour test failures outside + the approved ticket paths. The base branch needs separately scoped repairs + before it can be considered merge-ready. diff --git a/project/ticket-030/changelog.md b/project/ticket-030/changelog.md new file mode 100644 index 0000000..dd6ad4a --- /dev/null +++ b/project/ticket-030/changelog.md @@ -0,0 +1,16 @@ +# Ticket Changelog (ticket-030) + +## [0.1.0] - 2026-08-04 + +- Initial governance scaffold created. +- No human participant identity or content was generated. + +## [0.1.1] - 2026-08-04 + +- Repaired the stale Markdown/NL confidence-clamp source locations in the + hierarchy test without changing production confidence behavior. +- Focused compiled NL LLM test passes 11/11. After rebasing onto `caf6551`, a + clean build passes; the complete fresh suite exposes ten inherited relation + and communication failures while the repaired hierarchy assertion stays + green. +- Renumbered the unpublished ticket to 030 to avoid colliding with active PR #3. diff --git a/project/ticket-030/intent.json b/project/ticket-030/intent.json new file mode 100644 index 0000000..499db70 --- /dev/null +++ b/project/ticket-030/intent.json @@ -0,0 +1,17 @@ +{ + "schema": "new-project.intent/v2", + "ticket": "ticket-030", + "summary": "Repair confidence hierarchy test after extractor refactor", + "workstream": "llm", + "allowedPaths": [ + "test/nl-llm.test.ts", + "project/ticket-030/**", + "TODO.md", + "project/TICKETS.md" + ], + "forbiddenPaths": ["project/ticket-*/user-*.md"], + "stacks": ["node"], + "dependsOn": [], + "conflictsWith": [], + "integrationTicket": null +} diff --git a/project/ticket-030/preprompt.md b/project/ticket-030/preprompt.md new file mode 100644 index 0000000..0086e90 --- /dev/null +++ b/project/ticket-030/preprompt.md @@ -0,0 +1,8 @@ +# Ticket preprompt + +- **Task ID**: ticket-030 +- **Task title**: Repair confidence hierarchy test after extractor refactor +- **Created**: 2026-08-04T09:30:20Z + +Keep executable implementation outside this governance/evidence directory. +Read a human-owned user-*.md file only when one exists. diff --git a/test/nl-llm.test.ts b/test/nl-llm.test.ts index b6eb2c1..7fa5cca 100644 --- a/test/nl-llm.test.ts +++ b/test/nl-llm.test.ts @@ -192,8 +192,8 @@ test('The documented confidence hierarchy holds across LLM extractors', async () // Read the TypeScript sources from the repository root: the compiled test // runs out of dist/, where the .ts files do not exist. const sources = await Promise.all([ - fs.readFile(path.resolve('src/extractors/markdown-llm.ts'), 'utf8'), - fs.readFile(path.resolve('src/extractors/nl-llm.ts'), 'utf8'), + fs.readFile(path.resolve('src/extractors/markdown-llm-helpers.ts'), 'utf8'), + fs.readFile(path.resolve('src/extractors/nl-llm-helpers.ts'), 'utf8'), fs.readFile(path.resolve('src/extractors/docs-record.ts'), 'utf8'), ]); const ceilings = sources.map((source) => {