Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

## Active tickets

- [ ] [`ticket-030`](project/ticket-030/README.md) — repair the stale
confidence-hierarchy source lookup after the NL and Markdown clamp logic
moved into focused helper modules. Current state: `IN_PROGRESS / VALIDATION`;
the production ceilings remain unchanged and the implementation is test-only.

- [ ] [`ticket-019`](project/ticket-019/README.md) — publish the dependency-free
Python SDK as the root PyPI distribution `todo2code` through `goal -a`, with
one root `pyproject.toml` and SDK-only artifacts. Current state:
Expand Down
1 change: 1 addition & 0 deletions project/TICKETS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,5 @@ of `project/README.md`, which remains a generated technical-analysis artifact.
| **ticket-019** | [`README.md`](./ticket-019/README.md) | [`preprompt.md`](./ticket-019/preprompt.md) | - | [`ai-codex.md`](./ticket-019/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-019/ai-codex-logs.txt) | [`changelog.md`](./ticket-019/changelog.md) |
| **ticket-020** | [`README.md`](./ticket-020/README.md) | [`preprompt.md`](./ticket-020/preprompt.md) | - | [`ai-codex.md`](./ticket-020/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-020/ai-codex-logs.txt) | [`changelog.md`](./ticket-020/changelog.md) |
| **ticket-022** | [`README.md`](./ticket-022/README.md) | [`preprompt.md`](./ticket-022/preprompt.md) | - | [`ai-codex.md`](./ticket-022/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-022/ai-codex-logs.txt) | [`changelog.md`](./ticket-022/changelog.md) |
| **ticket-030** | [`README.md`](./ticket-030/README.md) | [`preprompt.md`](./ticket-030/preprompt.md) | - | [`ai-codex.md`](./ticket-030/ai-codex.md) | [`ai-codex-logs.txt`](./ticket-030/ai-codex-logs.txt) | [`changelog.md`](./ticket-030/changelog.md) |
<!-- AUTO:TICKET_INDEX:END -->
71 changes: 71 additions & 0 deletions project/ticket-030/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Ticket 030: Repair confidence hierarchy test after extractor refactor

- **ID**: ticket-030
- **Owner**: unresolved:human
- **Status**: IN_PROGRESS
- **Workflow state**: VALIDATION
- **Created**: 2026-08-04

## Goal and scope

Repair the confidence-hierarchy regression test that failed after the Markdown
and NL confidence clamping logic moved from the orchestration modules into
focused helper modules. The production ceilings remain correct (`0.94`,
`0.90`, `0.85`); the failure is a test-discovery defect, not evidence that an
Intent record contains `confidence: null`.

The implementation is deliberately test-only. It will update the hierarchy
test so it verifies the canonical current locations without changing any
extractor behavior, confidence value, documentation contract or LLM boundary.
No unrelated refactor or generated artifact belongs to this ticket.

## Planned changed paths

- `test/nl-llm.test.ts`: replace the stale source-file assumptions with a
deterministic check of the current confidence-clamping implementations.
- `project/ticket-030/**`, `TODO.md`, `project/TICKETS.md`: ticket intent and
verification evidence only.

## Acceptance criteria

- [x] AC-01: Scope is approved by a human owner.
- [x] AC-02: The test distinguishes a missing source-code match from a runtime
confidence value and no longer reports the former as extractor failure.
- [x] AC-03: The hierarchy remains Markdown `0.94` > NL `0.90` > documentation
`0.85`, with all three ceilings below deterministic observation levels.
- [x] AC-04: Focused NL/Markdown/document tests pass without a live provider,
changing production code or accepting a looser confidence ceiling.
- [x] AC-05: The ticket diff contains only its allowed test and governance
paths; unrelated dirty/generated files remain untouched.

## Participants

- Human participant: unresolved; no user-* file was created by this script.
- Agent participant: [ai-codex.md](ai-codex.md)

## Approval boundary

- Current state: `IN_PROGRESS / VALIDATION`.
- Required response from: `unresolved:human`.
- Chat approval authorizes implementation in this interactive session but is
not trusted merge evidence.
- The ticket was renumbered from the locally drafted `ticket-023` to
`ticket-030` before publication because remote PR #3 already owns 023. The
approved implementation scope and code delta did not change.

## Validation evidence

- The freshly emitted `dist/test/nl-llm.test.js` passes `11/11`; the repaired
hierarchy assertion passes and no live provider is called.
- The reported 338-test run used an older `dist` tree: it correctly exposed the
stale hierarchy lookup, but by itself was not proof that current TypeScript
sources build cleanly. After rebasing onto base `caf6551`, a clean
`npm run build` passes.
- The full freshly built suite reports 338 tests: 327 passed, 10 failed and one
skipped. The repaired confidence-hierarchy test passes. The ten inherited
failures concern AST/linker relations, communication identity/enrichment and
gold-dataset relations; none is in ticket-030 scope or caused by its two-line
source-location update.
- Scoped `git diff --check` passes. Whole-repository governance continues to
report only the pre-existing ticket-018/ticket-019 conflict, dependency,
workstream ownership and shared `Makefile` overlap.
8 changes: 8 additions & 0 deletions project/ticket-030/ai-codex-logs.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
2026-08-04 ticket-030 validation
- `node --test dist/test/nl-llm.test.js`: PASS (11/11).
- `npm run build`: FAIL (17 inherited TypeScript diagnostics outside allowed implementation path).
- `npm test` after the failed fresh emit: FAIL (292 total, 253 pass, 38 fail, 1 skip); base import/runtime regressions exposed.
- The user-reported 338-test run was based on stale `dist`; useful for reproducing the hierarchy assertion only, not for clean-source verification.
- `make governance`: FAIL with the four inherited ticket-018/ticket-019 diagnostics (`GOV-CONFLICT-001`, `GOV-DEPENDENCY-002`, `GOV-WORKSTREAM-003`, `GOV-WORKSTREAM-004`); no ticket-030 diagnostic was reported.
- Rebased onto updated PR-5 base `caf6551`: `npm run build` PASS.
- Fresh `npm test`: FAIL (338 total, 327 pass, 10 fail, 1 skip). Failures are AST/linker relation (2 direct plus gold cohorts), communication identity/enrichment (3), and configuration/document relation coverage; the ticket-030 test passes.
56 changes: 56 additions & 0 deletions project/ticket-030/ai-codex.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
participant-id: agent:codex
participant: codex
role: agent
ticket: ticket-030
---
# Participant: codex (AI agent)

## Understanding

The failed assertion's `actual: null` is the result of
`source.match(...)`, not the confidence stored in an Intent record. The test
still reads `markdown-llm.ts` and `nl-llm.ts`, while their clamping expressions
now live in `markdown-llm-helpers.ts` and `nl-llm-helpers.ts`. The documented
and runtime ceilings remain `0.94`, `0.90` and `0.85`.

The narrow repair belongs to the `llm` workstream and does not overlap the
active governance or SDK implementation paths. It must not absorb the user's
concurrent synthesis refactor or generated `project/` artifacts.

## Execution plan

1. Stop at `WAIT_FOR_APPROVAL` before editing the test.
2. Update only the confidence-hierarchy test to inspect the canonical current
implementations, retaining strict numeric assertions and ordering.
3. Run the focused compiled/TypeScript tests without a live OpenRouter call.
4. Review the exact diff and run scoped governance; record any inherited
repository failures separately from this test-only change.

## Actual changes

- The user explicitly approved the bounded ticket-030 plan in chat. The
implementation session is authorized; merge approval remains external.
- Updated only `test/nl-llm.test.ts`: the hierarchy test now reads the Markdown
and NL helper modules that own their confidence clamps after refactoring.
- Ran the freshly emitted compiled test file: 11 passed, 0 failed, with the
hierarchy assertion passing. No provider request was made.
- Rechecked the updated base from a clean dependency installation. Base
`caf6551` builds successfully, and the freshly built complete suite reports
327 passed, 10 failed and one skipped out of 338. The confidence-hierarchy
test passes; the remaining failures are inherited relation and communication
regressions outside this ticket.
- Renumbered the unpublished local ticket from 023 to 030 after discovering
that GitHub PR #3 already owns ticket-023; no implementation path or
acceptance criterion changed.

## Blockers

- Whole-repository governance already fails because ticket-018 and ticket-019
declare a conflict and unfinished dependency. This ticket does not rewrite
either intent or claim that unrelated blocker.
- The main worktree contains user-owned generated and synthesis changes. Work
remains isolated in a separate branch/worktree.
- Full verification is blocked by ten source-behaviour test failures outside
the approved ticket paths. The base branch needs separately scoped repairs
before it can be considered merge-ready.
16 changes: 16 additions & 0 deletions project/ticket-030/changelog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Ticket Changelog (ticket-030)

## [0.1.0] - 2026-08-04

- Initial governance scaffold created.
- No human participant identity or content was generated.

## [0.1.1] - 2026-08-04

- Repaired the stale Markdown/NL confidence-clamp source locations in the
hierarchy test without changing production confidence behavior.
- Focused compiled NL LLM test passes 11/11. After rebasing onto `caf6551`, a
clean build passes; the complete fresh suite exposes ten inherited relation
and communication failures while the repaired hierarchy assertion stays
green.
- Renumbered the unpublished ticket to 030 to avoid colliding with active PR #3.
17 changes: 17 additions & 0 deletions project/ticket-030/intent.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"schema": "new-project.intent/v2",
"ticket": "ticket-030",
"summary": "Repair confidence hierarchy test after extractor refactor",
"workstream": "llm",
"allowedPaths": [
"test/nl-llm.test.ts",
"project/ticket-030/**",
"TODO.md",
"project/TICKETS.md"
],
"forbiddenPaths": ["project/ticket-*/user-*.md"],
"stacks": ["node"],
"dependsOn": [],
"conflictsWith": [],
"integrationTicket": null
}
8 changes: 8 additions & 0 deletions project/ticket-030/preprompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Ticket preprompt

- **Task ID**: ticket-030
- **Task title**: Repair confidence hierarchy test after extractor refactor
- **Created**: 2026-08-04T09:30:20Z

Keep executable implementation outside this governance/evidence directory.
Read a human-owned user-*.md file only when one exists.
4 changes: 2 additions & 2 deletions test/nl-llm.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,8 @@ test('The documented confidence hierarchy holds across LLM extractors', async ()
// Read the TypeScript sources from the repository root: the compiled test
// runs out of dist/, where the .ts files do not exist.
const sources = await Promise.all([
fs.readFile(path.resolve('src/extractors/markdown-llm.ts'), 'utf8'),
fs.readFile(path.resolve('src/extractors/nl-llm.ts'), 'utf8'),
fs.readFile(path.resolve('src/extractors/markdown-llm-helpers.ts'), 'utf8'),
fs.readFile(path.resolve('src/extractors/nl-llm-helpers.ts'), 'utf8'),
fs.readFile(path.resolve('src/extractors/docs-record.ts'), 'utf8'),
]);
const ceilings = sources.map((source) => {
Expand Down
Loading