Skip to content

red-team area 15: two paired passes over the legacy pure-R search API, plus two inline fixes - #150

Merged
ms609 merged 9 commits into
cpp-searchfrom
feature/red-team-area15
Aug 22, 2026
Merged

red-team area 15: two paired passes over the legacy pure-R search API, plus two inline fixes#150
ms609 merged 9 commits into
cpp-searchfrom
feature/red-team-area15

Conversation

@ms609-agent

Copy link
Copy Markdown
Collaborator

Two red-team passes over area 15 (legacy pure-R search API), its first review since the row was created. This PR carries two small inline fixes and the round records; the 31 findings themselves are filed as issues and are not fixed here.

No Fixes #N lines: nothing filed is fixed by this branch. The two defects that were fixed were fixed inline and deliberately not filed, per the skill's rule against filing an issue and closing it the same round.

Code changes (2 files, 8 lines)

R/CustomSearch.REdgeListSearch() looped for (iter in 1:maxIter). 1:0 is c(1, 0), so maxIter = 0 silently performed two rearrangement iterations instead of none; because RearrangeEdges() accepts any candidate scoring <= scoreToBeat, a caller asking for zero rearrangements could get a different tree back. maxIter is user-facing (default 100) and reaches this loop from Bootstrap(), Jackknife() and Ratchet(), so maxIter = 0 is a reachable "score without searching" idiom.

Now seq_len(maxIter), with iter <- 0L pre-initialised — that part is load-bearing, since iter is read after the loop at :111.

R/SuccessiveApproximations.R — the "Stability not reached" branch emitted its message regardless of verbosity while the "converged" branch was gated, so a default verbosity = 0 call was noisy. Both branches now sit inside one gate. Nothing in tests/, vignettes/, man/ or R/ referenced the string.

Test

tests/testthat/test-CustomSearch.R gains a regression test pinning the maxIter = 0 contract. EdgeListSearch() is exported, so it uses mocks that error if called — no dataset, no C++, instant. Verified to discriminate: post-fix the loop body is never entered; with the pre-fix loop bound restored in an isolated copy of the function, it is.

Findings filed

Issue Severity Summary
#136 high Ratchet(stopAtScore=) returns the input tree carrying the improved score, plus two more early-exit bookkeeping failures
#137 high Ratchet() never forwards TreeScorer to its Bootstrapper, so the two search phases optimise different objectives
#138 high TreeSearch()'s default swapper holds the root fixed, confining a mid-rooted start to ~5% of tree space — needs a design decision
#139 high Fractional character weights truncate to zero before bootstrap resampling, silently degrading BootstrapTree()/JackknifeTree() to a random walk
#143 med The pure-R samplers waste up to 40% of draws on identity moves and cannot reach part of the neighbourhood they document
#144 med Exported swappers' argument validation is unreachable or absent — TBRSwap() hangs on a trifurcating root
#125 med The documented edgeToBreak = -1 contract is broken at 4 of 6 exported entry points, one silently
#126 low SuccessiveApproximations() runs without the sectorial/fuse/pool machinery and never says so
#131 low Eight contract/documentation drifts, grouped for a single sweep

31 candidates were grouped into 9 issues by shared root cause rather than filed one-per-finding — the tracker is at 72 open with 3 claimed, and 31 tickets would have made triage worse, not better.

Why two passes

These were the two arms of a deliberate tier-economics experiment: is it cheaper to let sonnet find what it can and have opus find only the remainder, than to send opus first? Area 15 was the only never-visited area, so a first pass measured the cheap tier honestly.

sonnet opus (given sonnet's entire yield as off-limits)
Candidates 5 26
sev:high 0 4
Confirmed / refuted 5 / 0 26 / 0
Finder tokens 137k 180k

The cheap pass removed no work from the expensive one — opus still read all 2,185 loc and found five times as much — so sonnet-first was an added pass, not a substituted one. The severity split is the substantive result: sonnet found broken documented contracts, opus found silent wrong answers. Both log.md entries record this, and focus-areas.md now says not to re-run it.

focus-areas.md row 15 keeps start_tier: sonnet — the maintainer's recorded choice on #42 — but the rationale beside it now notes that the reasoning behind it is falsified, and that the field is inert for a visited area anyway. One word to change if you want it.

Verification notes worth reading

Verification corrected the finder three times, so the issues do not simply restate what the finders reported:

  • One high-severity candidate was downgraded to med (a loud error, not a silent one).
  • A15-06: TreeSearch()'s default swapper holds the root fixed, confining a mid-rooted start to ~5% of tree space (needs a design decision) #138's mechanism proved worse than claimed (the root split is a hard invariant, not a per-step bias — a 150k-step walk reached 45 of 945 topologies) while its motivating example was wrong: NJTree() is tip-rooted, so this package's own ?TreeSearch example is in the benign regime.
  • Three sampler findings came back worse: the NNISwap self-hit rate is 2/(nTips-2), not 1/(nTips-2) — which changes the fix, since both root-adjacent edges must be excluded; NNI(-1) is missing an entire pair of true neighbours rather than merely padded with duplicates; and TBRSwap() on a trifurcating root hangs indefinitely rather than misbehaving.

Both peer verifiers built independent unrooted-neighbourhood enumerators and validated them against the closed forms 2(n-3) and 2(n-3)(2n-7) before using them as ground truth, which is what let them contradict the finders' numbers rather than defer to them. One recorded a trap for whoever rebuilds it: tied n/2|n/2 splits need lexicographic, not size-based, tie-breaking.

Checks

agent-check.yml green on ubuntu-arm64 and windows (runs 31014552218, 31037706585). spelling::spell_check_package(vignettes = TRUE) clean. No roxygen or C++ signature changed, so check_man() and compile-attrs.R were not triggered; vignettes/custom.Rmd documents this API but uses maxIter = 50L throughout, so the degenerate-input fix changes nothing it describes.

🤖 Generated with Claude Code

ms609 and others added 2 commits August 5, 2026 15:18
Fixes the one defect small enough to fix inline, and records the round.

`EdgeListSearch()` looped `for (iter in 1:maxIter)`. `1:0` is `c(1, 0)`, so
`maxIter = 0` silently performed two rearrangement iterations instead of none
-- and since `RearrangeEdges()` accepts any candidate scoring `<= scoreToBeat`,
a caller asking for zero rearrangements could get a different tree back.
`maxIter` is user-facing and reaches this loop from `Bootstrap()`,
`Jackknife()` and `Ratchet()`, so `maxIter = 0` is a reachable "score without
searching" idiom.

Switched to `seq_len(maxIter)`, pre-initialising `iter <- 0L` because `iter` is
read after the loop. Pinned by a regression test using mocks that error if
called; `EdgeListSearch()` is exported, so it needs no dataset and no C++.

The round's other findings are filed as #125 (the
documented `edgeToBreak = -1` contract is broken at 4 of 6 exported entry
points, one of them silently) and #126 (`SuccessiveApproximations()` runs
without the sectorial/fuse/pool machinery, undocumented).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second pass over the same scope with the sonnet arm's entire yield in the
do-not-re-investigate block. 26 candidates, 26 confirmed, 0 refuted -- four of
them sev:high, against the sonnet arm's zero. Filed as seven grouped issues:
#136, #137, #138, #139 (high), #143, #144 (med), #131 (low).

The paired result answers the question the two passes were run to settle: a
cheap first pass did not reduce the expensive pass's work, so sonnet-first is
an added pass rather than a substituted one. Recorded in log.md; the rationale
in focus-areas.md now says not to re-run it.

Fixes one defect inline: the "Stability not reached" branch of
`SuccessiveApproximations()` emitted its message regardless of `verbosity`
while the "converged" branch was gated, so a default `verbosity = 0` call was
noisy. Both branches now sit inside one gate. Nothing in tests/, vignettes/,
man/ or R/ referenced the string.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ms609-agent
ms609-agent requested a review from ms609 August 6, 2026 07:50
ms609-agent and others added 7 commits August 6, 2026 09:13
The round record has no business waiting on a code review. This file sits on
a protected branch, and four completed rounds with 56 filed findings once sat
stranded on an unmerged PR while `last_focus:` here still named a stale area
-- so the next dispatch would have re-swept an area already reviewed twice
that day.

Area 15's three rounds now live as one Discussion post each, under that
area's category. Removes the two entries added earlier on this branch and
replaces them with a pointer.

`last_focus:` stays live and stays current: the new scheme picks the stalest
category rather than following a pointer, but that ordering cannot be
computed until every area has a discussion, and the backfill has to post
oldest-first so createdAt reproduces true staleness. Only area 15 is
migrated so far.

The historical entries below the pointer stay put -- eleven in-repo files and
the /red-team skill cite this path, and the T-nnn ids are frozen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Area 15's scope row grew on both sides. Kept as a union, except that
cpp-search's "#16 gives this row higher urgency, take it first" is dropped:
this branch's round discharged that question (#16 closed, guard landed in
PR #50, the pure-R layer measured doubly guarded), so the two cannot both
stand and the later assessment wins. cpp-search's src/rearrange.cpp note
is kept verbatim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
All 15 areas have a Discussion record, so rotation reads staleness from
createdAt and the pointer is dead. Records the invariant createdAt relies on
— creation order equals review-recency order — which the backfill broke and
discussion #184 restored.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The header claimed /red-team appends an entry and updates last_focus, which
the same file now says is retired. Replaced with what the file actually is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The `iter <- 0L` added alongside the seq_len() fix was dead: `for` binds its
variable unconditionally, so a zero-length sequence leaves `iter` NULL rather
than falling through to the earlier assignment. The verbosity report then
printed "after  rearrangements." with a blank. Handle it where it is read,
and cover it — the existing maxIter = 0 test runs at verbosity 0, so it could
not see this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
These rows argued with superseded versions of themselves — why a start_tier
was chosen and why it no longer binds, which #42 urgency justified the row and
why it is discharged. A scope row briefs the next round; it is not a record of
how it came to say what it says. Keeps the operative content: scope, seam
verdict, the standing prohibition on re-running the tier experiment, the
transferable lesson, and the next-visit leads.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Performing tree search" was emitted before maxIter was consulted, so
maxIter = 0 claimed a search had started and only the closing summary
contradicted it. Gate the claim instead, and ask maxIter directly rather
than inferring it from the loop variable being left NULL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ms609
ms609 enabled auto-merge August 22, 2026 07:04
@ms609
ms609 merged commit a3743dc into cpp-search Aug 22, 2026
11 of 12 checks passed
@ms609
ms609 deleted the feature/red-team-area15 branch August 22, 2026 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants