Skip to content

fix: preserve Strix PR scope on provider exhaustion - #1213

Open
seonghobae wants to merge 4 commits into
mainfrom
fix/strix-baseline-provider-exhaustion
Open

fix: preserve Strix PR scope on provider exhaustion#1213
seonghobae wants to merge 4 commits into
mainfrom
fix/strix-baseline-provider-exhaustion

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Root cause

The trusted Strix PR mapper correctly excluded a hallucinated finding in an unchanged, nonexistent file, but later provider retirement failures reused the same exit status as changed-file findings. The outer workflow then saw the retained severity text and failed the required check even though the exact PR diff had already been cleared.

Repair

  • return a dedicated trusted-gate status only after every finding is mapped outside the PR diff and all providers are unavailable
  • neutralize only that status in the reusable workflow
  • keep changed, unmapped, manifest, configuration, and unexpected failures blocking
  • record the reproduced incident and regression contract

Exact-head verification

  • 1333 passed, 1 skipped, 16 subtests
  • complete Strix shell regression suite passed
  • focused provider-exhaustion and neighboring workflow tests passed
  • actionlint, Bash syntax, CodeGraph sync, and diff hygiene passed

No secret, approval, protection, or reviewer-key boundary changes.


Open in Devin Review

@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 33 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8059ed57-5697-46d3-8690-bc2cd81220d1

📥 Commits

Reviewing files that changed from the base of the PR and between 634a41a and 6b03822.

📒 Files selected for processing (6)
  • .github/workflows/strix.yml
  • CHANGELOG.md
  • docs/doctoring/strix-pr-baseline-provider-exhaustion.md
  • scripts/ci/strix_quick_gate.sh
  • scripts/ci/test_strix_quick_gate.sh
  • tests/test_strix_nvidia_nim_not_found_fallback.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent Review only exact current head 5dbf903. Verify the trusted status-3 boundary, unchanged-finding attribution, provider-exhaustion neutralization, and fail-closed changed/unmapped/configuration paths. Submit a formal Reviews API verdict without changing the branch, self-approving, or bypassing protection.

@seonghobae
seonghobae enabled auto-merge (squash) August 22, 2026 00:05
devin-ai-integration[bot]

This comment was marked as resolved.

@opencode-agent
opencode-agent Bot disabled auto-merge August 22, 2026 00:08
@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent Re-review exact current head 5dbf903 now that all hosted checks are terminal-success and the informational Devin thread is resolved. Submit the formal Reviews API verdict; do not mutate the branch or bypass protection.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent Please review exact head d065e69fa48fbf96251433f8be77934ccf03b159. The follow-up narrowly recognizes the observed scanner-local RuntimeError: loginAsGuest ... curl exit 7 ... 127.0.0.1 port 48080 Caido bootstrap outage only when no positive vulnerability/severity signal exists; all other runtime failures and any finding remain fail-closed. Local evidence: 1,333 Python tests passed, 1 skipped, 16 subtests passed; bash scripts/ci/test_strix_quick_gate.sh passed; targeted workflow classifier tests (13) and actionlint passed. Please provide an independent exact-head review.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revalidated exact head 6b03822 after merging protected main 634a41a: 1333 passed, 1 skipped, 16 subtests; 100% statement and branch coverage; 100% production docstrings; focused Strix shell, Bash syntax, and workflow validation passed. Existing review threads remain resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment on lines +4043 to +4053
# The trusted PR-scope mapper has already proved these reports belong only
# to unchanged files. Keep that decision distinct from a real changed-file
# finding when every remaining provider is unavailable.
if [ "$PR_FINDINGS_DECISION" = "allow_baseline" ] && [ "$INFRA_ERROR_DETECTED" -eq 1 ]; then
if is_vertex_model "$PRIMARY_MODEL"; then
echo "Configured Vertex model and fallback models were unavailable after unchanged-file findings were excluded." >&2
else
echo "Configured model and fallback models were unavailable after unchanged-file findings were excluded." >&2
fi
return 3
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Exit-3 neutral path skipped when no fallbacks are configured

The new return 3 (strix_quick_gate.sh) sits after the guard that returns 1 when no fallback ran (strix_quick_gate.sh). With zero configured fallbacks, a primary infra failure on baseline-only findings still returns 1 and the workflow hard-fails on the severity text. The workflow always sets STRIX_FALLBACK_MODELS, so the reproduced incident is covered.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant