Skip to content

fix(llm): retry rate-limited (429) batches instead of dropping them - #403

Draft
keshprad wants to merge 18 commits into
mainfrom
keshavp/codex/llm-429-retries
Draft

fix(llm): retry rate-limited (429) batches instead of dropping them#403
keshprad wants to merge 18 commits into
mainfrom
keshavp/codex/llm-429-retries

Conversation

@keshprad

Copy link
Copy Markdown
Member

Summary

  • retry provider rate-limit failures in both synchronous and asynchronous LLM batch loops with a bounded 5s/15s/30s/60s/60s schedule
  • honor numeric Retry-After values up to 120 seconds, including for models with native provider retries
  • record exhausted rate-limit retries as llm_rate_limit_retries_exhausted in the inspection ledger

Validation

  • make lint — passed
  • make format-check — passed
  • pytest -q tests/nodes/test_llm_analyzer_base.py tests/test_inspection_ledger.py — 171 passed
  • unit suite — 2213 passed, 13 skipped, 4 xfailed; seven SSRF hostname tests fail because GitHub/GitLab resolve to private/internal addresses in this environment. The exact same seven tests fail on the untouched 29b0dc8 base.

Attribution

Applied from the attached signed-off patch while preserving Devin AI as the commit author and the original Signed-off-by/Co-Authored-By trailers.

devin-ai-integration Bot and others added 18 commits August 20, 2026 00:40
Add a bounded rate-limit retry policy (5s, 15s, 30s, 60s, 60s, honoring a numeric Retry-After up to 120s) to both the sync and async LLM batch retry loops, applied even for models with native provider retries. Exhausted retries record LLM_RATE_LIMIT_RETRIES_EXHAUSTED in the inspection ledger.

Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Keshav Pradeep <keshavp@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant