Skip to content

Wait out the rate limit instead of failing the file - #11

Merged
pavel-te merged 1 commit into
mainfrom
fix-rate-limit-backoff
Aug 5, 2026
Merged

Wait out the rate limit instead of failing the file#11
pavel-te merged 1 commit into
mainfrom
fix-rate-limit-backoff

Conversation

@pavel-te

@pavel-te pavel-te commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

The API allows 10 requests per minute on a bucket shared by create, process and bulk, and a run spends two of them per file. Any project past five files meets a 429 partway through.

Before this, the CLI reported that 429 as a failed upload and moved to the next file. The run ended with some files registered and some not, and HTTP 429 as the only clue in the log — no reason text, because the rate-limit body uses a response shape the CLI did not read.

What changes

A 429 now asks for a retry instead of reporting failure. The wrapper waits and calls again, up to 5 times:

  • honours Retry-After when the server sends one (capped at 300s so a bad header cannot hang a job);
  • otherwise backs off 15s → 30s → 45s → 60s, walking towards the one-minute window the limit is measured over.

PTC does not send Retry-After today, so the backoff is what actually runs. The header path exists because pacing is the server's call the moment it wants it back — and it is exercised by the end-to-end check below.

Response headers reach the retry through one optional -D inside ptc_curl, so the five branching curl invocations are untouched.

A non-429 failure is passed straight through and still fails fast — only the rate limit is retried.

Failure text

describe_api_failure now also reads the {"error": "..."} shape. source_files#create and #process answer that way, so reading only message turned a real rejection into a bare HTTP 422 in the CI log. When both keys are present, both are kept — they say different things (TRIAL_EXPIRED vs Your trial has ended).

Verification

35 new tests in tests/test-rate-limit.sh (215 across the suite, 0 failures): the backoff ladder, Retry-After including CRLF, HTTP-date and absurd values, retry exhaustion, non-429 pass-through, and that the internal retry signal never escapes to the shell as an exit code.

End to end against a mock that 429s the first two attempts of both calls:

[WARNING] PTC rate limit reached (HTTP 429). Waiting 2s, then retry 1 of 5.
[WARNING] PTC rate limit reached (HTTP 429). Waiting 2s, then retry 2 of 5.
[SUCCESS] File uploaded successfully: locales/en.json
...
[SUCCESS] Translations unpacked successfully

It waited 2s rather than the 15s default, which is the Retry-After header being read. The run finished and wrote de.json; before this change that file was lost.

Not included

No VERSION bump — this repository does that in its own release commit.

create, process and bulk share one bucket of 10 requests per minute, and a
run spends two of them per file. Every project past five files therefore
meets a 429 partway through, and the CLI treated it as a failed upload and
carried on: the run ended with some files registered and some not, and
"HTTP 429" as the only clue in the log.

A 429 now asks for a retry rather than reporting failure. The wrapper waits
and calls again, up to 5 times, honouring Retry-After when the server sends
one and otherwise backing off 15s, 30s, 45s, 60s towards the one-minute
window the limit is measured over. PTC does not send Retry-After today, so
the backoff is what runs; the header path is there because the answer to
"who paces this" belongs to the server the moment it wants it back.

Response headers reach the retry through a single optional -D in ptc_curl,
so the five branching curl invocations stay untouched.

Failure descriptions also gained the {"error": "..."} shape. source_files
create and process answer that way, and reading only "message" turned them
into a bare "HTTP 422" in the CI log — the one place the reason was needed.

35 new tests. Verified end to end against a mock that 429s the first two
attempts of each call: the run waits, retries, and downloads the
translations, where before it lost the file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant