refactor(transport)!: share fan-out execution across services

[thodson-usgs]

Rebased onto main now that #350 has merged. Single commit. Independent of #351 — the 9 shared files are NEWS.md, the ADR index, and test modules.

The distinction

Chunking is how you divide the data structurally; fan-out is how you distribute the work operationally.

The two are orthogonal, and only the first is protocol knowledge: dividing a query needs the byte budget, the CQL2 grammar, and which parameters are list-valued. Distributing the pieces needs none of it.

The package had not drawn that line. ChunkPlan (division) and ChunkedCall (distribution) sat side by side in dataretrieval/ogc/ as siblings — and ADR 0006 grouped them together deliberately, which was right while a byte plan was the only thing anyone fanned out over.

Measuring OGC domain vocabulary in each class's code, docstrings and comments stripped:

ChunkPlan     7 distinct -> CQL, axes, axis, axis_chunks, bytes, filter_lang, url_limit
ChunkedCall   0 distinct -> []

What that cost

Water Use fans out for an entirely different reason — the NWDC accepts one location= per request, and its URLs run ~63 bytes against an 8000-byte budget. Unable to reach an OGC-internal executor, wateruse._fan_out re-implemented the semaphore, the gather, and the cancellation-beats-HTTP-error precedence rule, with a comment at wateruse.py:398 naming ChunkedCall._run as the original. One subtle rule, two copies, synchronized by prose.

The duplicate also lost three things:

	OGC	Water Use (before)
resume after 429	.call.resume()	none — one exhausted location discarded all completed ones
progress line	yes	no
concurrency knob	API_USGS_CONCURRENT	module global MAX_CONCURRENT_REQUESTS = 4

The resume gap is the one that bites: a multi-state county pull is hundreds of requests against a 1000/hr quota.

What this does

Moves execution down, leaves planning up. transport/fanout.py::FanOut drives any FanOutPlan — a Protocol of exactly the three members the executor already used (total, canonical_url, iter_sub_args()).

Structural rather than nominal because the two implementations share an interface and no implementation at all: ChunkPlan derives sub-requests from a byte budget over multi-value axes; a Water Use plan lists locations the caller already named separately. Neither has anything the other could inherit, so an ABC would be ceremony. ChunkPlan needed no edits — it already satisfied the protocol.

Water Use sheds ~75 lines and gains resume, progress, and the shared setting.

Breaking changes

1. A Water Use fan-out interrupted by 5xx/429 now raises ServiceInterrupted/QuotaExhausted, not ServiceUnavailable/RateLimited. Both remain DataRetrievalError so broad handlers are unaffected, but a narrow except ServiceUnavailable around a Water Use call must widen. This is convergence with the OGC getters — and it is precisely what makes the failure resumable.
2. wateruse.MAX_CONCURRENT_REQUESTS is removed in favor of API_USGS_CONCURRENT and wateruse.DEFAULT_CONCURRENT_REQUESTS.

Not breaking: ChunkInterrupted is a permanent alias of the renamed FanOutInterrupted — the same class object, not a deprecation — so except ChunkInterrupted keeps working. The rename is one name, because QuotaExhausted and ServiceInterrupted were already fan-out-neutral.

Concurrency: one setting, per-service defaults

API_USGS_CONCURRENT is general; a service declares a default for when it is unset (Water Use 4, package-wide 32). An explicitly set env var outranks a service default, never the reverse — a service able to override it would make API_USGS_CONCURRENT=1 a lie, which is the original defect.

Layout

From	To
ogc/chunking.py::ChunkedCall	transport/fanout.py::FanOut
ogc/retry.py::_classify_chunk_error	interruptions.py (beside the classes it produces)
transport/retry.py::_deterministic_failure	interruptions.py; transport imports it back
ogc/interruptions.py	interruptions.py (top-level leaf)

Stays in ogc: ChunkPlan, multi_value_chunked, parallel_chunks, _OGC_URL_BYTE_LIMIT. Compatibility aliases (ChunkedCall, get_active_client, _chunked_client) remain importable from ogc.chunking; _chunked_client is the same ambient object transport publishes, not a copy.

interruptions.py is a top-level leaf for the reason ADR 0006 gives for combining/progress/credentials: adapters need it whether or not they went through transport, and an exception taxonomy is not HTTP execution policy.

_deterministic_failure follows it down. Whether a failure is worth retrying and whether it can be resumed are one judgement about what an exception means, not two — and the leaf is where meaning lives. Leaving it in transport would have forced the leaf to import transport just to ask. The fix that made those two answers agree (31b01420) is preserved: tests/transport_test.py still asserts both on the same failures, so they cannot drift apart.

Verification

• 685 tests pass (5 new), ruff clean, mypy --strict clean, pre-commit clean.
• New fitness functions: wateruse may contain no asyncio.gather/Semaphore/TaskGroup; both plans satisfy FanOutPlan (including that iter_sub_args() is stable across passes and agrees with total, since resume keys by position); the Water Use plan does not inherit ChunkPlan; no interruption taxonomy inside transport.
• New adapter tests: Water Use resume re-issues only unfinished locations, progress ticks, concurrency precedence.

One deviation from plan: I predicted the moves would change zero OGC tests. They changed three call sites — 9 patches of _chunking.asyncio.sleep (the backoff sleep is issued by transport.retry; patching it through the chunker only ever worked because asyncio is a shared module object) and one client-factory patch. No assertion changed; each now names the module that actually owns the behavior.

Known costs

• Resume re-issues a failed location's entire page walk, so pages fetched before the failure are fetched again. Already true for OGC — a partial walk never enters the completion map — so a cost, not a correctness problem.
• Water Use frames carry huc12_id, not id, so _combine_chunk_frames concatenates without deduplicating. Correct (locations partition by construction), but the dedup safety net does not apply there.

Open question

.completed_chunks / .total_chunks are left as-is. They are read rather than caught, and the message text already says "sub-requests", so renaming them would churn ~30 assertions for cosmetics. Happy to do it if you want the vocabulary uniform.

Supersedes one clause of ADR 0006; see the new ADR 0008 (numbered around #351's 0007).

🤖 Generated with Claude Code