feat(transport)!: page large results in parallel by offset; remove parallel_chunks

NEWS.md

@@ -1,3 +1,5 @@
+**08/06/2026:** Large Water Data pulls are now paged in parallel automatically. A multi-page result previously cost one round trip per page, because a cursor page's URL only exists once the previous page has been parsed. Where the API honors `offset`, every page's URL is computable up front, so the pages are fetched concurrently — the request count is unchanged, only their timing, and since the USGS quota is volume-based the speedup costs no extra quota. Measured 2.1× on 8 sites × 2 years (16,000 rows) and 3.1–4.2× on a single site's full daily history (~19,000 rows). **Breaking change:** `parallel_chunks(n)` is removed (along with `ChunkPlan.max_chunks`), because it bought parallelism the other way — splitting a request that already fit into more sub-requests, which spent extra quota and did nothing at all for a single-site query, the case with no multi-value axis to split. Delete the `with parallel_chunks(...):` wrapper; the pages inside it are now overlapped without it. Byte-driven chunking is unchanged and still required for correctness (the ~8 KB URL limit is real). `API_USGS_CONCURRENT` now also caps the page-fetch wave width; set it to `1` to page strictly sequentially. Two fallbacks keep results correct rather than merely fast: a server that ignores `offset` (a non-standard extension, so ignoring it is conventional) is detected before any rows are returned and the query re-runs via standard `next`-link paging, and the API's hard `offset` ceiling of 40,000 hands the remainder off to the sequential walk, so an arbitrarily deep pull is still returned in full.
+
 **08/02/2026:** Added an internal API-neutral transport layer for guarded HTTP clients, host-scoped authentication, cursor pagination, bounded retry, response aggregation, progress, and sync-over-async dispatch. Water Use and the non-OGC Statistics API now consume transport directly instead of private OGC execution helpers; WQP, NLDI, and StreamStats opt into bounded transient retry while deprecated NWIS behavior remains unchanged. OGC retains CQL2, request construction, feature shaping, chunk planning, resumable calls, and interruption types, with compatibility imports at previous private paths. Failed pagination and fan-out still raise rather than returning partial data, and no public signatures or return shapes changed.
 
 **08/02/2026:** Phase 1 OGC boundary stabilization: the `dataretrieval.ogc` package now exposes a deliberate, small facade (`OgcDialect`, `prepare_request_args`, `get_ogc_data`, `fetch_ogc_request`) that service adapters (NGWMN, Water Data's generic wrapper) import. Internal request-construction helpers moved to a new `ogc.requests` module; the dialect type and endpoint constants live in the leaf `ogc.policy` module. `ogc.shaping` no longer depends on `ogc.engine` at all, and the complete runtime OGC import graph is now acyclic. `_default_headers` now accepts a target URL and adds `X-Api-Key` only for `api.waterdata.usgs.gov`; shared sync and async clients also strip the key before following any cross-host redirect, including external rating-asset downloads. `waterdata.utils` no longer bulk-re-exports private OGC symbols, and consumers import implementation helpers from their canonical modules. No public API, return-value, or deprecation changes.

README.md

@@ -106,57 +106,50 @@ df, metadata = waterdata.get_continuous(
 print(f"Retrieved {len(df)} continuous gage height measurements")
 ```
 
-#### Speeding up large downloads with `parallel_chunks`
-
-By default the getters split a multi-value request only as far as the server's
-~8 KB URL limit forces — the fewest sub-requests. For a **large, paginated**
-pull that is needlessly conservative: every sub-request pages through its own
-results, so dividing the query into more, smaller sub-requests lets those pages
-be fetched **in parallel**. `parallel_chunks(n)` opts a single call into that
-finer split, fanning it out into `n` sub-requests. It pays off only when the
-result is large enough to span many pages *and* the query has a multi-value
-argument to divide (such as a list of monitoring locations); on a small query —
-or one with nothing to split — it just adds requests, so it is a deliberate,
-scoped `with` block, never the default.
+#### Large downloads are paged in parallel automatically
+
+A large result arrives one page at a time, and cursor pagination is inherently
+sequential: page *N+1*'s URL is only revealed by page *N*, so a 10-page result
+costs 10 round trips end to end. The Water Data API also accepts an `offset`
+parameter, which means every page's URL is computable up front
+(`offset = i * limit`) — so `dataretrieval` fetches them **concurrently**
+instead. Nothing to opt into; it is the default path for every getter.
 
 ```python
 from dataretrieval import waterdata
 
-# All stream gages in Ohio, then 20 years of their daily discharge — large
-# enough to span many pages, so it profits from a finer split.
-sites, _ = waterdata.get_monitoring_locations(state="Ohio", site_type_code="ST")
-
-with waterdata.parallel_chunks(32):  # fan out into 32 sub-requests
-    df, md = waterdata.get_daily(
-        monitoring_location_id=sites["monitoring_location_id"],
-        parameter_code="00060",  # discharge
-        time="2004-01-01/2023-12-31",
-    )
+# 20 years of daily discharge for one gage: ~7,300 rows over several pages,
+# fetched concurrently rather than one after another.
+df, md = waterdata.get_daily(
+    monitoring_location_id="USGS-01646500",
+    parameter_code="00060",  # discharge
+    time="2004-01-01/2023-12-31",
+)
 ```
 
-`n` is the number of sub-requests to fan the call out into. It is capped by how
-many values there are to split, and each sub-request costs a request against
-your hourly [rate limit](https://api.waterdata.usgs.gov/signup/); since how many
-run *at once* is capped separately by `API_USGS_CONCURRENT` (default 32), the
-useful range is roughly `2` up to that value.
-
-Benchmark — a fixed 271-site subset of Ohio stream gages
-(`get_daily`, `parameter_code="00060"`), with a small fixed page size
-(`limit=250`) so every run fetches roughly the same number of pages (isolating
-the effect of parallelism). Each `n` was run against its own cold 1-year time
-window so no run is served from the server's data-window cache:
-
-| `n`  | parallelism | pages | wall-clock              | speedup |
-| ---- | ----------- | ----- | ----------------------- | ------- |
-| off  | 1           | ~30   | 9.5 s / 9.1 s (2 runs)  | 1×      |
-| `8`  | 8           | ~32   | 2.2 s / 1.9 s           | ~4.5×   |
-| `32` | 32          | 54    | 1.2 s                   | ~8×     |
-
-The gain comes from overlapping each sub-request's per-page latency and
-server-side work, so the exact multiplier scales with how many pages the pull
-spans — a larger pull (more pages) has more parallelism to exploit. The extra
-sub-requests each cost quota, so reserve a large `n` for pulls you know are
-large.
+The **request count is unchanged** — only their timing is. That matters because
+the hourly [rate limit](https://api.waterdata.usgs.gov/signup/) counts requests,
+so the speedup is free of quota. How many pages are in flight at once is capped
+by `API_USGS_CONCURRENT` (default 32); set it to `1` to page sequentially.
+
+Measured, against the live API with `limit=2000`:
+
+| pull                                       | pages | sequential | offset-parallel | speedup     |
+| ------------------------------------------ | ----- | ---------- | --------------- | ----------- |
+| `get_daily`, 8 sites, 2 years              | 8     | 4.6 s      | 2.2 s           | 2.1×        |
+| `get_daily`, 1 site, full history (~19k rows) | 10  | 6.3 s      | 1.5–2.0 s       | 3.1–4.2×    |
+
+The second row is the case that previously had no answer at all: a single-site
+deep history has no multi-value argument to divide, so splitting the *query* to
+gain parallelism was impossible. Paging by offset parallelizes the pages
+themselves, so it applies whether or not the query can be split.
+
+Because `offset` is a Water Data extension rather than part of OGC API -
+Features, the walk is defensive: it verifies the server is honoring the
+parameter before trusting any rows, and falls back to standard cursor
+pagination if not. Past the API's `offset` ceiling of 40,000 rows it continues
+with a sequential cursor walk, so arbitrarily deep results still come back
+complete.
 
 Visit the
 [API Reference](https://doi-usgs.github.io/dataretrieval-python/reference/waterdata.html)

dataretrieval/__init__.py

@@ -45,11 +45,6 @@
     URLTooLong,
 )
 
-# Parallel-chunks control (a context manager). Defined with the chunker in
-# ``dataretrieval.ogc.chunking``; surfaced here for a stable public path
-# ``from dataretrieval import parallel_chunks``.
-from dataretrieval.ogc.chunking import parallel_chunks
-
 # Resumable chunk-interruption exceptions. They are defined in
 # ``dataretrieval.ogc.interruptions`` rather than ``dataretrieval.exceptions``
 # because they carry pandas/httpx state and a resumable ``ChunkedCall`` handle,
@@ -101,6 +96,5 @@
     "QuotaExhausted",
     "ServiceInterrupted",
     # parallel-chunks control (defined in ogc.chunking)
-    "parallel_chunks",
     "__version__",
 ]