Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- `open_result_arrow` and `ArrowResultStream`: fetch a result as an Arrow IPC
stream decoded directly off the socket, so peak memory is one record batch
rather than the whole result. `get_result_arrow` and `stream_result_arrow`
both collect the entire body first and are unchanged. Also available as
`Client::open_result_arrow`.

## [0.17.0] - 2026-09-10

Expand Down
6 changes: 5 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,18 @@ bytes = "^1"
futures-core = "^0.3"
log = "^0.4"
arrow-ipc = { version = "59", optional = true }
# Only for `Buffer`, which `arrow-ipc`'s push decoder names in its public
# signature and does not re-export. Backs the zero-copy handoff from a reqwest
# body chunk (`bytes::Bytes`) to `StreamDecoder::decode`.
arrow-buffer = { version = "59", optional = true }
arrow-array = { version = "59", optional = true }
arrow-schema = { version = "59", optional = true }

[features]
default = ["native-tls"]
native-tls = ["reqwest/native-tls"]
rustls = ["reqwest/rustls"]
arrow = ["dep:arrow-ipc", "dep:arrow-array", "dep:arrow-schema"]
arrow = ["dep:arrow-ipc", "dep:arrow-array", "dep:arrow-schema", "dep:arrow-buffer"]

[dev-dependencies]
tokio = { version = "^1.46.0", features = ["rt-multi-thread", "macros", "time"] }
Expand Down
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,19 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
}
```

Both methods accept `offset` and `limit` for pagination, and both carry the same authentication and workspace headers as the generated operations. They return `ArrowError::NotReady` if the result is still pending or processing — poll `client.get_result(result_id, database_id)` until its status is `ready` first. `ArrowResult` also surfaces the `X-Total-Row-Count` header (`total_row_count`) and the `rel="next"` pagination `Link` (`next_link`).
A third, `open_result_arrow`, decodes straight off the socket: it pulls body chunks as batches are asked for, so peak memory is one record batch rather than the whole result. Use it for a result larger than memory — the other two collect the entire body before returning. Its schema is available before the first batch, and the pooled connection stays checked out until the stream is drained or dropped.

```rust
let mut stream = client
.open_result_arrow(&result_id, &database_id, None, None)
.await?;
println!("columns: {:?}", stream.schema().fields());
while let Some(batch) = stream.next_batch().await? {
// ... one batch at a time; the rest is still on the wire
}
```

All three accept `offset` and `limit` for pagination, and all carry the same authentication and workspace headers as the generated operations. They return `ArrowError::NotReady` if the result is still pending or processing — poll `client.get_result(result_id, database_id)` until its status is `ready` first. `ArrowResult` also surfaces the `X-Total-Row-Count` header (`total_row_count`) and the `rel="next"` pagination `Link` (`next_link`).

To run a query and get its result as Arrow in a single call — submit, await
`ready`, and decode — use `query_to_arrow`:
Expand Down
Loading
Loading