Skip to content

Commit 3bcef4c

Browse files
committed
feat: add create_index to HotdataClient
Bring the framework client to parity with `hotdata indexes create`, which already creates BM25 and vector indexes from the CLI. Previously callers had to drop to the raw `hotdata.IndexesApi`, leaving a managed database's data loaded but not searchable. Covers both vector-index modes: a plain index over an existing vector column, and a provider-backed index over a source text column that the server embeds. The returned `source_column` names the column to pass to `vector_distance` in the latter. The build runs as a background job whose submit call reports success even when the build later fails, so the method polls the job to a terminal state and raises RuntimeError with the job's error_message. `wait=False` mirrors the CLI's `--async`. `index_name` derives from `{table}_{columns}_{index_type}` when omitted, matching the CLI so both surfaces name the same index identically. `index_type` is required rather than inheriting the API's "sorted" default. Combinations the server would silently ignore — a multi-column vector index, or vector-only options on a non-vector index — raise ValueError, since each otherwise surfaces only as an unaccelerated query. Release 0.10.0.
1 parent 0fb624d commit 3bcef4c

10 files changed

Lines changed: 1124 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010

11+
## [0.10.0] - 2026-08-06
12+
13+
### Added
14+
15+
- `create_index(database, table, columns=..., index_type=...)` builds an index on a
16+
managed table, bringing the framework client to parity with `hotdata indexes
17+
create` in the CLI. It covers all three index kinds the API accepts: `"bm25"` for
18+
full-text search, `"vector"` for nearest-neighbour search, and `"sorted"`.
19+
Previously the framework had no index API at all and callers had to drop to the raw
20+
`hotdata.IndexesApi`, which left a managed database's data loaded but not
21+
searchable: full-text queries error without an index, and vector queries run at
22+
full-scan speed. Like the other managed-table operations, `database` accepts a
23+
name/id or an already-resolved `ManagedDatabase`. Indexing a table on a plain
24+
(non-managed) connection is not covered — the CLI's `--catalog` handles that.
25+
26+
`index_name` is optional and defaults to `{table}_{columns}_{index_type}`, the same
27+
derivation the CLI uses when `--name` is omitted, so both surfaces name the same
28+
index identically. `index_type` is required, unlike the API's `"sorted"` default,
29+
because the wrong kind only fails at query time.
30+
31+
The server builds the index as a background job whose submit call reports success
32+
even when the build later fails, so `create_index` polls the job to a terminal
33+
state and raises `RuntimeError` carrying the job's `error_message`. Pass
34+
`wait=False` to return once the job is accepted (`status="pending"` plus a
35+
`job_id`) and own the outcome check yourself, as the CLI's `--async` does;
36+
`timeout_s` and `poll_interval_s` tune the wait.
37+
38+
Both vector-index modes are supported. Omitting `embedding_provider_id` indexes an
39+
existing vector column, queried with a literal vector — and there `metric` must
40+
match the distance function the query uses (`cosine``cosine_distance`,
41+
`l2``l2_distance`, `dot``negative_dot_product`), since a mismatch silently
42+
reverts to a full table scan rather than erroring. Setting
43+
`embedding_provider_id` indexes a *text* column instead: the provider embeds it
44+
into `output_column`, queries pass text via `vector_distance(source_col, 'query')`,
45+
and the server resolves the distance function itself.
46+
47+
Argument combinations that the server would silently ignore raise `ValueError`
48+
before any request is sent: an unknown `index_type` or `metric`, a vector index
49+
with more than one column (the engine indexes only the first), and
50+
`metric`/`dimensions`/`embedding_provider_id`/`output_column`/`description` on a
51+
non-vector index.
52+
53+
- `CreateIndexResult`, the frozen dataclass `create_index` returns, is exported from
54+
`hotdata_framework` and added to the public contract surface. Its `source_column`
55+
names the text column to query for a provider-backed vector index, and is `None`
56+
for BM25, sorted, and plain vector indexes.
57+
1158
## [0.9.0] - 2026-07-23
1259

1360
### Added

CONTRACT.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ The supported import surface is:
3333
- `ManagedDatabase`
3434
- `ManagedTable`
3535
- `LoadManagedTableResult`
36+
- `CreateIndexResult`
3637
- `DEFAULT_SCHEMA`
3738
- `is_parquet_path`
3839

@@ -63,7 +64,8 @@ Adapters should import from `hotdata_framework` and treat this surface as the st
6364
- `upload_parquet(path)` uploads a local parquet file and returns an upload id.
6465
- `load_managed_table(database, table, schema=..., upload_id=..., file=...)` publishes parquet data into a declared managed table.
6566
- `delete_managed_table(database, table, schema=...)` deletes a managed table.
66-
- The `database` argument of `list_managed_tables`, `load_managed_table`, `add_managed_table`, `delete_managed_table`, `delete_managed_database`, and `execute_sql` accepts a name/id **or** an already-resolved `ManagedDatabase`. Passing a `ManagedDatabase` skips the name/id read probe, so a create-scoped key that cannot read `/databases` can load into a database it just created.
67+
- `create_index(database, table, schema=..., columns=..., index_type=..., index_name=...)` builds a `"sorted"`, `"bm25"`, or `"vector"` index on a managed table and returns a `CreateIndexResult`. It is the framework-side equivalent of the CLI's `hotdata indexes create`; indexing a table on a plain (non-managed) connection is out of scope. `index_name` defaults to `{table}_{columns}_{index_type}`, matching the CLI's derivation when `--name` is omitted. `index_type` is required rather than defaulting to the API's `"sorted"`. The build runs as a background job; the call polls it to a terminal state and raises `RuntimeError` with the job's `error_message` when it fails, because the submit call reports success regardless. `wait=False` returns as soon as the job is accepted, with `status="pending"` and a `job_id` for the caller to poll. For `index_type="vector"`, omitting `embedding_provider_id` indexes an existing vector column and `metric` (`"l2"`, `"cosine"`, `"dot"`) selects the distance function the index accelerates — a query using a different function silently falls back to a full scan; setting `embedding_provider_id` indexes a source *text* column instead, and the returned `source_column` names the column to pass to `vector_distance`. Argument combinations the server would silently ignore raise `ValueError` before any request is sent.
68+
- The `database` argument of `list_managed_tables`, `load_managed_table`, `add_managed_table`, `delete_managed_table`, `delete_managed_database`, `create_index`, and `execute_sql` accepts a name/id **or** an already-resolved `ManagedDatabase`. Passing a `ManagedDatabase` skips the name/id read probe, so a create-scoped key that cannot read `/databases` can load into a database it just created.
6769

6870
### `QueryResult`
6971

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ Runtime boundary and guarantees are defined in `CONTRACT.md`.
1616
- **Result utilities** — convert query results to records, pandas DataFrames, or metadata dictionaries for adapter display layers.
1717
- **History helpers** — list recent results and query run history with normalized dataclasses.
1818
- **Managed databases** — create Hotdata-owned catalogs, declare tables, upload parquet, and load managed tables (mirrors `hotdata databases` in the CLI).
19+
- **Indexes** — build BM25, vector, or sorted indexes on managed tables (parity with `hotdata indexes create`), waiting on the background build and surfacing its failure instead of reporting a phantom success.
1920
- **Health helpers** — build compact API/workspace health summaries for UI integrations.
2021

2122
Install:

hotdata_framework/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
)
1111
from hotdata_framework.databases import (
1212
DEFAULT_SCHEMA,
13+
CreateIndexResult,
1314
LoadManagedTableResult,
1415
ManagedDatabase,
1516
ManagedTable,
@@ -43,6 +44,7 @@
4344

4445
__all__ = [
4546
"DEFAULT_SCHEMA",
47+
"CreateIndexResult",
4648
"HotdataClient",
4749
"HotdataError",
4850
"HotdataTerminalError",

0 commit comments

Comments
 (0)