feat(api): add search, filtering, and sorting to the v2 list endpoints - #6189
Conversation
One convention across every v2 list, documented on lib/api/contracts/v2/shared.ts: `search` (case-insensitive substring on the resource's natural name field), `sortBy` + `sortOrder` (per-resource enum, never a free string), and enumerated resource-specific filters. Reuses the sortBy/sortOrder pair v2 logs and v2 knowledge-documents already ship rather than inventing a third dialect alongside the Logs filters and the Tables predicate grammar. Every filter and sort is pushed into SQL. GET /api/v2/files previously read the whole scope and sorted/sliced it in JS; it now goes through a new queryWorkspaceFiles that filters, orders, and bounds the page in one query. Cursors are stamped with the sort they were minted under, so replaying one under a different sort is a 400 instead of silently duplicated or skipped rows.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
PR SummaryMedium Risk Overview Query layer: New Cursors: Other: Reviewed by Cursor Bugbot for commit 0888ee0. Bugbot is set up for automated code reviews on this repo. Configure here. |
Greptile SummaryThe PR adds a consistent search, filtering, and sorting convention to v2 list endpoints and moves file filtering, ordering, and keyset pagination into SQL.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains; the previously reported malformed-cursor path now rejects invalid key values before SQL and maps the rejection to a 400 response.
|
| Filename | Overview |
|---|---|
| apps/sim/lib/api/list-query.ts | Adds reusable escaped search predicates, deterministic ordering, and typed keyset cursor codecs; malformed key values are rejected before SQL. |
| apps/sim/app/api/v2/lib/response.ts | Adds sort-stamped cursor envelope encoding and decoding while delegating value validation to the keyset codecs. |
| apps/sim/app/api/v2/files/route.ts | Replaces in-memory file pagination with validated, SQL-backed filtering, sorting, and keyset pagination. |
| apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts | Implements bounded workspace-file queries and classifies invalid cursor keys as validation errors. |
| apps/sim/app/api/v2/workflows/route.ts | Makes workflow keyset pagination follow the selected sort and returns a client error for incompatible cursor keys. |
| apps/sim/lib/api/contracts/v2/shared.ts | Defines the shared v2 list search and sort conventions used by resource-specific schemas. |
Sequence Diagram
sequenceDiagram
participant Client
participant Route as V2 List Route
participant Codec as Cursor Codec
participant Query as SQL Query Layer
participant DB as PostgreSQL
Client->>Route: GET with search/filter/sort/cursor
Route->>Route: Validate query parameters
Route->>Codec: Decode sort-stamped cursor
Codec-->>Route: Typed keys or invalid
alt Invalid envelope or keys
Route-->>Client: 400 BAD_REQUEST
else Valid request
Route->>Query: Filters, ordering, limit, cursor keys
Query->>DB: Bounded keyset query
DB-->>Query: Page and next-row marker
Query-->>Route: Items and next keys
Route->>Codec: Encode next cursor
Route-->>Client: Items and nextCursor
end
Reviews (2): Last reviewed commit: "fix(api): validate v2 cursor key values ..." | Re-trigger Greptile
…precision
Two review findings, fixed at the root by making a keyset key own its cursor
codec instead of hand-writing a decoder per sort.
Cursor key values are caller-controlled, and matching the sort stamp and key
count was not enough: an unparseable timestamp or a non-numeric size reached
the query as an Invalid Date or NaN and surfaced as a 500. Each key now type-
checks its own value and rejects a cursor it cannot hold, which both routes
render as the documented 400.
Timestamp keys now order and compare on date_trunc('milliseconds', col).
Postgres keeps microseconds and defaultNow() populates them, but a cursor value
round-trips through a millisecond-only JS Date — comparing the raw column
against the truncated value re-admitted the page's own last row, duplicating it
and stalling pagination outright at a page size of one. Reachable today via
workspace_files.updated_at, which insertFileMetadata leaves to defaultNow().
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 0888ee0. Configure here.
Summary
lib/api/contracts/v2/shared.ts:search(case-insensitive substring on the resource's single natural name field),sortBy+sortOrder(per-resource enum, never a free string), and enumerated resource-specific filters. Reuses thesortBy/sortOrderpair v2 logs and v2 knowledge-documents already ship — no third dialect alongside the Logs filters or the Tables predicate grammar.GET /api/v2/filesused to read the whole scope and sort/slice it in JS; it now goes through a newqueryWorkspaceFilesthat filters, orders, and bounds the page in one query. Workflows' keyset now followssortBy.folderIdto the files, tables, and knowledge lists; credentials had noORDER BYat all and now has a deterministic default.Notes
positionnames a resource's stored manual arrangement (thesort_ordercolumn on workflows/folders), spelled differently from thesortOrderparam on purpose.mapWithto@sim/testing's mocksqlfragment (a real gap that madegetKnowledgeBasesthrow under test). That surfaced one existing knowledge test that was green only because of the throw; corrected its assertion to what it actually claims.Type of Change
Testing
bun run check:api-validation:strict,bun run check:openapi(7 specs, 89 operations),bun run type-check,bun run lint:check— all passlist-convention.test.tspinning each resource's search column and ordering%/_; an unknown sort field 400s before reaching SQL; pagination terminates with a filter applied; a filter plus a cursor returns a consistent pageChecklist