Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 114 additions & 0 deletions docs/design/2026-09-08-lab-mode-stream.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Lab widget for `ModeStream`

**Status:** approved (chat: looks good, implement it)
**Branch (later):** `feat/lab-mode-stream`
**Date:** 2026-09-08

## Overview

[ModeStream](./2026-09-08-mode-stream.md) shipped without a Lab chapter (explicit non-goal). The explorer still stops at VectorSet. This PR adds the missing Stream chapter so the same 3-node Lab can append, range, trim, and delete a named log.

Lab-only. No Engine / proto / RF change.

## Problem

`examples/lab` registers every other structured mode (`cluster.go` `labKeyspaces`, `ui/src/api.ts` `CHAPTERS`, `Playground.tsx`). There is no `stream` keyspace, no `/v1/op` dispatch for `xadd` / `xrange` / …, and no widget. Callers cannot see owner-minted ids or snapshot fan-out the way they can for List / VectorSet.

## Non-goals

- Consumer groups, blocking `XREAD`, caller-chosen ids.
- Standalone `examples/stream` binary (Lab is the walkthrough).
- Adding `stream` to stock `supercache-node -demo-keyspace`.
- Changing Stream contract, Flags, or peer `StreamAdd`.
- New scbench cells.

## Contract

- **API / proto:** no public change. Lab HTTP `/v1/op` gains Stream verbs; same `client.Client` calls.
- **Keyspace (in-process Lab mesh only):** name `stream`, `ModeStream`, RF=2, `StreamMaxLen=0` (no auto-trim; widget has XTrim).
- **Identity:** stream `name` (default `logs`). Owner = `Owner(name)`.
- **Existing clients:** unchanged. Remote attach works if the remote mesh already has a `stream` keyspace; otherwise ops return the usual missing-keyspace / invalid-argument errors.

## Approach

Same pattern as VectorSet / List.

**Backend (`examples/lab`):**

| File | Change |
|------|--------|
| `cluster.go` | `labKeyspaces` + `modeNames` include `stream` / `ModeStream` |
| `op.go` | `opArgs` fields `ID`, `Start`, `End` (string), `Count`, `MaxLen`; dispatch `xadd` / `xrange` / `xrevrange` / `xlen` / `xdel` / `xtrim`; `isWriteOp` adds `xadd`, `xdel`, `xtrim` |
| `scene.go` | scene `stream`: two `XAdd`, then `XRange - +`; demo name `logs` |
| `lab_test.go` | keyspace list + a Stream op test |

Dispatch shapes (JSON `result`):

| Op | Args | Result |
|----|------|--------|
| `xadd` | `value` (payload string) | `{id}` |
| `xrange` / `xrevrange` | `start`, `end`, `count` | `{entries:[{id,payload}]}` |
| `xlen` | — | `{n, present}` |
| `xdel` | `id` | `{acked}` |
| `xtrim` | `max_len` | `{acked}` |
| `delete` | existing | `{acked}` |

`start` / `end` are strings (`-`, `+`, `millis-seq`, exclusive `(id`). `count<=0` means no cap (engine clamps >512). `XRevRange` uses the same min/max window as `XRange` (newest first) — not Redis high-then-low.

**UI:**

- Chapter `{ id: "stream", label: "Stream", ks: "stream", name: "logs" }`
- `widgets/Stream.tsx`: stacked labeled controls + placeholders (same as List / Geo)
- name, payload → **XAdd** (prints minted id in the results pane)
- start / end / count → **XRange** / **XRevRange** / **XLen**
- id → **XDel**
- maxLen → **XTrim**
- **Delete** whole stream
- Visual: after a range, a vertical log of `id` + payload (order from the RPC, no client-side filter). Empty range is empty, not an error. Missing name: `XLen` `present=false`.
- Results stay in the existing right-hand results pane (`last` JSON). Do not add a second results column inside the widget.

**Product docs (same PR):** `examples/lab/README.md` chapter table. `docs/design/README.md` shipped row. No API/OpenAPI/PLAN change (no new public verb).

### Rejected alternatives

| Idea | Why not |
|------|---------|
| `examples/stream` binary | Second walkthrough; Lab already covers modes |
| StreamMaxLen=16 in Lab | Auto-trim would surprise; XTrim is enough |
| Redis `XREVRANGE + -` in the widget | Engine contract is start/end min/max |
| Client-side hit filter / hollow “window” | Same class of bug as the Geo radius ring |

## Tests (write these first, after approval)

| Test | Package | Asserts |
|------|---------|---------|
| `TestLabHTTPCluster` | `examples/lab` | keyspace list includes `stream` |
| `TestLabStreamXAddRange` | `examples/lab` | two `xadd`; `xlen` n=2 present; `xrange - +` both ids oldest-first; `xrevrange - +` newest first |
| `TestLabStreamWrongVerb` | `examples/lab` | `get` on `stream` → HTTP 400 + `invalid_argument` |
| `TestLabHoldFalse` | `examples/lab` | still passes (walkthrough unchanged) |

## Bench risk

- **Hot path?** no. Lab HTTP + demo keyspace only.
- Gate: shared smoke ±10%; Get-hit / StoreGetHit allocs flat.

## Key Decisions

1. Lab chapter only — reuse shipped Stream RPCs.
2. Default name `logs`; `StreamMaxLen=0`.
3. Timeline visual from the last range RPC, no extra filtering.
4. Same PR updates Lab README only (plus design index).

## Open Questions

None.

## PR Plan

### PR 1 — `feat: Lab ModeStream chapter`

- **Title:** feat: Lab ModeStream chapter
- **Branch:** `feat/lab-mode-stream`
- **Files:** `examples/lab` backend + `ui/src` widget + rebuilt `ui/dist`; this design; Lab README
- **Dependencies:** ModeStream on `main` (PR #50)
- **Description:** Explorer only. No Engine contract change.
1 change: 1 addition & 0 deletions docs/design/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Do not implement from a draft.
| [2026-09-02-feature-folders.md](./2026-09-02-feature-folders.md) | Feature folders (List first) |
| [2026-09-04-mode-vector-set.md](./2026-09-04-mode-vector-set.md) | `ModeVectorSet` |
| [2026-09-08-mode-stream.md](./2026-09-08-mode-stream.md) | `ModeStream` |
| [2026-09-08-lab-mode-stream.md](./2026-09-08-lab-mode-stream.md) | Lab Stream chapter |
| [2026-08-25-list-counter-version.md](./2026-08-25-list-counter-version.md) | List/Counter snapshot version |
| [2026-08-13-unify-grpc-error-map.md](./2026-08-13-unify-grpc-error-map.md) | grpcmap |

Expand Down
1 change: 1 addition & 0 deletions examples/lab/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ runs the same verbs by hand. After each call the UI polls `LocalView` on every n
| LoadThrough | `loadthrough` | SoT miss, hit, singleflight |
| Tombstone | `cacheonly` | delete marker vs live |
| Bloom … VectorSet | matching mode | thin widget + wrong-verb on Set |
| Stream | `stream` | append-only log; XAdd returns id; XRange `-` `+` |

Lab HTTP defaults to `127.0.0.1:19080`. `-cluster` uses ephemeral cache/peer
ports (`internal/testcluster`). Mock SoT latency: `-sot-latency` (in-process only).
Expand Down
3 changes: 2 additions & 1 deletion examples/lab/cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ const (

var modeNames = []string{
"cacheonly", "loadthrough", "bloom", "set", "zset", "geo", "list",
"hash", "counter", "json", "bitmap", "hll", "topk", "cms", "vectorset",
"hash", "counter", "json", "bitmap", "hll", "topk", "cms", "vectorset", "stream",
}

type mockSoT struct {
Expand Down Expand Up @@ -79,6 +79,7 @@ func labKeyspaces(src datasource.DataSource) []keyspace.Config {
c.VectorDim = 2
return c
}(),
base("stream", keyspace.ModeStream),
}
}

Expand Down
84 changes: 83 additions & 1 deletion examples/lab/lab_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ func TestLabHTTPCluster(t *testing.T) {
}
want := []string{
"cacheonly", "loadthrough", "bloom", "set", "zset", "geo", "list",
"hash", "counter", "json", "bitmap", "hll", "topk", "cms", "vectorset",
"hash", "counter", "json", "bitmap", "hll", "topk", "cms", "vectorset", "stream",
}
got := map[string]bool{}
for _, raw := range asSlice(body["keyspaces"]) {
Expand Down Expand Up @@ -197,6 +197,88 @@ func TestLabBloomGrid(t *testing.T) {
}
}

func TestLabStreamXAddRange(t *testing.T) {
lab := startTestLab(t)
name := "logs-" + fmt.Sprint(time.Now().UnixNano())
a := postJSON(t, lab, "/v1/op", map[string]any{
"ks": "stream", "op": "xadd", "name": name,
"args": map[string]any{"value": "old"},
})
if ok, _ := a["ok"].(bool); !ok {
t.Fatalf("xadd1: %v", a)
}
id1, _ := resultMap(a)["id"].(string)
if !strings.Contains(id1, "-") {
t.Fatalf("id1: %v", a)
}
b := postJSON(t, lab, "/v1/op", map[string]any{
"ks": "stream", "op": "xadd", "name": name,
"args": map[string]any{"value": "new"},
})
id2, _ := resultMap(b)["id"].(string)
ln := postJSON(t, lab, "/v1/op", map[string]any{
"ks": "stream", "op": "xlen", "name": name,
})
res := resultMap(ln)
if n, _ := res["n"].(float64); n != 2 {
t.Fatalf("xlen: %v", ln)
}
if present, _ := res["present"].(bool); !present {
t.Fatalf("present: %v", ln)
}
fwd := postJSON(t, lab, "/v1/op", map[string]any{
"ks": "stream", "op": "xrange", "name": name,
"args": map[string]any{"start": "-", "end": "+", "count": 0},
})
ents := resultEntries(fwd)
if len(ents) != 2 || ents[0]["id"] != id1 || ents[1]["id"] != id2 || ents[0]["payload"] != "old" {
t.Fatalf("xrange: %v", fwd)
}
rev := postJSON(t, lab, "/v1/op", map[string]any{
"ks": "stream", "op": "xrevrange", "name": name,
"args": map[string]any{"start": "-", "end": "+", "count": 1},
})
rents := resultEntries(rev)
if len(rents) != 1 || rents[0]["id"] != id2 || rents[0]["payload"] != "new" {
t.Fatalf("xrevrange: %v", rev)
}
}

func TestLabStreamWrongVerb(t *testing.T) {
lab := startTestLab(t)
code, body := postJSONStatus(t, lab, "/v1/op", map[string]any{
"ks": "stream", "op": "get", "name": "logs",
})
if code != http.StatusBadRequest {
t.Fatalf("status %d body=%s", code, body)
}
var resp map[string]any
if err := json.Unmarshal(body, &resp); err != nil {
t.Fatal(err)
}
if inv, _ := resp["invalid_argument"].(bool); !inv {
t.Fatalf("want invalid_argument: %v", resp)
}
}

func resultMap(resp map[string]any) map[string]any {
m, _ := resp["result"].(map[string]any)
if m == nil {
return map[string]any{}
}
return m
}

func resultEntries(resp map[string]any) []map[string]any {
raw, _ := resultMap(resp)["entries"].([]any)
out := make([]map[string]any, 0, len(raw))
for _, v := range raw {
m, _ := v.(map[string]any)
out = append(out, m)
}
return out
}

func TestLabHoldFalse(t *testing.T) {
var buf bytes.Buffer
if err := runWalkthrough(&buf); err != nil {
Expand Down
Loading
Loading