Skip to content

docs — V32 à V34 au plan : documentation et tutoriels, en trois vagues #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dapiced merged 1 commit into from
Aug 22, 2026
+4 −1
Merged

docs — V32 à V34 au plan : documentation et tutoriels, en trois vagues #49

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion PLAN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -442,8 +442,11 @@ data budget asks.
| **V29 — delivered** | **Analytical SQL in the browser (DuckDB-Wasm, MIT)**: the Data Studio gains a real OLAP engine — joins, window functions, aggregations — over the file you just loaded, with no server and no upload. The file is queried **as dropped, before the cleaning recipe**: the recipe belongs to the studio, and a result traceable to nothing the user can reopen would be worse than no SQL at all. Extra CSV / **Parquet** / JSON files can be attached in the same session (Parquet is a new input format for the lab), each exposed as a view named after the file; a result exports to CSV or goes to the ML Lab in one click, through the handoff path V4 already built. Errors show **DuckDB's own message** — it names the line and the token, which no paraphrase of ours would. **The measurement that set the version**: `@duckdb/duckdb-wasm` is pinned to **1.28.0**, not `latest`. From 1.29 the binaries cross Cloudflare Pages' hard 25 MiB per-file limit (eh 34.2 MiB, mvp 39.4 MiB); at 1.28.0 they are **17.3 and 21.1 MiB** and fit. Newer would have meant sharding the wasm and either widening `connect-src` to `blob:` — days after publishing a page that quotes that very directive — or rebuilding the service worker in injectManifest mode. An older engine was the cheaper honest trade, and it is written here so the next upgrade re-measures instead of rediscovering. Self-hosted under `/duckdb/` (the library defaults to jsDelivr, which the CSP refuses), **never precached** — cached on first use like the vision models, so nobody pays 18 MiB before opening the console — and the `coi` threaded build is left out entirely: no COOP/COEP, no SharedArrayBuffer, single-threaded as the assumed mode. Remote S3/HTTP querying stays out, by CSP and by intent. 352 unit tests, 61 e2e. | Owner request (21/08/2026): real analytical SQL on ~100 MB files with zero backend. Delivered after the /privacy page at the owner's request (22/08/2026). |
| V30 | **Chat that reads better, measured before it is made bigger.** The V27.1–V27.3 measurement stands at **5 of 6** reference questions, and the one failure is a _shape_ error, not missing knowledge: the model finds `fare` every time and picks the wrong intent. **First, the numbers that kill the obvious idea** — a « 600 MB model » is not an upgrade: at q4f16, Qwen3-0.6B **non-DQ is 570 MB and the same brain**, only its embeddings unquantised. The real rungs are gemma-3-1b-it **764 MB** (2×), Llama-3.2-1B **1.09 GB**, SmolLM2-1.7B **1.11 GB**, Qwen2.5-1.5B **1.22 GB**, Qwen3-1.7B **1.43 GB** — against 370 MB today. So the plan spends nothing on weights until the cheap levers are exhausted. **(A) A bench worth the name** — 40–60 FR/EN questions including the phrasings that fail, runnable and reported; today's 18 cases cannot run in CI, and without this nothing that follows is measurable. **(B) Constrained decoding** — a hand-written `LogitsProcessor` masking every token outside the grammar _during_ generation: after `{"kind":"` only seven tokens are legal. The shape error becomes unrepresentable rather than caught after the fact, and on this task that can beat a model four times larger. **(C) Examples drawn from the user's own columns** instead of frozen Titanic ones — 0 MB, and it removes the temptation to copy an example column. **(D) Two samples, one vote**, keeping the candidate that validates and invents no column the question never names — 0 MB, 2× the time. **Only then** the bigger model, and as a SECOND announced download (« reinforced model », 764 MB) with Qwen 370 MB staying the default: the V27 sharding infrastructure already handles it (32 parts of 24 MiB). VRAM (~1.2–1.5 GB estimated) and first-token latency to be measured before promising anything. | Owner question (22/08/2026): would a bigger model raise the share of correct answers? The measured failure is structural, so the plan tests that hypothesis for 0 MB before asking a visitor for twice the bandwidth. |
| V31 | **Vision that stops being asked the impossible.** Today's three models weigh **18.6 MB total** (EfficientNet-Lite4 int8 13.6, YOLOX-Nano 3.7, UltraFace 1.3) against 370 MB for the chat model — the headroom is enormous. **The main cause of the mistakes is not the network**: ImageNet-1k has **no « person » class** — 1000 labels, ~120 of them dog breeds, none for a human being — so a photo of someone comes back as « suit » or « jersey ». The model is not wrong; it is being asked a question whose answer is absent from its vocabulary. **(A) Measure first**: 30–50 public-domain images with expected label and expected boxes, replayed in e2e, so « it still makes mistakes » becomes a percentage. **(B) Fix the label space — the real correction**: CLIP ViT-B/32 zero-shot, vision q4f16 **126 MB** + text int8 **64 MB** ≈ **190 MB**, letting the visitor type their own labels (« a cat », « an invoice », « a houseplant »). It repairs the defect and makes a far better demonstration than 1000 frozen classes; open weights, self-hosted, local execution — the doctrine holds. **(C) What costs no download**: check the crop (squashing a 16:9 photo into a square skews everything — `preprocess.ts` is the suspect), average over two crops, recalibrate `OBJECT_THRESHOLD` (0.35) and `FACE_THRESHOLD` (0.9), and above all **refuse below a confidence floor** — « I am not sure » rather than a label picked at random, which is the chat's doctrine applied to pixels. **(D) A better detector**: YOLOX-S (Apache-2.0), ~35 MB, roughly +14 mAP over Nano — with acquisition and licence verified first, as in V23: the YOLOX ONNX files on the Hub are community re-uploads, not official releases. | Owner report (22/08/2026): the vision playground is better than the chat but still makes mistakes. Naming the label-space mismatch is what turns a vague complaint into a fixable defect. |
| V32 | **Documentation, the scaffolding and one finished tutorial.** A `/docs` route, linked from the footer beside « Comment ça marche », built on the **Diátaxis** split — tutorial (learning), how-to (a task), reference (lookup), explanation (the why) — because the usual failure of documentation is mixing all four on one page: a tutorial that pauses to weigh an alternative loses the beginner it was written for. A tutorial offers **no choices** and **guarantees the result**. Five rules specific to this project: **(1) the docs are tested like the code** — everything here is seeded at 42, so « you will get 0.821 accuracy » becomes an assertion in `e2e/docs.spec.ts` and a drifting page **breaks the build**; a documentation that cannot lie is the same promise as the rest of the site. **(2) Screenshots are generated** with Playwright, never hand-taken — one that cannot be regenerated does not ship. **(3) Better than a screenshot, a link that does the thing**: « try it » deep-links landing on the panel with the demo already loaded (needs small URL-parameter support), which never goes stale. **(4) Markdown lives in the repo** (`src/content/docs/**`), compiled at build with prerendered shells like every other route; no Algolia, no third-party doc host — a third-party call on a site that publishes `/privacy` would be indefensible, so search is a local index. **(5) The docs are not PLAN.md**: this file is the engineering record in English with the trade-offs; the docs are for users, FR/EN. Scope of this wave: the route, the Markdown pipeline, the table of contents, local search, and **one** complete tutorial — « premier modèle en 10 minutes » — tested end to end. It is the template every later page copies: tone, length, how figures are quoted. | Owner request (22/08/2026): document every shipped feature across /ml, /data and /ai, linked from the footer. One finished tutorial first, on purpose — writing the full reference before the template is settled means rewriting all of it. |
| V33 | **The reference, and the table of refusals.** Page-per-panel coverage of the three sections: ML Lab (leaderboard, tuning, thresholds, segments, uncertainty, learning curves, run comparison, model export/import, batch scoring), Data Studio (quality score, recipe, forced types, join, drift, anomalies, SQL console) and AI (vision, assistant, the two interpreters). Reference is dry, exhaustive and structured like the software — not prose. The page that no competitor has: **a complete table of the named refusals** — `filter-not-numeric`, `llm-part-missing`, `too-large`, `no-webgpu`, « neither interpreter understood », « the interval is not conclusive » — with what triggers each one, what it means and what to do about it. Refusing well is this project's distinguishing feature; documenting the refusals is the most honest page it can publish. Plus a formats page (CSV, Parquet, JSON, the model manifest). **Honest sizing**: this is 1–2 days of _writing_ for ~25 features in two languages. It does not automate into anything but mush. | A feature nobody can look up is a feature that does not exist for the reader; and a refusal nobody can decode reads as a bug rather than as the design it is. |
| V34 | **Explanations, how-to guides, and the pages that make the project readable as engineering.** The why-pages: why a baseline before anything else, why intervals instead of a single figure, why seed 42 everywhere, why everything runs locally (pointing at `/privacy` rather than repeating it — duplicated prose diverges), and **what LabML does not do, and why** — a project that names its limits reads as a serious one, and the limits are already measured here (the comparison question a 0.6B model does not read, the bench that needs `shader-f16`, SQL over the file before the recipe). Task-shaped how-to guides for readers who already know their way around: score a new batch, compare two runs, read a learning curve, hand a SQL result to the lab. Generated screenshots and the « try it » deep-links land here too. Every page ends with « et ensuite ? » — documentation without a next step is a dead end. | Three audiences, deliberately: the curious visitor (five minutes), the practitioner (one task), and the evaluator judging whether the engineering is rigorous. The explanation pages are what the third one reads. |

**Ordering**: V30 and V31 both start with a bench, because neither « a bigger model » nor « it still makes mistakes » is a measurable statement today; no wave starts without an explicit launch command. V23 first (owner request); V24 keeps its vocabulary capped —
**Ordering**: V32 ships one finished tutorial before any reference page — the tutorial is the template the rest copies, and settling it late means rewriting everything. V30 and V31 both start with a bench, because neither « a bigger model » nor « it still makes mistakes » is a measurable statement today; no wave starts without an explicit launch command. V23 first (owner request); V24 keeps its vocabulary capped —
V25 (delivered) chose announced sampling and a named memory guard over the
typed-array rewrite, which measurement showed unnecessary; widening the
vocabulary stays possible later. Set aside for now: multiclass thresholds. V12
Expand Down
Loading