docs — V30 et V31 au plan : chat plus juste, vision qui cesse d'être piégée - #48
Merged
Merged
Conversation
…iégée Deux pistes demandées par Dominic, écrites avec les chiffres qui les rendent décidables plutôt qu'avec des impressions. V30 — chat. La mesure tient à 5 bonnes réponses sur 6, et l'unique échec est une erreur de FORME, pas un manque de connaissances : le modèle trouve fare à chaque fois et se trompe d'intention. Le plan commence donc par tuer l'idée évidente, tailles à l'appui : un « modèle de 600 Mo » n'est pas une montée en gamme, c'est Qwen3-0.6B non-DQ (570 Mo), le même cerveau avec des embeddings non quantifiés. Les vrais paliers sont gemma-3-1b 764 Mo, Llama-3.2-1B 1,09 Go, SmolLM2-1.7B 1,11 Go, Qwen2.5-1.5B 1,22 Go, Qwen3-1.7B 1,43 Go — contre 370 Mo aujourd'hui. Avant de dépenser cette bande passante : un banc de 40–60 questions, puis un décodage contraint qui rend l'erreur de forme non représentable, puis des exemples tirés des colonnes réelles, puis deux tirages et un vote. Le modèle plus gros ensuite, en second téléchargement annoncé. V31 — vision. Les trois modèles pèsent 18,6 Mo au total contre 370 Mo pour le chat. Et la cause principale des fautes n'est pas le réseau : ImageNet-1k n'a pas de classe « personne » — mille étiquettes, cent vingt races de chiens, aucun humain — donc une photo de quelqu'un ressort en « suit ». Le plan : mesurer sur 30–50 images étiquetées, corriger l'espace d'étiquettes avec CLIP zero-shot (~190 Mo, l'utilisateur écrit ses propres étiquettes), vérifier le recadrage et poser un refus sous seuil de confiance, et seulement ensuite monter le détecteur en gamme. Les deux commencent par un banc : ni « un modèle plus gros » ni « ça fait encore des fautes » n'est aujourd'hui une phrase mesurable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UKw6oNC8iZ9Kn7q6x4qom4
There was a problem hiding this comment.
🟢 Approval recommended
The changes are documentation-only and the identified issues are minor clarity/consistency nits that don’t affect runtime behavior.
Pull request overview
This PR updates PLAN.md to document two new planned waves (V30 for improving the local chat over user data and V31 for improving the vision playground), emphasizing measurement-first benches and low-cost levers before increasing model sizes/downloads.
Changes:
- Adds V30 plan details: benchmarking (40–60 Qs), constrained decoding, user-column examples, and multi-sample voting before considering a larger LLM download.
- Adds V31 plan details: benchmarking (30–50 images), addressing label-space mismatch (CLIP zero-shot), preprocessing/threshold checks, and evaluating a stronger detector if justified.
- Updates the Ordering paragraph to explicitly state that new waves start with measurable benches and require an explicit launch command.
File summaries
| File | Description |
|---|---|
| PLAN.md | Adds V30/V31 planning entries and updates ordering rationale to emphasize measurable benches before launching. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+443
to
+444
| | V30 | **Chat that reads better, measured before it is made bigger.** The V27.1–V27.3 measurement stands at **5 of 6** reference questions, and the one failure is a _shape_ error, not missing knowledge: the model finds `fare` every time and picks the wrong intent. **First, the numbers that kill the obvious idea** — a « 600 MB model » is not an upgrade: at q4f16, Qwen3-0.6B **non-DQ is 570 MB and the same brain**, only its embeddings unquantised. The real rungs are gemma-3-1b-it **764 MB** (2×), Llama-3.2-1B **1.09 GB**, SmolLM2-1.7B **1.11 GB**, Qwen2.5-1.5B **1.22 GB**, Qwen3-1.7B **1.43 GB** — against 370 MB today. So the plan spends nothing on weights until the cheap levers are exhausted. **(A) A bench worth the name** — 40–60 FR/EN questions including the phrasings that fail, runnable and reported; today's 18 cases cannot run in CI, and without this nothing that follows is measurable. **(B) Constrained decoding** — a hand-written `LogitsProcessor` masking every token outside the grammar _during_ generation: after `{"kind":"` only seven tokens are legal. The shape error becomes unrepresentable rather than caught after the fact, and on this task that can beat a model four times larger. **(C) Examples drawn from the user's own columns** instead of frozen Titanic ones — 0 MB, and it removes the temptation to copy an example column. **(D) Two samples, one vote**, keeping the candidate that validates and invents no column the question never names — 0 MB, 2× the time. **Only then** the bigger model, and as a SECOND announced download (« reinforced model », 764 MB) with Qwen 370 MB staying the default: the V27 sharding infrastructure already handles it (32 parts of 24 MiB). VRAM (~1.2–1.5 GB estimated) and first-token latency to be measured before promising anything. | Owner question (22/08/2026): would a bigger model raise the share of correct answers? The measured failure is structural, so the plan tests that hypothesis for 0 MB before asking a visitor for twice the bandwidth. | | ||
| | V31 | **Vision that stops being asked the impossible.** Today's three models weigh **18.6 MB total** (EfficientNet-Lite4 int8 13.6, YOLOX-Nano 3.7, UltraFace 1.3) against 370 MB for the chat model — the headroom is enormous. **The main cause of the mistakes is not the network**: ImageNet-1k has **no « person » class** — 1000 labels, ~120 of them dog breeds, none for a human being — so a photo of someone comes back as « suit » or « jersey ». The model is not wrong; it is being asked a question whose answer is absent from its vocabulary. **(A) Measure first**: 30–50 public-domain images with expected label and expected boxes, replayed in e2e, so « it still makes mistakes » becomes a percentage. **(B) Fix the label space — the real correction**: CLIP ViT-B/32 zero-shot, vision q4f16 **126 MB** + text int8 **64 MB** ≈ **190 MB**, letting the visitor type their own labels (« a cat », « an invoice », « a houseplant »). It repairs the defect and makes a far better demonstration than 1000 frozen classes; open weights, self-hosted, local execution — the doctrine holds. **(C) What costs no download**: check the crop (squashing a 16:9 photo into a square skews everything — `preprocess.ts` is the suspect), average over two crops, recalibrate `OBJECT_THRESHOLD` (0.35) and `FACE_THRESHOLD` (0.9), and above all **refuse below a confidence floor** — « I am not sure » rather than a label picked at random, which is the chat's doctrine applied to pixels. **(D) A better detector**: YOLOX-S (Apache-2.0), ~35 MB, roughly +14 mAP over Nano — with acquisition and licence verified first, as in V23: the YOLOX ONNX files on the Hub are community re-uploads, not official releases. | Owner report (22/08/2026): the vision playground is better than the chat but still makes mistakes. Naming the label-space mismatch is what turns a vague complaint into a fixable defect. | |
| | V31 | **Vision that stops being asked the impossible.** Today's three models weigh **18.6 MB total** (EfficientNet-Lite4 int8 13.6, YOLOX-Nano 3.7, UltraFace 1.3) against 370 MB for the chat model — the headroom is enormous. **The main cause of the mistakes is not the network**: ImageNet-1k has **no « person » class** — 1000 labels, ~120 of them dog breeds, none for a human being — so a photo of someone comes back as « suit » or « jersey ». The model is not wrong; it is being asked a question whose answer is absent from its vocabulary. **(A) Measure first**: 30–50 public-domain images with expected label and expected boxes, replayed in e2e, so « it still makes mistakes » becomes a percentage. **(B) Fix the label space — the real correction**: CLIP ViT-B/32 zero-shot, vision q4f16 **126 MB** + text int8 **64 MB** ≈ **190 MB**, letting the visitor type their own labels (« a cat », « an invoice », « a houseplant »). It repairs the defect and makes a far better demonstration than 1000 frozen classes; open weights, self-hosted, local execution — the doctrine holds. **(C) What costs no download**: check the crop (squashing a 16:9 photo into a square skews everything — `preprocess.ts` is the suspect), average over two crops, recalibrate `OBJECT_THRESHOLD` (0.35) and `FACE_THRESHOLD` (0.9), and above all **refuse below a confidence floor** — « I am not sure » rather than a label picked at random, which is the chat's doctrine applied to pixels. **(D) A better detector**: YOLOX-S (Apache-2.0), ~35 MB, roughly +14 mAP over Nano — with acquisition and licence verified first, as in V23: the YOLOX ONNX files on the Hub are community re-uploads, not official releases. | Owner report (22/08/2026): the vision playground is better than the chat but still makes mistakes. Naming the label-space mismatch is what turns a vague complaint into a fixable defect. | | ||
|
|
||
| **Ordering**: V23 first (owner request); V24 keeps its vocabulary capped — | ||
| **Ordering**: V30 and V31 both start with a bench, because neither « a bigger model » nor « it still makes mistakes » is a measurable statement today; no wave starts without an explicit launch command. V23 first (owner request); V24 keeps its vocabulary capped — |
| | **V28 — delivered** | **« Ne nous croyez pas sur parole »** — a `/privacy` route that states the local-only promise once, in full, and then hands the reader the means to check it without trusting a word of it. Four verification steps, ordered by how hard they are to fake: cut the network (DevTools → Network → Offline, or the Wi-Fi switch) and watch the whole lab keep working; watch the Network tab while loading a file and training, and see nothing happen; read `Content-Security-Policy` on the document itself; open Application → IndexedDB and see exactly what was kept. The served policy is **quoted verbatim on the page and pinned to `public/_headers` by a unit test** — a page that claims a protection the site quietly dropped is worse than no page. A schematic of the Network panel is drawn rather than screenshotted (DevTools chrome differs per browser and per locale) and captioned as a diagram, not a capture. A live audit panel counts this page's own resource timings by origin and says, in the same breath, what it cannot see: worker timelines and requests the CSP blocked — a proof that oversells itself is worth less than none. Last section lists what _does_ cross the network (app files, demo datasets on click, vision models on entering Vision, LLM weights on explicit consent) and what never does. FR/EN, prerendered shell, WCAG AA verified by axe including the audit result. 344 unit tests, 60 e2e. | Owner request (22/08/2026): the promise is repeated across the site, but a user has no way to tell a true claim from a comforting one. Verifiability is the product here — anyone can write « your data stays local » in a footer. | | ||
| | **V29 — delivered** | **Analytical SQL in the browser (DuckDB-Wasm, MIT)**: the Data Studio gains a real OLAP engine — joins, window functions, aggregations — over the file you just loaded, with no server and no upload. The file is queried **as dropped, before the cleaning recipe**: the recipe belongs to the studio, and a result traceable to nothing the user can reopen would be worse than no SQL at all. Extra CSV / **Parquet** / JSON files can be attached in the same session (Parquet is a new input format for the lab), each exposed as a view named after the file; a result exports to CSV or goes to the ML Lab in one click, through the handoff path V4 already built. Errors show **DuckDB's own message** — it names the line and the token, which no paraphrase of ours would. **The measurement that set the version**: `@duckdb/duckdb-wasm` is pinned to **1.28.0**, not `latest`. From 1.29 the binaries cross Cloudflare Pages' hard 25 MiB per-file limit (eh 34.2 MiB, mvp 39.4 MiB); at 1.28.0 they are **17.3 and 21.1 MiB** and fit. Newer would have meant sharding the wasm and either widening `connect-src` to `blob:` — days after publishing a page that quotes that very directive — or rebuilding the service worker in injectManifest mode. An older engine was the cheaper honest trade, and it is written here so the next upgrade re-measures instead of rediscovering. Self-hosted under `/duckdb/` (the library defaults to jsDelivr, which the CSP refuses), **never precached** — cached on first use like the vision models, so nobody pays 18 MiB before opening the console — and the `coi` threaded build is left out entirely: no COOP/COEP, no SharedArrayBuffer, single-threaded as the assumed mode. Remote S3/HTTP querying stays out, by CSP and by intent. 352 unit tests, 61 e2e. | Owner request (21/08/2026): real analytical SQL on ~100 MB files with zero backend. Delivered after the /privacy page at the owner's request (22/08/2026). | | ||
| | V30 | **Chat that reads better, measured before it is made bigger.** The V27.1–V27.3 measurement stands at **5 of 6** reference questions, and the one failure is a _shape_ error, not missing knowledge: the model finds `fare` every time and picks the wrong intent. **First, the numbers that kill the obvious idea** — a « 600 MB model » is not an upgrade: at q4f16, Qwen3-0.6B **non-DQ is 570 MB and the same brain**, only its embeddings unquantised. The real rungs are gemma-3-1b-it **764 MB** (2×), Llama-3.2-1B **1.09 GB**, SmolLM2-1.7B **1.11 GB**, Qwen2.5-1.5B **1.22 GB**, Qwen3-1.7B **1.43 GB** — against 370 MB today. So the plan spends nothing on weights until the cheap levers are exhausted. **(A) A bench worth the name** — 40–60 FR/EN questions including the phrasings that fail, runnable and reported; today's 18 cases cannot run in CI, and without this nothing that follows is measurable. **(B) Constrained decoding** — a hand-written `LogitsProcessor` masking every token outside the grammar _during_ generation: after `{"kind":"` only seven tokens are legal. The shape error becomes unrepresentable rather than caught after the fact, and on this task that can beat a model four times larger. **(C) Examples drawn from the user's own columns** instead of frozen Titanic ones — 0 MB, and it removes the temptation to copy an example column. **(D) Two samples, one vote**, keeping the candidate that validates and invents no column the question never names — 0 MB, 2× the time. **Only then** the bigger model, and as a SECOND announced download (« reinforced model », 764 MB) with Qwen 370 MB staying the default: the V27 sharding infrastructure already handles it (32 parts of 24 MiB). VRAM (~1.2–1.5 GB estimated) and first-token latency to be measured before promising anything. | Owner question (22/08/2026): would a bigger model raise the share of correct answers? The measured failure is structural, so the plan tests that hypothesis for 0 MB before asking a visitor for twice the bandwidth. | | ||
| | V31 | **Vision that stops being asked the impossible.** Today's three models weigh **18.6 MB total** (EfficientNet-Lite4 int8 13.6, YOLOX-Nano 3.7, UltraFace 1.3) against 370 MB for the chat model — the headroom is enormous. **The main cause of the mistakes is not the network**: ImageNet-1k has **no « person » class** — 1000 labels, ~120 of them dog breeds, none for a human being — so a photo of someone comes back as « suit » or « jersey ». The model is not wrong; it is being asked a question whose answer is absent from its vocabulary. **(A) Measure first**: 30–50 public-domain images with expected label and expected boxes, replayed in e2e, so « it still makes mistakes » becomes a percentage. **(B) Fix the label space — the real correction**: CLIP ViT-B/32 zero-shot, vision q4f16 **126 MB** + text int8 **64 MB** ≈ **190 MB**, letting the visitor type their own labels (« a cat », « an invoice », « a houseplant »). It repairs the defect and makes a far better demonstration than 1000 frozen classes; open weights, self-hosted, local execution — the doctrine holds. **(C) What costs no download**: check the crop (squashing a 16:9 photo into a square skews everything — `preprocess.ts` is the suspect), average over two crops, recalibrate `OBJECT_THRESHOLD` (0.35) and `FACE_THRESHOLD` (0.9), and above all **refuse below a confidence floor** — « I am not sure » rather than a label picked at random, which is the chat's doctrine applied to pixels. **(D) A better detector**: YOLOX-S (Apache-2.0), ~35 MB, roughly +14 mAP over Nano — with acquisition and licence verified first, as in V23: the YOLOX ONNX files on the Hub are community re-uploads, not official releases. | Owner report (22/08/2026): the vision playground is better than the chat but still makes mistakes. Naming the label-space mismatch is what turns a vague complaint into a fixable defect. | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Deux pistes demandées aujourd'hui, inscrites au PLAN avec les chiffres qui les rendent décidables plutôt qu'avec des impressions. Aucune n'est lancée : elles attendent une commande explicite.
V30 — chat sur vos données
La mesure tient à 5 bonnes réponses sur 6, et l'unique échec est une erreur de forme, pas un manque de connaissances : le modèle trouve
fareà chaque fois et se trompe d'intention.Le plan commence donc par tuer l'idée évidente, tailles réelles à l'appui (q4f16, format déjà utilisé) :
Un « modèle de 600 Mo » n'est pas une montée en gamme. Avant de demander le double de bande passante à un visiteur : (A) un banc de 40–60 questions FR/EN — les 18 cas actuels ne tournent pas en CI, donc rien n'est mesurable ; (B) un décodage contraint (
LogitsProcessormasquant tout token hors grammaire pendant la génération, l'erreur de forme devient non représentable) ; (C) des exemples tirés des colonnes réelles du fichier ; (D) deux tirages et un vote. Le modèle plus gros ensuite, en second téléchargement annoncé, Qwen 370 Mo restant le défaut.V31 — terrain de jeu vision
Les trois modèles pèsent 18,6 Mo au total contre 370 Mo pour le chat. Et la cause principale des fautes n'est pas le réseau : ImageNet-1k n'a pas de classe « personne » — mille étiquettes, environ cent vingt races de chiens, aucun humain — donc une photo de quelqu'un ressort en « suit ». Le modèle ne se trompe pas ; on lui pose une question dont la réponse est absente de son vocabulaire.
(A) Mesurer sur 30–50 images étiquetées rejouées en e2e ; (B) corriger l'espace d'étiquettes avec CLIP ViT-B/32 zero-shot (vision q4f16 126 Mo + texte int8 64 Mo ≈ 190 Mo) pour que l'utilisateur écrive ses propres étiquettes ; (C) ce qui ne coûte rien — vérifier le recadrage (
preprocess.tsest le suspect), recalibrer les seuils, et refuser sous un seuil de confiance plutôt que d'inventer une étiquette ; (D) YOLOX-S (~35 Mo, +14 mAP) si le banc le justifie, licence vérifiée d'abord comme en V23.Le point commun
Les deux commencent par un banc, parce que ni « un modèle plus gros » ni « ça fait encore des fautes » n'est aujourd'hui une phrase mesurable. C'est noté dans le paragraphe d'ordonnancement du PLAN.
Documentation seule — aucun code touché.
Generated by Claude Code