A small example of Retrieval-Augmented Generation (RAG) that runs entirely in the browser – no server, no API key. Questions and document contents are not sent to a model server; once the app, index, and model are loaded, inference runs locally. Answers are generated by Gemma 4 E2B via Transformers.js and WebGPU.
The setup: one Angular page, one RAG service, three web workers (vector database, embedder, language model), and an offline pipeline in TypeScript.
The bundled demo corpus is a German-language computer-science archive: around 120 Wikipedia articles (CC BY-SA 4.0) on software bugs and IT security (Heartbleed, Ariane 5, Stuxnet, Meltdown/Spectre), on programming languages, algorithms, cryptography, and computing history. They are split heading-based into roughly 3,000 chunks (up to ~1,200 characters); each chunk points back to its document and section in the corpus.
The corpus is in German, so ask your questions in German for the demo to work well. The embedding model is multilingual, so other languages also work, but retrieval is best when the question language matches the corpus.
- A chat window that streams the answer token by token.
- A token counter for the assembled prompt.
- Thought process: an expandable trace of the intermediate steps (query rewriting, loaded sections) – for display only, it does not flow back into the model context.
- Context inspector: a stacked bar showing how the token budget is distributed across system prompt, conversation history, question, loaded sections, and free space – and lists the matched chunks with their similarity score and a link to the Wikipedia article.
- RAG on/off toggle: shows how the model answers without retrieval – no context, no sources.
- DE/EN switch (Transloco) and Markdown-rendered answers.
- Node.js 24+ – the offline pipeline runs with native TypeScript (
node pipeline/*.ts), and the Angular CLI requires at least v24.15. - A browser with WebGPU (current Chrome/Edge; Safari Technology Preview). Without WebGPU the model falls back to the slower WASM backend.
- On first start the browser downloads the Gemma 4 E2B model (~2–3 GB, once, then cached) as well as the embedding model and the ~25 MB index.
pnpm install
pnpm start # Angular dev server at http://localhost:4200The retrieval index (public/rag-index.json) is precomputed, so the app runs
immediately.
The pipeline consists of TypeScript scripts that run with native Node (no build
step). Rebuild the index whenever the corpus under pipeline/corpus/ changes:
pnpm fetch:corpus # fetches the full Wikipedia articles (URL from the file header)
pnpm build:index # corpus/*.md -> public/rag-index.json
pnpm verify:recall # measures Recall@1/3/5 against pipeline/eval-queries.json
pnpm lint # ESLint over app + pipeline
pnpm typecheck:pipeline # tsc --noEmit for the pipelinefetch:corpus reads the Wikipedia URL from the --- url: … --- header of each file and
replaces the body with the full article text (plain-text extract from the MediaWiki API).
build:index splits the Markdown files into chunks heading-based, links each chunk to
its document and section (see below), embeds it with multilingual-e5-base, and writes a
compact JSON index.
verify:recall checks model-free whether the vector search ranks the right chunks at the
top.
Each chunk in the index carries its links into the corpus graph: documentId and
title (document), sectionId with parentSectionId (section tree), sectionPath
and depth (breadcrumb path), and position (reading order). This makes it possible to
navigate from a hit to the surrounding section and the whole document – the app uses this
for “small-to-big” retrieval (hit chunk → whole document).
Offline (Node) In the browser (Angular)
────────────── ────────────────────────
corpus/*.md Question
│ chunk + embed (E5) │ embed (E5, worker)
▼ ▼
public/rag-index.json ──import──▶ pgvector (PGlite, HNSW) ──▶ Top-K chunks
│
▼
Context + system prompt
│
▼
Gemma 4 E2B (worker) ──▶ Answer (stream)
| File | Purpose |
|---|---|
pipeline/fetch-corpus.ts |
Fetches the full Wikipedia articles (URL from the file header) |
pipeline/chunking.ts |
Markdown → linked chunks (frontmatter, section tree, budget split) |
pipeline/embedding.ts |
E5 embedder + model constants |
pipeline/build-index.ts |
Corpus → rag-index.json |
pipeline/verify-recall.ts |
Recall@k measurement (own cosine search) |
src/app/vector-db.service.ts |
Vector database: PGlite + pgvector (HNSW), import & search |
src/app/rag.service.ts |
The RAG loop: load, embed question, search, build context, generate |
src/app/workers/embeddings.worker.ts |
Question → vector (E5) |
src/app/workers/llm.worker.ts |
Gemma 4 E2B, token streaming |
src/app/app.component.ts |
Chat page, token counter, context inspector |
-
Place Markdown files in
pipeline/corpus/. Each file starts with a three-line header naming the source, followed by structure via#/##/###:--- url: https://de.wikipedia.org/wiki/Therac-25 --- # Therac-25 ## Overview …
-
Optionally run
pnpm fetch:corpusto replace the bodies with the full Wikipedia articles from the headerurl. -
Run
pnpm build:index. -
Optionally adjust
pipeline/eval-queries.jsonand runpnpm verify:recall.
Retrieval uses PostgreSQL + pgvector (via PGlite/WebAssembly), the same technique as a backend, just locally in the browser with persistence in IndexedDB. Left out are BM25/hybrid search and reranking. Query rewriting (for follow-up questions) and a conversation memory are built in.
- Language model:
onnx-community/gemma-4-E2B-it-ONNX(Gemma 4, Apache 2.0) - Embeddings:
Xenova/multilingual-e5-base(768-dim)