perf(highlight): load Shiki WASM bytes without base64 decoding

[elucid]

Follow-up to #1063. Second of two first-interaction fixes; independent of the help/menu overlay PR.

Problem

Pierre requests the Oniguruma engine via import("shiki/wasm"), which resolves to @shikijs/engine-oniguruma/wasm-inlined: a 622 KB base64 literal decoded with Uint8Array.from(atob(...)) on the main thread the first time a highlighter is prepared. That runs right after the first frame, so it lands between the user's first keypress and its render. Measured (n=5, median / max):

	Linux 4 vCPU	macOS M-series
synchronous string→bytes	29.2 / 33.6 ms	5.9 / 7.3 ms
first highlighter prep, wall	51.9 / 56.2 ms	12.5 / 16.7 ms
largest event-loop gap during prep	33.9 / 38.5 ms	6.7 / 8.6 ms

Approach

A tsconfig paths alias maps shiki/wasm → packages/hunk/src/lib/shikiWasm.ts, which reads Shiki's public shiki/onig.wasm asset through Bun's file loader and calls WebAssembly.instantiate. Same bytes, engine, grammars, themes and Pierre lifecycle; token output is byte-identical (SHA-256 checked in tests). No dependency patching or vendoring. shiki@3.23.0 becomes an explicit root devDependency (already present transitively at that version).

After: string→bytes conversions 1 → 0; Linux prep 51.9 → 20.0 ms, largest gap 33.9 → 12.9 ms; macOS prep 12.5 → 5.8 ms.

The alias applies to source runs, build:bin (asset embedded — verified /$bunfs/root/onig-*.wasm, and B:/~BUN/root/... on a Windows cross-compile), build:npm (asset emitted beside main.js, resolved against the module directory, not cwd), the highlight worker, and bun test. The OpenTUI Node facade externalizes @pierre/diffs, so downstream Node consumers keep upstream Shiki behaviour — this adapter is Bun-only by design.

Tests / checks

• packages/hunk/src/lib/shikiWasm.test.ts: fresh-process zero-decode assertions for a source entry, the real highlight worker entry, a relocated JS bundle, and a compiled executable; stock-engine vs adapter token parity (TypeScript + Elixir); exact asset size so an upstream change forces review.
• test/pty/highlighting.test.ts: new small-diff (main-thread) highlighting case alongside the existing worker/Elixir cases; passes for source, compiled binary and npm main on macOS and Linux.
• bun run typecheck, deps:check (alias-only entry accounted for in .dependency-cruiser.cjs), lint, format:check, build:bin, build:npm — pass.
• bun run test — same 6 pre-existing failures as main; 1 remaining (jj not installed) with isolated config.
• bun run test:integration — the 5 failures reproduce on main (macOS /var vs /private/var TMPDIR trust-state cases; 3 signal-lifecycle cases).
• bun run test:tty-smoke — Linux 10/10; macOS skipped (no util-linux script).
• bench:highlight-prefetch / bench:interaction-latency: no material change; macOS next_file_ready_ms is bimodal (10 / 50 ms) in both stock and candidate runs with adjacent_ready_before_move=1 throughout, so it is measuring act/render settling rather than highlighting.
• Not run on Windows (path handling inspected via cross-compile only).

Pre-existing, not addressed

build:npm does not emit highlightWorkerEntry.js, so the npm large-file worker PTY test fails on main and here identically.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
hunk-web	Ignored	Preview	Sep 10, 2026 4:05am UTC

[greptile-apps]

Greptile Summary

This PR redirects Pierre’s Bun-targeted shiki/wasm import to an adapter that loads Shiki’s raw Oniguruma WASM asset, avoiding synchronous base64 decoding during initial highlighting.

• Adds a Bun-specific raw-WASM instantiation adapter and TypeScript asset declaration.
• Pins the existing Shiki version as an explicit root development dependency.
• Adds source, worker, bundle, executable, token-parity, and PTY coverage.
• Updates dependency-cruiser entry points and release notes.

Confidence Score: 4/5

The runtime change appears sound, but the explicit filename and test-import requirements must be satisfied before merging.

The adapter’s important source, worker, bundle, and executable paths have targeted regression coverage, and no behavioral or security failure remains; the accepted findings are repository-rule compliance issues.

Files Needing Attention: packages/hunk/src/lib/shikiWasm.ts, packages/hunk/src/lib/shikiWasmAssets.d.ts, packages/hunk/src/lib/shikiWasm.test.ts

Important Files Changed

Filename	Overview
packages/hunk/src/lib/shikiWasm.ts	Adds the Bun-specific raw-WASM loader; runtime behavior is well covered, but the new filename violates the dash-case rule.
packages/hunk/src/lib/shikiWasm.test.ts	Adds broad fresh-process and build-mode regression coverage, while using disallowed dynamic imports and a nonconforming filename.
packages/hunk/src/lib/shikiWasmAssets.d.ts	Declares the WASM file-loader module shape, but its filename does not follow the required dash-case convention.
tsconfig.json	Redirects bundled shiki/wasm imports to the Bun adapter.
test/pty/highlighting.test.ts	Adds main-thread highlighting coverage and parameterizes the existing fixture helper.
package.json	Pins Shiki 3.23.0 explicitly to align the imported asset with Pierre’s resolved engine.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Pierre["@pierre/diffs highlighter"] --> Alias["tsconfig alias: shiki/wasm"]
    Alias --> Adapter["Bun WASM adapter"]
    Adapter --> Asset["shiki/onig.wasm"]
    Asset --> Read["Bun.file(...).arrayBuffer()"]
    Read --> Instantiate["WebAssembly.instantiate(imports)"]
    Instantiate --> Engine["Oniguruma highlighting engine"]

Loading

Prompt To Fix All With AI

### Issue 1
packages/hunk/src/lib/shikiWasm.ts:1
**TypeScript filenames violate convention**

The new `shikiWasm.ts` file uses camel case, while the repository requires dash-case names for TypeScript files. The same violation appears in `shikiWasmAssets.d.ts` and `shikiWasm.test.ts`. Rename these files and update their references before merging.

### Issue 2
packages/hunk/src/lib/shikiWasm.test.ts:23
**Tests use dynamic imports**

The fresh-process test program dynamically imports `@pierre/diffs` here and again in the worker program at line 57. This violates the repository testing directive to hoist imports to the top level. Preserve the required instrumentation order through a preload or fixture entry instead, and satisfy this requirement before merging.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "perf(highlight): load Shiki WASM bytes w..." | Re-trigger Greptile}