diff --git a/.changeset/inline-source-parsers.md b/.changeset/inline-source-parsers.md new file mode 100644 index 0000000..305a3ef --- /dev/null +++ b/.changeset/inline-source-parsers.md @@ -0,0 +1,5 @@ +--- +"@tanstack/markdown": minor +--- + +Add opt-in source-level inline parsers to Markdown extensions. Declare starting characters and return a standard inline node with an explicit consumed length, before built-in formatting changes the source. Preserve escape and code precedence, expose link-label context, and share parser budgets with nested parsing. diff --git a/.changeset/optional-http-autolinks.md b/.changeset/optional-http-autolinks.md new file mode 100644 index 0000000..0dae13f --- /dev/null +++ b/.changeset/optional-http-autolinks.md @@ -0,0 +1,5 @@ +--- +"@tanstack/markdown": minor +--- + +Add a separately imported HTTP(S) autolink extension using the inline source parser API. Recognize pasted URLs and angle-bracket URLs before Markdown punctuation changes their contents, preserve code and explicit links, and apply the existing application URL policy. Core and docs-preset defaults are unchanged. diff --git a/docs/comparison.md b/docs/comparison.md index e2cb35c..fbe97c2 100644 --- a/docs/comparison.md +++ b/docs/comparison.md @@ -40,11 +40,11 @@ These repository benchmarks bundle representative browser entry points from pinn | Entry | Gzip | Brotli | | --- | ---: | ---: | -| `@tanstack/markdown/parser` | 5.0 KB | 4.6 KB | -| `@tanstack/markdown/html` | 6.8 KB | 6.2 KB | -| `@tanstack/markdown/react` | 6.7 KB | 6.2 KB | +| `@tanstack/markdown/parser` | 5.3 KB | 4.9 KB | +| `@tanstack/markdown/html` | 7.1 KB | 6.5 KB | +| `@tanstack/markdown/react` | 7.0 KB | 6.5 KB | | React with streaming extension | 6.9 KB | 6.4 KB | -| `@tanstack/markdown/octane` | 6.7 KB | 6.2 KB | +| `@tanstack/markdown/octane` | 7.0 KB | 6.5 KB | | Marked | 12.5 KB | 11.5 KB | | micromark | 15.4 KB | 13.7 KB | | markdown-wasm JS + WASM | 31.3 KB | 26.4 KB | diff --git a/docs/core-concepts/syntax-profile.md b/docs/core-concepts/syntax-profile.md index 2e378e5..cbb1dcf 100644 --- a/docs/core-concepts/syntax-profile.md +++ b/docs/core-concepts/syntax-profile.md @@ -35,7 +35,7 @@ TanStack Markdown implements a documented subset aimed at repository-authored bl | Reference links and images | Yes | Full, collapsed, and shortcut forms; normalized reference labels | | Hard breaks | Yes | Backslash before a newline | | Raw inline HTML | Opt-in | Requires `allowHtml: true` | -| Autolink literals | No | Write an explicit link | +| Autolink literals | Opt-in | HTTP(S) only via the [autolinks extension](../guides/extensions#optional-url-linking) | | Entity decoding | Partial | HTML is escaped; full CommonMark entity behavior is not a goal | ## Docs metadata @@ -69,7 +69,7 @@ The following are not project goals: - complete CommonMark or GFM conformance - MDX, JSX parsing, or arbitrary code evaluation -- automatic URL linking +- automatic URL linking by default or full GFM autolink conformance - a complete HTML parser or sanitizer - every delimiter, indentation, entity, or reference-label interaction - syntax highlighting, themes, or language grammars diff --git a/docs/guides/extensions.md b/docs/guides/extensions.md index fc00090..1886885 100644 --- a/docs/guides/extensions.md +++ b/docs/guides/extensions.md @@ -40,9 +40,68 @@ The context includes: Nested parsing shares the parser depth budget and heading slugger. +## Inline source parsing + +Use `inlineParser` when syntax must see source characters before they become emphasis, links, or decoded escapes. `transformInline` cannot recover an escaped opener or the original spelling of an already-parsed node. + +```ts +import type { MarkdownExtension } from '@tanstack/markdown' + +const issueReferences: MarkdownExtension = { + name: 'issue-references', + inlineParser: { + markers: '#', + parse({ source, index, inLink }) { + if (inLink) return undefined + const match = /^#([0-9]+)\b/.exec(source.slice(index)) + if (!match) return undefined + return { + length: match[0].length, + node: { + type: 'link', + href: `/issues/${match[1]}`, + children: [{ type: 'text', value: match[0] }], + }, + } + }, + }, +} +``` + +`markers` lists literal possible first characters, not a regular expression. The parser skips ordinary text to the next built-in or extension marker. At a matching position, extensions run in array order after built-in escapes and code spans, before the other built-in inline rules. The first returned result owns that range. Return `undefined` to let the next extension or built-in rule handle it. + +The context provides `source`, `index`, `options`, `inLink`, and a nested `parseInline(value)` helper. Source and UTF-16 indices refer to the current inline container, not offsets in the original document. Hooks also run within emphasis and explicit link labels; `inLink` remains true through their nested content. They do not run inside code, image alt text, or link destinations. A hook cannot consume across an enclosing inline or block boundary. + +Return one standard `InlineNode` and a positive integer `length` within the remaining source. Invalid lengths throw `RangeError`. The child parser shares the existing depth and scan limits; use that helper instead of calling the top-level parser recursively. Hook dispatch counts against the scan budget. Extension code remains trusted: these limits do not bound work done inside a callback. Keep recognition deterministic and avoid repeatedly scanning a suffix after unsuccessful matches. + +Returned nodes follow the same trust contract as supplied ASTs: URL destinations and component names/properties must be validated by the extension. The fixed, numeric issue path above needs no user-supplied URL. Extensions accepting arbitrary URLs should use the application URL policy. Return a portable `InlineComponentNode` for custom presentation; no HTML renderer changes are required. + +## Optional URL linking + +```ts +import { renderHtml } from '@tanstack/markdown/html' +import { autolinksExtension } from '@tanstack/markdown/extensions/autolinks' + +const html = renderHtml('See https://example.com/~alice~/notes.', { + extensions: [autolinksExtension()], +}) +``` + +This extension recognizes bare HTTP(S) URLs and `` / `` notation, without changing the core or docs-preset defaults. URLs retain their original source spelling, including Markdown punctuation. Host/port validity follows the platform `URL` implementation. Link nodes pass through the existing `urlTransform(url, 'link', defaultUrl)` policy and render consistently in HTML, React, and Octane; `null` keeps only the URL label. Application replacements remain trusted, as with explicit links. + +The bounded profile is intentionally smaller than GFM autolink literals: + +- Bare links must start at the container boundary or after punctuation/whitespace, excluding letters, numbers, `_`, `/`, `@`, `<`, and backslash. They stop at whitespace, controls, quotes, backticks, backslashes, angle brackets, or an unmatched closing parenthesis/bracket/brace. +- Balanced parentheses, brackets, and braces stay in bare URLs. Trailing `. , ! ? ; :` characters stay outside the link. Use angle notation or an explicit Markdown link when those trailing characters belong to the URL. +- Angle notation preserves trailing punctuation and requires a closing `>` before whitespace, controls, quotes, backticks, backslashes, or another `<`. Escaping the opening `<` keeps it literal. +- Explicit links (including their formatted labels), images, code, destinations, and enabled raw HTML are handled by the existing parser. Autolinks also work in ordinary emphasis, headings, lists, quotes, and table cells, within the enclosing inline boundaries. +- `www.` addresses, email detection, other schemes, entity decoding, and full CommonMark/GFM autolink conformance are not included. Malformed HTTP(S) candidates are consumed as literal text to avoid rescanning their suffixes. + +This uses the same public `inlineParser` contract as third-party extensions and adds no runtime dependency. Import it only where URL linking is wanted. + ## Inline transformation -`transformInline` receives built-in inline nodes after parsing. Return the replacement array. Keep transforms deterministic and avoid repeated full-array scans for every node. +`transformInline` receives built-in and extension inline nodes after parsing. Return the replacement array. Keep transforms deterministic and avoid repeated full-array scans for every node. The hook runs once per inline container. Recurse through inline `children` when your transform also needs to handle content inside emphasis or links. Code spans and image alt text are not separate inline containers. @@ -93,4 +152,4 @@ Document transforms are already represented in a pre-parsed AST. HTML render hoo ## Admission rule -An extension is appropriate when syntax is broadly useful to docs, has a deterministic block boundary, and does not justify cost in the core entry. Use a larger processing ecosystem when the job requires async plugins, arbitrary tree pipelines, compiler integration, or MDX evaluation. +An extension is appropriate when syntax is broadly useful to docs, has a deterministic source boundary, and does not justify cost in the core entry. Use a larger processing ecosystem when the job requires async plugins, arbitrary tree pipelines, compiler integration, or MDX evaluation. diff --git a/docs/guides/performance.md b/docs/guides/performance.md index 070f162..5820a1e 100644 --- a/docs/guides/performance.md +++ b/docs/guides/performance.md @@ -12,10 +12,10 @@ The generated browser bundle report records: | Entry | Gzip | Brotli | | --- | ---: | ---: | -| parser | 5.0 KB | 4.6 KB | -| HTML renderer | 6.8 KB | 6.2 KB | -| React adapter | 6.7 KB | 6.2 KB | -| Octane adapter | 6.7 KB | 6.2 KB | +| parser | 5.3 KB | 4.9 KB | +| HTML renderer | 7.1 KB | 6.5 KB | +| React adapter | 7.0 KB | 6.5 KB | +| Octane adapter | 7.0 KB | 6.5 KB | | React adapter with streaming extension | 6.9 KB | 6.4 KB | | Streaming extension | 0.3 KB | 0.3 KB | | docs preset | 2.3 KB | 2.1 KB | diff --git a/docs/overview.md b/docs/overview.md index f814fbb..025d606 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -25,13 +25,13 @@ General Markdown processors optimize for broad conformance, plugin ecosystems, o - code metadata for documentation UI - a small browser bundle -TanStack Markdown spends its complexity budget on that path. It deliberately does not implement every CommonMark edge case, MDX evaluation, automatic linkification, or a general asynchronous processing ecosystem. +TanStack Markdown spends its complexity budget on that path. It deliberately does not implement every CommonMark edge case, MDX evaluation, automatic linkification by default, or a general asynchronous processing ecosystem. ## Core properties ### Small entry points -Current minified browser bundles are 5.0 KB gzip for the parser, 6.8 KB for HTML rendering, and 6.7 KB for either UI adapter with its framework runtime externalized. The generated [bundle report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) is the source of truth. +Current minified browser bundles are 5.3 KB gzip for the parser, 7.1 KB for HTML rendering, and 7.0 KB for either UI adapter with its framework runtime externalized. The generated [bundle report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) is the source of truth. ### Parse once, render many diff --git a/docs/reference/extensions.md b/docs/reference/extensions.md index 3215d15..43ceaf2 100644 --- a/docs/reference/extensions.md +++ b/docs/reference/extensions.md @@ -6,6 +6,18 @@ title: Extensions Every built-in extension is available through a separate package entry. +## Autolinks + +Import from `@tanstack/markdown/extensions/autolinks`. + +### `autolinksExtension` + +```ts +function autolinksExtension(): MarkdownExtension +``` + +Creates an inline source parser for bare `http://` / `https://` URLs and explicit `` links. Returns ordinary link nodes and applies the active `urlTransform` policy. It is not included in the core entry or docs preset. See the [autolink profile](../guides/extensions#optional-url-linking) for boundaries and deliberate limits. + ## Callouts Import from `@tanstack/markdown/extensions/callouts`. diff --git a/docs/reference/index.md b/docs/reference/index.md index e56eb47..043dcc8 100644 --- a/docs/reference/index.md +++ b/docs/reference/index.md @@ -13,6 +13,7 @@ TanStack Markdown uses explicit subpath exports so applications can import only | `@tanstack/markdown/html` | HTML rendering functions | | `@tanstack/markdown/react` | React component and React rendering functions | | `@tanstack/markdown/octane` | Octane component and descriptor rendering functions | +| `@tanstack/markdown/extensions/autolinks` | optional HTTP(S) URL linking | | `@tanstack/markdown/extensions/callouts` | callout block parser | | `@tanstack/markdown/extensions/comment-components` | comment-delimited component parser | | `@tanstack/markdown/extensions/docs` | composed docs extension preset | diff --git a/docs/reference/types.md b/docs/reference/types.md index fd7a737..b8bebbb 100644 --- a/docs/reference/types.md +++ b/docs/reference/types.md @@ -158,12 +158,24 @@ Configures anchor `content`, `className`, `ariaHidden`, and `tabIndex`. ### `MarkdownExtension` -Named hook object with optional `parseBlock`, `transformDocument`, `transformInline`, and `renderHtml` functions. +Named hook object with optional `parseBlock`, `transformDocument`, `transformInline`, and `renderHtml` functions, plus an optional `inlineParser`. ### `BlockParseContext` Provides source `lines`, current `index`, active `options`, nested `parseInline` and `parseBlocks` helpers, and `consume`. +### `InlineParser` + +Contains literal first-character `markers` and a synchronous `parse(context)` callback returning `InlineParseResult | undefined`. Runs at matching source positions after escapes and code spans, before other built-in inline rules. + +### `InlineParseContext` + +Provides current-container `source`, UTF-16 `index`, active `options`, `inLink`, and a nested `parseInline(value)` helper that shares the recursion and scan budget. + +### `InlineParseResult` + +Contains one standard `InlineNode` as `node` and a positive integer `length` in UTF-16 code units. The range must fit in the current source. Invalid lengths throw `RangeError`. + ### `InlineTransformContext` Provides active parse `options` to inline transforms. diff --git a/package.json b/package.json index b215175..7af96ca 100644 --- a/package.json +++ b/package.json @@ -69,6 +69,10 @@ "./extensions/tabs": { "types": "./dist/extensions/tabs.d.ts", "import": "./dist/extensions/tabs.js" + }, + "./extensions/autolinks": { + "types": "./dist/extensions/autolinks.d.ts", + "import": "./dist/extensions/autolinks.js" } }, "scripts": { diff --git a/reports/autolinks.md b/reports/autolinks.md new file mode 100644 index 0000000..5b1d6c9 --- /dev/null +++ b/reports/autolinks.md @@ -0,0 +1,28 @@ +# Optional HTTP(S) autolinks + +This proposal depends on the inline source parser API. It keeps automatic URL linking opt-in, addressing the core profile’s existing non-goal without changing default rendering. It intentionally does not promise full GFM autolink behavior. + +The separate public extension bundle is 994 bytes minified, 632 bytes gzip, and 568 bytes Brotli. All 22 existing measured import shapes are byte-for-byte unchanged in size relative to the inline-parser commit. No runtime dependency or renderer-specific implementation is added. + +The revision comparison retains all 403 established CommonMark matches with the extension disabled. The 45 extension tests cover URL punctuation, delimiters, source spelling, code/link exclusions, URL policy, malformed input, serialization, and HTML/React/Octane rendering. + +These synthetic measurements use `v26.7.0`, seven alternating warmed rounds, and median milliseconds per render. Enabled and disabled profiles perform different work and can produce different HTML; this is an overhead measurement, not an equivalent-feature renderer comparison. Browser performance is unmeasured. + +| Fixture | Core (ms) | Autolinks enabled (ms) | +| --- | ---: | ---: | +| plain comment | 0.00094 | 0.00305 | +| URL comment | 0.00211 | 0.00394 | +| punctuation | 0.00124 | 0.00319 | +| malformed 9000 chars | 0.00336 | 0.13189 | +| malformed 18000 chars | 0.00585 | 0.27066 | +| malformed 36000 chars | 0.01047 | 0.49776 | + +The repeated malformed-prefix cases exercise increasing input sizes; the implementation consumes a failed HTTP(S) candidate as one text range instead of retrying every prefix in its suffix. These observations cover this fixture, not arbitrary extension code. + +Reproduce: + +```sh +pnpm exec tsx scripts/compare-revision.mjs b70affe --no-bench +pnpm exec tsx scripts/bench-autolinks.ts +pnpm run verify +``` diff --git a/reports/benchmarks.json b/reports/benchmarks.json index 1ba7450..953f457 100644 --- a/reports/benchmarks.json +++ b/reports/benchmarks.json @@ -1,5 +1,5 @@ { - "generatedAt": "2026-09-11T20:23:30.357Z", + "generatedAt": "2026-09-12T16:34:43.746Z", "sink": 112892330, "results": [ { @@ -8,9 +8,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.01844806249999999, + "msPerOp": 0.022017473499999995, "outputBytes": 15, - "heapDeltaKb": 898.9609375 + "heapDeltaKb": 977.6328125 }, { "group": "markdown", @@ -18,9 +18,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.005263854499999994, + "msPerOp": 0.004601603999999994, "outputBytes": 1399, - "heapDeltaKb": 1238.703125 + "heapDeltaKb": 2604.671875 }, { "group": "markdown", @@ -28,9 +28,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.00399779199999999, + "msPerOp": 0.003958596499999999, "outputBytes": 1143, - "heapDeltaKb": -3027.4296875 + "heapDeltaKb": -1731.7109375 }, { "group": "markdown", @@ -38,9 +38,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.023797770500000013, + "msPerOp": 0.02425558550000001, "outputBytes": 1399, - "heapDeltaKb": -2596.171875 + "heapDeltaKb": -3691.7734375 }, { "group": "markdown", @@ -48,9 +48,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.023606604000000003, + "msPerOp": 0.022545705999999995, "outputBytes": 1143, - "heapDeltaKb": -1860.5859375 + "heapDeltaKb": 1486.6484375 }, { "group": "markdown", @@ -58,9 +58,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.02952339599999999, + "msPerOp": 0.028566675500000003, "outputBytes": 1035, - "heapDeltaKb": -853.609375 + "heapDeltaKb": 711.0078125 }, { "group": "markdown", @@ -68,9 +68,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.019339646000000016, + "msPerOp": 0.01768149349999999, "outputBytes": 1087, - "heapDeltaKb": 4088.890625 + "heapDeltaKb": -4008.578125 }, { "group": "markdown", @@ -78,9 +78,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.17314687500000003, + "msPerOp": 0.17639268349999998, "outputBytes": 825, - "heapDeltaKb": -8298.203125 + "heapDeltaKb": 8797.984375 }, { "group": "markdown", @@ -88,9 +88,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.021329833500000006, + "msPerOp": 0.013538402500000019, "outputBytes": 825, - "heapDeltaKb": 54.8671875 + "heapDeltaKb": 409.8125 }, { "group": "markdown", @@ -98,9 +98,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.010540166499999998, + "msPerOp": 0.007687698000000012, "outputBytes": 1117, - "heapDeltaKb": 4372.3984375 + "heapDeltaKb": 4377.6796875 }, { "group": "markdown", @@ -108,9 +108,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 2000, - "msPerOp": 0.19910575, + "msPerOp": 0.22415825150000002, "outputBytes": 824, - "heapDeltaKb": 13756 + "heapDeltaKb": 9968.046875 }, { "group": "markdown", @@ -118,9 +118,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.006807167000000163, + "msPerOp": 0.008376180999999861, "outputBytes": 15, - "heapDeltaKb": -6378.1484375 + "heapDeltaKb": -5036.9296875 }, { "group": "markdown", @@ -128,9 +128,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.007484500000000026, + "msPerOp": 0.008498019999999997, "outputBytes": 4596, - "heapDeltaKb": -1122.5703125 + "heapDeltaKb": -772.2265625 }, { "group": "markdown", @@ -138,9 +138,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.003155708999999888, + "msPerOp": 0.0033726779999999506, "outputBytes": 1927, - "heapDeltaKb": 10476.2890625 + "heapDeltaKb": 10347.921875 }, { "group": "markdown", @@ -148,9 +148,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.014249915999999984, + "msPerOp": 0.015233486999999967, "outputBytes": 4596, - "heapDeltaKb": -6339.6796875 + "heapDeltaKb": -7885.09375 }, { "group": "markdown", @@ -158,9 +158,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.009854332999999996, + "msPerOp": 0.01033086400000002, "outputBytes": 1927, - "heapDeltaKb": 4603.2109375 + "heapDeltaKb": 3101.0859375 }, { "group": "markdown", @@ -168,9 +168,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.006169208000000026, + "msPerOp": 0.007242064999999911, "outputBytes": 1330, - "heapDeltaKb": -19757.0859375 + "heapDeltaKb": -19906.671875 }, { "group": "markdown", @@ -178,9 +178,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.008338584000000083, + "msPerOp": 0.008980608999999959, "outputBytes": 1330, - "heapDeltaKb": -5321.0390625 + "heapDeltaKb": -5375.328125 }, { "group": "markdown", @@ -188,9 +188,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.13680366700000013, + "msPerOp": 0.16169720499999993, "outputBytes": 1330, - "heapDeltaKb": -11833.328125 + "heapDeltaKb": -11467.953125 }, { "group": "markdown", @@ -198,9 +198,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.009240166999999928, + "msPerOp": 0.00974678599999993, "outputBytes": 1330, - "heapDeltaKb": 24577.0234375 + "heapDeltaKb": 24610.8203125 }, { "group": "markdown", @@ -208,9 +208,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.004289874999999938, + "msPerOp": 0.004391055000000052, "outputBytes": 1512, - "heapDeltaKb": 2559.9921875 + "heapDeltaKb": 2559.9296875 }, { "group": "markdown", @@ -218,9 +218,9 @@ "fixture": "code-heavy.md", "bytes": 1011, "iterations": 1000, - "msPerOp": 0.15214312500000005, + "msPerOp": 0.17077694300000007, "outputBytes": 1200, - "heapDeltaKb": -7037.953125 + "heapDeltaKb": -6403.96875 }, { "group": "markdown", @@ -228,9 +228,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.0027057704999999713, + "msPerOp": 0.0032395934999999553, "outputBytes": 15, - "heapDeltaKb": 18580.609375 + "heapDeltaKb": 20140.4140625 }, { "group": "markdown", @@ -238,9 +238,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.0016762294999999768, + "msPerOp": 0.0019173435000000153, "outputBytes": 1067, - "heapDeltaKb": -17067.3203125 + "heapDeltaKb": -17021.609375 }, { "group": "markdown", @@ -248,9 +248,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.0007344790000000785, + "msPerOp": 0.0008988014999999905, "outputBytes": 361, - "heapDeltaKb": 5314.8828125 + "heapDeltaKb": 5298.171875 }, { "group": "markdown", @@ -258,9 +258,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.004606062500000007, + "msPerOp": 0.004847802999999999, "outputBytes": 1067, - "heapDeltaKb": 1452.21875 + "heapDeltaKb": 2224.421875 }, { "group": "markdown", @@ -268,9 +268,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.0036469999999999346, + "msPerOp": 0.0038574614999999993, "outputBytes": 361, - "heapDeltaKb": -8892.2109375 + "heapDeltaKb": -8430.7890625 }, { "group": "markdown", @@ -278,9 +278,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.005531416499999977, + "msPerOp": 0.005880379500000004, "outputBytes": 350, - "heapDeltaKb": 16140.453125 + "heapDeltaKb": 16325.5 }, { "group": "markdown", @@ -288,9 +288,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.005058562499999994, + "msPerOp": 0.005102377000000047, "outputBytes": 300, - "heapDeltaKb": -7253.6015625 + "heapDeltaKb": -7808.4375 }, { "group": "markdown", @@ -298,9 +298,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.05207070850000002, + "msPerOp": 0.055343931500000054, "outputBytes": 300, - "heapDeltaKb": 7694.546875 + "heapDeltaKb": 7211.1796875 }, { "group": "markdown", @@ -308,9 +308,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.0038984790000000658, + "msPerOp": 0.00415123600000004, "outputBytes": 300, - "heapDeltaKb": -9655.921875 + "heapDeltaKb": -9635.296875 }, { "group": "markdown", @@ -318,9 +318,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.0018649789999999485, + "msPerOp": 0.0021862974999999098, "outputBytes": 408, - "heapDeltaKb": 2938.3984375 + "heapDeltaKb": 2951.2109375 }, { "group": "markdown", @@ -328,9 +328,9 @@ "fixture": "malformed.md", "bytes": 237, "iterations": 2000, - "msPerOp": 0.06917156250000005, + "msPerOp": 0.06116631100000006, "outputBytes": 297, - "heapDeltaKb": 2219.1953125 + "heapDeltaKb": 1570.0703125 }, { "group": "markdown", @@ -338,9 +338,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.017162290999999868, + "msPerOp": 0.0195029320000001, "outputBytes": 15, - "heapDeltaKb": -17554.0625 + "heapDeltaKb": -15331.234375 }, { "group": "markdown", @@ -348,9 +348,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.002673625000000129, + "msPerOp": 0.002325791999999865, "outputBytes": 1903, - "heapDeltaKb": 10295.9765625 + "heapDeltaKb": 10463.9375 }, { "group": "markdown", @@ -358,9 +358,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.002687332999999853, + "msPerOp": 0.0024216610000000857, "outputBytes": 1903, - "heapDeltaKb": 10321.3203125 + "heapDeltaKb": -22236.140625 }, { "group": "markdown", @@ -368,9 +368,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.017694124999999984, + "msPerOp": 0.02115736700000002, "outputBytes": 1903, - "heapDeltaKb": -7354.9765625 + "heapDeltaKb": 26786.2734375 }, { "group": "markdown", @@ -378,9 +378,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.016582792000000155, + "msPerOp": 0.02099837699999989, "outputBytes": 1903, - "heapDeltaKb": -7244.9453125 + "heapDeltaKb": -5999.953125 }, { "group": "markdown", @@ -388,9 +388,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.03447937500000012, + "msPerOp": 0.03640275299999985, "outputBytes": 1860, - "heapDeltaKb": 6729.9375 + "heapDeltaKb": -25857.6796875 }, { "group": "markdown", @@ -398,9 +398,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.01885091699999998, + "msPerOp": 0.017336059000000206, "outputBytes": 1860, - "heapDeltaKb": -18343.15625 + "heapDeltaKb": 14484.9921875 }, { "group": "markdown", @@ -408,9 +408,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.24291566599999986, + "msPerOp": 0.2602433900000001, "outputBytes": 1862, - "heapDeltaKb": 24865.828125 + "heapDeltaKb": 6955.8203125 }, { "group": "markdown", @@ -418,9 +418,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.011920082999999977, + "msPerOp": 0.013677292000000308, "outputBytes": 1862, - "heapDeltaKb": 46458.671875 + "heapDeltaKb": 15720.5859375 }, { "group": "markdown", @@ -428,9 +428,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.006297250000000076, + "msPerOp": 0.0061069580000003044, "outputBytes": 2340, - "heapDeltaKb": 3368.3671875 + "heapDeltaKb": 3368.5703125 }, { "group": "markdown", @@ -438,9 +438,9 @@ "fixture": "prose-heavy.md", "bytes": 1700, "iterations": 1000, - "msPerOp": 0.28966795899999986, + "msPerOp": 0.2977656790000001, "outputBytes": 1859, - "heapDeltaKb": 50897.1328125 + "heapDeltaKb": 51638.1171875 }, { "group": "markdown", @@ -448,9 +448,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.0093737500000002, + "msPerOp": 0.012297751999999947, "outputBytes": 15, - "heapDeltaKb": 7577.4453125 + "heapDeltaKb": 9680.671875 }, { "group": "markdown", @@ -458,9 +458,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.003080417000000125, + "msPerOp": 0.003950981000000183, "outputBytes": 1370, - "heapDeltaKb": 27814.0859375 + "heapDeltaKb": -4751.8828125 }, { "group": "markdown", @@ -468,9 +468,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.0026590625000001184, + "msPerOp": 0.0029309299999999896, "outputBytes": 1044, - "heapDeltaKb": -42766.875 + "heapDeltaKb": -10698.0078125 }, { "group": "markdown", @@ -478,9 +478,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.012628958000000011, + "msPerOp": 0.015319171500000039, "outputBytes": 1370, - "heapDeltaKb": 31414.875 + "heapDeltaKb": 142.5625 }, { "group": "markdown", @@ -488,9 +488,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.012084312499999895, + "msPerOp": 0.014412205000000086, "outputBytes": 1044, - "heapDeltaKb": -40133.328125 + "heapDeltaKb": -5808.3125 }, { "group": "markdown", @@ -498,9 +498,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.012065562499999943, + "msPerOp": 0.014474229499999864, "outputBytes": 724, - "heapDeltaKb": 50632.2109375 + "heapDeltaKb": 17843.921875 }, { "group": "markdown", @@ -508,9 +508,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.00776693750000004, + "msPerOp": 0.008581575000000157, "outputBytes": 776, - "heapDeltaKb": 3160.15625 + "heapDeltaKb": -29482.9921875 }, { "group": "markdown", @@ -518,9 +518,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.09741200000000004, + "msPerOp": 0.11313335449999999, "outputBytes": 535, - "heapDeltaKb": 44970.375 + "heapDeltaKb": 9311.3671875 }, { "group": "markdown", @@ -528,9 +528,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.007062458500000048, + "msPerOp": 0.007051335000000108, "outputBytes": 535, - "heapDeltaKb": -15145.1796875 + "heapDeltaKb": -15395.9140625 }, { "group": "markdown", @@ -538,9 +538,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.003928562499999998, + "msPerOp": 0.0036258374999999887, "outputBytes": 854, - "heapDeltaKb": 3580.1796875 + "heapDeltaKb": 3820.015625 }, { "group": "markdown", @@ -548,9 +548,9 @@ "fixture": "small-doc.md", "bytes": 432, "iterations": 2000, - "msPerOp": 0.129259542, + "msPerOp": 0.13716429049999987, "outputBytes": 534, - "heapDeltaKb": 4179.84375 + "heapDeltaKb": 2126.9765625 }, { "group": "markdown", @@ -558,9 +558,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.0232009579999999, + "msPerOp": 0.0305964684999999, "outputBytes": 15, - "heapDeltaKb": -43744.359375 + "heapDeltaKb": -1160.2734375 }, { "group": "markdown", @@ -568,9 +568,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.004354895999999826, + "msPerOp": 0.004948552500000005, "outputBytes": 1315, - "heapDeltaKb": 42277.9921875 + "heapDeltaKb": 10472.2109375 }, { "group": "markdown", @@ -578,9 +578,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.00404427099999998, + "msPerOp": 0.004566613999999845, "outputBytes": 1315, - "heapDeltaKb": -23161.15625 + "heapDeltaKb": -22139.1796875 }, { "group": "markdown", @@ -588,9 +588,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.02736354150000011, + "msPerOp": 0.03415823050000017, "outputBytes": 1315, - "heapDeltaKb": -1853.140625 + "heapDeltaKb": 7724.90625 }, { "group": "markdown", @@ -598,9 +598,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.027197875000000066, + "msPerOp": 0.033351428499999884, "outputBytes": 1315, - "heapDeltaKb": -2112.7109375 + "heapDeltaKb": 5238.578125 }, { "group": "markdown", @@ -608,9 +608,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.029182874999999966, + "msPerOp": 0.032062967999999956, "outputBytes": 1102, - "heapDeltaKb": -14329.2421875 + "heapDeltaKb": -13272.3828125 }, { "group": "markdown", @@ -618,9 +618,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.017972145500000123, + "msPerOp": 0.01842763049999985, "outputBytes": 1325, - "heapDeltaKb": 26395.5078125 + "heapDeltaKb": -5843.8046875 }, { "group": "markdown", @@ -628,9 +628,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.16772906250000005, + "msPerOp": 0.19467189949999988, "outputBytes": 627, - "heapDeltaKb": -39153.3359375 + "heapDeltaKb": -12006.1875 }, { "group": "markdown", @@ -638,9 +638,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.011388625000000048, + "msPerOp": 0.013239073499999905, "outputBytes": 627, - "heapDeltaKb": 28363.8125 + "heapDeltaKb": -4626.6484375 }, { "group": "markdown", @@ -648,9 +648,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.005910791999999901, + "msPerOp": 0.00600950899999998, "outputBytes": 1202, - "heapDeltaKb": 4506.171875 + "heapDeltaKb": 4504.3828125 }, { "group": "markdown", @@ -658,9 +658,9 @@ "fixture": "tables-lists.md", "bytes": 454, "iterations": 2000, - "msPerOp": 0.1964307080000001, + "msPerOp": 0.24243692200000033, "outputBytes": 622, - "heapDeltaKb": 7601.8671875 + "heapDeltaKb": 5114.359375 }, { "group": "streaming", @@ -668,9 +668,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 250, - "msPerOp": 0.22356500000000232, + "msPerOp": 0.284965400000001, "outputBytes": 1112, - "heapDeltaKb": -8191.453125 + "heapDeltaKb": -25768.0625 }, { "group": "streaming", @@ -678,9 +678,9 @@ "fixture": "ai-response.md", "bytes": 652, "iterations": 250, - "msPerOp": 0.2804184999999998, + "msPerOp": 0.31935131999999794, "outputBytes": 1035, - "heapDeltaKb": 3232.0078125 + "heapDeltaKb": 3579.3984375 } ] } \ No newline at end of file diff --git a/reports/benchmarks.md b/reports/benchmarks.md index adb64ed..7f51013 100644 --- a/reports/benchmarks.md +++ b/reports/benchmarks.md @@ -1,6 +1,6 @@ # Benchmark Results -Generated: 2026-09-11T20:23:30.359Z +Generated: 2026-09-12T16:34:43.747Z Lower `ms/op` is better. Benchmarks run in Node with production package builds where available; heap delta is a coarse process-level signal, not an allocation profiler. Streaming rows replay the complete response in 32-character chunks, so one operation is one progressive response. @@ -8,94 +8,94 @@ Lower `ms/op` is better. Benchmarks run in Node with production package builds w | Name | Fixture | Bytes | Iterations | ms/op | Output bytes | Heap delta KB | | :--- | :--- | ---: | ---: | ---: | ---: | ---: | -| @tanstack/markdown parse | ai-response.md | 652 | 2000 | 0.0184 | 15 | 899.0 | -| @tanstack/markdown render AST with external highlighter | ai-response.md | 652 | 2000 | 0.0053 | 1399 | 1238.7 | -| @tanstack/markdown render AST | ai-response.md | 652 | 2000 | 0.0040 | 1143 | -3027.4 | -| @tanstack/markdown parse+render with external highlighter | ai-response.md | 652 | 2000 | 0.0238 | 1399 | -2596.2 | -| @tanstack/markdown parse+render | ai-response.md | 652 | 2000 | 0.0236 | 1143 | -1860.6 | -| marked parse+render | ai-response.md | 652 | 2000 | 0.0295 | 1035 | -853.6 | -| markdown-it parse+render | ai-response.md | 652 | 2000 | 0.0193 | 1087 | 4088.9 | -| micromark render | ai-response.md | 652 | 2000 | 0.1731 | 825 | -8298.2 | -| commonmark parse+render | ai-response.md | 652 | 2000 | 0.0213 | 825 | 54.9 | -| markdown-wasm render | ai-response.md | 652 | 2000 | 0.0105 | 1117 | 4372.4 | -| unified remark+rehype render | ai-response.md | 652 | 2000 | 0.1991 | 824 | 13756.0 | -| @tanstack/markdown parse | code-heavy.md | 1011 | 1000 | 0.0068 | 15 | -6378.1 | -| @tanstack/markdown render AST with external highlighter | code-heavy.md | 1011 | 1000 | 0.0075 | 4596 | -1122.6 | -| @tanstack/markdown render AST | code-heavy.md | 1011 | 1000 | 0.0032 | 1927 | 10476.3 | -| @tanstack/markdown parse+render with external highlighter | code-heavy.md | 1011 | 1000 | 0.0142 | 4596 | -6339.7 | -| @tanstack/markdown parse+render | code-heavy.md | 1011 | 1000 | 0.0099 | 1927 | 4603.2 | -| marked parse+render | code-heavy.md | 1011 | 1000 | 0.0062 | 1330 | -19757.1 | -| markdown-it parse+render | code-heavy.md | 1011 | 1000 | 0.0083 | 1330 | -5321.0 | -| micromark render | code-heavy.md | 1011 | 1000 | 0.1368 | 1330 | -11833.3 | -| commonmark parse+render | code-heavy.md | 1011 | 1000 | 0.0092 | 1330 | 24577.0 | -| markdown-wasm render | code-heavy.md | 1011 | 1000 | 0.0043 | 1512 | 2560.0 | -| unified remark+rehype render | code-heavy.md | 1011 | 1000 | 0.1521 | 1200 | -7038.0 | -| @tanstack/markdown parse | malformed.md | 237 | 2000 | 0.0027 | 15 | 18580.6 | -| @tanstack/markdown render AST with external highlighter | malformed.md | 237 | 2000 | 0.0017 | 1067 | -17067.3 | -| @tanstack/markdown render AST | malformed.md | 237 | 2000 | 0.0007 | 361 | 5314.9 | -| @tanstack/markdown parse+render with external highlighter | malformed.md | 237 | 2000 | 0.0046 | 1067 | 1452.2 | -| @tanstack/markdown parse+render | malformed.md | 237 | 2000 | 0.0036 | 361 | -8892.2 | -| marked parse+render | malformed.md | 237 | 2000 | 0.0055 | 350 | 16140.5 | -| markdown-it parse+render | malformed.md | 237 | 2000 | 0.0051 | 300 | -7253.6 | -| micromark render | malformed.md | 237 | 2000 | 0.0521 | 300 | 7694.5 | -| commonmark parse+render | malformed.md | 237 | 2000 | 0.0039 | 300 | -9655.9 | -| markdown-wasm render | malformed.md | 237 | 2000 | 0.0019 | 408 | 2938.4 | -| unified remark+rehype render | malformed.md | 237 | 2000 | 0.0692 | 297 | 2219.2 | -| @tanstack/markdown parse | prose-heavy.md | 1700 | 1000 | 0.0172 | 15 | -17554.1 | -| @tanstack/markdown render AST with external highlighter | prose-heavy.md | 1700 | 1000 | 0.0027 | 1903 | 10296.0 | -| @tanstack/markdown render AST | prose-heavy.md | 1700 | 1000 | 0.0027 | 1903 | 10321.3 | -| @tanstack/markdown parse+render with external highlighter | prose-heavy.md | 1700 | 1000 | 0.0177 | 1903 | -7355.0 | -| @tanstack/markdown parse+render | prose-heavy.md | 1700 | 1000 | 0.0166 | 1903 | -7244.9 | -| marked parse+render | prose-heavy.md | 1700 | 1000 | 0.0345 | 1860 | 6729.9 | -| markdown-it parse+render | prose-heavy.md | 1700 | 1000 | 0.0189 | 1860 | -18343.2 | -| micromark render | prose-heavy.md | 1700 | 1000 | 0.2429 | 1862 | 24865.8 | -| commonmark parse+render | prose-heavy.md | 1700 | 1000 | 0.0119 | 1862 | 46458.7 | -| markdown-wasm render | prose-heavy.md | 1700 | 1000 | 0.0063 | 2340 | 3368.4 | -| unified remark+rehype render | prose-heavy.md | 1700 | 1000 | 0.2897 | 1859 | 50897.1 | -| @tanstack/markdown parse | small-doc.md | 432 | 2000 | 0.0094 | 15 | 7577.4 | -| @tanstack/markdown render AST with external highlighter | small-doc.md | 432 | 2000 | 0.0031 | 1370 | 27814.1 | -| @tanstack/markdown render AST | small-doc.md | 432 | 2000 | 0.0027 | 1044 | -42766.9 | -| @tanstack/markdown parse+render with external highlighter | small-doc.md | 432 | 2000 | 0.0126 | 1370 | 31414.9 | -| @tanstack/markdown parse+render | small-doc.md | 432 | 2000 | 0.0121 | 1044 | -40133.3 | -| marked parse+render | small-doc.md | 432 | 2000 | 0.0121 | 724 | 50632.2 | -| markdown-it parse+render | small-doc.md | 432 | 2000 | 0.0078 | 776 | 3160.2 | -| micromark render | small-doc.md | 432 | 2000 | 0.0974 | 535 | 44970.4 | -| commonmark parse+render | small-doc.md | 432 | 2000 | 0.0071 | 535 | -15145.2 | -| markdown-wasm render | small-doc.md | 432 | 2000 | 0.0039 | 854 | 3580.2 | -| unified remark+rehype render | small-doc.md | 432 | 2000 | 0.1293 | 534 | 4179.8 | -| @tanstack/markdown parse | tables-lists.md | 454 | 2000 | 0.0232 | 15 | -43744.4 | -| @tanstack/markdown render AST with external highlighter | tables-lists.md | 454 | 2000 | 0.0044 | 1315 | 42278.0 | -| @tanstack/markdown render AST | tables-lists.md | 454 | 2000 | 0.0040 | 1315 | -23161.2 | -| @tanstack/markdown parse+render with external highlighter | tables-lists.md | 454 | 2000 | 0.0274 | 1315 | -1853.1 | -| @tanstack/markdown parse+render | tables-lists.md | 454 | 2000 | 0.0272 | 1315 | -2112.7 | -| marked parse+render | tables-lists.md | 454 | 2000 | 0.0292 | 1102 | -14329.2 | -| markdown-it parse+render | tables-lists.md | 454 | 2000 | 0.0180 | 1325 | 26395.5 | -| micromark render | tables-lists.md | 454 | 2000 | 0.1677 | 627 | -39153.3 | -| commonmark parse+render | tables-lists.md | 454 | 2000 | 0.0114 | 627 | 28363.8 | -| markdown-wasm render | tables-lists.md | 454 | 2000 | 0.0059 | 1202 | 4506.2 | -| unified remark+rehype render | tables-lists.md | 454 | 2000 | 0.1964 | 622 | 7601.9 | +| @tanstack/markdown parse | ai-response.md | 652 | 2000 | 0.0220 | 15 | 977.6 | +| @tanstack/markdown render AST with external highlighter | ai-response.md | 652 | 2000 | 0.0046 | 1399 | 2604.7 | +| @tanstack/markdown render AST | ai-response.md | 652 | 2000 | 0.0040 | 1143 | -1731.7 | +| @tanstack/markdown parse+render with external highlighter | ai-response.md | 652 | 2000 | 0.0243 | 1399 | -3691.8 | +| @tanstack/markdown parse+render | ai-response.md | 652 | 2000 | 0.0225 | 1143 | 1486.6 | +| marked parse+render | ai-response.md | 652 | 2000 | 0.0286 | 1035 | 711.0 | +| markdown-it parse+render | ai-response.md | 652 | 2000 | 0.0177 | 1087 | -4008.6 | +| micromark render | ai-response.md | 652 | 2000 | 0.1764 | 825 | 8798.0 | +| commonmark parse+render | ai-response.md | 652 | 2000 | 0.0135 | 825 | 409.8 | +| markdown-wasm render | ai-response.md | 652 | 2000 | 0.0077 | 1117 | 4377.7 | +| unified remark+rehype render | ai-response.md | 652 | 2000 | 0.2242 | 824 | 9968.0 | +| @tanstack/markdown parse | code-heavy.md | 1011 | 1000 | 0.0084 | 15 | -5036.9 | +| @tanstack/markdown render AST with external highlighter | code-heavy.md | 1011 | 1000 | 0.0085 | 4596 | -772.2 | +| @tanstack/markdown render AST | code-heavy.md | 1011 | 1000 | 0.0034 | 1927 | 10347.9 | +| @tanstack/markdown parse+render with external highlighter | code-heavy.md | 1011 | 1000 | 0.0152 | 4596 | -7885.1 | +| @tanstack/markdown parse+render | code-heavy.md | 1011 | 1000 | 0.0103 | 1927 | 3101.1 | +| marked parse+render | code-heavy.md | 1011 | 1000 | 0.0072 | 1330 | -19906.7 | +| markdown-it parse+render | code-heavy.md | 1011 | 1000 | 0.0090 | 1330 | -5375.3 | +| micromark render | code-heavy.md | 1011 | 1000 | 0.1617 | 1330 | -11468.0 | +| commonmark parse+render | code-heavy.md | 1011 | 1000 | 0.0097 | 1330 | 24610.8 | +| markdown-wasm render | code-heavy.md | 1011 | 1000 | 0.0044 | 1512 | 2559.9 | +| unified remark+rehype render | code-heavy.md | 1011 | 1000 | 0.1708 | 1200 | -6404.0 | +| @tanstack/markdown parse | malformed.md | 237 | 2000 | 0.0032 | 15 | 20140.4 | +| @tanstack/markdown render AST with external highlighter | malformed.md | 237 | 2000 | 0.0019 | 1067 | -17021.6 | +| @tanstack/markdown render AST | malformed.md | 237 | 2000 | 0.0009 | 361 | 5298.2 | +| @tanstack/markdown parse+render with external highlighter | malformed.md | 237 | 2000 | 0.0048 | 1067 | 2224.4 | +| @tanstack/markdown parse+render | malformed.md | 237 | 2000 | 0.0039 | 361 | -8430.8 | +| marked parse+render | malformed.md | 237 | 2000 | 0.0059 | 350 | 16325.5 | +| markdown-it parse+render | malformed.md | 237 | 2000 | 0.0051 | 300 | -7808.4 | +| micromark render | malformed.md | 237 | 2000 | 0.0553 | 300 | 7211.2 | +| commonmark parse+render | malformed.md | 237 | 2000 | 0.0042 | 300 | -9635.3 | +| markdown-wasm render | malformed.md | 237 | 2000 | 0.0022 | 408 | 2951.2 | +| unified remark+rehype render | malformed.md | 237 | 2000 | 0.0612 | 297 | 1570.1 | +| @tanstack/markdown parse | prose-heavy.md | 1700 | 1000 | 0.0195 | 15 | -15331.2 | +| @tanstack/markdown render AST with external highlighter | prose-heavy.md | 1700 | 1000 | 0.0023 | 1903 | 10463.9 | +| @tanstack/markdown render AST | prose-heavy.md | 1700 | 1000 | 0.0024 | 1903 | -22236.1 | +| @tanstack/markdown parse+render with external highlighter | prose-heavy.md | 1700 | 1000 | 0.0212 | 1903 | 26786.3 | +| @tanstack/markdown parse+render | prose-heavy.md | 1700 | 1000 | 0.0210 | 1903 | -6000.0 | +| marked parse+render | prose-heavy.md | 1700 | 1000 | 0.0364 | 1860 | -25857.7 | +| markdown-it parse+render | prose-heavy.md | 1700 | 1000 | 0.0173 | 1860 | 14485.0 | +| micromark render | prose-heavy.md | 1700 | 1000 | 0.2602 | 1862 | 6955.8 | +| commonmark parse+render | prose-heavy.md | 1700 | 1000 | 0.0137 | 1862 | 15720.6 | +| markdown-wasm render | prose-heavy.md | 1700 | 1000 | 0.0061 | 2340 | 3368.6 | +| unified remark+rehype render | prose-heavy.md | 1700 | 1000 | 0.2978 | 1859 | 51638.1 | +| @tanstack/markdown parse | small-doc.md | 432 | 2000 | 0.0123 | 15 | 9680.7 | +| @tanstack/markdown render AST with external highlighter | small-doc.md | 432 | 2000 | 0.0040 | 1370 | -4751.9 | +| @tanstack/markdown render AST | small-doc.md | 432 | 2000 | 0.0029 | 1044 | -10698.0 | +| @tanstack/markdown parse+render with external highlighter | small-doc.md | 432 | 2000 | 0.0153 | 1370 | 142.6 | +| @tanstack/markdown parse+render | small-doc.md | 432 | 2000 | 0.0144 | 1044 | -5808.3 | +| marked parse+render | small-doc.md | 432 | 2000 | 0.0145 | 724 | 17843.9 | +| markdown-it parse+render | small-doc.md | 432 | 2000 | 0.0086 | 776 | -29483.0 | +| micromark render | small-doc.md | 432 | 2000 | 0.1131 | 535 | 9311.4 | +| commonmark parse+render | small-doc.md | 432 | 2000 | 0.0071 | 535 | -15395.9 | +| markdown-wasm render | small-doc.md | 432 | 2000 | 0.0036 | 854 | 3820.0 | +| unified remark+rehype render | small-doc.md | 432 | 2000 | 0.1372 | 534 | 2127.0 | +| @tanstack/markdown parse | tables-lists.md | 454 | 2000 | 0.0306 | 15 | -1160.3 | +| @tanstack/markdown render AST with external highlighter | tables-lists.md | 454 | 2000 | 0.0049 | 1315 | 10472.2 | +| @tanstack/markdown render AST | tables-lists.md | 454 | 2000 | 0.0046 | 1315 | -22139.2 | +| @tanstack/markdown parse+render with external highlighter | tables-lists.md | 454 | 2000 | 0.0342 | 1315 | 7724.9 | +| @tanstack/markdown parse+render | tables-lists.md | 454 | 2000 | 0.0334 | 1315 | 5238.6 | +| marked parse+render | tables-lists.md | 454 | 2000 | 0.0321 | 1102 | -13272.4 | +| markdown-it parse+render | tables-lists.md | 454 | 2000 | 0.0184 | 1325 | -5843.8 | +| micromark render | tables-lists.md | 454 | 2000 | 0.1947 | 627 | -12006.2 | +| commonmark parse+render | tables-lists.md | 454 | 2000 | 0.0132 | 627 | -4626.6 | +| markdown-wasm render | tables-lists.md | 454 | 2000 | 0.0060 | 1202 | 4504.4 | +| unified remark+rehype render | tables-lists.md | 454 | 2000 | 0.2424 | 622 | 5114.4 | ## Streaming | Name | Fixture | Bytes | Iterations | ms/op | Output bytes | Heap delta KB | | :--- | :--- | ---: | ---: | ---: | ---: | ---: | -| @tanstack/markdown streaming profile | ai-response.md | 652 | 250 | 0.2236 | 1112 | -8191.5 | -| marked progressive parse+render | ai-response.md | 652 | 250 | 0.2804 | 1035 | 3232.0 | +| @tanstack/markdown streaming profile | ai-response.md | 652 | 250 | 0.2850 | 1112 | -25768.1 | +| marked progressive parse+render | ai-response.md | 652 | 250 | 0.3194 | 1035 | 3579.4 | ## Averages | Group | Name | Mean ms/op | | :--- | :--- | ---: | -| markdown | @tanstack/markdown parse | 0.0129 | -| markdown | @tanstack/markdown render AST with external highlighter | 0.0041 | -| markdown | @tanstack/markdown render AST | 0.0029 | -| markdown | @tanstack/markdown parse+render with external highlighter | 0.0167 | -| markdown | @tanstack/markdown parse+render | 0.0155 | -| markdown | marked parse+render | 0.0195 | -| markdown | markdown-it parse+render | 0.0129 | -| markdown | micromark render | 0.1450 | -| markdown | commonmark parse+render | 0.0108 | -| markdown | markdown-wasm render | 0.0055 | -| markdown | unified remark+rehype render | 0.1726 | -| streaming | @tanstack/markdown streaming profile | 0.2236 | -| streaming | marked progressive parse+render | 0.2804 | +| markdown | @tanstack/markdown parse | 0.0160 | +| markdown | @tanstack/markdown render AST with external highlighter | 0.0044 | +| markdown | @tanstack/markdown render AST | 0.0030 | +| markdown | @tanstack/markdown parse+render with external highlighter | 0.0192 | +| markdown | @tanstack/markdown parse+render | 0.0176 | +| markdown | marked parse+render | 0.0208 | +| markdown | markdown-it parse+render | 0.0127 | +| markdown | micromark render | 0.1602 | +| markdown | commonmark parse+render | 0.0102 | +| markdown | markdown-wasm render | 0.0050 | +| markdown | unified remark+rehype render | 0.1889 | +| streaming | @tanstack/markdown streaming profile | 0.2850 | +| streaming | marked progressive parse+render | 0.3194 | diff --git a/reports/conformance.json b/reports/conformance.json index 4b4dee6..df79fae 100644 --- a/reports/conformance.json +++ b/reports/conformance.json @@ -1,5 +1,5 @@ { - "generatedAt": "2026-09-11T20:23:24.612Z", + "generatedAt": "2026-09-12T16:34:37.763Z", "commonMarkVersion": "0.31.2", "baseline": 403, "lost": [], diff --git a/reports/conformance.md b/reports/conformance.md index b84d0cd..14718b5 100644 --- a/reports/conformance.md +++ b/reports/conformance.md @@ -1,6 +1,6 @@ # Conformance Report -Generated: 2026-09-11T20:23:24.613Z +Generated: 2026-09-12T16:34:37.763Z This is compatibility accounting against CommonMark 0.31.2, not a claim of CommonMark conformance. diff --git a/reports/inline-parsers.md b/reports/inline-parsers.md new file mode 100644 index 0000000..231b1d4 --- /dev/null +++ b/reports/inline-parsers.md @@ -0,0 +1,37 @@ +# Inline source parser proposal + +Baseline: `eb6ef72496aa2c5b2916a1545064307041529024`. Runtime: `v26.7.0`. + +This new extension API adds core dispatch code. The proposed measured ceilings below replace the previous exact ceilings; accepting the API therefore also requires accepting this explicit size increase. Standalone existing extensions are unchanged. No runtime dependency is added. + +| Entry | Gzip before | Gzip after | Delta | +| --- | ---: | ---: | ---: | +| parser only | 4975 | 5283 | +308 | +| html renderer no highlighter | 6807 | 7116 | +309 | +| react adapter | 6722 | 7026 | +304 | +| octane adapter | 6727 | 7033 | +306 | + +CommonMark accounting: 403 → 403; no established matches lost. + +The existing revision comparison warms both implementations and alternates their execution over nine rounds. These are local Node measurements with the extension disabled, not browser performance claims. + +| Fixture | Parse + render before (ms) | After (ms) | Ratio | +| --- | ---: | ---: | ---: | +| ai-response.md (parseRender) | 0.01933 | 0.01951 | 1.01× | +| ai-response.md (streaming) | 0.22550 | 0.23448 | 1.04× | +| code-heavy.md (parseRender) | 0.01066 | 0.01088 | 1.02× | +| malformed.md (parseRender) | 0.00374 | 0.00383 | 1.02× | +| prose-heavy.md (parseRender) | 0.01852 | 0.01860 | 1.00× | +| small-doc.md (parseRender) | 0.01291 | 0.01298 | 1.01× | +| tables-lists.md (parseRender) | 0.02851 | 0.02877 | 1.01× | +| long-prose (parseRender) | 0.01856 | 0.01857 | 1.00× | +| unmatched-brackets (parseRender) | 0.03279 | 0.03293 | 1.00× | + +Reproduce: + +```sh +pnpm exec tsx scripts/compare-revision.mjs eb6ef72496aa2c5b2916a1545064307041529024 +pnpm run verify +``` + +The comparison writes full paired samples to `artifacts/audit-comparison.json`. Hook-specific tests cover ordinary-text dispatch, literal markers, escape and code precedence, source ownership, shared scan/depth limits, invalid consumption, link context, AST serialization, and HTML/React/Octane parity. diff --git a/reports/sizes.json b/reports/sizes.json index 5347214..cacd19d 100644 --- a/reports/sizes.json +++ b/reports/sizes.json @@ -1,40 +1,40 @@ { - "generatedAt": "2026-09-11T20:23:25.725Z", + "generatedAt": "2026-09-12T16:41:31.750Z", "results": [ { "group": "tanstack", "name": "parser only", - "minBytes": 13157, - "gzipBytes": 4975, - "brotliBytes": 4593 + "minBytes": 13797, + "gzipBytes": 5283, + "brotliBytes": 4854 }, { "group": "tanstack", "name": "html renderer no highlighter", - "minBytes": 18337, - "gzipBytes": 6807, - "brotliBytes": 6240 + "minBytes": 18979, + "gzipBytes": 7116, + "brotliBytes": 6514 }, { "group": "tanstack", "name": "html renderer with external highlighter stub", - "minBytes": 18373, - "gzipBytes": 6829, - "brotliBytes": 6260 + "minBytes": 19015, + "gzipBytes": 7137, + "brotliBytes": 6536 }, { "group": "tanstack", "name": "react adapter", - "minBytes": 18271, - "gzipBytes": 6722, - "brotliBytes": 6184 + "minBytes": 18913, + "gzipBytes": 7026, + "brotliBytes": 6454 }, { "group": "tanstack", "name": "octane adapter", - "minBytes": 18283, - "gzipBytes": 6727, - "brotliBytes": 6194 + "minBytes": 18925, + "gzipBytes": 7033, + "brotliBytes": 6462 }, { "group": "tanstack", @@ -60,9 +60,9 @@ { "group": "tanstack", "name": "react adapter with streaming extension", - "minBytes": 18962, - "gzipBytes": 6907, - "brotliBytes": 6356 + "minBytes": 19604, + "gzipBytes": 7212, + "brotliBytes": 6624 }, { "group": "tanstack", @@ -116,37 +116,37 @@ { "group": "tanstack-public", "name": ".", - "minBytes": 18598, - "gzipBytes": 6936, - "brotliBytes": 6345 + "minBytes": 19241, + "gzipBytes": 7240, + "brotliBytes": 6622 }, { "group": "tanstack-public", "name": "./html", - "minBytes": 18560, - "gzipBytes": 6920, - "brotliBytes": 6331 + "minBytes": 19203, + "gzipBytes": 7227, + "brotliBytes": 6606 }, { "group": "tanstack-public", "name": "./parser", - "minBytes": 13289, - "gzipBytes": 5061, - "brotliBytes": 4664 + "minBytes": 13929, + "gzipBytes": 5372, + "brotliBytes": 4925 }, { "group": "tanstack-public", "name": "./react", - "minBytes": 18470, - "gzipBytes": 6828, - "brotliBytes": 6279 + "minBytes": 19113, + "gzipBytes": 7133, + "brotliBytes": 6554 }, { "group": "tanstack-public", "name": "./octane", - "minBytes": 18485, - "gzipBytes": 6833, - "brotliBytes": 6287 + "minBytes": 19128, + "gzipBytes": 7140, + "brotliBytes": 6563 }, { "group": "tanstack-public", @@ -196,6 +196,13 @@ "minBytes": 3537, "gzipBytes": 1338, "brotliBytes": 1185 + }, + { + "group": "tanstack-public", + "name": "./extensions/autolinks", + "minBytes": 994, + "gzipBytes": 632, + "brotliBytes": 568 } ] } \ No newline at end of file diff --git a/reports/sizes.md b/reports/sizes.md index a30d23f..eae7a33 100644 --- a/reports/sizes.md +++ b/reports/sizes.md @@ -1,20 +1,20 @@ # Bundle Size Results -Generated: 2026-09-11T20:23:25.726Z +Generated: 2026-09-12T16:41:31.751Z Bundles are ESM, browser-targeted, minified with esbuild, then gzip and brotli compressed. Framework runtimes are externalized for the React and Octane adapters. | Group | Entry | Min bytes | Gzip bytes | Brotli bytes | | :--- | :--- | ---: | ---: | ---: | -| tanstack | parser only | 13157 | 4975 | 4593 | -| tanstack | html renderer no highlighter | 18337 | 6807 | 6240 | -| tanstack | html renderer with external highlighter stub | 18373 | 6829 | 6260 | -| tanstack | react adapter | 18271 | 6722 | 6184 | -| tanstack | octane adapter | 18283 | 6727 | 6194 | +| tanstack | parser only | 13797 | 5283 | 4854 | +| tanstack | html renderer no highlighter | 18979 | 7116 | 6514 | +| tanstack | html renderer with external highlighter stub | 19015 | 7137 | 6536 | +| tanstack | react adapter | 18913 | 7026 | 6454 | +| tanstack | octane adapter | 18925 | 7033 | 6462 | | tanstack | docs extension preset | 6423 | 2292 | 2073 | | tanstack | callouts extension | 506 | 335 | 278 | | tanstack | streaming extension | 699 | 311 | 253 | -| tanstack | react adapter with streaming extension | 18962 | 6907 | 6356 | +| tanstack | react adapter with streaming extension | 19604 | 7212 | 6624 | | tanstack | tabs transforms | 3290 | 1221 | 1082 | | markdown | marked | 41415 | 12548 | 11509 | | markdown | markdown-it | 148242 | 52655 | 44023 | @@ -22,11 +22,11 @@ Bundles are ESM, browser-targeted, minified with esbuild, then gzip and brotli c | markdown | commonmark | 159687 | 48084 | 39793 | | markdown | markdown-wasm browser js+wasm | 66387 | 31275 | 26431 | | markdown | unified remark+rehype | 119588 | 36843 | 32686 | -| tanstack-public | . | 18598 | 6936 | 6345 | -| tanstack-public | ./html | 18560 | 6920 | 6331 | -| tanstack-public | ./parser | 13289 | 5061 | 4664 | -| tanstack-public | ./react | 18470 | 6828 | 6279 | -| tanstack-public | ./octane | 18485 | 6833 | 6287 | +| tanstack-public | . | 19241 | 7240 | 6622 | +| tanstack-public | ./html | 19203 | 7227 | 6606 | +| tanstack-public | ./parser | 13929 | 5372 | 4925 | +| tanstack-public | ./react | 19113 | 7133 | 6554 | +| tanstack-public | ./octane | 19128 | 7140 | 6563 | | tanstack-public | ./extensions/callouts | 660 | 432 | 360 | | tanstack-public | ./extensions/comment-components | 1073 | 647 | 542 | | tanstack-public | ./extensions/docs | 6587 | 2392 | 2162 | @@ -34,3 +34,4 @@ Bundles are ESM, browser-targeted, minified with esbuild, then gzip and brotli c | tanstack-public | ./extensions/headings | 1038 | 573 | 480 | | tanstack-public | ./extensions/streaming | 838 | 403 | 334 | | tanstack-public | ./extensions/tabs | 3537 | 1338 | 1185 | +| tanstack-public | ./extensions/autolinks | 994 | 632 | 568 | diff --git a/scripts/bench-autolinks.ts b/scripts/bench-autolinks.ts new file mode 100644 index 0000000..4144314 --- /dev/null +++ b/scripts/bench-autolinks.ts @@ -0,0 +1,29 @@ +import { performance } from 'node:perf_hooks' +import { autolinksExtension } from '../src/extensions/autolinks.js' +import { renderHtml } from '../src/html.js' + +const extensions = [autolinksExtension()] +const fixtures = [ + ['plain comment', 'Thanks for the thoughtful review. This paragraph reads clearly now.'], + ['URL comment', 'See https://example.com/~alice~/notes and **https://example.com/docs**.'], + ['punctuation', '(https://example.com/a_(b)). '], + ...[1000, 2000, 4000].map(size => [`malformed ${size * 9} chars`, 'https://%'.repeat(size)]), +] as const +let sink = 0 +const results = fixtures.map(([name, source]) => { + const runs = [() => renderHtml(source!), () => renderHtml(source!, { extensions })] + const iterations = Math.max(100, Math.floor(1_000_000 / source!.length)) + const sample = (run: () => string) => { + const start = performance.now() + for (let index = 0; index < iterations; index++) sink += run().length + return (performance.now() - start) / iterations + } + runs.forEach(sample) + const samples: number[][] = [[], []] + for (let round = 0; round < 7; round++) { + for (const index of round % 2 ? [1, 0] : [0, 1]) samples[index]!.push(sample(runs[index]!)) + } + const medians = samples.map(values => [...values].sort((a, b) => a - b)[3]!) + return { name, bytes: Buffer.byteLength(source!), iterations, coreMs: medians[0], autolinksMs: medians[1], samples } +}) +console.log(JSON.stringify({ node: process.version, results, sink }, null, 2)) diff --git a/scripts/compare-revision.mjs b/scripts/compare-revision.mjs index b8ee413..b6e7f6c 100644 --- a/scripts/compare-revision.mjs +++ b/scripts/compare-revision.mjs @@ -16,6 +16,9 @@ const revision = process.argv[2] assert(revision, 'Usage: node --import tsx scripts/compare-revision.mjs [--no-bench] [--corpus]') const directory = await stat(revision).then(value => value.isDirectory()).catch(() => false) const baseline = directory ? resolve(revision) : execFileSync('git', ['rev-parse', '--verify', `${revision}^{commit}`], { encoding: 'utf8' }).trim() +const baselinePackage = JSON.parse(directory + ? await readFile(resolve(baseline, 'package.json'), 'utf8') + : execFileSync('git', ['show', `${baseline}:package.json`], { encoding: 'utf8' })) const sourceCache = new Map() const baselinePlugin = { name: 'baseline-source', @@ -52,12 +55,16 @@ const sizes = [] for (const entry of [...entries.filter(entry => entry.group === 'tanstack'), ...publicEntries]) { const results = [] for (const previous of [true, false]) { + if (previous && entry.group === 'tanstack-public' && !Object.hasOwn(baselinePackage.exports, entry.name)) { + results.push(null) + continue + } const code = await bundle(entry.contents, previous, entry.external) results.push({ min: code.length, gzip: gzipSync(code, { level: 9 }).length, brotli: brotliCompressSync(code).length }) } sizes.push({ name: entry.name, before: results[0], after: results[1] }) } -console.table(sizes.map(row => ({ name: row.name, before: row.before.gzip, after: row.after.gzip, delta: row.after.gzip - row.before.gzip }))) +console.table(sizes.map(row => ({ name: row.name, before: row.before?.gzip ?? 'new entry', after: row.after.gzip, delta: row.before ? row.after.gzip - row.before.gzip : null }))) const apis = [] for (const previous of [true, false]) { diff --git a/scripts/verify-docs.mjs b/scripts/verify-docs.mjs index a6a016d..77a47d8 100644 --- a/scripts/verify-docs.mjs +++ b/scripts/verify-docs.mjs @@ -55,6 +55,7 @@ const apiDocuments = new Map([ ['./parser', 'docs/reference/parser.md'], ['./react', 'docs/reference/react.md'], ['./octane', 'docs/reference/octane.md'], + ['./extensions/autolinks', 'docs/reference/extensions.md'], ['./extensions/callouts', 'docs/reference/extensions.md'], ['./extensions/comment-components', 'docs/reference/extensions.md'], ['./extensions/docs', 'docs/reference/extensions.md'], diff --git a/skills/custom-extensions/SKILL.md b/skills/custom-extensions/SKILL.md index 4786b4e..2ec4965 100644 --- a/skills/custom-extensions/SKILL.md +++ b/skills/custom-extensions/SKILL.md @@ -1,7 +1,7 @@ --- name: 'custom-extensions' description: > - Implement MarkdownExtension block parsers, inline and document transforms, + Implement MarkdownExtension block and inline source parsers, inline and document transforms, HTML hooks, and portable block and inline component output. Load when adding deterministic custom syntax or rendering behavior across HTML, React, and Octane. metadata: @@ -76,6 +76,14 @@ console.log(html) ## Core Patterns +### Recognize inline source before formatting + +Use `inlineParser: { markers, parse(context) }` for syntax that needs original source characters. `markers` is a string of literal first characters. The context supplies `source`, UTF-16 `index`, `options`, `inLink`, and `parseInline(value)` sharing the current depth and scan budget. Return `{ node, length }` with one standard `InlineNode` and a positive in-bounds integer length, or `undefined` to decline. Invalid lengths throw `RangeError`. + +Escapes and code spans take precedence. Hooks do not run in code, image alt text, or link destinations; they do run in emphasis and link labels. Respect `inLink` when creating automatic links. Indices are local to the current inline container, and ranges cannot cross enclosing inline or block boundaries. Hook dispatch is budgeted, but callback work is trusted and must avoid repeated suffix scans. Returned AST nodes are trusted; validate URLs and component metadata yourself. + +Use `transformInline` for changes to already-parsed nodes. It cannot distinguish an escaped opener from an ordinary text opener or restore raw Markdown spelling. See the extension guide for a complete source-parser example. + ### Transform parsed inline nodes ```ts @@ -109,7 +117,7 @@ const html = renderHtml('IMPORTANT: Back up the database.', { console.log(html) ``` -Transforms receive built-in inline nodes and must return a deterministic replacement array. +Transforms receive both built-in inline nodes and extension nodes returned by `inlineParser`, and must return a deterministic replacement array. For custom inline UI, return an `InlineComponentNode` with `type: 'inlineComponent'`, `name`, `attributes`, inline `children`, and optional `tagName` and string `properties`. diff --git a/skills/render-markdown/SKILL.md b/skills/render-markdown/SKILL.md index 906c24c..882cd0f 100644 --- a/skills/render-markdown/SKILL.md +++ b/skills/render-markdown/SKILL.md @@ -201,7 +201,7 @@ const html = renderHtml(controlledMarkdown) console.log(html) ``` -Setext headings and automatic URL linking are outside the supported syntax +Setext headings are outside the supported syntax profile, so unsupported input can remain literal or have different structure. Source: `docs/core-concepts/syntax-profile.md` @@ -321,6 +321,10 @@ footnote definitions unless internal parser state is supplied explicitly. Source: `docs/reference/default-entry.md` +## Optional pasted URLs + +Import `autolinksExtension` from `@tanstack/markdown/extensions/autolinks` and pass it in `extensions` to recognize bare HTTP(S) URLs and `` notation. This is opt-in and is not part of the docs preset. It uses the application `urlTransform` policy and produces ordinary portable link nodes. Code, images, and explicit link labels retain existing behavior. Bare links exclude trailing sentence punctuation and unmatched closing delimiters; use angle notation for meaningful trailing punctuation. `www.` addresses, emails, other schemes, and full GFM autolink conformance are not included. + ## Related Skills - `production-pipelines/SKILL.md` - trust boundaries, syntax highlighting, diff --git a/skills/render-markdown/references/ast-and-options.md b/skills/render-markdown/references/ast-and-options.md index afdc8ff..204b6b0 100644 --- a/skills/render-markdown/references/ast-and-options.md +++ b/skills/render-markdown/references/ast-and-options.md @@ -229,7 +229,7 @@ interface ParseOptions { | `urlTransform` | built-in policy | Synchronous `(url, kind, defaultUrl) => string \| null`; return the screened default, a trusted replacement, or `null` to keep only label content. Applies to Markdown destinations, not raw HTML or supplied ASTs. | | `frontmatter` | `true` | Extract one leading `---` block into `document.frontmatter`. The library does not parse YAML. | | `headingIds` | `true` | Generate duplicate-safe IDs, disable IDs with `false`, or return an ID from `(text, normalizedLineIndex)`. | -| `extensions` | `[]` | Run block parsers and inline/document transforms in array order. | +| `extensions` | `[]` | Run block/inline source parsers and inline/document transforms in array order. | | `references` | internal | Carry normalized reference definitions through nested parsing. | | `footnotes` | internal | Carry normalized footnote definitions through nested parsing. | | `footnoteOrder` | internal | Track first-reference order through nested parsing. | diff --git a/src/extensions/autolinks.ts b/src/extensions/autolinks.ts new file mode 100644 index 0000000..bef7fb2 --- /dev/null +++ b/src/extensions/autolinks.ts @@ -0,0 +1,54 @@ +import type { InlineParseContext, InlineParseResult, MarkdownExtension } from '../types.js' + +const protocol = /https?:\/\//iy +const boundary = /[\p{L}\p{N}_/@<\\]/u +const terminator = /[\s\u0000-\u0020\u007f<>"'`\\]/ +const trailing = /[.,!?;:]/ + +/** Opt-in HTTP(S) links; leaves the core Markdown profile unchanged. */ +export function autolinksExtension(): MarkdownExtension { + return { name: 'autolinks', inlineParser: { markers: 'hH<', parse: parseAutolink } } +} + +function parseAutolink({ source, index, options, inLink }: InlineParseContext): InlineParseResult | undefined { + if (inLink) return + const angle = source[index] === '<' + if (!angle && index > 0 && boundary.test(source[index - 1]!)) return + const start = index + Number(angle) + protocol.lastIndex = start + const match = protocol.exec(source) + if (!match) return + + let end = protocol.lastIndex + const depth = [0, 0, 0] + while (end < source.length) { + const char = source[end]! + if (terminator.test(char)) break + if (!angle) { + const open = '([{'.indexOf(char) + const close = ')]}'.indexOf(char) + if (open !== -1) depth[open]!++ + if (close !== -1 && depth[close]!-- === 0) break + } + end++ + } + if (!angle) while (end > protocol.lastIndex && trailing.test(source[end - 1]!)) end-- + + const url = source.slice(start, end) + const closed = !angle || source[end] === '>' + const length = end + Number(angle && closed) - index + // Consume malformed candidates literally so their suffixes are not rescanned. + const literal: InlineParseResult = { length, node: { type: 'text', value: source.slice(index, index + length) } } + if (!closed) return literal + try { + new URL(url) + } catch { + return literal + } + + // Only validated http(s) source reaches this point, so the default is safe. + // As with core links, an application's replacement is trusted. + const href = options.urlTransform ? options.urlTransform(url, 'link', url) : url + const label = { type: 'text' as const, value: url } + return { length, node: href === null ? label : { type: 'link', href, children: [label] } } +} diff --git a/src/inline.ts b/src/inline.ts index 32cbd28..a9248b1 100644 --- a/src/inline.ts +++ b/src/inline.ts @@ -1,8 +1,14 @@ -import type { InlineNode, ParseOptions } from './types.js' +import type { InlineNode, InlineParser, ParseOptions } from './types.js' import { footnoteId, normalizeReferenceLabel, parseDestination, plainText, sanitizeUrl } from './utils.js' export function parseInline(value: string, options: ParseOptions = {}): InlineNode[] { - let result = parseInlineRaw(value, options) + const parsers = options.extensions?.flatMap(extension => extension.inlineParser ? [extension.inlineParser] : []) ?? [] + const marker = parsers.length + ? new RegExp(inlineMarker.source + '|[' + parsers.map(parser => parser.markers).join('').replace(/[\\\]\[\-^]/g, '\\$&') + ']', 'g') + : inlineMarker + let result = parseInlineRaw(value, options, { + scans: Math.max(value.length * scansPerCharacter, 1024), depth: 0, links: 0, parsers, marker, + }) for (const extension of options.extensions ?? []) { result = extension.transformInline?.(result, { options }) ?? result @@ -15,6 +21,8 @@ interface InlineParseBudget { scans: number depth: number links: number + parsers: InlineParser[] + marker: RegExp } const maxInlineDepth = 32 @@ -24,10 +32,14 @@ const inlineMarker = /[\\`!\[_*~<]/g function parseInlineRaw( value: string, options: ParseOptions, - budget: InlineParseBudget = { scans: Math.max(value.length * scansPerCharacter, 1024), depth: 0, links: 0 }, + budget: InlineParseBudget, + inLink = false, ): InlineNode[] { if (budget.depth >= maxInlineDepth) return value ? [{ type: 'text', value }] : [] budget.depth++ + // Image alt text is parsed with stripped options and shares only the budget. + const parsers = options.extensions ? budget.parsers : [] + const marker = parsers.length ? budget.marker : inlineMarker const nodes: InlineNode[] = [] let index = 0 @@ -40,7 +52,7 @@ function parseInlineRaw( } } - while (index < value.length) { + scan: while (index < value.length) { const char = value[index]! const next = value[index + 1] @@ -77,6 +89,23 @@ function parseInlineRaw( continue } + for (const parser of parsers) { + if (!parser.markers.includes(char) || budget.scans-- <= 0) continue + const result = parser.parse({ + source: value, index, options, inLink, + parseInline: source => parseInlineRaw(source, options, budget, inLink), + }) + if (!result) continue + if (!Number.isInteger(result.length) || result.length <= 0 || result.length > value.length - index) { + throw new RangeError('Inline parser must consume a positive length within the source') + } + pushText() + nodes.push(result.node) + if (result.node.type === 'link') budget.links++ + index += result.length + continue scan + } + if (char === '[' || (char === '!' && next === '[')) { const image = char === '!' const footnote = next === '^' && parseFootnoteReference(value, index, options, budget) @@ -91,7 +120,7 @@ function parseInlineRaw( const parsed = parseLinkish(value, index + Number(image), options, budget) if (parsed) { const links = budget.links - const children = parseInlineRaw(parsed.label, image ? (options.references ? { references: options.references } : {}) : options, budget) + const children = parseInlineRaw(parsed.label, image ? (options.references ? { references: options.references } : {}) : options, budget, !image) const nested = !image && budget.links !== links const defaultUrl = sanitizeUrl(parsed.href) const href = options.urlTransform ? options.urlTransform(parsed.href, image ? 'image' : 'link', defaultUrl) : defaultUrl @@ -124,7 +153,7 @@ function parseInlineRaw( pushText() nodes.push({ type: 'emphasis', - children: parseInlineRaw(value.slice(index + 1, close + 2), options, budget), + children: parseInlineRaw(value.slice(index + 1, close + 2), options, budget, inLink), }) index = close + 3 continue @@ -145,7 +174,7 @@ function parseInlineRaw( pushText() nodes.push({ type: char === '~' ? 'strike' : size === 2 ? 'strong' : 'emphasis', - children: parseInlineRaw(value.slice(index + size, close), options, budget), + children: parseInlineRaw(value.slice(index + size, close), options, budget, inLink), }) index = close + size } else { @@ -166,8 +195,8 @@ function parseInlineRaw( } // Reset after recursive parsing; exhausted budgets still allow escapes. - inlineMarker.lastIndex = index + 1 - const end = budget.scans > 0 ? inlineMarker.exec(value)?.index ?? value.length : value.indexOf('\\', index + 1) + marker.lastIndex = index + 1 + const end = budget.scans > 0 ? marker.exec(value)?.index ?? value.length : value.indexOf('\\', index + 1) text += value.slice(index, end < 0 ? value.length : end) index = end < 0 ? value.length : end } diff --git a/src/types.ts b/src/types.ts index 46a1d5d..9a29246 100644 --- a/src/types.ts +++ b/src/types.ts @@ -188,11 +188,34 @@ export interface HtmlInlineNode { export interface MarkdownExtension { name: string parseBlock?: (context: BlockParseContext) => BlockNode | undefined + inlineParser?: InlineParser transformDocument?: (document: MarkdownDocument, context: DocumentTransformContext) => MarkdownDocument | void transformInline?: (nodes: InlineNode[], context: InlineTransformContext) => InlineNode[] renderHtml?: (node: BlockNode | InlineNode, context: HtmlRenderContext) => string | undefined } +export interface InlineParser { + /** Possible first characters, used to skip ordinary text efficiently. */ + markers: string + parse: (context: InlineParseContext) => InlineParseResult | undefined +} + +export interface InlineParseContext { + source: string + index: number + options: ParseOptions + /** True while parsing an explicit link label. */ + inLink: boolean + /** Parse children with the current recursion and scan budget. */ + parseInline: (value: string) => InlineNode[] +} + +export interface InlineParseResult { + node: InlineNode + /** Positive number of UTF-16 code units consumed at context.index. */ + length: number +} + export interface BlockParseContext { lines: string[] index: number diff --git a/tests/autolinks.test.tsx b/tests/autolinks.test.tsx new file mode 100644 index 0000000..71be2f5 --- /dev/null +++ b/tests/autolinks.test.tsx @@ -0,0 +1,113 @@ +import { createElement } from 'react' +import { renderToStaticMarkup as renderReact } from 'react-dom/server' +import { renderToStaticMarkup as renderOctane } from 'octane/server' +import { describe, expect, it, vi } from 'vitest' +import { autolinksExtension } from '../src/extensions/autolinks.js' +import { parseMarkdown, renderHtml } from '../src/index.js' +import { Markdown as ReactMarkdown } from '../src/react.js' +import { Markdown as OctaneMarkdown } from '../src/octane.js' +import type { MarkdownDocument, ParseOptions } from '../src/types.js' +import { normalizeStaticMarkup } from './helpers/normalize-html.js' + +const extensions = [autolinksExtension()] +const escape = (value: string) => value.replaceAll('&', '&').replaceAll('<', '<').replaceAll('>', '>').replaceAll('"', '"') +const link = (url: string) => `${escape(url)}` +function renderAll(source: string | MarkdownDocument, options: ParseOptions = {}) { + options = { extensions, ...options } + const html = renderHtml(source, options) + expect(normalizeStaticMarkup(renderReact(createElement(ReactMarkdown, { children: source, ...options })))).toBe(normalizeStaticMarkup(html)) + expect(normalizeStaticMarkup(renderOctane(OctaneMarkdown, { children: source, ...options }).html)).toBe(normalizeStaticMarkup(html)) + return html +} + +describe('optional HTTP(S) autolinks', () => { + it('does not change core behavior unless enabled', () => { + expect(renderHtml('https://example.com')).toBe('

https://example.com

') + expect(renderAll('See https://example.com.')).toBe(`

See ${link('https://example.com')}.

`) + }) + + it.each([ + 'http://example.com', 'HTTPS://EXAMPLE.COM/path', 'https://example.com/~alice~/notes', + 'https://example.com/?q=*hello*', 'https://example.com/a_b_c', + 'https://example.com/?a=1&b=2#part', 'http://localhost:3000/docs', 'http://[::1]:3000/docs', + 'https://例え.jp/道', 'https://example.com/a%20b', + ])('preserves the source URL: %s', url => { + expect(renderAll(`See ${url}`)).toBe(`

See ${link(url)}

`) + }) + + it.each(['.', ',', '!', '?', ';', ':', '...?!'])('leaves sentence punctuation outside the link: %s', punctuation => { + expect(renderAll(`https://example.com/path${punctuation}`)).toBe(`

${link('https://example.com/path')}${punctuation}

`) + }) + + it.each([ + ['(https://example.com/a_(b)).', `(${link('https://example.com/a_(b)')}).`], + ['[https://example.com/a[b]]', `[${link('https://example.com/a[b]')}]`], + ['https://example.com/a{b}}', `${link('https://example.com/a{b}')}}`], + ['https://example.com/a_(b(c))', link('https://example.com/a_(b(c))')], + ])('balances URL delimiters: %s', (source, html) => { + expect(renderAll(source)).toBe(`

${html}

`) + }) + + it.each(['https://example.com/path.', 'https://example.com/a)', 'http://[::1]:8080', 'https://example.com/?q=*hello*'])('honors explicit angle boundaries: %s', url => { + expect(renderAll(`<${url}>`)).toBe(`

${link(url)}

`) + }) + + it('works in ordinary formatting, headings, lists, quotes, and table cells', () => { + const url = 'https://example.com/~alice~/notes' + const source = `# ${url}\n\n**${url}** *${url}*\n\n> ${url}\n\n- ${url}\n\n| Site |\n| --- |\n| ${url} |` + const html = renderAll(source) + expect(html.match(/') + expect(html).toContain(`${link(url)}`) + const document = parseMarkdown(source, { extensions }) + expect(renderAll(JSON.parse(JSON.stringify(document)))).toBe(html) + }) + + it('leaves explicit links, reference links, images, and code alone', () => { + const url = 'https://example.com' + const source = `[${url}](/target) [**${url}**](/target) ![${url}](/image) \`${url}\`\n\n[${url}][id]\n\n[id]: /target\n\n\`\`\`\n${url}\n\`\`\`` + expect(renderAll(source)).toBe(renderHtml(source)) + }) + + it('leaves raw HTML tag attributes alone when HTML is enabled', () => { + const source = 'text' + expect(parseMarkdown(source, { extensions, allowHtml: true })).toEqual(parseMarkdown(source, { allowHtml: true })) + }) + + it.each([ + 'www.example.com', 'person@example.com', 'ftp://example.com', 'javascript:alert(1)', + 'xhttps://example.com', 'éhttps://example.com', '/https://example.com', + 'https://', 'https://%', 'https://example.com:invalid/path', '', + ])('keeps unsupported or malformed input literal: %s', source => { + expect(renderAll(source)).toBe(renderHtml(source)) + }) + + it('passes complete URLs to application policy before inline transforms', () => { + const url = 'https://example.com/~alice~/notes' + const urlTransform = vi.fn((raw: string, kind: string, fallback: string) => { + expect([raw, kind, fallback]).toEqual([url, 'link', url]) + return '/redirect' + }) + expect(renderAll(url, { urlTransform })).toBe(`

${url}

`) + expect(urlTransform).toHaveBeenCalledTimes(3) + expect(renderAll(`<${url}>`, { urlTransform: () => null })).toBe(`

${url}

`) + }) + + it('supports a mixture of rejected and accepted links', () => { + expect(renderAll('https://blocked.test https://allowed.test', { + urlTransform: (url, _kind, safe) => url.includes('blocked') ? null : safe, + })).toBe(`

https://blocked.test ${link('https://allowed.test')}

`) + }) + + it('escapes labels and attributes and excludes controls and quotes', () => { + expect(renderAll('https://example.com/?x=