Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/inline-source-parsers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@tanstack/markdown": minor
---

Add opt-in source-level inline parsers to Markdown extensions. Declare starting characters and return a standard inline node with an explicit consumed length, before built-in formatting changes the source. Preserve escape and code precedence, expose link-label context, and share parser budgets with nested parsing.
8 changes: 4 additions & 4 deletions docs/comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,11 @@ These repository benchmarks bundle representative browser entry points from pinn

| Entry | Gzip | Brotli |
| --- | ---: | ---: |
| `@tanstack/markdown/parser` | 5.0 KB | 4.6 KB |
| `@tanstack/markdown/html` | 6.8 KB | 6.2 KB |
| `@tanstack/markdown/react` | 6.7 KB | 6.2 KB |
| `@tanstack/markdown/parser` | 5.3 KB | 4.9 KB |
| `@tanstack/markdown/html` | 7.1 KB | 6.5 KB |
| `@tanstack/markdown/react` | 7.0 KB | 6.5 KB |
| React with streaming extension | 6.9 KB | 6.4 KB |
| `@tanstack/markdown/octane` | 6.7 KB | 6.2 KB |
| `@tanstack/markdown/octane` | 7.0 KB | 6.5 KB |
| Marked | 12.5 KB | 11.5 KB |
| micromark | 15.4 KB | 13.7 KB |
| markdown-wasm JS + WASM | 31.3 KB | 26.4 KB |
Expand Down
40 changes: 38 additions & 2 deletions docs/guides/extensions.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,45 @@ The context includes:

Nested parsing shares the parser depth budget and heading slugger.

## Inline source parsing

Use `inlineParser` when syntax must see source characters before they become emphasis, links, or decoded escapes. `transformInline` cannot recover an escaped opener or the original spelling of an already-parsed node.

```ts
import type { MarkdownExtension } from '@tanstack/markdown'

const issueReferences: MarkdownExtension = {
name: 'issue-references',
inlineParser: {
markers: '#',
parse({ source, index, inLink }) {
if (inLink) return undefined
const match = /^#([0-9]+)\b/.exec(source.slice(index))
if (!match) return undefined
return {
length: match[0].length,
node: {
type: 'link',
href: `/issues/${match[1]}`,
children: [{ type: 'text', value: match[0] }],
},
}
},
},
}
```

`markers` lists literal possible first characters, not a regular expression. The parser skips ordinary text to the next built-in or extension marker. At a matching position, extensions run in array order after built-in escapes and code spans, before the other built-in inline rules. The first returned result owns that range. Return `undefined` to let the next extension or built-in rule handle it.

The context provides `source`, `index`, `options`, `inLink`, and a nested `parseInline(value)` helper. Source and UTF-16 indices refer to the current inline container, not offsets in the original document. Hooks also run within emphasis and explicit link labels; `inLink` remains true through their nested content. They do not run inside code, image alt text, or link destinations. A hook cannot consume across an enclosing inline or block boundary.

Return one standard `InlineNode` and a positive integer `length` within the remaining source. Invalid lengths throw `RangeError`. The child parser shares the existing depth and scan limits; use that helper instead of calling the top-level parser recursively. Hook dispatch counts against the scan budget. Extension code remains trusted: these limits do not bound work done inside a callback. Keep recognition deterministic and avoid repeatedly scanning a suffix after unsuccessful matches.

Returned nodes follow the same trust contract as supplied ASTs: URL destinations and component names/properties must be validated by the extension. The fixed, numeric issue path above needs no user-supplied URL. Extensions accepting arbitrary URLs should use the application URL policy. Return a portable `InlineComponentNode` for custom presentation; no HTML renderer changes are required.

## Inline transformation

`transformInline` receives built-in inline nodes after parsing. Return the replacement array. Keep transforms deterministic and avoid repeated full-array scans for every node.
`transformInline` receives built-in and extension inline nodes after parsing. Return the replacement array. Keep transforms deterministic and avoid repeated full-array scans for every node.

The hook runs once per inline container. Recurse through inline `children` when your transform also needs to handle content inside emphasis or links. Code spans and image alt text are not separate inline containers.

Expand Down Expand Up @@ -93,4 +129,4 @@ Document transforms are already represented in a pre-parsed AST. HTML render hoo

## Admission rule

An extension is appropriate when syntax is broadly useful to docs, has a deterministic block boundary, and does not justify cost in the core entry. Use a larger processing ecosystem when the job requires async plugins, arbitrary tree pipelines, compiler integration, or MDX evaluation.
An extension is appropriate when syntax is broadly useful to docs, has a deterministic source boundary, and does not justify cost in the core entry. Use a larger processing ecosystem when the job requires async plugins, arbitrary tree pipelines, compiler integration, or MDX evaluation.
8 changes: 4 additions & 4 deletions docs/guides/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ The generated browser bundle report records:

| Entry | Gzip | Brotli |
| --- | ---: | ---: |
| parser | 5.0 KB | 4.6 KB |
| HTML renderer | 6.8 KB | 6.2 KB |
| React adapter | 6.7 KB | 6.2 KB |
| Octane adapter | 6.7 KB | 6.2 KB |
| parser | 5.3 KB | 4.9 KB |
| HTML renderer | 7.1 KB | 6.5 KB |
| React adapter | 7.0 KB | 6.5 KB |
| Octane adapter | 7.0 KB | 6.5 KB |
| React adapter with streaming extension | 6.9 KB | 6.4 KB |
| Streaming extension | 0.3 KB | 0.3 KB |
| docs preset | 2.3 KB | 2.1 KB |
Expand Down
2 changes: 1 addition & 1 deletion docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ TanStack Markdown spends its complexity budget on that path. It deliberately doe

### Small entry points

Current minified browser bundles are 5.0 KB gzip for the parser, 6.8 KB for HTML rendering, and 6.7 KB for either UI adapter with its framework runtime externalized. The generated [bundle report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) is the source of truth.
Current minified browser bundles are 5.3 KB gzip for the parser, 7.1 KB for HTML rendering, and 7.0 KB for either UI adapter with its framework runtime externalized. The generated [bundle report](https://github.com/TanStack/markdown/blob/main/reports/sizes.md) is the source of truth.

### Parse once, render many

Expand Down
14 changes: 13 additions & 1 deletion docs/reference/types.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,12 +158,24 @@ Configures anchor `content`, `className`, `ariaHidden`, and `tabIndex`.

### `MarkdownExtension`

Named hook object with optional `parseBlock`, `transformDocument`, `transformInline`, and `renderHtml` functions.
Named hook object with optional `parseBlock`, `transformDocument`, `transformInline`, and `renderHtml` functions, plus an optional `inlineParser`.

### `BlockParseContext`

Provides source `lines`, current `index`, active `options`, nested `parseInline` and `parseBlocks` helpers, and `consume`.

### `InlineParser`

Contains literal first-character `markers` and a synchronous `parse(context)` callback returning `InlineParseResult | undefined`. Runs at matching source positions after escapes and code spans, before other built-in inline rules.

### `InlineParseContext`

Provides current-container `source`, UTF-16 `index`, active `options`, `inLink`, and a nested `parseInline(value)` helper that shares the recursion and scan budget.

### `InlineParseResult`

Contains one standard `InlineNode` as `node` and a positive integer `length` in UTF-16 code units. The range must fit in the current source. Invalid lengths throw `RangeError`.

### `InlineTransformContext`

Provides active parse `options` to inline transforms.
Expand Down
Loading