diff --git a/nbdev/__init__.py b/nbdev/__init__.py index af7a19b9f..0d794c9ec 100644 --- a/nbdev/__init__.py +++ b/nbdev/__init__.py @@ -4,7 +4,7 @@ - `nbdev.extract_attachments`: A preprocessor that extracts all of the attachments from the notebook file. The extracted attachments are returned in the 'resources' dictionary. - `nbdev.moddocs`: # Module docs: creating them from existing notebooks -- `nbdev.skill`: Author clear, executable nbdev notebooks where code, prose, examples, outputs, and tests form one coherent narrative. Trigger: ALWAYS read this before ANY notebook edit.""" +- `nbdev.skill`: Author nbdev notebooks as source, documentation, examples, and tests. Ensure this guidance is in context before any notebook edit.""" __version__ = "3.3.16" diff --git a/nbdev/skill.py b/nbdev/skill.py index 04ab4ec73..1ed910d65 100644 --- a/nbdev/skill.py +++ b/nbdev/skill.py @@ -1,91 +1,70 @@ -"""Author clear, executable nbdev notebooks where code, prose, examples, outputs, and tests form one coherent narrative. Trigger: ALWAYS read this before ANY notebook edit. +"""Author nbdev notebooks as source, documentation, examples, and tests. Ensure this guidance is in context before any notebook edit. -# The notebook is the product +# Source -An nbdev notebook is source, docs, examples, and tests at once. The rendered page explains the public API, and executing the same cells builds and verifies it. Write it to be read top to bottom, interleaving implementation with explanation, examples, outputs, and failure demonstrations. These conventions matter most for published libraries with docs sites. In internal projects, match the surrounding notebooks instead. +Write notebooks to be read top-to-bottom, interleaving implementation, explanation, examples, outputs, and failure demonstrations. These conventions target published libraries; match surrounding notebooks in internal projects. -# Notebooks generate modules +Check the file, not the repo: `# AUTOGENERATED! DO NOT EDIT!` names the source notebook; `# %% ../nbs/04_usage.ipynb #a45f753a` identifies the source cell. Edit that notebook, never generated modules, `__all__`, `_modidx.py`, or module docstrings; `nbdev-export` regenerates them. Unmarked modules are hand-written. Use notebooks where narrative/examples teach the API, plain `.py` for dense mechanism iterated under pytest. Tests follow source form: lesson cells for notebooks, `tests/*.py` for plain modules. -`nbdev-export` writes exported cells to the module named by `#| default_exp`. Never hand-edit generated `.py` files, `__all__`, `_modidx.py`, or generated module docstrings. The next export overwrites them. Each exported section's marker (`# %% ../nbs/04_usage.ipynb #a45f753a`) leads back to the cell to change. The module docstring is the title cell's `>` description joined with every exported markdown cell after the H1. For a long docstring, prefer one exported markdown cell per section. +For project-level questions, read `nbs/index.ipynb`, the source of README.md; regenerate with `nbdev-readme` after changes. -Projects mix notebook-sourced and plain modules: notebooks where the narrative is worth having (public API with a docs page, code best explained through examples, tests that double as documentation), plain `.py` for dense mechanism iterated quickly under `pytest`. Check the file, never the repo. A generated module opens with the `# AUTOGENERATED! DO NOT EDIT!` warning naming its source notebook, and `# %%` cell markers precede each definition, so a few lines of context around any `def` also answer it. No marker means hand-written and edited directly. Tests follow the source form: cells for notebook modules, `tests/*.py` for plain ones. +# Structure and authoring -For project-level questions, read `nbs/index.ipynb`, not `README.md`. The README is generated from it, renders worse, and can be stale if regeneration lagged. When they disagree, `index.ipynb` is the truth. Regenerate with `nbdev-readme` after editing it. +Choose tutorial, how-to, explanation, or reference. Open with an H1 title cell and `>` subtitle, then a brief introduction. Use H2s for major concepts without fragmenting short narratives. Each heading belongs in its own markdown cell so Jupyter can collapse sections. -# Structure +Before changing a notebook, read the whole notebook or a summary of every cell. For a new notebook, read a role model such as claudette's `00_core.ipynb` to learn the literate form. -Pick one documentation form rather than mixing accidentally: tutorial (guided learning), how-to (a practical task), explanation (a focused topic), or reference (a component and its API, easy to scan). +Develop one idea at a time: small implementation, explanation, executable lesson, assertions where useful. Introduce each lesson in markdown; split it when the introduction becomes complicated. Extend existing examples rather than adding near-duplicates. Build classes with `@patch` beside each method's explanation unless that obscures the API. -Open with an H1 title cell plus `>` subtitle line, introduce the page in a sentence or two, and get to the subject quickly. H2 headings divide major concepts. Don't fragment a short narrative into tiny sections. Every heading goes in its own markdown cell, separate from body prose, since Jupyter collapses sections by cell. +For new behavior, choose its narrative position, write the lesson first, run preceding cells as needed, and see it fail. Add its markdown introduction and implementation above it, then run implementation and lesson to see it pass. For bugs, revise an existing example to exercise the failing path; if no revision fits, add an assertion to it. If no documented behavior changes, change no lesson cells. Never add a cell merely to witness a fix. -# Develop one idea at a time +Read existing examples before experimenting. Explore in the notebook, not in a separate kernel check; retain useful explorations as lessons and delete those that teach nothing worth keeping. -Add the smallest useful implementation, explain what it does and why it has that form, demonstrate it executably, assert where that improves the example, display the result a reader should notice, move on. A cell may bundle several closely related checks, but needs a markdown introduction saying what it establishes. If the introduction gets complicated, split the cell. Prefer extending an existing example over adding a near-duplicate cell. Build classes incrementally with `@patch` so each method sits beside its explanation, unless splitting makes the API harder to understand. +Reserve underscore names for machinery without a coherent independent contract. Internal use can justify exposing an abstraction; helpers useful only internally may indicate awkward design. -# Authoring sequence +# Prose and module documentation -Read the whole notebook before changing it. The interleaved prose, examples, and stored outputs are the design rationale. For a new notebook, first read enough of a role-model notebook, such as claudette's `00_core.ipynb`, to learn the literate form. +Markdown explains rationale, distinctions, and guarantees, not code line-by-line or empty transitions. Place it where it reads correctly on the generated page, normally after the definition. Use short symbol docstrings, parameter/return docments beside signatures, and backticked symbol names for nbdev links. Extended explanation, examples, and warnings belong in markdown. -Pick the place in the narrative where the change belongs. Write the lesson cell for the new behavior first. Run the cells above that place when it needs earlier state. Run the new cell and see it fail. Add a markdown cell above it that introduces the behavior it demonstrates. Add the implementation cell above that. Run the implementation, then the lesson cell, and see it pass. +The module docstring combines the title's `>` description with exported markdown after H1. Use one exported markdown cell per major section for long docstrings. Since `doc(module)` also lists functions, module prose should teach shared constraints and workflows, not repeat individual API documentation. -For a bug fix, do not add a new lesson cell by default. A regression test per fix is a pytest habit. Revise an existing lesson cell so its example passes through the bug's path, and see it fail before the fix. When no revision fits the example, add one assertion line to it. When the fix changes no documented behavior, change no cells. +`#| exportd` includes a code cell as fenced source in the module docstring, not module code. Use compact runnable demos after their required definitions; end the preceding exported summary with a colon. Only exported cells join the docstring, so definitions between that summary and demo do not separate them there. -Explore inside the notebook, not in the kernel. A check that answered your question while building answers it for the next reader. Keep it as a lesson cell, or delete it before finishing when it taught nothing worth keeping. -# Helpers - -Internal use is often the first evidence an abstraction is worth exposing, not a reason to hide it. Reserve underscores for machinery with no coherent independent contract. A helper only useful to the library itself hints at an awkward internal design. - -# Prose and docstrings - -Markdown explains what the code cannot: why an abstraction exists, what distinction the next example demonstrates, which details are guarantees and which merely describe an example. Never narrate code line by line or write empty transitions ("Now we test the function"). State the lesson instead ("A missing leaf returns `None`, so success status alone does not establish existence"). Place prose where it reads correctly on the generated page, normally right after the definition it explains. - -Keep docstrings to a short statement of what the symbol does. Extended explanation, examples, and warnings go in markdown cells, where they render properly and can include executable results. Document parameters and returns with docments, keeping their docs beside the signature instead of repeating it in the docstring. Backtick symbol names in prose. nbdev links them, so prefer names over hand-maintained URLs. - -# Module docstrings - -`doc(module)` shows the docstring with the API listing, and for an LLM that is usually the whole read. Treat the docstring as the module's TL;DR. Write a short summary under each major section heading and tag it `#| export`, teaching what a reader cannot guess from signatures and leaving per-function detail to the page. - -`#| exportd` on a code cell puts its source in the docstring as a fenced block and keeps it out of the module code. Use it for a compact runnable demo, placed after the definitions it needs. End the summary before it with a colon so the two read as one. Only exported cells join the docstring, so the pair stays adjacent there even when definition cells sit between them in the notebook. - -Preview the assembled docstring with `nbdev.export.nb_mdoc`, and run code to check each claim before writing it. `nbdev-export` adds the module to the generated package docstring and `llms.txt` once its docstring says more than the default summary line, so write these cells only for modules meant to be read on their own. For the package intro, tag the opening paragraphs of `index.ipynb` with `#| export`. +Check claims by running code before documenting them; preview assembly with `nbdev.export.nb_mdoc`. A docstring beyond the default summary causes `nbdev-export` to include the module in package documentation and `llms.txt`; write these summaries for modules meant to be read independently. Export the opening paragraphs of `index.ipynb` for the package intro. # Lesson cells -An nbdev notebook has no test cells (rare `#| hide` checks aside). It has *lesson cells*: code cells that teach a point on the page, whose displayed result is the evidence and whose assertions keep the lesson honest forever. Write each one as page content first, then make it verify behavior: realistic values, the shortest path to the idea, an informative displayed result, direct assertions that reinforce the lesson, reuse of objects introduced earlier, and important errors demonstrated executably with `expect_fail` (one focused example per contract). Keep check plumbing out of reader-facing cells. Mocks, dense comprehensions, and long setup make poor documentation, so extract a tiny helper or hide the check. Assertions verify, but only the final expression's display teaches. End cells with the value worth showing, and design a compact `_repr_markdown_` or structured summary when it turns later examples into documentation for free. Plots, tables, images, and rich HTML all count as evidence. Stored outputs are part of the explanation. Keep them focused, and never dump a large structure without saying what matters in it. Assertion helpers come from `fastcore.test` (`test_eq`, `expect_fail`, ...), in plain code cells. - -# Tells +Write page content, not standalone test cells (except rare hidden checks): realistic values, minimal setup, reused objects, and direct assertions that support the lesson. End lesson cells with an informative displayed value when there is something useful to show. Compact `_repr_markdown_` or structured summaries can make later examples readable; plots, tables, images, and HTML also provide evidence. Explain what matters in large outputs and keep stored results focused. -Some patterns in a lesson cell can be spotted mechanically, and each reliably signals a rewrite that would improve the page. Each is a strong hint rather than a law. In particular, a comment sometimes states a constraint the code cannot show, and such a comment stays. +Use `fastcore.test` assertions and `expect_fail` in ordinary code cells, with one focused error example per contract. Keep check plumbing, mocks, dense comprehensions, and long setup out of reader-facing examples: extract a small helper or hide the check. -- A comment in an example cell usually marks where the cell should split in two. Split there, and grow the comment into a markdown cell introducing what the next code cell shows. -- Comments numbering steps mark a tutorial sequence. Give each step its own markdown and code pair. -- A `print` whose f-string wraps a result in a sentence is prose in code. The sentence belongs in markdown, and the value belongs at the end of the cell as its displayed result. -- A `print(x)` as a cell's last line hides the rich repr. End with bare `x` instead. -- A lesson cell that doesn't end in an evaluation to display (its last line is an assertion or an assignment) verifies without teaching. End with the value the checks are about. -- One name reassigned through stages in a single cell hides the intermediate values, which are the point. Give each stage its own cell, ending with a display. -- Blank lines dividing a cell into groups mark candidate cell boundaries, and each group needs its own sentence of markdown. -- Adjacent near-duplicate cells differing in one argument are a comparison written as copies. Make the difference the narrative ("with `strict=True` the same call raises..."). -- `try`/`except` written to demonstrate an error is what `expect_fail` is for. -- A triple-quoted docstring holds explanation that belongs in markdown cells. Keep the docstring to a single short line (see "Prose and docstrings"). -- `# TODO` or `# FIXME` in an example belongs nowhere on a docs page. It is an issue, or it is fixed. +Review these as rewrite signals, not absolute rules: -# State flows downward +- Comments often mark a split into prose and code; numbered steps suggest separate markdown/code pairs. Keep comments that state constraints code cannot show. +- A sentence-wrapped `print` belongs in prose plus a displayed value; final `print(x)` loses rich repr compared with bare `x`. +- Reassignments through stages can hide useful intermediate displays; blank-line groups suggest cell boundaries with their own introductions. +- Near-duplicate examples should make the differing argument the narrative, rather than repeat setup. +- Error-demonstration `try`/`except` usually wants `expect_fail`; long docstrings want markdown. +- Resolve TODO/FIXME example comments or move them to issues. -Keep imports in dedicated import cells, define values near first use, reuse established objects, don't reassign names later cells depend on, introduce shared setup explicitly, and end exploratory cells with the expression whose output records what was learned, so a reader never searches far upward for where a value came from. The import rule is strict and covers lesson cells. The docs build runs each import-containing cell in a fresh namespace where no other cell has run, so a cell mixing imports with other code breaks the build or silently runs at documentation time. +# State and execution -# Directives +Define values near first use, introduce shared setup, reuse established objects, and don't reassign names later cells depend on. End exploratory cells with the expression recording what was learned when useful. -`#| default_exp` names the module. `#| export` marks exported cells. Underscore-prefixed helpers may export without joining the public API. `#| exportd` exports a code cell's source to the module docstring instead of the module. `#| hide` keeps necessary but distracting material off the page. `#| eval: false` is for cells that genuinely must not run, not for suppressing broken ones. An unevaluated cell cannot create state for later evaluated cells. +Keep imports in dedicated cells, including in lessons: docs builds execute import-containing cells in fresh namespaces, so mixing imports with state-dependent code can fail or run code at documentation time. -# nbdev v3 +Stored outputs are generated artifacts and may be stale during editing. Do not stop, clear them manually, or manipulate notebook JSON because output no longer matches source. At PR time, regenerate with `nbdev-test --save` when the project needs saved outputs updated. -Everything here describes nbdev v3 (released Jan 2026, possibly after a model's training cutoff). Key user-visible changes from v2: config moved from `settings.ini` to `pyproject.toml`, with standard metadata in `[project]`, nbdev-specific keys in `[tool.nbdev]` (defaults `nbs_path='nbs'`, `doc_path='_docs'`), version in `__init__.py` via `dynamic = ["version"]`, and `_modidx` registered under `[project.entry-points.nbdev]`. CLI commands use hyphens (`nbdev-export`, `nbdev-test`, ...) though Python functions keep underscores. GitHub workflows use the v3 actions (`fastai/workflows/nbdev3-ci`, `quarto-ghp3`). +# Directives and nbdev v3 -Directives can also live in cell *metadata*, under the `nbdev` key (`{"nbdev": {"export": "true"}}`), instead of `#|` comment lines. Export and the rest of the toolchain honor both; `mk_cell(source, metadata=...).directives` is the merged read, with the comment winning when both set the same key. The editing toolkit keeps the meta form visible: summaries show it bracketed after the type char (`id:c[export]:...` in nbio's `CellRow` and aidialog's previews), XML views render it as attrs (a bare `export` on the tag), and aidialog's `Message` offers `exported` (either form, read-only) and `meta_exported` (meta only, assignable), with `export=`/`meta=` params on the add functions. +- `#| default_exp` names the generated module; `#| export` includes a cell there. Exported underscore helpers need not join the public API. +- `#| hide` hides distracting necessities from the page. +- `#| eval: false` is for code that must not run, not broken code; it cannot supply state to evaluated cells. +- Directives may instead be metadata: `{"nbdev": {"export": "true"}}`. The toolchain honors both; `mk_cell(...).directives` merges them with content winning conflicts. -# Before you finish +nbdev v3 (Jan 2026) uses `pyproject.toml`, not `settings.ini`: metadata in `[project]`, nbdev settings in `[tool.nbdev]` (default `nbs_path='nbs'`, `doc_path='_docs'`), version in `__init__.py` via `dynamic = ["version"]`, and `_modidx` in `[project.entry-points.nbdev]`. CLIs use hyphens (`nbdev-export`, `nbdev-test`); Python names retain underscores. GitHub actions are `fastai/workflows/nbdev3-ci` and `quarto-ghp3`. -A behavior change means revising the lesson cells it touches: prose, displayed output, and assertions move together. Where you revised or added an assertion, check it fails against the old code and passes against the new. Many changes need no new cell at all - never add one merely to witness a change. Re-read the touched section as a reader would, against the conventions above. Style damage breaks no test and no export, so the harm only shows on the docs page. +# Finish -Run notebooks with `nbdev-test `. +Revise affected prose, displays, and assertions together; changed assertions must fail on old code and pass on new. Re-read the touched narrative for readability: tests cannot catch documentation damage. Run `nbdev-test `. """