feat: add wizard subcommand (unstructured input → markdown)

[m-messer]

in2lambda wizard INPUT -o draft.md turns a PDF/docx/tex/md document into the #/## markdown the Markdown filter reads, for a human to review before in2lambda convert draft.md Markdown.

Commits on this branch

• feat: add wizard subcommand — wizard/run.py routes input through Mathpix (PDF) / pandoc (docx) / raw read, runs one LLM extraction pass, renders #/## markdown, echoes check_markdown() warnings, writes the file. wizard/extract.py: pydantic WizardSet/WizardQuestion/WizardPart + extract_set() via the OpenAI structured-output parse against OpenRouter (system prompt + one few-shot). to_markdown() renders the filter's contract. run.py imported lazily so the rest of the CLI works without the llm extra. docs/source/wizard.md + toctree, quickstart + README pointers.
• feat: --name/-n for convert — optional set name; sanitised into a filesystem-safe slug for set_<slug>.json / <slug>/ / <slug>.zip, raw name kept in the JSON name field (what Lambda Feedback shows on import). Default unchanged (set).
• feat: repair mangled LaTeX commands — structured-output models sometimes emit \text/\frac/\beta with a single backslash; \t/\f/\b/\r are valid JSON escapes so the backslash is swallowed and the field ends up with a bare control char glued to the command. extract_set now re-escapes any TAB/CR/FF/BS immediately followed by a letter. Newlines left as-is.
• chore — .gitignore (docs/_bt/, /e2e/); CI test matrix reduced to Python 3.11 (.github/workflows/test.yml; the commit is mislabelled "Updated gitignore").

Testing done

• black --check ., isort, pydocstyle, full pytest --cov — green (93 passed, 92%; wizard/extract.py 100%).
• Live end-to-end against OpenRouter (openai/gpt-4o-mini) for .tex, .md and .docx inputs: wizard → review → convert → valid set.zip. The \text→TAB corruption is fixed and verified on the real model output.

Known gaps / notes

• PDF/Mathpix path (from feat: add Mathpix PDF extraction #26) still untested against the live service.
• pyproject.toml keeps python = "^3.10" while CI now tests only 3.11.
• Wizard extraction is non-deterministic (a question was dropped on one of two .tex runs) — hence the mandatory draft.md review step.

---

Stack (top): … ← mathpix ← wizard-command
Base: mathpix — #26. Merges after the whole stack (#21–#26).

🤖 Generated with Claude Code

[peterbjohnson]

Notes from review. The overall shape (document in, reviewable markdown out, human checks it before conversion) looks right to me. Four things I'd fix first.

The Mathpix step runs before .env is loaded. In run_wizard, _load_markdown() (which does the OCR) runs at run.py:44, and get_client() — the only thing that loads .env — isn't called until :46. So for a PDF, Mathpix credentials kept in .env are never seen, though docs/source/wizard.md:20 says they are. Related: a missing OpenRouter key is only discovered after the paid OCR call has run. Checking credentials up front would save people money. (See also the .env bug in #25, which makes this worse: once installed normally, .env isn't picked up at all.)

_demangle misses the \n commands. The escapes it restores are \t, \r, \f and \b, so:

\text{m}   -> \text{m}   OK
\rho_0     -> \rho_0     OK
\nu        -> newline + "u"    not restored
\nabla u   -> newline + "abla u"   not restored
\neq 0     -> newline + "eq 0"     not restored

I realise newlines are left alone deliberately, and I agree you can't tell a real line break from a mangled \nu after the fact. But \nu, \nabla and \neq are common in the material we run this on, and they'd come out as a broken line break rather than maths. The comment explains the trade-off well; the docstring should probably warn the user too. Also, a genuine tab followed by a letter (a table row, say) becomes a literal \t in the output.

A question-level solution is dropped when the question has parts. In to_markdown, question.solution is only emitted in the elif branch (extract.py:158), so a document with both per-part answers and a closing overall solution loses the latter silently.

CI quietly stopped testing Python 3.10. .github/workflows/test.yml now runs ['3.11'] only, while pyproject.toml still claims python = "^3.10". The change arrived in a commit labelled "Updated gitignore", so it looks accidental. Either restore 3.10 or raise the stated minimum.

One thing to think about rather than change now: the model is asked for JSON and we then repair LaTeX that JSON escaping mangled, when the thing we actually want is markdown. Asking for markdown directly would remove the schema, the escaping repair and to_markdown in one go. It's a trade against how reliably models follow a free-form format, so it may well be the right call as it stands — worth a note in the PR either way.

[m-messer]

Pushed a fix in 4974a31 for the four points above:

• .env/Mathpix ordering: run_wizard now calls get_client() (which loads .env and validates OPENROUTER_API_KEY) before _load_markdown() runs Mathpix OCR, so a missing OpenRouter key is caught before the paid OCR call, and .env-only Mathpix credentials are loaded in time for PDFs. Added a test asserting the call order.
• _demangle's \n limitation: left the logic as-is since it's genuinely unfixable, but expanded the public docstring to explicitly warn that \n-prefixed commands (\nu, \nabla, \neq) can't be repaired (indistinguishable from a real line break), and that a genuine control char before a letter (e.g. a table row) can equally false-positive into a literal escape.
• Dropped question-level solution: dug into the Markdown filter and the Question/Part data model — there's no question-level solution slot at all; a ## Solution heading after the last part is parsed as belonging to that part. So to_markdown now folds an overall/closing question.solution into the last part's solution instead of silently dropping it, which is the only representable, round-trippable option. Updated the field description and system prompt to match, and added a test.
• CI matrix: restored ['3.10', '3.11'] in test.yml — pyproject.toml still declares python = "^3.10" so this keeps CI honest with the stated minimum.

On the JSON-vs-markdown design note: not changing now, per the framing above — leaving it as a follow-up thought.

All lint (black/isort/pydocstyle) and the full pytest --cov suite are green (105 passed), wizard/extract.py still at 100% coverage.