feat(svd_circuits): singular-direction readout, projection, and causal patch gate

[janmenjayap]

---

Description

Adds the causal-validation layer on top of PR1's per-head QK/OV SVD (#1768): a direction can now be read out to vocab/logit space, projected against a real forward pass, and, critically, causally patched before any subfunction claim is accepted. Second of three PRs implementing per-head singular-vector decomposition of a head's QK (W_Q W_K^T) and OV (W_V W_O) maps (tracking issue: #1767).

• vocab_readout(model, head_svd, k=10) — projects the top-k OV output directions (HeadSVD.V) through the unembedding, gated on TransformerBridge compatibility mode.
• logit_signature(model, head_svd, direction, tokens) — signed logit effect of one OV direction's rank-1 reconstruction; requires the direction to pass require_isolated first, since a rotation-ambiguous or null direction's signature is not attributable to it alone.
• project_activations(model, head_svd, prompt) — per-position firing coefficients of a head's actual hook_result output against its V basis; summing the coefficients against V reconstructs the real output to numerical precision, since both read the same basis the head actually writes in.
• patch_along_directions(model, head_svd, prompt, metric, keep=..., ablate=...) — the mandatory causal gate. Reconstructs or ablates a head's output onto a chosen span of V directions via a direct hook_result hook, and reports delta_metric against an equally-sized random-subspace baseline (gated is true only when the requested subspace beats that baseline, not merely when the metric moves at all). Degenerate blocks reported by HeadSVD.degenerate_blocks() must be kept or dropped whole; a caller cannot route around require_isolated by hand-picking part of a rotation-ambiguous block through keep/ablate.
• Public exports: decompose_head, project_activations, patch_along_directions, vocab_readout, logit_signature, and their supporting types now ship from transformer_lens.tools.analysis — the first time this module is part of the public API, by design, so a readout was never importable without the causal gate attached to it.
• tests/integration/test_svd_circuits.py (new): exercises vocab_readout and patch_along_directions end to end against GPT-2 small's layer 9 head 9 (name-mover head) on an IOI-style prompt.

Correctness fix folded in first: HeadSVD's docstring had the OV output direction backwards — U's columns are the value-computation input space, V's are the residual-stream output space this head writes into and the one to project through W_U. The swap never raised a shape error because both spaces are d_model-dimensional, so every downstream readout, projection, or patch would have silently used the wrong basis. Fixed first, with a regression test that cross-checks against the already-shipped SVDInterpreter (which projects the OV map the other way already), so the assertion cannot pass under either labeling.

Two implementation notes for anyone diffing this against the original proposal:

• vocab_readout does not wrap SVDInterpreter.get_singular_vectors. That method indexes W_V/W_K with a raw query-head index, which is wrong on grouped-query models (the same bug PR1 already fixed for this module's own weight reads). vocab_readout reuses PR1's already-kv-mapped HeadSVD.V and projects it through W_U directly instead of introducing a second, buggy SVD path.
• patch_along_directions calls model.run_with_hooks directly rather than generic_activation_patch. That helper is built for a clean-vs-corrupted two-run sweep over an index grid; this tool is a single-prompt reconstruct/ablate on a live projection, which does not fit the sweep shape.

Unit-tested with a tiny no-download TransformerBridge (real forward passes, no Hub access) for projection/patch behavior, plus model-free tests for the pure weight-space math. Integration test downloads gpt2-small.

PR3 (docs, demo notebook, slow multi-head oracle-parity check) follows this one.

Part of #1767

---

Type of change

• New feature (non-breaking change which adds functionality)

---

Checklist:

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (will be done in PR3)
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have not rewritten tests relating to key interfaces which would affect backward compatibility

---