Skip to content

Reject fabricated MoE weights in SVDInterpreter - #1780

Open
emerardd wants to merge 2 commits into
TransformerLensOrg:devfrom
emerardd:fix/svd-interpreter-moe-placeholder-weights
Open

Reject fabricated MoE weights in SVDInterpreter#1780
emerardd wants to merge 2 commits into
TransformerLensOrg:devfrom
emerardd:fix/svd-interpreter-moe-placeholder-weights

Conversation

@emerardd

Copy link
Copy Markdown
Contributor

Description

Sparse-MoE layers do not have a single dense W_in or W_out, but get_bridge_params() previously represented these structurally unavailable weights with zero tensors. SVDInterpreter then decomposed those zero matrices and projected an arbitrary singular basis through the unembedding, returning plausible non-zero directions that were independent of the real expert weights.

This PR:

  • omits dense MLP weight keys when a real Bridge MLP cannot expose them;
  • makes SVDInterpreter raise a layer-specific NotImplementedError before SVD for unavailable w_in or w_out;
  • documents the unsupported sparse-MoE contract;
  • adds unit coverage for parameter omission and a download-free tiny Mixtral regression that proves torch.linalg.svd is not reached.

The change deliberately does not select, average, or router-weight experts. Those behaviors need a separately defined expert-aware interpretation contract.

Closes #1779

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Validation

  • Relevant parameter, SVD, Mixtral, hybrid-MoE, optimizer, and parameter-count tests: 103 passed, 3 skipped
  • Dense tiny GPT-2 control: both w_in and w_out still return non-zero SVD results
  • uv run --no-cache --no-sync mypy .: no issues in 398 source files
  • pycln, isort, Black, and git diff --check: passed

The complete test suite was not run locally.

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@koriyoshi2041 koriyoshi2041 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 1efd56e0. Omitting structurally unavailable sparse-MoE weights closes the dangerous path where a fabricated zero matrix can produce plausible non-zero singular directions, while the dense/absent-MLP cases remain explicit. I also checked the partial-weight path: an available W_in is preserved while an unavailable W_out is omitted, so the interpreter fails at the requested boundary rather than erasing usable data. The focused parameter, Mixtral, and dense-dispatch tests pass locally (20 passed). The full hosted compatibility/quality matrix is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants