Skip to content

Preserve prompt padding masks during conditioning - #2

Open
zjn20030811 wants to merge 1 commit into
H-EmbodVis:mainfrom
zjn20030811:fix/text-guidance
Open

Preserve prompt padding masks during conditioning#2
zjn20030811 wants to merge 1 commit into
H-EmbodVis:mainfrom
zjn20030811:fix/text-guidance

Conversation

@zjn20030811

Copy link
Copy Markdown

Summary

The SimWAM prompt-conditioning path currently replaces the tokenizer validity
mask with an all-ones mask after zeroing padded embeddings. As a result,
padded rows can still participate in cross-attention (for example after a
projection bias or in attention implementations that consume the mask).

This change preserves the tokenizer mask and defensively zeros invalid rows
with an out-of-place operation that also works for non-contiguous encoder
outputs.

Validation

  • pytest -q tests/test_prompt_mask.py (2 passed)
  • python -m compileall -q src/simwam tests
  • git diff --check

The regression tests cover both the returned prompt mask/embeddings and the
invariance of cross-attention to changes in padded context rows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant