fix(svm): decode the text, and close out the plan - #793
Merged
Conversation
Closes #772, whose last open defect this is. `read_string_with_encoding` handed back the file's own bytes for every encoding but `UCS2`, so a label in `MS_1252` or an ISO-8859 part emitted invalid utf-8 - and an xml parser refuses that exactly as hard as the unescaped `&` the same defect was raised for, costing the whole image rather than one label. The charset a `FONT` action names now picks a decoder from `internal/encoding`, which already has the single-byte tables. An encoding we have no decoder for is taken for `MS_1252`, what a file of this age most likely means and what vcl falls back to: a label in the wrong characters still draws. The font's own family and style names go through it too - they are bytes as well, and they go straight into `font-family`. The run a text action names is measured in utf-16 code units, because vcl decodes the string before it indexes it. That was a byte `substr` before, which was right only while the string was the raw bytes. Numbers checked against `include/rtl/textenc.h`: `SYMBOL` is 10 and `ASCII_US` 11, 9 being unused. The plan is spent, so `svm/PLAN.md` goes. What had forward value moved rather than went: the corpus frequency table, which is the evidence behind every "occurs nowhere" in the module, and the `FLOATTRANSPARENT` shortcut, both now in `svm/AGENTS.md`. `svm/README.md`'s feature matrix was left behind by it now says what is true, including where the drawing is deliberately approximate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XQpLmJpJ87qbKoG8B7kbLY
andiwand
force-pushed
the
fix/svm-text-encoding
branch
from
August 30, 2026 19:33
680adb3 to
fbe0bc9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 Generated with Claude Code
Closes #772. This is the last of its ten defects, plus the doc pass that
retires the plan.
The defect
Defect 1 was raised as "text is not XML-escaped". Escaping landed in the first
stage; the other half did not.
read_string_with_encodinghanded back thefile's own bytes for every encoding but
UCS2:So a label in
MS_1252or an ISO-8859 part emitted invalid utf-8, and an xmlparser refuses that exactly as hard as the unescaped
&the defect was raisedfor — the whole image, not one label.
README.mdcarried it as the "Knowndefect" and
svm/AGENTS.mdas "escaping is not yet enough".The charset a
FONTaction names now picks a decoder frominternal/encoding, which already has the single-byte tables. 29 of thertl_TextEncodingvalues map onto one; anything else is taken forMS_1252,which is what a file of this age most likely means and what vcl falls back to
— a label in the wrong characters still draws, where the bytes cost the image.
Two things that came with it:
font-family.They are decoded with the charset the same action reads next. (vcl uses the
stream charset for these, which a metafile never sets.)
SvmReaderdecodes thestring before it indexes the
OUString. It was a bytesubstr, which wascorrect only while the string was the raw bytes; after decoding it would
have cut a character in half.
The numbers are from
include/rtl/textenc.hrather than memory —
SYMBOLis 10 andASCII_US11, with 9 unused, so theconstant already in the enum was right.
The docs
svm/PLAN.mdis spent — every stage is done — so it goes. Two parts hadforward value and moved to
svm/AGENTS.mdrather than going with it: thecorpus frequency table, which is the evidence behind every "occurs
nowhere" claim in the module and exists nowhere else, and the
FLOATTRANSPARENTshortcut.svm/README.md's feature matrix was left behind by #784–#791 — it stilladvertised unchecked boxes for the poly-polygon fill rule, the primitives,
béziers, bitmaps, gradients, clipping and the state stack, all of which
shipped. It now says what is true (38 of the 54 action types we name are
drawn), lists what is not implemented and why, and names the two places the
drawing is deliberately approximate (
SQUARE/RECTgradients as ellipses,the triple hatch's diagonal spacing).
What is left in the module
None of it occurs in the corpus, and each is logged where it is skipped:
TEXTRECTandTEXTLINE(the reader has both, nothing draws them),MOVECLIPREGION,WALLPAPER,FLOATTRANSPARENT,EPS, theMASKfamily,ZCOMPRESSed dibs, and version-1 (pre-VCLMTF) files.Verification
3 new tests (50 in the svm suite): the same bytes decode differently under
MS_1252and latin-1 (0x92 is a curly apostrophe in one and a controlcharacter in the other), an unknown encoding still yields valid utf-8, and a
run offset survives a character that is more than one utf-8 byte. Full suite
1326 passed, 6 skipped, no reference output moves — the corpus text is
UCS2or ascii, so nothing rendered changes.
Checked as CI compiles,
-Wall -Wextra -Werror, not just as the local builddoes.