LgRegexpWordCharacters = 'MECAB' is a magic string: the field that should hold a word-characters regex instead holds a literal marker meaning "tokenize with MeCab". LgParserType now exists to say that properly, so the magic word should go — but it is not a one-line swap, because it carries a second fact that LgParserType does not.
It means two different things
Ten PHP sites read it, and they do not agree on what it is.
As a parser selector — genuinely superseded by LgParserType:
Language::getEffectiveParserType() (Language.php:434)
ParserRegistry::resolveParserType() / resolveParserTypeFromRow() (ParserRegistry.php:199,228)
ExternalParser.php:551
As "this language has no spaces between words" — not superseded, and this is the blocker. In SentenceService it appears in six places alongside LgSplitEachChar and LgRemoveSpaces, governing how spaces are reconstructed around tokens:
// SentenceService.php:333, 362 (and 136, 290, 455, 548)
if (!$removeSpaces && !($splitEachChar || 'MECAB' == strtoupper(trim($termchar)))) {
Setting LgParserType = 'mecab' and clearing the field would silently change Japanese spacing, because nothing else tells SentenceService the language is space-free. That fact needs an explicit home first — LgRemoveSpaces already exists, or it could be derived from the parser's own capabilities.
It also consumes the field it occupies: a MeCab language has no word-characters regex at all, which interacts badly with anything reasoning about word characters (see #278).
It is still a live input path
Not a legacy artifact to read-only-support. The word-characters selector still offers "MeCab (recommended)" (locale/en/language.json → form.regexp_alt_mecab), and language_form.ts::wordCharChange() writes the literal string into the field. New magic-word languages are still being created today.
Meanwhile the modern path has already moved on without it: langdefs.json's Japanese preset declares "parserType": "mecab", and #281 is what makes that preset take effect.
A bug it has already caused
Every reader normalizes before comparing — strtoupper(trim($x)) === 'MECAB' — except one:
// GetPhoneticReading.php:54
if ($wordCharacters !== "mecab") {
return $text;
}
A language whose field holds MECAB in any other casing (an old install, an import, a hand-edit) silently gets no phonetic reading: no error, just the input back. Worth fixing on its own, independently of this issue.
Why it matters beyond tidiness
Parser identity currently lives in three places — the magic word, LgSplitEachChar, and LgParserType — and 20251223_120000_add_parser_type.sql backfills the third from the first two. That ambiguity is what made #281 change tokenization for every CJK language on an upgraded install: it started trusting a column holding derived values. The fix there (17262f5) reads a type that merely restates its neighbouring flag as the flag rather than as intent — correct for now, and a workaround for this issue. Retiring the magic word is what would let LgParserType simply mean what the user chose, and let that workaround be deleted.
Suggested plan
- Give "no spaces between words" an explicit home, so
SentenceService stops asking about the magic word.
- Migration: for
UPPER(TRIM(LgRegexpWordCharacters)) = 'MECAB', set LgParserType = 'mecab' (the backfill already did), set the space flag, and put a real regex (or empty) back in the word-characters field.
- Drop the MeCab option from the word-characters selector, so nothing new is written; the parser dropdown already covers it.
- Keep the readers accepting the magic word for one release, logging a deprecation.
- Next release: delete the readers, and with them the
restatesALegacySignal() workaround in ParserRegistry.
Roughly 10 PHP sites, 2 TS files, 1 migration, 1 locale key. Small, but it changes how Japanese renders, so it wants a before/after parse comparison across every language — the same one that caught the #281 regression.
LgRegexpWordCharacters = 'MECAB'is a magic string: the field that should hold a word-characters regex instead holds a literal marker meaning "tokenize with MeCab".LgParserTypenow exists to say that properly, so the magic word should go — but it is not a one-line swap, because it carries a second fact thatLgParserTypedoes not.It means two different things
Ten PHP sites read it, and they do not agree on what it is.
As a parser selector — genuinely superseded by
LgParserType:Language::getEffectiveParserType()(Language.php:434)ParserRegistry::resolveParserType()/resolveParserTypeFromRow()(ParserRegistry.php:199,228)ExternalParser.php:551As "this language has no spaces between words" — not superseded, and this is the blocker. In
SentenceServiceit appears in six places alongsideLgSplitEachCharandLgRemoveSpaces, governing how spaces are reconstructed around tokens:Setting
LgParserType = 'mecab'and clearing the field would silently change Japanese spacing, because nothing else tellsSentenceServicethe language is space-free. That fact needs an explicit home first —LgRemoveSpacesalready exists, or it could be derived from the parser's own capabilities.It also consumes the field it occupies: a MeCab language has no word-characters regex at all, which interacts badly with anything reasoning about word characters (see #278).
It is still a live input path
Not a legacy artifact to read-only-support. The word-characters selector still offers "MeCab (recommended)" (
locale/en/language.json→form.regexp_alt_mecab), andlanguage_form.ts::wordCharChange()writes the literal string into the field. New magic-word languages are still being created today.Meanwhile the modern path has already moved on without it:
langdefs.json's Japanese preset declares"parserType": "mecab", and #281 is what makes that preset take effect.A bug it has already caused
Every reader normalizes before comparing —
strtoupper(trim($x)) === 'MECAB'— except one:A language whose field holds
MECABin any other casing (an old install, an import, a hand-edit) silently gets no phonetic reading: no error, just the input back. Worth fixing on its own, independently of this issue.Why it matters beyond tidiness
Parser identity currently lives in three places — the magic word,
LgSplitEachChar, andLgParserType— and20251223_120000_add_parser_type.sqlbackfills the third from the first two. That ambiguity is what made #281 change tokenization for every CJK language on an upgraded install: it started trusting a column holding derived values. The fix there (17262f5) reads a type that merely restates its neighbouring flag as the flag rather than as intent — correct for now, and a workaround for this issue. Retiring the magic word is what would letLgParserTypesimply mean what the user chose, and let that workaround be deleted.Suggested plan
SentenceServicestops asking about the magic word.UPPER(TRIM(LgRegexpWordCharacters)) = 'MECAB', setLgParserType = 'mecab'(the backfill already did), set the space flag, and put a real regex (or empty) back in the word-characters field.restatesALegacySignal()workaround inParserRegistry.Roughly 10 PHP sites, 2 TS files, 1 migration, 1 locale key. Small, but it changes how Japanese renders, so it wants a before/after parse comparison across every language — the same one that caught the #281 regression.