Thanks for kaalin — we're using it to build an open Karakalpak speech corpus, and it's been very useful.
While validating ~15,000 sentences we found that latin2cyrillic is lossy for borrowed words, in a way that cyrillic2latin is not.
Reproduction
kaalin 3.3.2.post1 — a correctly-spelled Cyrillic word does not survive a round trip:
from kaalin.converter import cyrillic2latin, latin2cyrillic
for w in ["компьютер", "февраль", "экономика", "объект", "счёт", "дүнья"]:
print(w, "->", cyrillic2latin(w), "->", latin2cyrillic(cyrillic2latin(w)))
# компьютер -> kompyuter -> компютер ь dropped
# февраль -> fevral -> феврал ь dropped
# экономика -> ekonomika -> економика э became е
# объект -> obyekt -> обйект ъ became й
# счёт -> schyot -> счйот ё became йо
# дүнья -> dúnya -> дүня ь droppedFour fault classes
All confined to borrowings — native Karakalpak words are unaffected.
| Correct |
Produced |
компьютер, февраль, король, модель, область |
soft sign dropped |
экономика, элемент, экология |
э → е |
объект, субъект |
ъ → й |
счёт, самолёт |
ё → йо |
How often
Measured against 5,007 human-reviewed sentences in our corpus, comparing each converter's output with the text a native speaker settled on:
| Direction |
Reproduced the reviewer's text |
cyrillic2latin |
2,169 / 2,170 (100.0%) |
latin2cyrillic |
2,565 / 2,837 (90.4%) |
cyrillic2latin is essentially exact. The ~10% error in the other direction was entirely this pattern, across 127 distinct word forms.
Why this is hard, and what might help
The Latin orthography doesn't encode these distinctions — kompyuter carries no information about whether the Cyrillic is компютер or компьютер. So this isn't a mapping bug; a correct conversion needs a loanword lexicon.
Possible directions:
- A built-in loanword list consulted during Latin → Cyrillic. A few hundred entries would cover most real text.
- An optional hook so callers can supply their own list.
- Documenting the limitation, so users know Latin → Cyrillic output needs review for borrowings.
We built a small version of (1) for our own use — a list of ~57 correct spellings, with the broken forms derived by applying the four faults in reverse, so only listed words are ever rewritten. Happy to share it if useful.
Minor, possibly out of scope
cyrillic2latin("обьектив") returns obektiv, where obyektiv is expected. The input is itself misspelled (ь where ъ belongs), so this may not be worth handling — noting it only in case robustness to such input is wanted.
Thanks for kaalin — we're using it to build an open Karakalpak speech corpus, and it's been very useful.
While validating ~15,000 sentences we found that
latin2cyrillicis lossy for borrowed words, in a way thatcyrillic2latinis not.Reproduction
kaalin 3.3.2.post1 — a correctly-spelled Cyrillic word does not survive a round trip:
Four fault classes
All confined to borrowings — native Karakalpak words are unaffected.
компьютер,февраль,король,модель,областьэкономика,элемент,экологияэ→еобъект,субъектъ→йсчёт,самолётё→йоHow often
Measured against 5,007 human-reviewed sentences in our corpus, comparing each converter's output with the text a native speaker settled on:
cyrillic2latinlatin2cyrilliccyrillic2latinis essentially exact. The ~10% error in the other direction was entirely this pattern, across 127 distinct word forms.Why this is hard, and what might help
The Latin orthography doesn't encode these distinctions —
kompyutercarries no information about whether the Cyrillic isкомпютерorкомпьютер. So this isn't a mapping bug; a correct conversion needs a loanword lexicon.Possible directions:
We built a small version of (1) for our own use — a list of ~57 correct spellings, with the broken forms derived by applying the four faults in reverse, so only listed words are ever rewritten. Happy to share it if useful.
Minor, possibly out of scope
cyrillic2latin("обьектив")returnsobektiv, whereobyektivis expected. The input is itself misspelled (ьwhereъbelongs), so this may not be worth handling — noting it only in case robustness to such input is wanted.