Skip to content

Dictionary-imported terms are never linked to text occurrences, so marking a word known fails with a duplicate-key error #283

Description

@sarahmccuan

Describe the bug

After importing a curated dictionary, marking a word known from the reading view fails: the term already exists in words, but the reader treats it as new and tries to INSERT it, colliding with the unique index WoTextLCLgID.

Two symptoms, depending on which control is used:

  • "Add Term" → save: POST /api/v1/terms/full500
    DatabaseException: Failed to execute statement: Duplicate entry 'ich-1' for key 'WoTextLCLgID'
      at TermCrudApiHandler::createTermFull() (TermCrudApiHandler.php:759)
    
  • Well Known / status shortcut: POST /api/v1/terms/quickHTTP 200, nothing happens. No PHP error, no row changed. (The error is swallowed — see the companion issue.)

To Reproduce

  1. Import a curated dictionary for a language (POST /api/v1/local-dictionaries/import-curated).
  2. Open a text in that language.
  3. Click any word the dictionary supplied and mark it Well Known.

Cause

CuratedDictImportService::importFromUrl() calls DictionaryFacade::createVocabularyFromEntries(), creating one words row per dictionary entry. It never links them to word_occurrences — linking normally happens in WordLinkingService::linkToTextItems(), which only runs when a user creates a term.

The reader decides a word is unknown from word_occurrences.Ti2WoID, not from whether the term exists in words. So every imported term is invisible to the reader:

-- text 27 (German), after importing a German dictionary
SELECT COUNT(*) AS occ, SUM(Ti2WoID IS NULL) AS unlinked
FROM word_occurrences WHERE Ti2TxID = 27 AND Ti2WordCount = 1;
-- occ = 526, unlinked = 526      <- none linked

SELECT WoID, WoText FROM words WHERE WoLgID = 1 AND WoTextLC IN ('wer','kann','mir');
-- 360152 Wer | 51 kann | 18 mir  <- all already present

WordDiscoveryService::insertWordWithStatus() then runs a plain INSERT INTO words (...) with no existence check or upsert, so it collides with WoTextLCLgID (WoTextLC, WoLgID).

getTermForEdit() shows the same assumption from the other direction: it sets isNew purely from whether a wid was passed, so /api/v1/terms/for-edit returns "isNew": true for a term that already exists in the vocabulary.

Because the import covers essentially the whole language, this affects effectively every word in a text, not isolated cases.

Which side is wrong isn't obvious: either the import should link occurrences as it creates terms, or the reader's write paths should look a term up by (text, language) before inserting. Flagging rather than assuming.

Server

  • LWT 3.4.2-fork (v003004002)
  • PHP 8.4.24, MariaDB 12.1.2, Apache 2.4.68 (Debian), Docker on Windows

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions