Derive the ISO 639 sets from the registration authority data

[jviotti]

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

[augmentcode]

🤖 Augment PR Summary

Summary: This PR updates the ISO 639 language datasets so the published “sets” are derived directly from the official registration authority sources.

Changes:

• Adds ISO 639-5 TSV (LOC) as a new vendored data source
• Extends the build pipeline to convert ISO 639-5 TSV to JSON and pass it into language set generation
• Reworks templates/build/iso/language/enriched.jq so Set 5 comes from ISO 639-5 data (instead of heuristics on ISO 639-2 names)
• Expands Set 2 (bibliographic + terminologic) to include all registered ISO 639-2 code elements (individual, collective, special)
• Models the reserved local-use range (qaa–qtz) as a regex branch and excludes the qaa-qtz range marker row
• Updates schema templates accordingly (per-code x-jsonld-self and new “Reserved for local use” branch)
• Updates tests to cover special/collective codes and the reserved local-use range

Technical Notes: Set 2 terminologic is generated as “bibliographic except for the ~20 divergent pairs”, and Set 5 now tracks the LOC ISO 639-5 vocabulary directly.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 1 suggestion posted.

Comment augment review to trigger a new review at any time.

[augmentcode]

The new ISO 639-5 source URL is plain http://; since https://id.loc.gov/vocabulary/iso639-5.tsv is available, sticking to HTTPS would avoid environments that block HTTP and provides transport integrity for vendored data downloads.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[cubic-dev-ai]

All reported issues were addressed

Not reviewed (too large): schemas/2020-12/iso/language/2023/set-2-terminologic.json (~3,939 lines), schemas/2020-12/iso/language/2023/set-2-bibliographic.json (~823 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

<sub>Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic</sub>