一个模型搞定73种语言的词形变化，轻量高效还支持生词。
Flexing in 73 Languages: A Single Small Model for Multilingual Inflection
- 用单一模型联合训练73种语言的词形变化数据。
- 在多数语言上超越单语模型，对未见词汇表现稳健。
- 适合需要多语言文本生成的开发者和研究者使用。
我们提出一种紧凑的单模型多语言词形变化方法，即从词基生成表达语法范畴的词形变化形式。该模型在73种语言的联合数据上训练，轻量、对未见词汇鲁棒，并在大多数语言上优于单语基线模型。这证明了多语言建模在词形变化中的有效性，凸显其实际优势：通过消除管理与重训数十个独立单语模型的需求，简化部署。除标准SIGMORPHON共享任务基准外，我们在73个通用依赖（UD）树库上评估了单语与多语言模型，提取词基-标签-词形三元组及其频次。为确保真实数据划分，引入新颖的频率加权、词基不相交的训练-验证-测试重采样流程。本工作填补了开源、通用、多语言词形变化系统在广泛语言（包括捷克语）中处理未见词的空白。所有代码已公开发布于：https://github.com/tomsouri/multilingual-inflection。
原文摘要 · Abstract (English)
We present a compact, single-model approach to multilingual inflection, the task of generating inflected word forms from base lemmas to express grammatical categories. Our model, trained jointly on data from 73 languages, is lightweight, robust to unseen words, and outperforms monolingual baselines in most languages. This demonstrates the effectiveness of multilingual modeling for inflection and highlights its practical benefits: simplifying deployment by eliminating the need to manage and retrain dozens of separate monolingual models. In addition to the standard SIGMORPHON shared task benchmarks, we evaluate our monolingual and multilingual models on 73 Universal Dependencies (UD) treebanks, extracting lemma-tag-form triples and their frequency counts. To ensure realistic data splits, we introduce a novel frequency-weighted, lemma-disjoint train-dev-test resampling procedure. Our work addresses the lack of an open-source, general-purpose, multilingual morphological inflection system capable of handling unseen words across a wide range of languages, including Czech. All code is publicly released at: https://github.com/tomsouri/multilingual-inflection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。