考虑词频的形态变化模型,让高频词表现更好。
Corpus Frequencies in Morphological Inflection: Do They Matter?
- 训练时按词频采样,提升模型对常见词的掌握
- 在43种语言中,26种语言表现优于传统方法
- 用词频加权评估更贴近真实文本使用场景
传统形态变化任务将词干-词性-形式三元组视为均等,忽略其在语料中的频率分布。为更好模拟实际应用中的输入分布,本文从三个维度引入词频信息:(i) 训练/验证/测试集划分采用词干不相交与词频加权结合策略,反映不同频率区间的真实分布;(ii) 评估时补充标准类型准确率,引入词频加权的词元准确率,使结果更贴近实际文本表现;(iii) 提出频率感知训练方法,首次在形态变化任务中将词频纳入采样过程。实验表明,该方法在43种语言中的26种上优于均匀采样。
原文摘要 · Abstract (English)
The traditional approach to morphological inflection (the task of modifying a base word (lemma) to express grammatical categories) has been, for decades, to consider lexical entries of lemma-tag-form triples uniformly, lacking any information about their frequency distribution. However, in production deployment, one might expect the user inputs to reflect a real-world distribution of frequencies in natural texts. With future deployment in mind, we explore the incorporation of corpus frequency information into the task of morphological inflection along three key dimensions during system development: (i) for train-dev-test split, we combine a lemma-disjoint approach, which evaluates the model's generalization capabilities, with a frequency-weighted strategy to better reflect the realistic distribution of items across different frequency bands in training and test sets; (ii) for evaluation, we complement the standard type accuracy (often referred to simply as accuracy), which treats all items equally regardless of frequency, with token accuracy, which assigns greater weight to frequent words and better approximates performance on running text; (iii) for training data sampling, we introduce a method novel in the context of inflection, frequency-aware training, which explicitly incorporates word frequency into the sampling process. We show that frequency-aware training outperforms uniform sampling in 26 out of 43 languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。