用模型评分替代人工标注，自动优选词语替换方案
Learning to Substitute Words with Model-based Score Ranking
- 以BARTScore量化句子质量，构建词替换的统计评估分布
- 新损失函数使模型预测与句子评分对齐，提升整体文本质量
- 无需人工标注，适合大规模文本优化场景
智能词语替换旨在通过改进词汇选择提升句子质量，但现有基准依赖人工标注数据。由于词汇选择具有主观性，少数标注者生成的真值替换往往不完整且难以泛化。为此，本文采用基于模型的评分（BARTScore）来量化句子质量，避免依赖人工标注。具体而言，利用该评分定义每个词替换的分布，从而检验某替换是否在统计上优于其他选项。同时，提出一种损失函数，直接优化模型预测与句子评分的一致性，同时提升替换后的整体质量得分。关键在于，模型训练不再需要人工标签，既降低标注成本，又保持修改后文本的质量。实验表明，该方法在性能上超越掩码语言模型（BERT、BART）和大语言模型（GPT-4、LLaMA）。代码已开源：https://github.com/Hyfred/Substitute-Words-with-Ranking。
原文摘要 · Abstract (English)
Smart word substitution aims to enhance sentence quality by improving word choices; however current benchmarks rely on human-labeled data. Since word choices are inherently subjective, ground-truth word substitutions generated by a small group of annotators are often incomplete and likely not generalizable. To circumvent this issue, we instead employ a model-based score (BARTScore) to quantify sentence quality, thus forgoing the need for human annotations. Specifically, we use this score to define a distribution for each word substitution, allowing one to test whether a substitution is statistically superior relative to others. In addition, we propose a loss function that directly optimizes the alignment between model predictions and sentence scores, while also enhancing the overall quality score of a substitution. Crucially, model learning no longer requires human labels, thus avoiding the cost of annotation while maintaining the quality of the text modified with substitutions. Experimental results show that the proposed approach outperforms both masked language models (BERT, BART) and large language models (GPT-4, LLaMA). The source code is available at https://github.com/Hyfred/Substitute-Words-with-Ranking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。