arXiv:2502.12924cs.CLcs.AI2025-02中稿 · LREC 2026被引 3

用回译法训练大模型生成英西混用文本,提升流畅度与质量。

Conditioning LLMs to Generate Code-Switched Text

  • 通过回译构建平行语料,微调大模型实现单语到混用文本生成
  • 微调后模型生成的混用文本在人类偏好测试中表现更优
  • 传统评估指标失效,大模型判别更贴近人类真实偏好

代码切换(CS)仍是自然语言处理中的关键挑战,主要因缺乏大规模、多样化的高质量CS数据集。本文研究大模型在英语-西班牙语混用文本生成中的能力,提出一种新方法:将自然发生的混用句回译为纯英语,构建平行语料,再用此数据微调大模型,使其能将单语句子转为混用文本。通过人工偏好测试、定性错误分析、主流参考指标及基于大模型的评判进行综合评估。结果表明,微调是确保大模型一致生成流畅混用文本的关键步骤;所提方法生成的文本质量高,拓展了代码切换研究的可能性。我们发现,传统评价指标与人类判断无相关性,而基于大模型的评判更接近人类偏好。代码与生成数据集已按CC-BY-NC-SA协议开源。

原文摘要 · Abstract (English)

Code-switching (CS) is still a critical challenge in Natural Language Processing (NLP), due to the limited availability of large-scale, diverse CS datasets for robust training and evaluation. Despite recent advances, the capabilities and limitations of LLMs in handling CS are still not fully understood. In this work, we investigate the extent to which LLMs can be used in a framework for CS text generation, focusing on the English-Spanish language pair. Our proposed methodology consists of back-translating natural CS sentences into monolingual English, and using the resulting parallel corpus to fine-tune LLMs to turn monolingual sentences into CS. We thoroughly analyse the models' performance through a study on human preferences, a qualitative error analysis, an evaluation with popular reference-based metrics and LLM-based judgment. Results show that fine-tuning can be a key step to ensure that current LLMs consistently generate fluent code-switched text and that our methodology generates high-quality outputs, expanding research opportunities in CS communication. We find that traditional metrics do not correlate with human judgement when assessing the quality of the generated CS data, but LLM-based judgment aligns more closely with human preferences. We release our code and generated dataset under a CC-BY-NC-SA license.

代码切换大模型文本生成双语

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。