用合成数据训练模型中和文本情绪，提升大模型在带情绪文本中的理解能力。
Reading with Intent -- Neutralizing Intent
- 构建11种情绪的合成文本数据集，系统化生成不同语气的上下文。
- 通过情绪中和模型，使带讽刺的文本转为中性，任务准确率提升3%。
- 适合研究大模型对语境情绪鲁棒性的研究人员参考。
大型语言模型（LLM）的查询通常由指令与上下文组成。现有检索增强生成（RAG）基准多使用维基百科类中性、客观文本作为上下文，但实际互联网内容风格多样，影响下游任务表现。本文提出‘带意图阅读’（Reading with Intent）任务，评估不同语气对模型性能的影响。在已有讽刺研究基础上，采用更优的合成数据生成方法，构建包含11种情感的上下文数据集，并训练情绪转换模型以系统化调整文本语气。人工评估显示，微调后的模型能有效利用合成数据。最终，该模型用于将上下文转化为中性语气，在任务中显著缓解讽刺文本带来的挑战，整体性能提升约3%。
原文摘要 · Abstract (English)
Queries to large language models (LLMs) can be divided into two parts: the instruction/question and the accompanying context. The context for retrieval-augmented generation (RAG) systems in most benchmarks comes from Wikipedia or Wikipedia-like texts which are written in a neutral and factual tone. However, when RAG systems retrieve internet-based content, they encounter text with diverse tones and linguistic styles, introducing challenges for downstream tasks. The Reading with Intent task addresses this issue by evaluating how varying tones in context passages affect model performance. Building on prior work that focused on sarcasm, we extend this paradigm by constructing a dataset where context passages are transformed to $11$ distinct emotions using a better synthetic data generation approach. Using this dataset, we train an emotion translation model to systematically adapt passages to specified emotional tones. The human evaluation shows that the LLM fine-tuned to become the emotion-translator benefited from the synthetically generated data. Finally, the emotion-translator is used in the Reading with Intent task to transform the passages to a neutral tone. By neutralizing the passages, it mitigates the challenges posed by sarcastic passages and improves overall results on this task by about $3\%$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。