通过增强查询对称性，让大模型在不同问法下保持推理一致性。
Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training
- 基于查询增强的数据扩增方法，提升模型从上下文中提取信息的能力。
- 在逻辑与算术推理任务中，对不同问法的鲁棒性显著提升。
- 适合关注模型泛化能力与抗扰动性能的研究者使用。
大型语言模型在各类任务中展现出强大的推理能力，但即使语义相同的查询，仅因表达方式微调也会显著影响其表现。为此，我们聚焦于提升模型对查询变体对称性的感知能力，提出一种数据驱动的方法——对称性增强数据扩增（syMmetry-ENhanceD, MEND）。该方法不依赖推理链增强，而是通过查询扩增提升模型在知识提取阶段的鲁棒性，实现更高效的数据利用和更强的分布外（OOD）泛化能力。在逻辑与算术推理任务上的大量实验表明，MEND能有效提升模型在多样化查询变体下的推理性能，为通过结构化数据集构建提升大模型鲁棒性提供了新思路。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated strong reasoning capabilities across various tasks. However, even minor variations in query phrasing, despite preserving the underlying semantic meaning, can significantly affect their performance. To address this, we focus on enhancing LLMs' awareness of symmetry in query variations and propose syMmetry-ENhanceD (MEND) Data Augmentation, a data-centric approach that improves the model's ability to extract useful information from context. Unlike existing methods that emphasize reasoning chain augmentation, our approach improves model robustness at the knowledge extraction stage through query augmentations, enabling more data-efficient training and stronger generalization to Out-of-Distribution (OOD) settings. Extensive experiments on both logical and arithmetic reasoning tasks show that MEND enhances reasoning performance across diverse query variations, providing new insight into improving LLM robustness through structured dataset curation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。