通过多格式训练提升大模型对答案形式的鲁棒性
Improving Cross-Format Robustness in Language Models with Multi-Format Training
- 用格式混合方法扩展部分训练数据的多种表达形式
- 仅扩增30%数据即可接近全量多格式训练效果
- 适合希望提升模型泛化能力但不想改模型的研究者
大型语言模型常对答案格式敏感：同一问题在一种形式下正确解答，换种等价形式却可能出错。我们定义跨格式鲁棒性为模型在不同格式下对同一问题回答的一致性。对比全格式训练与FormatMix方法（仅对部分训练样本使用随机或定向选择生成多格式），在GLM4和Llama-3.1上，多格式监督显著提升任务性能与跨格式鲁棒性；而仅使用多选题训练则收益甚微，甚至降低鲁棒性。进一步发现，仅将约30%训练集扩展为多格式，即可恢复大部分全格式训练收益，该现象在不同模型家族与规模中均成立。结果表明，格式多样性是鲁棒性的关键驱动因素，而非额外监督本身。轻量级多格式增强是一种无需修改基础模型即可降低格式敏感性的实用方法。
原文摘要 · Abstract (English)
Large language models often remain sensitive to answer format: a question solved correctly in one form may fail in another semantically equivalent form. To study this gap, we define cross-format robustness as the extent to which a model answers the same underlying question consistently across formats. We then compare full-format training with FormatMix, which expands only a subset of training items into multiple equivalent formats using either random or targeted selection. Across GLM4 and Llama-3.1, multi-format supervision consistently improves both task performance and cross-format robustness, whereas Multiple-choice question (MCQ)-only supervision alone brings little benefit and can even reduce robustness. We further find that expanding only about 30% of the training set into multiple formats often recovers most of the gain from full-format training, and this effect appears across the model families and sizes we study. These results suggest that format diversity, rather than additional supervision alone, is the key driver of robustness. That lightweight multi-format augmentation is a practical way to make LLMs less sensitive to answer format without changing the base model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。