通过约束生成词元提升大模型问答鲁棒性，抗干扰能力显著增强。
Token Constraint Decoding Improves Robustness on Question Answering for Large Language Models
- 在推理时强制词元预测一致，提升模型对输入扰动的抵抗力。
- 在多个数据集上使弱模型性能提升最高达39个百分点。
- 无需修改模型即可通用，适合部署于真实场景的高可靠性需求。
大语言模型在多项选择题问答任务中表现优异，但对微小输入扰动仍十分敏感。本文提出并评估了一种名为词元约束解码（Token Constraint Decoding, TCD）的简单而有效的推理阶段算法，通过强制词元级预测的一致性来增强噪声环境下的鲁棒性。在CommonsenseQA、MMLU和MMLU-Pro上的大量实验表明，TCD尤其与提示工程（PE）结合时，能显著恢复因输入噪声导致的性能下降，使Gemma3 1B等弱模型的准确率最高提升39%。惩罚扫描分析进一步揭示，TCD隐式正则化了过度自信的输出，不同模型需采用不同的惩罚策略以最大化抗扰能力。研究结果表明，TCD是一种实用、模型无关的改进方法，可提升复杂推理任务中的稳定性，为大模型在安全关键或用户交互场景中的可靠部署提供支持。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated impressive performance on multiple-choice question answering (MCQA) benchmarks, yet they remain highly vulnerable to minor input perturbations. In this paper, we introduce and evaluate Token Constraint Decoding (TCD). This simple yet effective inference-time algorithm enforces alignment between token-level predictions to enhance robustness in noisy settings. Through extensive experiments on CommonsenseQA, MMLU, and MMLU-Pro, we show that TCD, especially when paired with prompt engineering (PE) fixes, significantly restores performance degraded by input noise, yielding up to +39\% absolute gains for weaker models like Gemma3 1B. Penalty sweep analyses further reveal that TCD implicitly regularizes overconfident outputs, with different models requiring distinct penalty schedules to maximize resilience. Our findings establish TCD as a practical, model-agnostic approach for improving reasoning stability under real-world imperfections and pave the way for more reliable deployment of LLMs in safety-critical or user-facing applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。